2024-02-15T17:51:34.563 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-15T17:51:34.563 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561652 --name lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi --verbose --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} --owner scheduled_lflores@teuthology -- /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561652/orig.config.yaml 2024-02-15T17:51:34.612 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-15T17:51:34.704 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561652 branch: wip-yuri10-testing-2024-02-08-0854-pacific description: rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} email: null first_in_suite: false job_id: '7561652' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/pacific v16.2.4 ' name: lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri10-testing-2024-02-08-0854-pacific ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(PG_DEGRADED\) - Degraded data redundancy - \(PG_ - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-tracker-64343-pacific sha1: bedfc49c6a823fb0ff75911e476e1f3cb464ad3e owner: scheduled_lflores@teuthology priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 964 sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-tracker-64343-pacific suite_path: /home/teuthworker/src/github.com_ljflores_ceph_bedfc49c6a823fb0ff75911e476e1f3cb464ad3e/qa suite_relpath: qa suite_repo: https://github.com/ljflores/ceph.git suite_sha1: bedfc49c6a823fb0ff75911e476e1f3cb464ad3e targets: smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKlhCUm2k1q1gsq3CSgb49Hu3cnWkSpNr0fOsXKpIwQANleMzfbTUWiP4y8V0TcnpowcO0hDB2U/GPAJRCoIUsc= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBLU5bsDTV0HTr8VBBaD3qxl333FTANBHQUsnaFRnkoACjZk/+YCDpW6l79Isbj8JbERr5d6JyU6yaIOBfCD4UI= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - pexec: clients: - sudo modprobe -r ceph - sudo modprobe ceph disable_send_metrics=on - install: exclude_packages: - ceph-volume tag: v16.2.4 - print: '**** done install task...' - cephadm: avoid_pacific_features: true cephadm_branch: v16.2.4 cephadm_git_url: https://github.com/ceph/ceph image: docker.io/ceph/ceph:v16.2.4 roleless: true - print: '**** done starting v16.2.4' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-15_17:32:20 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; sleep 30 ; done - ceph orch ps - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: lflores verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-02-15T17:51:34.705 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ljflores_ceph_bedfc49c6a823fb0ff75911e476e1f3cb464ad3e/qa; will attempt to use it 2024-02-15T17:51:34.705 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ljflores_ceph_bedfc49c6a823fb0ff75911e476e1f3cb464ad3e/qa/tasks 2024-02-15T17:51:34.705 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-15T17:51:34.707 INFO:teuthology.task.internal:Checking packages... 2024-02-15T17:51:34.730 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '0e714d9a4bd2a821113e6318adb87bd06cf81ec1' 2024-02-15T17:51:34.730 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-15T17:51:34.730 INFO:teuthology.packaging:ref: None 2024-02-15T17:51:34.730 INFO:teuthology.packaging:tag: None 2024-02-15T17:51:34.731 INFO:teuthology.packaging:branch: wip-yuri10-testing-2024-02-08-0854-pacific 2024-02-15T17:51:34.731 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T17:51:34.731 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri10-testing-2024-02-08-0854-pacific 2024-02-15T17:51:34.961 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-521.g0e714d9a 2024-02-15T17:51:34.962 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-15T17:51:34.971 INFO:teuthology.task.internal:no buildpackages task found 2024-02-15T17:51:34.971 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-15T17:51:35.015 INFO:teuthology.task.internal:Saving configuration 2024-02-15T17:51:35.033 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-15T17:51:35.048 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-15T17:51:35.072 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-15 17:46:24.426094', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKlhCUm2k1q1gsq3CSgb49Hu3cnWkSpNr0fOsXKpIwQANleMzfbTUWiP4y8V0TcnpowcO0hDB2U/GPAJRCoIUsc='} 2024-02-15T17:51:35.093 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-15 17:46:24.427538', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBLU5bsDTV0HTr8VBBaD3qxl333FTANBHQUsnaFRnkoACjZk/+YCDpW6l79Isbj8JbERr5d6JyU6yaIOBfCD4UI='} 2024-02-15T17:51:35.093 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-15T17:51:35.104 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-02-15T17:51:35.104 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-02-15T17:51:35.104 INFO:teuthology.run_tasks:Running task console_log... 2024-02-15T17:51:35.199 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f81890e1f70>, signals=[15]) 2024-02-15T17:51:35.199 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-15T17:51:35.208 INFO:teuthology.task.internal:Opening connections... 2024-02-15T17:51:35.208 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-02-15T17:51:35.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T17:51:35.283 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2024-02-15T17:51:35.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T17:51:35.355 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-15T17:51:35.363 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-02-15T17:51:35.386 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-02-15T17:51:35.386 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-02-15T17:51:35.449 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-02-15T17:51:35.449 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="8" 2024-02-15T17:51:35.449 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-02-15T17:51:35.449 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-02-15T17:51:35.449 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="8" 2024-02-15T17:51:35.449 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el8" 2024-02-15T17:51:35.449 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-15T17:51:35.449 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-02-15T17:51:35.449 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-15T17:51:35.449 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-02-15T17:51:35.449 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-15T17:51:35.450 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-15T17:51:35.450 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-15T17:51:35.450 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-02-15T17:51:35.475 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-02-15T17:51:35.497 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-02-15T17:51:35.497 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-02-15T17:51:35.558 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-02-15T17:51:35.558 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8" 2024-02-15T17:51:35.558 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-02-15T17:51:35.558 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-02-15T17:51:35.558 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8" 2024-02-15T17:51:35.558 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8" 2024-02-15T17:51:35.558 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-15T17:51:35.558 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-02-15T17:51:35.558 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-15T17:51:35.558 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-02-15T17:51:35.558 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-15T17:51:35.558 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-15T17:51:35.559 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-15T17:51:35.559 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-02-15T17:51:35.583 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-15T17:51:35.597 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-15T17:51:35.633 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-15T17:51:35.634 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-02-15T17:51:35.636 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2024-02-15T17:51:35.656 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-15T17:51:35.670 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-15T17:51:35.670 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-02-15T17:51:35.700 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2024-02-15T17:51:35.730 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-15T17:51:35.783 INFO:teuthology.run_tasks:Running task kernel... 2024-02-15T17:51:35.800 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-15T17:51:35.800 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-15T17:51:35.801 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-15T17:51:35.801 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-15T17:51:35.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T17:51:35.822 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-02-15T17:51:35.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T17:51:35.823 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-02-15T17:51:35.883 INFO:teuthology.orchestra.run.smithi191.stdout:4.18.0-489.el8.x86_64 2024-02-15T17:51:35.883 INFO:teuthology.task.kernel:Running kernel on smithi191: 4.18.0-489.el8.x86_64 2024-02-15T17:51:35.883 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum install -y kernel 2024-02-15T17:51:35.885 INFO:teuthology.orchestra.run.smithi042.stdout:4.18.0-489.el8.x86_64 2024-02-15T17:51:35.886 INFO:teuthology.task.kernel:Running kernel on smithi042: 4.18.0-489.el8.x86_64 2024-02-15T17:51:35.886 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum install -y kernel 2024-02-15T17:51:52.304 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - AppStream 2.0 MB/s | 27 MB 00:13 2024-02-15T17:51:53.789 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 8 - AppStream 1.8 MB/s | 27 MB 00:15 2024-02-15T17:51:57.705 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - BaseOS 17 MB/s | 10 MB 00:00 2024-02-15T17:51:59.621 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - Extras 59 kB/s | 18 kB 00:00 2024-02-15T17:51:59.631 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 8 - BaseOS 17 MB/s | 10 MB 00:00 2024-02-15T17:52:00.272 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - Extras common packages 13 kB/s | 6.9 kB 00:00 2024-02-15T17:52:00.516 INFO:teuthology.orchestra.run.smithi191.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2024-02-15T17:52:01.592 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2024-02-15T17:52:01.623 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 8 - Extras 62 kB/s | 18 kB 00:00 2024-02-15T17:52:02.140 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 8 - Extras common packages 17 kB/s | 6.9 kB 00:00 2024-02-15T17:52:02.400 INFO:teuthology.orchestra.run.smithi042.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-02-15T17:52:05.359 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 432 kB/s | 24 kB 00:00 2024-02-15T17:52:08.323 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-02-15T17:52:08.323 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-02-15T17:52:08.425 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T17:52:08.425 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T17:52:08.425 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-02-15T17:52:08.426 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T17:52:08.426 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T17:52:08.426 INFO:teuthology.orchestra.run.smithi191.stdout: kernel x86_64 4.18.0-539.el8 baseos 10 M 2024-02-15T17:52:08.426 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-02-15T17:52:08.426 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-core x86_64 4.18.0-539.el8 baseos 43 M 2024-02-15T17:52:08.426 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules x86_64 4.18.0-539.el8 baseos 36 M 2024-02-15T17:52:08.426 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T17:52:08.426 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T17:52:08.426 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T17:52:08.426 INFO:teuthology.orchestra.run.smithi191.stdout:Install 3 Packages 2024-02-15T17:52:08.426 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T17:52:08.426 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 90 M 2024-02-15T17:52:08.426 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 97 M 2024-02-15T17:52:08.426 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T17:52:08.828 INFO:teuthology.orchestra.run.smithi191.stdout:(1/3): kernel-4.18.0-539.el8.x86_64.rpm 29 MB/s | 10 MB 00:00 2024-02-15T17:52:09.445 INFO:teuthology.orchestra.run.smithi191.stdout:(2/3): kernel-modules-4.18.0-539.el8.x86_64.rpm 37 MB/s | 36 MB 00:00 2024-02-15T17:52:09.970 INFO:teuthology.orchestra.run.smithi191.stdout:(3/3): kernel-core-4.18.0-539.el8.x86_64.rpm 29 MB/s | 43 MB 00:01 2024-02-15T17:52:09.970 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T17:52:09.970 INFO:teuthology.orchestra.run.smithi191.stdout:Total 58 MB/s | 90 MB 00:01 2024-02-15T17:52:10.654 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T17:52:11.085 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T17:52:11.085 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T17:52:12.314 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T17:52:12.341 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T17:52:15.094 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T17:52:15.600 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-15T17:52:19.133 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-15T17:52:19.344 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 981 kB/s | 16 MB 00:16 2024-02-15T17:52:19.557 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-15T17:52:22.990 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-15T17:52:23.289 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-4.18.0-539.el8.x86_64 3/3 2024-02-15T17:52:23.425 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 316 kB/s | 24 kB 00:00 2024-02-15T17:52:26.588 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-02-15T17:52:26.589 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-02-15T17:52:26.715 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T17:52:26.716 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T17:52:26.716 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-02-15T17:52:26.716 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T17:52:26.716 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T17:52:26.716 INFO:teuthology.orchestra.run.smithi042.stdout: kernel x86_64 4.18.0-539.el8 baseos 10 M 2024-02-15T17:52:26.716 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-02-15T17:52:26.717 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-core x86_64 4.18.0-539.el8 baseos 43 M 2024-02-15T17:52:26.717 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules x86_64 4.18.0-539.el8 baseos 36 M 2024-02-15T17:52:26.717 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T17:52:26.717 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T17:52:26.717 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T17:52:26.717 INFO:teuthology.orchestra.run.smithi042.stdout:Install 3 Packages 2024-02-15T17:52:26.717 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T17:52:26.717 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 90 M 2024-02-15T17:52:26.718 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 97 M 2024-02-15T17:52:26.718 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T17:52:27.110 INFO:teuthology.orchestra.run.smithi042.stdout:(1/3): kernel-4.18.0-539.el8.x86_64.rpm 29 MB/s | 10 MB 00:00 2024-02-15T17:52:27.752 INFO:teuthology.orchestra.run.smithi042.stdout:(2/3): kernel-modules-4.18.0-539.el8.x86_64.rpm 36 MB/s | 36 MB 00:00 2024-02-15T17:52:28.211 INFO:teuthology.orchestra.run.smithi042.stdout:(3/3): kernel-core-4.18.0-539.el8.x86_64.rpm 30 MB/s | 43 MB 00:01 2024-02-15T17:52:28.211 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T17:52:28.211 INFO:teuthology.orchestra.run.smithi042.stdout:Total 60 MB/s | 90 MB 00:01 2024-02-15T17:52:28.926 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T17:52:29.376 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T17:52:29.376 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T17:52:30.687 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T17:52:30.702 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T17:52:33.640 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T17:52:34.249 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-15T17:52:38.018 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 1/3 2024-02-15T17:52:38.399 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-15T17:52:42.077 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 2/3 2024-02-15T17:52:42.359 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-4.18.0-539.el8.x86_64 3/3 2024-02-15T17:52:45.176 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 3/3 2024-02-15T17:52:45.199 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-15T17:52:45.537 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-4.18.0-539.el8.x86_64 3/3 2024-02-15T17:52:45.537 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-4.18.0-539.el8.x86_64 1/3 2024-02-15T17:52:45.537 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-core-4.18.0-539.el8.x86_64 2/3 2024-02-15T17:52:46.057 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-15T17:52:46.057 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T17:52:46.057 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T17:52:46.057 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-4.18.0-539.el8.x86_64 kernel-core-4.18.0-539.el8.x86_64 2024-02-15T17:52:46.057 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules-4.18.0-539.el8.x86_64 2024-02-15T17:52:46.058 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T17:52:46.058 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T17:52:46.201 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2024-02-15T17:52:46.456 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-15T17:52:46.456 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-15T17:52:46.456 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-539.el8.x86_64 2024-02-15T17:52:46.457 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-15T17:52:46.457 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-15T17:52:46.457 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-02-15T17:52:46.457 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi191.front.sepia.ceph.com, path=None, version=distro) 2024-02-15T17:52:46.457 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2024-02-15T17:52:46.667 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-15T17:52:46.667 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-15T17:52:46.667 DEBUG:teuthology.orchestra.run.smithi191:> sudo rpm -qi grub2-tools 2024-02-15T17:52:46.700 INFO:teuthology.orchestra.run.smithi191.stdout:Name : grub2-tools 2024-02-15T17:52:46.701 INFO:teuthology.orchestra.run.smithi191.stdout:Epoch : 1 2024-02-15T17:52:46.701 INFO:teuthology.orchestra.run.smithi191.stdout:Version : 2.02 2024-02-15T17:52:46.701 INFO:teuthology.orchestra.run.smithi191.stdout:Release : 129.el8 2024-02-15T17:52:46.701 INFO:teuthology.orchestra.run.smithi191.stdout:Architecture: x86_64 2024-02-15T17:52:46.701 INFO:teuthology.orchestra.run.smithi191.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-02-15T17:52:46.701 INFO:teuthology.orchestra.run.smithi191.stdout:Group : System Environment/Base 2024-02-15T17:52:46.701 INFO:teuthology.orchestra.run.smithi191.stdout:Size : 9265586 2024-02-15T17:52:46.701 INFO:teuthology.orchestra.run.smithi191.stdout:License : GPLv3+ 2024-02-15T17:52:46.701 INFO:teuthology.orchestra.run.smithi191.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-02-15T17:52:46.701 INFO:teuthology.orchestra.run.smithi191.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-02-15T17:52:46.701 INFO:teuthology.orchestra.run.smithi191.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-02-15T17:52:46.701 INFO:teuthology.orchestra.run.smithi191.stdout:Build Host : kbuilder.bsys.centos.org 2024-02-15T17:52:46.701 INFO:teuthology.orchestra.run.smithi191.stdout:Relocations : (not relocatable) 2024-02-15T17:52:46.701 INFO:teuthology.orchestra.run.smithi191.stdout:Packager : CentOS BuildSystem 2024-02-15T17:52:46.702 INFO:teuthology.orchestra.run.smithi191.stdout:Vendor : CentOS 2024-02-15T17:52:46.702 INFO:teuthology.orchestra.run.smithi191.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-15T17:52:46.702 INFO:teuthology.orchestra.run.smithi191.stdout:Summary : Support tools for GRUB. 2024-02-15T17:52:46.702 INFO:teuthology.orchestra.run.smithi191.stdout:Description : 2024-02-15T17:52:46.702 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T17:52:46.702 INFO:teuthology.orchestra.run.smithi191.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-15T17:52:46.702 INFO:teuthology.orchestra.run.smithi191.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-15T17:52:46.702 INFO:teuthology.orchestra.run.smithi191.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-15T17:52:46.702 INFO:teuthology.orchestra.run.smithi191.stdout:hardware devices. 2024-02-15T17:52:46.702 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T17:52:46.702 INFO:teuthology.orchestra.run.smithi191.stdout:This subpackage provides tools for support of all platforms. 2024-02-15T17:52:46.703 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-15T17:52:46.703 INFO:teuthology.task.kernel:Updating grub on smithi191 to boot 4.18.0-539.el8.x86_64 2024-02-15T17:52:46.703 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-15T17:52:48.438 INFO:teuthology.orchestra.run.smithi191.stderr:Generating grub configuration file ... 2024-02-15T17:52:50.352 INFO:teuthology.orchestra.run.smithi191.stderr:done 2024-02-15T17:52:50.354 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2024-02-15T17:52:50.377 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.bXWvs0Fjtd 2024-02-15T17:52:50.377 DEBUG:teuthology.orchestra.run.smithi191:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bXWvs0Fjtd 2024-02-15T17:52:50.451 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.bXWvs0Fjtd 2024-02-15T17:52:50.583 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.bXWvs0Fjtd is 6KB 2024-02-15T17:52:50.631 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.bXWvs0Fjtd 2024-02-15T17:52:50.648 DEBUG:teuthology.orchestra.run.smithi191:> sudo /bin/ls /boot/loader/entries || true 2024-02-15T17:52:50.717 INFO:teuthology.orchestra.run.smithi191.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-02-15T17:52:50.717 INFO:teuthology.orchestra.run.smithi191.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-02-15T17:52:50.717 INFO:teuthology.orchestra.run.smithi191.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-02-15T17:52:50.717 INFO:teuthology.orchestra.run.smithi191.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64.conf 2024-02-15T17:52:50.717 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64 2024-02-15T17:52:50.865 DEBUG:teuthology.orchestra.run.smithi191:> sudo shutdown -r now 2024-02-15T17:53:05.849 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-core-4.18.0-539.el8.x86_64 3/3 2024-02-15T17:53:05.872 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-15T17:53:06.262 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-4.18.0-539.el8.x86_64 3/3 2024-02-15T17:53:06.262 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-4.18.0-539.el8.x86_64 1/3 2024-02-15T17:53:06.262 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-core-4.18.0-539.el8.x86_64 2/3 2024-02-15T17:53:06.803 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-modules-4.18.0-539.el8.x86_64 3/3 2024-02-15T17:53:06.803 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T17:53:06.803 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T17:53:06.803 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-4.18.0-539.el8.x86_64 kernel-core-4.18.0-539.el8.x86_64 2024-02-15T17:53:06.803 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules-4.18.0-539.el8.x86_64 2024-02-15T17:53:06.803 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T17:53:06.803 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T17:53:06.934 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q kernel | sort -rV | head -n 1 2024-02-15T17:53:07.165 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-15T17:53:07.165 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-15T17:53:07.166 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-539.el8.x86_64 2024-02-15T17:53:07.166 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-15T17:53:07.166 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-15T17:53:07.166 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-02-15T17:53:07.166 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi042.front.sepia.ceph.com, path=None, version=distro) 2024-02-15T17:53:07.166 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q kernel | sort -rV | head -n 1 2024-02-15T17:53:07.384 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-4.18.0-539.el8.x86_64 2024-02-15T17:53:07.385 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-539.el8.x86_64 2024-02-15T17:53:07.385 DEBUG:teuthology.orchestra.run.smithi042:> sudo rpm -qi grub2-tools 2024-02-15T17:53:07.419 INFO:teuthology.orchestra.run.smithi042.stdout:Name : grub2-tools 2024-02-15T17:53:07.419 INFO:teuthology.orchestra.run.smithi042.stdout:Epoch : 1 2024-02-15T17:53:07.419 INFO:teuthology.orchestra.run.smithi042.stdout:Version : 2.02 2024-02-15T17:53:07.419 INFO:teuthology.orchestra.run.smithi042.stdout:Release : 129.el8 2024-02-15T17:53:07.419 INFO:teuthology.orchestra.run.smithi042.stdout:Architecture: x86_64 2024-02-15T17:53:07.420 INFO:teuthology.orchestra.run.smithi042.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-02-15T17:53:07.420 INFO:teuthology.orchestra.run.smithi042.stdout:Group : System Environment/Base 2024-02-15T17:53:07.420 INFO:teuthology.orchestra.run.smithi042.stdout:Size : 9265586 2024-02-15T17:53:07.420 INFO:teuthology.orchestra.run.smithi042.stdout:License : GPLv3+ 2024-02-15T17:53:07.420 INFO:teuthology.orchestra.run.smithi042.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-02-15T17:53:07.420 INFO:teuthology.orchestra.run.smithi042.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-02-15T17:53:07.420 INFO:teuthology.orchestra.run.smithi042.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-02-15T17:53:07.420 INFO:teuthology.orchestra.run.smithi042.stdout:Build Host : kbuilder.bsys.centos.org 2024-02-15T17:53:07.420 INFO:teuthology.orchestra.run.smithi042.stdout:Relocations : (not relocatable) 2024-02-15T17:53:07.420 INFO:teuthology.orchestra.run.smithi042.stdout:Packager : CentOS BuildSystem 2024-02-15T17:53:07.420 INFO:teuthology.orchestra.run.smithi042.stdout:Vendor : CentOS 2024-02-15T17:53:07.420 INFO:teuthology.orchestra.run.smithi042.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-15T17:53:07.420 INFO:teuthology.orchestra.run.smithi042.stdout:Summary : Support tools for GRUB. 2024-02-15T17:53:07.421 INFO:teuthology.orchestra.run.smithi042.stdout:Description : 2024-02-15T17:53:07.421 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T17:53:07.421 INFO:teuthology.orchestra.run.smithi042.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-15T17:53:07.421 INFO:teuthology.orchestra.run.smithi042.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-15T17:53:07.421 INFO:teuthology.orchestra.run.smithi042.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-15T17:53:07.421 INFO:teuthology.orchestra.run.smithi042.stdout:hardware devices. 2024-02-15T17:53:07.421 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T17:53:07.421 INFO:teuthology.orchestra.run.smithi042.stdout:This subpackage provides tools for support of all platforms. 2024-02-15T17:53:07.422 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-15T17:53:07.422 INFO:teuthology.task.kernel:Updating grub on smithi042 to boot 4.18.0-539.el8.x86_64 2024-02-15T17:53:07.422 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-15T17:53:08.938 INFO:teuthology.orchestra.run.smithi042.stderr:Generating grub configuration file ... 2024-02-15T17:53:10.615 INFO:teuthology.orchestra.run.smithi042.stderr:done 2024-02-15T17:53:10.617 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-02-15T17:53:10.644 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.puSIAz4A2P 2024-02-15T17:53:10.644 DEBUG:teuthology.orchestra.run.smithi042:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.puSIAz4A2P 2024-02-15T17:53:10.716 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.puSIAz4A2P 2024-02-15T17:53:10.860 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.puSIAz4A2P is 6KB 2024-02-15T17:53:10.908 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.puSIAz4A2P 2024-02-15T17:53:10.926 DEBUG:teuthology.orchestra.run.smithi042:> sudo /bin/ls /boot/loader/entries || true 2024-02-15T17:53:10.992 INFO:teuthology.orchestra.run.smithi042.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-02-15T17:53:10.992 INFO:teuthology.orchestra.run.smithi042.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-02-15T17:53:10.992 INFO:teuthology.orchestra.run.smithi042.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-02-15T17:53:10.993 INFO:teuthology.orchestra.run.smithi042.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64.conf 2024-02-15T17:53:10.993 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-539.el8.x86_64 2024-02-15T17:53:11.154 DEBUG:teuthology.orchestra.run.smithi042:> sudo shutdown -r now 2024-02-15T17:53:20.876 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-02-15T17:53:20.877 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-15T17:53:20.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T17:53:39.388 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-02-15T17:53:41.158 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-02-15T17:53:41.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-02-15T17:53:41.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T17:53:48.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-15T17:53:48.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T17:53:51.456 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-02-15T17:53:59.616 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-02-15T17:54:03.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-15T17:54:03.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T17:54:08.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-02-15T17:54:08.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T17:54:21.884 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-02-15T17:54:36.887 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-15T17:54:36.888 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T17:54:37.965 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-02-15T17:54:39.142 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-15T17:54:39.142 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "4.18.0-539.el8.x86_64"... 2024-02-15T17:54:39.142 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-02-15T17:54:39.176 INFO:teuthology.orchestra.run.smithi191.stdout:4.18.0-539.el8.x86_64 2024-02-15T17:54:39.177 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-539.el8.x86_64 vs 4.18.0-539.el8.x86_64 2024-02-15T17:54:39.177 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-15T17:54:39.177 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-15T17:54:40.178 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-02-15T17:54:40.178 DEBUG:teuthology.orchestra.run.smithi191:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-15T17:54:40.256 INFO:teuthology.orchestra.run.smithi191.stdout:ttyS1 2024-02-15T17:54:40.270 DEBUG:teuthology.parallel:result is None 2024-02-15T17:55:08.627 DEBUG:teuthology.orchestra.remote:timed out 2024-02-15T17:55:20.628 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-02-15T17:55:20.629 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T17:55:20.950 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-02-15T17:55:21.243 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-02-15T17:55:21.243 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "4.18.0-539.el8.x86_64"... 2024-02-15T17:55:21.243 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-02-15T17:55:21.304 INFO:teuthology.orchestra.run.smithi042.stdout:4.18.0-539.el8.x86_64 2024-02-15T17:55:21.304 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-539.el8.x86_64 vs 4.18.0-539.el8.x86_64 2024-02-15T17:55:21.304 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-15T17:55:21.304 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-15T17:55:22.305 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-02-15T17:55:22.305 DEBUG:teuthology.orchestra.run.smithi042:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-15T17:55:22.385 INFO:teuthology.orchestra.run.smithi042.stdout:ttyS1 2024-02-15T17:55:22.397 DEBUG:teuthology.parallel:result is None 2024-02-15T17:55:22.398 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-15T17:55:22.405 INFO:teuthology.task.internal:Creating test directory... 2024-02-15T17:55:22.405 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-15T17:55:22.408 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-15T17:55:22.430 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-15T17:55:22.436 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-15T17:55:22.470 INFO:teuthology.task.internal:Creating archive directory... 2024-02-15T17:55:22.471 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-15T17:55:22.474 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-15T17:55:22.508 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-15T17:55:22.523 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-15T17:55:22.524 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-15T17:55:22.545 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-15T17:55:22.588 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-15T17:55:22.590 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-15T17:55:22.600 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-15T17:55:22.602 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-15T17:55:22.603 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-15T17:55:22.610 INFO:teuthology.task.internal:Configuring sudo... 2024-02-15T17:55:22.610 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-15T17:55:22.612 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-15T17:55:22.642 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-15T17:55:22.654 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-15T17:55:22.654 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-15T17:55:22.685 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-15T17:55:22.704 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-15T17:55:22.786 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-15T17:55:22.851 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-15T17:55:22.893 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-15T17:55:22.940 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T17:55:22.940 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-15T17:55:23.008 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-15T17:55:23.054 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-15T17:55:23.118 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-15T17:55:23.166 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-15T17:55:23.209 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-15T17:55:23.209 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-15T17:55:23.277 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-02-15T17:55:23.279 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2024-02-15T17:55:23.332 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-15T17:55:23.383 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-15T17:55:23.576 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-15T17:55:23.586 INFO:teuthology.task.internal:Starting timer... 2024-02-15T17:55:23.587 INFO:teuthology.run_tasks:Running task pcp... 2024-02-15T17:55:23.615 INFO:teuthology.run_tasks:Running task selinux... 2024-02-15T17:55:23.656 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-15T17:55:23.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo service auditd rotate 2024-02-15T17:55:23.740 INFO:teuthology.orchestra.run.smithi042.stdout:Rotating logs: 2024-02-15T17:55:23.742 DEBUG:teuthology.orchestra.run.smithi191:> sudo service auditd rotate 2024-02-15T17:55:23.803 INFO:teuthology.orchestra.run.smithi191.stdout:Rotating logs: 2024-02-15T17:55:23.804 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-15T17:55:23.805 DEBUG:teuthology.orchestra.run.smithi042:> /usr/sbin/getenforce 2024-02-15T17:55:23.829 INFO:teuthology.orchestra.run.smithi042.stdout:Permissive 2024-02-15T17:55:23.830 DEBUG:teuthology.orchestra.run.smithi191:> /usr/sbin/getenforce 2024-02-15T17:55:23.989 INFO:teuthology.orchestra.run.smithi191.stdout:Permissive 2024-02-15T17:55:23.990 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi042.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi191.front.sepia.ceph.com': 'permissive'} 2024-02-15T17:55:23.990 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-15T17:55:24.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T17:55:24.023 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-15T17:55:24.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T17:55:24.051 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-15T17:55:24.051 DEBUG:teuthology.orchestra.run.smithi042:> sudo /usr/sbin/setenforce permissive 2024-02-15T17:55:24.092 DEBUG:teuthology.orchestra.run.smithi191:> sudo /usr/sbin/setenforce permissive 2024-02-15T17:55:24.119 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-15T17:55:24.130 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-15T17:55:24.153 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-15T17:55:24.154 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi042.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-15T18:05:28.766 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi042.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2024-02-15T18:05:28.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-02-15T18:05:28.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T18:05:28.845 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-02-15T18:05:28.915 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-02-15T18:05:28.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-15T18:05:28.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-15T18:05:28.989 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-02-15T18:05:29.018 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-15T18:05:29.019 INFO:teuthology.run_tasks:Running task clock... 2024-02-15T18:05:29.030 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-15T18:05:29.030 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-15T18:05:29.031 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-15T18:05:29.033 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-15T18:05:29.033 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-15T18:05:29.065 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-15T18:05:29.080 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-15T18:05:29.092 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-15T18:05:29.107 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-15T18:05:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: ntpd: command not found 2024-02-15T18:05:29.165 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: ntpd: command not found 2024-02-15T18:05:29.169 INFO:teuthology.orchestra.run.smithi042.stdout:506 Cannot talk to daemon 2024-02-15T18:05:29.176 INFO:teuthology.orchestra.run.smithi191.stdout:506 Cannot talk to daemon 2024-02-15T18:05:29.184 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-15T18:05:29.190 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-15T18:05:29.198 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-15T18:05:29.203 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-15T18:05:29.261 INFO:teuthology.orchestra.run.smithi042.stderr:bash: ntpq: command not found 2024-02-15T18:05:29.265 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-15T18:05:29.265 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-02-15T18:05:29.266 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:05:29.266 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:05:29.266 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:05:29.266 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:05:29.274 INFO:teuthology.orchestra.run.smithi191.stderr:bash: ntpq: command not found 2024-02-15T18:05:29.277 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-15T18:05:29.277 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-02-15T18:05:29.277 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:05:29.277 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:05:29.277 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:05:29.277 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:05:29.278 INFO:teuthology.run_tasks:Running task pexec... 2024-02-15T18:05:29.288 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-15T18:05:29.289 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi042.front.sepia.ceph.com 2024-02-15T18:05:29.289 DEBUG:teuthology.orchestra.run.smithi042:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-15T18:05:29.289 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi191.front.sepia.ceph.com 2024-02-15T18:05:29.289 DEBUG:teuthology.orchestra.run.smithi191:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-15T18:05:30.106 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:05:39 ago on Thu 15 Feb 2024 05:59:50 PM UTC. 2024-02-15T18:05:30.126 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:05:35 ago on Thu 15 Feb 2024 05:59:54 PM UTC. 2024-02-15T18:05:32.131 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:05:32.132 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:05:32.132 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-02-15T18:05:32.132 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:05:32.132 INFO:teuthology.orchestra.run.smithi191.stdout:Resetting modules: 2024-02-15T18:05:32.132 INFO:teuthology.orchestra.run.smithi191.stdout: container-tools 2024-02-15T18:05:32.132 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:05:32.132 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:05:32.132 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:05:32.132 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:05:32.323 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:05:32.324 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:05:32.324 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-02-15T18:05:32.324 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:05:32.324 INFO:teuthology.orchestra.run.smithi042.stdout:Resetting modules: 2024-02-15T18:05:32.324 INFO:teuthology.orchestra.run.smithi042.stdout: container-tools 2024-02-15T18:05:32.324 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:05:32.324 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:05:32.324 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:05:32.324 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:05:32.378 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:05:32.591 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:05:33.233 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:05:43 ago on Thu 15 Feb 2024 05:59:50 PM UTC. 2024-02-15T18:05:33.475 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:05:39 ago on Thu 15 Feb 2024 05:59:54 PM UTC. 2024-02-15T18:05:35.262 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:05:35.297 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:05:35.297 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-02-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi191.stdout:Installing group/module packages: 2024-02-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi191.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-02-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi191.stdout: buildah x86_64 2:1.33.5-1.module_el8+885+7da147f3 appstream 9.7 M 2024-02-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi191.stdout: cockpit-podman noarch 83-1.module_el8+847+7863d4e6 appstream 688 k 2024-02-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi191.stdout: crun x86_64 1.14-1.module_el8+871+552be958 appstream 257 k 2024-02-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi191.stdout: netavark x86_64 2:1.10.2-1.module_el8+885+7da147f3 appstream 4.3 M 2024-02-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi191.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-02-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi191.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-02-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi191.stdout: toolbox x86_64 0.0.99.5-1.module_el8+847+7863d4e6 appstream 2.5 M 2024-02-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi191.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-02-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-02-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-02-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-02-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi191.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-02-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi191.stdout:Installing weak dependencies: 2024-02-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi191.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-02-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi191.stdout:Installing module profiles: 2024-02-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi191.stdout: container-tools/common 2024-02-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi191.stdout:Enabling module streams: 2024-02-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi191.stdout: container-tools rhel8 2024-02-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi191.stdout:Install 13 Packages 2024-02-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:05:35.300 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 28 M 2024-02-15T18:05:35.300 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 92 M 2024-02-15T18:05:35.300 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:05:35.643 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:05:35.682 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:05:35.682 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-02-15T18:05:35.682 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:05:35.682 INFO:teuthology.orchestra.run.smithi042.stdout:Installing group/module packages: 2024-02-15T18:05:35.682 INFO:teuthology.orchestra.run.smithi042.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-02-15T18:05:35.682 INFO:teuthology.orchestra.run.smithi042.stdout: buildah x86_64 2:1.33.5-1.module_el8+885+7da147f3 appstream 9.7 M 2024-02-15T18:05:35.682 INFO:teuthology.orchestra.run.smithi042.stdout: cockpit-podman noarch 83-1.module_el8+847+7863d4e6 appstream 688 k 2024-02-15T18:05:35.682 INFO:teuthology.orchestra.run.smithi042.stdout: crun x86_64 1.14-1.module_el8+871+552be958 appstream 257 k 2024-02-15T18:05:35.682 INFO:teuthology.orchestra.run.smithi042.stdout: netavark x86_64 2:1.10.2-1.module_el8+885+7da147f3 appstream 4.3 M 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout: toolbox x86_64 0.0.99.5-1.module_el8+847+7863d4e6 appstream 2.5 M 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout:Installing weak dependencies: 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout:Installing module profiles: 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout: container-tools/common 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling module streams: 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout: container-tools rhel8 2024-02-15T18:05:35.683 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:05:35.684 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:05:35.684 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:05:35.684 INFO:teuthology.orchestra.run.smithi042.stdout:Install 13 Packages 2024-02-15T18:05:35.684 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:05:35.684 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 28 M 2024-02-15T18:05:35.684 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 92 M 2024-02-15T18:05:35.685 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:05:41.813 INFO:teuthology.orchestra.run.smithi191.stdout:(1/13): aardvark-dns-1.10.0-1.module_el8+871+55 159 kB/s | 1.0 MB 00:06 2024-02-15T18:05:41.863 INFO:teuthology.orchestra.run.smithi191.stdout:(2/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 758 kB/s | 38 kB 00:00 2024-02-15T18:05:41.997 INFO:teuthology.orchestra.run.smithi191.stdout:(3/13): crun-1.14-1.module_el8+871+552be958.x86 1.9 MB/s | 257 kB 00:00 2024-02-15T18:05:43.205 INFO:teuthology.orchestra.run.smithi191.stdout:(4/13): netavark-1.10.2-1.module_el8+885+7da147 3.6 MB/s | 4.3 MB 00:01 2024-02-15T18:05:43.597 INFO:teuthology.orchestra.run.smithi191.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 460 kB/s | 180 kB 00:00 2024-02-15T18:05:43.647 INFO:teuthology.orchestra.run.smithi191.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 497 kB/s | 25 kB 00:00 2024-02-15T18:05:43.756 INFO:teuthology.orchestra.run.smithi191.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 868 kB/s | 94 kB 00:00 2024-02-15T18:05:44.889 INFO:teuthology.orchestra.run.smithi191.stdout:(8/13): skopeo-1.14.3-0.1.module_el8+885+7da147 7.8 MB/s | 8.8 MB 00:01 2024-02-15T18:05:45.448 INFO:teuthology.orchestra.run.smithi191.stdout:(9/13): toolbox-0.0.99.5-1.module_el8+847+7863d 4.5 MB/s | 2.5 MB 00:00 2024-02-15T18:05:45.498 INFO:teuthology.orchestra.run.smithi191.stdout:(10/13): udica-0.2.6-20.module_el8+804+f131391c 1.0 MB/s | 53 kB 00:00 2024-02-15T18:05:45.556 INFO:teuthology.orchestra.run.smithi191.stdout:(11/13): yajl-2.1.0-12.el8.x86_64.rpm 702 kB/s | 41 kB 00:00 2024-02-15T18:05:45.832 INFO:teuthology.orchestra.run.smithi191.stdout:(12/13): cockpit-podman-83-1.module_el8+847+786 66 kB/s | 688 kB 00:10 2024-02-15T18:05:47.057 INFO:teuthology.orchestra.run.smithi191.stdout:(13/13): buildah-1.33.5-1.module_el8+885+7da147 844 kB/s | 9.7 MB 00:11 2024-02-15T18:05:47.057 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:05:47.057 INFO:teuthology.orchestra.run.smithi191.stdout:Total 2.4 MB/s | 28 MB 00:11 2024-02-15T18:05:47.330 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:05:47.350 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:05:47.350 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:05:47.631 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:05:47.634 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:05:48.350 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:05:49.291 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-15T18:05:49.472 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-02-15T18:05:49.826 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-02-15T18:05:50.046 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-02-15T18:05:50.261 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-02-15T18:05:50.890 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-02-15T18:05:51.884 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 7/13 2024-02-15T18:05:52.132 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 8/13 2024-02-15T18:05:52.346 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : crun-1.14-1.module_el8+871+552be958.x86_64 9/13 2024-02-15T18:05:52.816 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-02-15T18:05:53.057 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-15T18:05:53.287 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-15T18:05:53.475 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-15T18:05:53.894 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-15T18:05:53.895 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-02-15T18:05:53.895 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2/13 2024-02-15T18:05:53.895 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 3/13 2024-02-15T18:05:53.895 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-02-15T18:05:53.895 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : crun-1.14-1.module_el8+871+552be958.x86_64 5/13 2024-02-15T18:05:53.895 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 6/13 2024-02-15T18:05:53.895 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-02-15T18:05:53.895 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-02-15T18:05:53.895 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-02-15T18:05:53.895 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-02-15T18:05:53.895 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-15T18:05:53.895 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-15T18:05:54.580 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-15T18:05:54.580 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:05:54.580 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:05:54.580 INFO:teuthology.orchestra.run.smithi191.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-02-15T18:05:54.580 INFO:teuthology.orchestra.run.smithi191.stdout: buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2024-02-15T18:05:54.580 INFO:teuthology.orchestra.run.smithi191.stdout: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 2024-02-15T18:05:54.580 INFO:teuthology.orchestra.run.smithi191.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-02-15T18:05:54.580 INFO:teuthology.orchestra.run.smithi191.stdout: crun-1.14-1.module_el8+871+552be958.x86_64 2024-02-15T18:05:54.580 INFO:teuthology.orchestra.run.smithi191.stdout: netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 2024-02-15T18:05:54.580 INFO:teuthology.orchestra.run.smithi191.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-02-15T18:05:54.580 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-15T18:05:54.580 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-15T18:05:54.581 INFO:teuthology.orchestra.run.smithi191.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-02-15T18:05:54.581 INFO:teuthology.orchestra.run.smithi191.stdout: toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 2024-02-15T18:05:54.581 INFO:teuthology.orchestra.run.smithi191.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-02-15T18:05:54.581 INFO:teuthology.orchestra.run.smithi191.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-15T18:05:54.581 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:05:54.581 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:05:54.886 DEBUG:teuthology.parallel:result is None 2024-02-15T18:07:44.371 INFO:teuthology.orchestra.run.smithi042.stdout:[MIRROR] aardvark-dns-1.10.0-1.module_el8+871+552be958.x86_64.rpm: Curl error (7): Couldn't connect to server for http://packages.oit.ncsu.edu/centos/8-stream/AppStream/x86_64/os/Packages/aardvark-dns-1.10.0-1.module_el8%2b871%2b552be958.x86_64.rpm [Failed to connect to packages.oit.ncsu.edu port 80: Connection timed out] 2024-02-15T18:07:44.371 INFO:teuthology.orchestra.run.smithi042.stdout:[MIRROR] buildah-1.33.5-1.module_el8+885+7da147f3.x86_64.rpm: Curl error (7): Couldn't connect to server for http://packages.oit.ncsu.edu/centos/8-stream/AppStream/x86_64/os/Packages/buildah-1.33.5-1.module_el8%2b885%2b7da147f3.x86_64.rpm [Failed to connect to packages.oit.ncsu.edu port 80: Connection timed out] 2024-02-15T18:07:44.371 INFO:teuthology.orchestra.run.smithi042.stdout:[MIRROR] cockpit-podman-83-1.module_el8+847+7863d4e6.noarch.rpm: Curl error (7): Couldn't connect to server for http://packages.oit.ncsu.edu/centos/8-stream/AppStream/x86_64/os/Packages/cockpit-podman-83-1.module_el8%2b847%2b7863d4e6.noarch.rpm [Failed to connect to packages.oit.ncsu.edu port 80: Connection timed out] 2024-02-15T18:07:44.478 INFO:teuthology.orchestra.run.smithi042.stdout:(1/13): cockpit-podman-83-1.module_el8+847+7863 5.4 kB/s | 688 kB 02:08 2024-02-15T18:07:44.520 INFO:teuthology.orchestra.run.smithi042.stdout:(2/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 911 kB/s | 38 kB 00:00 2024-02-15T18:07:44.562 INFO:teuthology.orchestra.run.smithi042.stdout:(3/13): aardvark-dns-1.10.0-1.module_el8+871+55 8.0 kB/s | 1.0 MB 02:08 2024-02-15T18:07:44.587 INFO:teuthology.orchestra.run.smithi042.stdout:(4/13): crun-1.14-1.module_el8+871+552be958.x86 3.7 MB/s | 257 kB 00:00 2024-02-15T18:07:44.646 INFO:teuthology.orchestra.run.smithi042.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 3.0 MB/s | 180 kB 00:00 2024-02-15T18:07:44.680 INFO:teuthology.orchestra.run.smithi042.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 744 kB/s | 25 kB 00:00 2024-02-15T18:07:44.747 INFO:teuthology.orchestra.run.smithi042.stdout:(7/13): netavark-1.10.2-1.module_el8+885+7da147 23 MB/s | 4.3 MB 00:00 2024-02-15T18:07:44.802 INFO:teuthology.orchestra.run.smithi042.stdout:(8/13): python3-pyxdg-0.25-16.el8.noarch.rpm 772 kB/s | 94 kB 00:00 2024-02-15T18:07:44.931 INFO:teuthology.orchestra.run.smithi042.stdout:(9/13): buildah-1.33.5-1.module_el8+885+7da147f 77 kB/s | 9.7 MB 02:09 2024-02-15T18:07:44.973 INFO:teuthology.orchestra.run.smithi042.stdout:(10/13): udica-0.2.6-20.module_el8+804+f131391c 1.2 MB/s | 53 kB 00:00 2024-02-15T18:07:45.023 INFO:teuthology.orchestra.run.smithi042.stdout:(11/13): toolbox-0.0.99.5-1.module_el8+847+7863 11 MB/s | 2.5 MB 00:00 2024-02-15T18:07:45.048 INFO:teuthology.orchestra.run.smithi042.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 546 kB/s | 41 kB 00:00 2024-02-15T18:07:45.165 INFO:teuthology.orchestra.run.smithi042.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 21 MB/s | 8.8 MB 00:00 2024-02-15T18:07:45.166 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:07:45.166 INFO:teuthology.orchestra.run.smithi042.stdout:Total 219 kB/s | 28 MB 02:09 2024-02-15T18:07:45.440 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:07:45.462 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:07:45.462 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:07:45.735 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:07:45.738 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:07:46.422 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:07:47.429 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-15T18:07:47.632 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-02-15T18:07:47.935 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-02-15T18:07:48.111 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-02-15T18:07:48.326 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-02-15T18:07:48.899 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-02-15T18:07:49.884 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 7/13 2024-02-15T18:07:50.090 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 8/13 2024-02-15T18:07:50.313 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : crun-1.14-1.module_el8+871+552be958.x86_64 9/13 2024-02-15T18:07:50.762 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-02-15T18:07:50.998 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-15T18:07:51.223 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-15T18:07:51.425 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-15T18:07:51.869 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 13/13 2024-02-15T18:07:51.869 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-02-15T18:07:51.869 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2/13 2024-02-15T18:07:51.869 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 3/13 2024-02-15T18:07:51.869 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-02-15T18:07:51.870 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : crun-1.14-1.module_el8+871+552be958.x86_64 5/13 2024-02-15T18:07:51.870 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 6/13 2024-02-15T18:07:51.870 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-02-15T18:07:51.870 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-02-15T18:07:51.870 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-02-15T18:07:51.870 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-02-15T18:07:51.870 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 11/13 2024-02-15T18:07:51.870 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-02-15T18:07:52.434 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-15T18:07:52.434 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:07:52.434 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:07:52.434 INFO:teuthology.orchestra.run.smithi042.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-02-15T18:07:52.434 INFO:teuthology.orchestra.run.smithi042.stdout: buildah-2:1.33.5-1.module_el8+885+7da147f3.x86_64 2024-02-15T18:07:52.434 INFO:teuthology.orchestra.run.smithi042.stdout: cockpit-podman-83-1.module_el8+847+7863d4e6.noarch 2024-02-15T18:07:52.434 INFO:teuthology.orchestra.run.smithi042.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-02-15T18:07:52.434 INFO:teuthology.orchestra.run.smithi042.stdout: crun-1.14-1.module_el8+871+552be958.x86_64 2024-02-15T18:07:52.434 INFO:teuthology.orchestra.run.smithi042.stdout: netavark-2:1.10.2-1.module_el8+885+7da147f3.x86_64 2024-02-15T18:07:52.435 INFO:teuthology.orchestra.run.smithi042.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-02-15T18:07:52.435 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-15T18:07:52.435 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-15T18:07:52.435 INFO:teuthology.orchestra.run.smithi042.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-02-15T18:07:52.435 INFO:teuthology.orchestra.run.smithi042.stdout: toolbox-0.0.99.5-1.module_el8+847+7863d4e6.x86_64 2024-02-15T18:07:52.435 INFO:teuthology.orchestra.run.smithi042.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-02-15T18:07:52.435 INFO:teuthology.orchestra.run.smithi042.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-15T18:07:52.435 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:07:52.435 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:07:52.748 DEBUG:teuthology.parallel:result is None 2024-02-15T18:07:52.748 INFO:teuthology.run_tasks:Running task pexec... 2024-02-15T18:07:52.757 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-15T18:07:52.758 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi042.front.sepia.ceph.com 2024-02-15T18:07:52.758 DEBUG:teuthology.orchestra.run.smithi042:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-15T18:07:52.759 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi191.front.sepia.ceph.com 2024-02-15T18:07:52.759 DEBUG:teuthology.orchestra.run.smithi191:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-15T18:07:52.954 DEBUG:teuthology.parallel:result is None 2024-02-15T18:07:52.956 DEBUG:teuthology.parallel:result is None 2024-02-15T18:07:52.956 INFO:teuthology.run_tasks:Running task install... 2024-02-15T18:07:52.965 DEBUG:teuthology.task.install:project ceph 2024-02-15T18:07:52.966 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1'}} 2024-02-15T18:07:52.966 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v16.2.4', 'flavor': 'default', 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1'} 2024-02-15T18:07:52.966 INFO:teuthology.task.install:Using flavor: default 2024-02-15T18:07:52.975 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-15T18:07:52.975 INFO:teuthology.task.install:extra packages: [] 2024-02-15T18:07:52.975 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1', 'tag': 'v16.2.4', 'wait_for_package': False} 2024-02-15T18:07:52.975 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-15T18:07:52.976 INFO:teuthology.packaging:ref: None 2024-02-15T18:07:52.976 INFO:teuthology.packaging:tag: v16.2.4 2024-02-15T18:07:52.976 INFO:teuthology.packaging:branch: None 2024-02-15T18:07:52.976 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:07:53.275 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph-ci v16.2.4^{} -> 3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2024-02-15T18:07:53.276 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2024-02-15T18:07:53.278 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1', 'tag': 'v16.2.4', 'wait_for_package': False} 2024-02-15T18:07:53.279 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-15T18:07:53.279 INFO:teuthology.packaging:ref: None 2024-02-15T18:07:53.279 INFO:teuthology.packaging:tag: v16.2.4 2024-02-15T18:07:53.279 INFO:teuthology.packaging:branch: None 2024-02-15T18:07:53.279 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:07:53.279 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2024-02-15T18:07:53.426 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/ 2024-02-15T18:07:53.426 INFO:teuthology.task.install.rpm:Package version is 16.2.4-0 2024-02-15T18:07:53.440 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/ 2024-02-15T18:07:53.441 INFO:teuthology.task.install.rpm:Package version is 16.2.4-0 2024-02-15T18:07:53.472 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-15T18:07:53.472 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:07:53.472 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-15T18:07:53.480 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-15T18:07:53.480 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-15T18:07:53.480 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-15T18:07:53.505 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-15T18:07:53.505 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-15T18:07:53.505 INFO:teuthology.packaging:ref: None 2024-02-15T18:07:53.505 INFO:teuthology.packaging:tag: v16.2.4 2024-02-15T18:07:53.505 INFO:teuthology.packaging:branch: None 2024-02-15T18:07:53.505 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:07:53.506 DEBUG:teuthology.orchestra.run.smithi042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v16.2.4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-15T18:07:53.512 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-15T18:07:53.512 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-15T18:07:53.513 INFO:teuthology.packaging:ref: None 2024-02-15T18:07:53.513 INFO:teuthology.packaging:tag: v16.2.4 2024-02-15T18:07:53.513 INFO:teuthology.packaging:branch: None 2024-02-15T18:07:53.513 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:07:53.513 DEBUG:teuthology.orchestra.run.smithi191:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v16.2.4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-15T18:07:53.577 DEBUG:teuthology.orchestra.run.smithi042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-15T18:07:53.584 DEBUG:teuthology.orchestra.run.smithi191:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-15T18:07:53.657 DEBUG:teuthology.orchestra.run.smithi042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-15T18:07:53.664 DEBUG:teuthology.orchestra.run.smithi191:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-15T18:07:53.725 INFO:teuthology.orchestra.run.smithi042.stdout:check_obsoletes = 1 2024-02-15T18:07:53.726 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum clean all 2024-02-15T18:07:53.732 INFO:teuthology.orchestra.run.smithi191.stdout:check_obsoletes = 1 2024-02-15T18:07:53.734 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum clean all 2024-02-15T18:07:54.005 INFO:teuthology.orchestra.run.smithi191.stdout:76 files removed 2024-02-15T18:07:54.012 INFO:teuthology.orchestra.run.smithi042.stdout:76 files removed 2024-02-15T18:07:54.033 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-radosgw 2024-02-15T18:07:54.041 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-radosgw 2024-02-15T18:07:54.537 INFO:teuthology.orchestra.run.smithi191.stdout:ceph packages for x86_64 764 kB/s | 74 kB 00:00 2024-02-15T18:07:54.566 INFO:teuthology.orchestra.run.smithi042.stdout:ceph packages for x86_64 765 kB/s | 74 kB 00:00 2024-02-15T18:07:54.654 INFO:teuthology.orchestra.run.smithi191.stdout:ceph noarch packages 194 kB/s | 15 kB 00:00 2024-02-15T18:07:54.683 INFO:teuthology.orchestra.run.smithi042.stdout:ceph noarch packages 199 kB/s | 15 kB 00:00 2024-02-15T18:07:54.738 INFO:teuthology.orchestra.run.smithi191.stdout:ceph source packages 32 kB/s | 1.7 kB 00:00 2024-02-15T18:07:54.825 INFO:teuthology.orchestra.run.smithi042.stdout:ceph source packages 16 kB/s | 1.7 kB 00:00 2024-02-15T18:08:19.676 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - AppStream 1.1 MB/s | 27 MB 00:24 2024-02-15T18:08:21.807 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 8 - AppStream 1.0 MB/s | 27 MB 00:26 2024-02-15T18:08:24.951 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - BaseOS 23 MB/s | 10 MB 00:00 2024-02-15T18:08:27.012 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - Extras 36 kB/s | 18 kB 00:00 2024-02-15T18:08:27.400 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 8 - BaseOS 25 MB/s | 10 MB 00:00 2024-02-15T18:08:27.736 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - Extras common packages 11 kB/s | 6.9 kB 00:00 2024-02-15T18:08:28.687 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 8 - PowerTools 5.6 MB/s | 4.8 MB 00:00 2024-02-15T18:08:29.517 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 8 - Extras 44 kB/s | 18 kB 00:00 2024-02-15T18:08:29.838 INFO:teuthology.orchestra.run.smithi191.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2024-02-15T18:08:30.034 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 8 - Extras common packages 17 kB/s | 6.9 kB 00:00 2024-02-15T18:08:31.136 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 8 - PowerTools 4.9 MB/s | 4.8 MB 00:00 2024-02-15T18:08:32.262 INFO:teuthology.orchestra.run.smithi042.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2024-02-15T18:08:33.690 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2024-02-15T18:08:37.680 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 397 kB/s | 24 kB 00:00 2024-02-15T18:08:41.448 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:08:41.451 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:08:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-02-15T18:08:41.451 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:08:41.451 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:08:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw x86_64 2:16.2.4-0.el8 ceph 13 M 2024-02-15T18:08:41.451 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrading: 2024-02-15T18:08:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: librados2 x86_64 2:16.2.4-0.el8 ceph 3.7 M 2024-02-15T18:08:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1 x86_64 2:16.2.4-0.el8 ceph 4.0 M 2024-02-15T18:08:41.451 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-02-15T18:08:41.451 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base x86_64 2:16.2.4-0.el8 ceph 6.0 M 2024-02-15T18:08:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common x86_64 2:16.2.4-0.el8 ceph 24 M 2024-02-15T18:08:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux x86_64 2:16.2.4-0.el8 ceph 25 k 2024-02-15T18:08:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: daxctl-libs x86_64 71.1-7.el8 baseos 42 k 2024-02-15T18:08:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2024-02-15T18:08:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-15T18:08:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-02-15T18:08:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2 x86_64 2:16.2.4-0.el8 ceph 801 k 2024-02-15T18:08:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-15T18:08:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj x86_64 1.6.1-1.el8 appstream 145 k 2024-02-15T18:08:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-02-15T18:08:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1 x86_64 2:16.2.4-0.el8 ceph 486 k 2024-02-15T18:08:41.452 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-02-15T18:08:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2 x86_64 2:16.2.4-0.el8 ceph 4.1 M 2024-02-15T18:08:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-15T18:08:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-02-15T18:08:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: ndctl-libs x86_64 71.1-7.el8 baseos 80 k 2024-02-15T18:08:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse x86_64 2:16.2.4-0.el8 ceph 44 k 2024-02-15T18:08:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common x86_64 2:16.2.4-0.el8 ceph 80 k 2024-02-15T18:08:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs x86_64 2:16.2.4-0.el8 ceph 213 k 2024-02-15T18:08:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-02-15T18:08:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados x86_64 2:16.2.4-0.el8 ceph 387 k 2024-02-15T18:08:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd x86_64 2:16.2.4-0.el8 ceph 368 k 2024-02-15T18:08:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw x86_64 2:16.2.4-0.el8 ceph 114 k 2024-02-15T18:08:41.453 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:08:41.454 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:08:41.454 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:08:41.454 INFO:teuthology.orchestra.run.smithi042.stdout:Install 25 Packages 2024-02-15T18:08:41.454 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrade 2 Packages 2024-02-15T18:08:41.454 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:08:41.454 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 59 M 2024-02-15T18:08:41.454 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:08:41.881 INFO:teuthology.orchestra.run.smithi042.stdout:(1/27): ceph-base-16.2.4-0.el8.x86_64.rpm 45 MB/s | 6.0 MB 00:00 2024-02-15T18:08:41.906 INFO:teuthology.orchestra.run.smithi042.stdout:(2/27): ceph-selinux-16.2.4-0.el8.x86_64.rpm 1.0 MB/s | 25 kB 00:00 2024-02-15T18:08:41.948 INFO:teuthology.orchestra.run.smithi042.stdout:(3/27): libcephfs2-16.2.4-0.el8.x86_64.rpm 19 MB/s | 801 kB 00:00 2024-02-15T18:08:41.990 INFO:teuthology.orchestra.run.smithi042.stdout:(4/27): libradosstriper1-16.2.4-0.el8.x86_64.rp 11 MB/s | 486 kB 00:00 2024-02-15T18:08:42.090 INFO:teuthology.orchestra.run.smithi042.stdout:(5/27): librgw2-16.2.4-0.el8.x86_64.rpm 41 MB/s | 4.1 MB 00:00 2024-02-15T18:08:42.107 INFO:teuthology.orchestra.run.smithi042.stdout:(6/27): python3-ceph-argparse-16.2.4-0.el8.x86_ 2.7 MB/s | 44 kB 00:00 2024-02-15T18:08:42.132 INFO:teuthology.orchestra.run.smithi042.stdout:(7/27): python3-ceph-common-16.2.4-0.el8.x86_64 3.1 MB/s | 80 kB 00:00 2024-02-15T18:08:42.158 INFO:teuthology.orchestra.run.smithi042.stdout:(8/27): python3-cephfs-16.2.4-0.el8.x86_64.rpm 8.3 MB/s | 213 kB 00:00 2024-02-15T18:08:42.192 INFO:teuthology.orchestra.run.smithi042.stdout:(9/27): python3-rados-16.2.4-0.el8.x86_64.rpm 11 MB/s | 387 kB 00:00 2024-02-15T18:08:42.225 INFO:teuthology.orchestra.run.smithi042.stdout:(10/27): python3-rbd-16.2.4-0.el8.x86_64.rpm 11 MB/s | 368 kB 00:00 2024-02-15T18:08:42.250 INFO:teuthology.orchestra.run.smithi042.stdout:(11/27): python3-rgw-16.2.4-0.el8.x86_64.rpm 4.5 MB/s | 114 kB 00:00 2024-02-15T18:08:42.410 INFO:teuthology.orchestra.run.smithi042.stdout:(12/27): ceph-radosgw-16.2.4-0.el8.x86_64.rpm 19 MB/s | 13 MB 00:00 2024-02-15T18:08:42.693 INFO:teuthology.orchestra.run.smithi042.stdout:(13/27): ceph-common-16.2.4-0.el8.x86_64.rpm 25 MB/s | 24 MB 00:00 2024-02-15T18:08:45.451 INFO:teuthology.orchestra.run.smithi042.stdout:(14/27): libpmemobj-1.6.1-1.el8.x86_64.rpm 45 kB/s | 145 kB 00:03 2024-02-15T18:08:45.543 INFO:teuthology.orchestra.run.smithi042.stdout:(15/27): python3-prettytable-0.7.2-14.el8.noarc 479 kB/s | 44 kB 00:00 2024-02-15T18:08:45.635 INFO:teuthology.orchestra.run.smithi042.stdout:(16/27): daxctl-libs-71.1-7.el8.x86_64.rpm 462 kB/s | 42 kB 00:00 2024-02-15T18:08:45.686 INFO:teuthology.orchestra.run.smithi042.stdout:(17/27): librabbitmq-0.9.0-5.el8.x86_64.rpm 939 kB/s | 47 kB 00:00 2024-02-15T18:08:45.736 INFO:teuthology.orchestra.run.smithi042.stdout:(18/27): ndctl-libs-71.1-7.el8.x86_64.rpm 1.6 MB/s | 80 kB 00:00 2024-02-15T18:08:45.778 INFO:teuthology.orchestra.run.smithi042.stdout:(19/27): librdkafka-1.6.1-1.el8.x86_64.rpm 201 kB/s | 677 kB 00:03 2024-02-15T18:08:46.253 INFO:teuthology.orchestra.run.smithi042.stdout:(20/27): fmt-6.2.1-1.el8.x86_64.rpm 225 kB/s | 116 kB 00:00 2024-02-15T18:08:46.403 INFO:teuthology.orchestra.run.smithi042.stdout:(21/27): gperftools-libs-2.7-9.el8.x86_64.rpm 490 kB/s | 306 kB 00:00 2024-02-15T18:08:46.462 INFO:teuthology.orchestra.run.smithi042.stdout:(22/27): leveldb-1.22-1.el8.x86_64.rpm 867 kB/s | 181 kB 00:00 2024-02-15T18:08:46.545 INFO:teuthology.orchestra.run.smithi042.stdout:(23/27): liboath-2.6.2-3.el8.x86_64.rpm 415 kB/s | 59 kB 00:00 2024-02-15T18:08:46.571 INFO:teuthology.orchestra.run.smithi042.stdout:(24/27): libunwind-1.3.1-3.el8.x86_64.rpm 691 kB/s | 75 kB 00:00 2024-02-15T18:08:46.646 INFO:teuthology.orchestra.run.smithi042.stdout:(25/27): librados2-16.2.4-0.el8.x86_64.rpm 37 MB/s | 3.7 MB 00:00 2024-02-15T18:08:46.722 INFO:teuthology.orchestra.run.smithi042.stdout:(26/27): librbd1-16.2.4-0.el8.x86_64.rpm 27 MB/s | 4.0 MB 00:00 2024-02-15T18:08:52.038 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 748 kB/s | 16 MB 00:22 2024-02-15T18:08:55.860 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 318 kB/s | 24 kB 00:00 2024-02-15T18:08:59.431 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:08:59.434 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:08:59.434 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-02-15T18:08:59.434 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:08:59.434 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:08:59.434 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw x86_64 2:16.2.4-0.el8 ceph 13 M 2024-02-15T18:08:59.434 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrading: 2024-02-15T18:08:59.434 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 x86_64 2:16.2.4-0.el8 ceph 3.7 M 2024-02-15T18:08:59.435 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1 x86_64 2:16.2.4-0.el8 ceph 4.0 M 2024-02-15T18:08:59.435 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-02-15T18:08:59.435 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base x86_64 2:16.2.4-0.el8 ceph 6.0 M 2024-02-15T18:08:59.435 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common x86_64 2:16.2.4-0.el8 ceph 24 M 2024-02-15T18:08:59.435 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux x86_64 2:16.2.4-0.el8 ceph 25 k 2024-02-15T18:08:59.435 INFO:teuthology.orchestra.run.smithi191.stdout: daxctl-libs x86_64 71.1-7.el8 baseos 42 k 2024-02-15T18:08:59.435 INFO:teuthology.orchestra.run.smithi191.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2024-02-15T18:08:59.435 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-15T18:08:59.435 INFO:teuthology.orchestra.run.smithi191.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2024-02-15T18:08:59.435 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2 x86_64 2:16.2.4-0.el8 ceph 801 k 2024-02-15T18:08:59.435 INFO:teuthology.orchestra.run.smithi191.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-15T18:08:59.435 INFO:teuthology.orchestra.run.smithi191.stdout: libpmemobj x86_64 1.6.1-1.el8 appstream 145 k 2024-02-15T18:08:59.436 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-02-15T18:08:59.436 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 x86_64 2:16.2.4-0.el8 ceph 486 k 2024-02-15T18:08:59.436 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-02-15T18:08:59.436 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2 x86_64 2:16.2.4-0.el8 ceph 4.1 M 2024-02-15T18:08:59.436 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-15T18:08:59.436 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-02-15T18:08:59.436 INFO:teuthology.orchestra.run.smithi191.stdout: ndctl-libs x86_64 71.1-7.el8 baseos 80 k 2024-02-15T18:08:59.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse x86_64 2:16.2.4-0.el8 ceph 44 k 2024-02-15T18:08:59.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common x86_64 2:16.2.4-0.el8 ceph 80 k 2024-02-15T18:08:59.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs x86_64 2:16.2.4-0.el8 ceph 213 k 2024-02-15T18:08:59.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-02-15T18:08:59.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados x86_64 2:16.2.4-0.el8 ceph 387 k 2024-02-15T18:08:59.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd x86_64 2:16.2.4-0.el8 ceph 368 k 2024-02-15T18:08:59.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw x86_64 2:16.2.4-0.el8 ceph 114 k 2024-02-15T18:08:59.437 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:08:59.437 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:08:59.437 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:08:59.437 INFO:teuthology.orchestra.run.smithi191.stdout:Install 25 Packages 2024-02-15T18:08:59.437 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrade 2 Packages 2024-02-15T18:08:59.437 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:08:59.437 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 59 M 2024-02-15T18:08:59.438 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:08:59.837 INFO:teuthology.orchestra.run.smithi191.stdout:(1/27): ceph-base-16.2.4-0.el8.x86_64.rpm 36 MB/s | 6.0 MB 00:00 2024-02-15T18:08:59.862 INFO:teuthology.orchestra.run.smithi191.stdout:(2/27): ceph-selinux-16.2.4-0.el8.x86_64.rpm 1.0 MB/s | 25 kB 00:00 2024-02-15T18:08:59.929 INFO:teuthology.orchestra.run.smithi191.stdout:(3/27): libcephfs2-16.2.4-0.el8.x86_64.rpm 12 MB/s | 801 kB 00:00 2024-02-15T18:08:59.971 INFO:teuthology.orchestra.run.smithi191.stdout:(4/27): libradosstriper1-16.2.4-0.el8.x86_64.rp 11 MB/s | 486 kB 00:00 2024-02-15T18:09:00.071 INFO:teuthology.orchestra.run.smithi191.stdout:(5/27): librgw2-16.2.4-0.el8.x86_64.rpm 41 MB/s | 4.1 MB 00:00 2024-02-15T18:09:00.088 INFO:teuthology.orchestra.run.smithi191.stdout:(6/27): python3-ceph-argparse-16.2.4-0.el8.x86_ 2.6 MB/s | 44 kB 00:00 2024-02-15T18:09:00.113 INFO:teuthology.orchestra.run.smithi191.stdout:(7/27): python3-ceph-common-16.2.4-0.el8.x86_64 3.1 MB/s | 80 kB 00:00 2024-02-15T18:09:00.147 INFO:teuthology.orchestra.run.smithi191.stdout:(8/27): python3-cephfs-16.2.4-0.el8.x86_64.rpm 6.3 MB/s | 213 kB 00:00 2024-02-15T18:09:00.181 INFO:teuthology.orchestra.run.smithi191.stdout:(9/27): python3-rados-16.2.4-0.el8.x86_64.rpm 11 MB/s | 387 kB 00:00 2024-02-15T18:09:00.356 INFO:teuthology.orchestra.run.smithi191.stdout:(10/27): ceph-radosgw-16.2.4-0.el8.x86_64.rpm 18 MB/s | 13 MB 00:00 2024-02-15T18:09:00.390 INFO:teuthology.orchestra.run.smithi191.stdout:(11/27): python3-rbd-16.2.4-0.el8.x86_64.rpm 1.7 MB/s | 368 kB 00:00 2024-02-15T18:09:00.415 INFO:teuthology.orchestra.run.smithi191.stdout:(12/27): python3-rgw-16.2.4-0.el8.x86_64.rpm 1.9 MB/s | 114 kB 00:00 2024-02-15T18:09:00.665 INFO:teuthology.orchestra.run.smithi191.stdout:(13/27): ceph-common-16.2.4-0.el8.x86_64.rpm 24 MB/s | 24 MB 00:00 2024-02-15T18:09:08.931 INFO:teuthology.orchestra.run.smithi191.stdout:(14/27): lttng-ust-2.8.1-11.el8.x86_64.rpm 31 kB/s | 259 kB 00:08 2024-02-15T18:09:08.990 INFO:teuthology.orchestra.run.smithi191.stdout:(15/27): python3-prettytable-0.7.2-14.el8.noarc 753 kB/s | 44 kB 00:00 2024-02-15T18:09:09.081 INFO:teuthology.orchestra.run.smithi191.stdout:(16/27): daxctl-libs-71.1-7.el8.x86_64.rpm 461 kB/s | 42 kB 00:00 2024-02-15T18:09:09.161 INFO:teuthology.orchestra.run.smithi191.stdout:(17/27): librabbitmq-0.9.0-5.el8.x86_64.rpm 598 kB/s | 47 kB 00:00 2024-02-15T18:09:09.215 INFO:teuthology.orchestra.run.smithi191.stdout:(18/27): ndctl-libs-71.1-7.el8.x86_64.rpm 1.4 MB/s | 80 kB 00:00 2024-02-15T18:09:09.357 INFO:teuthology.orchestra.run.smithi191.stdout:(19/27): fmt-6.2.1-1.el8.x86_64.rpm 822 kB/s | 116 kB 00:00 2024-02-15T18:09:09.516 INFO:teuthology.orchestra.run.smithi191.stdout:(20/27): gperftools-libs-2.7-9.el8.x86_64.rpm 1.9 MB/s | 306 kB 00:00 2024-02-15T18:09:09.633 INFO:teuthology.orchestra.run.smithi191.stdout:(21/27): leveldb-1.22-1.el8.x86_64.rpm 1.5 MB/s | 181 kB 00:00 2024-02-15T18:09:09.766 INFO:teuthology.orchestra.run.smithi191.stdout:(22/27): liboath-2.6.2-3.el8.x86_64.rpm 441 kB/s | 59 kB 00:00 2024-02-15T18:09:09.875 INFO:teuthology.orchestra.run.smithi191.stdout:(23/27): libunwind-1.3.1-3.el8.x86_64.rpm 694 kB/s | 75 kB 00:00 2024-02-15T18:09:10.000 INFO:teuthology.orchestra.run.smithi191.stdout:(24/27): librados2-16.2.4-0.el8.x86_64.rpm 30 MB/s | 3.7 MB 00:00 2024-02-15T18:09:10.076 INFO:teuthology.orchestra.run.smithi191.stdout:(25/27): librbd1-16.2.4-0.el8.x86_64.rpm 54 MB/s | 4.0 MB 00:00 2024-02-15T18:09:10.459 INFO:teuthology.orchestra.run.smithi191.stdout:(26/27): librdkafka-1.6.1-1.el8.x86_64.rpm 67 kB/s | 677 kB 00:10 2024-02-15T18:09:42.000 INFO:teuthology.orchestra.run.smithi042.stdout:(27/27): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.4 kB/s | 259 kB 00:59 2024-02-15T18:09:42.003 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:09:42.004 INFO:teuthology.orchestra.run.smithi042.stdout:Total 997 kB/s | 59 MB 01:00 2024-02-15T18:09:42.074 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:09:42.127 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:09:42.127 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:09:42.507 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:09:42.510 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:09:43.655 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:09:43.857 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/29 2024-02-15T18:09:44.079 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/29 2024-02-15T18:09:44.220 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/29 2024-02-15T18:09:44.645 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/29 2024-02-15T18:09:44.853 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librados2-2:16.2.4-0.el8.x86_64 4/29 2024-02-15T18:09:44.954 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:16.2.4-0.el8.x86_64 4/29 2024-02-15T18:09:45.307 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rados-2:16.2.4-0.el8.x86_64 5/29 2024-02-15T18:09:45.503 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2024-02-15T18:09:45.555 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2024-02-15T18:09:45.776 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 7/29 2024-02-15T18:09:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 8/29 2024-02-15T18:09:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librgw2-2:16.2.4-0.el8.x86_64 9/29 2024-02-15T18:09:46.537 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librgw2-2:16.2.4-0.el8.x86_64 9/29 2024-02-15T18:09:46.725 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/29 2024-02-15T18:09:47.053 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : daxctl-libs-71.1-7.el8.x86_64 11/29 2024-02-15T18:09:47.119 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: daxctl-libs-71.1-7.el8.x86_64 11/29 2024-02-15T18:09:47.311 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 12/29 2024-02-15T18:09:47.497 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cephfs-2:16.2.4-0.el8.x86_64 13/29 2024-02-15T18:09:47.645 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ndctl-libs-71.1-7.el8.x86_64 14/29 2024-02-15T18:09:47.724 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ndctl-libs-71.1-7.el8.x86_64 14/29 2024-02-15T18:09:47.870 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libpmemobj-1.6.1-1.el8.x86_64 15/29 2024-02-15T18:09:48.328 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libpmemobj-1.6.1-1.el8.x86_64 15/29 2024-02-15T18:09:48.561 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librbd1-2:16.2.4-0.el8.x86_64 16/29 2024-02-15T18:09:48.649 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:16.2.4-0.el8.x86_64 16/29 2024-02-15T18:09:48.832 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rbd-2:16.2.4-0.el8.x86_64 17/29 2024-02-15T18:09:49.041 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rgw-2:16.2.4-0.el8.x86_64 18/29 2024-02-15T18:09:49.178 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2024-02-15T18:09:49.249 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2024-02-15T18:09:49.463 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 20/29 2024-02-15T18:09:49.695 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 21/29 2024-02-15T18:09:50.005 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 22/29 2024-02-15T18:09:50.253 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-common-2:16.2.4-0.el8.x86_64 23/29 2024-02-15T18:09:52.897 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2024-02-15T18:09:53.145 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-common-2:16.2.4-0.el8.x86_64 24/29 2024-02-15T18:09:53.887 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2024-02-15T18:09:54.262 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-base-2:16.2.4-0.el8.x86_64 25/29 2024-02-15T18:09:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-base-2:16.2.4-0.el8.x86_64 25/29 2024-02-15T18:09:54.485 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2024-02-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2024-02-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /sys 2024-02-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /proc 2024-02-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /dev 2024-02-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /run 2024-02-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /mnt 2024-02-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /var/tmp 2024-02-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /home 2024-02-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /tmp 2024-02-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /dev 2024-02-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:10:16.325 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2024-02-15T18:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2024-02-15T18:10:16.403 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 28/29 2024-02-15T18:10:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 28/29 2024-02-15T18:10:16.610 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 29/29 2024-02-15T18:10:17.297 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 29/29 2024-02-15T18:10:17.298 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-base-2:16.2.4-0.el8.x86_64 1/29 2024-02-15T18:10:17.298 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-common-2:16.2.4-0.el8.x86_64 2/29 2024-02-15T18:10:17.298 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-radosgw-2:16.2.4-0.el8.x86_64 3/29 2024-02-15T18:10:17.298 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-selinux-2:16.2.4-0.el8.x86_64 4/29 2024-02-15T18:10:17.298 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs2-2:16.2.4-0.el8.x86_64 5/29 2024-02-15T18:10:17.298 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libradosstriper1-2:16.2.4-0.el8.x86_64 6/29 2024-02-15T18:10:17.298 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librgw2-2:16.2.4-0.el8.x86_64 7/29 2024-02-15T18:10:17.298 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 8/29 2024-02-15T18:10:17.298 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-common-2:16.2.4-0.el8.x86_64 9/29 2024-02-15T18:10:17.298 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cephfs-2:16.2.4-0.el8.x86_64 10/29 2024-02-15T18:10:17.298 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rados-2:16.2.4-0.el8.x86_64 11/29 2024-02-15T18:10:17.298 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rbd-2:16.2.4-0.el8.x86_64 12/29 2024-02-15T18:10:17.299 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rgw-2:16.2.4-0.el8.x86_64 13/29 2024-02-15T18:10:17.299 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libpmemobj-1.6.1-1.el8.x86_64 14/29 2024-02-15T18:10:17.299 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 15/29 2024-02-15T18:10:17.299 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 16/29 2024-02-15T18:10:17.299 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 17/29 2024-02-15T18:10:17.299 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : daxctl-libs-71.1-7.el8.x86_64 18/29 2024-02-15T18:10:17.299 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 19/29 2024-02-15T18:10:17.299 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ndctl-libs-71.1-7.el8.x86_64 20/29 2024-02-15T18:10:17.299 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 21/29 2024-02-15T18:10:17.299 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 22/29 2024-02-15T18:10:17.299 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : leveldb-1.22-1.el8.x86_64 23/29 2024-02-15T18:10:17.299 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 24/29 2024-02-15T18:10:17.299 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 25/29 2024-02-15T18:10:17.299 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:16.2.4-0.el8.x86_64 26/29 2024-02-15T18:10:17.300 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 27/29 2024-02-15T18:10:17.300 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:16.2.4-0.el8.x86_64 28/29 2024-02-15T18:10:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 29/29 2024-02-15T18:10:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:10:18.111 INFO:teuthology.orchestra.run.smithi042.stdout:Upgraded: 2024-02-15T18:10:18.111 INFO:teuthology.orchestra.run.smithi042.stdout: librados2-2:16.2.4-0.el8.x86_64 librbd1-2:16.2.4-0.el8.x86_64 2024-02-15T18:10:18.112 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:10:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base-2:16.2.4-0.el8.x86_64 2024-02-15T18:10:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common-2:16.2.4-0.el8.x86_64 2024-02-15T18:10:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw-2:16.2.4-0.el8.x86_64 2024-02-15T18:10:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux-2:16.2.4-0.el8.x86_64 2024-02-15T18:10:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: daxctl-libs-71.1-7.el8.x86_64 2024-02-15T18:10:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: fmt-6.2.1-1.el8.x86_64 2024-02-15T18:10:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-15T18:10:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: leveldb-1.22-1.el8.x86_64 2024-02-15T18:10:18.112 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2-2:16.2.4-0.el8.x86_64 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj-1.6.1-1.el8.x86_64 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1-2:16.2.4-0.el8.x86_64 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2-2:16.2.4-0.el8.x86_64 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: ndctl-libs-71.1-7.el8.x86_64 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse-2:16.2.4-0.el8.x86_64 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common-2:16.2.4-0.el8.x86_64 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs-2:16.2.4-0.el8.x86_64 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados-2:16.2.4-0.el8.x86_64 2024-02-15T18:10:18.113 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd-2:16.2.4-0.el8.x86_64 2024-02-15T18:10:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw-2:16.2.4-0.el8.x86_64 2024-02-15T18:10:18.114 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:10:18.114 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:10:18.397 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-test 2024-02-15T18:10:19.188 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:42 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:10:20.842 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:10:20.843 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:10:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-02-15T18:10:20.843 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:10:20.843 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:10:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test x86_64 2:16.2.4-0.el8 ceph 52 M 2024-02-15T18:10:20.843 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-02-15T18:10:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-02-15T18:10:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite x86_64 2:16.2.4-0.el8 ceph 169 k 2024-02-15T18:10:20.843 INFO:teuthology.orchestra.run.smithi042.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-02-15T18:10:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-02-15T18:10:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-02-15T18:10:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:10:20.844 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:10:20.844 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:10:20.844 INFO:teuthology.orchestra.run.smithi042.stdout:Install 6 Packages 2024-02-15T18:10:20.844 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:10:20.845 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 53 M 2024-02-15T18:10:20.845 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 229 M 2024-02-15T18:10:20.845 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:10:22.102 INFO:teuthology.orchestra.run.smithi042.stdout:(1/6): libcephsqlite-16.2.4-0.el8.x86_64.rpm 3.1 MB/s | 169 kB 00:00 2024-02-15T18:10:22.786 INFO:teuthology.orchestra.run.smithi042.stdout:(2/6): ceph-test-16.2.4-0.el8.x86_64.rpm 70 MB/s | 52 MB 00:00 2024-02-15T18:10:32.202 INFO:teuthology.orchestra.run.smithi042.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 34 kB/s | 323 kB 00:09 2024-02-15T18:10:32.277 INFO:teuthology.orchestra.run.smithi042.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 944 kB/s | 71 kB 00:00 2024-02-15T18:10:40.818 INFO:teuthology.orchestra.run.smithi042.stdout:(5/6): oniguruma-6.8.2-3.el8.x86_64.rpm 10 kB/s | 188 kB 00:18 2024-02-15T18:11:09.473 INFO:teuthology.orchestra.run.smithi191.stdout:[MIRROR] libpmemobj-1.6.1-1.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://packages.oit.ncsu.edu/centos/8-stream/AppStream/x86_64/os/Packages/libpmemobj-1.6.1-1.el8.x86_64.rpm [Failed to connect to packages.oit.ncsu.edu port 80: Connection timed out] 2024-02-15T18:11:09.574 INFO:teuthology.orchestra.run.smithi191.stdout:(27/27): libpmemobj-1.6.1-1.el8.x86_64.rpm 1.1 kB/s | 145 kB 02:09 2024-02-15T18:11:09.574 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:11:09.574 INFO:teuthology.orchestra.run.smithi191.stdout:Total 464 kB/s | 59 MB 02:10 2024-02-15T18:11:09.639 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:11:09.687 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:11:09.688 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:11:10.032 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:11:10.035 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:11:11.191 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:11:11.415 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/29 2024-02-15T18:11:11.621 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/29 2024-02-15T18:11:11.826 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/29 2024-02-15T18:11:12.217 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/29 2024-02-15T18:11:12.426 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librados2-2:16.2.4-0.el8.x86_64 4/29 2024-02-15T18:11:12.524 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:16.2.4-0.el8.x86_64 4/29 2024-02-15T18:11:12.790 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rados-2:16.2.4-0.el8.x86_64 5/29 2024-02-15T18:11:12.976 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2024-02-15T18:11:13.045 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2024-02-15T18:11:13.268 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 7/29 2024-02-15T18:11:13.810 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 8/29 2024-02-15T18:11:13.942 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librgw2-2:16.2.4-0.el8.x86_64 9/29 2024-02-15T18:11:14.025 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librgw2-2:16.2.4-0.el8.x86_64 9/29 2024-02-15T18:11:14.231 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/29 2024-02-15T18:11:14.392 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : daxctl-libs-71.1-7.el8.x86_64 11/29 2024-02-15T18:11:14.441 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: daxctl-libs-71.1-7.el8.x86_64 11/29 2024-02-15T18:11:14.656 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 12/29 2024-02-15T18:11:14.818 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cephfs-2:16.2.4-0.el8.x86_64 13/29 2024-02-15T18:11:15.011 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ndctl-libs-71.1-7.el8.x86_64 14/29 2024-02-15T18:11:15.085 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ndctl-libs-71.1-7.el8.x86_64 14/29 2024-02-15T18:11:15.425 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libpmemobj-1.6.1-1.el8.x86_64 15/29 2024-02-15T18:11:15.849 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libpmemobj-1.6.1-1.el8.x86_64 15/29 2024-02-15T18:11:16.033 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librbd1-2:16.2.4-0.el8.x86_64 16/29 2024-02-15T18:11:16.129 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:16.2.4-0.el8.x86_64 16/29 2024-02-15T18:11:16.312 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rbd-2:16.2.4-0.el8.x86_64 17/29 2024-02-15T18:11:16.502 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rgw-2:16.2.4-0.el8.x86_64 18/29 2024-02-15T18:11:16.667 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2024-02-15T18:11:16.735 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2024-02-15T18:11:16.917 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 20/29 2024-02-15T18:11:17.175 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 21/29 2024-02-15T18:11:17.340 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 22/29 2024-02-15T18:11:17.567 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-common-2:16.2.4-0.el8.x86_64 23/29 2024-02-15T18:11:20.176 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2024-02-15T18:11:20.417 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-common-2:16.2.4-0.el8.x86_64 24/29 2024-02-15T18:11:21.112 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2024-02-15T18:11:21.410 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-base-2:16.2.4-0.el8.x86_64 25/29 2024-02-15T18:11:21.521 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-base-2:16.2.4-0.el8.x86_64 25/29 2024-02-15T18:11:21.649 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2024-02-15T18:11:41.050 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2024-02-15T18:11:41.050 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /sys 2024-02-15T18:11:41.050 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /proc 2024-02-15T18:11:41.050 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /dev 2024-02-15T18:11:41.050 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /run 2024-02-15T18:11:41.050 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /mnt 2024-02-15T18:11:41.050 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /var/tmp 2024-02-15T18:11:41.050 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /home 2024-02-15T18:11:41.051 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /tmp 2024-02-15T18:11:41.051 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /dev 2024-02-15T18:11:41.051 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:11:42.389 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2024-02-15T18:11:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2024-02-15T18:11:42.466 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 28/29 2024-02-15T18:11:42.664 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 28/29 2024-02-15T18:11:42.665 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 29/29 2024-02-15T18:11:43.364 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 29/29 2024-02-15T18:11:43.364 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-base-2:16.2.4-0.el8.x86_64 1/29 2024-02-15T18:11:43.364 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-common-2:16.2.4-0.el8.x86_64 2/29 2024-02-15T18:11:43.364 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-radosgw-2:16.2.4-0.el8.x86_64 3/29 2024-02-15T18:11:43.364 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-selinux-2:16.2.4-0.el8.x86_64 4/29 2024-02-15T18:11:43.364 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs2-2:16.2.4-0.el8.x86_64 5/29 2024-02-15T18:11:43.365 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libradosstriper1-2:16.2.4-0.el8.x86_64 6/29 2024-02-15T18:11:43.365 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librgw2-2:16.2.4-0.el8.x86_64 7/29 2024-02-15T18:11:43.365 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 8/29 2024-02-15T18:11:43.365 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-common-2:16.2.4-0.el8.x86_64 9/29 2024-02-15T18:11:43.365 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cephfs-2:16.2.4-0.el8.x86_64 10/29 2024-02-15T18:11:43.365 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rados-2:16.2.4-0.el8.x86_64 11/29 2024-02-15T18:11:43.365 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rbd-2:16.2.4-0.el8.x86_64 12/29 2024-02-15T18:11:43.365 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rgw-2:16.2.4-0.el8.x86_64 13/29 2024-02-15T18:11:43.365 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libpmemobj-1.6.1-1.el8.x86_64 14/29 2024-02-15T18:11:43.365 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 15/29 2024-02-15T18:11:43.365 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 16/29 2024-02-15T18:11:43.365 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 17/29 2024-02-15T18:11:43.365 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : daxctl-libs-71.1-7.el8.x86_64 18/29 2024-02-15T18:11:43.365 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 19/29 2024-02-15T18:11:43.366 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ndctl-libs-71.1-7.el8.x86_64 20/29 2024-02-15T18:11:43.366 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 21/29 2024-02-15T18:11:43.366 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 22/29 2024-02-15T18:11:43.366 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : leveldb-1.22-1.el8.x86_64 23/29 2024-02-15T18:11:43.366 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 24/29 2024-02-15T18:11:43.366 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 25/29 2024-02-15T18:11:43.366 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:16.2.4-0.el8.x86_64 26/29 2024-02-15T18:11:43.366 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 27/29 2024-02-15T18:11:43.366 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:16.2.4-0.el8.x86_64 28/29 2024-02-15T18:11:44.276 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 29/29 2024-02-15T18:11:44.277 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:11:44.277 INFO:teuthology.orchestra.run.smithi191.stdout:Upgraded: 2024-02-15T18:11:44.277 INFO:teuthology.orchestra.run.smithi191.stdout: librados2-2:16.2.4-0.el8.x86_64 librbd1-2:16.2.4-0.el8.x86_64 2024-02-15T18:11:44.277 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:11:44.277 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base-2:16.2.4-0.el8.x86_64 2024-02-15T18:11:44.277 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common-2:16.2.4-0.el8.x86_64 2024-02-15T18:11:44.277 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw-2:16.2.4-0.el8.x86_64 2024-02-15T18:11:44.277 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux-2:16.2.4-0.el8.x86_64 2024-02-15T18:11:44.277 INFO:teuthology.orchestra.run.smithi191.stdout: daxctl-libs-71.1-7.el8.x86_64 2024-02-15T18:11:44.277 INFO:teuthology.orchestra.run.smithi191.stdout: fmt-6.2.1-1.el8.x86_64 2024-02-15T18:11:44.277 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-15T18:11:44.277 INFO:teuthology.orchestra.run.smithi191.stdout: leveldb-1.22-1.el8.x86_64 2024-02-15T18:11:44.277 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2-2:16.2.4-0.el8.x86_64 2024-02-15T18:11:44.278 INFO:teuthology.orchestra.run.smithi191.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-15T18:11:44.278 INFO:teuthology.orchestra.run.smithi191.stdout: libpmemobj-1.6.1-1.el8.x86_64 2024-02-15T18:11:44.278 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-02-15T18:11:44.278 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1-2:16.2.4-0.el8.x86_64 2024-02-15T18:11:44.278 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-02-15T18:11:44.278 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2-2:16.2.4-0.el8.x86_64 2024-02-15T18:11:44.278 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-15T18:11:44.278 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-15T18:11:44.278 INFO:teuthology.orchestra.run.smithi191.stdout: ndctl-libs-71.1-7.el8.x86_64 2024-02-15T18:11:44.278 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse-2:16.2.4-0.el8.x86_64 2024-02-15T18:11:44.278 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common-2:16.2.4-0.el8.x86_64 2024-02-15T18:11:44.278 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs-2:16.2.4-0.el8.x86_64 2024-02-15T18:11:44.278 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-15T18:11:44.278 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados-2:16.2.4-0.el8.x86_64 2024-02-15T18:11:44.279 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd-2:16.2.4-0.el8.x86_64 2024-02-15T18:11:44.279 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw-2:16.2.4-0.el8.x86_64 2024-02-15T18:11:44.279 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:11:44.279 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:11:44.549 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-test 2024-02-15T18:11:45.307 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:02:50 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:11:46.851 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:11:46.852 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:11:46.852 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-02-15T18:11:46.852 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:11:46.852 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:11:46.852 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test x86_64 2:16.2.4-0.el8 ceph 52 M 2024-02-15T18:11:46.852 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-02-15T18:11:46.853 INFO:teuthology.orchestra.run.smithi191.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-02-15T18:11:46.853 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite x86_64 2:16.2.4-0.el8 ceph 169 k 2024-02-15T18:11:46.853 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-02-15T18:11:46.853 INFO:teuthology.orchestra.run.smithi191.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-02-15T18:11:46.853 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-02-15T18:11:46.853 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:11:46.853 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:11:46.853 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:11:46.853 INFO:teuthology.orchestra.run.smithi191.stdout:Install 6 Packages 2024-02-15T18:11:46.853 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:11:46.854 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 53 M 2024-02-15T18:11:46.854 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 229 M 2024-02-15T18:11:46.854 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:11:47.128 INFO:teuthology.orchestra.run.smithi191.stdout:(1/6): libcephsqlite-16.2.4-0.el8.x86_64.rpm 1.9 MB/s | 169 kB 00:00 2024-02-15T18:11:47.898 INFO:teuthology.orchestra.run.smithi191.stdout:(2/6): ceph-test-16.2.4-0.el8.x86_64.rpm 60 MB/s | 52 MB 00:00 2024-02-15T18:12:08.420 INFO:teuthology.orchestra.run.smithi191.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 16 kB/s | 323 kB 00:20 2024-02-15T18:12:08.478 INFO:teuthology.orchestra.run.smithi191.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-02-15T18:12:23.207 INFO:teuthology.orchestra.run.smithi042.stdout:[MIRROR] jq-1.6-8.el8.x86_64.rpm: Curl error (56): Failure when receiving data from the peer for http://packages.oit.ncsu.edu/centos/8-stream/AppStream/x86_64/os/Packages/jq-1.6-8.el8.x86_64.rpm [Recv failure: Connection reset by peer] 2024-02-15T18:12:23.298 INFO:teuthology.orchestra.run.smithi042.stdout:(6/6): jq-1.6-8.el8.x86_64.rpm 1.7 kB/s | 203 kB 02:01 2024-02-15T18:12:23.299 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:12:23.299 INFO:teuthology.orchestra.run.smithi042.stdout:Total 442 kB/s | 53 MB 02:02 2024-02-15T18:12:23.339 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:12:23.363 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:12:23.363 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:12:23.688 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:12:23.692 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:12:24.221 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:12:24.464 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-02-15T18:12:24.677 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-02-15T18:12:24.833 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-15T18:12:24.912 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-15T18:12:25.100 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-02-15T18:12:25.250 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephsqlite-2:16.2.4-0.el8.x86_64 5/6 2024-02-15T18:12:30.435 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephsqlite-2:16.2.4-0.el8.x86_64 5/6 2024-02-15T18:12:30.815 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-test-2:16.2.4-0.el8.x86_64 6/6 2024-02-15T18:12:31.121 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-test-2:16.2.4-0.el8.x86_64 6/6 2024-02-15T18:12:31.121 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-test-2:16.2.4-0.el8.x86_64 1/6 2024-02-15T18:12:31.121 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephsqlite-2:16.2.4-0.el8.x86_64 2/6 2024-02-15T18:12:31.122 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-02-15T18:12:31.122 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-02-15T18:12:31.122 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-02-15T18:12:31.657 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-02-15T18:12:31.657 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:12:31.657 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:12:31.657 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test-2:16.2.4-0.el8.x86_64 jq-1.6-8.el8.x86_64 2024-02-15T18:12:31.657 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite-2:16.2.4-0.el8.x86_64 oniguruma-6.8.2-3.el8.x86_64 2024-02-15T18:12:31.658 INFO:teuthology.orchestra.run.smithi042.stdout: socat-1.7.4.1-1.el8.x86_64 xmlstarlet-1.6.1-20.el8.x86_64 2024-02-15T18:12:31.658 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:12:31.658 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:12:31.915 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph 2024-02-15T18:12:32.707 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:03:55 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:12:34.374 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:12:34.377 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:12:34.378 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-02-15T18:12:34.378 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:12:34.378 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:12:34.378 INFO:teuthology.orchestra.run.smithi042.stdout: ceph x86_64 2:16.2.4-0.el8 ceph 6.3 k 2024-02-15T18:12:34.378 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-02-15T18:12:34.378 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds x86_64 2:16.2.4-0.el8 ceph 2.4 M 2024-02-15T18:12:34.378 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr x86_64 2:16.2.4-0.el8 ceph 1.6 M 2024-02-15T18:12:34.378 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core noarch 2:16.2.4-0.el8 ceph-noarch 220 k 2024-02-15T18:12:34.378 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon x86_64 2:16.2.4-0.el8 ceph 5.0 M 2024-02-15T18:12:34.378 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd x86_64 2:16.2.4-0.el8 ceph 18 M 2024-02-15T18:12:34.378 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-15T18:12:34.378 INFO:teuthology.orchestra.run.smithi042.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-15T18:12:34.379 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-02-15T18:12:34.379 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-15T18:12:34.379 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-15T18:12:34.379 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-02-15T18:12:34.379 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-15T18:12:34.379 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-15T18:12:34.379 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-15T18:12:34.379 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-02-15T18:12:34.379 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-02-15T18:12:34.379 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-15T18:12:34.379 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-15T18:12:34.379 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-15T18:12:34.379 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-02-15T18:12:34.379 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-02-15T18:12:34.380 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-02-15T18:12:34.380 INFO:teuthology.orchestra.run.smithi042.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-15T18:12:34.380 INFO:teuthology.orchestra.run.smithi042.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-15T18:12:34.380 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-15T18:12:34.380 INFO:teuthology.orchestra.run.smithi042.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-15T18:12:34.380 INFO:teuthology.orchestra.run.smithi042.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-15T18:12:34.380 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-15T18:12:34.380 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-15T18:12:34.380 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-02-15T18:12:34.380 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-15T18:12:34.380 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:12:34.380 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:12:34.380 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:12:34.381 INFO:teuthology.orchestra.run.smithi042.stdout:Install 32 Packages 2024-02-15T18:12:34.381 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:12:34.381 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 31 M 2024-02-15T18:12:34.381 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 111 M 2024-02-15T18:12:34.381 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:12:35.143 INFO:teuthology.orchestra.run.smithi042.stdout:(1/32): ceph-16.2.4-0.el8.x86_64.rpm 120 kB/s | 6.3 kB 00:00 2024-02-15T18:12:35.202 INFO:teuthology.orchestra.run.smithi042.stdout:(2/32): ceph-mgr-16.2.4-0.el8.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-02-15T18:12:35.260 INFO:teuthology.orchestra.run.smithi042.stdout:(3/32): ceph-mds-16.2.4-0.el8.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-02-15T18:12:35.294 INFO:teuthology.orchestra.run.smithi042.stdout:(4/32): ceph-mgr-modules-core-16.2.4-0.el8.noar 6.5 MB/s | 220 kB 00:00 2024-02-15T18:12:35.403 INFO:teuthology.orchestra.run.smithi042.stdout:(5/32): ceph-mon-16.2.4-0.el8.x86_64.rpm 19 MB/s | 5.0 MB 00:00 2024-02-15T18:12:35.662 INFO:teuthology.orchestra.run.smithi042.stdout:(6/32): ceph-osd-16.2.4-0.el8.x86_64.rpm 40 MB/s | 18 MB 00:00 2024-02-15T18:12:38.887 INFO:teuthology.orchestra.run.smithi042.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 11 kB/s | 39 kB 00:03 2024-02-15T18:12:38.962 INFO:teuthology.orchestra.run.smithi042.stdout:(8/32): python3-pytz-2017.2-11.el8.noarch.rpm 762 kB/s | 57 kB 00:00 2024-02-15T18:12:39.212 INFO:teuthology.orchestra.run.smithi042.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 1.8 MB/s | 457 kB 00:00 2024-02-15T18:12:39.362 INFO:teuthology.orchestra.run.smithi042.stdout:(10/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.6 MB/s | 249 kB 00:00 2024-02-15T18:12:39.437 INFO:teuthology.orchestra.run.smithi042.stdout:(11/32): python3-cryptography-3.2.1-7.el8.x86_6 8.3 MB/s | 641 kB 00:00 2024-02-15T18:12:39.479 INFO:teuthology.orchestra.run.smithi042.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 2.6 MB/s | 109 kB 00:00 2024-02-15T18:12:39.688 INFO:teuthology.orchestra.run.smithi042.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 209 kB/s | 44 kB 00:00 2024-02-15T18:12:39.846 INFO:teuthology.orchestra.run.smithi042.stdout:(14/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.1 MB/s | 185 kB 00:00 2024-02-15T18:12:39.953 INFO:teuthology.orchestra.run.smithi042.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.6 MB/s | 173 kB 00:00 2024-02-15T18:12:40.080 INFO:teuthology.orchestra.run.smithi042.stdout:(16/32): python3-cherrypy-18.4.0-1.el8.noarch.r 3.0 MB/s | 384 kB 00:00 2024-02-15T18:12:40.147 INFO:teuthology.orchestra.run.smithi042.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 167 kB/s | 11 kB 00:00 2024-02-15T18:12:40.231 INFO:teuthology.orchestra.run.smithi042.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 211 kB/s | 18 kB 00:00 2024-02-15T18:12:40.315 INFO:teuthology.orchestra.run.smithi042.stdout:(19/32): python3-logutils-0.3.5-11.el8.noarch.r 582 kB/s | 49 kB 00:00 2024-02-15T18:12:40.390 INFO:teuthology.orchestra.run.smithi042.stdout:(20/32): python3-more-itertools-7.2.0-3.el8.noa 785 kB/s | 59 kB 00:00 2024-02-15T18:12:40.482 INFO:teuthology.orchestra.run.smithi042.stdout:(21/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.0 MB/s | 283 kB 00:00 2024-02-15T18:12:40.557 INFO:teuthology.orchestra.run.smithi042.stdout:(22/32): python3-portend-2.6-1.el8.noarch.rpm 208 kB/s | 16 kB 00:00 2024-02-15T18:12:40.641 INFO:teuthology.orchestra.run.smithi042.stdout:(23/32): python3-simplegeneric-0.8.1-17.el8.noa 224 kB/s | 19 kB 00:00 2024-02-15T18:12:40.749 INFO:teuthology.orchestra.run.smithi042.stdout:(24/32): python3-singledispatch-3.4.0.3-18.el8. 221 kB/s | 24 kB 00:00 2024-02-15T18:12:40.833 INFO:teuthology.orchestra.run.smithi042.stdout:(25/32): python3-tempora-1.14.1-5.el8.noarch.rp 351 kB/s | 29 kB 00:00 2024-02-15T18:12:40.908 INFO:teuthology.orchestra.run.smithi042.stdout:(26/32): python3-trustme-0.6.0-4.el8.noarch.rpm 367 kB/s | 27 kB 00:00 2024-02-15T18:12:41.000 INFO:teuthology.orchestra.run.smithi042.stdout:(27/32): python3-waitress-1.4.3-1.el8.noarch.rp 2.7 MB/s | 255 kB 00:00 2024-02-15T18:12:41.092 INFO:teuthology.orchestra.run.smithi042.stdout:(28/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.7 MB/s | 251 kB 00:00 2024-02-15T18:12:41.176 INFO:teuthology.orchestra.run.smithi042.stdout:(29/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.0 MB/s | 85 kB 00:00 2024-02-15T18:12:41.251 INFO:teuthology.orchestra.run.smithi042.stdout:(30/32): python3-zc-lockfile-2.0-2.el8.noarch.r 306 kB/s | 23 kB 00:00 2024-02-15T18:12:42.892 INFO:teuthology.orchestra.run.smithi042.stdout:(31/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 14 kB/s | 103 kB 00:07 2024-02-15T18:13:11.050 INFO:teuthology.orchestra.run.smithi042.stdout:(32/32): python3-mako-1.0.6-14.el8.noarch.rpm 4.4 kB/s | 157 kB 00:35 2024-02-15T18:13:11.051 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:13:11.052 INFO:teuthology.orchestra.run.smithi042.stdout:Total 866 kB/s | 31 MB 00:36 2024-02-15T18:13:11.132 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:13:11.174 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:13:11.174 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:13:11.503 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:13:11.506 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:13:12.625 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:13:12.845 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-02-15T18:13:13.070 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-02-15T18:13:13.324 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-02-15T18:13:13.542 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-02-15T18:13:13.698 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-02-15T18:13:13.914 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-02-15T18:13:14.105 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-02-15T18:13:14.308 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-02-15T18:13:14.503 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-02-15T18:13:14.757 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-02-15T18:13:14.970 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-02-15T18:13:15.203 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-02-15T18:13:15.609 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-02-15T18:13:15.847 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-02-15T18:13:16.027 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-02-15T18:13:16.316 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-02-15T18:13:16.496 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-15T18:13:16.795 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-02-15T18:13:17.051 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-02-15T18:13:17.239 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-02-15T18:13:17.408 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-02-15T18:13:17.662 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-02-15T18:13:17.876 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-02-15T18:13:18.138 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-02-15T18:13:18.470 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-02-15T18:13:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-02-15T18:13:19.166 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 27/32 2024-02-15T18:13:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-2:16.2.4-0.el8.x86_64 28/32 2024-02-15T18:13:21.013 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-2:16.2.4-0.el8.x86_64 28/32 2024-02-15T18:13:21.227 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-osd-2:16.2.4-0.el8.x86_64 29/32 2024-02-15T18:13:21.687 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-osd-2:16.2.4-0.el8.x86_64 29/32 2024-02-15T18:13:21.918 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mon-2:16.2.4-0.el8.x86_64 30/32 2024-02-15T18:13:22.154 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mon-2:16.2.4-0.el8.x86_64 30/32 2024-02-15T18:13:22.368 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mds-2:16.2.4-0.el8.x86_64 31/32 2024-02-15T18:13:22.386 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mds-2:16.2.4-0.el8.x86_64 31/32 2024-02-15T18:13:22.551 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-2:16.2.4-0.el8.x86_64 32/32 2024-02-15T18:13:22.977 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-2:16.2.4-0.el8.x86_64 32/32 2024-02-15T18:13:22.977 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-2:16.2.4-0.el8.x86_64 1/32 2024-02-15T18:13:22.977 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mds-2:16.2.4-0.el8.x86_64 2/32 2024-02-15T18:13:22.977 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-2:16.2.4-0.el8.x86_64 3/32 2024-02-15T18:13:22.978 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mon-2:16.2.4-0.el8.x86_64 4/32 2024-02-15T18:13:22.978 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-osd-2:16.2.4-0.el8.x86_64 5/32 2024-02-15T18:13:22.978 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 6/32 2024-02-15T18:13:22.978 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-02-15T18:13:22.978 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-02-15T18:13:22.978 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-02-15T18:13:22.978 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-02-15T18:13:22.978 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-02-15T18:13:22.978 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-02-15T18:13:22.978 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-02-15T18:13:22.978 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-02-15T18:13:22.978 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-02-15T18:13:22.978 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-02-15T18:13:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-15T18:13:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-02-15T18:13:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-02-15T18:13:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-02-15T18:13:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-02-15T18:13:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-02-15T18:13:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-02-15T18:13:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-02-15T18:13:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-02-15T18:13:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-02-15T18:13:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-02-15T18:13:22.979 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-02-15T18:13:22.980 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-02-15T18:13:22.980 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-02-15T18:13:22.980 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-02-15T18:13:23.840 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-02-15T18:13:23.840 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:23.840 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:13:23.840 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-2:16.2.4-0.el8.x86_64 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds-2:16.2.4-0.el8.x86_64 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-2:16.2.4-0.el8.x86_64 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon-2:16.2.4-0.el8.x86_64 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd-2:16.2.4-0.el8.x86_64 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-15T18:13:23.841 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-15T18:13:23.842 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-15T18:13:23.842 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-15T18:13:23.842 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-15T18:13:23.842 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend-2.6-1.el8.noarch 2024-02-15T18:13:23.842 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-15T18:13:23.842 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-15T18:13:23.842 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-15T18:13:23.842 INFO:teuthology.orchestra.run.smithi042.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-15T18:13:23.842 INFO:teuthology.orchestra.run.smithi042.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-15T18:13:23.842 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-15T18:13:23.842 INFO:teuthology.orchestra.run.smithi042.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-15T18:13:23.842 INFO:teuthology.orchestra.run.smithi042.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-15T18:13:23.842 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-15T18:13:23.842 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-15T18:13:23.843 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-15T18:13:23.843 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-15T18:13:23.843 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:23.843 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:13:24.096 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-base 2024-02-15T18:13:24.887 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:04:47 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:13:26.498 INFO:teuthology.orchestra.run.smithi042.stdout:Package ceph-base-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:13:26.548 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:13:26.549 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-02-15T18:13:26.549 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:13:26.793 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install cephadm 2024-02-15T18:13:27.579 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:04:50 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:13:29.255 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:13:29.256 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:13:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-02-15T18:13:29.256 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:13:29.256 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:13:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm noarch 2:16.2.4-0.el8 ceph-noarch 71 k 2024-02-15T18:13:29.256 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:29.257 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:13:29.257 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:13:29.257 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-02-15T18:13:29.257 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:29.257 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 71 k 2024-02-15T18:13:29.257 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 290 k 2024-02-15T18:13:29.257 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:13:29.306 INFO:teuthology.orchestra.run.smithi042.stdout:cephadm-16.2.4-0.el8.noarch.rpm 1.5 MB/s | 71 kB 00:00 2024-02-15T18:13:29.306 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:13:29.306 INFO:teuthology.orchestra.run.smithi042.stdout:Total 1.4 MB/s | 71 kB 00:00 2024-02-15T18:13:29.306 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:13:29.314 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:13:29.314 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:13:29.316 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:13:29.319 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:13:29.366 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:13:29.832 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:13:30.039 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:13:30.273 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:13:30.755 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:13:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:30.756 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:13:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm-2:16.2.4-0.el8.noarch 2024-02-15T18:13:30.756 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:30.756 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:13:30.993 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-immutable-object-cache 2024-02-15T18:13:31.778 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:04:54 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:13:33.447 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:13:33.448 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:13:33.448 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repo Size 2024-02-15T18:13:33.449 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:13:33.449 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:13:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache x86_64 2:16.2.4-0.el8 ceph 163 k 2024-02-15T18:13:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:33.449 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:13:33.449 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:13:33.449 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-02-15T18:13:33.449 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:33.449 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 163 k 2024-02-15T18:13:33.450 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 587 k 2024-02-15T18:13:33.450 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:13:33.494 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-immutable-object-cache-16.2.4-0.el8.x86_64 3.6 MB/s | 163 kB 00:00 2024-02-15T18:13:33.494 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:13:33.495 INFO:teuthology.orchestra.run.smithi042.stdout:Total 3.5 MB/s | 163 kB 00:00 2024-02-15T18:13:33.495 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:13:33.506 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:13:33.506 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:13:33.558 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:13:33.561 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:13:33.677 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:13:33.891 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:13:34.322 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:13:34.714 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:13:34.714 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:34.714 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:13:34.714 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 2024-02-15T18:13:34.714 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:34.714 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:13:34.926 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr 2024-02-15T18:13:35.717 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:04:58 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:13:37.320 INFO:teuthology.orchestra.run.smithi042.stdout:Package ceph-mgr-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:13:37.371 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:13:37.372 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-02-15T18:13:37.372 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:13:37.486 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-dashboard 2024-02-15T18:13:38.276 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:05:01 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:13:39.935 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:13:39.936 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:13:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-02-15T18:13:39.936 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:13:39.936 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:13:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard noarch 2:16.2.4-0.el8 ceph-noarch 2.7 M 2024-02-15T18:13:39.936 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-02-15T18:13:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards noarch 2:16.2.4-0.el8 ceph-noarch 24 k 2024-02-15T18:13:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts noarch 2:16.2.4-0.el8 ceph-noarch 8.9 k 2024-02-15T18:13:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-02-15T18:13:39.936 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-15T18:13:39.937 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-15T18:13:39.937 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:39.937 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:13:39.937 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:13:39.937 INFO:teuthology.orchestra.run.smithi042.stdout:Install 6 Packages 2024-02-15T18:13:39.937 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:39.937 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 3.0 M 2024-02-15T18:13:39.938 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 15 M 2024-02-15T18:13:39.938 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:13:40.097 INFO:teuthology.orchestra.run.smithi042.stdout:(1/6): ceph-grafana-dashboards-16.2.4-0.el8.noa 426 kB/s | 24 kB 00:00 2024-02-15T18:13:40.122 INFO:teuthology.orchestra.run.smithi042.stdout:(2/6): ceph-prometheus-alerts-16.2.4-0.el8.noar 109 kB/s | 8.9 kB 00:00 2024-02-15T18:13:40.189 INFO:teuthology.orchestra.run.smithi042.stdout:(3/6): ceph-mgr-dashboard-16.2.4-0.el8.noarch.r 18 MB/s | 2.7 MB 00:00 2024-02-15T18:13:40.264 INFO:teuthology.orchestra.run.smithi042.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 255 kB/s | 43 kB 00:00 2024-02-15T18:13:40.590 INFO:teuthology.orchestra.run.smithi042.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 72 kB/s | 34 kB 00:00 2024-02-15T18:13:40.882 INFO:teuthology.orchestra.run.smithi042.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 284 kB/s | 196 kB 00:00 2024-02-15T18:13:40.885 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:13:40.885 INFO:teuthology.orchestra.run.smithi042.stdout:Total 3.1 MB/s | 3.0 MB 00:00 2024-02-15T18:13:40.898 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:13:40.909 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:13:40.909 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:13:41.093 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:13:41.097 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:13:41.375 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:13:41.616 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-02-15T18:13:42.052 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-02-15T18:13:42.267 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-02-15T18:13:42.389 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 4/6 2024-02-15T18:13:43.445 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 5/6 2024-02-15T18:13:43.682 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:13:44.020 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:13:44.020 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 1/6 2024-02-15T18:13:44.020 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2/6 2024-02-15T18:13:44.021 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 3/6 2024-02-15T18:13:44.021 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-02-15T18:13:44.021 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-02-15T18:13:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-02-15T18:13:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:44.431 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:13:44.431 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 2024-02-15T18:13:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2024-02-15T18:13:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 2024-02-15T18:13:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-15T18:13:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-15T18:13:44.432 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-15T18:13:44.433 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:44.433 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:13:44.557 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-15T18:13:45.348 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:05:08 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:13:47.011 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:13:47.013 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================================ 2024-02-15T18:13:47.013 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-02-15T18:13:47.013 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================================ 2024-02-15T18:13:47.013 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:13:47.013 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.4-0.el8 ceph-noarch 8.6 M 2024-02-15T18:13:47.013 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-02-15T18:13:47.013 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-02-15T18:13:47.013 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-02-15T18:13:47.014 INFO:teuthology.orchestra.run.smithi042.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-02-15T18:13:47.014 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-02-15T18:13:47.014 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-02-15T18:13:47.014 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-02-15T18:13:47.014 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-02-15T18:13:47.014 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:47.014 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:13:47.014 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================================ 2024-02-15T18:13:47.014 INFO:teuthology.orchestra.run.smithi042.stdout:Install 8 Packages 2024-02-15T18:13:47.014 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:13:47.015 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 37 M 2024-02-15T18:13:47.015 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 210 M 2024-02-15T18:13:47.015 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:13:47.423 INFO:teuthology.orchestra.run.smithi042.stdout:(1/8): ceph-mgr-diskprediction-local-16.2.4-0.e 58 MB/s | 8.6 MB 00:00 2024-02-15T18:13:51.406 INFO:teuthology.orchestra.run.smithi042.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 959 kB/s | 3.7 MB 00:03 2024-02-15T18:13:51.982 INFO:teuthology.orchestra.run.smithi042.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 392 kB/s | 225 kB 00:00 2024-02-15T18:13:53.150 INFO:teuthology.orchestra.run.smithi042.stdout:(4/8): python3-scipy-1.0.0-21.module_el8+762+77 12 MB/s | 14 MB 00:01 2024-02-15T18:13:53.309 INFO:teuthology.orchestra.run.smithi042.stdout:(5/8): libgfortran-8.5.0-21.el8.x86_64.rpm 4.0 MB/s | 645 kB 00:00 2024-02-15T18:13:53.350 INFO:teuthology.orchestra.run.smithi042.stdout:(6/8): libquadmath-8.5.0-21.el8.x86_64.rpm 4.1 MB/s | 172 kB 00:00 2024-02-15T18:13:57.408 INFO:teuthology.orchestra.run.smithi191.stdout:[MIRROR] jq-1.6-8.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://packages.oit.ncsu.edu/centos/8-stream/AppStream/x86_64/os/Packages/jq-1.6-8.el8.x86_64.rpm [Failed to connect to packages.oit.ncsu.edu port 80: Connection timed out] 2024-02-15T18:13:57.409 INFO:teuthology.orchestra.run.smithi191.stdout:[MIRROR] oniguruma-6.8.2-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://packages.oit.ncsu.edu/centos/8-stream/AppStream/x86_64/os/Packages/oniguruma-6.8.2-3.el8.x86_64.rpm [Failed to connect to packages.oit.ncsu.edu port 80: Connection timed out] 2024-02-15T18:13:57.510 INFO:teuthology.orchestra.run.smithi191.stdout:(5/6): oniguruma-6.8.2-3.el8.x86_64.rpm 1.4 kB/s | 188 kB 02:10 2024-02-15T18:13:57.536 INFO:teuthology.orchestra.run.smithi191.stdout:(6/6): jq-1.6-8.el8.x86_64.rpm 1.6 kB/s | 203 kB 02:10 2024-02-15T18:13:57.536 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:13:57.536 INFO:teuthology.orchestra.run.smithi191.stdout:Total 414 kB/s | 53 MB 02:10 2024-02-15T18:13:57.576 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:13:57.597 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:13:57.597 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:13:57.921 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:13:57.924 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:13:58.429 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:13:58.592 INFO:teuthology.orchestra.run.smithi042.stdout:(7/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 439 kB/s | 4.9 MB 00:11 2024-02-15T18:13:58.656 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-02-15T18:13:58.658 INFO:teuthology.orchestra.run.smithi042.stdout:(8/8): openblas-0.3.15-4.el8.x86_64.rpm 425 kB/s | 4.7 MB 00:11 2024-02-15T18:13:58.660 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:13:58.660 INFO:teuthology.orchestra.run.smithi042.stdout:Total 3.2 MB/s | 37 MB 00:11 2024-02-15T18:13:58.882 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-02-15T18:13:58.898 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:13:58.918 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:13:58.918 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:13:59.039 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-15T18:13:59.140 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-02-15T18:13:59.263 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:13:59.266 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:13:59.318 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-02-15T18:13:59.462 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephsqlite-2:16.2.4-0.el8.x86_64 5/6 2024-02-15T18:13:59.884 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:14:00.055 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-15T18:14:00.287 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-15T18:14:00.461 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-15T18:14:01.090 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-15T18:14:01.263 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-02-15T18:14:01.910 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-02-15T18:14:02.064 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-15T18:14:02.795 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-15T18:14:03.155 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-02-15T18:14:03.672 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-15T18:14:04.374 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephsqlite-2:16.2.4-0.el8.x86_64 5/6 2024-02-15T18:14:04.593 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-test-2:16.2.4-0.el8.x86_64 6/6 2024-02-15T18:14:04.862 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-test-2:16.2.4-0.el8.x86_64 6/6 2024-02-15T18:14:04.862 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-test-2:16.2.4-0.el8.x86_64 1/6 2024-02-15T18:14:04.862 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephsqlite-2:16.2.4-0.el8.x86_64 2/6 2024-02-15T18:14:04.862 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-02-15T18:14:04.862 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-02-15T18:14:04.863 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-02-15T18:14:05.296 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-02-15T18:14:05.296 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:05.297 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:14:05.297 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test-2:16.2.4-0.el8.x86_64 jq-1.6-8.el8.x86_64 2024-02-15T18:14:05.297 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite-2:16.2.4-0.el8.x86_64 oniguruma-6.8.2-3.el8.x86_64 2024-02-15T18:14:05.297 INFO:teuthology.orchestra.run.smithi191.stdout: socat-1.7.4.1-1.el8.x86_64 xmlstarlet-1.6.1-20.el8.x86_64 2024-02-15T18:14:05.297 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:05.297 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:14:05.513 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph 2024-02-15T18:14:05.555 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-15T18:14:06.282 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:05:11 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:14:06.909 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-02-15T18:14:07.122 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2024-02-15T18:14:07.445 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2024-02-15T18:14:07.446 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 1/8 2024-02-15T18:14:07.446 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-02-15T18:14:07.446 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-02-15T18:14:07.446 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-02-15T18:14:07.446 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-02-15T18:14:07.446 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-02-15T18:14:07.446 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-02-15T18:14:07.834 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:14:07.838 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:14:07.838 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-02-15T18:14:07.838 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:14:07.838 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:14:07.838 INFO:teuthology.orchestra.run.smithi191.stdout: ceph x86_64 2:16.2.4-0.el8 ceph 6.3 k 2024-02-15T18:14:07.838 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-02-15T18:14:07.838 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds x86_64 2:16.2.4-0.el8 ceph 2.4 M 2024-02-15T18:14:07.838 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr x86_64 2:16.2.4-0.el8 ceph 1.6 M 2024-02-15T18:14:07.838 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core noarch 2:16.2.4-0.el8 ceph-noarch 220 k 2024-02-15T18:14:07.838 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon x86_64 2:16.2.4-0.el8 ceph 5.0 M 2024-02-15T18:14:07.838 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd x86_64 2:16.2.4-0.el8 ceph 18 M 2024-02-15T18:14:07.839 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-15T18:14:07.839 INFO:teuthology.orchestra.run.smithi191.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-15T18:14:07.839 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-02-15T18:14:07.839 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-15T18:14:07.839 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-15T18:14:07.839 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-02-15T18:14:07.839 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-15T18:14:07.839 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-15T18:14:07.839 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-15T18:14:07.839 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-02-15T18:14:07.839 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-02-15T18:14:07.839 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-15T18:14:07.839 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-15T18:14:07.839 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:14:07.840 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:14:07.841 INFO:teuthology.orchestra.run.smithi191.stdout:Install 32 Packages 2024-02-15T18:14:07.841 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:07.841 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 31 M 2024-02-15T18:14:07.841 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 111 M 2024-02-15T18:14:07.841 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:14:07.870 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-02-15T18:14:07.871 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:14:07.871 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:14:07.871 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 2024-02-15T18:14:07.871 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-02-15T18:14:07.871 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-02-15T18:14:07.871 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-0.3.15-4.el8.x86_64 2024-02-15T18:14:07.871 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-02-15T18:14:07.871 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-15T18:14:07.871 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-15T18:14:07.871 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-02-15T18:14:07.871 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:14:07.871 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:14:07.992 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-rook 2024-02-15T18:14:08.787 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:05:31 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:14:09.445 INFO:teuthology.orchestra.run.smithi191.stdout:(1/32): ceph-16.2.4-0.el8.x86_64.rpm 98 kB/s | 6.3 kB 00:00 2024-02-15T18:14:09.503 INFO:teuthology.orchestra.run.smithi191.stdout:(2/32): ceph-mgr-16.2.4-0.el8.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-02-15T18:14:09.612 INFO:teuthology.orchestra.run.smithi191.stdout:(3/32): ceph-mds-16.2.4-0.el8.x86_64.rpm 10 MB/s | 2.4 MB 00:00 2024-02-15T18:14:09.646 INFO:teuthology.orchestra.run.smithi191.stdout:(4/32): ceph-mgr-modules-core-16.2.4-0.el8.noar 6.5 MB/s | 220 kB 00:00 2024-02-15T18:14:09.746 INFO:teuthology.orchestra.run.smithi191.stdout:(5/32): ceph-mon-16.2.4-0.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2024-02-15T18:14:10.030 INFO:teuthology.orchestra.run.smithi191.stdout:(6/32): ceph-osd-16.2.4-0.el8.x86_64.rpm 35 MB/s | 18 MB 00:00 2024-02-15T18:14:10.454 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:14:10.455 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:14:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-02-15T18:14:10.456 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:14:10.456 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:14:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook noarch 2:16.2.4-0.el8 ceph-noarch 27 k 2024-02-15T18:14:10.456 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-02-15T18:14:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-15T18:14:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-15T18:14:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-15T18:14:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-02-15T18:14:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-02-15T18:14:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-15T18:14:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-02-15T18:14:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-02-15T18:14:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-02-15T18:14:10.456 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-02-15T18:14:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-15T18:14:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-15T18:14:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:14:10.457 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:14:10.457 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:14:10.457 INFO:teuthology.orchestra.run.smithi042.stdout:Install 13 Packages 2024-02-15T18:14:10.457 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:14:10.457 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 1.9 M 2024-02-15T18:14:10.458 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 24 M 2024-02-15T18:14:10.458 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:14:11.315 INFO:teuthology.orchestra.run.smithi042.stdout:(1/13): ceph-mgr-rook-16.2.4-0.el8.noarch.rpm 538 kB/s | 27 kB 00:00 2024-02-15T18:14:11.696 INFO:teuthology.orchestra.run.smithi191.stdout:(7/32): python3-markupsafe-0.23-19.el8.x86_64.r 20 kB/s | 39 kB 00:01 2024-02-15T18:14:11.721 INFO:teuthology.orchestra.run.smithi191.stdout:(8/32): python3-mako-1.0.6-14.el8.noarch.rpm 76 kB/s | 157 kB 00:02 2024-02-15T18:14:11.780 INFO:teuthology.orchestra.run.smithi191.stdout:(9/32): python3-pytz-2017.2-11.el8.noarch.rpm 684 kB/s | 57 kB 00:00 2024-02-15T18:14:11.880 INFO:teuthology.orchestra.run.smithi191.stdout:(10/32): python3-werkzeug-0.12.2-4.el8.noarch.r 2.8 MB/s | 457 kB 00:00 2024-02-15T18:14:11.922 INFO:teuthology.orchestra.run.smithi191.stdout:(11/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.7 MB/s | 249 kB 00:00 2024-02-15T18:14:11.981 INFO:teuthology.orchestra.run.smithi191.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 1.8 MB/s | 109 kB 00:00 2024-02-15T18:14:12.056 INFO:teuthology.orchestra.run.smithi191.stdout:(13/32): python3-cryptography-3.2.1-7.el8.x86_6 3.6 MB/s | 641 kB 00:00 2024-02-15T18:14:12.090 INFO:teuthology.orchestra.run.smithi191.stdout:(14/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 50 kB/s | 103 kB 00:02 2024-02-15T18:14:12.265 INFO:teuthology.orchestra.run.smithi191.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 154 kB/s | 44 kB 00:00 2024-02-15T18:14:12.415 INFO:teuthology.orchestra.run.smithi191.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 523 kB/s | 185 kB 00:00 2024-02-15T18:14:12.441 INFO:teuthology.orchestra.run.smithi191.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 492 kB/s | 173 kB 00:00 2024-02-15T18:14:12.491 INFO:teuthology.orchestra.run.smithi191.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2024-02-15T18:14:12.516 INFO:teuthology.orchestra.run.smithi191.stdout:(19/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.5 MB/s | 384 kB 00:00 2024-02-15T18:14:12.541 INFO:teuthology.orchestra.run.smithi191.stdout:(20/32): python3-jaraco-functools-2.0-4.el8.noa 175 kB/s | 18 kB 00:00 2024-02-15T18:14:12.575 INFO:teuthology.orchestra.run.smithi191.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 583 kB/s | 49 kB 00:00 2024-02-15T18:14:12.600 INFO:teuthology.orchestra.run.smithi191.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 701 kB/s | 59 kB 00:00 2024-02-15T18:14:12.651 INFO:teuthology.orchestra.run.smithi191.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 207 kB/s | 16 kB 00:00 2024-02-15T18:14:12.676 INFO:teuthology.orchestra.run.smithi191.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 249 kB/s | 19 kB 00:00 2024-02-15T18:14:12.701 INFO:teuthology.orchestra.run.smithi191.stdout:(25/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2024-02-15T18:14:12.727 INFO:teuthology.orchestra.run.smithi191.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 317 kB/s | 24 kB 00:00 2024-02-15T18:14:12.777 INFO:teuthology.orchestra.run.smithi191.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 291 kB/s | 29 kB 00:00 2024-02-15T18:14:12.802 INFO:teuthology.orchestra.run.smithi191.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 274 kB/s | 27 kB 00:00 2024-02-15T18:14:12.844 INFO:teuthology.orchestra.run.smithi191.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 2.1 MB/s | 255 kB 00:00 2024-02-15T18:14:12.869 INFO:teuthology.orchestra.run.smithi191.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.7 MB/s | 251 kB 00:00 2024-02-15T18:14:12.894 INFO:teuthology.orchestra.run.smithi191.stdout:(31/32): python3-webtest-2.0.33-1.el8.noarch.rp 928 kB/s | 85 kB 00:00 2024-02-15T18:14:12.911 INFO:teuthology.orchestra.run.smithi191.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 343 kB/s | 23 kB 00:00 2024-02-15T18:14:12.916 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:14:12.916 INFO:teuthology.orchestra.run.smithi191.stdout:Total 6.1 MB/s | 31 MB 00:05 2024-02-15T18:14:12.996 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:14:13.035 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:14:13.035 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:14:13.368 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:14:13.371 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:14:14.509 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:14:14.739 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-02-15T18:14:14.957 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-02-15T18:14:15.242 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-02-15T18:14:15.424 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-02-15T18:14:15.597 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-02-15T18:14:15.751 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-02-15T18:14:15.912 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-02-15T18:14:16.087 INFO:teuthology.orchestra.run.smithi042.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 5.5 kB/s | 27 kB 00:04 2024-02-15T18:14:16.098 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-02-15T18:14:16.118 INFO:teuthology.orchestra.run.smithi042.stdout:(3/13): python3-jsonpointer-1.10-11.el8.noarch. 4.1 kB/s | 20 kB 00:04 2024-02-15T18:14:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 1.2 MB/s | 110 kB 00:00 2024-02-15T18:14:16.227 INFO:teuthology.orchestra.run.smithi042.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.4 MB/s | 155 kB 00:00 2024-02-15T18:14:16.269 INFO:teuthology.orchestra.run.smithi042.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 464 kB/s | 43 kB 00:00 2024-02-15T18:14:16.272 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-02-15T18:14:16.411 INFO:teuthology.orchestra.run.smithi042.stdout:(7/13): python3-cachetools-3.1.1-4.el8.noarch.r 180 kB/s | 33 kB 00:00 2024-02-15T18:14:16.436 INFO:teuthology.orchestra.run.smithi042.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 97 kB/s | 16 kB 00:00 2024-02-15T18:14:16.492 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-02-15T18:14:16.545 INFO:teuthology.orchestra.run.smithi042.stdout:(9/13): python3-google-auth-1.1.1-10.el8.noarch 608 kB/s | 81 kB 00:00 2024-02-15T18:14:16.628 INFO:teuthology.orchestra.run.smithi042.stdout:(10/13): python3-rsa-4.9-2.el8.noarch.rpm 770 kB/s | 64 kB 00:00 2024-02-15T18:14:16.698 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-02-15T18:14:16.712 INFO:teuthology.orchestra.run.smithi042.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 735 kB/s | 61 kB 00:00 2024-02-15T18:14:16.770 INFO:teuthology.orchestra.run.smithi042.stdout:(12/13): python3-kubernetes-11.0.0-6.el8.noarch 3.5 MB/s | 1.2 MB 00:00 2024-02-15T18:14:16.964 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-02-15T18:14:17.329 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-02-15T18:14:17.578 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-02-15T18:14:17.759 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-02-15T18:14:17.964 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-02-15T18:14:18.168 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-15T18:14:18.433 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-02-15T18:14:18.672 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-02-15T18:14:18.828 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-02-15T18:14:19.039 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-02-15T18:14:19.281 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-02-15T18:14:19.495 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-02-15T18:14:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-02-15T18:14:20.076 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-02-15T18:14:20.382 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-02-15T18:14:20.798 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 27/32 2024-02-15T18:14:21.027 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-2:16.2.4-0.el8.x86_64 28/32 2024-02-15T18:14:22.575 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-2:16.2.4-0.el8.x86_64 28/32 2024-02-15T18:14:22.793 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-osd-2:16.2.4-0.el8.x86_64 29/32 2024-02-15T18:14:23.226 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-osd-2:16.2.4-0.el8.x86_64 29/32 2024-02-15T18:14:23.485 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mon-2:16.2.4-0.el8.x86_64 30/32 2024-02-15T18:14:23.707 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mon-2:16.2.4-0.el8.x86_64 30/32 2024-02-15T18:14:23.933 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mds-2:16.2.4-0.el8.x86_64 31/32 2024-02-15T18:14:23.951 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mds-2:16.2.4-0.el8.x86_64 31/32 2024-02-15T18:14:24.074 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-2:16.2.4-0.el8.x86_64 32/32 2024-02-15T18:14:24.490 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-2:16.2.4-0.el8.x86_64 32/32 2024-02-15T18:14:24.490 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-2:16.2.4-0.el8.x86_64 1/32 2024-02-15T18:14:24.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mds-2:16.2.4-0.el8.x86_64 2/32 2024-02-15T18:14:24.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-2:16.2.4-0.el8.x86_64 3/32 2024-02-15T18:14:24.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mon-2:16.2.4-0.el8.x86_64 4/32 2024-02-15T18:14:24.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-osd-2:16.2.4-0.el8.x86_64 5/32 2024-02-15T18:14:24.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 6/32 2024-02-15T18:14:24.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-02-15T18:14:24.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-02-15T18:14:24.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-02-15T18:14:24.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-02-15T18:14:24.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-02-15T18:14:24.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-02-15T18:14:24.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-02-15T18:14:24.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-02-15T18:14:24.491 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-02-15T18:14:24.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-02-15T18:14:24.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-02-15T18:14:24.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-02-15T18:14:24.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-02-15T18:14:24.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-02-15T18:14:24.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-02-15T18:14:24.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-02-15T18:14:24.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-02-15T18:14:24.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-02-15T18:14:24.492 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-02-15T18:14:24.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-02-15T18:14:24.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-02-15T18:14:24.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-02-15T18:14:24.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-02-15T18:14:24.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-02-15T18:14:24.493 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-02-15T18:14:25.434 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-02-15T18:14:25.434 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:25.434 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:14:25.434 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-2:16.2.4-0.el8.x86_64 2024-02-15T18:14:25.434 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds-2:16.2.4-0.el8.x86_64 2024-02-15T18:14:25.434 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-2:16.2.4-0.el8.x86_64 2024-02-15T18:14:25.434 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon-2:16.2.4-0.el8.x86_64 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd-2:16.2.4-0.el8.x86_64 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-15T18:14:25.435 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend-2.6-1.el8.noarch 2024-02-15T18:14:25.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-15T18:14:25.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-15T18:14:25.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-15T18:14:25.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-15T18:14:25.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-15T18:14:25.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-15T18:14:25.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-15T18:14:25.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-15T18:14:25.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-15T18:14:25.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-15T18:14:25.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-15T18:14:25.436 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-15T18:14:25.436 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:25.437 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:14:25.703 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-base 2024-02-15T18:14:26.460 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:05:31 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:14:27.973 INFO:teuthology.orchestra.run.smithi191.stdout:Package ceph-base-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:14:28.020 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:14:28.021 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-02-15T18:14:28.021 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:14:28.124 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install cephadm 2024-02-15T18:14:28.870 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:05:33 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:14:30.438 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:14:30.439 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:14:30.439 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-02-15T18:14:30.439 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:14:30.439 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:14:30.439 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm noarch 2:16.2.4-0.el8 ceph-noarch 71 k 2024-02-15T18:14:30.439 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:30.439 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:14:30.440 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:14:30.440 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-02-15T18:14:30.440 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:30.440 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 71 k 2024-02-15T18:14:30.440 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 290 k 2024-02-15T18:14:30.440 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:14:30.482 INFO:teuthology.orchestra.run.smithi191.stdout:cephadm-16.2.4-0.el8.noarch.rpm 1.6 MB/s | 71 kB 00:00 2024-02-15T18:14:30.483 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:14:30.483 INFO:teuthology.orchestra.run.smithi191.stdout:Total 1.6 MB/s | 71 kB 00:00 2024-02-15T18:14:30.483 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:14:30.490 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:14:30.490 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:14:30.492 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:14:30.495 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:14:30.539 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:14:31.112 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:14:31.280 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:14:31.505 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:14:31.898 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephadm-2:16.2.4-0.el8.noarch 1/1 2024-02-15T18:14:31.899 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:31.899 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:14:31.899 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm-2:16.2.4-0.el8.noarch 2024-02-15T18:14:31.899 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:31.899 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:14:32.116 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-immutable-object-cache 2024-02-15T18:14:32.881 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:05:37 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:14:34.437 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:14:34.437 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:14:34.437 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repo Size 2024-02-15T18:14:34.437 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:14:34.437 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:14:34.437 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache x86_64 2:16.2.4-0.el8 ceph 163 k 2024-02-15T18:14:34.437 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:34.438 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:14:34.438 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:14:34.438 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-02-15T18:14:34.438 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:34.438 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 163 k 2024-02-15T18:14:34.438 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 587 k 2024-02-15T18:14:34.438 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:14:34.845 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-immutable-object-cache-16.2.4-0.el8.x86_64 401 kB/s | 163 kB 00:00 2024-02-15T18:14:34.846 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:14:34.846 INFO:teuthology.orchestra.run.smithi191.stdout:Total 399 kB/s | 163 kB 00:00 2024-02-15T18:14:34.846 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:14:34.858 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:14:34.859 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:14:34.910 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:14:34.914 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:14:35.808 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:14:37.056 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:14:37.464 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:14:37.846 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:14:37.846 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:37.846 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:14:37.846 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 2024-02-15T18:14:37.846 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:37.847 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:14:38.055 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-mgr 2024-02-15T18:14:38.820 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:05:43 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:14:40.334 INFO:teuthology.orchestra.run.smithi191.stdout:Package ceph-mgr-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:14:40.381 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:14:40.382 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-02-15T18:14:40.382 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:14:40.493 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-mgr-dashboard 2024-02-15T18:14:41.250 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:05:46 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:14:42.803 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:14:42.804 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:14:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-02-15T18:14:42.804 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:14:42.804 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:14:42.804 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard noarch 2:16.2.4-0.el8 ceph-noarch 2.7 M 2024-02-15T18:14:42.805 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-02-15T18:14:42.805 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards noarch 2:16.2.4-0.el8 ceph-noarch 24 k 2024-02-15T18:14:42.805 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts noarch 2:16.2.4-0.el8 ceph-noarch 8.9 k 2024-02-15T18:14:42.805 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-02-15T18:14:42.805 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-15T18:14:42.805 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-15T18:14:42.805 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:42.805 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:14:42.805 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:14:42.805 INFO:teuthology.orchestra.run.smithi191.stdout:Install 6 Packages 2024-02-15T18:14:42.805 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:42.806 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 3.0 M 2024-02-15T18:14:42.806 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 15 M 2024-02-15T18:14:42.806 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:14:43.482 INFO:teuthology.orchestra.run.smithi191.stdout:(1/6): ceph-grafana-dashboards-16.2.4-0.el8.noa 507 kB/s | 24 kB 00:00 2024-02-15T18:14:43.507 INFO:teuthology.orchestra.run.smithi191.stdout:(2/6): ceph-prometheus-alerts-16.2.4-0.el8.noar 123 kB/s | 8.9 kB 00:00 2024-02-15T18:14:43.574 INFO:teuthology.orchestra.run.smithi191.stdout:(3/6): ceph-mgr-dashboard-16.2.4-0.el8.noarch.r 19 MB/s | 2.7 MB 00:00 2024-02-15T18:14:43.698 INFO:teuthology.orchestra.run.smithi191.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 198 kB/s | 43 kB 00:00 2024-02-15T18:14:43.741 INFO:teuthology.orchestra.run.smithi191.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 144 kB/s | 34 kB 00:00 2024-02-15T18:14:43.925 INFO:teuthology.orchestra.run.smithi191.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 568 kB/s | 196 kB 00:00 2024-02-15T18:14:43.928 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:14:43.928 INFO:teuthology.orchestra.run.smithi191.stdout:Total 2.6 MB/s | 3.0 MB 00:01 2024-02-15T18:14:43.940 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:14:43.951 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:14:43.951 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:14:44.128 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:14:44.131 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:14:44.424 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:14:44.651 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-02-15T18:14:44.863 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-02-15T18:14:45.098 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-02-15T18:14:45.210 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 4/6 2024-02-15T18:14:46.241 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 5/6 2024-02-15T18:14:46.531 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:14:46.816 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:14:46.816 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 1/6 2024-02-15T18:14:46.816 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2/6 2024-02-15T18:14:46.816 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 3/6 2024-02-15T18:14:46.817 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-02-15T18:14:46.817 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-02-15T18:14:47.323 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-02-15T18:14:47.323 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:47.323 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:14:47.323 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 2024-02-15T18:14:47.323 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2024-02-15T18:14:47.323 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 2024-02-15T18:14:47.323 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-15T18:14:47.323 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-15T18:14:47.323 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-15T18:14:47.323 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:47.324 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:14:47.443 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-15T18:14:48.198 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:05:53 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:14:49.739 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:14:49.740 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-02-15T18:14:49.741 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-02-15T18:14:49.741 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-02-15T18:14:49.741 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:14:49.741 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.4-0.el8 ceph-noarch 8.6 M 2024-02-15T18:14:49.741 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-02-15T18:14:49.741 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-02-15T18:14:49.741 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-02-15T18:14:49.741 INFO:teuthology.orchestra.run.smithi191.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-02-15T18:14:49.741 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-02-15T18:14:49.741 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-02-15T18:14:49.741 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-02-15T18:14:49.742 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-02-15T18:14:49.742 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:49.742 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:14:49.742 INFO:teuthology.orchestra.run.smithi191.stdout:============================================================================================ 2024-02-15T18:14:49.742 INFO:teuthology.orchestra.run.smithi191.stdout:Install 8 Packages 2024-02-15T18:14:49.742 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:14:49.742 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 37 M 2024-02-15T18:14:49.742 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 210 M 2024-02-15T18:14:49.743 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:14:50.288 INFO:teuthology.orchestra.run.smithi191.stdout:(1/8): ceph-mgr-diskprediction-local-16.2.4-0.e 50 MB/s | 8.6 MB 00:00 2024-02-15T18:14:53.648 INFO:teuthology.orchestra.run.smithi191.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 1.1 MB/s | 3.7 MB 00:03 2024-02-15T18:14:54.465 INFO:teuthology.orchestra.run.smithi191.stdout:(3/8): openblas-0.3.15-4.el8.x86_64.rpm 1.1 MB/s | 4.7 MB 00:04 2024-02-15T18:14:54.523 INFO:teuthology.orchestra.run.smithi191.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 257 kB/s | 225 kB 00:00 2024-02-15T18:14:54.682 INFO:teuthology.orchestra.run.smithi191.stdout:(5/8): libgfortran-8.5.0-21.el8.x86_64.rpm 4.0 MB/s | 645 kB 00:00 2024-02-15T18:14:54.724 INFO:teuthology.orchestra.run.smithi191.stdout:(6/8): libquadmath-8.5.0-21.el8.x86_64.rpm 4.1 MB/s | 172 kB 00:00 2024-02-15T18:14:55.374 INFO:teuthology.orchestra.run.smithi191.stdout:(7/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 946 kB/s | 4.9 MB 00:05 2024-02-15T18:15:01.540 INFO:teuthology.orchestra.run.smithi191.stdout:(8/8): python3-scipy-1.0.0-21.module_el8+762+77 2.0 MB/s | 14 MB 00:07 2024-02-15T18:15:01.541 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:15:01.541 INFO:teuthology.orchestra.run.smithi191.stdout:Total 3.1 MB/s | 37 MB 00:11 2024-02-15T18:15:01.772 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:15:01.790 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:15:01.790 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:15:02.127 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:15:02.130 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:15:02.759 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:15:02.900 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-15T18:15:03.041 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-02-15T18:15:03.258 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-15T18:15:03.835 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-02-15T18:15:04.059 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-02-15T18:15:04.655 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-02-15T18:15:04.832 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-15T18:15:05.524 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-02-15T18:15:05.836 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-02-15T18:15:06.090 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-15T18:15:07.840 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-02-15T18:15:09.165 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-02-15T18:15:09.369 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2024-02-15T18:15:09.687 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2024-02-15T18:15:09.687 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 1/8 2024-02-15T18:15:09.687 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-02-15T18:15:09.687 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-02-15T18:15:09.687 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-02-15T18:15:09.687 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-02-15T18:15:09.687 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-02-15T18:15:09.687 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-02-15T18:15:10.112 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-02-15T18:15:10.112 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:10.112 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:15:10.112 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 2024-02-15T18:15:10.112 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-02-15T18:15:10.112 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-02-15T18:15:10.112 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-0.3.15-4.el8.x86_64 2024-02-15T18:15:10.112 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-02-15T18:15:10.112 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-15T18:15:10.112 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-15T18:15:10.112 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-02-15T18:15:10.113 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:10.113 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:15:10.218 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-mgr-rook 2024-02-15T18:15:10.985 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:06:15 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:15:12.536 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:15:12.537 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:12.537 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-02-15T18:15:12.537 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:12.538 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:15:12.538 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook noarch 2:16.2.4-0.el8 ceph-noarch 27 k 2024-02-15T18:15:12.538 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-02-15T18:15:12.538 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-15T18:15:12.538 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-15T18:15:12.538 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-15T18:15:12.538 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-02-15T18:15:12.538 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-02-15T18:15:12.538 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-15T18:15:12.538 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-02-15T18:15:12.538 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-02-15T18:15:12.538 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-02-15T18:15:12.538 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-02-15T18:15:12.539 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-15T18:15:12.539 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-15T18:15:12.539 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:12.539 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:15:12.539 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:12.539 INFO:teuthology.orchestra.run.smithi191.stdout:Install 13 Packages 2024-02-15T18:15:12.539 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:12.540 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 1.9 M 2024-02-15T18:15:12.540 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 24 M 2024-02-15T18:15:12.540 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:15:13.179 INFO:teuthology.orchestra.run.smithi191.stdout:(1/13): ceph-mgr-rook-16.2.4-0.el8.noarch.rpm 518 kB/s | 27 kB 00:00 2024-02-15T18:15:13.387 INFO:teuthology.orchestra.run.smithi191.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 77 kB/s | 20 kB 00:00 2024-02-15T18:15:13.604 INFO:teuthology.orchestra.run.smithi191.stdout:(3/13): python3-pyasn1-modules-0.3.7-6.el8.noar 508 kB/s | 110 kB 00:00 2024-02-15T18:15:13.721 INFO:teuthology.orchestra.run.smithi191.stdout:(4/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.3 MB/s | 155 kB 00:00 2024-02-15T18:15:13.763 INFO:teuthology.orchestra.run.smithi191.stdout:(5/13): python3-requests-oauthlib-1.0.0-1.el8.n 1.0 MB/s | 43 kB 00:00 2024-02-15T18:15:13.872 INFO:teuthology.orchestra.run.smithi191.stdout:(6/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 182 kB/s | 126 kB 00:00 2024-02-15T18:15:14.022 INFO:teuthology.orchestra.run.smithi191.stdout:(7/13): python3-cachetools-3.1.1-4.el8.noarch.r 128 kB/s | 33 kB 00:00 2024-02-15T18:15:14.097 INFO:teuthology.orchestra.run.smithi191.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 73 kB/s | 16 kB 00:00 2024-02-15T18:15:14.164 INFO:teuthology.orchestra.run.smithi191.stdout:(9/13): python3-google-auth-1.1.1-10.el8.noarch 573 kB/s | 81 kB 00:00 2024-02-15T18:15:14.248 INFO:teuthology.orchestra.run.smithi191.stdout:(10/13): python3-rsa-4.9-2.el8.noarch.rpm 773 kB/s | 64 kB 00:00 2024-02-15T18:15:14.348 INFO:teuthology.orchestra.run.smithi191.stdout:(11/13): python3-jsonpatch-1.21-2.el8.noarch.rp 22 kB/s | 27 kB 00:01 2024-02-15T18:15:14.373 INFO:teuthology.orchestra.run.smithi191.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 488 kB/s | 61 kB 00:00 2024-02-15T18:15:14.490 INFO:teuthology.orchestra.run.smithi191.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 3.0 MB/s | 1.2 MB 00:00 2024-02-15T18:15:14.492 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:15:14.492 INFO:teuthology.orchestra.run.smithi191.stdout:Total 1.0 MB/s | 1.9 MB 00:01 2024-02-15T18:15:14.549 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:15:14.565 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:15:14.565 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:15:14.653 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:15:14.659 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:15:15.200 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:15:15.434 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-02-15T18:15:15.655 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-02-15T18:15:15.888 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-02-15T18:15:16.155 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-02-15T18:15:16.404 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-02-15T18:15:16.618 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-02-15T18:15:16.785 INFO:teuthology.orchestra.run.smithi042.stdout:(13/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.9 kB/s | 126 kB 01:05 2024-02-15T18:15:16.786 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:15:16.786 INFO:teuthology.orchestra.run.smithi042.stdout:Total 29 kB/s | 1.9 MB 01:06 2024-02-15T18:15:16.841 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:15:16.858 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:15:16.858 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:15:16.934 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-02-15T18:15:16.953 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:15:16.957 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:15:17.194 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-02-15T18:15:17.473 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:15:17.758 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-02-15T18:15:17.939 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-02-15T18:15:18.203 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-02-15T18:15:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-02-15T18:15:18.504 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-02-15T18:15:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-02-15T18:15:18.695 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-02-15T18:15:18.753 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-02-15T18:15:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-02-15T18:15:19.131 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-02-15T18:15:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-02-15T18:15:19.323 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2024-02-15T18:15:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-02-15T18:15:19.628 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2024-02-15T18:15:19.628 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-rook-2:16.2.4-0.el8.noarch 1/13 2024-02-15T18:15:19.628 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-02-15T18:15:19.628 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-02-15T18:15:19.628 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-02-15T18:15:19.628 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-02-15T18:15:19.628 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-02-15T18:15:19.628 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-02-15T18:15:19.628 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-02-15T18:15:19.628 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-02-15T18:15:19.629 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-02-15T18:15:19.629 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-02-15T18:15:19.629 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-02-15T18:15:20.092 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-02-15T18:15:20.092 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:20.092 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:15:20.092 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook-2:16.2.4-0.el8.noarch 2024-02-15T18:15:20.092 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-15T18:15:20.092 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-15T18:15:20.093 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-15T18:15:20.093 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-15T18:15:20.093 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-15T18:15:20.093 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-15T18:15:20.093 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-15T18:15:20.093 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-15T18:15:20.093 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-15T18:15:20.093 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-15T18:15:20.093 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-15T18:15:20.093 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-15T18:15:20.093 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:20.093 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:15:20.208 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-mgr-cephadm 2024-02-15T18:15:20.626 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-02-15T18:15:20.927 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-02-15T18:15:20.962 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:06:25 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:15:21.136 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-02-15T18:15:21.336 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-02-15T18:15:21.519 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2024-02-15T18:15:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2024-02-15T18:15:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-rook-2:16.2.4-0.el8.noarch 1/13 2024-02-15T18:15:21.842 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-02-15T18:15:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-02-15T18:15:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-02-15T18:15:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-02-15T18:15:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-02-15T18:15:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-02-15T18:15:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-02-15T18:15:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-02-15T18:15:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-02-15T18:15:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-02-15T18:15:21.843 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-02-15T18:15:22.443 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-02-15T18:15:22.443 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:22.443 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:15:22.443 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook-2:16.2.4-0.el8.noarch 2024-02-15T18:15:22.443 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-15T18:15:22.443 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-15T18:15:22.443 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-15T18:15:22.443 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-15T18:15:22.443 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-15T18:15:22.444 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-15T18:15:22.444 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-15T18:15:22.444 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-15T18:15:22.444 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-15T18:15:22.444 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-15T18:15:22.444 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-15T18:15:22.444 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-15T18:15:22.444 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:22.444 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:15:22.535 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:15:22.536 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:22.536 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-02-15T18:15:22.536 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:22.536 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:15:22.536 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm noarch 2:16.2.4-0.el8 ceph-noarch 89 k 2024-02-15T18:15:22.536 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-02-15T18:15:22.536 INFO:teuthology.orchestra.run.smithi191.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2024-02-15T18:15:22.537 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-02-15T18:15:22.537 INFO:teuthology.orchestra.run.smithi191.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2024-02-15T18:15:22.537 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-02-15T18:15:22.537 INFO:teuthology.orchestra.run.smithi191.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2024-02-15T18:15:22.537 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:22.537 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:15:22.537 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:22.537 INFO:teuthology.orchestra.run.smithi191.stdout:Install 6 Packages 2024-02-15T18:15:22.537 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:22.538 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 5.9 M 2024-02-15T18:15:22.538 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 24 M 2024-02-15T18:15:22.538 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:15:22.661 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-cephadm 2024-02-15T18:15:22.875 INFO:teuthology.orchestra.run.smithi191.stdout:(1/6): ceph-mgr-cephadm-16.2.4-0.el8.noarch.rpm 1.5 MB/s | 89 kB 00:00 2024-02-15T18:15:23.092 INFO:teuthology.orchestra.run.smithi191.stdout:(2/6): python3-apipkg-1.5-6.el8.noarch.rpm 98 kB/s | 21 kB 00:00 2024-02-15T18:15:23.401 INFO:teuthology.orchestra.run.smithi191.stdout:(3/6): python3-execnet-1.7.1-1.el8.noarch.rpm 1.1 MB/s | 355 kB 00:00 2024-02-15T18:15:23.450 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:06:46 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:15:23.493 INFO:teuthology.orchestra.run.smithi191.stdout:(4/6): python3-remoto-1.2.1-1.el8.noarch.rpm 533 kB/s | 49 kB 00:00 2024-02-15T18:15:24.559 INFO:teuthology.orchestra.run.smithi191.stdout:(5/6): python3-jinja2-2.10.1-4.el8.noarch.rpm 332 kB/s | 578 kB 00:01 2024-02-15T18:15:25.104 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:15:25.105 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:15:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-02-15T18:15:25.105 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:15:25.105 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:15:25.105 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm noarch 2:16.2.4-0.el8 ceph-noarch 89 k 2024-02-15T18:15:25.106 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-02-15T18:15:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2024-02-15T18:15:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-02-15T18:15:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2024-02-15T18:15:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-02-15T18:15:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2024-02-15T18:15:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:25.106 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:15:25.106 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:15:25.106 INFO:teuthology.orchestra.run.smithi042.stdout:Install 6 Packages 2024-02-15T18:15:25.106 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:25.107 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 5.9 M 2024-02-15T18:15:25.107 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 24 M 2024-02-15T18:15:25.107 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:15:25.426 INFO:teuthology.orchestra.run.smithi042.stdout:(1/6): ceph-mgr-cephadm-16.2.4-0.el8.noarch.rpm 1.9 MB/s | 89 kB 00:00 2024-02-15T18:15:25.701 INFO:teuthology.orchestra.run.smithi042.stdout:(2/6): python3-apipkg-1.5-6.el8.noarch.rpm 77 kB/s | 21 kB 00:00 2024-02-15T18:15:26.060 INFO:teuthology.orchestra.run.smithi042.stdout:(3/6): python3-execnet-1.7.1-1.el8.noarch.rpm 992 kB/s | 355 kB 00:00 2024-02-15T18:15:26.168 INFO:teuthology.orchestra.run.smithi042.stdout:(4/6): python3-remoto-1.2.1-1.el8.noarch.rpm 451 kB/s | 49 kB 00:00 2024-02-15T18:15:26.760 INFO:teuthology.orchestra.run.smithi042.stdout:(5/6): python3-jinja2-2.10.1-4.el8.noarch.rpm 419 kB/s | 578 kB 00:01 2024-02-15T18:15:31.118 INFO:teuthology.orchestra.run.smithi042.stdout:(6/6): python3-babel-2.5.1-7.el8.noarch.rpm 854 kB/s | 4.8 MB 00:05 2024-02-15T18:15:31.119 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:15:31.120 INFO:teuthology.orchestra.run.smithi042.stdout:Total 997 kB/s | 5.9 MB 00:06 2024-02-15T18:15:31.175 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:15:31.185 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:15:31.185 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:15:31.296 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:15:31.300 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:15:31.601 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:15:31.865 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2024-02-15T18:15:32.185 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2024-02-15T18:15:33.203 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2024-02-15T18:15:33.316 INFO:teuthology.orchestra.run.smithi191.stdout:(6/6): python3-babel-2.5.1-7.el8.noarch.rpm 467 kB/s | 4.8 MB 00:10 2024-02-15T18:15:33.317 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:15:33.318 INFO:teuthology.orchestra.run.smithi191.stdout:Total 556 kB/s | 5.9 MB 00:10 2024-02-15T18:15:33.371 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:15:33.382 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:15:33.382 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:15:33.487 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:15:33.491 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:15:33.524 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 4/6 2024-02-15T18:15:33.741 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 5/6 2024-02-15T18:15:33.815 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:15:33.993 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:15:34.052 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2024-02-15T18:15:34.294 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:15:34.295 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 1/6 2024-02-15T18:15:34.295 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/6 2024-02-15T18:15:34.295 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/6 2024-02-15T18:15:34.295 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2024-02-15T18:15:34.295 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2024-02-15T18:15:34.355 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2024-02-15T18:15:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2024-02-15T18:15:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:34.710 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:15:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch python3-apipkg-1.5-6.el8.noarch 2024-02-15T18:15:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel-2.5.1-7.el8.noarch python3-execnet-1.7.1-1.el8.noarch 2024-02-15T18:15:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2-2.10.1-4.el8.noarch python3-remoto-1.2.1-1.el8.noarch 2024-02-15T18:15:34.710 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:34.710 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:15:34.859 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-fuse 2024-02-15T18:15:35.314 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2024-02-15T18:15:35.657 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:06:58 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:15:35.931 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 4/6 2024-02-15T18:15:36.199 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 5/6 2024-02-15T18:15:36.444 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:15:36.733 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2024-02-15T18:15:36.733 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 1/6 2024-02-15T18:15:36.733 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/6 2024-02-15T18:15:36.733 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/6 2024-02-15T18:15:36.733 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2024-02-15T18:15:36.733 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2024-02-15T18:15:37.156 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2024-02-15T18:15:37.156 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:37.156 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:15:37.156 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch python3-apipkg-1.5-6.el8.noarch 2024-02-15T18:15:37.156 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel-2.5.1-7.el8.noarch python3-execnet-1.7.1-1.el8.noarch 2024-02-15T18:15:37.156 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2-2.10.1-4.el8.noarch python3-remoto-1.2.1-1.el8.noarch 2024-02-15T18:15:37.156 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:37.156 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:15:37.308 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:15:37.308 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:15:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-02-15T18:15:37.309 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:15:37.309 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:15:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse x86_64 2:16.2.4-0.el8 ceph 885 k 2024-02-15T18:15:37.309 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-02-15T18:15:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2024-02-15T18:15:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:37.309 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:15:37.309 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:15:37.309 INFO:teuthology.orchestra.run.smithi042.stdout:Install 2 Packages 2024-02-15T18:15:37.309 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:37.310 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 968 k 2024-02-15T18:15:37.310 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 3.2 M 2024-02-15T18:15:37.310 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:15:37.311 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-fuse 2024-02-15T18:15:37.570 INFO:teuthology.orchestra.run.smithi042.stdout:(1/2): ceph-fuse-16.2.4-0.el8.x86_64.rpm 17 MB/s | 885 kB 00:00 2024-02-15T18:15:37.661 INFO:teuthology.orchestra.run.smithi042.stdout:(2/2): fuse-2.9.7-17.el8.x86_64.rpm 576 kB/s | 83 kB 00:00 2024-02-15T18:15:37.661 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:15:37.662 INFO:teuthology.orchestra.run.smithi042.stdout:Total 2.7 MB/s | 968 kB 00:00 2024-02-15T18:15:37.672 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:15:37.695 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:15:37.695 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:15:37.790 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:15:37.793 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:15:37.980 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:15:38.069 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:06:43 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:15:38.239 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-02-15T18:15:38.451 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2024-02-15T18:15:38.867 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2024-02-15T18:15:38.867 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-fuse-2:16.2.4-0.el8.x86_64 1/2 2024-02-15T18:15:39.291 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-02-15T18:15:39.291 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:39.291 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:15:39.291 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse-2:16.2.4-0.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-02-15T18:15:39.292 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:39.292 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:15:39.522 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librados-devel 2024-02-15T18:15:39.628 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:15:39.628 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:39.628 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-02-15T18:15:39.628 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:39.629 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:15:39.629 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse x86_64 2:16.2.4-0.el8 ceph 885 k 2024-02-15T18:15:39.629 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-02-15T18:15:39.629 INFO:teuthology.orchestra.run.smithi191.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2024-02-15T18:15:39.629 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:39.629 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:15:39.629 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:39.629 INFO:teuthology.orchestra.run.smithi191.stdout:Install 2 Packages 2024-02-15T18:15:39.629 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:39.630 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 968 k 2024-02-15T18:15:39.630 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 3.2 M 2024-02-15T18:15:39.630 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:15:39.951 INFO:teuthology.orchestra.run.smithi191.stdout:(1/2): ceph-fuse-16.2.4-0.el8.x86_64.rpm 14 MB/s | 885 kB 00:00 2024-02-15T18:15:40.043 INFO:teuthology.orchestra.run.smithi191.stdout:(2/2): fuse-2.9.7-17.el8.x86_64.rpm 536 kB/s | 83 kB 00:00 2024-02-15T18:15:40.043 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:15:40.043 INFO:teuthology.orchestra.run.smithi191.stdout:Total 2.3 MB/s | 968 kB 00:00 2024-02-15T18:15:40.054 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:15:40.075 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:15:40.075 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:15:40.162 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:15:40.165 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:15:40.323 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:07:03 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:15:40.334 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:15:40.619 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-02-15T18:15:40.881 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2024-02-15T18:15:41.613 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2024-02-15T18:15:41.614 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-fuse-2:16.2.4-0.el8.x86_64 1/2 2024-02-15T18:15:41.984 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:15:41.984 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:15:41.984 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-02-15T18:15:41.984 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:15:41.984 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:15:41.985 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel x86_64 2:16.2.4-0.el8 ceph 131 k 2024-02-15T18:15:41.985 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:41.985 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:15:41.985 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:15:41.985 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-02-15T18:15:41.985 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:41.985 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 131 k 2024-02-15T18:15:41.986 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 439 k 2024-02-15T18:15:41.986 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:15:42.032 INFO:teuthology.orchestra.run.smithi042.stdout:librados-devel-16.2.4-0.el8.x86_64.rpm 2.8 MB/s | 131 kB 00:00 2024-02-15T18:15:42.032 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:15:42.032 INFO:teuthology.orchestra.run.smithi042.stdout:Total 2.7 MB/s | 131 kB 00:00 2024-02-15T18:15:42.033 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:15:42.038 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:15:42.038 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:15:42.092 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:15:42.095 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:15:42.212 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:15:42.449 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:15:42.591 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-02-15T18:15:42.591 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:42.591 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:15:42.592 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse-2:16.2.4-0.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-02-15T18:15:42.592 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:42.592 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:15:42.759 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:15:42.855 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install librados-devel 2024-02-15T18:15:43.165 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:15:43.165 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:43.165 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:15:43.165 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel-2:16.2.4-0.el8.x86_64 2024-02-15T18:15:43.165 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:43.165 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:15:43.372 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install libcephfs2 2024-02-15T18:15:43.637 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:06:48 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:15:44.167 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:07:07 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:15:45.194 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:15:45.195 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:45.195 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-02-15T18:15:45.195 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:45.195 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:15:45.195 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel x86_64 2:16.2.4-0.el8 ceph 131 k 2024-02-15T18:15:45.196 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:45.196 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:15:45.196 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:45.196 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-02-15T18:15:45.196 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:45.196 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 131 k 2024-02-15T18:15:45.196 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 439 k 2024-02-15T18:15:45.196 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:15:45.233 INFO:teuthology.orchestra.run.smithi191.stdout:librados-devel-16.2.4-0.el8.x86_64.rpm 3.5 MB/s | 131 kB 00:00 2024-02-15T18:15:45.234 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:15:45.234 INFO:teuthology.orchestra.run.smithi191.stdout:Total 3.4 MB/s | 131 kB 00:00 2024-02-15T18:15:45.234 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:15:45.239 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:15:45.239 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:15:45.289 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:15:45.292 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:15:45.412 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:15:45.650 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:15:45.785 INFO:teuthology.orchestra.run.smithi042.stdout:Package libcephfs2-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:15:45.835 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:15:45.837 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-02-15T18:15:45.837 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:15:45.934 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install libcephfs-devel 2024-02-15T18:15:45.947 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:15:46.319 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:15:46.319 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:46.319 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:15:46.320 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel-2:16.2.4-0.el8.x86_64 2024-02-15T18:15:46.320 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:46.320 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:15:46.545 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install libcephfs2 2024-02-15T18:15:46.731 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:07:09 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:15:47.356 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:06:52 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:15:48.384 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:15:48.384 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:15:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-02-15T18:15:48.384 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:15:48.384 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:15:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel x86_64 2:16.2.4-0.el8 ceph 24 k 2024-02-15T18:15:48.384 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:48.384 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:15:48.385 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:15:48.385 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-02-15T18:15:48.385 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:48.385 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 24 k 2024-02-15T18:15:48.385 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 94 k 2024-02-15T18:15:48.385 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:15:48.427 INFO:teuthology.orchestra.run.smithi042.stdout:libcephfs-devel-16.2.4-0.el8.x86_64.rpm 592 kB/s | 24 kB 00:00 2024-02-15T18:15:48.427 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:15:48.427 INFO:teuthology.orchestra.run.smithi042.stdout:Total 575 kB/s | 24 kB 00:00 2024-02-15T18:15:48.428 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:15:48.431 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:15:48.431 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:15:48.433 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:15:48.437 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:15:48.483 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:15:48.703 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:15:48.845 INFO:teuthology.orchestra.run.smithi191.stdout:Package libcephfs2-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:15:48.891 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:15:48.893 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-02-15T18:15:48.893 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:15:48.964 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:15:48.980 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install libcephfs-devel 2024-02-15T18:15:49.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:15:49.339 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:49.339 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:15:49.339 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel-2:16.2.4-0.el8.x86_64 2024-02-15T18:15:49.339 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:15:49.340 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:15:49.457 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librados2 2024-02-15T18:15:49.739 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:06:54 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:15:50.249 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:07:13 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:15:51.302 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:15:51.303 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:51.303 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-02-15T18:15:51.303 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:51.303 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:15:51.304 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel x86_64 2:16.2.4-0.el8 ceph 24 k 2024-02-15T18:15:51.304 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:51.304 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:15:51.304 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:15:51.304 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-02-15T18:15:51.304 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:51.304 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 24 k 2024-02-15T18:15:51.304 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 94 k 2024-02-15T18:15:51.304 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:15:51.353 INFO:teuthology.orchestra.run.smithi191.stdout:libcephfs-devel-16.2.4-0.el8.x86_64.rpm 492 kB/s | 24 kB 00:00 2024-02-15T18:15:51.354 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:15:51.354 INFO:teuthology.orchestra.run.smithi191.stdout:Total 481 kB/s | 24 kB 00:00 2024-02-15T18:15:51.354 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:15:51.358 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:15:51.358 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:15:51.360 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:15:51.363 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:15:51.411 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:15:51.670 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:15:51.865 INFO:teuthology.orchestra.run.smithi042.stdout:Package librados2-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:15:51.915 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:15:51.917 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-02-15T18:15:51.917 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:15:51.934 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:15:52.148 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librbd1 2024-02-15T18:15:52.292 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:15:52.292 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:52.292 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:15:52.292 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel-2:16.2.4-0.el8.x86_64 2024-02-15T18:15:52.292 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:15:52.292 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:15:52.400 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install librados2 2024-02-15T18:15:52.945 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:07:15 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:15:53.157 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:06:58 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:15:54.556 INFO:teuthology.orchestra.run.smithi042.stdout:Package librbd1-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:15:54.606 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:15:54.607 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-02-15T18:15:54.607 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:15:54.665 INFO:teuthology.orchestra.run.smithi191.stdout:Package librados2-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:15:54.703 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rados 2024-02-15T18:15:54.712 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:15:54.713 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-02-15T18:15:54.713 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:15:54.800 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install librbd1 2024-02-15T18:15:55.496 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:07:18 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:15:55.554 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:07:00 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:15:57.068 INFO:teuthology.orchestra.run.smithi191.stdout:Package librbd1-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:15:57.114 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:15:57.116 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-02-15T18:15:57.116 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:15:57.119 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rados-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:15:57.169 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:15:57.171 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-02-15T18:15:57.171 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:15:57.214 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install python3-rados 2024-02-15T18:15:57.275 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rgw 2024-02-15T18:15:57.970 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:07:02 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:15:58.068 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:07:21 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:15:59.480 INFO:teuthology.orchestra.run.smithi191.stdout:Package python3-rados-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:15:59.527 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:15:59.528 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-02-15T18:15:59.528 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:15:59.621 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install python3-rgw 2024-02-15T18:15:59.677 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rgw-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:15:59.727 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:15:59.729 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-02-15T18:15:59.729 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:15:59.831 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-cephfs 2024-02-15T18:16:00.375 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:07:05 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:16:00.616 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:07:23 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:16:01.853 INFO:teuthology.orchestra.run.smithi191.stdout:Package python3-rgw-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:16:01.900 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:16:01.901 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-02-15T18:16:01.901 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:16:02.000 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install python3-cephfs 2024-02-15T18:16:02.306 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-cephfs-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:16:02.357 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:16:02.358 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-02-15T18:16:02.358 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:16:02.462 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rbd 2024-02-15T18:16:02.769 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:07:07 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:16:03.254 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:07:26 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:16:04.273 INFO:teuthology.orchestra.run.smithi191.stdout:Package python3-cephfs-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:16:04.319 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:16:04.320 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-02-15T18:16:04.320 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:16:04.422 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install python3-rbd 2024-02-15T18:16:04.863 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rbd-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:16:04.913 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:16:04.914 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-02-15T18:16:04.914 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:16:05.064 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-fuse 2024-02-15T18:16:05.180 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:07:10 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:16:05.867 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:07:28 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:16:06.685 INFO:teuthology.orchestra.run.smithi191.stdout:Package python3-rbd-2:16.2.4-0.el8.x86_64 is already installed. 2024-02-15T18:16:06.732 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:16:06.733 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2024-02-15T18:16:06.733 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:16:06.834 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install rbd-fuse 2024-02-15T18:16:07.524 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:16:07.526 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:16:07.526 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-02-15T18:16:07.526 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:16:07.526 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:16:07.526 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse x86_64 2:16.2.4-0.el8 ceph 89 k 2024-02-15T18:16:07.527 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:16:07.527 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:16:07.527 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:16:07.527 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-02-15T18:16:07.527 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:16:07.527 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 89 k 2024-02-15T18:16:07.528 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 298 k 2024-02-15T18:16:07.528 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:16:07.585 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:07:12 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:16:07.599 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-fuse-16.2.4-0.el8.x86_64.rpm 1.2 MB/s | 89 kB 00:00 2024-02-15T18:16:07.600 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:16:07.600 INFO:teuthology.orchestra.run.smithi042.stdout:Total 1.2 MB/s | 89 kB 00:00 2024-02-15T18:16:07.601 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:16:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:16:07.611 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:16:07.665 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:16:07.668 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:16:07.982 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:16:08.154 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:08.457 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:08.853 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:16:08.854 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:16:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse-2:16.2.4-0.el8.x86_64 2024-02-15T18:16:08.854 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:16:08.854 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:16:09.072 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-mirror 2024-02-15T18:16:09.139 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:16:09.140 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:16:09.140 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-02-15T18:16:09.140 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:16:09.140 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:16:09.140 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse x86_64 2:16.2.4-0.el8 ceph 89 k 2024-02-15T18:16:09.140 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:16:09.140 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:16:09.140 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:16:09.140 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-02-15T18:16:09.140 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:16:09.141 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 89 k 2024-02-15T18:16:09.141 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 298 k 2024-02-15T18:16:09.141 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:16:09.191 INFO:teuthology.orchestra.run.smithi191.stdout:rbd-fuse-16.2.4-0.el8.x86_64.rpm 1.8 MB/s | 89 kB 00:00 2024-02-15T18:16:09.191 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:16:09.192 INFO:teuthology.orchestra.run.smithi191.stdout:Total 1.7 MB/s | 89 kB 00:00 2024-02-15T18:16:09.192 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:16:09.202 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:16:09.202 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:16:09.251 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:16:09.253 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:16:09.361 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:16:09.562 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:09.849 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:09.867 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:07:32 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:16:10.234 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:10.234 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:16:10.234 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:16:10.234 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse-2:16.2.4-0.el8.x86_64 2024-02-15T18:16:10.234 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:16:10.234 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:16:10.423 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install rbd-mirror 2024-02-15T18:16:11.187 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:07:16 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:16:11.531 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:16:11.531 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:16:11.531 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-02-15T18:16:11.532 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:16:11.532 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:16:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror x86_64 2:16.2.4-0.el8 ceph 3.7 M 2024-02-15T18:16:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:16:11.532 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:16:11.532 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:16:11.532 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-02-15T18:16:11.532 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:16:11.533 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 3.7 M 2024-02-15T18:16:11.533 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 18 M 2024-02-15T18:16:11.533 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:16:11.604 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-mirror-16.2.4-0.el8.x86_64.rpm 52 MB/s | 3.7 MB 00:00 2024-02-15T18:16:11.605 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:16:11.605 INFO:teuthology.orchestra.run.smithi042.stdout:Total 51 MB/s | 3.7 MB 00:00 2024-02-15T18:16:11.605 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:16:11.618 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:16:11.618 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:16:11.685 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:16:11.688 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:16:12.187 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:16:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:12.743 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:16:12.743 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:16:12.743 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-02-15T18:16:12.743 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:16:12.743 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:16:12.743 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror x86_64 2:16.2.4-0.el8 ceph 3.7 M 2024-02-15T18:16:12.743 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:16:12.744 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:16:12.744 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:16:12.744 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-02-15T18:16:12.744 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:16:12.744 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 3.7 M 2024-02-15T18:16:12.744 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 18 M 2024-02-15T18:16:12.745 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:16:12.808 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:12.866 INFO:teuthology.orchestra.run.smithi191.stdout:rbd-mirror-16.2.4-0.el8.x86_64.rpm 31 MB/s | 3.7 MB 00:00 2024-02-15T18:16:12.866 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:16:12.867 INFO:teuthology.orchestra.run.smithi191.stdout:Total 30 MB/s | 3.7 MB 00:00 2024-02-15T18:16:12.867 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:16:12.880 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:16:12.881 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:16:12.941 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:16:12.944 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:16:13.222 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:13.222 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:16:13.222 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:16:13.222 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror-2:16.2.4-0.el8.x86_64 2024-02-15T18:16:13.222 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:16:13.223 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:16:13.403 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:16:13.483 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-nbd 2024-02-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:14.279 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:07:37 ago on Thu 15 Feb 2024 06:08:37 PM UTC. 2024-02-15T18:16:14.302 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:14.699 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:14.699 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:16:14.699 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:16:14.700 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror-2:16.2.4-0.el8.x86_64 2024-02-15T18:16:14.700 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:16:14.700 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:16:14.900 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install rbd-nbd 2024-02-15T18:16:15.667 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:07:20 ago on Thu 15 Feb 2024 06:08:55 PM UTC. 2024-02-15T18:16:15.936 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-02-15T18:16:15.937 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:16:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-02-15T18:16:15.937 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:16:15.937 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-02-15T18:16:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd x86_64 2:16.2.4-0.el8 ceph 173 k 2024-02-15T18:16:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:16:15.937 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-02-15T18:16:15.937 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-02-15T18:16:15.937 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-02-15T18:16:15.937 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:16:15.938 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 173 k 2024-02-15T18:16:15.938 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 649 k 2024-02-15T18:16:15.938 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-02-15T18:16:15.991 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-nbd-16.2.4-0.el8.x86_64.rpm 3.2 MB/s | 173 kB 00:00 2024-02-15T18:16:15.992 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:16:15.992 INFO:teuthology.orchestra.run.smithi042.stdout:Total 3.1 MB/s | 173 kB 00:00 2024-02-15T18:16:15.992 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-02-15T18:16:16.004 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-02-15T18:16:16.004 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-02-15T18:16:16.056 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-02-15T18:16:16.059 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-02-15T18:16:16.179 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-02-15T18:16:16.366 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:16.665 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:16:17.060 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-02-15T18:16:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd-2:16.2.4-0.el8.x86_64 2024-02-15T18:16:17.060 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:16:17.060 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-02-15T18:16:17.237 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-15T18:16:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:16:17.312 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-02-15T18:16:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:16:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-15T18:16:17.312 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd x86_64 2:16.2.4-0.el8 ceph 173 k 2024-02-15T18:16:17.312 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:16:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-15T18:16:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-15T18:16:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-02-15T18:16:17.312 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:16:17.313 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 173 k 2024-02-15T18:16:17.313 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 649 k 2024-02-15T18:16:17.313 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-15T18:16:17.313 INFO:teuthology.orchestra.run.smithi191.stdout:rbd-nbd-16.2.4-0.el8.x86_64.rpm 4.2 MB/s | 173 kB 00:00 2024-02-15T18:16:17.313 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-15T18:16:17.313 INFO:teuthology.orchestra.run.smithi191.stdout:Total 4.0 MB/s | 173 kB 00:00 2024-02-15T18:16:17.313 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-15T18:16:17.313 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-15T18:16:17.313 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-15T18:16:17.314 DEBUG:teuthology.parallel:result is None 2024-02-15T18:16:17.341 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-15T18:16:17.344 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-15T18:16:17.476 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-15T18:16:17.705 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:18.030 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:18.404 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2024-02-15T18:16:18.404 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:16:18.404 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-15T18:16:18.405 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd-2:16.2.4-0.el8.x86_64 2024-02-15T18:16:18.405 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:16:18.405 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-15T18:16:18.652 DEBUG:teuthology.parallel:result is None 2024-02-15T18:16:18.653 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-15T18:16:18.653 INFO:teuthology.packaging:ref: None 2024-02-15T18:16:18.653 INFO:teuthology.packaging:tag: v16.2.4 2024-02-15T18:16:18.653 INFO:teuthology.packaging:branch: None 2024-02-15T18:16:18.653 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:16:18.654 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2024-02-15T18:16:18.928 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-15T18:16:18.956 INFO:teuthology.orchestra.run.smithi042.stdout:16.2.4-0.el8 2024-02-15T18:16:18.956 INFO:teuthology.packaging:The installed version of ceph is 16.2.4-0.el8 2024-02-15T18:16:18.956 INFO:teuthology.task.install:The correct ceph version 16.2.4-0 is installed. 2024-02-15T18:16:18.959 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-15T18:16:18.959 INFO:teuthology.packaging:ref: None 2024-02-15T18:16:18.959 INFO:teuthology.packaging:tag: v16.2.4 2024-02-15T18:16:18.959 INFO:teuthology.packaging:branch: None 2024-02-15T18:16:18.959 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:16:18.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2024-02-15T18:16:19.098 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-15T18:16:19.121 INFO:teuthology.orchestra.run.smithi191.stdout:16.2.4-0.el8 2024-02-15T18:16:19.121 INFO:teuthology.packaging:The installed version of ceph is 16.2.4-0.el8 2024-02-15T18:16:19.122 INFO:teuthology.task.install:The correct ceph version 16.2.4-0 is installed. 2024-02-15T18:16:19.123 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-15T18:16:19.124 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:16:19.124 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-15T18:16:19.158 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-15T18:16:19.158 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-15T18:16:19.189 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-15T18:16:19.189 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:16:19.189 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-02-15T18:16:19.231 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-15T18:16:19.302 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-15T18:16:19.303 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2024-02-15T18:16:19.329 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-15T18:16:19.396 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-15T18:16:19.397 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:16:19.397 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-15T18:16:19.431 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-15T18:16:19.501 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-15T18:16:19.502 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-15T18:16:19.529 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-15T18:16:19.596 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-15T18:16:19.596 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:16:19.596 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-02-15T18:16:19.629 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-15T18:16:19.702 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-15T18:16:19.703 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/stdin-killer 2024-02-15T18:16:19.730 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-15T18:16:19.797 INFO:teuthology.run_tasks:Running task print... 2024-02-15T18:16:19.807 INFO:teuthology.task.print:**** done install task... 2024-02-15T18:16:19.807 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-15T18:16:19.913 INFO:tasks.cephadm:Config: {'avoid_pacific_features': True, 'cephadm_branch': 'v16.2.4', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'image': 'docker.io/ceph/ceph:v16.2.4', 'roleless': True, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(PG_DEGRADED\\)', 'Degraded data redundancy', '\\(PG_', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1'} 2024-02-15T18:16:19.913 INFO:tasks.cephadm:Cluster image is docker.io/ceph/ceph:v16.2.4 2024-02-15T18:16:19.913 INFO:tasks.cephadm:Cluster fsid is 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:16:19.914 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-15T18:16:19.914 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-02-15T18:16:19.914 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi042': '172.21.15.42', 'mon.smithi191': '172.21.15.191'} 2024-02-15T18:16:19.914 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-15T18:16:19.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo hostname $(hostname -s) 2024-02-15T18:16:19.949 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname $(hostname -s) 2024-02-15T18:16:19.977 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref v16.2.4)... 2024-02-15T18:16:19.978 DEBUG:teuthology.orchestra.run.smithi042:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v16.2.4/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-15T18:16:20.132 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-rw-r--. 1 ubuntu ubuntu 292005 Feb 15 18:16 /home/ubuntu/cephtest/cephadm 2024-02-15T18:16:20.132 DEBUG:teuthology.orchestra.run.smithi191:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v16.2.4/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-15T18:16:20.324 INFO:teuthology.orchestra.run.smithi191.stdout:-rw-rw-r--. 1 ubuntu ubuntu 292005 Feb 15 18:16 /home/ubuntu/cephtest/cephadm 2024-02-15T18:16:20.325 DEBUG:teuthology.orchestra.run.smithi042:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-15T18:16:20.349 DEBUG:teuthology.orchestra.run.smithi191:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-15T18:16:20.379 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph 2024-02-15T18:16:20.422 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph 2024-02-15T18:16:20.449 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /etc/ceph 2024-02-15T18:16:20.493 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /etc/ceph 2024-02-15T18:16:20.519 INFO:tasks.cephadm:Writing seed config... 2024-02-15T18:16:20.521 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-15T18:16:20.521 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-15T18:16:20.521 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-15T18:16:20.521 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-15T18:16:20.521 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-15T18:16:20.521 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-02-15T18:16:20.521 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-15T18:16:20.521 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-15T18:16:20.521 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-15T18:16:20.521 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-15T18:16:20.521 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-15T18:16:20.521 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-15T18:16:20.521 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-15T18:16:20.522 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-15T18:16:20.522 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-15T18:16:20.522 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-15T18:16:20.522 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-15T18:16:20.522 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-15T18:16:20.522 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-15T18:16:20.522 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-15T18:16:20.522 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-15T18:16:20.522 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-15T18:16:20.522 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-02-15T18:16:20.522 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-02-15T18:16:20.522 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-02-15T18:16:20.522 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-15T18:16:20.522 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-15T18:16:20.523 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-02-15T18:16:20.523 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-02-15T18:16:20.523 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-15T18:16:20.523 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-15T18:16:20.523 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-02-15T18:16:20.523 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-15T18:16:20.523 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-15T18:16:20.523 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-15T18:16:20.523 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-15T18:16:20.523 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-15T18:16:20.523 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-15T18:16:20.524 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:16:20.524 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-15T18:16:20.554 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 51458a44-cc2e-11ee-95ba-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-15T18:16:20.554 DEBUG:teuthology.orchestra.run.smithi042:mon.smithi042> sudo journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi042.service 2024-02-15T18:16:20.596 INFO:tasks.cephadm:Bootstrapping... 2024-02-15T18:16:20.597 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 -v bootstrap --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.42 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-15T18:16:20.627 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:-- Logs begin at Thu 2024-02-15 17:54:32 UTC. -- 2024-02-15T18:16:20.892 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman version --format {{.Client.Version}} 2024-02-15T18:16:21.883 INFO:teuthology.orchestra.run.smithi042.stderr:/bin/podman: 4.9.3-dev 2024-02-15T18:16:21.893 INFO:teuthology.orchestra.run.smithi042.stderr:Verifying podman|docker is present... 2024-02-15T18:16:21.894 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman version --format {{.Client.Version}} 2024-02-15T18:16:21.918 INFO:teuthology.orchestra.run.smithi042.stderr:/bin/podman: 4.9.3-dev 2024-02-15T18:16:21.932 INFO:teuthology.orchestra.run.smithi042.stderr:Verifying lvm2 is present... 2024-02-15T18:16:21.932 INFO:teuthology.orchestra.run.smithi042.stderr:Verifying time synchronization is in place... 2024-02-15T18:16:21.932 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-enabled chrony.service 2024-02-15T18:16:21.939 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-15T18:16:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-active chrony.service 2024-02-15T18:16:21.946 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: inactive 2024-02-15T18:16:21.947 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-enabled chronyd.service 2024-02-15T18:16:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: enabled 2024-02-15T18:16:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-active chronyd.service 2024-02-15T18:16:21.959 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: active 2024-02-15T18:16:21.960 INFO:teuthology.orchestra.run.smithi042.stderr:Unit chronyd.service is enabled and running 2024-02-15T18:16:21.960 INFO:teuthology.orchestra.run.smithi042.stderr:Repeating the final host check... 2024-02-15T18:16:21.960 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman version --format {{.Client.Version}} 2024-02-15T18:16:21.981 INFO:teuthology.orchestra.run.smithi042.stderr:/bin/podman: 4.9.3-dev 2024-02-15T18:16:21.993 INFO:teuthology.orchestra.run.smithi042.stderr:podman|docker (/bin/podman) is present 2024-02-15T18:16:21.994 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl is present 2024-02-15T18:16:21.994 INFO:teuthology.orchestra.run.smithi042.stderr:lvcreate is present 2024-02-15T18:16:21.994 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-enabled chrony.service 2024-02-15T18:16:21.999 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-15T18:16:22.000 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-active chrony.service 2024-02-15T18:16:22.005 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: inactive 2024-02-15T18:16:22.006 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-enabled chronyd.service 2024-02-15T18:16:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: enabled 2024-02-15T18:16:22.012 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-active chronyd.service 2024-02-15T18:16:22.017 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: active 2024-02-15T18:16:22.018 INFO:teuthology.orchestra.run.smithi042.stderr:Unit chronyd.service is enabled and running 2024-02-15T18:16:22.018 INFO:teuthology.orchestra.run.smithi042.stderr:Host looks OK 2024-02-15T18:16:22.018 INFO:teuthology.orchestra.run.smithi042.stderr:Cluster fsid: 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:16:22.018 INFO:teuthology.orchestra.run.smithi042.stderr:Acquiring lock 139730620135912 on /run/cephadm/51458a44-cc2e-11ee-95ba-87774f69a715.lock 2024-02-15T18:16:22.019 INFO:teuthology.orchestra.run.smithi042.stderr:Lock 139730620135912 acquired on /run/cephadm/51458a44-cc2e-11ee-95ba-87774f69a715.lock 2024-02-15T18:16:22.019 INFO:teuthology.orchestra.run.smithi042.stderr:Verifying IP 172.21.15.42 port 3300 ... 2024-02-15T18:16:22.020 INFO:teuthology.orchestra.run.smithi042.stderr:Verifying IP 172.21.15.42 port 6789 ... 2024-02-15T18:16:22.020 INFO:teuthology.orchestra.run.smithi042.stderr:Base mon IP is 172.21.15.42, final addrv is [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-02-15T18:16:22.020 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /sbin/ip route ls 2024-02-15T18:16:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:/sbin/ip: default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.42 metric 100 2024-02-15T18:16:22.022 INFO:teuthology.orchestra.run.smithi042.stderr:/sbin/ip: 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.42 metric 100 2024-02-15T18:16:22.023 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /sbin/ip -6 route ls 2024-02-15T18:16:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:/sbin/ip: ::1 dev lo proto kernel metric 256 pref medium 2024-02-15T18:16:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:/sbin/ip: fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-02-15T18:16:22.025 INFO:teuthology.orchestra.run.smithi042.stderr:/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1758sec hoplimit 64 pref medium 2024-02-15T18:16:22.026 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /sbin/ip -6 addr ls 2024-02-15T18:16:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-15T18:16:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:/sbin/ip: inet6 ::1/128 scope host 2024-02-15T18:16:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-15T18:16:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:/sbin/ip: 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-15T18:16:22.028 INFO:teuthology.orchestra.run.smithi042.stderr:/sbin/ip: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2024-02-15T18:16:22.029 INFO:teuthology.orchestra.run.smithi042.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-15T18:16:22.029 INFO:teuthology.orchestra.run.smithi042.stderr:Mon IP 172.21.15.42 is in CIDR network 172.21.0.0/20 2024-02-15T18:16:22.029 INFO:teuthology.orchestra.run.smithi042.stderr:- internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-15T18:16:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:Pulling container image docker.io/ceph/ceph:v16.2.4... 2024-02-15T18:16:22.031 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman pull docker.io/ceph/ceph:v16.2.4 2024-02-15T18:16:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:/bin/podman: Trying to pull docker.io/ceph/ceph:v16.2.4... 2024-02-15T18:16:28.727 INFO:teuthology.orchestra.run.smithi042.stderr:/bin/podman: Getting image source signatures 2024-02-15T18:16:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:/bin/podman: Copying blob sha256:551101690791055467b1840be133b2e6b029d263771bc6c46a96bb7acdacac39 2024-02-15T18:16:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:/bin/podman: Copying blob sha256:7a0437f04f83f084b7ed68ad9c4a4947e12fc4e1b006b38129bac89114ec3621 2024-02-15T18:16:43.896 INFO:teuthology.orchestra.run.smithi042.stderr:/bin/podman: Copying config sha256:8d91d370c2b86c07de46146aba8d36718eaefa69b1880c77fa312fda6efd7d29 2024-02-15T18:16:44.054 INFO:teuthology.orchestra.run.smithi042.stderr:/bin/podman: Writing manifest to image destination 2024-02-15T18:16:44.325 INFO:teuthology.orchestra.run.smithi042.stderr:/bin/podman: 8d91d370c2b86c07de46146aba8d36718eaefa69b1880c77fa312fda6efd7d29 2024-02-15T18:16:44.337 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --version 2024-02-15T18:16:45.083 INFO:teuthology.orchestra.run.smithi042.stderr:ceph: ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable) 2024-02-15T18:16:45.716 INFO:teuthology.orchestra.run.smithi042.stderr:Ceph version: ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable) 2024-02-15T18:16:45.716 INFO:teuthology.orchestra.run.smithi042.stderr:Extracting ceph user uid/gid from container image... 2024-02-15T18:16:45.716 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 -c %u %g /var/lib/ceph 2024-02-15T18:16:46.241 INFO:teuthology.orchestra.run.smithi042.stderr:stat: 167 167 2024-02-15T18:16:46.761 INFO:teuthology.orchestra.run.smithi042.stderr:Creating initial keys... 2024-02-15T18:16:46.762 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --gen-print-key 2024-02-15T18:16:47.307 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-authtool: AQAPVc5lIEAaDRAArO7BBpF3UZ8XuBVev+mXLg== 2024-02-15T18:16:47.893 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --gen-print-key 2024-02-15T18:16:48.412 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-authtool: AQAQVc5l24BjERAAvsGt4CfLG3tXY3apTZpxkQ== 2024-02-15T18:16:48.976 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --gen-print-key 2024-02-15T18:16:49.584 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-authtool: AQARVc5lEFa3HRAAmj47e+Af0uY1wxHfr+UaGA== 2024-02-15T18:16:50.138 INFO:teuthology.orchestra.run.smithi042.stderr:Creating initial monmap... 2024-02-15T18:16:50.138 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/monmaptool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /tmp/ceph-tmphj50zvcn:/tmp/monmap:z docker.io/ceph/ceph:v16.2.4 --create --clobber --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 --addv smithi042 [v2:172.21.15.42:3300,v1:172.21.15.42:6789] /tmp/monmap 2024-02-15T18:16:50.674 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-15T18:16:50.674 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:16:50.674 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-15T18:16:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:monmaptool for smithi042 [v2:172.21.15.42:3300,v1:172.21.15.42:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-15T18:16:51.211 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/monmaptool: set fsid to 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:16:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-15T18:16:51.212 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-02-15T18:16:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:Creating mon... 2024-02-15T18:16:51.212 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-mon --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042:/var/lib/ceph/mon/ceph-smithi042:z -v /tmp/ceph-tmpx_ifhxid:/tmp/keyring:z -v /tmp/ceph-tmphj50zvcn:/tmp/monmap:z docker.io/ceph/ceph:v16.2.4 --mkfs -i smithi042 --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -c /dev/null --monmap /tmp/monmap --keyring /tmp/keyring --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true 2024-02-15T18:16:51.815 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.719+0000 7f2afbf0c700 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-15T18:16:51.815 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.719+0000 7f2afbf0c700 1 imported monmap: 2024-02-15T18:16:51.815 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: epoch 0 2024-02-15T18:16:51.815 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: fsid 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:16:51.815 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: last_changed 2024-02-15T18:16:50.575606+0000 2024-02-15T18:16:51.815 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: created 2024-02-15T18:16:50.575606+0000 2024-02-15T18:16:51.815 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: min_mon_release 0 (unknown) 2024-02-15T18:16:51.815 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: election_strategy: 1 2024-02-15T18:16:51.816 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.smithi042 2024-02-15T18:16:51.816 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.816 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.719+0000 7f2afbf0c700 0 /usr/bin/ceph-mon: set fsid to 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:16:51.816 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: RocksDB version: 6.8.1 2024-02-15T18:16:51.816 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.816 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-15T18:16:51.816 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Compile date May 13 2021 2024-02-15T18:16:51.838 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: DB SUMMARY 2024-02-15T18:16:51.838 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.838 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi042/store.db dir, Total Num: 0, files: 2024-02-15T18:16:51.838 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.838 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi042/store.db: 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.error_if_exists: 0 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.create_if_missing: 1 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.paranoid_checks: 1 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.env: 0x55c7435dc1c0 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.fs: Posix File System 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.info_log: 0x55c7452acec0 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.statistics: (nil) 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.use_fsync: 0 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.max_log_file_size: 0 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-15T18:16:51.839 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.allow_fallocate: 1 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.use_direct_reads: 0 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.create_missing_column_families: 0 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.db_log_dir: 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi042/store.db 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.max_subcompactions: 1 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.max_background_flushes: -1 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-15T18:16:51.840 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.advise_random_on_open: 1 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.write_buffer_manager: 0x55c7452b3530 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.rate_limiter: (nil) 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.unordered_write: 0 2024-02-15T18:16:51.841 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.row_cache: None 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.wal_filter: None 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.preserve_deletes: 0 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.two_write_queues: 0 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.manual_wal_flush: 0 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.atomic_flush: 0 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-15T18:16:51.842 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.log_readahead_size: 0 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.max_background_jobs: 2 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.max_background_compactions: -1 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.max_total_wal_size: 0 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.max_open_files: -1 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.bytes_per_sync: 0 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-15T18:16:51.843 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-15T18:16:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-15T18:16:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Compression algorithms supported: 2024-02-15T18:16:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-15T18:16:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: kZSTD supported: 0 2024-02-15T18:16:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: kXpressCompression supported: 0 2024-02-15T18:16:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-15T18:16:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: kLZ4Compression supported: 1 2024-02-15T18:16:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: kBZip2Compression supported: 0 2024-02-15T18:16:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: kZlibCompression supported: 1 2024-02-15T18:16:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: kSnappyCompression supported: 1 2024-02-15T18:16:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.720+0000 7f2afbf0c700 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-15T18:16:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: [db_impl/db_impl_open.cc:273] Creating manifest 1 2024-02-15T18:16:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.844 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi042/store.db/MANIFEST-000001 2024-02-15T18:16:51.845 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.845 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-15T18:16:51.845 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.845 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-15T18:16:51.845 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.merge_operator: 2024-02-15T18:16:51.845 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compaction_filter: None 2024-02-15T18:16:51.845 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compaction_filter_factory: None 2024-02-15T18:16:51.845 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-15T18:16:51.845 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-15T18:16:51.845 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c7451c5cb8) 2024-02-15T18:16:51.845 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks: 1 2024-02-15T18:16:51.845 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-15T18:16:51.845 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-15T18:16:51.845 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: pin_top_level_index_and_filter: 1 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: index_type: 0 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: data_block_index_type: 0 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: index_shortening: 1 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: data_block_hash_table_util_ratio: 0.750000 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: hash_index_allow_collision: 1 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: checksum: 1 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: no_block_cache: 0 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: block_cache: 0x55c7451fad10 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: block_cache_name: BinnedLRUCache 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: block_cache_options: 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: capacity : 536870912 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: num_shard_bits : 4 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: strict_capacity_limit : 0 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: high_pri_pool_ratio: 0.000 2024-02-15T18:16:51.846 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: block_cache_compressed: (nil) 2024-02-15T18:16:51.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: persistent_cache: (nil) 2024-02-15T18:16:51.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: block_size: 4096 2024-02-15T18:16:51.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: block_size_deviation: 10 2024-02-15T18:16:51.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: block_restart_interval: 16 2024-02-15T18:16:51.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: index_block_restart_interval: 1 2024-02-15T18:16:51.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: metadata_block_size: 4096 2024-02-15T18:16:51.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: partition_filters: 0 2024-02-15T18:16:51.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: use_delta_encoding: 1 2024-02-15T18:16:51.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-15T18:16:51.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: whole_key_filtering: 1 2024-02-15T18:16:51.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: verify_compression: 0 2024-02-15T18:16:51.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: read_amp_bytes_per_bit: 0 2024-02-15T18:16:51.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: format_version: 2 2024-02-15T18:16:51.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: enable_index_compression: 1 2024-02-15T18:16:51.848 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: block_align: 0 2024-02-15T18:16:51.848 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.848 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-15T18:16:51.848 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-15T18:16:51.848 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compression: NoCompression 2024-02-15T18:16:51.848 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-15T18:16:51.848 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-15T18:16:51.848 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-15T18:16:51.848 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.num_levels: 7 2024-02-15T18:16:51.848 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-15T18:16:51.848 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-15T18:16:51.848 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-15T18:16:51.848 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-15T18:16:51.848 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-15T18:16:51.849 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-15T18:16:51.849 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-15T18:16:51.849 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:16:51.849 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-15T18:16:51.849 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-15T18:16:51.849 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compression_opts.level: 32767 2024-02-15T18:16:51.849 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-15T18:16:51.849 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-15T18:16:51.849 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:16:51.849 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compression_opts.enabled: false 2024-02-15T18:16:51.849 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-15T18:16:51.849 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-15T18:16:51.849 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-15T18:16:51.849 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.arena_block_size: 4194304 2024-02-15T18:16:51.850 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.table_properties_collectors: 2024-02-15T18:16:51.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.inplace_update_support: 0 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.bloom_locality: 0 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.max_successive_merges: 0 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.force_consistency_checks: 0 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.ttl: 2592000 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.721+0000 7f2afbf0c700 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.722+0000 7f2afbf0c700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi042/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.722+0000 7f2afbf0c700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-15T18:16:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.722+0000 7f2afbf0c700 4 rocksdb: DB pointer 0x55c7452bf800 2024-02-15T18:16:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.723+0000 7f2ae4da5700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-15T18:16:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.723+0000 7f2ae4da5700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-15T18:16:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: ** DB Stats ** 2024-02-15T18:16:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:16:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-15T18:16:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-15T18:16:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:16:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-15T18:16:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-15T18:16:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:16:51.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-15T18:16:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:16:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:16:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:16:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:16:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-15T18:16:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:16:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:16:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:16:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-15T18:16:51.858 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:16:51.859 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:16:51.860 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: 2024-02-15T18:16:51.861 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.724+0000 7f2afbf0c700 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work 2024-02-15T18:16:51.861 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.724+0000 7f2afbf0c700 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete 2024-02-15T18:16:51.861 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph-mon: debug 2024-02-15T18:16:51.724+0000 7f2afbf0c700 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-smithi042 for mon.smithi042 2024-02-15T18:16:52.383 INFO:teuthology.orchestra.run.smithi042.stderr:create mon.smithi042 on 2024-02-15T18:16:52.384 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: install -d -m0770 -o 167 -g 167 /var/run/ceph/51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:16:52.390 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl disable ceph.target 2024-02-15T18:16:52.398 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-02-15T18:16:52.490 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl enable ceph.target 2024-02-15T18:16:52.498 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-15T18:16:52.586 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl start ceph.target 2024-02-15T18:16:52.593 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl enable ceph-51458a44-cc2e-11ee-95ba-87774f69a715.target 2024-02-15T18:16:52.599 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-51458a44-cc2e-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-51458a44-cc2e-11ee-95ba-87774f69a715.target. 2024-02-15T18:16:52.599 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-51458a44-cc2e-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-51458a44-cc2e-11ee-95ba-87774f69a715.target. 2024-02-15T18:16:52.682 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl start ceph-51458a44-cc2e-11ee-95ba-87774f69a715.target 2024-02-15T18:16:52.690 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl daemon-reload 2024-02-15T18:16:52.782 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl stop ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi042 2024-02-15T18:16:52.810 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl reset-failed ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi042 2024-02-15T18:16:52.816 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: Failed to reset failed state of unit ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi042.service: Unit ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi042.service not loaded. 2024-02-15T18:16:52.816 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl enable ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi042 2024-02-15T18:16:52.823 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: Created symlink /etc/systemd/system/ceph-51458a44-cc2e-11ee-95ba-87774f69a715.target.wants/ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi042.service → /etc/systemd/system/ceph-51458a44-cc2e-11ee-95ba-87774f69a715@.service. 2024-02-15T18:16:52.911 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl start ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi042 2024-02-15T18:16:53.450 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-enabled firewalld.service 2024-02-15T18:16:53.459 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: disabled 2024-02-15T18:16:53.459 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-active firewalld.service 2024-02-15T18:16:53.467 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: inactive 2024-02-15T18:16:53.468 INFO:teuthology.orchestra.run.smithi042.stderr:firewalld.service is not enabled 2024-02-15T18:16:53.468 INFO:teuthology.orchestra.run.smithi042.stderr:Not possible to enable service . firewalld.service is not available 2024-02-15T18:16:53.468 INFO:teuthology.orchestra.run.smithi042.stderr:Not possible to open ports <[]>. firewalld.service is not available 2024-02-15T18:16:53.469 INFO:teuthology.orchestra.run.smithi042.stderr:Waiting for mon to start... 2024-02-15T18:16:53.469 INFO:teuthology.orchestra.run.smithi042.stderr:Waiting for mon... 2024-02-15T18:16:53.469 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042:/var/lib/ceph/mon/ceph-smithi042:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status 2024-02-15T18:16:53.671 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:53 smithi042 bash[37816]: 8505e5a18c43211d382436f2d2e77dcaefb4c9972bc6e14e2ca491f3ca643e89 2024-02-15T18:16:53.671 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:53 smithi042 systemd[1]: Started Ceph mon.smithi042 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:16:53.941 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.940+0000 7f9a3a063700 1 Processor -- start 2024-02-15T18:16:53.941 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.940+0000 7f9a3a063700 1 -- start start 2024-02-15T18:16:53.941 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.940+0000 7f9a3a063700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a340fabe0 0x7f9a340fb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:16:53.941 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.940+0000 7f9a3a063700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a340fb4f0 con 0x7f9a340fabe0 2024-02-15T18:16:53.941 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.940+0000 7f9a39061700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a340fabe0 0x7f9a340fb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:16:53.944 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.940+0000 7f9a39061700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a340fabe0 0x7f9a340fb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58040/0 (socket says 172.21.15.42:58040) 2024-02-15T18:16:53.944 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.940+0000 7f9a39061700 1 -- 172.21.15.42:0/3676331283 learned_addr learned my addr 172.21.15.42:0/3676331283 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:16:53.944 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.941+0000 7f9a39061700 1 -- 172.21.15.42:0/3676331283 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a340fb630 con 0x7f9a340fabe0 2024-02-15T18:16:53.944 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.941+0000 7f9a39061700 1 --2- 172.21.15.42:0/3676331283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a340fabe0 0x7f9a340fb000 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f9a3000b5f0 tx=0x7f9a30006ee0).ready entity=mon.0 client_cookie=3b011bfa1fa405bc server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:16:53.944 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.941+0000 7f9a23fff700 1 -- 172.21.15.42:0/3676331283 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a3000d600 con 0x7f9a340fabe0 2024-02-15T18:16:53.944 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.941+0000 7f9a23fff700 1 -- 172.21.15.42:0/3676331283 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3000d760 con 0x7f9a340fabe0 2024-02-15T18:16:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.941+0000 7f9a23fff700 1 -- 172.21.15.42:0/3676331283 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a3000da30 con 0x7f9a340fabe0 2024-02-15T18:16:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.942+0000 7f9a3a063700 1 -- 172.21.15.42:0/3676331283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a340fabe0 msgr2=0x7f9a340fb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:16:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.942+0000 7f9a3a063700 1 --2- 172.21.15.42:0/3676331283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a340fabe0 0x7f9a340fb000 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f9a3000b5f0 tx=0x7f9a30006ee0).stop 2024-02-15T18:16:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.942+0000 7f9a3a063700 1 -- 172.21.15.42:0/3676331283 shutdown_connections 2024-02-15T18:16:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.942+0000 7f9a3a063700 1 --2- 172.21.15.42:0/3676331283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a340fabe0 0x7f9a340fb000 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:16:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.942+0000 7f9a3a063700 1 -- 172.21.15.42:0/3676331283 >> 172.21.15.42:0/3676331283 conn(0x7f9a340f6420 msgr2=0x7f9a340f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:16:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.942+0000 7f9a3a063700 1 -- 172.21.15.42:0/3676331283 shutdown_connections 2024-02-15T18:16:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.942+0000 7f9a3a063700 1 -- 172.21.15.42:0/3676331283 wait complete. 2024-02-15T18:16:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.942+0000 7f9a3a063700 1 Processor -- start 2024-02-15T18:16:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.942+0000 7f9a3a063700 1 -- start start 2024-02-15T18:16:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.942+0000 7f9a3a063700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a340fabe0 0x7f9a34107f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:16:53.945 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.942+0000 7f9a3a063700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a34108450 con 0x7f9a340fabe0 2024-02-15T18:16:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.943+0000 7f9a39061700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a340fabe0 0x7f9a34107f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:16:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.943+0000 7f9a39061700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a340fabe0 0x7f9a34107f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58048/0 (socket says 172.21.15.42:58048) 2024-02-15T18:16:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.943+0000 7f9a39061700 1 -- 172.21.15.42:0/1164048470 learned_addr learned my addr 172.21.15.42:0/1164048470 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:16:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.943+0000 7f9a39061700 1 -- 172.21.15.42:0/1164048470 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3000b200 con 0x7f9a340fabe0 2024-02-15T18:16:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.943+0000 7f9a39061700 1 --2- 172.21.15.42:0/1164048470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a340fabe0 0x7f9a34107f60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f9a30006c70 tx=0x7f9a30016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:16:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.943+0000 7f9a21ffb700 1 -- 172.21.15.42:0/1164048470 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a30011b40 con 0x7f9a340fabe0 2024-02-15T18:16:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.943+0000 7f9a21ffb700 1 -- 172.21.15.42:0/1164048470 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a30011ca0 con 0x7f9a340fabe0 2024-02-15T18:16:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.943+0000 7f9a3a063700 1 -- 172.21.15.42:0/1164048470 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a34108650 con 0x7f9a340fabe0 2024-02-15T18:16:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.943+0000 7f9a21ffb700 1 -- 172.21.15.42:0/1164048470 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a3001d430 con 0x7f9a340fabe0 2024-02-15T18:16:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.944+0000 7f9a3a063700 1 -- 172.21.15.42:0/1164048470 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a34108ae0 con 0x7f9a340fabe0 2024-02-15T18:16:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.944+0000 7f9a21ffb700 1 -- 172.21.15.42:0/1164048470 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f9a3000f070 con 0x7f9a340fabe0 2024-02-15T18:16:53.946 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.944+0000 7f9a21ffb700 1 -- 172.21.15.42:0/1164048470 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f9a30027460 con 0x7f9a340fabe0 2024-02-15T18:16:53.947 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.944+0000 7f9a3a063700 1 -- 172.21.15.42:0/1164048470 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a34045ad0 con 0x7f9a340fabe0 2024-02-15T18:16:53.947 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:53.947+0000 7f9a21ffb700 1 -- 172.21.15.42:0/1164048470 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f9a30016070 con 0x7f9a340fabe0 2024-02-15T18:16:54.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:54.017+0000 7f9a3a063700 1 -- 172.21.15.42:0/1164048470 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f9a34101930 con 0x7f9a340fabe0 2024-02-15T18:16:54.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:54.017+0000 7f9a21ffb700 1 -- 172.21.15.42:0/1164048470 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f9a30035160 con 0x7f9a340fabe0 2024-02-15T18:16:54.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: cluster: 2024-02-15T18:16:54.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: id: 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:16:54.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: health: HEALTH_OK 2024-02-15T18:16:54.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:54.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: services: 2024-02-15T18:16:54.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: mon: 1 daemons, quorum smithi042 (age 0.612218s) 2024-02-15T18:16:54.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: mgr: no daemons active 2024-02-15T18:16:54.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2024-02-15T18:16:54.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:54.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: data: 2024-02-15T18:16:54.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2024-02-15T18:16:54.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2024-02-15T18:16:54.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2024-02-15T18:16:54.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: pgs: 2024-02-15T18:16:54.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:54.022 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:54.018+0000 7f9a3a063700 1 -- 172.21.15.42:0/1164048470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a340fabe0 msgr2=0x7f9a34107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:16:54.022 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:54.018+0000 7f9a3a063700 1 --2- 172.21.15.42:0/1164048470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a340fabe0 0x7f9a34107f60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f9a30006c70 tx=0x7f9a30016000).stop 2024-02-15T18:16:54.022 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:54.018+0000 7f9a3a063700 1 -- 172.21.15.42:0/1164048470 shutdown_connections 2024-02-15T18:16:54.022 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:54.018+0000 7f9a3a063700 1 --2- 172.21.15.42:0/1164048470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a340fabe0 0x7f9a34107f60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:16:54.022 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:54.018+0000 7f9a3a063700 1 -- 172.21.15.42:0/1164048470 >> 172.21.15.42:0/1164048470 conn(0x7f9a340f6420 msgr2=0x7f9a340f85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:16:54.022 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:54.018+0000 7f9a3a063700 1 -- 172.21.15.42:0/1164048470 shutdown_connections 2024-02-15T18:16:54.022 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:54.018+0000 7f9a3a063700 1 -- 172.21.15.42:0/1164048470 wait complete. 2024-02-15T18:16:54.622 INFO:teuthology.orchestra.run.smithi042.stderr:mon is available 2024-02-15T18:16:54.622 INFO:teuthology.orchestra.run.smithi042.stderr:Assimilating anything we can from ceph.conf... 2024-02-15T18:16:54.623 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z -v /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042:/var/lib/ceph/mon/ceph-smithi042:z docker.io/ceph/ceph:v16.2.4 config assimilate-conf -i /var/lib/ceph/mon/ceph-smithi042/config 2024-02-15T18:16:54.625 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[37831]: cluster 2024-02-15T18:16:53.406334+0000 mon.smithi042 (mon.0) 2 : cluster [INF] mon.smithi042 is new leader, mons smithi042 in quorum (ranks 0) 2024-02-15T18:16:54.625 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[37831]: cluster 2024-02-15T18:16:53.406456+0000 mon.smithi042 (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {smithi042=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0]} 2024-02-15T18:16:54.625 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[37831]: cluster 2024-02-15T18:16:53.407843+0000 mon.smithi042 (mon.0) 4 : cluster [DBG] fsmap 2024-02-15T18:16:54.625 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[37831]: cluster 2024-02-15T18:16:53.409731+0000 mon.smithi042 (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-15T18:16:54.625 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[37831]: cluster 2024-02-15T18:16:53.410090+0000 mon.smithi042 (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2024-02-15T18:16:54.625 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[37831]: audit 2024-02-15T18:16:54.018515+0000 mon.smithi042 (mon.0) 7 : audit [DBG] from='client.? 172.21.15.42:0/1164048470' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-15T18:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.141+0000 7ff0cdd57700 1 Processor -- start 2024-02-15T18:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.141+0000 7ff0cdd57700 1 -- start start 2024-02-15T18:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.142+0000 7ff0cdd57700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0c80703d0 0x7ff0c80feab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.142+0000 7ff0cdd57700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80fefa0 con 0x7ff0c80703d0 2024-02-15T18:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.142+0000 7ff0c77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0c80703d0 0x7ff0c80feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.142+0000 7ff0c77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0c80703d0 0x7ff0c80feab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58058/0 (socket says 172.21.15.42:58058) 2024-02-15T18:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.142+0000 7ff0c77fe700 1 -- 172.21.15.42:0/946746331 learned_addr learned my addr 172.21.15.42:0/946746331 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.142+0000 7ff0c77fe700 1 -- 172.21.15.42:0/946746331 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c80ff0e0 con 0x7ff0c80703d0 2024-02-15T18:16:55.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.143+0000 7ff0c77fe700 1 --2- 172.21.15.42:0/946746331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0c80703d0 0x7ff0c80feab0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7ff0b800b5f0 tx=0x7ff0b8006ee0).ready entity=mon.0 client_cookie=60fdcc33e8dd0d1e server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:16:55.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.143+0000 7ff0c67fc700 1 -- 172.21.15.42:0/946746331 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0b800d600 con 0x7ff0c80703d0 2024-02-15T18:16:55.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.143+0000 7ff0c67fc700 1 -- 172.21.15.42:0/946746331 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b800d760 con 0x7ff0c80703d0 2024-02-15T18:16:55.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.143+0000 7ff0c67fc700 1 -- 172.21.15.42:0/946746331 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0b800da30 con 0x7ff0c80703d0 2024-02-15T18:16:55.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.143+0000 7ff0cdd57700 1 -- 172.21.15.42:0/946746331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0c80703d0 msgr2=0x7ff0c80feab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:16:55.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.143+0000 7ff0cdd57700 1 --2- 172.21.15.42:0/946746331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0c80703d0 0x7ff0c80feab0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7ff0b800b5f0 tx=0x7ff0b8006ee0).stop 2024-02-15T18:16:55.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.143+0000 7ff0cdd57700 1 -- 172.21.15.42:0/946746331 shutdown_connections 2024-02-15T18:16:55.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.143+0000 7ff0cdd57700 1 --2- 172.21.15.42:0/946746331 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0c80703d0 0x7ff0c80feab0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:16:55.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.143+0000 7ff0cdd57700 1 -- 172.21.15.42:0/946746331 >> 172.21.15.42:0/946746331 conn(0x7ff0c80f6530 msgr2=0x7ff0c80f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.143+0000 7ff0cdd57700 1 -- 172.21.15.42:0/946746331 shutdown_connections 2024-02-15T18:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.143+0000 7ff0cdd57700 1 -- 172.21.15.42:0/946746331 wait complete. 2024-02-15T18:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.144+0000 7ff0cdd57700 1 Processor -- start 2024-02-15T18:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.144+0000 7ff0cdd57700 1 -- start start 2024-02-15T18:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.145+0000 7ff0cdd57700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0c80703d0 0x7ff0c80fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.145+0000 7ff0cdd57700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80fa9b0 con 0x7ff0c80703d0 2024-02-15T18:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.145+0000 7ff0c77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0c80703d0 0x7ff0c80fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.145+0000 7ff0c77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0c80703d0 0x7ff0c80fc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58064/0 (socket says 172.21.15.42:58064) 2024-02-15T18:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.145+0000 7ff0c77fe700 1 -- 172.21.15.42:0/4211236772 learned_addr learned my addr 172.21.15.42:0/4211236772 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.145+0000 7ff0c77fe700 1 -- 172.21.15.42:0/4211236772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b800b200 con 0x7ff0c80703d0 2024-02-15T18:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.145+0000 7ff0c77fe700 1 --2- 172.21.15.42:0/4211236772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0c80703d0 0x7ff0c80fc530 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7ff0b8006c70 tx=0x7ff0b8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.145+0000 7ff0bffff700 1 -- 172.21.15.42:0/4211236772 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0b8011b40 con 0x7ff0c80703d0 2024-02-15T18:16:55.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.145+0000 7ff0bffff700 1 -- 172.21.15.42:0/4211236772 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b8011ca0 con 0x7ff0c80703d0 2024-02-15T18:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.145+0000 7ff0cdd57700 1 -- 172.21.15.42:0/4211236772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c80fab50 con 0x7ff0c80703d0 2024-02-15T18:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.145+0000 7ff0bffff700 1 -- 172.21.15.42:0/4211236772 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0b801d430 con 0x7ff0c80703d0 2024-02-15T18:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.146+0000 7ff0cdd57700 1 -- 172.21.15.42:0/4211236772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c80fafe0 con 0x7ff0c80703d0 2024-02-15T18:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.146+0000 7ff0bffff700 1 -- 172.21.15.42:0/4211236772 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7ff0b800f070 con 0x7ff0c80703d0 2024-02-15T18:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.146+0000 7ff0bffff700 1 -- 172.21.15.42:0/4211236772 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7ff0b8027460 con 0x7ff0c80703d0 2024-02-15T18:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.147+0000 7ff0cdd57700 1 -- 172.21.15.42:0/4211236772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c8045ad0 con 0x7ff0c80703d0 2024-02-15T18:16:55.149 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.149+0000 7ff0bffff700 1 -- 172.21.15.42:0/4211236772 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7ff0b8016070 con 0x7ff0c80703d0 2024-02-15T18:16:55.216 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.216+0000 7ff0cdd57700 1 -- 172.21.15.42:0/4211236772 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff0c80ff390 con 0x7ff0c80703d0 2024-02-15T18:16:55.219 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.219+0000 7ff0bffff700 1 -- 172.21.15.42:0/4211236772 <== mon.0 v2:172.21.15.42:3300/0 7 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7ff0b801d660 con 0x7ff0c80703d0 2024-02-15T18:16:55.219 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.219+0000 7ff0bffff700 1 -- 172.21.15.42:0/4211236772 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+489 (secure 0 0 0) 0x7ff0c80ff390 con 0x7ff0c80703d0 2024-02-15T18:16:55.220 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: [global] 2024-02-15T18:16:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: fsid = 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:16:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-15T18:16:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: mon_host = [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-02-15T18:16:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-15T18:16:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-15T18:16:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-15T18:16:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-15T18:16:55.221 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: [mgr] 2024-02-15T18:16:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-15T18:16:55.222 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: [osd] 2024-02-15T18:16:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: mon_osd_backfillfull_ratio = 0.85 2024-02-15T18:16:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: mon_osd_full_ratio = 0.9 2024-02-15T18:16:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: mon_osd_nearfull_ratio = 0.8 2024-02-15T18:16:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-15T18:16:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: osd_memory_target_autotune = true 2024-02-15T18:16:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-15T18:16:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.220+0000 7ff0cdd57700 1 -- 172.21.15.42:0/4211236772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0c80703d0 msgr2=0x7ff0c80fc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:16:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.220+0000 7ff0cdd57700 1 --2- 172.21.15.42:0/4211236772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0c80703d0 0x7ff0c80fc530 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7ff0b8006c70 tx=0x7ff0b8016000).stop 2024-02-15T18:16:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.221+0000 7ff0cdd57700 1 -- 172.21.15.42:0/4211236772 shutdown_connections 2024-02-15T18:16:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.221+0000 7ff0cdd57700 1 --2- 172.21.15.42:0/4211236772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0c80703d0 0x7ff0c80fc530 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:16:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.221+0000 7ff0cdd57700 1 -- 172.21.15.42:0/4211236772 >> 172.21.15.42:0/4211236772 conn(0x7ff0c80f6530 msgr2=0x7ff0c80f81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:16:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.221+0000 7ff0cdd57700 1 -- 172.21.15.42:0/4211236772 shutdown_connections 2024-02-15T18:16:55.223 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:55.221+0000 7ff0cdd57700 1 -- 172.21.15.42:0/4211236772 wait complete. 2024-02-15T18:16:55.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[37831]: audit 2024-02-15T18:16:55.217399+0000 mon.smithi042 (mon.0) 8 : audit [INF] from='client.? 172.21.15.42:0/4211236772' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-15T18:16:55.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[37831]: audit 2024-02-15T18:16:55.220436+0000 mon.smithi042 (mon.0) 9 : audit [INF] from='client.? 172.21.15.42:0/4211236772' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-15T18:16:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:Generating new minimal ceph.conf... 2024-02-15T18:16:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z -v /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042:/var/lib/ceph/mon/ceph-smithi042:z docker.io/ceph/ceph:v16.2.4 config generate-minimal-conf -o /var/lib/ceph/mon/ceph-smithi042/config 2024-02-15T18:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.419+0000 7f9e0e634700 1 Processor -- start 2024-02-15T18:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.419+0000 7f9e0e634700 1 -- start start 2024-02-15T18:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.419+0000 7f9e0e634700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e08070430 0x7f9e080fea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.419+0000 7f9e0e634700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e080fef40 con 0x7f9e08070430 2024-02-15T18:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.419+0000 7f9e07fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e08070430 0x7f9e080fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.419+0000 7f9e07fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e08070430 0x7f9e080fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58072/0 (socket says 172.21.15.42:58072) 2024-02-15T18:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.419+0000 7f9e07fff700 1 -- 172.21.15.42:0/1495508324 learned_addr learned my addr 172.21.15.42:0/1495508324 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.420+0000 7f9e07fff700 1 -- 172.21.15.42:0/1495508324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e080ff080 con 0x7f9e08070430 2024-02-15T18:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.420+0000 7f9e07fff700 1 --2- 172.21.15.42:0/1495508324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e08070430 0x7f9e080fea50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f9df800b5f0 tx=0x7f9df8006ee0).ready entity=mon.0 client_cookie=f4a2f39d99c7d119 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.420+0000 7f9e077fe700 1 -- 172.21.15.42:0/1495508324 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9df800d600 con 0x7f9e08070430 2024-02-15T18:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.420+0000 7f9e077fe700 1 -- 172.21.15.42:0/1495508324 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f9df800dbf0 con 0x7f9e08070430 2024-02-15T18:16:56.422 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.420+0000 7f9e077fe700 1 -- 172.21.15.42:0/1495508324 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9df8011cb0 con 0x7f9e08070430 2024-02-15T18:16:56.423 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.420+0000 7f9e0e634700 1 -- 172.21.15.42:0/1495508324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e08070430 msgr2=0x7f9e080fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:16:56.423 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.420+0000 7f9e0e634700 1 --2- 172.21.15.42:0/1495508324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e08070430 0x7f9e080fea50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f9df800b5f0 tx=0x7f9df8006ee0).stop 2024-02-15T18:16:56.423 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.421+0000 7f9e0e634700 1 -- 172.21.15.42:0/1495508324 shutdown_connections 2024-02-15T18:16:56.423 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.421+0000 7f9e0e634700 1 --2- 172.21.15.42:0/1495508324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e08070430 0x7f9e080fea50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:16:56.423 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.421+0000 7f9e0e634700 1 -- 172.21.15.42:0/1495508324 >> 172.21.15.42:0/1495508324 conn(0x7f9e080f6530 msgr2=0x7f9e080f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:16:56.423 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.421+0000 7f9e0e634700 1 -- 172.21.15.42:0/1495508324 shutdown_connections 2024-02-15T18:16:56.423 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.421+0000 7f9e0e634700 1 -- 172.21.15.42:0/1495508324 wait complete. 2024-02-15T18:16:56.424 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.422+0000 7f9e0e634700 1 Processor -- start 2024-02-15T18:16:56.424 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.422+0000 7f9e0e634700 1 -- start start 2024-02-15T18:16:56.424 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.422+0000 7f9e0e634700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e08070430 0x7f9e0810ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:16:56.424 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.422+0000 7f9e0e634700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0810b080 con 0x7f9e08070430 2024-02-15T18:16:56.424 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.422+0000 7f9e07fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e08070430 0x7f9e0810ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.422+0000 7f9e07fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e08070430 0x7f9e0810ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58078/0 (socket says 172.21.15.42:58078) 2024-02-15T18:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.422+0000 7f9e07fff700 1 -- 172.21.15.42:0/1004468748 learned_addr learned my addr 172.21.15.42:0/1004468748 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.422+0000 7f9e07fff700 1 -- 172.21.15.42:0/1004468748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df800b200 con 0x7f9e08070430 2024-02-15T18:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.422+0000 7f9e07fff700 1 --2- 172.21.15.42:0/1004468748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e08070430 0x7f9e0810ab90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f9df800df90 tx=0x7f9df8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.423+0000 7f9e057fa700 1 -- 172.21.15.42:0/1004468748 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9df801b410 con 0x7f9e08070430 2024-02-15T18:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.423+0000 7f9e057fa700 1 -- 172.21.15.42:0/1004468748 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f9df801ba00 con 0x7f9e08070430 2024-02-15T18:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.423+0000 7f9e0e634700 1 -- 172.21.15.42:0/1004468748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0810b280 con 0x7f9e08070430 2024-02-15T18:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.423+0000 7f9e057fa700 1 -- 172.21.15.42:0/1004468748 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9df801aa30 con 0x7f9e08070430 2024-02-15T18:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.423+0000 7f9e0e634700 1 -- 172.21.15.42:0/1004468748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0810b710 con 0x7f9e08070430 2024-02-15T18:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.423+0000 7f9e057fa700 1 -- 172.21.15.42:0/1004468748 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f9df800f070 con 0x7f9e08070430 2024-02-15T18:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.423+0000 7f9e057fa700 1 -- 172.21.15.42:0/1004468748 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f9df8025c80 con 0x7f9e08070430 2024-02-15T18:16:56.425 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.424+0000 7f9e0e634700 1 -- 172.21.15.42:0/1004468748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e08045ad0 con 0x7f9e08070430 2024-02-15T18:16:56.426 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.426+0000 7f9e057fa700 1 -- 172.21.15.42:0/1004468748 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f9df8016070 con 0x7f9e08070430 2024-02-15T18:16:56.493 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.492+0000 7f9e0e634700 1 -- 172.21.15.42:0/1004468748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9e080ffdc0 con 0x7f9e08070430 2024-02-15T18:16:56.493 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.493+0000 7f9e057fa700 1 -- 172.21.15.42:0/1004468748 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f9df801ae90 con 0x7f9e08070430 2024-02-15T18:16:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.493+0000 7f9e0e634700 1 -- 172.21.15.42:0/1004468748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e08070430 msgr2=0x7f9e0810ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:16:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.493+0000 7f9e0e634700 1 --2- 172.21.15.42:0/1004468748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e08070430 0x7f9e0810ab90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f9df800df90 tx=0x7f9df8006ee0).stop 2024-02-15T18:16:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.494+0000 7f9e0e634700 1 -- 172.21.15.42:0/1004468748 shutdown_connections 2024-02-15T18:16:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.494+0000 7f9e0e634700 1 --2- 172.21.15.42:0/1004468748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e08070430 0x7f9e0810ab90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:16:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.494+0000 7f9e0e634700 1 -- 172.21.15.42:0/1004468748 >> 172.21.15.42:0/1004468748 conn(0x7f9e080f6530 msgr2=0x7f9e080f7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:16:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.494+0000 7f9e0e634700 1 -- 172.21.15.42:0/1004468748 shutdown_connections 2024-02-15T18:16:56.494 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:56.494+0000 7f9e0e634700 1 -- 172.21.15.42:0/1004468748 wait complete. 2024-02-15T18:16:56.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[37831]: audit 2024-02-15T18:16:56.494138+0000 mon.smithi042 (mon.0) 10 : audit [DBG] from='client.? 172.21.15.42:0/1004468748' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:16:57.060 INFO:teuthology.orchestra.run.smithi042.stderr:Restarting the monitor... 2024-02-15T18:16:57.060 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl restart ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi042 2024-02-15T18:16:57.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:57 smithi042 systemd[1]: Stopping Ceph mon.smithi042 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:16:57.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[37831]: debug 2024-02-15T18:16:57.175+0000 7feef8eac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi042 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-15T18:16:57.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[37831]: debug 2024-02-15T18:16:57.175+0000 7feef8eac700 -1 mon.smithi042@0(leader) e1 *** Got Signal Terminated *** 2024-02-15T18:16:58.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:57 smithi042 podman[38155]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042 2024-02-15T18:16:59.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:58 smithi042 systemd[1]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi042.service: Succeeded. 2024-02-15T18:16:59.061 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:58 smithi042 systemd[1]: Stopped Ceph mon.smithi042 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:16:59.061 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:58 smithi042 systemd[1]: Starting Ceph mon.smithi042 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:16:59.226 INFO:teuthology.orchestra.run.smithi042.stderr:Setting mon public_network to 172.21.0.0/20 2024-02-15T18:16:59.226 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config set mon public_network 172.21.0.0/20 2024-02-15T18:16:59.440 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.185+0000 7f0b746df700 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-15T18:16:59.440 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.185+0000 7f0b746df700 0 ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable), process ceph-mon, pid 7 2024-02-15T18:16:59.440 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.185+0000 7f0b746df700 0 pidfile_write: ignore empty --pid-file 2024-02-15T18:16:59.441 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 0 load: jerasure load: lrc load: isa 2024-02-15T18:16:59.441 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: RocksDB version: 6.8.1 2024-02-15T18:16:59.441 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.441 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-15T18:16:59.441 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Compile date May 13 2021 2024-02-15T18:16:59.441 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: DB SUMMARY 2024-02-15T18:16:59.441 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.441 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: CURRENT file: CURRENT 2024-02-15T18:16:59.441 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.441 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: IDENTITY file: IDENTITY 2024-02-15T18:16:59.441 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.441 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 131 Bytes 2024-02-15T18:16:59.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi042/store.db dir, Total Num: 1, files: 000004.sst 2024-02-15T18:16:59.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi042/store.db: 000006.log size: 98708 ; 2024-02-15T18:16:59.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.error_if_exists: 0 2024-02-15T18:16:59.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.create_if_missing: 0 2024-02-15T18:16:59.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.paranoid_checks: 1 2024-02-15T18:16:59.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.env: 0x55bf03ae21c0 2024-02-15T18:16:59.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.fs: Posix File System 2024-02-15T18:16:59.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.info_log: 0x55bf04721960 2024-02-15T18:16:59.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-15T18:16:59.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.statistics: (nil) 2024-02-15T18:16:59.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.use_fsync: 0 2024-02-15T18:16:59.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.max_log_file_size: 0 2024-02-15T18:16:59.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-15T18:16:59.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-15T18:16:59.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-15T18:16:59.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-15T18:16:59.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.allow_fallocate: 1 2024-02-15T18:16:59.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-15T18:16:59.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-15T18:16:59.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.use_direct_reads: 0 2024-02-15T18:16:59.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-15T18:16:59.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.create_missing_column_families: 0 2024-02-15T18:16:59.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.db_log_dir: 2024-02-15T18:16:59.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi042/store.db 2024-02-15T18:16:59.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-15T18:16:59.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.max_subcompactions: 1 2024-02-15T18:16:59.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.max_background_flushes: -1 2024-02-15T18:16:59.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-15T18:16:59.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-15T18:16:59.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-15T18:16:59.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-15T18:16:59.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-15T18:16:59.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.advise_random_on_open: 1 2024-02-15T18:16:59.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-15T18:16:59.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.write_buffer_manager: 0x55bf05396030 2024-02-15T18:16:59.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-15T18:16:59.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-15T18:16:59.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-15T18:16:59.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-15T18:16:59.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.rate_limiter: (nil) 2024-02-15T18:16:59.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-15T18:16:59.446 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-15T18:16:59.446 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-15T18:16:59.446 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-15T18:16:59.446 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.unordered_write: 0 2024-02-15T18:16:59.446 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-15T18:16:59.446 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-15T18:16:59.446 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-15T18:16:59.446 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-15T18:16:59.446 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.row_cache: None 2024-02-15T18:16:59.446 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.190+0000 7f0b746df700 4 rocksdb: Options.wal_filter: None 2024-02-15T18:16:59.446 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-15T18:16:59.446 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-15T18:16:59.447 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.preserve_deletes: 0 2024-02-15T18:16:59.447 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.two_write_queues: 0 2024-02-15T18:16:59.447 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.manual_wal_flush: 0 2024-02-15T18:16:59.447 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.atomic_flush: 0 2024-02-15T18:16:59.447 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-15T18:16:59.447 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-15T18:16:59.447 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-15T18:16:59.447 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.log_readahead_size: 0 2024-02-15T18:16:59.447 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-15T18:16:59.447 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_background_jobs: 2 2024-02-15T18:16:59.447 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_background_compactions: -1 2024-02-15T18:16:59.447 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-15T18:16:59.447 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-15T18:16:59.447 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-15T18:16:59.448 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_total_wal_size: 0 2024-02-15T18:16:59.448 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-15T18:16:59.448 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-15T18:16:59.448 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-15T18:16:59.448 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-15T18:16:59.448 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_open_files: -1 2024-02-15T18:16:59.448 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.bytes_per_sync: 0 2024-02-15T18:16:59.448 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-15T18:16:59.448 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-15T18:16:59.448 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-15T18:16:59.448 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Compression algorithms supported: 2024-02-15T18:16:59.449 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-15T18:16:59.449 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: kZSTD supported: 0 2024-02-15T18:16:59.449 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: kXpressCompression supported: 0 2024-02-15T18:16:59.449 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-15T18:16:59.449 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: kLZ4Compression supported: 1 2024-02-15T18:16:59.449 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: kBZip2Compression supported: 0 2024-02-15T18:16:59.449 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: kZlibCompression supported: 1 2024-02-15T18:16:59.449 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: kSnappyCompression supported: 1 2024-02-15T18:16:59.449 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-15T18:16:59.449 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi042/store.db/MANIFEST-000005 2024-02-15T18:16:59.450 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.450 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-15T18:16:59.450 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.450 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-15T18:16:59.450 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.merge_operator: 2024-02-15T18:16:59.450 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compaction_filter: None 2024-02-15T18:16:59.450 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compaction_filter_factory: None 2024-02-15T18:16:59.450 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-15T18:16:59.450 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-15T18:16:59.450 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bf04639cb0) 2024-02-15T18:16:59.450 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cache_index_and_filter_blocks: 1 2024-02-15T18:16:59.450 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-15T18:16:59.451 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-15T18:16:59.451 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: pin_top_level_index_and_filter: 1 2024-02-15T18:16:59.451 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: index_type: 0 2024-02-15T18:16:59.451 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: data_block_index_type: 0 2024-02-15T18:16:59.451 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: index_shortening: 1 2024-02-15T18:16:59.451 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: data_block_hash_table_util_ratio: 0.750000 2024-02-15T18:16:59.451 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: hash_index_allow_collision: 1 2024-02-15T18:16:59.451 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: checksum: 1 2024-02-15T18:16:59.451 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: no_block_cache: 0 2024-02-15T18:16:59.451 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: block_cache: 0x55bf0466ef10 2024-02-15T18:16:59.451 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: block_cache_name: BinnedLRUCache 2024-02-15T18:16:59.451 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: block_cache_options: 2024-02-15T18:16:59.452 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: capacity : 536870912 2024-02-15T18:16:59.452 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: num_shard_bits : 4 2024-02-15T18:16:59.452 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: strict_capacity_limit : 0 2024-02-15T18:16:59.452 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: high_pri_pool_ratio: 0.000 2024-02-15T18:16:59.453 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: block_cache_compressed: (nil) 2024-02-15T18:16:59.454 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: persistent_cache: (nil) 2024-02-15T18:16:59.454 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: block_size: 4096 2024-02-15T18:16:59.454 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: block_size_deviation: 10 2024-02-15T18:16:59.454 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: block_restart_interval: 16 2024-02-15T18:16:59.454 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: index_block_restart_interval: 1 2024-02-15T18:16:59.454 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: metadata_block_size: 4096 2024-02-15T18:16:59.454 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: partition_filters: 0 2024-02-15T18:16:59.454 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: use_delta_encoding: 1 2024-02-15T18:16:59.454 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-15T18:16:59.454 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: whole_key_filtering: 1 2024-02-15T18:16:59.454 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: verify_compression: 0 2024-02-15T18:16:59.454 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: read_amp_bytes_per_bit: 0 2024-02-15T18:16:59.455 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: format_version: 2 2024-02-15T18:16:59.455 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: enable_index_compression: 1 2024-02-15T18:16:59.455 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: block_align: 0 2024-02-15T18:16:59.455 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.455 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-15T18:16:59.455 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-15T18:16:59.455 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compression: NoCompression 2024-02-15T18:16:59.455 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-15T18:16:59.455 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-15T18:16:59.455 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-15T18:16:59.455 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.num_levels: 7 2024-02-15T18:16:59.456 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-15T18:16:59.456 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-15T18:16:59.456 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-15T18:16:59.456 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-15T18:16:59.456 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-15T18:16:59.456 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-15T18:16:59.456 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-15T18:16:59.456 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:16:59.456 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-15T18:16:59.456 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-15T18:16:59.456 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compression_opts.level: 32767 2024-02-15T18:16:59.457 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-15T18:16:59.457 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-15T18:16:59.457 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:16:59.457 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compression_opts.enabled: false 2024-02-15T18:16:59.457 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-15T18:16:59.457 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-15T18:16:59.457 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-15T18:16:59.457 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-15T18:16:59.457 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-15T18:16:59.457 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-15T18:16:59.457 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-15T18:16:59.457 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-15T18:16:59.458 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-15T18:16:59.458 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-15T18:16:59.458 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-15T18:16:59.458 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-15T18:16:59.458 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-15T18:16:59.458 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-15T18:16:59.458 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-15T18:16:59.458 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-15T18:16:59.458 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-15T18:16:59.458 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.arena_block_size: 4194304 2024-02-15T18:16:59.458 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-15T18:16:59.458 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-15T18:16:59.459 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-15T18:16:59.459 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-15T18:16:59.459 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-15T18:16:59.459 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-15T18:16:59.459 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-15T18:16:59.459 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-15T18:16:59.459 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-15T18:16:59.459 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-15T18:16:59.459 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-15T18:16:59.459 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-15T18:16:59.459 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-15T18:16:59.460 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-15T18:16:59.460 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.table_properties_collectors: 2024-02-15T18:16:59.460 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.inplace_update_support: 0 2024-02-15T18:16:59.460 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-15T18:16:59.460 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-15T18:16:59.460 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-15T18:16:59.460 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-15T18:16:59.460 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.bloom_locality: 0 2024-02-15T18:16:59.460 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.max_successive_merges: 0 2024-02-15T18:16:59.460 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-15T18:16:59.460 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-15T18:16:59.460 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.force_consistency_checks: 0 2024-02-15T18:16:59.461 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-15T18:16:59.461 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.ttl: 2592000 2024-02-15T18:16:59.461 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-15T18:16:59.461 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi042/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-15T18:16:59.461 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.461 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 4 2024-02-15T18:16:59.461 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.461 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708021019192957, "job": 1, "event": "recovery_started", "log_files": [6]} 2024-02-15T18:16:59.461 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.191+0000 7f0b746df700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #6 mode 2 2024-02-15T18:16:59.461 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.192+0000 7f0b746df700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-15T18:16:59.461 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.192+0000 7f0b746df700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708021019193940, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 94282, "table_properties": {"data_size": 92370, "index_size": 389, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 709, "raw_key_size": 12563, "raw_average_key_size": 49, "raw_value_size": 85401, "raw_average_value_size": 336, "num_data_blocks": 12, "num_entries": 254, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708021019, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-15T18:16:59.462 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.192+0000 7f0b746df700 4 rocksdb: [version_set.cc:3826] Creating manifest 8 2024-02-15T18:16:59.462 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.462 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.193+0000 7f0b746df700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708021019194394, "job": 1, "event": "recovery_finished"} 2024-02-15T18:16:59.462 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.194+0000 7f0b746df700 4 rocksdb: DB pointer 0x55bf04733800 2024-02-15T18:16:59.462 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.194+0000 7f0b5b4f7700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-15T18:16:59.462 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.194+0000 7f0b5b4f7700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-15T18:16:59.462 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: ** DB Stats ** 2024-02-15T18:16:59.462 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:16:59.462 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-15T18:16:59.462 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-15T18:16:59.462 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:16:59.462 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-15T18:16:59.463 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-15T18:16:59.463 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:16:59.463 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.463 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: ** Compaction Stats [default] ** 2024-02-15T18:16:59.463 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:16:59.463 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:16:59.463 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: L0 2/0 93.76 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.3 0.00 0.00 1 0.001 0 0 2024-02-15T18:16:59.463 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Sum 2/0 93.76 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.3 0.00 0.00 1 0.001 0 0 2024-02-15T18:16:59.463 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.3 0.00 0.00 1 0.001 0 0 2024-02-15T18:16:59.463 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: ** Compaction Stats [default] ** 2024-02-15T18:16:59.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:16:59.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:16:59.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.3 0.00 0.00 1 0.001 0 0 2024-02-15T18:16:59.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:16:59.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-15T18:16:59.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:16:59.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:16:59.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:16:59.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:16:59.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Cumulative compaction: 0.00 GB write, 29.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:16:59.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Interval compaction: 0.00 GB write, 29.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:16:59.465 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:16:59.465 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.465 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:16:59.465 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.465 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: ** Compaction Stats [default] ** 2024-02-15T18:16:59.465 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:16:59.465 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:16:59.465 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: L0 2/0 93.76 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.3 0.00 0.00 1 0.001 0 0 2024-02-15T18:16:59.465 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Sum 2/0 93.76 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.3 0.00 0.00 1 0.001 0 0 2024-02-15T18:16:59.465 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:16:59.465 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: ** Compaction Stats [default] ** 2024-02-15T18:16:59.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:16:59.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:16:59.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.3 0.00 0.00 1 0.001 0 0 2024-02-15T18:16:59.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:16:59.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-15T18:16:59.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:16:59.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:16:59.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:16:59.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:16:59.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Cumulative compaction: 0.00 GB write, 29.09 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:16:59.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:16:59.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:16:59.467 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.467 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:16:59.467 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.467 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.194+0000 7f0b746df700 0 starting mon.smithi042 rank 0 at public addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] at bind addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi042 fsid 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:16:59.467 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.195+0000 7f0b746df700 1 mon.smithi042@-1(???) e1 preinit fsid 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:16:59.467 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.196+0000 7f0b746df700 0 mon.smithi042@-1(???).mds e1 new map 2024-02-15T18:16:59.467 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.196+0000 7f0b746df700 0 mon.smithi042@-1(???).mds e1 print_map 2024-02-15T18:16:59.467 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: e1 2024-02-15T18:16:59.467 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:16:59.467 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:16:59.467 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: legacy client fscid: -1 2024-02-15T18:16:59.468 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.468 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: No filesystems configured 2024-02-15T18:16:59.468 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: 2024-02-15T18:16:59.468 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.196+0000 7f0b746df700 0 mon.smithi042@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-15T18:16:59.468 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.196+0000 7f0b746df700 0 mon.smithi042@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-15T18:16:59.468 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.196+0000 7f0b746df700 0 mon.smithi042@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-15T18:16:59.468 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.196+0000 7f0b746df700 0 mon.smithi042@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-15T18:16:59.468 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:16:56.494138+0000 mon.smithi042 (mon.0) 10 : audit [DBG] from='client.? 172.21.15.42:0/1004468748' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:16:59.468 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.196+0000 7f0b746df700 1 mon.smithi042@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-15T18:16:59.468 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.197+0000 7f0b746df700 20 expand_channel_meta expand map: {default=false} 2024-02-15T18:16:59.468 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.197+0000 7f0b746df700 20 expand_channel_meta from 'false' to 'false' 2024-02-15T18:16:59.469 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.197+0000 7f0b746df700 20 expand_channel_meta expanded map: {default=false} 2024-02-15T18:16:59.469 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.197+0000 7f0b746df700 20 expand_channel_meta expand map: {default=info} 2024-02-15T18:16:59.469 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.197+0000 7f0b746df700 20 expand_channel_meta from 'info' to 'info' 2024-02-15T18:16:59.469 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.197+0000 7f0b746df700 20 expand_channel_meta expanded map: {default=info} 2024-02-15T18:16:59.469 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.197+0000 7f0b746df700 20 expand_channel_meta expand map: {default=daemon} 2024-02-15T18:16:59.469 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.197+0000 7f0b746df700 20 expand_channel_meta from 'daemon' to 'daemon' 2024-02-15T18:16:59.469 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.197+0000 7f0b746df700 20 expand_channel_meta expanded map: {default=daemon} 2024-02-15T18:16:59.469 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.197+0000 7f0b746df700 20 expand_channel_meta expand map: {default=debug} 2024-02-15T18:16:59.469 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.197+0000 7f0b746df700 20 expand_channel_meta from 'debug' to 'debug' 2024-02-15T18:16:59.469 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:16:59.197+0000 7f0b746df700 20 expand_channel_meta expanded map: {default=debug} 2024-02-15T18:16:59.469 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 bash[38228]: 9dcbc5c49e053a7d5e3034317648123b0d35be3d08e9b32525a59f556f7ac027 2024-02-15T18:16:59.469 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 systemd[1]: Started Ceph mon.smithi042 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:16:59.470 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:16:59.214022+0000 mon.smithi042 (mon.0) 1 : cluster [INF] mon.smithi042 is new leader, mons smithi042 in quorum (ranks 0) 2024-02-15T18:16:59.470 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:16:59.214075+0000 mon.smithi042 (mon.0) 2 : cluster [DBG] monmap e1: 1 mons at {smithi042=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0]} 2024-02-15T18:16:59.470 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:16:59.214130+0000 mon.smithi042 (mon.0) 3 : cluster [DBG] fsmap 2024-02-15T18:16:59.470 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:16:59.214159+0000 mon.smithi042 (mon.0) 4 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-15T18:16:59.470 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:16:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:16:59.214488+0000 mon.smithi042 (mon.0) 5 : cluster [DBG] mgrmap e1: no daemons active 2024-02-15T18:16:59.727 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.725+0000 7fe497ad1700 1 Processor -- start 2024-02-15T18:16:59.727 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.725+0000 7fe497ad1700 1 -- start start 2024-02-15T18:16:59.727 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.726+0000 7fe497ad1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4900703b0 0x7fe490100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:16:59.727 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.726+0000 7fe497ad1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900fc8c0 con 0x7fe4900703b0 2024-02-15T18:16:59.727 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.726+0000 7fe49586d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4900703b0 0x7fe490100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:16:59.727 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.726+0000 7fe49586d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4900703b0 0x7fe490100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58090/0 (socket says 172.21.15.42:58090) 2024-02-15T18:16:59.727 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.726+0000 7fe49586d700 1 -- 172.21.15.42:0/2717755429 learned_addr learned my addr 172.21.15.42:0/2717755429 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:16:59.731 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.727+0000 7fe49586d700 1 -- 172.21.15.42:0/2717755429 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4901013c0 con 0x7fe4900703b0 2024-02-15T18:16:59.731 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.727+0000 7fe49586d700 1 --2- 172.21.15.42:0/2717755429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4900703b0 0x7fe490100ed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fe48400b5f0 tx=0x7fe484006ee0).ready entity=mon.0 client_cookie=1ee4368535de746e server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:16:59.731 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.727+0000 7fe49486b700 1 -- 172.21.15.42:0/2717755429 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe48400d600 con 0x7fe4900703b0 2024-02-15T18:16:59.731 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.727+0000 7fe49486b700 1 -- 172.21.15.42:0/2717755429 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fe48400dbf0 con 0x7fe4900703b0 2024-02-15T18:16:59.731 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.727+0000 7fe49486b700 1 -- 172.21.15.42:0/2717755429 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe484011cb0 con 0x7fe4900703b0 2024-02-15T18:16:59.731 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.727+0000 7fe497ad1700 1 -- 172.21.15.42:0/2717755429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4900703b0 msgr2=0x7fe490100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:16:59.731 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.727+0000 7fe497ad1700 1 --2- 172.21.15.42:0/2717755429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4900703b0 0x7fe490100ed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fe48400b5f0 tx=0x7fe484006ee0).stop 2024-02-15T18:16:59.731 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.727+0000 7fe497ad1700 1 -- 172.21.15.42:0/2717755429 shutdown_connections 2024-02-15T18:16:59.731 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.727+0000 7fe497ad1700 1 --2- 172.21.15.42:0/2717755429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4900703b0 0x7fe490100ed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:16:59.731 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.727+0000 7fe497ad1700 1 -- 172.21.15.42:0/2717755429 >> 172.21.15.42:0/2717755429 conn(0x7fe4900f6530 msgr2=0x7fe4900f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:16:59.731 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.728+0000 7fe497ad1700 1 -- 172.21.15.42:0/2717755429 shutdown_connections 2024-02-15T18:16:59.732 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.728+0000 7fe497ad1700 1 -- 172.21.15.42:0/2717755429 wait complete. 2024-02-15T18:16:59.732 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.728+0000 7fe497ad1700 1 Processor -- start 2024-02-15T18:16:59.732 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.728+0000 7fe497ad1700 1 -- start start 2024-02-15T18:16:59.732 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.728+0000 7fe497ad1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4900703b0 0x7fe49010cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:16:59.732 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.728+0000 7fe497ad1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49010d270 con 0x7fe4900703b0 2024-02-15T18:16:59.732 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.729+0000 7fe49586d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4900703b0 0x7fe49010cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:16:59.732 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.729+0000 7fe49586d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4900703b0 0x7fe49010cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58092/0 (socket says 172.21.15.42:58092) 2024-02-15T18:16:59.732 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.729+0000 7fe49586d700 1 -- 172.21.15.42:0/140950318 learned_addr learned my addr 172.21.15.42:0/140950318 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:16:59.732 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.729+0000 7fe49586d700 1 -- 172.21.15.42:0/140950318 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48400b200 con 0x7fe4900703b0 2024-02-15T18:16:59.732 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.729+0000 7fe49586d700 1 --2- 172.21.15.42:0/140950318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4900703b0 0x7fe49010cd80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fe484006800 tx=0x7fe484006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:16:59.732 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.729+0000 7fe4827fc700 1 -- 172.21.15.42:0/140950318 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe48401b570 con 0x7fe4900703b0 2024-02-15T18:16:59.732 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.729+0000 7fe497ad1700 1 -- 172.21.15.42:0/140950318 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49010d470 con 0x7fe4900703b0 2024-02-15T18:16:59.732 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.729+0000 7fe4827fc700 1 -- 172.21.15.42:0/140950318 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fe48401bb60 con 0x7fe4900703b0 2024-02-15T18:16:59.733 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.729+0000 7fe4827fc700 1 -- 172.21.15.42:0/140950318 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe48401a9f0 con 0x7fe4900703b0 2024-02-15T18:16:59.733 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.729+0000 7fe497ad1700 1 -- 172.21.15.42:0/140950318 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49010e080 con 0x7fe4900703b0 2024-02-15T18:16:59.733 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.729+0000 7fe4827fc700 1 -- 172.21.15.42:0/140950318 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fe48400f070 con 0x7fe4900703b0 2024-02-15T18:16:59.733 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.729+0000 7fe4827fc700 1 -- 172.21.15.42:0/140950318 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fe484025d80 con 0x7fe4900703b0 2024-02-15T18:16:59.733 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.730+0000 7fe497ad1700 1 -- 172.21.15.42:0/140950318 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe490059100 con 0x7fe4900703b0 2024-02-15T18:16:59.733 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.732+0000 7fe4827fc700 1 -- 172.21.15.42:0/140950318 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7fe484016070 con 0x7fe4900703b0 2024-02-15T18:16:59.799 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.799+0000 7fe497ad1700 1 -- 172.21.15.42:0/140950318 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fe490045ad0 con 0x7fe4900703b0 2024-02-15T18:16:59.801 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.801+0000 7fe4827fc700 1 -- 172.21.15.42:0/140950318 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe484025460 con 0x7fe4900703b0 2024-02-15T18:16:59.802 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.802+0000 7fe497ad1700 1 -- 172.21.15.42:0/140950318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4900703b0 msgr2=0x7fe49010cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:16:59.802 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.802+0000 7fe497ad1700 1 --2- 172.21.15.42:0/140950318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4900703b0 0x7fe49010cd80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fe484006800 tx=0x7fe484006ee0).stop 2024-02-15T18:16:59.803 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.802+0000 7fe497ad1700 1 -- 172.21.15.42:0/140950318 shutdown_connections 2024-02-15T18:16:59.803 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.802+0000 7fe497ad1700 1 --2- 172.21.15.42:0/140950318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4900703b0 0x7fe49010cd80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:16:59.803 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.802+0000 7fe497ad1700 1 -- 172.21.15.42:0/140950318 >> 172.21.15.42:0/140950318 conn(0x7fe4900f6530 msgr2=0x7fe4900f7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:16:59.803 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.802+0000 7fe497ad1700 1 -- 172.21.15.42:0/140950318 shutdown_connections 2024-02-15T18:16:59.803 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:16:59.802+0000 7fe497ad1700 1 -- 172.21.15.42:0/140950318 wait complete. 2024-02-15T18:17:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:Wrote config to /etc/ceph/ceph.conf 2024-02-15T18:17:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-15T18:17:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:Creating mgr... 2024-02-15T18:17:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:Verifying port 9283 ... 2024-02-15T18:17:00.534 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl daemon-reload 2024-02-15T18:17:00.642 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl stop ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mgr.smithi042.kfoihb 2024-02-15T18:17:00.668 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl reset-failed ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mgr.smithi042.kfoihb 2024-02-15T18:17:00.674 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: Failed to reset failed state of unit ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mgr.smithi042.kfoihb.service: Unit ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mgr.smithi042.kfoihb.service not loaded. 2024-02-15T18:17:00.674 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl enable ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mgr.smithi042.kfoihb 2024-02-15T18:17:00.680 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: Created symlink /etc/systemd/system/ceph-51458a44-cc2e-11ee-95ba-87774f69a715.target.wants/ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mgr.smithi042.kfoihb.service → /etc/systemd/system/ceph-51458a44-cc2e-11ee-95ba-87774f69a715@.service. 2024-02-15T18:17:00.772 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl start ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mgr.smithi042.kfoihb 2024-02-15T18:17:01.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:16:59.802373+0000 mon.smithi042 (mon.0) 6 : audit [INF] from='client.? 172.21.15.42:0/140950318' entity='client.admin' 2024-02-15T18:17:01.399 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-enabled firewalld.service 2024-02-15T18:17:01.407 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: disabled 2024-02-15T18:17:01.408 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-active firewalld.service 2024-02-15T18:17:01.416 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: inactive 2024-02-15T18:17:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:firewalld.service is not enabled 2024-02-15T18:17:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:Not possible to enable service . firewalld.service is not available 2024-02-15T18:17:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:Not possible to open ports <[]>. firewalld.service is not available 2024-02-15T18:17:01.417 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-enabled firewalld.service 2024-02-15T18:17:01.425 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: disabled 2024-02-15T18:17:01.425 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-active firewalld.service 2024-02-15T18:17:01.432 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: inactive 2024-02-15T18:17:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:firewalld.service is not enabled 2024-02-15T18:17:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2024-02-15T18:17:01.434 INFO:teuthology.orchestra.run.smithi042.stderr:Waiting for mgr to start... 2024-02-15T18:17:01.435 INFO:teuthology.orchestra.run.smithi042.stderr:Waiting for mgr... 2024-02-15T18:17:01.435 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status -f json-pretty 2024-02-15T18:17:02.054 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.048+0000 7fc61cb8b700 1 Processor -- start 2024-02-15T18:17:02.054 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.048+0000 7fc61cb8b700 1 -- start start 2024-02-15T18:17:02.054 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.049+0000 7fc61cb8b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180703d0 0x7fc618100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:02.054 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.049+0000 7fc61cb8b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6180fc8a0 con 0x7fc6180703d0 2024-02-15T18:17:02.054 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.049+0000 7fc61659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180703d0 0x7fc618100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:02.054 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.049+0000 7fc61659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180703d0 0x7fc618100eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47322/0 (socket says 172.21.15.42:47322) 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.049+0000 7fc61659c700 1 -- 172.21.15.42:0/3172975723 learned_addr learned my addr 172.21.15.42:0/3172975723 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.049+0000 7fc61659c700 1 -- 172.21.15.42:0/3172975723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6181013a0 con 0x7fc6180703d0 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.050+0000 7fc61659c700 1 --2- 172.21.15.42:0/3172975723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180703d0 0x7fc618100eb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fc60800b5f0 tx=0x7fc608006ee0).ready entity=mon.0 client_cookie=4dfca5d153e379a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.050+0000 7fc61559a700 1 -- 172.21.15.42:0/3172975723 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc60800d600 con 0x7fc6180703d0 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.050+0000 7fc61559a700 1 -- 172.21.15.42:0/3172975723 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fc60800dbf0 con 0x7fc6180703d0 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.050+0000 7fc61559a700 1 -- 172.21.15.42:0/3172975723 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc608011cb0 con 0x7fc6180703d0 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.051+0000 7fc61cb8b700 1 -- 172.21.15.42:0/3172975723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180703d0 msgr2=0x7fc618100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.051+0000 7fc61cb8b700 1 --2- 172.21.15.42:0/3172975723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180703d0 0x7fc618100eb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fc60800b5f0 tx=0x7fc608006ee0).stop 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.051+0000 7fc61cb8b700 1 -- 172.21.15.42:0/3172975723 shutdown_connections 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.051+0000 7fc61cb8b700 1 --2- 172.21.15.42:0/3172975723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180703d0 0x7fc618100eb0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.051+0000 7fc61cb8b700 1 -- 172.21.15.42:0/3172975723 >> 172.21.15.42:0/3172975723 conn(0x7fc6180f6530 msgr2=0x7fc6180f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.051+0000 7fc61cb8b700 1 -- 172.21.15.42:0/3172975723 shutdown_connections 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.051+0000 7fc61cb8b700 1 -- 172.21.15.42:0/3172975723 wait complete. 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.052+0000 7fc61cb8b700 1 Processor -- start 2024-02-15T18:17:02.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.052+0000 7fc61cb8b700 1 -- start start 2024-02-15T18:17:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.052+0000 7fc61cb8b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180703d0 0x7fc61810cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.052+0000 7fc61cb8b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61810d2a0 con 0x7fc6180703d0 2024-02-15T18:17:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.052+0000 7fc61659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180703d0 0x7fc61810cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.052+0000 7fc61659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180703d0 0x7fc61810cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47334/0 (socket says 172.21.15.42:47334) 2024-02-15T18:17:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.052+0000 7fc61659c700 1 -- 172.21.15.42:0/2015156306 learned_addr learned my addr 172.21.15.42:0/2015156306 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.053+0000 7fc61659c700 1 -- 172.21.15.42:0/2015156306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60800b200 con 0x7fc6180703d0 2024-02-15T18:17:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.053+0000 7fc61659c700 1 --2- 172.21.15.42:0/2015156306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180703d0 0x7fc61810cdb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fc608006a20 tx=0x7fc608006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.053+0000 7fc60effd700 1 -- 172.21.15.42:0/2015156306 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc60801b570 con 0x7fc6180703d0 2024-02-15T18:17:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.053+0000 7fc60effd700 1 -- 172.21.15.42:0/2015156306 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fc60801bb60 con 0x7fc6180703d0 2024-02-15T18:17:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.053+0000 7fc61cb8b700 1 -- 172.21.15.42:0/2015156306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc61810d4a0 con 0x7fc6180703d0 2024-02-15T18:17:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.053+0000 7fc60effd700 1 -- 172.21.15.42:0/2015156306 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc60801aa20 con 0x7fc6180703d0 2024-02-15T18:17:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.053+0000 7fc61cb8b700 1 -- 172.21.15.42:0/2015156306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc61810e0e0 con 0x7fc6180703d0 2024-02-15T18:17:02.056 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.053+0000 7fc60effd700 1 -- 172.21.15.42:0/2015156306 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fc60800f070 con 0x7fc6180703d0 2024-02-15T18:17:02.057 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.053+0000 7fc60effd700 1 -- 172.21.15.42:0/2015156306 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fc608025d00 con 0x7fc6180703d0 2024-02-15T18:17:02.057 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.054+0000 7fc61cb8b700 1 -- 172.21.15.42:0/2015156306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc61810d930 con 0x7fc6180703d0 2024-02-15T18:17:02.057 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.056+0000 7fc60effd700 1 -- 172.21.15.42:0/2015156306 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7fc608016070 con 0x7fc6180703d0 2024-02-15T18:17:02.127 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.126+0000 7fc61cb8b700 1 -- 172.21.15.42:0/2015156306 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc6181020c0 con 0x7fc6180703d0 2024-02-15T18:17:02.127 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.126+0000 7fc60effd700 1 -- 172.21.15.42:0/2015156306 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1230 (secure 0 0 0) 0x7fc60801a370 con 0x7fc6180703d0 2024-02-15T18:17:02.127 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.128 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: { 2024-02-15T18:17:02.128 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "fsid": "51458a44-cc2e-11ee-95ba-87774f69a715", 2024-02-15T18:17:02.128 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "health": { 2024-02-15T18:17:02.128 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-15T18:17:02.128 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "checks": {}, 2024-02-15T18:17:02.128 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "mutes": [] 2024-02-15T18:17:02.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:02.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-15T18:17:02.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "quorum": [ 2024-02-15T18:17:02.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 0 2024-02-15T18:17:02.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: ], 2024-02-15T18:17:02.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-15T18:17:02.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "smithi042" 2024-02-15T18:17:02.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: ], 2024-02-15T18:17:02.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "quorum_age": 2, 2024-02-15T18:17:02.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "monmap": { 2024-02-15T18:17:02.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "osdmap": { 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "pgmap": { 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-15T18:17:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-15T18:17:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-15T18:17:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-15T18:17:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-15T18:17:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-15T18:17:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-15T18:17:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "fsmap": { 2024-02-15T18:17:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:17:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-15T18:17:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-15T18:17:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-15T18:17:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "available": false, 2024-02-15T18:17:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "modules": [ 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "iostat", 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "restful" 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: ], 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "services": {} 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "servicemap": { 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "modified": "2024-02-15T18:16:53.407128+0000", 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "services": {} 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: } 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.127+0000 7fc61cb8b700 1 -- 172.21.15.42:0/2015156306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180703d0 msgr2=0x7fc61810cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:02.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.127+0000 7fc61cb8b700 1 --2- 172.21.15.42:0/2015156306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180703d0 0x7fc61810cdb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fc608006a20 tx=0x7fc608006ee0).stop 2024-02-15T18:17:02.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.127+0000 7fc61cb8b700 1 -- 172.21.15.42:0/2015156306 shutdown_connections 2024-02-15T18:17:02.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.127+0000 7fc61cb8b700 1 --2- 172.21.15.42:0/2015156306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6180703d0 0x7fc61810cdb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:02.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.127+0000 7fc61cb8b700 1 -- 172.21.15.42:0/2015156306 >> 172.21.15.42:0/2015156306 conn(0x7fc6180f6530 msgr2=0x7fc6180f7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:02.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.127+0000 7fc61cb8b700 1 -- 172.21.15.42:0/2015156306 shutdown_connections 2024-02-15T18:17:02.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:02.127+0000 7fc61cb8b700 1 -- 172.21.15.42:0/2015156306 wait complete. 2024-02-15T18:17:02.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:02.127677+0000 mon.smithi042 (mon.0) 7 : audit [DBG] from='client.? 172.21.15.42:0/2015156306' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-15T18:17:03.009 INFO:teuthology.orchestra.run.smithi042.stderr:mgr not available, waiting (1/15)... 2024-02-15T18:17:05.012 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status -f json-pretty 2024-02-15T18:17:06.059 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.058+0000 7fae279e1700 1 Processor -- start 2024-02-15T18:17:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.058+0000 7fae279e1700 1 -- start start 2024-02-15T18:17:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.058+0000 7fae279e1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae20070430 0x7fae200fea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.058+0000 7fae279e1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae200fef40 con 0x7fae20070430 2024-02-15T18:17:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.058+0000 7fae2577d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae20070430 0x7fae200fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.058+0000 7fae2577d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae20070430 0x7fae200fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47338/0 (socket says 172.21.15.42:47338) 2024-02-15T18:17:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.058+0000 7fae2577d700 1 -- 172.21.15.42:0/1561168549 learned_addr learned my addr 172.21.15.42:0/1561168549 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.058+0000 7fae2577d700 1 -- 172.21.15.42:0/1561168549 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae200ff080 con 0x7fae20070430 2024-02-15T18:17:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.059+0000 7fae2577d700 1 --2- 172.21.15.42:0/1561168549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae20070430 0x7fae200fea50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fae1c00b5f0 tx=0x7fae1c006ee0).ready entity=mon.0 client_cookie=3b4ea5f863aae377 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.059+0000 7fae0ffff700 1 -- 172.21.15.42:0/1561168549 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae1c00d600 con 0x7fae20070430 2024-02-15T18:17:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.059+0000 7fae0ffff700 1 -- 172.21.15.42:0/1561168549 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fae1c00dbf0 con 0x7fae20070430 2024-02-15T18:17:06.060 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.059+0000 7fae0ffff700 1 -- 172.21.15.42:0/1561168549 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae1c011cb0 con 0x7fae20070430 2024-02-15T18:17:06.062 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.060+0000 7fae279e1700 1 -- 172.21.15.42:0/1561168549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae20070430 msgr2=0x7fae200fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.060+0000 7fae279e1700 1 --2- 172.21.15.42:0/1561168549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae20070430 0x7fae200fea50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fae1c00b5f0 tx=0x7fae1c006ee0).stop 2024-02-15T18:17:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.060+0000 7fae279e1700 1 -- 172.21.15.42:0/1561168549 shutdown_connections 2024-02-15T18:17:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.060+0000 7fae279e1700 1 --2- 172.21.15.42:0/1561168549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae20070430 0x7fae200fea50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.060+0000 7fae279e1700 1 -- 172.21.15.42:0/1561168549 >> 172.21.15.42:0/1561168549 conn(0x7fae200f6530 msgr2=0x7fae200f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.060+0000 7fae279e1700 1 -- 172.21.15.42:0/1561168549 shutdown_connections 2024-02-15T18:17:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.060+0000 7fae279e1700 1 -- 172.21.15.42:0/1561168549 wait complete. 2024-02-15T18:17:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.060+0000 7fae279e1700 1 Processor -- start 2024-02-15T18:17:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.061+0000 7fae279e1700 1 -- start start 2024-02-15T18:17:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.061+0000 7fae279e1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae20070430 0x7fae2010ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.061+0000 7fae279e1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2010b080 con 0x7fae20070430 2024-02-15T18:17:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.061+0000 7fae2577d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae20070430 0x7fae2010ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.061+0000 7fae2577d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae20070430 0x7fae2010ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47352/0 (socket says 172.21.15.42:47352) 2024-02-15T18:17:06.063 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.061+0000 7fae2577d700 1 -- 172.21.15.42:0/1910021374 learned_addr learned my addr 172.21.15.42:0/1910021374 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:06.064 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.061+0000 7fae2577d700 1 -- 172.21.15.42:0/1910021374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c00b200 con 0x7fae20070430 2024-02-15T18:17:06.064 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.061+0000 7fae2577d700 1 --2- 172.21.15.42:0/1910021374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae20070430 0x7fae2010ab90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fae1c00df90 tx=0x7fae1c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:06.064 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.062+0000 7fae0dffb700 1 -- 172.21.15.42:0/1910021374 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae1c01b410 con 0x7fae20070430 2024-02-15T18:17:06.064 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.062+0000 7fae279e1700 1 -- 172.21.15.42:0/1910021374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae2010b280 con 0x7fae20070430 2024-02-15T18:17:06.064 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.062+0000 7fae0dffb700 1 -- 172.21.15.42:0/1910021374 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fae1c01ba00 con 0x7fae20070430 2024-02-15T18:17:06.064 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.062+0000 7fae0dffb700 1 -- 172.21.15.42:0/1910021374 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fae1c01aa30 con 0x7fae20070430 2024-02-15T18:17:06.064 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.062+0000 7fae279e1700 1 -- 172.21.15.42:0/1910021374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2010b710 con 0x7fae20070430 2024-02-15T18:17:06.064 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.062+0000 7fae0dffb700 1 -- 172.21.15.42:0/1910021374 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fae1c00f070 con 0x7fae20070430 2024-02-15T18:17:06.064 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.062+0000 7fae0dffb700 1 -- 172.21.15.42:0/1910021374 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fae1c025c80 con 0x7fae20070430 2024-02-15T18:17:06.064 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.063+0000 7fae279e1700 1 -- 172.21.15.42:0/1910021374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae20045ad0 con 0x7fae20070430 2024-02-15T18:17:06.065 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.065+0000 7fae0dffb700 1 -- 172.21.15.42:0/1910021374 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7fae1c016070 con 0x7fae20070430 2024-02-15T18:17:06.142 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.135+0000 7fae279e1700 1 -- 172.21.15.42:0/1910021374 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fae200fff30 con 0x7fae20070430 2024-02-15T18:17:06.142 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.135+0000 7fae0dffb700 1 -- 172.21.15.42:0/1910021374 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1230 (secure 0 0 0) 0x7fae1c025460 con 0x7fae20070430 2024-02-15T18:17:06.142 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.142 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: { 2024-02-15T18:17:06.143 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "fsid": "51458a44-cc2e-11ee-95ba-87774f69a715", 2024-02-15T18:17:06.143 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "health": { 2024-02-15T18:17:06.143 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-15T18:17:06.143 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "checks": {}, 2024-02-15T18:17:06.143 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "mutes": [] 2024-02-15T18:17:06.143 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:06.143 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-15T18:17:06.143 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "quorum": [ 2024-02-15T18:17:06.143 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 0 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: ], 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "smithi042" 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: ], 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "quorum_age": 6, 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "monmap": { 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "osdmap": { 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-15T18:17:06.144 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-15T18:17:06.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-15T18:17:06.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-15T18:17:06.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-15T18:17:06.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:06.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "pgmap": { 2024-02-15T18:17:06.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-15T18:17:06.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-15T18:17:06.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-15T18:17:06.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-15T18:17:06.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-15T18:17:06.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-15T18:17:06.145 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-15T18:17:06.146 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-15T18:17:06.146 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:06.146 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "fsmap": { 2024-02-15T18:17:06.146 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:17:06.146 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-15T18:17:06.146 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-15T18:17:06.146 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:06.146 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "available": false, 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "modules": [ 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "iostat", 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "restful" 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: ], 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "services": {} 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "servicemap": { 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "modified": "2024-02-15T18:16:53.407128+0000", 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "services": {} 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-15T18:17:06.147 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: } 2024-02-15T18:17:06.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.136+0000 7fae279e1700 1 -- 172.21.15.42:0/1910021374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae20070430 msgr2=0x7fae2010ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:06.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.136+0000 7fae279e1700 1 --2- 172.21.15.42:0/1910021374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae20070430 0x7fae2010ab90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fae1c00df90 tx=0x7fae1c006ee0).stop 2024-02-15T18:17:06.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.136+0000 7fae279e1700 1 -- 172.21.15.42:0/1910021374 shutdown_connections 2024-02-15T18:17:06.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.136+0000 7fae279e1700 1 --2- 172.21.15.42:0/1910021374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae20070430 0x7fae2010ab90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:06.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.136+0000 7fae279e1700 1 -- 172.21.15.42:0/1910021374 >> 172.21.15.42:0/1910021374 conn(0x7fae200f6530 msgr2=0x7fae200f7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:06.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.136+0000 7fae279e1700 1 -- 172.21.15.42:0/1910021374 shutdown_connections 2024-02-15T18:17:06.148 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:06.136+0000 7fae279e1700 1 -- 172.21.15.42:0/1910021374 wait complete. 2024-02-15T18:17:06.499 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:06.136655+0000 mon.smithi042 (mon.0) 8 : audit [DBG] from='client.? 172.21.15.42:0/1910021374' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-15T18:17:06.761 INFO:teuthology.orchestra.run.smithi042.stderr:mgr not available, waiting (2/15)... 2024-02-15T18:17:07.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:06.474316+0000 mon.smithi042 (mon.0) 9 : cluster [INF] Activating manager daemon smithi042.kfoihb 2024-02-15T18:17:07.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:06.476766+0000 mon.smithi042 (mon.0) 10 : cluster [DBG] mgrmap e2: smithi042.kfoihb(active, starting, since 0.00253675s) 2024-02-15T18:17:07.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:06.477570+0000 mon.smithi042 (mon.0) 11 : audit [DBG] from='mgr.14100 172.21.15.42:0/2504217797' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:17:07.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:06.477773+0000 mon.smithi042 (mon.0) 12 : audit [DBG] from='mgr.14100 172.21.15.42:0/2504217797' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:17:07.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:06.477884+0000 mon.smithi042 (mon.0) 13 : audit [DBG] from='mgr.14100 172.21.15.42:0/2504217797' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:17:07.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:06.478010+0000 mon.smithi042 (mon.0) 14 : audit [DBG] from='mgr.14100 172.21.15.42:0/2504217797' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-02-15T18:17:07.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:06.478432+0000 mon.smithi042 (mon.0) 15 : audit [DBG] from='mgr.14100 172.21.15.42:0/2504217797' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr metadata", "who": "smithi042.kfoihb", "id": "smithi042.kfoihb"}]: dispatch 2024-02-15T18:17:07.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:06.480288+0000 mon.smithi042 (mon.0) 16 : cluster [INF] Manager daemon smithi042.kfoihb is now available 2024-02-15T18:17:07.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:06.487579+0000 mon.smithi042 (mon.0) 17 : audit [INF] from='mgr.14100 172.21.15.42:0/2504217797' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:07.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:06.492739+0000 mon.smithi042 (mon.0) 18 : audit [INF] from='mgr.14100 172.21.15.42:0/2504217797' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:17:07.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:06.493898+0000 mon.smithi042 (mon.0) 19 : audit [INF] from='mgr.14100 172.21.15.42:0/2504217797' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:17:07.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:06.498610+0000 mon.smithi042 (mon.0) 20 : audit [INF] from='mgr.14100 172.21.15.42:0/2504217797' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:07.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:06.501012+0000 mon.smithi042 (mon.0) 21 : audit [INF] from='mgr.14100 172.21.15.42:0/2504217797' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:08.763 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status -f json-pretty 2024-02-15T18:17:08.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:07.479411+0000 mon.smithi042 (mon.0) 22 : cluster [DBG] mgrmap e3: smithi042.kfoihb(active, since 1.00518s) 2024-02-15T18:17:09.290 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.283+0000 7f26b39b7700 1 Processor -- start 2024-02-15T18:17:09.290 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.283+0000 7f26b39b7700 1 -- start start 2024-02-15T18:17:09.290 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.283+0000 7f26b39b7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26ac0fbea0 0x7f26ac0fc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:09.290 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.283+0000 7f26b39b7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac0fc7b0 con 0x7f26ac0fbea0 2024-02-15T18:17:09.290 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.283+0000 7f26b1753700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26ac0fbea0 0x7f26ac0fc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:09.290 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.283+0000 7f26b1753700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26ac0fbea0 0x7f26ac0fc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47418/0 (socket says 172.21.15.42:47418) 2024-02-15T18:17:09.290 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.283+0000 7f26b1753700 1 -- 172.21.15.42:0/1996858313 learned_addr learned my addr 172.21.15.42:0/1996858313 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:09.290 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.284+0000 7f26b1753700 1 -- 172.21.15.42:0/1996858313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ac0fc8f0 con 0x7f26ac0fbea0 2024-02-15T18:17:09.290 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.284+0000 7f26b1753700 1 --2- 172.21.15.42:0/1996858313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26ac0fbea0 0x7f26ac0fc2c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f26a800b5f0 tx=0x7f26a8006ee0).ready entity=mon.0 client_cookie=9d4be4cace067092 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:09.291 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.284+0000 7f269ffff700 1 -- 172.21.15.42:0/1996858313 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26a800d600 con 0x7f26ac0fbea0 2024-02-15T18:17:09.291 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.284+0000 7f269ffff700 1 -- 172.21.15.42:0/1996858313 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f26a800dbf0 con 0x7f26ac0fbea0 2024-02-15T18:17:09.291 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.284+0000 7f269ffff700 1 -- 172.21.15.42:0/1996858313 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26a8011cb0 con 0x7f26ac0fbea0 2024-02-15T18:17:09.291 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.284+0000 7f26b39b7700 1 -- 172.21.15.42:0/1996858313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26ac0fbea0 msgr2=0x7f26ac0fc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:09.291 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.284+0000 7f26b39b7700 1 --2- 172.21.15.42:0/1996858313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26ac0fbea0 0x7f26ac0fc2c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f26a800b5f0 tx=0x7f26a8006ee0).stop 2024-02-15T18:17:09.291 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.284+0000 7f26b39b7700 1 -- 172.21.15.42:0/1996858313 shutdown_connections 2024-02-15T18:17:09.291 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.284+0000 7f26b39b7700 1 --2- 172.21.15.42:0/1996858313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26ac0fbea0 0x7f26ac0fc2c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:09.291 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.284+0000 7f26b39b7700 1 -- 172.21.15.42:0/1996858313 >> 172.21.15.42:0/1996858313 conn(0x7f26ac0f76a0 msgr2=0x7f26ac0f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:09.291 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.285+0000 7f26b39b7700 1 -- 172.21.15.42:0/1996858313 shutdown_connections 2024-02-15T18:17:09.291 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.285+0000 7f26b39b7700 1 -- 172.21.15.42:0/1996858313 wait complete. 2024-02-15T18:17:09.291 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.285+0000 7f26b39b7700 1 Processor -- start 2024-02-15T18:17:09.291 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.285+0000 7f26b39b7700 1 -- start start 2024-02-15T18:17:09.291 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.286+0000 7f26b39b7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26ac0fbea0 0x7f26ac10ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:09.291 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.286+0000 7f26b39b7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac10b080 con 0x7f26ac0fbea0 2024-02-15T18:17:09.292 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.286+0000 7f26b1753700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26ac0fbea0 0x7f26ac10ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:09.292 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.286+0000 7f26b1753700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26ac0fbea0 0x7f26ac10ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47428/0 (socket says 172.21.15.42:47428) 2024-02-15T18:17:09.292 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.286+0000 7f26b1753700 1 -- 172.21.15.42:0/3517826638 learned_addr learned my addr 172.21.15.42:0/3517826638 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:09.292 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.286+0000 7f26b1753700 1 -- 172.21.15.42:0/3517826638 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a800b200 con 0x7f26ac0fbea0 2024-02-15T18:17:09.292 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.286+0000 7f26b1753700 1 --2- 172.21.15.42:0/3517826638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26ac0fbea0 0x7f26ac10ab90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f26a800bf40 tx=0x7f26a8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:09.292 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.286+0000 7f269dffb700 1 -- 172.21.15.42:0/3517826638 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26a801b570 con 0x7f26ac0fbea0 2024-02-15T18:17:09.292 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.286+0000 7f26b39b7700 1 -- 172.21.15.42:0/3517826638 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ac10b280 con 0x7f26ac0fbea0 2024-02-15T18:17:09.292 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.286+0000 7f269dffb700 1 -- 172.21.15.42:0/3517826638 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f26a801bb60 con 0x7f26ac0fbea0 2024-02-15T18:17:09.292 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.286+0000 7f269dffb700 1 -- 172.21.15.42:0/3517826638 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26a801aa20 con 0x7f26ac0fbea0 2024-02-15T18:17:09.292 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.287+0000 7f26b39b7700 1 -- 172.21.15.42:0/3517826638 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ac10b710 con 0x7f26ac0fbea0 2024-02-15T18:17:09.292 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.287+0000 7f269dffb700 1 -- 172.21.15.42:0/3517826638 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 3) v1 ==== 36029+0+0 (secure 0 0 0) 0x7f26a800f070 con 0x7f26ac0fbea0 2024-02-15T18:17:09.292 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.288+0000 7f26b39b7700 1 -- 172.21.15.42:0/3517826638 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ac045ad0 con 0x7f26ac0fbea0 2024-02-15T18:17:09.292 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.288+0000 7f269dffb700 1 --2- 172.21.15.42:0/3517826638 >> [v2:172.21.15.42:6800/1746226634,v1:172.21.15.42:6801/1746226634] conn(0x7f269402fd10 0x7f26940321d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:09.292 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.288+0000 7f269dffb700 1 -- 172.21.15.42:0/3517826638 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f26a8041320 con 0x7f26ac0fbea0 2024-02-15T18:17:09.293 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.288+0000 7f26b0f52700 1 --2- 172.21.15.42:0/3517826638 >> [v2:172.21.15.42:6800/1746226634,v1:172.21.15.42:6801/1746226634] conn(0x7f269402fd10 0x7f26940321d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:09.293 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.288+0000 7f26b0f52700 1 --2- 172.21.15.42:0/3517826638 >> [v2:172.21.15.42:6800/1746226634,v1:172.21.15.42:6801/1746226634] conn(0x7f269402fd10 0x7f26940321d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f26a00096a0 tx=0x7f26a0006b40).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:09.293 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.293+0000 7f269dffb700 1 -- 172.21.15.42:0/3517826638 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f26a8025980 con 0x7f26ac0fbea0 2024-02-15T18:17:09.478 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.477+0000 7f269dffb700 1 -- 172.21.15.42:0/3517826638 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 4) v1 ==== 36109+0+0 (secure 0 0 0) 0x7f26a8025460 con 0x7f26ac0fbea0 2024-02-15T18:17:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.510+0000 7f26b39b7700 1 -- 172.21.15.42:0/3517826638 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f26ac0fdc00 con 0x7f26ac0fbea0 2024-02-15T18:17:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.511+0000 7f269dffb700 1 -- 172.21.15.42:0/3517826638 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1230 (secure 0 0 0) 0x7f26a8022b40 con 0x7f26ac0fbea0 2024-02-15T18:17:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: { 2024-02-15T18:17:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "fsid": "51458a44-cc2e-11ee-95ba-87774f69a715", 2024-02-15T18:17:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "health": { 2024-02-15T18:17:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-15T18:17:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "checks": {}, 2024-02-15T18:17:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "mutes": [] 2024-02-15T18:17:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-15T18:17:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "quorum": [ 2024-02-15T18:17:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 0 2024-02-15T18:17:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: ], 2024-02-15T18:17:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-15T18:17:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "smithi042" 2024-02-15T18:17:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: ], 2024-02-15T18:17:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "quorum_age": 10, 2024-02-15T18:17:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "monmap": { 2024-02-15T18:17:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:17:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-15T18:17:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-15T18:17:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "osdmap": { 2024-02-15T18:17:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:17:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-15T18:17:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-15T18:17:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-15T18:17:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-15T18:17:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-15T18:17:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-15T18:17:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "pgmap": { 2024-02-15T18:17:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-15T18:17:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-15T18:17:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-15T18:17:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-15T18:17:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-15T18:17:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-15T18:17:09.519 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-15T18:17:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-15T18:17:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "fsmap": { 2024-02-15T18:17:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:17:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-15T18:17:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-15T18:17:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-15T18:17:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "available": true, 2024-02-15T18:17:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-15T18:17:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "modules": [ 2024-02-15T18:17:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "iostat", 2024-02-15T18:17:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "restful" 2024-02-15T18:17:09.520 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: ], 2024-02-15T18:17:09.521 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "services": {} 2024-02-15T18:17:09.521 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:09.521 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "servicemap": { 2024-02-15T18:17:09.521 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-15T18:17:09.522 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "modified": "2024-02-15T18:16:53.407128+0000", 2024-02-15T18:17:09.523 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "services": {} 2024-02-15T18:17:09.523 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: }, 2024-02-15T18:17:09.523 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-15T18:17:09.523 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: } 2024-02-15T18:17:09.523 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.514+0000 7f26b39b7700 1 -- 172.21.15.42:0/3517826638 >> [v2:172.21.15.42:6800/1746226634,v1:172.21.15.42:6801/1746226634] conn(0x7f269402fd10 msgr2=0x7f26940321d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:09.523 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.514+0000 7f26b39b7700 1 --2- 172.21.15.42:0/3517826638 >> [v2:172.21.15.42:6800/1746226634,v1:172.21.15.42:6801/1746226634] conn(0x7f269402fd10 0x7f26940321d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f26a00096a0 tx=0x7f26a0006b40).stop 2024-02-15T18:17:09.523 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.514+0000 7f26b39b7700 1 -- 172.21.15.42:0/3517826638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26ac0fbea0 msgr2=0x7f26ac10ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:09.523 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.514+0000 7f26b39b7700 1 --2- 172.21.15.42:0/3517826638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26ac0fbea0 0x7f26ac10ab90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f26a800bf40 tx=0x7f26a8006ee0).stop 2024-02-15T18:17:09.523 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.514+0000 7f26b39b7700 1 -- 172.21.15.42:0/3517826638 shutdown_connections 2024-02-15T18:17:09.524 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.514+0000 7f26b39b7700 1 --2- 172.21.15.42:0/3517826638 >> [v2:172.21.15.42:6800/1746226634,v1:172.21.15.42:6801/1746226634] conn(0x7f269402fd10 0x7f26940321d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:09.524 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.514+0000 7f26b39b7700 1 --2- 172.21.15.42:0/3517826638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26ac0fbea0 0x7f26ac10ab90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:09.524 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.514+0000 7f26b39b7700 1 -- 172.21.15.42:0/3517826638 >> 172.21.15.42:0/3517826638 conn(0x7f26ac0f76a0 msgr2=0x7f26ac0f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:09.524 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.514+0000 7f26b39b7700 1 -- 172.21.15.42:0/3517826638 shutdown_connections 2024-02-15T18:17:09.524 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:09.514+0000 7f26b39b7700 1 -- 172.21.15.42:0/3517826638 wait complete. 2024-02-15T18:17:10.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:09.479830+0000 mon.smithi042 (mon.0) 23 : cluster [DBG] mgrmap e4: smithi042.kfoihb(active, since 3s) 2024-02-15T18:17:10.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:09.512015+0000 mon.smithi042 (mon.0) 24 : audit [DBG] from='client.? 172.21.15.42:0/3517826638' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-15T18:17:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:mgr is available 2024-02-15T18:17:10.927 INFO:teuthology.orchestra.run.smithi042.stderr:Enabling cephadm module... 2024-02-15T18:17:10.928 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr module enable cephadm 2024-02-15T18:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.213+0000 7f6e362a1700 1 Processor -- start 2024-02-15T18:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.213+0000 7f6e362a1700 1 -- start start 2024-02-15T18:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.214+0000 7f6e362a1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e30070390 0x7f6e30100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.214+0000 7f6e362a1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300fc8e0 con 0x7f6e30070390 2024-02-15T18:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.214+0000 7f6e2ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e30070390 0x7f6e30100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.214+0000 7f6e2ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e30070390 0x7f6e30100ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53320/0 (socket says 172.21.15.42:53320) 2024-02-15T18:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.214+0000 7f6e2ffff700 1 -- 172.21.15.42:0/3351272034 learned_addr learned my addr 172.21.15.42:0/3351272034 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.214+0000 7f6e2ffff700 1 -- 172.21.15.42:0/3351272034 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e301013e0 con 0x7f6e30070390 2024-02-15T18:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.214+0000 7f6e2ffff700 1 --2- 172.21.15.42:0/3351272034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e30070390 0x7f6e30100ef0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f6e2000b5f0 tx=0x7f6e20006ee0).ready entity=mon.0 client_cookie=a986f1038e593098 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:13.626 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.214+0000 7f6e2effd700 1 -- 172.21.15.42:0/3351272034 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e2000d600 con 0x7f6e30070390 2024-02-15T18:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.214+0000 7f6e2effd700 1 -- 172.21.15.42:0/3351272034 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f6e2000dbf0 con 0x7f6e30070390 2024-02-15T18:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.215+0000 7f6e2effd700 1 -- 172.21.15.42:0/3351272034 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e20011cb0 con 0x7f6e30070390 2024-02-15T18:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.215+0000 7f6e362a1700 1 -- 172.21.15.42:0/3351272034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e30070390 msgr2=0x7f6e30100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.215+0000 7f6e362a1700 1 --2- 172.21.15.42:0/3351272034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e30070390 0x7f6e30100ef0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f6e2000b5f0 tx=0x7f6e20006ee0).stop 2024-02-15T18:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.215+0000 7f6e362a1700 1 -- 172.21.15.42:0/3351272034 shutdown_connections 2024-02-15T18:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.215+0000 7f6e362a1700 1 --2- 172.21.15.42:0/3351272034 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e30070390 0x7f6e30100ef0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.215+0000 7f6e362a1700 1 -- 172.21.15.42:0/3351272034 >> 172.21.15.42:0/3351272034 conn(0x7f6e300f6530 msgr2=0x7f6e300f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.215+0000 7f6e362a1700 1 -- 172.21.15.42:0/3351272034 shutdown_connections 2024-02-15T18:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.215+0000 7f6e362a1700 1 -- 172.21.15.42:0/3351272034 wait complete. 2024-02-15T18:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.216+0000 7f6e362a1700 1 Processor -- start 2024-02-15T18:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.216+0000 7f6e362a1700 1 -- start start 2024-02-15T18:17:13.627 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.216+0000 7f6e362a1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e30070390 0x7f6e3010cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:13.628 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.216+0000 7f6e362a1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3010d2c0 con 0x7f6e30070390 2024-02-15T18:17:13.628 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.216+0000 7f6e2ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e30070390 0x7f6e3010cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:13.628 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.216+0000 7f6e2ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e30070390 0x7f6e3010cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53330/0 (socket says 172.21.15.42:53330) 2024-02-15T18:17:13.628 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.216+0000 7f6e2ffff700 1 -- 172.21.15.42:0/3525031227 learned_addr learned my addr 172.21.15.42:0/3525031227 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:13.628 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.216+0000 7f6e2ffff700 1 -- 172.21.15.42:0/3525031227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2000b200 con 0x7f6e30070390 2024-02-15T18:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.217+0000 7f6e2ffff700 1 --2- 172.21.15.42:0/3525031227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e30070390 0x7f6e3010cdd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f6e2000df90 tx=0x7f6e20006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.217+0000 7f6e2cff9700 1 -- 172.21.15.42:0/3525031227 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e2001b410 con 0x7f6e30070390 2024-02-15T18:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.217+0000 7f6e2cff9700 1 -- 172.21.15.42:0/3525031227 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f6e2001ba00 con 0x7f6e30070390 2024-02-15T18:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.217+0000 7f6e362a1700 1 -- 172.21.15.42:0/3525031227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3010d4c0 con 0x7f6e30070390 2024-02-15T18:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.217+0000 7f6e2cff9700 1 -- 172.21.15.42:0/3525031227 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e2001aa30 con 0x7f6e30070390 2024-02-15T18:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.217+0000 7f6e362a1700 1 -- 172.21.15.42:0/3525031227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e3010e100 con 0x7f6e30070390 2024-02-15T18:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.218+0000 7f6e2cff9700 1 -- 172.21.15.42:0/3525031227 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) v1 ==== 36109+0+0 (secure 0 0 0) 0x7f6e2000f070 con 0x7f6e30070390 2024-02-15T18:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.218+0000 7f6e2cff9700 1 --2- 172.21.15.42:0/3525031227 >> [v2:172.21.15.42:6800/1746226634,v1:172.21.15.42:6801/1746226634] conn(0x7f6e1c02fdc0 0x7f6e1c032280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.218+0000 7f6e2cff9700 1 -- 172.21.15.42:0/3525031227 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f6e200410b0 con 0x7f6e30070390 2024-02-15T18:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.218+0000 7f6e362a1700 1 -- 172.21.15.42:0/3525031227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3010d950 con 0x7f6e30070390 2024-02-15T18:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.218+0000 7f6e2f7fe700 1 --2- 172.21.15.42:0/3525031227 >> [v2:172.21.15.42:6800/1746226634,v1:172.21.15.42:6801/1746226634] conn(0x7f6e1c02fdc0 0x7f6e1c032280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:13.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.218+0000 7f6e2f7fe700 1 --2- 172.21.15.42:0/3525031227 >> [v2:172.21.15.42:6800/1746226634,v1:172.21.15.42:6801/1746226634] conn(0x7f6e1c02fdc0 0x7f6e1c032280 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f6e240096a0 tx=0x7f6e24006b40).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.222+0000 7f6e2cff9700 1 -- 172.21.15.42:0/3525031227 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6e20016070 con 0x7f6e30070390 2024-02-15T18:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.422+0000 7f6e362a1700 1 -- 172.21.15.42:0/3525031227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f6e301020a0 con 0x7f6e30070390 2024-02-15T18:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.474+0000 7f6e2cff9700 1 -- 172.21.15.42:0/3525031227 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 5) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f6e2001ab90 con 0x7f6e30070390 2024-02-15T18:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.475+0000 7f6e2cff9700 1 -- 172.21.15.42:0/3525031227 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f6e2002f430 con 0x7f6e30070390 2024-02-15T18:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.477+0000 7f6e362a1700 1 -- 172.21.15.42:0/3525031227 >> [v2:172.21.15.42:6800/1746226634,v1:172.21.15.42:6801/1746226634] conn(0x7f6e1c02fdc0 msgr2=0x7f6e1c032280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.477+0000 7f6e362a1700 1 --2- 172.21.15.42:0/3525031227 >> [v2:172.21.15.42:6800/1746226634,v1:172.21.15.42:6801/1746226634] conn(0x7f6e1c02fdc0 0x7f6e1c032280 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f6e240096a0 tx=0x7f6e24006b40).stop 2024-02-15T18:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.477+0000 7f6e362a1700 1 -- 172.21.15.42:0/3525031227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e30070390 msgr2=0x7f6e3010cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.477+0000 7f6e362a1700 1 --2- 172.21.15.42:0/3525031227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e30070390 0x7f6e3010cdd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f6e2000df90 tx=0x7f6e20006ee0).stop 2024-02-15T18:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.477+0000 7f6e362a1700 1 -- 172.21.15.42:0/3525031227 shutdown_connections 2024-02-15T18:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.477+0000 7f6e362a1700 1 --2- 172.21.15.42:0/3525031227 >> [v2:172.21.15.42:6800/1746226634,v1:172.21.15.42:6801/1746226634] conn(0x7f6e1c02fdc0 0x7f6e1c032280 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:13.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.477+0000 7f6e362a1700 1 --2- 172.21.15.42:0/3525031227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e30070390 0x7f6e3010cdd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:13.631 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.477+0000 7f6e362a1700 1 -- 172.21.15.42:0/3525031227 >> 172.21.15.42:0/3525031227 conn(0x7f6e300f6530 msgr2=0x7f6e300f7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:13.631 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.478+0000 7f6e362a1700 1 -- 172.21.15.42:0/3525031227 shutdown_connections 2024-02-15T18:17:13.631 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:13.478+0000 7f6e362a1700 1 -- 172.21.15.42:0/3525031227 wait complete. 2024-02-15T18:17:13.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:13.423812+0000 mon.smithi042 (mon.0) 25 : audit [INF] from='client.? 172.21.15.42:0/3525031227' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-15T18:17:14.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:13.476786+0000 mon.smithi042 (mon.0) 26 : audit [INF] from='client.? 172.21.15.42:0/3525031227' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-15T18:17:14.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:13.476831+0000 mon.smithi042 (mon.0) 27 : cluster [DBG] mgrmap e5: smithi042.kfoihb(active, since 7s) 2024-02-15T18:17:18.573 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr stat 2024-02-15T18:17:19.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:18.618932+0000 mon.smithi042 (mon.0) 28 : cluster [INF] Active manager daemon smithi042.kfoihb restarted 2024-02-15T18:17:19.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:18.619295+0000 mon.smithi042 (mon.0) 29 : cluster [INF] Activating manager daemon smithi042.kfoihb 2024-02-15T18:17:19.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:18.621091+0000 mon.smithi042 (mon.0) 30 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2024-02-15T18:17:20.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:18.671540+0000 mon.smithi042 (mon.0) 31 : cluster [DBG] mgrmap e6: smithi042.kfoihb(active, starting, since 0.052317s) 2024-02-15T18:17:20.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:18.674028+0000 mon.smithi042 (mon.0) 32 : audit [DBG] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-02-15T18:17:20.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:18.674654+0000 mon.smithi042 (mon.0) 33 : audit [DBG] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr metadata", "who": "smithi042.kfoihb", "id": "smithi042.kfoihb"}]: dispatch 2024-02-15T18:17:20.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:18.675409+0000 mon.smithi042 (mon.0) 34 : audit [DBG] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:17:20.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:18.675645+0000 mon.smithi042 (mon.0) 35 : audit [DBG] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:17:20.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:18.676000+0000 mon.smithi042 (mon.0) 36 : audit [DBG] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:17:20.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:18.690358+0000 mon.smithi042 (mon.0) 37 : cluster [INF] Manager daemon smithi042.kfoihb is now available 2024-02-15T18:17:20.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:18.704034+0000 mon.smithi042 (mon.0) 38 : audit [INF] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:20.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:18.706629+0000 mon.smithi042 (mon.0) 39 : audit [INF] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:20.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:18.708997+0000 mon.smithi042 (mon.0) 40 : audit [INF] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:20.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:18.803427+0000 mon.smithi042 (mon.0) 41 : audit [INF] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:20.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:18.804129+0000 mon.smithi042 (mon.0) 42 : audit [DBG] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:17:20.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:18.805940+0000 mon.smithi042 (mon.0) 43 : audit [DBG] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:17:20.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:18.807942+0000 mon.smithi042 (mon.0) 44 : audit [DBG] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:17:20.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:18.817509+0000 mon.smithi042 (mon.0) 45 : audit [INF] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:17:20.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:18.818663+0000 mon.smithi042 (mon.0) 46 : audit [INF] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:17:20.133 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.132+0000 7f79e94ce700 1 Processor -- start 2024-02-15T18:17:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.132+0000 7f79e94ce700 1 -- start start 2024-02-15T18:17:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.132+0000 7f79e94ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79e40703b0 0x7f79e4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.132+0000 7f79e94ce700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e40fc8c0 con 0x7f79e40703b0 2024-02-15T18:17:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.133+0000 7f79e2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79e40703b0 0x7f79e4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.133+0000 7f79e2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79e40703b0 0x7f79e4100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53418/0 (socket says 172.21.15.42:53418) 2024-02-15T18:17:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.133+0000 7f79e2ffd700 1 -- 172.21.15.42:0/3488578432 learned_addr learned my addr 172.21.15.42:0/3488578432 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.133+0000 7f79e2ffd700 1 -- 172.21.15.42:0/3488578432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e41013c0 con 0x7f79e40703b0 2024-02-15T18:17:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.133+0000 7f79e2ffd700 1 --2- 172.21.15.42:0/3488578432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79e40703b0 0x7f79e4100ed0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f79cc00b5f0 tx=0x7f79cc006ee0).ready entity=mon.0 client_cookie=5b6decc4adf0a454 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.133+0000 7f79e1ffb700 1 -- 172.21.15.42:0/3488578432 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79cc00d600 con 0x7f79e40703b0 2024-02-15T18:17:20.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.133+0000 7f79e1ffb700 1 -- 172.21.15.42:0/3488578432 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f79cc00dbf0 con 0x7f79e40703b0 2024-02-15T18:17:20.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.133+0000 7f79e1ffb700 1 -- 172.21.15.42:0/3488578432 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79cc011cb0 con 0x7f79e40703b0 2024-02-15T18:17:20.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.134+0000 7f79e94ce700 1 -- 172.21.15.42:0/3488578432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79e40703b0 msgr2=0x7f79e4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:20.137 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.134+0000 7f79e94ce700 1 --2- 172.21.15.42:0/3488578432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79e40703b0 0x7f79e4100ed0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f79cc00b5f0 tx=0x7f79cc006ee0).stop 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.134+0000 7f79e94ce700 1 -- 172.21.15.42:0/3488578432 shutdown_connections 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.134+0000 7f79e94ce700 1 --2- 172.21.15.42:0/3488578432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79e40703b0 0x7f79e4100ed0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.134+0000 7f79e94ce700 1 -- 172.21.15.42:0/3488578432 >> 172.21.15.42:0/3488578432 conn(0x7f79e40f6530 msgr2=0x7f79e40f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.134+0000 7f79e94ce700 1 -- 172.21.15.42:0/3488578432 shutdown_connections 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.134+0000 7f79e94ce700 1 -- 172.21.15.42:0/3488578432 wait complete. 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.135+0000 7f79e94ce700 1 Processor -- start 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.135+0000 7f79e94ce700 1 -- start start 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.135+0000 7f79e94ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79e40703b0 0x7f79e410cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.135+0000 7f79e94ce700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e410d270 con 0x7f79e40703b0 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.135+0000 7f79e2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79e40703b0 0x7f79e410cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.135+0000 7f79e2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79e40703b0 0x7f79e410cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53424/0 (socket says 172.21.15.42:53424) 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.135+0000 7f79e2ffd700 1 -- 172.21.15.42:0/1246339125 learned_addr learned my addr 172.21.15.42:0/1246339125 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.135+0000 7f79e2ffd700 1 -- 172.21.15.42:0/1246339125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc00b200 con 0x7f79e40703b0 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.135+0000 7f79e2ffd700 1 --2- 172.21.15.42:0/1246339125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79e40703b0 0x7f79e410cd80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f79cc006800 tx=0x7f79cc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:20.138 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.135+0000 7f79db7fe700 1 -- 172.21.15.42:0/1246339125 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79cc01b570 con 0x7f79e40703b0 2024-02-15T18:17:20.139 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.136+0000 7f79e94ce700 1 -- 172.21.15.42:0/1246339125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e410d470 con 0x7f79e40703b0 2024-02-15T18:17:20.139 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.136+0000 7f79db7fe700 1 -- 172.21.15.42:0/1246339125 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f79cc01bb60 con 0x7f79e40703b0 2024-02-15T18:17:20.139 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.136+0000 7f79db7fe700 1 -- 172.21.15.42:0/1246339125 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f79cc01a9f0 con 0x7f79e40703b0 2024-02-15T18:17:20.139 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.136+0000 7f79e94ce700 1 -- 172.21.15.42:0/1246339125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e410e080 con 0x7f79e40703b0 2024-02-15T18:17:20.139 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.136+0000 7f79db7fe700 1 -- 172.21.15.42:0/1246339125 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) v1 ==== 36040+0+0 (secure 0 0 0) 0x7f79cc00f070 con 0x7f79e40703b0 2024-02-15T18:17:20.139 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.137+0000 7f79db7fe700 1 --2- 172.21.15.42:0/1246339125 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f79c402fd60 0x7f79c4032220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:20.139 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.137+0000 7f79db7fe700 1 -- 172.21.15.42:0/1246339125 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f79cc0421c0 con 0x7f79e40703b0 2024-02-15T18:17:20.139 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.137+0000 7f79e94ce700 1 -- 172.21.15.42:0/1246339125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e4059100 con 0x7f79e40703b0 2024-02-15T18:17:20.139 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.137+0000 7f79e27fc700 1 --2- 172.21.15.42:0/1246339125 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f79c402fd60 0x7f79c4032220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:20.139 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.137+0000 7f79e27fc700 1 --2- 172.21.15.42:0/1246339125 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f79c402fd60 0x7f79c4032220 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f79d40096a0 tx=0x7f79d4006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:20.141 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.141+0000 7f79db7fe700 1 -- 172.21.15.42:0/1246339125 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f79cc016070 con 0x7f79e40703b0 2024-02-15T18:17:20.338 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.337+0000 7f79e94ce700 1 -- 172.21.15.42:0/1246339125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f79e4045ad0 con 0x7f79e40703b0 2024-02-15T18:17:20.338 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.337+0000 7f79db7fe700 1 -- 172.21.15.42:0/1246339125 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v7) v1 ==== 56+0+103 (secure 0 0 0) 0x7f79cc010360 con 0x7f79e40703b0 2024-02-15T18:17:20.338 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: { 2024-02-15T18:17:20.339 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "epoch": 7, 2024-02-15T18:17:20.339 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "available": true, 2024-02-15T18:17:20.339 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "active_name": "smithi042.kfoihb", 2024-02-15T18:17:20.339 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-15T18:17:20.339 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: } 2024-02-15T18:17:20.340 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.339+0000 7f79e94ce700 1 -- 172.21.15.42:0/1246339125 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f79c402fd60 msgr2=0x7f79c4032220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.339+0000 7f79e94ce700 1 --2- 172.21.15.42:0/1246339125 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f79c402fd60 0x7f79c4032220 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f79d40096a0 tx=0x7f79d4006b40).stop 2024-02-15T18:17:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.339+0000 7f79e94ce700 1 -- 172.21.15.42:0/1246339125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79e40703b0 msgr2=0x7f79e410cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.339+0000 7f79e94ce700 1 --2- 172.21.15.42:0/1246339125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79e40703b0 0x7f79e410cd80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f79cc006800 tx=0x7f79cc006ee0).stop 2024-02-15T18:17:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.339+0000 7f79e94ce700 1 -- 172.21.15.42:0/1246339125 shutdown_connections 2024-02-15T18:17:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.339+0000 7f79e94ce700 1 --2- 172.21.15.42:0/1246339125 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f79c402fd60 0x7f79c4032220 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.339+0000 7f79e94ce700 1 --2- 172.21.15.42:0/1246339125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f79e40703b0 0x7f79e410cd80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.339+0000 7f79e94ce700 1 -- 172.21.15.42:0/1246339125 >> 172.21.15.42:0/1246339125 conn(0x7f79e40f6530 msgr2=0x7f79e40f7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.339+0000 7f79e94ce700 1 -- 172.21.15.42:0/1246339125 shutdown_connections 2024-02-15T18:17:20.341 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:20.339+0000 7f79e94ce700 1 -- 172.21.15.42:0/1246339125 wait complete. 2024-02-15T18:17:20.980 INFO:teuthology.orchestra.run.smithi042.stderr:Waiting for the mgr to restart... 2024-02-15T18:17:20.980 INFO:teuthology.orchestra.run.smithi042.stderr:Waiting for mgr epoch 7... 2024-02-15T18:17:20.980 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 tell mgr mgr_status 2024-02-15T18:17:20.982 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:19.679298+0000 mon.smithi042 (mon.0) 47 : cluster [DBG] mgrmap e7: smithi042.kfoihb(active, since 1.06007s) 2024-02-15T18:17:20.982 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:20.338767+0000 mon.smithi042 (mon.0) 48 : audit [DBG] from='client.? 172.21.15.42:0/1246339125' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-15T18:17:21.501 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.499+0000 7f7c4359e700 1 Processor -- start 2024-02-15T18:17:21.501 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.499+0000 7f7c4359e700 1 -- start start 2024-02-15T18:17:21.501 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.499+0000 7f7c4359e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c440703d0 0x7f7c44100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.499+0000 7f7c4359e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440fc6e0 con 0x7f7c440703d0 2024-02-15T18:17:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.499+0000 7f7c4259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c440703d0 0x7f7c44100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.499+0000 7f7c4259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c440703d0 0x7f7c44100cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37696/0 (socket says 172.21.15.42:37696) 2024-02-15T18:17:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.499+0000 7f7c4259c700 1 -- 172.21.15.42:0/548931344 learned_addr learned my addr 172.21.15.42:0/548931344 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.500+0000 7f7c4259c700 1 -- 172.21.15.42:0/548931344 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c441011e0 con 0x7f7c440703d0 2024-02-15T18:17:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.500+0000 7f7c4259c700 1 --2- 172.21.15.42:0/548931344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c440703d0 0x7f7c44100cf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f7c3400b5f0 tx=0x7f7c34006ee0).ready entity=mon.0 client_cookie=ceeee3b20f954c34 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.500+0000 7f7c4159a700 1 -- 172.21.15.42:0/548931344 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c3400d600 con 0x7f7c440703d0 2024-02-15T18:17:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.500+0000 7f7c4159a700 1 -- 172.21.15.42:0/548931344 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f7c3400dbf0 con 0x7f7c440703d0 2024-02-15T18:17:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.500+0000 7f7c4159a700 1 -- 172.21.15.42:0/548931344 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c34011cb0 con 0x7f7c440703d0 2024-02-15T18:17:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.500+0000 7f7c4359e700 1 -- 172.21.15.42:0/548931344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c440703d0 msgr2=0x7f7c44100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.500+0000 7f7c4359e700 1 --2- 172.21.15.42:0/548931344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c440703d0 0x7f7c44100cf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f7c3400b5f0 tx=0x7f7c34006ee0).stop 2024-02-15T18:17:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.501+0000 7f7c4359e700 1 -- 172.21.15.42:0/548931344 shutdown_connections 2024-02-15T18:17:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.501+0000 7f7c4359e700 1 --2- 172.21.15.42:0/548931344 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c440703d0 0x7f7c44100cf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.501+0000 7f7c4359e700 1 -- 172.21.15.42:0/548931344 >> 172.21.15.42:0/548931344 conn(0x7f7c440f6390 msgr2=0x7f7c440f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:21.503 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.501+0000 7f7c4359e700 1 -- 172.21.15.42:0/548931344 shutdown_connections 2024-02-15T18:17:21.503 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.501+0000 7f7c4359e700 1 -- 172.21.15.42:0/548931344 wait complete. 2024-02-15T18:17:21.503 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.501+0000 7f7c4359e700 1 Processor -- start 2024-02-15T18:17:21.503 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.501+0000 7f7c4359e700 1 -- start start 2024-02-15T18:17:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.501+0000 7f7c4359e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c440703d0 0x7f7c4410cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.501+0000 7f7c4359e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4410d090 con 0x7f7c440703d0 2024-02-15T18:17:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.502+0000 7f7c4259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c440703d0 0x7f7c4410cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.502+0000 7f7c4259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c440703d0 0x7f7c4410cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37706/0 (socket says 172.21.15.42:37706) 2024-02-15T18:17:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.502+0000 7f7c4259c700 1 -- 172.21.15.42:0/2771257434 learned_addr learned my addr 172.21.15.42:0/2771257434 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.502+0000 7f7c4259c700 1 -- 172.21.15.42:0/2771257434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3400b200 con 0x7f7c440703d0 2024-02-15T18:17:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.502+0000 7f7c4259c700 1 --2- 172.21.15.42:0/2771257434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c440703d0 0x7f7c4410cba0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f7c34006a20 tx=0x7f7c34006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.502+0000 7f7c32ffd700 1 -- 172.21.15.42:0/2771257434 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c3401b570 con 0x7f7c440703d0 2024-02-15T18:17:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.502+0000 7f7c32ffd700 1 -- 172.21.15.42:0/2771257434 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f7c3401bb60 con 0x7f7c440703d0 2024-02-15T18:17:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.502+0000 7f7c32ffd700 1 -- 172.21.15.42:0/2771257434 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7c3401aa20 con 0x7f7c440703d0 2024-02-15T18:17:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.502+0000 7f7c4359e700 1 -- 172.21.15.42:0/2771257434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4410d290 con 0x7f7c440703d0 2024-02-15T18:17:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.502+0000 7f7c4359e700 1 -- 172.21.15.42:0/2771257434 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4410d720 con 0x7f7c440703d0 2024-02-15T18:17:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.503+0000 7f7c32ffd700 1 -- 172.21.15.42:0/2771257434 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f7c3400f070 con 0x7f7c440703d0 2024-02-15T18:17:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.503+0000 7f7c32ffd700 1 --2- 172.21.15.42:0/2771257434 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f7c2802fe10 0x7f7c280322d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.503+0000 7f7c32ffd700 1 -- 172.21.15.42:0/2771257434 --> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c28032990 con 0x7f7c2802fe10 2024-02-15T18:17:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.503+0000 7f7c32ffd700 1 -- 172.21.15.42:0/2771257434 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f7c34041360 con 0x7f7c440703d0 2024-02-15T18:17:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.503+0000 7f7c41d9b700 1 --2- 172.21.15.42:0/2771257434 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f7c2802fe10 0x7f7c280322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.504+0000 7f7c41d9b700 1 --2- 172.21.15.42:0/2771257434 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f7c2802fe10 0x7f7c280322d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f7c380096a0 tx=0x7f7c38006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.504+0000 7f7c32ffd700 1 -- 172.21.15.42:0/2771257434 <== mgr.14114 v2:172.21.15.42:6800/3051492708 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7f7c28032990 con 0x7f7c2802fe10 2024-02-15T18:17:21.512 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.509+0000 7f7c4359e700 1 -- 172.21.15.42:0/2771257434 --> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f7c44059100 con 0x7f7c2802fe10 2024-02-15T18:17:21.512 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.510+0000 7f7c32ffd700 1 -- 172.21.15.42:0/2771257434 <== mgr.14114 v2:172.21.15.42:6800/3051492708 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f7c44059100 con 0x7f7c2802fe10 2024-02-15T18:17:21.512 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: { 2024-02-15T18:17:21.512 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "mgrmap_epoch": 8, 2024-02-15T18:17:21.512 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "initialized": true 2024-02-15T18:17:21.512 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: } 2024-02-15T18:17:21.512 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.510+0000 7f7c4359e700 1 -- 172.21.15.42:0/2771257434 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f7c2802fe10 msgr2=0x7f7c280322d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:21.512 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.510+0000 7f7c4359e700 1 --2- 172.21.15.42:0/2771257434 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f7c2802fe10 0x7f7c280322d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f7c380096a0 tx=0x7f7c38006b40).stop 2024-02-15T18:17:21.512 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.510+0000 7f7c4359e700 1 -- 172.21.15.42:0/2771257434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c440703d0 msgr2=0x7f7c4410cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:21.512 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.510+0000 7f7c4359e700 1 --2- 172.21.15.42:0/2771257434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c440703d0 0x7f7c4410cba0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f7c34006a20 tx=0x7f7c34006ee0).stop 2024-02-15T18:17:21.512 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.510+0000 7f7c4359e700 1 -- 172.21.15.42:0/2771257434 shutdown_connections 2024-02-15T18:17:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.510+0000 7f7c4359e700 1 --2- 172.21.15.42:0/2771257434 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f7c2802fe10 0x7f7c280322d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.510+0000 7f7c4359e700 1 --2- 172.21.15.42:0/2771257434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c440703d0 0x7f7c4410cba0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.510+0000 7f7c4359e700 1 -- 172.21.15.42:0/2771257434 >> 172.21.15.42:0/2771257434 conn(0x7f7c440f6390 msgr2=0x7f7c440f7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.510+0000 7f7c4359e700 1 -- 172.21.15.42:0/2771257434 shutdown_connections 2024-02-15T18:17:21.513 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:21.510+0000 7f7c4359e700 1 -- 172.21.15.42:0/2771257434 wait complete. 2024-02-15T18:17:21.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:20.680691+0000 mon.smithi042 (mon.0) 49 : cluster [DBG] mgrmap e8: smithi042.kfoihb(active, since 2s) 2024-02-15T18:17:21.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:21.505144+0000 mgr.smithi042.kfoihb (mgr.14114) 1 : audit [DBG] from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-15T18:17:21.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:21.511008+0000 mgr.smithi042.kfoihb (mgr.14114) 2 : audit [DBG] from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-15T18:17:22.111 INFO:teuthology.orchestra.run.smithi042.stderr:mgr epoch 7 is available 2024-02-15T18:17:22.111 INFO:teuthology.orchestra.run.smithi042.stderr:Setting orchestrator backend to cephadm... 2024-02-15T18:17:22.111 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch set backend cephadm 2024-02-15T18:17:22.606 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.604+0000 7fbe81cae700 1 Processor -- start 2024-02-15T18:17:22.606 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.604+0000 7fbe81cae700 1 -- start start 2024-02-15T18:17:22.606 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.605+0000 7fbe81cae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7c0703f0 0x7fbe7c0fea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:22.606 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.605+0000 7fbe81cae700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c0fef80 con 0x7fbe7c0703f0 2024-02-15T18:17:22.606 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.605+0000 7fbe7b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7c0703f0 0x7fbe7c0fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:22.606 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.605+0000 7fbe7b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7c0703f0 0x7fbe7c0fea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37722/0 (socket says 172.21.15.42:37722) 2024-02-15T18:17:22.606 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.605+0000 7fbe7b7fe700 1 -- 172.21.15.42:0/1456079291 learned_addr learned my addr 172.21.15.42:0/1456079291 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:22.606 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.605+0000 7fbe7b7fe700 1 -- 172.21.15.42:0/1456079291 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7c0ff0c0 con 0x7fbe7c0703f0 2024-02-15T18:17:22.609 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.605+0000 7fbe7b7fe700 1 --2- 172.21.15.42:0/1456079291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7c0703f0 0x7fbe7c0fea90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fbe7000b5f0 tx=0x7fbe70006ee0).ready entity=mon.0 client_cookie=e848c759e3106e6b server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:22.609 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.605+0000 7fbe7a7fc700 1 -- 172.21.15.42:0/1456079291 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe7000d600 con 0x7fbe7c0703f0 2024-02-15T18:17:22.609 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.605+0000 7fbe7a7fc700 1 -- 172.21.15.42:0/1456079291 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fbe7000dbf0 con 0x7fbe7c0703f0 2024-02-15T18:17:22.609 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.605+0000 7fbe7a7fc700 1 -- 172.21.15.42:0/1456079291 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe70011ce0 con 0x7fbe7c0703f0 2024-02-15T18:17:22.609 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.606+0000 7fbe81cae700 1 -- 172.21.15.42:0/1456079291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7c0703f0 msgr2=0x7fbe7c0fea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:22.609 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.606+0000 7fbe81cae700 1 --2- 172.21.15.42:0/1456079291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7c0703f0 0x7fbe7c0fea90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fbe7000b5f0 tx=0x7fbe70006ee0).stop 2024-02-15T18:17:22.609 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.606+0000 7fbe81cae700 1 -- 172.21.15.42:0/1456079291 shutdown_connections 2024-02-15T18:17:22.609 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.606+0000 7fbe81cae700 1 --2- 172.21.15.42:0/1456079291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7c0703f0 0x7fbe7c0fea90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:22.609 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.606+0000 7fbe81cae700 1 -- 172.21.15.42:0/1456079291 >> 172.21.15.42:0/1456079291 conn(0x7fbe7c0f6530 msgr2=0x7fbe7c0f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:22.609 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.606+0000 7fbe81cae700 1 -- 172.21.15.42:0/1456079291 shutdown_connections 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.606+0000 7fbe81cae700 1 -- 172.21.15.42:0/1456079291 wait complete. 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.607+0000 7fbe81cae700 1 Processor -- start 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.607+0000 7fbe81cae700 1 -- start start 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.607+0000 7fbe81cae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7c0703f0 0x7fbe7c10ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.607+0000 7fbe81cae700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c10b010 con 0x7fbe7c0703f0 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.607+0000 7fbe7b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7c0703f0 0x7fbe7c10ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.607+0000 7fbe7b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7c0703f0 0x7fbe7c10ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37728/0 (socket says 172.21.15.42:37728) 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.607+0000 7fbe7b7fe700 1 -- 172.21.15.42:0/1134587102 learned_addr learned my addr 172.21.15.42:0/1134587102 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.608+0000 7fbe7b7fe700 1 -- 172.21.15.42:0/1134587102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7000b200 con 0x7fbe7c0703f0 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.608+0000 7fbe7b7fe700 1 --2- 172.21.15.42:0/1134587102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7c0703f0 0x7fbe7c10ab20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbe70011ed0 tx=0x7fbe70006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.608+0000 7fbe63fff700 1 -- 172.21.15.42:0/1134587102 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe7001b570 con 0x7fbe7c0703f0 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.608+0000 7fbe63fff700 1 -- 172.21.15.42:0/1134587102 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fbe7001bb60 con 0x7fbe7c0703f0 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.608+0000 7fbe81cae700 1 -- 172.21.15.42:0/1134587102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7c10b210 con 0x7fbe7c0703f0 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.608+0000 7fbe63fff700 1 -- 172.21.15.42:0/1134587102 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbe7001aad0 con 0x7fbe7c0703f0 2024-02-15T18:17:22.610 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.608+0000 7fbe81cae700 1 -- 172.21.15.42:0/1134587102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7c10b6a0 con 0x7fbe7c0703f0 2024-02-15T18:17:22.611 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.609+0000 7fbe63fff700 1 -- 172.21.15.42:0/1134587102 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fbe7000f070 con 0x7fbe7c0703f0 2024-02-15T18:17:22.611 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.609+0000 7fbe63fff700 1 --2- 172.21.15.42:0/1134587102 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fbe6802fe10 0x7fbe680322d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:22.612 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.609+0000 7fbe63fff700 1 -- 172.21.15.42:0/1134587102 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fbe70041390 con 0x7fbe7c0703f0 2024-02-15T18:17:22.612 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.609+0000 7fbe81cae700 1 -- 172.21.15.42:0/1134587102 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe7c045ad0 con 0x7fbe7c0703f0 2024-02-15T18:17:22.612 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.609+0000 7fbe7affd700 1 --2- 172.21.15.42:0/1134587102 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fbe6802fe10 0x7fbe680322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:22.612 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.610+0000 7fbe7affd700 1 --2- 172.21.15.42:0/1134587102 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fbe6802fe10 0x7fbe680322d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fbe6c0096a0 tx=0x7fbe6c006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:22.614 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.614+0000 7fbe63fff700 1 -- 172.21.15.42:0/1134587102 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fbe70016070 con 0x7fbe7c0703f0 2024-02-15T18:17:22.806 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.806+0000 7fbe81cae700 1 -- 172.21.15.42:0/1134587102 --> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe7c10bfc0 con 0x7fbe6802fe10 2024-02-15T18:17:22.811 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.811+0000 7fbe63fff700 1 -- 172.21.15.42:0/1134587102 <== mgr.14114 v2:172.21.15.42:6800/3051492708 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbe7c10bfc0 con 0x7fbe6802fe10 2024-02-15T18:17:22.813 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.812+0000 7fbe81cae700 1 -- 172.21.15.42:0/1134587102 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fbe6802fe10 msgr2=0x7fbe680322d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:22.814 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.812+0000 7fbe81cae700 1 --2- 172.21.15.42:0/1134587102 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fbe6802fe10 0x7fbe680322d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fbe6c0096a0 tx=0x7fbe6c006b40).stop 2024-02-15T18:17:22.814 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.812+0000 7fbe81cae700 1 -- 172.21.15.42:0/1134587102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7c0703f0 msgr2=0x7fbe7c10ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:22.814 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.812+0000 7fbe81cae700 1 --2- 172.21.15.42:0/1134587102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7c0703f0 0x7fbe7c10ab20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbe70011ed0 tx=0x7fbe70006ee0).stop 2024-02-15T18:17:22.814 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.812+0000 7fbe81cae700 1 -- 172.21.15.42:0/1134587102 shutdown_connections 2024-02-15T18:17:22.814 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.812+0000 7fbe81cae700 1 --2- 172.21.15.42:0/1134587102 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fbe6802fe10 0x7fbe680322d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:22.814 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.812+0000 7fbe81cae700 1 --2- 172.21.15.42:0/1134587102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe7c0703f0 0x7fbe7c10ab20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:22.814 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.812+0000 7fbe81cae700 1 -- 172.21.15.42:0/1134587102 >> 172.21.15.42:0/1134587102 conn(0x7fbe7c0f6530 msgr2=0x7fbe7c0f7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:22.814 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.813+0000 7fbe81cae700 1 -- 172.21.15.42:0/1134587102 shutdown_connections 2024-02-15T18:17:22.814 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:22.813+0000 7fbe81cae700 1 -- 172.21.15.42:0/1134587102 wait complete. 2024-02-15T18:17:23.409 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 cephadm set-user root 2024-02-15T18:17:23.948 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.946+0000 7f5a3947a700 1 Processor -- start 2024-02-15T18:17:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.946+0000 7f5a3947a700 1 -- start start 2024-02-15T18:17:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.946+0000 7f5a3947a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a34070430 0x7f5a340fea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.946+0000 7f5a3947a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a340fef40 con 0x7f5a34070430 2024-02-15T18:17:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.947+0000 7f5a32ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a34070430 0x7f5a340fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.947+0000 7f5a32ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a34070430 0x7f5a340fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37744/0 (socket says 172.21.15.42:37744) 2024-02-15T18:17:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.947+0000 7f5a32ffd700 1 -- 172.21.15.42:0/3554486522 learned_addr learned my addr 172.21.15.42:0/3554486522 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.947+0000 7f5a32ffd700 1 -- 172.21.15.42:0/3554486522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a340ff080 con 0x7f5a34070430 2024-02-15T18:17:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.947+0000 7f5a32ffd700 1 --2- 172.21.15.42:0/3554486522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a34070430 0x7f5a340fea50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f5a2400b5f0 tx=0x7f5a24006ee0).ready entity=mon.0 client_cookie=afc928003fa05180 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.947+0000 7f5a31ffb700 1 -- 172.21.15.42:0/3554486522 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a2400d600 con 0x7f5a34070430 2024-02-15T18:17:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.947+0000 7f5a31ffb700 1 -- 172.21.15.42:0/3554486522 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f5a2400dbf0 con 0x7f5a34070430 2024-02-15T18:17:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.947+0000 7f5a31ffb700 1 -- 172.21.15.42:0/3554486522 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a24011cb0 con 0x7f5a34070430 2024-02-15T18:17:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.948+0000 7f5a3947a700 1 -- 172.21.15.42:0/3554486522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a34070430 msgr2=0x7f5a340fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:23.949 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.948+0000 7f5a3947a700 1 --2- 172.21.15.42:0/3554486522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a34070430 0x7f5a340fea50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f5a2400b5f0 tx=0x7f5a24006ee0).stop 2024-02-15T18:17:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.948+0000 7f5a3947a700 1 -- 172.21.15.42:0/3554486522 shutdown_connections 2024-02-15T18:17:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.948+0000 7f5a3947a700 1 --2- 172.21.15.42:0/3554486522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a34070430 0x7f5a340fea50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.948+0000 7f5a3947a700 1 -- 172.21.15.42:0/3554486522 >> 172.21.15.42:0/3554486522 conn(0x7f5a340f6530 msgr2=0x7f5a340f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.948+0000 7f5a3947a700 1 -- 172.21.15.42:0/3554486522 shutdown_connections 2024-02-15T18:17:23.950 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.948+0000 7f5a3947a700 1 -- 172.21.15.42:0/3554486522 wait complete. 2024-02-15T18:17:23.951 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.949+0000 7f5a3947a700 1 Processor -- start 2024-02-15T18:17:23.952 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.949+0000 7f5a3947a700 1 -- start start 2024-02-15T18:17:23.952 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.949+0000 7f5a3947a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a34070430 0x7f5a3410ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:23.952 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.949+0000 7f5a3947a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3410b030 con 0x7f5a34070430 2024-02-15T18:17:23.952 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.949+0000 7f5a32ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a34070430 0x7f5a3410ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:23.952 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.949+0000 7f5a32ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a34070430 0x7f5a3410ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37756/0 (socket says 172.21.15.42:37756) 2024-02-15T18:17:23.952 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.949+0000 7f5a32ffd700 1 -- 172.21.15.42:0/3589071762 learned_addr learned my addr 172.21.15.42:0/3589071762 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:23.952 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.950+0000 7f5a32ffd700 1 -- 172.21.15.42:0/3589071762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2400b200 con 0x7f5a34070430 2024-02-15T18:17:23.952 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.950+0000 7f5a32ffd700 1 --2- 172.21.15.42:0/3589071762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a34070430 0x7f5a3410ab40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f5a24006800 tx=0x7f5a24006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:23.952 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.950+0000 7f5a2b7fe700 1 -- 172.21.15.42:0/3589071762 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a2401b570 con 0x7f5a34070430 2024-02-15T18:17:23.952 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.950+0000 7f5a3947a700 1 -- 172.21.15.42:0/3589071762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3410b230 con 0x7f5a34070430 2024-02-15T18:17:23.952 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.950+0000 7f5a2b7fe700 1 -- 172.21.15.42:0/3589071762 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f5a2401bb60 con 0x7f5a34070430 2024-02-15T18:17:23.952 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.950+0000 7f5a2b7fe700 1 -- 172.21.15.42:0/3589071762 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a2401a9f0 con 0x7f5a34070430 2024-02-15T18:17:23.953 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.950+0000 7f5a3947a700 1 -- 172.21.15.42:0/3589071762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3410b6c0 con 0x7f5a34070430 2024-02-15T18:17:23.953 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.951+0000 7f5a2b7fe700 1 -- 172.21.15.42:0/3589071762 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f5a2400f070 con 0x7f5a34070430 2024-02-15T18:17:23.953 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.951+0000 7f5a2b7fe700 1 --2- 172.21.15.42:0/3589071762 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f5a1402fe10 0x7f5a140322d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:23.953 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.951+0000 7f5a2b7fe700 1 -- 172.21.15.42:0/3589071762 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f5a240413f0 con 0x7f5a34070430 2024-02-15T18:17:23.953 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.951+0000 7f5a327fc700 1 --2- 172.21.15.42:0/3589071762 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f5a1402fe10 0x7f5a140322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:23.953 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.951+0000 7f5a3947a700 1 -- 172.21.15.42:0/3589071762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a34045ad0 con 0x7f5a34070430 2024-02-15T18:17:23.953 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.951+0000 7f5a327fc700 1 --2- 172.21.15.42:0/3589071762 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f5a1402fe10 0x7f5a140322d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f5a1c0096a0 tx=0x7f5a1c006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:23.955 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:23.955+0000 7f5a2b7fe700 1 -- 172.21.15.42:0/3589071762 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5a24016070 con 0x7f5a34070430 2024-02-15T18:17:24.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:22.807813+0000 mgr.smithi042.kfoihb (mgr.14114) 3 : audit [DBG] from='client.14124 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:17:24.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:22.811648+0000 mon.smithi042 (mon.0) 50 : audit [INF] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:24.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:22.815067+0000 mon.smithi042 (mon.0) 51 : audit [DBG] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:17:24.115 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:24.114+0000 7f5a3947a700 1 -- 172.21.15.42:0/3589071762 --> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a3410bf20 con 0x7f5a1402fe10 2024-02-15T18:17:24.116 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:24.115+0000 7f5a2b7fe700 1 -- 172.21.15.42:0/3589071762 <== mgr.14114 v2:172.21.15.42:6800/3051492708 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f5a3410bf20 con 0x7f5a1402fe10 2024-02-15T18:17:24.116 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: value unchanged 2024-02-15T18:17:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:24.117+0000 7f5a3947a700 1 -- 172.21.15.42:0/3589071762 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f5a1402fe10 msgr2=0x7f5a140322d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:24.117+0000 7f5a3947a700 1 --2- 172.21.15.42:0/3589071762 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f5a1402fe10 0x7f5a140322d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f5a1c0096a0 tx=0x7f5a1c006b40).stop 2024-02-15T18:17:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:24.117+0000 7f5a3947a700 1 -- 172.21.15.42:0/3589071762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a34070430 msgr2=0x7f5a3410ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:24.117+0000 7f5a3947a700 1 --2- 172.21.15.42:0/3589071762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a34070430 0x7f5a3410ab40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f5a24006800 tx=0x7f5a24006ee0).stop 2024-02-15T18:17:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:24.117+0000 7f5a3947a700 1 -- 172.21.15.42:0/3589071762 shutdown_connections 2024-02-15T18:17:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:24.117+0000 7f5a3947a700 1 --2- 172.21.15.42:0/3589071762 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f5a1402fe10 0x7f5a140322d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:24.117+0000 7f5a3947a700 1 --2- 172.21.15.42:0/3589071762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a34070430 0x7f5a3410ab40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:24.117+0000 7f5a3947a700 1 -- 172.21.15.42:0/3589071762 >> 172.21.15.42:0/3589071762 conn(0x7f5a340f6530 msgr2=0x7f5a340f7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:24.118 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:24.117+0000 7f5a3947a700 1 -- 172.21.15.42:0/3589071762 shutdown_connections 2024-02-15T18:17:24.119 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:24.117+0000 7f5a3947a700 1 -- 172.21.15.42:0/3589071762 wait complete. 2024-02-15T18:17:24.640 INFO:teuthology.orchestra.run.smithi042.stderr:Generating ssh key... 2024-02-15T18:17:24.640 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 cephadm generate-key 2024-02-15T18:17:25.271 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.270+0000 7fd14b59e700 1 Processor -- start 2024-02-15T18:17:25.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.270+0000 7fd14b59e700 1 -- start start 2024-02-15T18:17:25.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.270+0000 7fd14b59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd14c0703d0 0x7fd14c100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:25.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.270+0000 7fd14b59e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0fc6e0 con 0x7fd14c0703d0 2024-02-15T18:17:25.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.270+0000 7fd14a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd14c0703d0 0x7fd14c100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:25.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.270+0000 7fd14a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd14c0703d0 0x7fd14c100cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37764/0 (socket says 172.21.15.42:37764) 2024-02-15T18:17:25.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.270+0000 7fd14a59c700 1 -- 172.21.15.42:0/3516953643 learned_addr learned my addr 172.21.15.42:0/3516953643 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:25.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.271+0000 7fd14a59c700 1 -- 172.21.15.42:0/3516953643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c1011e0 con 0x7fd14c0703d0 2024-02-15T18:17:25.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.271+0000 7fd14a59c700 1 --2- 172.21.15.42:0/3516953643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd14c0703d0 0x7fd14c100cf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fd13c00b5f0 tx=0x7fd13c006ee0).ready entity=mon.0 client_cookie=7a302fab7a544fa7 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:25.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.271+0000 7fd14959a700 1 -- 172.21.15.42:0/3516953643 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd13c00d600 con 0x7fd14c0703d0 2024-02-15T18:17:25.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.271+0000 7fd14959a700 1 -- 172.21.15.42:0/3516953643 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fd13c00dbf0 con 0x7fd14c0703d0 2024-02-15T18:17:25.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.271+0000 7fd14959a700 1 -- 172.21.15.42:0/3516953643 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd13c011cb0 con 0x7fd14c0703d0 2024-02-15T18:17:25.275 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.271+0000 7fd14b59e700 1 -- 172.21.15.42:0/3516953643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd14c0703d0 msgr2=0x7fd14c100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:25.275 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.271+0000 7fd14b59e700 1 --2- 172.21.15.42:0/3516953643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd14c0703d0 0x7fd14c100cf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fd13c00b5f0 tx=0x7fd13c006ee0).stop 2024-02-15T18:17:25.275 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.272+0000 7fd14b59e700 1 -- 172.21.15.42:0/3516953643 shutdown_connections 2024-02-15T18:17:25.276 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.272+0000 7fd14b59e700 1 --2- 172.21.15.42:0/3516953643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd14c0703d0 0x7fd14c100cf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:25.276 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.272+0000 7fd14b59e700 1 -- 172.21.15.42:0/3516953643 >> 172.21.15.42:0/3516953643 conn(0x7fd14c0f6390 msgr2=0x7fd14c0f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:25.276 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.272+0000 7fd14b59e700 1 -- 172.21.15.42:0/3516953643 shutdown_connections 2024-02-15T18:17:25.276 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.272+0000 7fd14b59e700 1 -- 172.21.15.42:0/3516953643 wait complete. 2024-02-15T18:17:25.276 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.272+0000 7fd14b59e700 1 Processor -- start 2024-02-15T18:17:25.276 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.272+0000 7fd14b59e700 1 -- start start 2024-02-15T18:17:25.276 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.272+0000 7fd14b59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd14c0703d0 0x7fd14c10cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:25.276 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.272+0000 7fd14b59e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c10d0e0 con 0x7fd14c0703d0 2024-02-15T18:17:25.276 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.273+0000 7fd14a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd14c0703d0 0x7fd14c10cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:25.276 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.273+0000 7fd14a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd14c0703d0 0x7fd14c10cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37774/0 (socket says 172.21.15.42:37774) 2024-02-15T18:17:25.276 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.273+0000 7fd14a59c700 1 -- 172.21.15.42:0/590071838 learned_addr learned my addr 172.21.15.42:0/590071838 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:25.276 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.273+0000 7fd14a59c700 1 -- 172.21.15.42:0/590071838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c00b200 con 0x7fd14c0703d0 2024-02-15T18:17:25.276 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.273+0000 7fd14a59c700 1 --2- 172.21.15.42:0/590071838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd14c0703d0 0x7fd14c10cbf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fd13c006c70 tx=0x7fd13c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:25.276 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.273+0000 7fd13affd700 1 -- 172.21.15.42:0/590071838 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd13c011400 con 0x7fd14c0703d0 2024-02-15T18:17:25.277 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.273+0000 7fd13affd700 1 -- 172.21.15.42:0/590071838 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fd13c01b8a0 con 0x7fd14c0703d0 2024-02-15T18:17:25.277 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.273+0000 7fd14b59e700 1 -- 172.21.15.42:0/590071838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c10d2e0 con 0x7fd14c0703d0 2024-02-15T18:17:25.277 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.273+0000 7fd13affd700 1 -- 172.21.15.42:0/590071838 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd13c01a7c0 con 0x7fd14c0703d0 2024-02-15T18:17:25.277 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.273+0000 7fd14b59e700 1 -- 172.21.15.42:0/590071838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c10d770 con 0x7fd14c0703d0 2024-02-15T18:17:25.277 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.274+0000 7fd13affd700 1 -- 172.21.15.42:0/590071838 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fd13c00f070 con 0x7fd14c0703d0 2024-02-15T18:17:25.277 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.274+0000 7fd13affd700 1 --2- 172.21.15.42:0/590071838 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fd13002fe10 0x7fd1300322d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:25.277 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.274+0000 7fd13affd700 1 -- 172.21.15.42:0/590071838 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fd13c041250 con 0x7fd14c0703d0 2024-02-15T18:17:25.277 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.274+0000 7fd149d9b700 1 --2- 172.21.15.42:0/590071838 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fd13002fe10 0x7fd1300322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:25.277 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.275+0000 7fd14b59e700 1 -- 172.21.15.42:0/590071838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c045ad0 con 0x7fd14c0703d0 2024-02-15T18:17:25.277 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.275+0000 7fd149d9b700 1 --2- 172.21.15.42:0/590071838 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fd13002fe10 0x7fd1300322d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fd1400096a0 tx=0x7fd140006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:25.279 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.279+0000 7fd13affd700 1 -- 172.21.15.42:0/590071838 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd13c016070 con 0x7fd14c0703d0 2024-02-15T18:17:25.439 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.438+0000 7fd14b59e700 1 -- 172.21.15.42:0/590071838 --> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd14c106c60 con 0x7fd13002fe10 2024-02-15T18:17:25.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:24.116467+0000 mgr.smithi042.kfoihb (mgr.14114) 4 : audit [DBG] from='client.14126 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:17:25.815 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.814+0000 7fd13affd700 1 -- 172.21.15.42:0/590071838 <== mgr.14114 v2:172.21.15.42:6800/3051492708 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd14c106c60 con 0x7fd13002fe10 2024-02-15T18:17:25.817 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.816+0000 7fd14b59e700 1 -- 172.21.15.42:0/590071838 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fd13002fe10 msgr2=0x7fd1300322d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:25.817 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.816+0000 7fd14b59e700 1 --2- 172.21.15.42:0/590071838 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fd13002fe10 0x7fd1300322d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fd1400096a0 tx=0x7fd140006b40).stop 2024-02-15T18:17:25.817 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.816+0000 7fd14b59e700 1 -- 172.21.15.42:0/590071838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd14c0703d0 msgr2=0x7fd14c10cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:25.817 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.816+0000 7fd14b59e700 1 --2- 172.21.15.42:0/590071838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd14c0703d0 0x7fd14c10cbf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fd13c006c70 tx=0x7fd13c006ee0).stop 2024-02-15T18:17:25.817 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.816+0000 7fd14b59e700 1 -- 172.21.15.42:0/590071838 shutdown_connections 2024-02-15T18:17:25.818 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.816+0000 7fd14b59e700 1 --2- 172.21.15.42:0/590071838 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fd13002fe10 0x7fd1300322d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:25.818 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.816+0000 7fd14b59e700 1 --2- 172.21.15.42:0/590071838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd14c0703d0 0x7fd14c10cbf0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:25.818 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.816+0000 7fd14b59e700 1 -- 172.21.15.42:0/590071838 >> 172.21.15.42:0/590071838 conn(0x7fd14c0f6390 msgr2=0x7fd14c0f7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:25.818 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.816+0000 7fd14b59e700 1 -- 172.21.15.42:0/590071838 shutdown_connections 2024-02-15T18:17:25.818 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:25.816+0000 7fd14b59e700 1 -- 172.21.15.42:0/590071838 wait complete. 2024-02-15T18:17:26.414 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 cephadm get-pub-key 2024-02-15T18:17:26.417 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:17:25.440156+0000 mgr.smithi042.kfoihb (mgr.14114) 6 : cephadm [INF] Generating ssh key... 2024-02-15T18:17:26.417 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:25.813244+0000 mon.smithi042 (mon.0) 52 : audit [INF] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:26.417 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:25.814880+0000 mon.smithi042 (mon.0) 53 : audit [INF] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:26.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.883+0000 7fb9e753e700 1 Processor -- start 2024-02-15T18:17:26.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.884+0000 7fb9e753e700 1 -- start start 2024-02-15T18:17:26.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.884+0000 7fb9e753e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e00fad90 0x7fb9e00fb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:26.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.884+0000 7fb9e753e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e00fb6a0 con 0x7fb9e00fad90 2024-02-15T18:17:26.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.884+0000 7fb9e52da700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e00fad90 0x7fb9e00fb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:26.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.884+0000 7fb9e52da700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e00fad90 0x7fb9e00fb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37786/0 (socket says 172.21.15.42:37786) 2024-02-15T18:17:26.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.884+0000 7fb9e52da700 1 -- 172.21.15.42:0/2017373964 learned_addr learned my addr 172.21.15.42:0/2017373964 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:26.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.884+0000 7fb9e52da700 1 -- 172.21.15.42:0/2017373964 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e00fb7e0 con 0x7fb9e00fad90 2024-02-15T18:17:26.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.885+0000 7fb9e52da700 1 --2- 172.21.15.42:0/2017373964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e00fad90 0x7fb9e00fb1b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fb9dc00b5f0 tx=0x7fb9dc006ee0).ready entity=mon.0 client_cookie=ab0d3f0a39191389 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:26.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.885+0000 7fb9d3fff700 1 -- 172.21.15.42:0/2017373964 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9dc00d600 con 0x7fb9e00fad90 2024-02-15T18:17:26.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.885+0000 7fb9d3fff700 1 -- 172.21.15.42:0/2017373964 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fb9dc00dbf0 con 0x7fb9e00fad90 2024-02-15T18:17:26.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.885+0000 7fb9d3fff700 1 -- 172.21.15.42:0/2017373964 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9dc011cb0 con 0x7fb9e00fad90 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.885+0000 7fb9e753e700 1 -- 172.21.15.42:0/2017373964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e00fad90 msgr2=0x7fb9e00fb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.885+0000 7fb9e753e700 1 --2- 172.21.15.42:0/2017373964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e00fad90 0x7fb9e00fb1b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fb9dc00b5f0 tx=0x7fb9dc006ee0).stop 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.885+0000 7fb9e753e700 1 -- 172.21.15.42:0/2017373964 shutdown_connections 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.885+0000 7fb9e753e700 1 --2- 172.21.15.42:0/2017373964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e00fad90 0x7fb9e00fb1b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.885+0000 7fb9e753e700 1 -- 172.21.15.42:0/2017373964 >> 172.21.15.42:0/2017373964 conn(0x7fb9e00f6530 msgr2=0x7fb9e00f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.885+0000 7fb9e753e700 1 -- 172.21.15.42:0/2017373964 shutdown_connections 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.885+0000 7fb9e753e700 1 -- 172.21.15.42:0/2017373964 wait complete. 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.886+0000 7fb9e753e700 1 Processor -- start 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.886+0000 7fb9e753e700 1 -- start start 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.886+0000 7fb9e753e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e00fad90 0x7fb9e010d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.886+0000 7fb9e753e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0070450 con 0x7fb9e00fad90 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.886+0000 7fb9e52da700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e00fad90 0x7fb9e010d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.886+0000 7fb9e52da700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e00fad90 0x7fb9e010d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37788/0 (socket says 172.21.15.42:37788) 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.886+0000 7fb9e52da700 1 -- 172.21.15.42:0/2509926500 learned_addr learned my addr 172.21.15.42:0/2509926500 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.886+0000 7fb9e52da700 1 -- 172.21.15.42:0/2509926500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9dc00b200 con 0x7fb9e00fad90 2024-02-15T18:17:26.895 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.886+0000 7fb9e52da700 1 --2- 172.21.15.42:0/2509926500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e00fad90 0x7fb9e010d850 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb9dc002f40 tx=0x7fb9dc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:26.896 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.886+0000 7fb9d1ffb700 1 -- 172.21.15.42:0/2509926500 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9dc01b410 con 0x7fb9e00fad90 2024-02-15T18:17:26.896 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.887+0000 7fb9d1ffb700 1 -- 172.21.15.42:0/2509926500 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fb9dc01ba00 con 0x7fb9e00fad90 2024-02-15T18:17:26.896 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.887+0000 7fb9e753e700 1 -- 172.21.15.42:0/2509926500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e0070650 con 0x7fb9e00fad90 2024-02-15T18:17:26.896 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.887+0000 7fb9d1ffb700 1 -- 172.21.15.42:0/2509926500 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9dc01a900 con 0x7fb9e00fad90 2024-02-15T18:17:26.896 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.887+0000 7fb9e753e700 1 -- 172.21.15.42:0/2509926500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e010df00 con 0x7fb9e00fad90 2024-02-15T18:17:26.896 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.887+0000 7fb9d1ffb700 1 -- 172.21.15.42:0/2509926500 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fb9dc00f070 con 0x7fb9e00fad90 2024-02-15T18:17:26.896 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.888+0000 7fb9d1ffb700 1 --2- 172.21.15.42:0/2509926500 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fb9c802fe70 0x7fb9c8032330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:26.896 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.888+0000 7fb9d1ffb700 1 -- 172.21.15.42:0/2509926500 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fb9dc041350 con 0x7fb9e00fad90 2024-02-15T18:17:26.897 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.888+0000 7fb9e753e700 1 -- 172.21.15.42:0/2509926500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e0045ad0 con 0x7fb9e00fad90 2024-02-15T18:17:26.897 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.888+0000 7fb9e4ad9700 1 --2- 172.21.15.42:0/2509926500 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fb9c802fe70 0x7fb9c8032330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:26.897 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.888+0000 7fb9e4ad9700 1 --2- 172.21.15.42:0/2509926500 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fb9c802fe70 0x7fb9c8032330 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fb9d40096a0 tx=0x7fb9d4006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:26.897 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:26.894+0000 7fb9d1ffb700 1 -- 172.21.15.42:0/2509926500 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb9dc016070 con 0x7fb9e00fad90 2024-02-15T18:17:27.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:27.054+0000 7fb9e753e700 1 -- 172.21.15.42:0/2509926500 --> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9e00fcb50 con 0x7fb9c802fe70 2024-02-15T18:17:27.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:27.055+0000 7fb9d1ffb700 1 -- 172.21.15.42:0/2509926500 <== mgr.14114 v2:172.21.15.42:6800/3051492708 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fb9e00fcb50 con 0x7fb9c802fe70 2024-02-15T18:17:27.055 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB57nZ/+QBwSmgjDC+P/l2s8YhtFjUbiG7YsOzwK9ID0UDwGlrBEP008ijmSXa6yJseQNacBFhFTdu3M6qoI+A+lKu2DPqPV/E0fCppCE5cdqd12ptfRxVYxoPhoVx9odHVbyQWGI3ATjLsXZIz6uE2AvJx1ZjDiigVYRC6LxFRJTaN/+xWW/NNSwPIo+8784rt9fpu9xQbEpeaSsp2xK66XkD++J/sdS84IOqG3mN7b+KbjC9eRWUuFZcgqf3cI3I+6F11wrvlBIEl5ZyFbWf6IzYNlu1V5dJAnJik+tG2Sy4dZ7VW4ouy1TnaIURlXs7hA+WMLmvMdVPCXTapbapwzRhOFV3mS5j7VW3M8+UjqYMTY7Fxl8xkVxbNN40xlbgOGxn/9gqxGqRQoEaOffU/L9DVT50fpoqXwxIvsXtghjpDuN/j/lznc2NgpOqfFerAqlUIygJRVF4kxFIoyLmXl1MEfeTurFtbwMloCdxo27NAyxmRSAoKDrNP5yL3k= ceph-51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:17:27.058 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:27.057+0000 7fb9e753e700 1 -- 172.21.15.42:0/2509926500 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fb9c802fe70 msgr2=0x7fb9c8032330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:27.058 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:27.057+0000 7fb9e753e700 1 --2- 172.21.15.42:0/2509926500 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fb9c802fe70 0x7fb9c8032330 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fb9d40096a0 tx=0x7fb9d4006b40).stop 2024-02-15T18:17:27.058 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:27.057+0000 7fb9e753e700 1 -- 172.21.15.42:0/2509926500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e00fad90 msgr2=0x7fb9e010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:27.058 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:27.057+0000 7fb9e753e700 1 --2- 172.21.15.42:0/2509926500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e00fad90 0x7fb9e010d850 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb9dc002f40 tx=0x7fb9dc006ee0).stop 2024-02-15T18:17:27.058 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:27.057+0000 7fb9e753e700 1 -- 172.21.15.42:0/2509926500 shutdown_connections 2024-02-15T18:17:27.058 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:27.057+0000 7fb9e753e700 1 --2- 172.21.15.42:0/2509926500 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fb9c802fe70 0x7fb9c8032330 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:27.058 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:27.057+0000 7fb9e753e700 1 --2- 172.21.15.42:0/2509926500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9e00fad90 0x7fb9e010d850 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:27.058 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:27.057+0000 7fb9e753e700 1 -- 172.21.15.42:0/2509926500 >> 172.21.15.42:0/2509926500 conn(0x7fb9e00f6530 msgr2=0x7fb9e00f7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:27.058 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:27.057+0000 7fb9e753e700 1 -- 172.21.15.42:0/2509926500 shutdown_connections 2024-02-15T18:17:27.058 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:27.057+0000 7fb9e753e700 1 -- 172.21.15.42:0/2509926500 wait complete. 2024-02-15T18:17:27.680 INFO:teuthology.orchestra.run.smithi042.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-15T18:17:27.680 INFO:teuthology.orchestra.run.smithi042.stderr:Adding key to root@localhost authorized_keys... 2024-02-15T18:17:27.680 INFO:teuthology.orchestra.run.smithi042.stderr:Adding host smithi042... 2024-02-15T18:17:27.681 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch host add smithi042 172.21.15.42 2024-02-15T18:17:27.682 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:27.056214+0000 mgr.smithi042.kfoihb (mgr.14114) 7 : audit [DBG] from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:17:28.181 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.178+0000 7f080b96a700 1 Processor -- start 2024-02-15T18:17:28.182 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.178+0000 7f080b96a700 1 -- start start 2024-02-15T18:17:28.182 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.179+0000 7f080b96a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08040fd560 0x7f08040fd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:28.182 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.179+0000 7f080b96a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08040fde70 con 0x7f08040fd560 2024-02-15T18:17:28.182 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.179+0000 7f0809706700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08040fd560 0x7f08040fd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:28.182 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.179+0000 7f0809706700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08040fd560 0x7f08040fd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37790/0 (socket says 172.21.15.42:37790) 2024-02-15T18:17:28.182 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.179+0000 7f0809706700 1 -- 172.21.15.42:0/1124886954 learned_addr learned my addr 172.21.15.42:0/1124886954 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:28.182 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.179+0000 7f0809706700 1 -- 172.21.15.42:0/1124886954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08040fdfb0 con 0x7f08040fd560 2024-02-15T18:17:28.182 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.179+0000 7f0809706700 1 --2- 172.21.15.42:0/1124886954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08040fd560 0x7f08040fd980 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f07f400b5f0 tx=0x7f07f4006ee0).ready entity=mon.0 client_cookie=b5450472373ca7ba server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:28.182 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.179+0000 7f0803fff700 1 -- 172.21.15.42:0/1124886954 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07f400d600 con 0x7f08040fd560 2024-02-15T18:17:28.182 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.179+0000 7f0803fff700 1 -- 172.21.15.42:0/1124886954 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f07f400dbf0 con 0x7f08040fd560 2024-02-15T18:17:28.182 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.179+0000 7f0803fff700 1 -- 172.21.15.42:0/1124886954 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07f4011cb0 con 0x7f08040fd560 2024-02-15T18:17:28.182 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.180+0000 7f080b96a700 1 -- 172.21.15.42:0/1124886954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08040fd560 msgr2=0x7f08040fd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.180+0000 7f080b96a700 1 --2- 172.21.15.42:0/1124886954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08040fd560 0x7f08040fd980 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f07f400b5f0 tx=0x7f07f4006ee0).stop 2024-02-15T18:17:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.180+0000 7f080b96a700 1 -- 172.21.15.42:0/1124886954 shutdown_connections 2024-02-15T18:17:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.180+0000 7f080b96a700 1 --2- 172.21.15.42:0/1124886954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08040fd560 0x7f08040fd980 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.180+0000 7f080b96a700 1 -- 172.21.15.42:0/1124886954 >> 172.21.15.42:0/1124886954 conn(0x7f08040f5d20 msgr2=0x7f08040f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.180+0000 7f080b96a700 1 -- 172.21.15.42:0/1124886954 shutdown_connections 2024-02-15T18:17:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.180+0000 7f080b96a700 1 -- 172.21.15.42:0/1124886954 wait complete. 2024-02-15T18:17:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.181+0000 7f080b96a700 1 Processor -- start 2024-02-15T18:17:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.181+0000 7f080b96a700 1 -- start start 2024-02-15T18:17:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.181+0000 7f080b96a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08040fd560 0x7f08040fce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.181+0000 7f080b96a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08040fb2b0 con 0x7f08040fd560 2024-02-15T18:17:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.181+0000 7f0809706700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08040fd560 0x7f08040fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.181+0000 7f0809706700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08040fd560 0x7f08040fce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37792/0 (socket says 172.21.15.42:37792) 2024-02-15T18:17:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.181+0000 7f0809706700 1 -- 172.21.15.42:0/49671868 learned_addr learned my addr 172.21.15.42:0/49671868 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.181+0000 7f0809706700 1 -- 172.21.15.42:0/49671868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f400b200 con 0x7f08040fd560 2024-02-15T18:17:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.181+0000 7f0809706700 1 --2- 172.21.15.42:0/49671868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08040fd560 0x7f08040fce30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f07f4002f40 tx=0x7f07f4006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.182+0000 7f0801ffb700 1 -- 172.21.15.42:0/49671868 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07f401b410 con 0x7f08040fd560 2024-02-15T18:17:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.182+0000 7f080b96a700 1 -- 172.21.15.42:0/49671868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08040fb450 con 0x7f08040fd560 2024-02-15T18:17:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.182+0000 7f0801ffb700 1 -- 172.21.15.42:0/49671868 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f07f401ba00 con 0x7f08040fd560 2024-02-15T18:17:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.182+0000 7f0801ffb700 1 -- 172.21.15.42:0/49671868 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f07f401a900 con 0x7f08040fd560 2024-02-15T18:17:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.182+0000 7f080b96a700 1 -- 172.21.15.42:0/49671868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08040fb8e0 con 0x7f08040fd560 2024-02-15T18:17:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.183+0000 7f0801ffb700 1 -- 172.21.15.42:0/49671868 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f07f400f070 con 0x7f08040fd560 2024-02-15T18:17:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.183+0000 7f0801ffb700 1 --2- 172.21.15.42:0/49671868 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f07f002fe10 0x7f07f00322d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.183+0000 7f080b96a700 1 -- 172.21.15.42:0/49671868 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0804045ad0 con 0x7f08040fd560 2024-02-15T18:17:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.183+0000 7f0801ffb700 1 -- 172.21.15.42:0/49671868 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f07f4041380 con 0x7f08040fd560 2024-02-15T18:17:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.183+0000 7f0808f05700 1 --2- 172.21.15.42:0/49671868 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f07f002fe10 0x7f07f00322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.183+0000 7f0808f05700 1 --2- 172.21.15.42:0/49671868 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f07f002fe10 0x7f07f00322d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f07f80096a0 tx=0x7f07f8006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:28.187 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.187+0000 7f0801ffb700 1 -- 172.21.15.42:0/49671868 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f07f4016070 con 0x7f08040fd560 2024-02-15T18:17:28.368 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.368+0000 7f080b96a700 1 -- 172.21.15.42:0/49671868 --> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}) v1 -- 0x7f08040fbb20 con 0x7f07f002fe10 2024-02-15T18:17:28.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:28.369970+0000 mgr.smithi042.kfoihb (mgr.14114) 8 : audit [DBG] from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:17:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: Added host 'smithi042' 2024-02-15T18:17:28.978 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.977+0000 7f0801ffb700 1 -- 172.21.15.42:0/49671868 <== mgr.14114 v2:172.21.15.42:6800/3051492708 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+23 (secure 0 0 0) 0x7f08040fbb20 con 0x7f07f002fe10 2024-02-15T18:17:28.980 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.979+0000 7f080b96a700 1 -- 172.21.15.42:0/49671868 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f07f002fe10 msgr2=0x7f07f00322d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:28.980 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.979+0000 7f080b96a700 1 --2- 172.21.15.42:0/49671868 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f07f002fe10 0x7f07f00322d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f07f80096a0 tx=0x7f07f8006b40).stop 2024-02-15T18:17:28.980 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.979+0000 7f080b96a700 1 -- 172.21.15.42:0/49671868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08040fd560 msgr2=0x7f08040fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:28.981 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.979+0000 7f080b96a700 1 --2- 172.21.15.42:0/49671868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08040fd560 0x7f08040fce30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f07f4002f40 tx=0x7f07f4006ee0).stop 2024-02-15T18:17:28.981 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.980+0000 7f080b96a700 1 -- 172.21.15.42:0/49671868 shutdown_connections 2024-02-15T18:17:28.981 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.980+0000 7f080b96a700 1 --2- 172.21.15.42:0/49671868 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f07f002fe10 0x7f07f00322d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:28.981 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.980+0000 7f080b96a700 1 --2- 172.21.15.42:0/49671868 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08040fd560 0x7f08040fce30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:28.981 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.980+0000 7f080b96a700 1 -- 172.21.15.42:0/49671868 >> 172.21.15.42:0/49671868 conn(0x7f08040f5d20 msgr2=0x7f08040ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:28.981 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.980+0000 7f080b96a700 1 -- 172.21.15.42:0/49671868 shutdown_connections 2024-02-15T18:17:28.981 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:28.980+0000 7f080b96a700 1 -- 172.21.15.42:0/49671868 wait complete. 2024-02-15T18:17:29.555 INFO:teuthology.orchestra.run.smithi042.stderr:Deploying mon service with default placement... 2024-02-15T18:17:29.556 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply mon 2024-02-15T18:17:30.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:28.977331+0000 mon.smithi042 (mon.0) 54 : audit [INF] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:30.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:17:28.977664+0000 mgr.smithi042.kfoihb (mgr.14114) 9 : cephadm [INF] Added host smithi042 2024-02-15T18:17:30.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:28.977908+0000 mon.smithi042 (mon.0) 55 : audit [DBG] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:17:30.104 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.103+0000 7f9b63fff700 1 Processor -- start 2024-02-15T18:17:30.106 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.103+0000 7f9b63fff700 1 -- start start 2024-02-15T18:17:30.106 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.103+0000 7f9b63fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b64070390 0x7f9b64100d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:30.106 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.103+0000 7f9b63fff700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b640fc720 con 0x7f9b64070390 2024-02-15T18:17:30.106 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.103+0000 7f9b62ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b64070390 0x7f9b64100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:30.106 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.103+0000 7f9b62ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b64070390 0x7f9b64100d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37808/0 (socket says 172.21.15.42:37808) 2024-02-15T18:17:30.106 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.103+0000 7f9b62ffd700 1 -- 172.21.15.42:0/3135481245 learned_addr learned my addr 172.21.15.42:0/3135481245 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:30.106 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.104+0000 7f9b62ffd700 1 -- 172.21.15.42:0/3135481245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b64101220 con 0x7f9b64070390 2024-02-15T18:17:30.106 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.104+0000 7f9b62ffd700 1 --2- 172.21.15.42:0/3135481245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b64070390 0x7f9b64100d30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f9b5400b5f0 tx=0x7f9b54006ee0).ready entity=mon.0 client_cookie=2c4ec5341daf4dc7 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:30.106 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.104+0000 7f9b61ffb700 1 -- 172.21.15.42:0/3135481245 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b5400d600 con 0x7f9b64070390 2024-02-15T18:17:30.106 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.104+0000 7f9b61ffb700 1 -- 172.21.15.42:0/3135481245 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f9b5400dbf0 con 0x7f9b64070390 2024-02-15T18:17:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.104+0000 7f9b61ffb700 1 -- 172.21.15.42:0/3135481245 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b54011cb0 con 0x7f9b64070390 2024-02-15T18:17:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.105+0000 7f9b63fff700 1 -- 172.21.15.42:0/3135481245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b64070390 msgr2=0x7f9b64100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.105+0000 7f9b63fff700 1 --2- 172.21.15.42:0/3135481245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b64070390 0x7f9b64100d30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f9b5400b5f0 tx=0x7f9b54006ee0).stop 2024-02-15T18:17:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.105+0000 7f9b63fff700 1 -- 172.21.15.42:0/3135481245 shutdown_connections 2024-02-15T18:17:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.105+0000 7f9b63fff700 1 --2- 172.21.15.42:0/3135481245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b64070390 0x7f9b64100d30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.105+0000 7f9b63fff700 1 -- 172.21.15.42:0/3135481245 >> 172.21.15.42:0/3135481245 conn(0x7f9b640f6390 msgr2=0x7f9b640f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.105+0000 7f9b63fff700 1 -- 172.21.15.42:0/3135481245 shutdown_connections 2024-02-15T18:17:30.107 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.105+0000 7f9b63fff700 1 -- 172.21.15.42:0/3135481245 wait complete. 2024-02-15T18:17:30.111 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.106+0000 7f9b63fff700 1 Processor -- start 2024-02-15T18:17:30.111 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.106+0000 7f9b63fff700 1 -- start start 2024-02-15T18:17:30.111 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.106+0000 7f9b63fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b64070390 0x7f9b6410cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:30.111 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.106+0000 7f9b63fff700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6410d060 con 0x7f9b64070390 2024-02-15T18:17:30.111 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.106+0000 7f9b62ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b64070390 0x7f9b6410cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:30.111 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.106+0000 7f9b62ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b64070390 0x7f9b6410cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37814/0 (socket says 172.21.15.42:37814) 2024-02-15T18:17:30.111 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.107+0000 7f9b62ffd700 1 -- 172.21.15.42:0/3932207889 learned_addr learned my addr 172.21.15.42:0/3932207889 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:30.111 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.107+0000 7f9b62ffd700 1 -- 172.21.15.42:0/3932207889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5400b200 con 0x7f9b64070390 2024-02-15T18:17:30.111 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.107+0000 7f9b62ffd700 1 --2- 172.21.15.42:0/3932207889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b64070390 0x7f9b6410cb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9b54006800 tx=0x7f9b54006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:30.111 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.107+0000 7f9b4b7fe700 1 -- 172.21.15.42:0/3932207889 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b5401b570 con 0x7f9b64070390 2024-02-15T18:17:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.107+0000 7f9b63fff700 1 -- 172.21.15.42:0/3932207889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6410d260 con 0x7f9b64070390 2024-02-15T18:17:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.107+0000 7f9b4b7fe700 1 -- 172.21.15.42:0/3932207889 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f9b5401bb60 con 0x7f9b64070390 2024-02-15T18:17:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.107+0000 7f9b4b7fe700 1 -- 172.21.15.42:0/3932207889 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b5401a9f0 con 0x7f9b64070390 2024-02-15T18:17:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.107+0000 7f9b63fff700 1 -- 172.21.15.42:0/3932207889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6410d6f0 con 0x7f9b64070390 2024-02-15T18:17:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.108+0000 7f9b4b7fe700 1 -- 172.21.15.42:0/3932207889 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f9b5400f070 con 0x7f9b64070390 2024-02-15T18:17:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.108+0000 7f9b4b7fe700 1 --2- 172.21.15.42:0/3932207889 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f9b5002fe10 0x7f9b500322d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.108+0000 7f9b4b7fe700 1 -- 172.21.15.42:0/3932207889 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f9b540422b0 con 0x7f9b64070390 2024-02-15T18:17:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.108+0000 7f9b627fc700 1 --2- 172.21.15.42:0/3932207889 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f9b5002fe10 0x7f9b500322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.108+0000 7f9b63fff700 1 -- 172.21.15.42:0/3932207889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b64045ad0 con 0x7f9b64070390 2024-02-15T18:17:30.112 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.109+0000 7f9b627fc700 1 --2- 172.21.15.42:0/3932207889 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f9b5002fe10 0x7f9b500322d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f9b580096a0 tx=0x7f9b58006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.113+0000 7f9b4b7fe700 1 -- 172.21.15.42:0/3932207889 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9b54016070 con 0x7f9b64070390 2024-02-15T18:17:30.298 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.298+0000 7f9b63fff700 1 -- 172.21.15.42:0/3932207889 --> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b6410eca0 con 0x7f9b5002fe10 2024-02-15T18:17:30.302 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.302+0000 7f9b4b7fe700 1 -- 172.21.15.42:0/3932207889 <== mgr.14114 v2:172.21.15.42:6800/3051492708 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9b6410eca0 con 0x7f9b5002fe10 2024-02-15T18:17:30.302 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: Scheduled mon update... 2024-02-15T18:17:30.304 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.303+0000 7f9b63fff700 1 -- 172.21.15.42:0/3932207889 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f9b5002fe10 msgr2=0x7f9b500322d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:30.304 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.303+0000 7f9b63fff700 1 --2- 172.21.15.42:0/3932207889 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f9b5002fe10 0x7f9b500322d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f9b580096a0 tx=0x7f9b58006b40).stop 2024-02-15T18:17:30.305 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.303+0000 7f9b63fff700 1 -- 172.21.15.42:0/3932207889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b64070390 msgr2=0x7f9b6410cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:30.305 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.303+0000 7f9b63fff700 1 --2- 172.21.15.42:0/3932207889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b64070390 0x7f9b6410cb70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9b54006800 tx=0x7f9b54006ee0).stop 2024-02-15T18:17:30.305 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.303+0000 7f9b63fff700 1 -- 172.21.15.42:0/3932207889 shutdown_connections 2024-02-15T18:17:30.305 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.303+0000 7f9b63fff700 1 --2- 172.21.15.42:0/3932207889 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f9b5002fe10 0x7f9b500322d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:30.305 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.303+0000 7f9b63fff700 1 --2- 172.21.15.42:0/3932207889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b64070390 0x7f9b6410cb70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:30.305 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.303+0000 7f9b63fff700 1 -- 172.21.15.42:0/3932207889 >> 172.21.15.42:0/3932207889 conn(0x7f9b640f6390 msgr2=0x7f9b640f7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:30.305 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.304+0000 7f9b63fff700 1 -- 172.21.15.42:0/3932207889 shutdown_connections 2024-02-15T18:17:30.305 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:30.304+0000 7f9b63fff700 1 -- 172.21.15.42:0/3932207889 wait complete. 2024-02-15T18:17:30.938 INFO:teuthology.orchestra.run.smithi042.stderr:Deploying mgr service with default placement... 2024-02-15T18:17:30.938 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply mgr 2024-02-15T18:17:31.495 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.493+0000 7fe8c0d4a700 1 Processor -- start 2024-02-15T18:17:31.495 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.493+0000 7fe8c0d4a700 1 -- start start 2024-02-15T18:17:31.496 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.494+0000 7fe8c0d4a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8bc0703b0 0x7fe8bc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:31.496 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.494+0000 7fe8c0d4a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0fc8c0 con 0x7fe8bc0703b0 2024-02-15T18:17:31.496 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.494+0000 7fe8ba59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8bc0703b0 0x7fe8bc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:31.496 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.494+0000 7fe8ba59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8bc0703b0 0x7fe8bc100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44802/0 (socket says 172.21.15.42:44802) 2024-02-15T18:17:31.496 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.494+0000 7fe8ba59c700 1 -- 172.21.15.42:0/3506958539 learned_addr learned my addr 172.21.15.42:0/3506958539 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:31.496 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.494+0000 7fe8ba59c700 1 -- 172.21.15.42:0/3506958539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc1013c0 con 0x7fe8bc0703b0 2024-02-15T18:17:31.496 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.495+0000 7fe8ba59c700 1 --2- 172.21.15.42:0/3506958539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8bc0703b0 0x7fe8bc100ed0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fe8b000b5f0 tx=0x7fe8b0006ee0).ready entity=mon.0 client_cookie=61a2e15b9268a358 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:31.496 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.495+0000 7fe8b959a700 1 -- 172.21.15.42:0/3506958539 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8b000d600 con 0x7fe8bc0703b0 2024-02-15T18:17:31.496 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.495+0000 7fe8b959a700 1 -- 172.21.15.42:0/3506958539 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fe8b000dbf0 con 0x7fe8bc0703b0 2024-02-15T18:17:31.496 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.495+0000 7fe8b959a700 1 -- 172.21.15.42:0/3506958539 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8b0011cb0 con 0x7fe8bc0703b0 2024-02-15T18:17:31.496 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.495+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/3506958539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8bc0703b0 msgr2=0x7fe8bc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:31.499 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.495+0000 7fe8c0d4a700 1 --2- 172.21.15.42:0/3506958539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8bc0703b0 0x7fe8bc100ed0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fe8b000b5f0 tx=0x7fe8b0006ee0).stop 2024-02-15T18:17:31.499 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.495+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/3506958539 shutdown_connections 2024-02-15T18:17:31.499 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.495+0000 7fe8c0d4a700 1 --2- 172.21.15.42:0/3506958539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8bc0703b0 0x7fe8bc100ed0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:31.499 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.495+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/3506958539 >> 172.21.15.42:0/3506958539 conn(0x7fe8bc0f6530 msgr2=0x7fe8bc0f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:31.499 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.496+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/3506958539 shutdown_connections 2024-02-15T18:17:31.499 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.496+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/3506958539 wait complete. 2024-02-15T18:17:31.499 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.496+0000 7fe8c0d4a700 1 Processor -- start 2024-02-15T18:17:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.496+0000 7fe8c0d4a700 1 -- start start 2024-02-15T18:17:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.496+0000 7fe8c0d4a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8bc0703b0 0x7fe8bc10cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.496+0000 7fe8c0d4a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc10d2c0 con 0x7fe8bc0703b0 2024-02-15T18:17:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.496+0000 7fe8ba59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8bc0703b0 0x7fe8bc10cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.496+0000 7fe8ba59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8bc0703b0 0x7fe8bc10cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44816/0 (socket says 172.21.15.42:44816) 2024-02-15T18:17:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.496+0000 7fe8ba59c700 1 -- 172.21.15.42:0/38163616 learned_addr learned my addr 172.21.15.42:0/38163616 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.497+0000 7fe8ba59c700 1 -- 172.21.15.42:0/38163616 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b000b200 con 0x7fe8bc0703b0 2024-02-15T18:17:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.497+0000 7fe8ba59c700 1 --2- 172.21.15.42:0/38163616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8bc0703b0 0x7fe8bc10cdd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fe8b000df90 tx=0x7fe8b0006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.497+0000 7fe8aaffd700 1 -- 172.21.15.42:0/38163616 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8b001b410 con 0x7fe8bc0703b0 2024-02-15T18:17:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.497+0000 7fe8aaffd700 1 -- 172.21.15.42:0/38163616 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fe8b001ba00 con 0x7fe8bc0703b0 2024-02-15T18:17:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.497+0000 7fe8aaffd700 1 -- 172.21.15.42:0/38163616 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe8b001aa30 con 0x7fe8bc0703b0 2024-02-15T18:17:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.497+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/38163616 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8bc10d4c0 con 0x7fe8bc0703b0 2024-02-15T18:17:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.497+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/38163616 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8bc10e100 con 0x7fe8bc0703b0 2024-02-15T18:17:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.498+0000 7fe8aaffd700 1 -- 172.21.15.42:0/38163616 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fe8b000f070 con 0x7fe8bc0703b0 2024-02-15T18:17:31.501 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.498+0000 7fe8aaffd700 1 --2- 172.21.15.42:0/38163616 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fe8a002fe70 0x7fe8a0032330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:31.501 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.498+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/38163616 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8bc10d950 con 0x7fe8bc0703b0 2024-02-15T18:17:31.501 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.498+0000 7fe8aaffd700 1 -- 172.21.15.42:0/38163616 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fe8b00411a0 con 0x7fe8bc0703b0 2024-02-15T18:17:31.501 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.498+0000 7fe8b9d9b700 1 --2- 172.21.15.42:0/38163616 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fe8a002fe70 0x7fe8a0032330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:31.501 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.499+0000 7fe8b9d9b700 1 --2- 172.21.15.42:0/38163616 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fe8a002fe70 0x7fe8a0032330 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe8ac0096a0 tx=0x7fe8ac006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:31.503 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.502+0000 7fe8aaffd700 1 -- 172.21.15.42:0/38163616 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe8b0016070 con 0x7fe8bc0703b0 2024-02-15T18:17:31.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:30.299789+0000 mgr.smithi042.kfoihb (mgr.14114) 10 : audit [DBG] from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:17:31.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:17:30.300554+0000 mgr.smithi042.kfoihb (mgr.14114) 11 : cephadm [INF] Saving service mon spec with placement count:5 2024-02-15T18:17:31.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:30.302759+0000 mon.smithi042 (mon.0) 56 : audit [INF] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:31.688 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.688+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/38163616 --> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8bc10e3b0 con 0x7fe8a002fe70 2024-02-15T18:17:31.692 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.692+0000 7fe8aaffd700 1 -- 172.21.15.42:0/38163616 <== mgr.14114 v2:172.21.15.42:6800/3051492708 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fe8bc10e3b0 con 0x7fe8a002fe70 2024-02-15T18:17:31.692 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: Scheduled mgr update... 2024-02-15T18:17:31.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.693+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/38163616 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fe8a002fe70 msgr2=0x7fe8a0032330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:31.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.693+0000 7fe8c0d4a700 1 --2- 172.21.15.42:0/38163616 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fe8a002fe70 0x7fe8a0032330 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe8ac0096a0 tx=0x7fe8ac006b40).stop 2024-02-15T18:17:31.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.693+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/38163616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8bc0703b0 msgr2=0x7fe8bc10cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:31.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.693+0000 7fe8c0d4a700 1 --2- 172.21.15.42:0/38163616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8bc0703b0 0x7fe8bc10cdd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fe8b000df90 tx=0x7fe8b0006ee0).stop 2024-02-15T18:17:31.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.693+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/38163616 shutdown_connections 2024-02-15T18:17:31.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.693+0000 7fe8c0d4a700 1 --2- 172.21.15.42:0/38163616 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fe8a002fe70 0x7fe8a0032330 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:31.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.693+0000 7fe8c0d4a700 1 --2- 172.21.15.42:0/38163616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8bc0703b0 0x7fe8bc10cdd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:31.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.693+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/38163616 >> 172.21.15.42:0/38163616 conn(0x7fe8bc0f6530 msgr2=0x7fe8bc0f7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:31.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.693+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/38163616 shutdown_connections 2024-02-15T18:17:31.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:31.693+0000 7fe8c0d4a700 1 -- 172.21.15.42:0/38163616 wait complete. 2024-02-15T18:17:32.312 INFO:teuthology.orchestra.run.smithi042.stderr:Deploying crash service with default placement... 2024-02-15T18:17:32.312 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply crash 2024-02-15T18:17:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.838+0000 7f51922bb700 1 Processor -- start 2024-02-15T18:17:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.838+0000 7f51922bb700 1 -- start start 2024-02-15T18:17:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.838+0000 7f51922bb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f518c0703f0 0x7f518c0fea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.838+0000 7f51922bb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0fef80 con 0x7f518c0703f0 2024-02-15T18:17:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.838+0000 7f518bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f518c0703f0 0x7f518c0fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.839+0000 7f518bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f518c0703f0 0x7f518c0fea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44822/0 (socket says 172.21.15.42:44822) 2024-02-15T18:17:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.839+0000 7f518bfff700 1 -- 172.21.15.42:0/4250862109 learned_addr learned my addr 172.21.15.42:0/4250862109 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.839+0000 7f518bfff700 1 -- 172.21.15.42:0/4250862109 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0ff0c0 con 0x7f518c0703f0 2024-02-15T18:17:32.851 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.839+0000 7f518bfff700 1 --2- 172.21.15.42:0/4250862109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f518c0703f0 0x7f518c0fea90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f517c00b5f0 tx=0x7f517c006ee0).ready entity=mon.0 client_cookie=680c0e3ebec7cc89 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.839+0000 7f518affd700 1 -- 172.21.15.42:0/4250862109 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f517c00d600 con 0x7f518c0703f0 2024-02-15T18:17:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.839+0000 7f518affd700 1 -- 172.21.15.42:0/4250862109 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f517c00dbf0 con 0x7f518c0703f0 2024-02-15T18:17:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.839+0000 7f518affd700 1 -- 172.21.15.42:0/4250862109 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f517c011ce0 con 0x7f518c0703f0 2024-02-15T18:17:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.840+0000 7f51922bb700 1 -- 172.21.15.42:0/4250862109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f518c0703f0 msgr2=0x7f518c0fea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.840+0000 7f51922bb700 1 --2- 172.21.15.42:0/4250862109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f518c0703f0 0x7f518c0fea90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f517c00b5f0 tx=0x7f517c006ee0).stop 2024-02-15T18:17:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.840+0000 7f51922bb700 1 -- 172.21.15.42:0/4250862109 shutdown_connections 2024-02-15T18:17:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.840+0000 7f51922bb700 1 --2- 172.21.15.42:0/4250862109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f518c0703f0 0x7f518c0fea90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.840+0000 7f51922bb700 1 -- 172.21.15.42:0/4250862109 >> 172.21.15.42:0/4250862109 conn(0x7f518c0f6530 msgr2=0x7f518c0f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.840+0000 7f51922bb700 1 -- 172.21.15.42:0/4250862109 shutdown_connections 2024-02-15T18:17:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.840+0000 7f51922bb700 1 -- 172.21.15.42:0/4250862109 wait complete. 2024-02-15T18:17:32.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.840+0000 7f51922bb700 1 Processor -- start 2024-02-15T18:17:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.840+0000 7f51922bb700 1 -- start start 2024-02-15T18:17:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.841+0000 7f51922bb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f518c0703f0 0x7f518c10ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.841+0000 7f51922bb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c10b080 con 0x7f518c0703f0 2024-02-15T18:17:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.841+0000 7f518bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f518c0703f0 0x7f518c10ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.841+0000 7f518bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f518c0703f0 0x7f518c10ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44834/0 (socket says 172.21.15.42:44834) 2024-02-15T18:17:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.841+0000 7f518bfff700 1 -- 172.21.15.42:0/404513335 learned_addr learned my addr 172.21.15.42:0/404513335 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.841+0000 7f518bfff700 1 -- 172.21.15.42:0/404513335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c00b200 con 0x7f518c0703f0 2024-02-15T18:17:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.841+0000 7f518bfff700 1 --2- 172.21.15.42:0/404513335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f518c0703f0 0x7f518c10ab90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f517c0115c0 tx=0x7f517c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.841+0000 7f5188ff9700 1 -- 172.21.15.42:0/404513335 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f517c01b410 con 0x7f518c0703f0 2024-02-15T18:17:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.841+0000 7f5188ff9700 1 -- 172.21.15.42:0/404513335 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f517c01ba00 con 0x7f518c0703f0 2024-02-15T18:17:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.841+0000 7f5188ff9700 1 -- 172.21.15.42:0/404513335 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f517c01aaa0 con 0x7f518c0703f0 2024-02-15T18:17:32.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.841+0000 7f51922bb700 1 -- 172.21.15.42:0/404513335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518c10b280 con 0x7f518c0703f0 2024-02-15T18:17:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.841+0000 7f51922bb700 1 -- 172.21.15.42:0/404513335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c10b710 con 0x7f518c0703f0 2024-02-15T18:17:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.842+0000 7f5188ff9700 1 -- 172.21.15.42:0/404513335 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f517c00f070 con 0x7f518c0703f0 2024-02-15T18:17:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.842+0000 7f5188ff9700 1 --2- 172.21.15.42:0/404513335 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f517802fe70 0x7f5178032330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.842+0000 7f5188ff9700 1 -- 172.21.15.42:0/404513335 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f517c0410b0 con 0x7f518c0703f0 2024-02-15T18:17:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.842+0000 7f51922bb700 1 -- 172.21.15.42:0/404513335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518c045ad0 con 0x7f518c0703f0 2024-02-15T18:17:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.842+0000 7f518b7fe700 1 --2- 172.21.15.42:0/404513335 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f517802fe70 0x7f5178032330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.843+0000 7f518b7fe700 1 --2- 172.21.15.42:0/404513335 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f517802fe70 0x7f5178032330 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f51800096a0 tx=0x7f5180006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:32.854 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:32.848+0000 7f5188ff9700 1 -- 172.21.15.42:0/404513335 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f517c016070 con 0x7f518c0703f0 2024-02-15T18:17:33.034 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:33.034+0000 7f51922bb700 1 -- 172.21.15.42:0/404513335 --> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f518c10bf70 con 0x7f517802fe70 2024-02-15T18:17:33.038 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:33.038+0000 7f5188ff9700 1 -- 172.21.15.42:0/404513335 <== mgr.14114 v2:172.21.15.42:6800/3051492708 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f518c10bf70 con 0x7f517802fe70 2024-02-15T18:17:33.038 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: Scheduled crash update... 2024-02-15T18:17:33.040 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:33.039+0000 7f51922bb700 1 -- 172.21.15.42:0/404513335 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f517802fe70 msgr2=0x7f5178032330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:33.041 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:33.039+0000 7f51922bb700 1 --2- 172.21.15.42:0/404513335 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f517802fe70 0x7f5178032330 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f51800096a0 tx=0x7f5180006b40).stop 2024-02-15T18:17:33.041 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:33.039+0000 7f51922bb700 1 -- 172.21.15.42:0/404513335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f518c0703f0 msgr2=0x7f518c10ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:33.041 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:33.039+0000 7f51922bb700 1 --2- 172.21.15.42:0/404513335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f518c0703f0 0x7f518c10ab90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f517c0115c0 tx=0x7f517c006ee0).stop 2024-02-15T18:17:33.041 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:33.039+0000 7f51922bb700 1 -- 172.21.15.42:0/404513335 shutdown_connections 2024-02-15T18:17:33.041 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:33.039+0000 7f51922bb700 1 --2- 172.21.15.42:0/404513335 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f517802fe70 0x7f5178032330 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:33.041 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:33.039+0000 7f51922bb700 1 --2- 172.21.15.42:0/404513335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f518c0703f0 0x7f518c10ab90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:33.041 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:33.039+0000 7f51922bb700 1 -- 172.21.15.42:0/404513335 >> 172.21.15.42:0/404513335 conn(0x7f518c0f6530 msgr2=0x7f518c0f7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:33.041 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:33.040+0000 7f51922bb700 1 -- 172.21.15.42:0/404513335 shutdown_connections 2024-02-15T18:17:33.041 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:33.040+0000 7f51922bb700 1 -- 172.21.15.42:0/404513335 wait complete. 2024-02-15T18:17:33.062 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:31.690068+0000 mgr.smithi042.kfoihb (mgr.14114) 12 : audit [DBG] from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:17:33.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:17:31.690611+0000 mgr.smithi042.kfoihb (mgr.14114) 13 : cephadm [INF] Saving service mgr spec with placement count:2 2024-02-15T18:17:33.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:31.692688+0000 mon.smithi042 (mon.0) 57 : audit [INF] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:33.702 INFO:teuthology.orchestra.run.smithi042.stderr:Enabling mgr prometheus module... 2024-02-15T18:17:33.702 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr module enable prometheus 2024-02-15T18:17:34.237 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.235+0000 7fa19af68700 1 Processor -- start 2024-02-15T18:17:34.237 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.235+0000 7fa19af68700 1 -- start start 2024-02-15T18:17:34.237 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.236+0000 7fa19af68700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1940fad90 0x7fa1940fb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:34.238 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.236+0000 7fa19af68700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1940fb6a0 con 0x7fa1940fad90 2024-02-15T18:17:34.238 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.236+0000 7fa198d04700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1940fad90 0x7fa1940fb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:34.238 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.236+0000 7fa198d04700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1940fad90 0x7fa1940fb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44848/0 (socket says 172.21.15.42:44848) 2024-02-15T18:17:34.238 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.236+0000 7fa198d04700 1 -- 172.21.15.42:0/3560433882 learned_addr learned my addr 172.21.15.42:0/3560433882 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:34.238 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.236+0000 7fa198d04700 1 -- 172.21.15.42:0/3560433882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1940fb7e0 con 0x7fa1940fad90 2024-02-15T18:17:34.238 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.236+0000 7fa198d04700 1 --2- 172.21.15.42:0/3560433882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1940fad90 0x7fa1940fb1b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fa18800b5f0 tx=0x7fa188006ee0).ready entity=mon.0 client_cookie=547675da888e3de5 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:34.238 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.236+0000 7fa1937fe700 1 -- 172.21.15.42:0/3560433882 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa18800d600 con 0x7fa1940fad90 2024-02-15T18:17:34.238 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.236+0000 7fa1937fe700 1 -- 172.21.15.42:0/3560433882 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fa18800dbf0 con 0x7fa1940fad90 2024-02-15T18:17:34.238 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.236+0000 7fa1937fe700 1 -- 172.21.15.42:0/3560433882 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa188011cb0 con 0x7fa1940fad90 2024-02-15T18:17:34.238 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.237+0000 7fa19af68700 1 -- 172.21.15.42:0/3560433882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1940fad90 msgr2=0x7fa1940fb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:34.240 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.237+0000 7fa19af68700 1 --2- 172.21.15.42:0/3560433882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1940fad90 0x7fa1940fb1b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fa18800b5f0 tx=0x7fa188006ee0).stop 2024-02-15T18:17:34.240 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.237+0000 7fa19af68700 1 -- 172.21.15.42:0/3560433882 shutdown_connections 2024-02-15T18:17:34.240 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.237+0000 7fa19af68700 1 --2- 172.21.15.42:0/3560433882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1940fad90 0x7fa1940fb1b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:34.240 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.237+0000 7fa19af68700 1 -- 172.21.15.42:0/3560433882 >> 172.21.15.42:0/3560433882 conn(0x7fa1940f6530 msgr2=0x7fa1940f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:34.240 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.237+0000 7fa19af68700 1 -- 172.21.15.42:0/3560433882 shutdown_connections 2024-02-15T18:17:34.240 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.237+0000 7fa19af68700 1 -- 172.21.15.42:0/3560433882 wait complete. 2024-02-15T18:17:34.240 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.238+0000 7fa19af68700 1 Processor -- start 2024-02-15T18:17:34.240 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.238+0000 7fa19af68700 1 -- start start 2024-02-15T18:17:34.240 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.238+0000 7fa19af68700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1940fad90 0x7fa19410cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:34.240 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.238+0000 7fa19af68700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19410d280 con 0x7fa1940fad90 2024-02-15T18:17:34.240 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.238+0000 7fa198d04700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1940fad90 0x7fa19410cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:34.240 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.238+0000 7fa198d04700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1940fad90 0x7fa19410cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44856/0 (socket says 172.21.15.42:44856) 2024-02-15T18:17:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.238+0000 7fa198d04700 1 -- 172.21.15.42:0/1810509045 learned_addr learned my addr 172.21.15.42:0/1810509045 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.238+0000 7fa198d04700 1 -- 172.21.15.42:0/1810509045 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18800b200 con 0x7fa1940fad90 2024-02-15T18:17:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.239+0000 7fa198d04700 1 --2- 172.21.15.42:0/1810509045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1940fad90 0x7fa19410cd90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fa188011ea0 tx=0x7fa188006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.239+0000 7fa1917fa700 1 -- 172.21.15.42:0/1810509045 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa18801b570 con 0x7fa1940fad90 2024-02-15T18:17:34.241 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.239+0000 7fa1917fa700 1 -- 172.21.15.42:0/1810509045 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7fa18801bb60 con 0x7fa1940fad90 2024-02-15T18:17:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.239+0000 7fa19af68700 1 -- 172.21.15.42:0/1810509045 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19410d480 con 0x7fa1940fad90 2024-02-15T18:17:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.239+0000 7fa1917fa700 1 -- 172.21.15.42:0/1810509045 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa18801a8d0 con 0x7fa1940fad90 2024-02-15T18:17:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.239+0000 7fa19af68700 1 -- 172.21.15.42:0/1810509045 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19410e090 con 0x7fa1940fad90 2024-02-15T18:17:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.240+0000 7fa1917fa700 1 -- 172.21.15.42:0/1810509045 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fa18800f070 con 0x7fa1940fad90 2024-02-15T18:17:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.240+0000 7fa1917fa700 1 --2- 172.21.15.42:0/1810509045 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fa17802fe10 0x7fa1780322d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.240+0000 7fa1917fa700 1 -- 172.21.15.42:0/1810509045 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fa188041400 con 0x7fa1940fad90 2024-02-15T18:17:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.240+0000 7fa193fff700 1 --2- 172.21.15.42:0/1810509045 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fa17802fe10 0x7fa1780322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.240+0000 7fa19af68700 1 -- 172.21.15.42:0/1810509045 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa194059100 con 0x7fa1940fad90 2024-02-15T18:17:34.242 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.240+0000 7fa193fff700 1 --2- 172.21.15.42:0/1810509045 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fa17802fe10 0x7fa1780322d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fa1800096a0 tx=0x7fa180006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:34.245 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.245+0000 7fa1917fa700 1 -- 172.21.15.42:0/1810509045 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa188016070 con 0x7fa1940fad90 2024-02-15T18:17:34.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:33.035844+0000 mgr.smithi042.kfoihb (mgr.14114) 14 : audit [DBG] from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:17:34.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:17:33.036505+0000 mgr.smithi042.kfoihb (mgr.14114) 15 : cephadm [INF] Saving service crash spec with placement * 2024-02-15T18:17:34.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:33.038722+0000 mon.smithi042 (mon.0) 58 : audit [INF] from='mgr.14114 172.21.15.42:0/3183418547' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:34.442 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:34.442+0000 7fa19af68700 1 -- 172.21.15.42:0/1810509045 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "prometheus"} v 0) v1 -- 0x7fa1940fcb10 con 0x7fa1940fad90 2024-02-15T18:17:35.209 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:35.209+0000 7fa1917fa700 1 -- 172.21.15.42:0/1810509045 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 9) v1 ==== 36134+0+0 (secure 0 0 0) 0x7fa188025980 con 0x7fa1940fad90 2024-02-15T18:17:35.211 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:35.210+0000 7fa1917fa700 1 -- 172.21.15.42:0/1810509045 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "prometheus"}]=0 v9) v1 ==== 89+0+0 (secure 0 0 0) 0x7fa18802f430 con 0x7fa1940fad90 2024-02-15T18:17:35.214 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:35.213+0000 7fa19af68700 1 -- 172.21.15.42:0/1810509045 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fa17802fe10 msgr2=0x7fa1780322d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:35.214 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:35.213+0000 7fa19af68700 1 --2- 172.21.15.42:0/1810509045 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fa17802fe10 0x7fa1780322d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fa1800096a0 tx=0x7fa180006b40).stop 2024-02-15T18:17:35.214 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:35.213+0000 7fa19af68700 1 -- 172.21.15.42:0/1810509045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1940fad90 msgr2=0x7fa19410cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:35.214 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:35.213+0000 7fa19af68700 1 --2- 172.21.15.42:0/1810509045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1940fad90 0x7fa19410cd90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fa188011ea0 tx=0x7fa188006ee0).stop 2024-02-15T18:17:35.214 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:35.213+0000 7fa19af68700 1 -- 172.21.15.42:0/1810509045 shutdown_connections 2024-02-15T18:17:35.214 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:35.213+0000 7fa19af68700 1 --2- 172.21.15.42:0/1810509045 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7fa17802fe10 0x7fa1780322d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:35.214 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:35.213+0000 7fa19af68700 1 --2- 172.21.15.42:0/1810509045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1940fad90 0x7fa19410cd90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:35.215 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:35.213+0000 7fa19af68700 1 -- 172.21.15.42:0/1810509045 >> 172.21.15.42:0/1810509045 conn(0x7fa1940f6530 msgr2=0x7fa194070050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:35.215 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:35.214+0000 7fa19af68700 1 -- 172.21.15.42:0/1810509045 shutdown_connections 2024-02-15T18:17:35.215 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:35.214+0000 7fa19af68700 1 -- 172.21.15.42:0/1810509045 wait complete. 2024-02-15T18:17:35.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:34.443384+0000 mon.smithi042 (mon.0) 59 : audit [INF] from='client.? 172.21.15.42:0/1810509045' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-15T18:17:35.905 INFO:teuthology.orchestra.run.smithi042.stderr:Deploying prometheus service with default placement... 2024-02-15T18:17:35.905 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply prometheus 2024-02-15T18:17:36.426 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.424+0000 7f3a7dbfb700 1 Processor -- start 2024-02-15T18:17:36.427 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.424+0000 7f3a7dbfb700 1 -- start start 2024-02-15T18:17:36.427 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.425+0000 7f3a7dbfb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a780fad90 0x7f3a780fb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:36.427 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.425+0000 7f3a7dbfb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a780fb6a0 con 0x7f3a780fad90 2024-02-15T18:17:36.427 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.425+0000 7f3a777fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a780fad90 0x7f3a780fb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:36.427 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.425+0000 7f3a777fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a780fad90 0x7f3a780fb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44882/0 (socket says 172.21.15.42:44882) 2024-02-15T18:17:36.427 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.425+0000 7f3a777fe700 1 -- 172.21.15.42:0/409883569 learned_addr learned my addr 172.21.15.42:0/409883569 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:36.427 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.425+0000 7f3a777fe700 1 -- 172.21.15.42:0/409883569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a780fb7e0 con 0x7f3a780fad90 2024-02-15T18:17:36.427 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.425+0000 7f3a777fe700 1 --2- 172.21.15.42:0/409883569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a780fad90 0x7f3a780fb1b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f3a6800b5f0 tx=0x7f3a68006ee0).ready entity=mon.0 client_cookie=7366886bd37471d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:36.427 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.426+0000 7f3a767fc700 1 -- 172.21.15.42:0/409883569 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a6800d600 con 0x7f3a780fad90 2024-02-15T18:17:36.427 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.426+0000 7f3a767fc700 1 -- 172.21.15.42:0/409883569 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f3a6800dbf0 con 0x7f3a780fad90 2024-02-15T18:17:36.427 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.426+0000 7f3a767fc700 1 -- 172.21.15.42:0/409883569 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a68011cb0 con 0x7f3a780fad90 2024-02-15T18:17:36.427 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.426+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/409883569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a780fad90 msgr2=0x7f3a780fb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:36.428 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.426+0000 7f3a7dbfb700 1 --2- 172.21.15.42:0/409883569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a780fad90 0x7f3a780fb1b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f3a6800b5f0 tx=0x7f3a68006ee0).stop 2024-02-15T18:17:36.428 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.426+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/409883569 shutdown_connections 2024-02-15T18:17:36.430 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.426+0000 7f3a7dbfb700 1 --2- 172.21.15.42:0/409883569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a780fad90 0x7f3a780fb1b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:36.430 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.426+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/409883569 >> 172.21.15.42:0/409883569 conn(0x7f3a780f6530 msgr2=0x7f3a780f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:36.430 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.426+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/409883569 shutdown_connections 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.426+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/409883569 wait complete. 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.427+0000 7f3a7dbfb700 1 Processor -- start 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.427+0000 7f3a7dbfb700 1 -- start start 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.427+0000 7f3a7dbfb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a780fad90 0x7f3a78108920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.427+0000 7f3a7dbfb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a78108e10 con 0x7f3a780fad90 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.428+0000 7f3a777fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a780fad90 0x7f3a78108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.428+0000 7f3a777fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a780fad90 0x7f3a78108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44896/0 (socket says 172.21.15.42:44896) 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.428+0000 7f3a777fe700 1 -- 172.21.15.42:0/3137666491 learned_addr learned my addr 172.21.15.42:0/3137666491 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.428+0000 7f3a777fe700 1 -- 172.21.15.42:0/3137666491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6800b200 con 0x7f3a780fad90 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.428+0000 7f3a777fe700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a780fad90 0x7f3a78108920 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3a68011ea0 tx=0x7f3a68006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.428+0000 7f3a5ffff700 1 -- 172.21.15.42:0/3137666491 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a6801b570 con 0x7f3a780fad90 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.428+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/3137666491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a78109010 con 0x7f3a780fad90 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.428+0000 7f3a5ffff700 1 -- 172.21.15.42:0/3137666491 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f3a6801bb60 con 0x7f3a780fad90 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.428+0000 7f3a5ffff700 1 -- 172.21.15.42:0/3137666491 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a6801a8d0 con 0x7f3a780fad90 2024-02-15T18:17:36.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.428+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/3137666491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a781094a0 con 0x7f3a780fad90 2024-02-15T18:17:36.432 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.429+0000 7f3a5ffff700 1 -- 172.21.15.42:0/3137666491 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 9) v1 ==== 36134+0+0 (secure 0 0 0) 0x7f3a6800f070 con 0x7f3a780fad90 2024-02-15T18:17:36.432 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.429+0000 7f3a5ffff700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f3a640320b0 0x7f3a64034570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:36.432 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.429+0000 7f3a5ffff700 1 -- 172.21.15.42:0/3137666491 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f3a68041580 con 0x7f3a780fad90 2024-02-15T18:17:36.432 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.429+0000 7f3a76ffd700 1 -- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f3a640320b0 msgr2=0x7f3a64034570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3051492708 2024-02-15T18:17:36.432 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.429+0000 7f3a76ffd700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f3a640320b0 0x7f3a64034570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:17:36.432 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.429+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/3137666491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a78045ad0 con 0x7f3a780fad90 2024-02-15T18:17:36.434 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.434+0000 7f3a5ffff700 1 -- 172.21.15.42:0/3137666491 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f3a68016070 con 0x7f3a780fad90 2024-02-15T18:17:36.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:35.211598+0000 mon.smithi042 (mon.0) 60 : audit [INF] from='client.? 172.21.15.42:0/1810509045' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-15T18:17:36.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:35.211699+0000 mon.smithi042 (mon.0) 61 : cluster [DBG] mgrmap e9: smithi042.kfoihb(active, since 16s) 2024-02-15T18:17:36.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.630+0000 7f3a76ffd700 1 -- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f3a640320b0 msgr2=0x7f3a64034570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3051492708 2024-02-15T18:17:36.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.630+0000 7f3a76ffd700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f3a640320b0 0x7f3a64034570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-15T18:17:36.681 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:36.681+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/3137666491 --> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a78102850 con 0x7f3a640320b0 2024-02-15T18:17:37.032 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:37.031+0000 7f3a76ffd700 1 -- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f3a640320b0 msgr2=0x7f3a64034570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3051492708 2024-02-15T18:17:37.032 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:37.031+0000 7f3a76ffd700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f3a640320b0 0x7f3a64034570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-15T18:17:37.832 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:37.832+0000 7f3a76ffd700 1 -- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f3a640320b0 msgr2=0x7f3a64034570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3051492708 2024-02-15T18:17:37.833 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:37.832+0000 7f3a76ffd700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f3a640320b0 0x7f3a64034570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-15T18:17:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:39.433+0000 7f3a76ffd700 1 -- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f3a640320b0 msgr2=0x7f3a64034570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3051492708 2024-02-15T18:17:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:39.434+0000 7f3a76ffd700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f3a640320b0 0x7f3a64034570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-02-15T18:17:40.492 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:40.491+0000 7f3a5ffff700 1 -- 172.21.15.42:0/3137666491 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 10) v1 ==== 35984+0+0 (secure 0 0 0) 0x7f3a68025980 con 0x7f3a780fad90 2024-02-15T18:17:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:40.492+0000 7f3a5ffff700 1 -- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f3a640320b0 msgr2=0x7f3a64034570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:17:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:40.492+0000 7f3a5ffff700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f3a640320b0 0x7f3a64034570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:17:40.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:40.441154+0000 mon.smithi042 (mon.0) 62 : cluster [INF] Active manager daemon smithi042.kfoihb restarted 2024-02-15T18:17:40.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:40.441525+0000 mon.smithi042 (mon.0) 63 : cluster [INF] Activating manager daemon smithi042.kfoihb 2024-02-15T18:17:40.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:40.443391+0000 mon.smithi042 (mon.0) 64 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2024-02-15T18:17:41.497 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.495+0000 7f3a5ffff700 1 -- 172.21.15.42:0/3137666491 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 11) v1 ==== 36054+0+0 (secure 0 0 0) 0x7f3a68043230 con 0x7f3a780fad90 2024-02-15T18:17:41.497 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.495+0000 7f3a5ffff700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f3a64035f20 0x7f3a64038310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:41.497 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.495+0000 7f3a5ffff700 1 -- 172.21.15.42:0/3137666491 --> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a78102850 con 0x7f3a64035f20 2024-02-15T18:17:41.497 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.496+0000 7f3a76ffd700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f3a64035f20 0x7f3a64038310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:41.497 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.496+0000 7f3a76ffd700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f3a64035f20 0x7f3a64038310 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f3a6c0036d0 tx=0x7f3a6c006cb0).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:41.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.502+0000 7f3a5ffff700 1 -- 172.21.15.42:0/3137666491 <== mgr.14142 v2:172.21.15.42:6800/481027993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f3a78102850 con 0x7f3a64035f20 2024-02-15T18:17:41.502 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: Scheduled prometheus update... 2024-02-15T18:17:41.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.504+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f3a64035f20 msgr2=0x7f3a64038310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:41.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.504+0000 7f3a7dbfb700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f3a64035f20 0x7f3a64038310 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f3a6c0036d0 tx=0x7f3a6c006cb0).stop 2024-02-15T18:17:41.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.504+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a780fad90 msgr2=0x7f3a78108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:41.505 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.504+0000 7f3a7dbfb700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a780fad90 0x7f3a78108920 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3a68011ea0 tx=0x7f3a68006ee0).stop 2024-02-15T18:17:41.506 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.504+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/3137666491 shutdown_connections 2024-02-15T18:17:41.506 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.504+0000 7f3a7dbfb700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f3a64035f20 0x7f3a64038310 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:41.506 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.504+0000 7f3a7dbfb700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a780fad90 0x7f3a78108920 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:41.506 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.504+0000 7f3a7dbfb700 1 --2- 172.21.15.42:0/3137666491 >> [v2:172.21.15.42:6800/3051492708,v1:172.21.15.42:6801/3051492708] conn(0x7f3a640320b0 0x7f3a64034570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:17:41.506 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.504+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/3137666491 >> 172.21.15.42:0/3137666491 conn(0x7f3a780f6530 msgr2=0x7f3a780ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:41.506 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.506+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/3137666491 shutdown_connections 2024-02-15T18:17:41.506 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:41.506+0000 7f3a7dbfb700 1 -- 172.21.15.42:0/3137666491 wait complete. 2024-02-15T18:17:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:40.493773+0000 mon.smithi042 (mon.0) 65 : cluster [DBG] mgrmap e10: smithi042.kfoihb(active, starting, since 0.0523307s) 2024-02-15T18:17:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:40.496595+0000 mon.smithi042 (mon.0) 66 : audit [DBG] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-02-15T18:17:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:40.497174+0000 mon.smithi042 (mon.0) 67 : audit [DBG] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr metadata", "who": "smithi042.kfoihb", "id": "smithi042.kfoihb"}]: dispatch 2024-02-15T18:17:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:40.497848+0000 mon.smithi042 (mon.0) 68 : audit [DBG] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:17:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:40.498034+0000 mon.smithi042 (mon.0) 69 : audit [DBG] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:17:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:40.498184+0000 mon.smithi042 (mon.0) 70 : audit [DBG] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:17:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:40.511553+0000 mon.smithi042 (mon.0) 71 : cluster [INF] Manager daemon smithi042.kfoihb is now available 2024-02-15T18:17:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:40.616024+0000 mon.smithi042 (mon.0) 72 : audit [DBG] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:17:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:40.617531+0000 mon.smithi042 (mon.0) 73 : audit [DBG] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:17:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:40.633584+0000 mon.smithi042 (mon.0) 74 : audit [INF] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:17:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:40.641183+0000 mon.smithi042 (mon.0) 75 : audit [INF] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:17:42.136 INFO:teuthology.orchestra.run.smithi042.stderr:Deploying grafana service with default placement... 2024-02-15T18:17:42.136 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply grafana 2024-02-15T18:17:42.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.690+0000 7fb00ee64700 1 Processor -- start 2024-02-15T18:17:42.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.690+0000 7fb00ee64700 1 -- start start 2024-02-15T18:17:42.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.690+0000 7fb00ee64700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0080703b0 0x7fb008100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:42.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.690+0000 7fb00ee64700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080fc8c0 con 0x7fb0080703b0 2024-02-15T18:17:42.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.690+0000 7fb00cc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0080703b0 0x7fb008100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:42.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.690+0000 7fb00cc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0080703b0 0x7fb008100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45330/0 (socket says 172.21.15.42:45330) 2024-02-15T18:17:42.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.690+0000 7fb00cc00700 1 -- 172.21.15.42:0/3041296538 learned_addr learned my addr 172.21.15.42:0/3041296538 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:42.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.691+0000 7fb00cc00700 1 -- 172.21.15.42:0/3041296538 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0081013c0 con 0x7fb0080703b0 2024-02-15T18:17:42.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.691+0000 7fb00cc00700 1 --2- 172.21.15.42:0/3041296538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0080703b0 0x7fb008100ed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7faffc00b5f0 tx=0x7faffc006ee0).ready entity=mon.0 client_cookie=ab5bb29574e6dcad server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:42.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.691+0000 7fb0077fe700 1 -- 172.21.15.42:0/3041296538 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faffc00d600 con 0x7fb0080703b0 2024-02-15T18:17:42.694 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.691+0000 7fb0077fe700 1 -- 172.21.15.42:0/3041296538 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7faffc00dbf0 con 0x7fb0080703b0 2024-02-15T18:17:42.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.691+0000 7fb0077fe700 1 -- 172.21.15.42:0/3041296538 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faffc011cb0 con 0x7fb0080703b0 2024-02-15T18:17:42.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.691+0000 7fb00ee64700 1 -- 172.21.15.42:0/3041296538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0080703b0 msgr2=0x7fb008100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:42.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.691+0000 7fb00ee64700 1 --2- 172.21.15.42:0/3041296538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0080703b0 0x7fb008100ed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7faffc00b5f0 tx=0x7faffc006ee0).stop 2024-02-15T18:17:42.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.692+0000 7fb00ee64700 1 -- 172.21.15.42:0/3041296538 shutdown_connections 2024-02-15T18:17:42.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.692+0000 7fb00ee64700 1 --2- 172.21.15.42:0/3041296538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0080703b0 0x7fb008100ed0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:42.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.692+0000 7fb00ee64700 1 -- 172.21.15.42:0/3041296538 >> 172.21.15.42:0/3041296538 conn(0x7fb0080f6530 msgr2=0x7fb0080f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:42.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.692+0000 7fb00ee64700 1 -- 172.21.15.42:0/3041296538 shutdown_connections 2024-02-15T18:17:42.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.692+0000 7fb00ee64700 1 -- 172.21.15.42:0/3041296538 wait complete. 2024-02-15T18:17:42.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.692+0000 7fb00ee64700 1 Processor -- start 2024-02-15T18:17:42.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.692+0000 7fb00ee64700 1 -- start start 2024-02-15T18:17:42.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.693+0000 7fb00ee64700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0080703b0 0x7fb00810cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:42.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.693+0000 7fb00ee64700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00810d2c0 con 0x7fb0080703b0 2024-02-15T18:17:42.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.693+0000 7fb00cc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0080703b0 0x7fb00810cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:42.695 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.693+0000 7fb00cc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0080703b0 0x7fb00810cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45342/0 (socket says 172.21.15.42:45342) 2024-02-15T18:17:42.696 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.693+0000 7fb00cc00700 1 -- 172.21.15.42:0/2045472135 learned_addr learned my addr 172.21.15.42:0/2045472135 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:42.696 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.693+0000 7fb00cc00700 1 -- 172.21.15.42:0/2045472135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc00b200 con 0x7fb0080703b0 2024-02-15T18:17:42.696 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.693+0000 7fb00cc00700 1 --2- 172.21.15.42:0/2045472135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0080703b0 0x7fb00810cdd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7faffc00df90 tx=0x7faffc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:42.696 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.693+0000 7fb0057fa700 1 -- 172.21.15.42:0/2045472135 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faffc01b410 con 0x7fb0080703b0 2024-02-15T18:17:42.696 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.693+0000 7fb0057fa700 1 -- 172.21.15.42:0/2045472135 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7faffc01ba00 con 0x7fb0080703b0 2024-02-15T18:17:42.697 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.693+0000 7fb00ee64700 1 -- 172.21.15.42:0/2045472135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00810d4c0 con 0x7fb0080703b0 2024-02-15T18:17:42.698 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.693+0000 7fb0057fa700 1 -- 172.21.15.42:0/2045472135 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faffc01aa30 con 0x7fb0080703b0 2024-02-15T18:17:42.698 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.694+0000 7fb00ee64700 1 -- 172.21.15.42:0/2045472135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00810e100 con 0x7fb0080703b0 2024-02-15T18:17:42.698 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.694+0000 7fb0057fa700 1 -- 172.21.15.42:0/2045472135 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7faffc00f070 con 0x7fb0080703b0 2024-02-15T18:17:42.698 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.694+0000 7fb0057fa700 1 --2- 172.21.15.42:0/2045472135 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7fafec02ff70 0x7fafec032430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:42.698 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.694+0000 7fb0057fa700 1 -- 172.21.15.42:0/2045472135 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7faffc0412e0 con 0x7fb0080703b0 2024-02-15T18:17:42.698 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.695+0000 7fb007fff700 1 --2- 172.21.15.42:0/2045472135 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7fafec02ff70 0x7fafec032430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:42.698 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.695+0000 7fb00ee64700 1 -- 172.21.15.42:0/2045472135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00810d950 con 0x7fb0080703b0 2024-02-15T18:17:42.698 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.695+0000 7fb007fff700 1 --2- 172.21.15.42:0/2045472135 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7fafec02ff70 0x7fafec032430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7faff40096a0 tx=0x7faff4006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:42.699 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.699+0000 7fb0057fa700 1 -- 172.21.15.42:0/2045472135 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7faffc022d20 con 0x7fb0080703b0 2024-02-15T18:17:42.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:41.498547+0000 mon.smithi042 (mon.0) 76 : cluster [DBG] mgrmap e11: smithi042.kfoihb(active, since 1.05711s) 2024-02-15T18:17:42.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:17:41.499966+0000 mgr.smithi042.kfoihb (mgr.14142) 2 : cephadm [INF] Saving service prometheus spec with placement count:1 2024-02-15T18:17:42.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:41.502946+0000 mon.smithi042 (mon.0) 77 : audit [INF] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:42.887 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.886+0000 7fb00ee64700 1 -- 172.21.15.42:0/2045472135 --> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fb008102150 con 0x7fafec02ff70 2024-02-15T18:17:42.891 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.890+0000 7fb0057fa700 1 -- 172.21.15.42:0/2045472135 <== mgr.14142 v2:172.21.15.42:6800/481027993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fb008102150 con 0x7fafec02ff70 2024-02-15T18:17:42.891 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: Scheduled grafana update... 2024-02-15T18:17:42.893 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.892+0000 7fb00ee64700 1 -- 172.21.15.42:0/2045472135 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7fafec02ff70 msgr2=0x7fafec032430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:42.893 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.892+0000 7fb00ee64700 1 --2- 172.21.15.42:0/2045472135 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7fafec02ff70 0x7fafec032430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7faff40096a0 tx=0x7faff4006b40).stop 2024-02-15T18:17:42.893 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.892+0000 7fb00ee64700 1 -- 172.21.15.42:0/2045472135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0080703b0 msgr2=0x7fb00810cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:42.893 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.892+0000 7fb00ee64700 1 --2- 172.21.15.42:0/2045472135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0080703b0 0x7fb00810cdd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7faffc00df90 tx=0x7faffc006ee0).stop 2024-02-15T18:17:42.893 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.892+0000 7fb00ee64700 1 -- 172.21.15.42:0/2045472135 shutdown_connections 2024-02-15T18:17:42.893 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.892+0000 7fb00ee64700 1 --2- 172.21.15.42:0/2045472135 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7fafec02ff70 0x7fafec032430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:42.893 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.892+0000 7fb00ee64700 1 --2- 172.21.15.42:0/2045472135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0080703b0 0x7fb00810cdd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:42.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.892+0000 7fb00ee64700 1 -- 172.21.15.42:0/2045472135 >> 172.21.15.42:0/2045472135 conn(0x7fb0080f6530 msgr2=0x7fb0080f7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:42.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.892+0000 7fb00ee64700 1 -- 172.21.15.42:0/2045472135 shutdown_connections 2024-02-15T18:17:42.894 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:42.892+0000 7fb00ee64700 1 -- 172.21.15.42:0/2045472135 wait complete. 2024-02-15T18:17:43.502 INFO:teuthology.orchestra.run.smithi042.stderr:Deploying node-exporter service with default placement... 2024-02-15T18:17:43.502 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply node-exporter 2024-02-15T18:17:43.760 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:42.505207+0000 mon.smithi042 (mon.0) 78 : cluster [DBG] mgrmap e12: smithi042.kfoihb(active, since 2s) 2024-02-15T18:17:43.760 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:42.888585+0000 mgr.smithi042.kfoihb (mgr.14142) 3 : audit [DBG] from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:17:43.761 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:17:42.889109+0000 mgr.smithi042.kfoihb (mgr.14142) 4 : cephadm [INF] Saving service grafana spec with placement count:1 2024-02-15T18:17:43.761 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:42.891384+0000 mon.smithi042 (mon.0) 79 : audit [INF] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:44.066 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.062+0000 7f0099be8700 1 Processor -- start 2024-02-15T18:17:44.066 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.062+0000 7f0099be8700 1 -- start start 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.062+0000 7f0099be8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00940fbea0 0x7f00940fc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.062+0000 7f0099be8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00940fc7b0 con 0x7f00940fbea0 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.063+0000 7f00937fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00940fbea0 0x7f00940fc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.063+0000 7f00937fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00940fbea0 0x7f00940fc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45350/0 (socket says 172.21.15.42:45350) 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.063+0000 7f00937fe700 1 -- 172.21.15.42:0/2422798016 learned_addr learned my addr 172.21.15.42:0/2422798016 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.063+0000 7f00937fe700 1 -- 172.21.15.42:0/2422798016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00940fc8f0 con 0x7f00940fbea0 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.063+0000 7f00937fe700 1 --2- 172.21.15.42:0/2422798016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00940fbea0 0x7f00940fc2c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f0084009960 tx=0x7f00840092d0).ready entity=mon.0 client_cookie=54364fb6a27ecb3b server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.063+0000 7f00927fc700 1 -- 172.21.15.42:0/2422798016 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f008400b9c0 con 0x7f00940fbea0 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.063+0000 7f00927fc700 1 -- 172.21.15.42:0/2422798016 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f0084011400 con 0x7f00940fbea0 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.063+0000 7f00927fc700 1 -- 172.21.15.42:0/2422798016 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0084010580 con 0x7f00940fbea0 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.064+0000 7f0099be8700 1 -- 172.21.15.42:0/2422798016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00940fbea0 msgr2=0x7f00940fc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.064+0000 7f0099be8700 1 --2- 172.21.15.42:0/2422798016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00940fbea0 0x7f00940fc2c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f0084009960 tx=0x7f00840092d0).stop 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.064+0000 7f0099be8700 1 -- 172.21.15.42:0/2422798016 shutdown_connections 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.064+0000 7f0099be8700 1 --2- 172.21.15.42:0/2422798016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00940fbea0 0x7f00940fc2c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:44.067 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.064+0000 7f0099be8700 1 -- 172.21.15.42:0/2422798016 >> 172.21.15.42:0/2422798016 conn(0x7f00940f76a0 msgr2=0x7f00940f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:44.068 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.064+0000 7f0099be8700 1 -- 172.21.15.42:0/2422798016 shutdown_connections 2024-02-15T18:17:44.068 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.064+0000 7f0099be8700 1 -- 172.21.15.42:0/2422798016 wait complete. 2024-02-15T18:17:44.068 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.065+0000 7f0099be8700 1 Processor -- start 2024-02-15T18:17:44.068 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.065+0000 7f0099be8700 1 -- start start 2024-02-15T18:17:44.068 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.065+0000 7f0099be8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00940fbea0 0x7f009410ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:44.068 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.065+0000 7f0099be8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009410b080 con 0x7f00940fbea0 2024-02-15T18:17:44.068 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.065+0000 7f00937fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00940fbea0 0x7f009410ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:44.068 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.065+0000 7f00937fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00940fbea0 0x7f009410ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45362/0 (socket says 172.21.15.42:45362) 2024-02-15T18:17:44.068 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.065+0000 7f00937fe700 1 -- 172.21.15.42:0/2633569379 learned_addr learned my addr 172.21.15.42:0/2633569379 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:44.068 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.066+0000 7f00937fe700 1 -- 172.21.15.42:0/2633569379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0084009570 con 0x7f00940fbea0 2024-02-15T18:17:44.068 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.066+0000 7f00937fe700 1 --2- 172.21.15.42:0/2633569379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00940fbea0 0x7f009410ab90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f008400bac0 tx=0x7f0084011f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:44.068 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.066+0000 7f007bfff700 1 -- 172.21.15.42:0/2633569379 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00840107f0 con 0x7f00940fbea0 2024-02-15T18:17:44.068 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.066+0000 7f007bfff700 1 -- 172.21.15.42:0/2633569379 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f0084010de0 con 0x7f00940fbea0 2024-02-15T18:17:44.069 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.066+0000 7f0099be8700 1 -- 172.21.15.42:0/2633569379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009410b280 con 0x7f00940fbea0 2024-02-15T18:17:44.070 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.066+0000 7f007bfff700 1 -- 172.21.15.42:0/2633569379 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f008401ae70 con 0x7f00940fbea0 2024-02-15T18:17:44.070 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.066+0000 7f0099be8700 1 -- 172.21.15.42:0/2633569379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009410b710 con 0x7f00940fbea0 2024-02-15T18:17:44.070 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.067+0000 7f007bfff700 1 -- 172.21.15.42:0/2633569379 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7f008400f070 con 0x7f00940fbea0 2024-02-15T18:17:44.070 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.067+0000 7f007bfff700 1 --2- 172.21.15.42:0/2633569379 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f008002ff20 0x7f00800323e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:44.070 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.067+0000 7f007bfff700 1 -- 172.21.15.42:0/2633569379 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f0084041d10 con 0x7f00940fbea0 2024-02-15T18:17:44.070 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.067+0000 7f0092ffd700 1 --2- 172.21.15.42:0/2633569379 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f008002ff20 0x7f00800323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:44.070 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.067+0000 7f0092ffd700 1 --2- 172.21.15.42:0/2633569379 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f008002ff20 0x7f00800323e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f00880096a0 tx=0x7f0088006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:44.070 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.067+0000 7f0099be8700 1 -- 172.21.15.42:0/2633569379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0094045ad0 con 0x7f00940fbea0 2024-02-15T18:17:44.072 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.072+0000 7f007bfff700 1 -- 172.21.15.42:0/2633569379 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f008401e020 con 0x7f00940fbea0 2024-02-15T18:17:44.259 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.259+0000 7f0099be8700 1 -- 172.21.15.42:0/2633569379 --> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f0094104bf0 con 0x7f008002ff20 2024-02-15T18:17:44.263 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.263+0000 7f007bfff700 1 -- 172.21.15.42:0/2633569379 <== mgr.14142 v2:172.21.15.42:6800/481027993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f0094104bf0 con 0x7f008002ff20 2024-02-15T18:17:44.263 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: Scheduled node-exporter update... 2024-02-15T18:17:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.264+0000 7f0099be8700 1 -- 172.21.15.42:0/2633569379 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f008002ff20 msgr2=0x7f00800323e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.264+0000 7f0099be8700 1 --2- 172.21.15.42:0/2633569379 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f008002ff20 0x7f00800323e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f00880096a0 tx=0x7f0088006b40).stop 2024-02-15T18:17:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.264+0000 7f0099be8700 1 -- 172.21.15.42:0/2633569379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00940fbea0 msgr2=0x7f009410ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.264+0000 7f0099be8700 1 --2- 172.21.15.42:0/2633569379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00940fbea0 0x7f009410ab90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f008400bac0 tx=0x7f0084011f90).stop 2024-02-15T18:17:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.265+0000 7f0099be8700 1 -- 172.21.15.42:0/2633569379 shutdown_connections 2024-02-15T18:17:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.265+0000 7f0099be8700 1 --2- 172.21.15.42:0/2633569379 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f008002ff20 0x7f00800323e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.265+0000 7f0099be8700 1 --2- 172.21.15.42:0/2633569379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00940fbea0 0x7f009410ab90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.265+0000 7f0099be8700 1 -- 172.21.15.42:0/2633569379 >> 172.21.15.42:0/2633569379 conn(0x7f00940f76a0 msgr2=0x7f00940f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.265+0000 7f0099be8700 1 -- 172.21.15.42:0/2633569379 shutdown_connections 2024-02-15T18:17:44.266 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:44.265+0000 7f0099be8700 1 -- 172.21.15.42:0/2633569379 wait complete. 2024-02-15T18:17:44.817 INFO:teuthology.orchestra.run.smithi042.stderr:Deploying alertmanager service with default placement... 2024-02-15T18:17:44.817 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply alertmanager 2024-02-15T18:17:45.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:44.261067+0000 mgr.smithi042.kfoihb (mgr.14142) 5 : audit [DBG] from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:17:45.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:17:44.261689+0000 mgr.smithi042.kfoihb (mgr.14142) 6 : cephadm [INF] Saving service node-exporter spec with placement * 2024-02-15T18:17:45.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:44.263734+0000 mon.smithi042 (mon.0) 80 : audit [INF] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:45.344 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.343+0000 7f72eeae4700 1 Processor -- start 2024-02-15T18:17:45.344 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.343+0000 7f72eeae4700 1 -- start start 2024-02-15T18:17:45.345 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.343+0000 7f72eeae4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80703f0 0x7f72e80fea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:45.345 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.343+0000 7f72eeae4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e80fef80 con 0x7f72e80703f0 2024-02-15T18:17:45.345 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.344+0000 7f72ec880700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80703f0 0x7f72e80fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:45.345 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.344+0000 7f72ec880700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80703f0 0x7f72e80fea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45378/0 (socket says 172.21.15.42:45378) 2024-02-15T18:17:45.347 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.344+0000 7f72ec880700 1 -- 172.21.15.42:0/3621096725 learned_addr learned my addr 172.21.15.42:0/3621096725 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:45.347 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.344+0000 7f72ec880700 1 -- 172.21.15.42:0/3621096725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e80ff0c0 con 0x7f72e80703f0 2024-02-15T18:17:45.347 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.344+0000 7f72ec880700 1 --2- 172.21.15.42:0/3621096725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80703f0 0x7f72e80fea90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f72dc00b5f0 tx=0x7f72dc006ee0).ready entity=mon.0 client_cookie=267440759a976d35 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:45.347 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.344+0000 7f72e77fe700 1 -- 172.21.15.42:0/3621096725 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f72dc00d600 con 0x7f72e80703f0 2024-02-15T18:17:45.347 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.344+0000 7f72e77fe700 1 -- 172.21.15.42:0/3621096725 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f72dc00dbf0 con 0x7f72e80703f0 2024-02-15T18:17:45.347 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.344+0000 7f72e77fe700 1 -- 172.21.15.42:0/3621096725 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f72dc011ce0 con 0x7f72e80703f0 2024-02-15T18:17:45.347 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.345+0000 7f72eeae4700 1 -- 172.21.15.42:0/3621096725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80703f0 msgr2=0x7f72e80fea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:45.347 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.345+0000 7f72eeae4700 1 --2- 172.21.15.42:0/3621096725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80703f0 0x7f72e80fea90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f72dc00b5f0 tx=0x7f72dc006ee0).stop 2024-02-15T18:17:45.347 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.345+0000 7f72eeae4700 1 -- 172.21.15.42:0/3621096725 shutdown_connections 2024-02-15T18:17:45.347 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.345+0000 7f72eeae4700 1 --2- 172.21.15.42:0/3621096725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80703f0 0x7f72e80fea90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:45.347 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.345+0000 7f72eeae4700 1 -- 172.21.15.42:0/3621096725 >> 172.21.15.42:0/3621096725 conn(0x7f72e80f6530 msgr2=0x7f72e80f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:45.348 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.345+0000 7f72eeae4700 1 -- 172.21.15.42:0/3621096725 shutdown_connections 2024-02-15T18:17:45.348 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.345+0000 7f72eeae4700 1 -- 172.21.15.42:0/3621096725 wait complete. 2024-02-15T18:17:45.348 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.345+0000 7f72eeae4700 1 Processor -- start 2024-02-15T18:17:45.348 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.345+0000 7f72eeae4700 1 -- start start 2024-02-15T18:17:45.348 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.346+0000 7f72eeae4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80703f0 0x7f72e80fc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:45.348 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.346+0000 7f72eeae4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e80fa990 con 0x7f72e80703f0 2024-02-15T18:17:45.348 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.346+0000 7f72ec880700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80703f0 0x7f72e80fc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:45.348 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.346+0000 7f72ec880700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80703f0 0x7f72e80fc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45380/0 (socket says 172.21.15.42:45380) 2024-02-15T18:17:45.349 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.346+0000 7f72ec880700 1 -- 172.21.15.42:0/1225318397 learned_addr learned my addr 172.21.15.42:0/1225318397 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:45.349 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.346+0000 7f72ec880700 1 -- 172.21.15.42:0/1225318397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc00b200 con 0x7f72e80703f0 2024-02-15T18:17:45.349 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.346+0000 7f72ec880700 1 --2- 172.21.15.42:0/1225318397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80703f0 0x7f72e80fc510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f72dc011ed0 tx=0x7f72dc006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:45.349 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.347+0000 7f72e57fa700 1 -- 172.21.15.42:0/1225318397 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f72dc01b570 con 0x7f72e80703f0 2024-02-15T18:17:45.349 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.347+0000 7f72eeae4700 1 -- 172.21.15.42:0/1225318397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e80fab30 con 0x7f72e80703f0 2024-02-15T18:17:45.349 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.347+0000 7f72e57fa700 1 -- 172.21.15.42:0/1225318397 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f72dc01bb60 con 0x7f72e80703f0 2024-02-15T18:17:45.349 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.347+0000 7f72e57fa700 1 -- 172.21.15.42:0/1225318397 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f72dc01aad0 con 0x7f72e80703f0 2024-02-15T18:17:45.349 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.347+0000 7f72eeae4700 1 -- 172.21.15.42:0/1225318397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e80fafc0 con 0x7f72e80703f0 2024-02-15T18:17:45.349 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.347+0000 7f72e57fa700 1 -- 172.21.15.42:0/1225318397 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7f72dc00f070 con 0x7f72e80703f0 2024-02-15T18:17:45.349 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.348+0000 7f72e57fa700 1 --2- 172.21.15.42:0/1225318397 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f72cc02ff20 0x7f72cc0323e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:45.349 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.348+0000 7f72eeae4700 1 -- 172.21.15.42:0/1225318397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e8045ad0 con 0x7f72e80703f0 2024-02-15T18:17:45.349 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.348+0000 7f72e57fa700 1 -- 172.21.15.42:0/1225318397 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f72dc041530 con 0x7f72e80703f0 2024-02-15T18:17:45.349 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.348+0000 7f72e7fff700 1 --2- 172.21.15.42:0/1225318397 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f72cc02ff20 0x7f72cc0323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:45.349 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.348+0000 7f72e7fff700 1 --2- 172.21.15.42:0/1225318397 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f72cc02ff20 0x7f72cc0323e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f72d40096a0 tx=0x7f72d4006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:45.352 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.352+0000 7f72e57fa700 1 -- 172.21.15.42:0/1225318397 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f72dc016070 con 0x7f72e80703f0 2024-02-15T18:17:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.538+0000 7f72eeae4700 1 -- 172.21.15.42:0/1225318397 --> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f72e80fb200 con 0x7f72cc02ff20 2024-02-15T18:17:45.542 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.541+0000 7f72e57fa700 1 -- 172.21.15.42:0/1225318397 <== mgr.14142 v2:172.21.15.42:6800/481027993 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f72e80fb200 con 0x7f72cc02ff20 2024-02-15T18:17:45.542 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: Scheduled alertmanager update... 2024-02-15T18:17:45.544 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.543+0000 7f72eeae4700 1 -- 172.21.15.42:0/1225318397 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f72cc02ff20 msgr2=0x7f72cc0323e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:45.544 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.543+0000 7f72eeae4700 1 --2- 172.21.15.42:0/1225318397 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f72cc02ff20 0x7f72cc0323e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f72d40096a0 tx=0x7f72d4006b40).stop 2024-02-15T18:17:45.544 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.543+0000 7f72eeae4700 1 -- 172.21.15.42:0/1225318397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80703f0 msgr2=0x7f72e80fc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:45.545 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.543+0000 7f72eeae4700 1 --2- 172.21.15.42:0/1225318397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80703f0 0x7f72e80fc510 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f72dc011ed0 tx=0x7f72dc006ee0).stop 2024-02-15T18:17:45.545 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.543+0000 7f72eeae4700 1 -- 172.21.15.42:0/1225318397 shutdown_connections 2024-02-15T18:17:45.545 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.543+0000 7f72eeae4700 1 --2- 172.21.15.42:0/1225318397 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f72cc02ff20 0x7f72cc0323e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:45.545 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.543+0000 7f72eeae4700 1 --2- 172.21.15.42:0/1225318397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80703f0 0x7f72e80fc510 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:45.545 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.543+0000 7f72eeae4700 1 -- 172.21.15.42:0/1225318397 >> 172.21.15.42:0/1225318397 conn(0x7f72e80f6530 msgr2=0x7f72e80f7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:45.545 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.543+0000 7f72eeae4700 1 -- 172.21.15.42:0/1225318397 shutdown_connections 2024-02-15T18:17:45.545 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:45.543+0000 7f72eeae4700 1 -- 172.21.15.42:0/1225318397 wait complete. 2024-02-15T18:17:46.116 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config set mgr mgr/cephadm/container_init True --force 2024-02-15T18:17:46.664 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.649+0000 7f855159f700 1 Processor -- start 2024-02-15T18:17:46.665 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.649+0000 7f855159f700 1 -- start start 2024-02-15T18:17:46.665 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.649+0000 7f855159f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f854c0fab30 0x7f854c0faf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:46.665 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.649+0000 7f855159f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854c0fb440 con 0x7f854c0fab30 2024-02-15T18:17:46.665 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.650+0000 7f854bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f854c0fab30 0x7f854c0faf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:46.665 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.650+0000 7f854bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f854c0fab30 0x7f854c0faf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45392/0 (socket says 172.21.15.42:45392) 2024-02-15T18:17:46.665 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.650+0000 7f854bfff700 1 -- 172.21.15.42:0/3679573983 learned_addr learned my addr 172.21.15.42:0/3679573983 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:46.665 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.650+0000 7f854bfff700 1 -- 172.21.15.42:0/3679573983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c0fb580 con 0x7f854c0fab30 2024-02-15T18:17:46.665 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.650+0000 7f854bfff700 1 --2- 172.21.15.42:0/3679573983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f854c0fab30 0x7f854c0faf50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f853c00b5f0 tx=0x7f853c006ee0).ready entity=mon.0 client_cookie=e3628e4e2952c7ec server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:46.665 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.650+0000 7f854b7fe700 1 -- 172.21.15.42:0/3679573983 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f853c00d600 con 0x7f854c0fab30 2024-02-15T18:17:46.665 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.650+0000 7f854b7fe700 1 -- 172.21.15.42:0/3679573983 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f853c00dbf0 con 0x7f854c0fab30 2024-02-15T18:17:46.665 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.650+0000 7f854b7fe700 1 -- 172.21.15.42:0/3679573983 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f853c011cb0 con 0x7f854c0fab30 2024-02-15T18:17:46.665 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.651+0000 7f855159f700 1 -- 172.21.15.42:0/3679573983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f854c0fab30 msgr2=0x7f854c0faf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:46.665 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.651+0000 7f855159f700 1 --2- 172.21.15.42:0/3679573983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f854c0fab30 0x7f854c0faf50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f853c00b5f0 tx=0x7f853c006ee0).stop 2024-02-15T18:17:46.666 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.651+0000 7f855159f700 1 -- 172.21.15.42:0/3679573983 shutdown_connections 2024-02-15T18:17:46.666 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.651+0000 7f855159f700 1 --2- 172.21.15.42:0/3679573983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f854c0fab30 0x7f854c0faf50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:46.666 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.651+0000 7f855159f700 1 -- 172.21.15.42:0/3679573983 >> 172.21.15.42:0/3679573983 conn(0x7f854c0f6390 msgr2=0x7f854c0f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:46.666 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.651+0000 7f855159f700 1 -- 172.21.15.42:0/3679573983 shutdown_connections 2024-02-15T18:17:46.666 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.651+0000 7f855159f700 1 -- 172.21.15.42:0/3679573983 wait complete. 2024-02-15T18:17:46.666 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.651+0000 7f855159f700 1 Processor -- start 2024-02-15T18:17:46.666 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.651+0000 7f855159f700 1 -- start start 2024-02-15T18:17:46.666 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.651+0000 7f855159f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f854c0fab30 0x7f854c10d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:46.666 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.652+0000 7f855159f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854c070470 con 0x7f854c0fab30 2024-02-15T18:17:46.666 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.652+0000 7f854bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f854c0fab30 0x7f854c10d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:46.666 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.652+0000 7f854bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f854c0fab30 0x7f854c10d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45400/0 (socket says 172.21.15.42:45400) 2024-02-15T18:17:46.666 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.652+0000 7f854bfff700 1 -- 172.21.15.42:0/1451568937 learned_addr learned my addr 172.21.15.42:0/1451568937 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:46.666 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.652+0000 7f854bfff700 1 -- 172.21.15.42:0/1451568937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c00b200 con 0x7f854c0fab30 2024-02-15T18:17:46.666 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.652+0000 7f854bfff700 1 --2- 172.21.15.42:0/1451568937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f854c0fab30 0x7f854c10d680 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f853c011ea0 tx=0x7f853c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:46.667 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.652+0000 7f85497fa700 1 -- 172.21.15.42:0/1451568937 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f853c01b570 con 0x7f854c0fab30 2024-02-15T18:17:46.667 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.652+0000 7f85497fa700 1 -- 172.21.15.42:0/1451568937 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7f853c01bb60 con 0x7f854c0fab30 2024-02-15T18:17:46.667 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.652+0000 7f855159f700 1 -- 172.21.15.42:0/1451568937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854c070670 con 0x7f854c0fab30 2024-02-15T18:17:46.667 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.652+0000 7f85497fa700 1 -- 172.21.15.42:0/1451568937 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f853c01a8d0 con 0x7f854c0fab30 2024-02-15T18:17:46.667 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.652+0000 7f855159f700 1 -- 172.21.15.42:0/1451568937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f854c10dd30 con 0x7f854c0fab30 2024-02-15T18:17:46.667 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.653+0000 7f85497fa700 1 -- 172.21.15.42:0/1451568937 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7f853c00f070 con 0x7f854c0fab30 2024-02-15T18:17:46.667 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.653+0000 7f85497fa700 1 --2- 172.21.15.42:0/1451568937 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f852c02ff70 0x7f852c032430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:46.667 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.653+0000 7f85497fa700 1 -- 172.21.15.42:0/1451568937 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f853c0417e0 con 0x7f854c0fab30 2024-02-15T18:17:46.667 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.653+0000 7f8543fff700 1 --2- 172.21.15.42:0/1451568937 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f852c02ff70 0x7f852c032430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:46.667 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.653+0000 7f855159f700 1 -- 172.21.15.42:0/1451568937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f854c045ad0 con 0x7f854c0fab30 2024-02-15T18:17:46.667 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.653+0000 7f8543fff700 1 --2- 172.21.15.42:0/1451568937 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f852c02ff70 0x7f852c032430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f85340096a0 tx=0x7f8534006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:46.667 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.657+0000 7f85497fa700 1 -- 172.21.15.42:0/1451568937 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f853c016070 con 0x7f854c0fab30 2024-02-15T18:17:46.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:45.540000+0000 mgr.smithi042.kfoihb (mgr.14142) 7 : audit [DBG] from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:17:46.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:17:45.540483+0000 mgr.smithi042.kfoihb (mgr.14142) 8 : cephadm [INF] Saving service alertmanager spec with placement count:1 2024-02-15T18:17:46.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:45.542567+0000 mon.smithi042 (mon.0) 81 : audit [INF] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:46.815 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.815+0000 7f855159f700 1 -- 172.21.15.42:0/1451568937 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f854c027a00 con 0x7f854c0fab30 2024-02-15T18:17:46.818 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.817+0000 7f85497fa700 1 -- 172.21.15.42:0/1451568937 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v8)=0 v8) v1 ==== 142+0+0 (secure 0 0 0) 0x7f853c022d20 con 0x7f854c0fab30 2024-02-15T18:17:46.821 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.819+0000 7f855159f700 1 -- 172.21.15.42:0/1451568937 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f852c02ff70 msgr2=0x7f852c032430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:46.821 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.819+0000 7f855159f700 1 --2- 172.21.15.42:0/1451568937 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f852c02ff70 0x7f852c032430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f85340096a0 tx=0x7f8534006b40).stop 2024-02-15T18:17:46.821 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.819+0000 7f855159f700 1 -- 172.21.15.42:0/1451568937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f854c0fab30 msgr2=0x7f854c10d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:46.821 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.819+0000 7f855159f700 1 --2- 172.21.15.42:0/1451568937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f854c0fab30 0x7f854c10d680 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f853c011ea0 tx=0x7f853c006ee0).stop 2024-02-15T18:17:46.821 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.820+0000 7f855159f700 1 -- 172.21.15.42:0/1451568937 shutdown_connections 2024-02-15T18:17:46.821 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.820+0000 7f855159f700 1 --2- 172.21.15.42:0/1451568937 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f852c02ff70 0x7f852c032430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:46.821 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.820+0000 7f855159f700 1 --2- 172.21.15.42:0/1451568937 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f854c0fab30 0x7f854c10d680 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:46.821 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.820+0000 7f855159f700 1 -- 172.21.15.42:0/1451568937 >> 172.21.15.42:0/1451568937 conn(0x7f854c0f6390 msgr2=0x7f854c0f7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:46.821 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.820+0000 7f855159f700 1 -- 172.21.15.42:0/1451568937 shutdown_connections 2024-02-15T18:17:46.821 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:46.820+0000 7f855159f700 1 -- 172.21.15.42:0/1451568937 wait complete. 2024-02-15T18:17:47.474 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config set mgr mgr/dashboard/ssl_server_port 8443 2024-02-15T18:17:48.122 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.087+0000 7faaedae7700 1 Processor -- start 2024-02-15T18:17:48.122 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.087+0000 7faaedae7700 1 -- start start 2024-02-15T18:17:48.122 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.088+0000 7faaedae7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faae8072520 0x7faae8070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:48.122 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.088+0000 7faaedae7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae8072940 con 0x7faae8072520 2024-02-15T18:17:48.122 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.088+0000 7faae77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faae8072520 0x7faae8070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:48.122 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.088+0000 7faae77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faae8072520 0x7faae8070ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45414/0 (socket says 172.21.15.42:45414) 2024-02-15T18:17:48.122 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.088+0000 7faae77fe700 1 -- 172.21.15.42:0/1230999771 learned_addr learned my addr 172.21.15.42:0/1230999771 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:48.122 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.088+0000 7faae77fe700 1 -- 172.21.15.42:0/1230999771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae8070fb0 con 0x7faae8072520 2024-02-15T18:17:48.122 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.088+0000 7faae77fe700 1 --2- 172.21.15.42:0/1230999771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faae8072520 0x7faae8070ac0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7faad800b5f0 tx=0x7faad8006ee0).ready entity=mon.0 client_cookie=a1baac3fc61cfbf0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:48.123 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.088+0000 7faae67fc700 1 -- 172.21.15.42:0/1230999771 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faad800d600 con 0x7faae8072520 2024-02-15T18:17:48.123 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.088+0000 7faae67fc700 1 -- 172.21.15.42:0/1230999771 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7faad800dbf0 con 0x7faae8072520 2024-02-15T18:17:48.123 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.088+0000 7faae67fc700 1 -- 172.21.15.42:0/1230999771 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faad8011cb0 con 0x7faae8072520 2024-02-15T18:17:48.123 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.089+0000 7faaedae7700 1 -- 172.21.15.42:0/1230999771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faae8072520 msgr2=0x7faae8070ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:48.123 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.089+0000 7faaedae7700 1 --2- 172.21.15.42:0/1230999771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faae8072520 0x7faae8070ac0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7faad800b5f0 tx=0x7faad8006ee0).stop 2024-02-15T18:17:48.123 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.089+0000 7faaedae7700 1 -- 172.21.15.42:0/1230999771 shutdown_connections 2024-02-15T18:17:48.123 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.089+0000 7faaedae7700 1 --2- 172.21.15.42:0/1230999771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faae8072520 0x7faae8070ac0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:48.123 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.089+0000 7faaedae7700 1 -- 172.21.15.42:0/1230999771 >> 172.21.15.42:0/1230999771 conn(0x7faae80f76a0 msgr2=0x7faae80f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:48.123 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.089+0000 7faaedae7700 1 -- 172.21.15.42:0/1230999771 shutdown_connections 2024-02-15T18:17:48.123 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.089+0000 7faaedae7700 1 -- 172.21.15.42:0/1230999771 wait complete. 2024-02-15T18:17:48.123 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.089+0000 7faaedae7700 1 Processor -- start 2024-02-15T18:17:48.123 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.090+0000 7faaedae7700 1 -- start start 2024-02-15T18:17:48.123 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.090+0000 7faaedae7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faae8072520 0x7faae810ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:48.124 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.090+0000 7faaedae7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae810d2f0 con 0x7faae8072520 2024-02-15T18:17:48.124 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.090+0000 7faae77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faae8072520 0x7faae810ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:48.124 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.090+0000 7faae77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faae8072520 0x7faae810ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45420/0 (socket says 172.21.15.42:45420) 2024-02-15T18:17:48.124 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.090+0000 7faae77fe700 1 -- 172.21.15.42:0/69608321 learned_addr learned my addr 172.21.15.42:0/69608321 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:48.124 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.090+0000 7faae77fe700 1 -- 172.21.15.42:0/69608321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad800b200 con 0x7faae8072520 2024-02-15T18:17:48.124 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.090+0000 7faae77fe700 1 --2- 172.21.15.42:0/69608321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faae8072520 0x7faae810ce00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7faad8006800 tx=0x7faad8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:48.124 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.091+0000 7faadffff700 1 -- 172.21.15.42:0/69608321 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faad801b570 con 0x7faae8072520 2024-02-15T18:17:48.124 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.091+0000 7faaedae7700 1 -- 172.21.15.42:0/69608321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae810d4f0 con 0x7faae8072520 2024-02-15T18:17:48.124 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.091+0000 7faadffff700 1 -- 172.21.15.42:0/69608321 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1045+0+0 (secure 0 0 0) 0x7faad801bb60 con 0x7faae8072520 2024-02-15T18:17:48.124 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.091+0000 7faadffff700 1 -- 172.21.15.42:0/69608321 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faad801a9f0 con 0x7faae8072520 2024-02-15T18:17:48.124 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.091+0000 7faaedae7700 1 -- 172.21.15.42:0/69608321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae810e100 con 0x7faae8072520 2024-02-15T18:17:48.125 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.091+0000 7faadffff700 1 -- 172.21.15.42:0/69608321 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7faad800f070 con 0x7faae8072520 2024-02-15T18:17:48.125 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.092+0000 7faadffff700 1 --2- 172.21.15.42:0/69608321 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7faac802ff20 0x7faac80323e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:48.125 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.092+0000 7faadffff700 1 -- 172.21.15.42:0/69608321 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7faad8041720 con 0x7faae8072520 2024-02-15T18:17:48.125 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.092+0000 7faaedae7700 1 -- 172.21.15.42:0/69608321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae8059100 con 0x7faae8072520 2024-02-15T18:17:48.125 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.092+0000 7faae6ffd700 1 --2- 172.21.15.42:0/69608321 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7faac802ff20 0x7faac80323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:48.125 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.092+0000 7faae6ffd700 1 --2- 172.21.15.42:0/69608321 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7faac802ff20 0x7faac80323e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7faad00096a0 tx=0x7faad0006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:48.125 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.096+0000 7faadffff700 1 -- 172.21.15.42:0/69608321 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7faad8016070 con 0x7faae8072520 2024-02-15T18:17:48.266 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.266+0000 7faaedae7700 1 -- 172.21.15.42:0/69608321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7faae8045ad0 con 0x7faae8072520 2024-02-15T18:17:48.269 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.269+0000 7faadffff700 1 -- 172.21.15.42:0/69608321 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v9)=0 v9) v1 ==== 130+0+0 (secure 0 0 0) 0x7faae8045ad0 con 0x7faae8072520 2024-02-15T18:17:48.271 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.270+0000 7faaedae7700 1 -- 172.21.15.42:0/69608321 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7faac802ff20 msgr2=0x7faac80323e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:48.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.270+0000 7faaedae7700 1 --2- 172.21.15.42:0/69608321 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7faac802ff20 0x7faac80323e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7faad00096a0 tx=0x7faad0006b40).stop 2024-02-15T18:17:48.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.270+0000 7faaedae7700 1 -- 172.21.15.42:0/69608321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faae8072520 msgr2=0x7faae810ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:48.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.270+0000 7faaedae7700 1 --2- 172.21.15.42:0/69608321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faae8072520 0x7faae810ce00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7faad8006800 tx=0x7faad8006ee0).stop 2024-02-15T18:17:48.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.270+0000 7faaedae7700 1 -- 172.21.15.42:0/69608321 shutdown_connections 2024-02-15T18:17:48.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.270+0000 7faaedae7700 1 --2- 172.21.15.42:0/69608321 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7faac802ff20 0x7faac80323e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:48.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.270+0000 7faaedae7700 1 --2- 172.21.15.42:0/69608321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faae8072520 0x7faae810ce00 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0x7faad8006800 tx=0x7faad8006ee0).stop 2024-02-15T18:17:48.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.270+0000 7faaedae7700 1 -- 172.21.15.42:0/69608321 >> 172.21.15.42:0/69608321 conn(0x7faae80f76a0 msgr2=0x7faae80f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:48.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.271+0000 7faaedae7700 1 -- 172.21.15.42:0/69608321 shutdown_connections 2024-02-15T18:17:48.272 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:48.271+0000 7faaedae7700 1 -- 172.21.15.42:0/69608321 wait complete. 2024-02-15T18:17:48.290 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:46.818789+0000 mon.smithi042 (mon.0) 82 : audit [INF] from='client.? 172.21.15.42:0/1451568937' entity='client.admin' 2024-02-15T18:17:49.458 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:48.270081+0000 mon.smithi042 (mon.0) 83 : audit [INF] from='client.? 172.21.15.42:0/69608321' entity='client.admin' 2024-02-15T18:17:49.799 INFO:teuthology.orchestra.run.smithi042.stderr:Enabling the dashboard module... 2024-02-15T18:17:49.799 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr module enable dashboard 2024-02-15T18:17:50.334 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.331+0000 7fe266b88700 1 Processor -- start 2024-02-15T18:17:50.334 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.331+0000 7fe266b88700 1 -- start start 2024-02-15T18:17:50.334 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.331+0000 7fe266b88700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2600fab30 0x7fe2600faf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:50.334 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.332+0000 7fe266b88700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2600fb440 con 0x7fe2600fab30 2024-02-15T18:17:50.334 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.332+0000 7fe265b86700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2600fab30 0x7fe2600faf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:50.334 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.332+0000 7fe265b86700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2600fab30 0x7fe2600faf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45432/0 (socket says 172.21.15.42:45432) 2024-02-15T18:17:50.334 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.332+0000 7fe265b86700 1 -- 172.21.15.42:0/2038254430 learned_addr learned my addr 172.21.15.42:0/2038254430 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:50.334 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.332+0000 7fe265b86700 1 -- 172.21.15.42:0/2038254430 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2600fb580 con 0x7fe2600fab30 2024-02-15T18:17:50.335 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.332+0000 7fe265b86700 1 --2- 172.21.15.42:0/2038254430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2600fab30 0x7fe2600faf50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fe25c00b5f0 tx=0x7fe25c006ee0).ready entity=mon.0 client_cookie=bd84c03a36c05000 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:50.335 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.332+0000 7fe264b84700 1 -- 172.21.15.42:0/2038254430 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe25c00d600 con 0x7fe2600fab30 2024-02-15T18:17:50.335 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.332+0000 7fe264b84700 1 -- 172.21.15.42:0/2038254430 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe25c00dc30 con 0x7fe2600fab30 2024-02-15T18:17:50.335 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.333+0000 7fe264b84700 1 -- 172.21.15.42:0/2038254430 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe25c011d30 con 0x7fe2600fab30 2024-02-15T18:17:50.335 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.333+0000 7fe266b88700 1 -- 172.21.15.42:0/2038254430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2600fab30 msgr2=0x7fe2600faf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:50.335 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.333+0000 7fe266b88700 1 --2- 172.21.15.42:0/2038254430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2600fab30 0x7fe2600faf50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fe25c00b5f0 tx=0x7fe25c006ee0).stop 2024-02-15T18:17:50.335 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.333+0000 7fe266b88700 1 -- 172.21.15.42:0/2038254430 shutdown_connections 2024-02-15T18:17:50.335 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.333+0000 7fe266b88700 1 --2- 172.21.15.42:0/2038254430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2600fab30 0x7fe2600faf50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:50.335 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.333+0000 7fe266b88700 1 -- 172.21.15.42:0/2038254430 >> 172.21.15.42:0/2038254430 conn(0x7fe2600f6390 msgr2=0x7fe2600f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:50.335 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.333+0000 7fe266b88700 1 -- 172.21.15.42:0/2038254430 shutdown_connections 2024-02-15T18:17:50.335 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.333+0000 7fe266b88700 1 -- 172.21.15.42:0/2038254430 wait complete. 2024-02-15T18:17:50.335 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.334+0000 7fe266b88700 1 Processor -- start 2024-02-15T18:17:50.336 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.334+0000 7fe266b88700 1 -- start start 2024-02-15T18:17:50.337 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.334+0000 7fe266b88700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2600fab30 0x7fe26010cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:50.337 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.334+0000 7fe266b88700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26010d0b0 con 0x7fe2600fab30 2024-02-15T18:17:50.337 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.334+0000 7fe265b86700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2600fab30 0x7fe26010cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:50.337 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.334+0000 7fe265b86700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2600fab30 0x7fe26010cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45442/0 (socket says 172.21.15.42:45442) 2024-02-15T18:17:50.337 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.334+0000 7fe265b86700 1 -- 172.21.15.42:0/182203233 learned_addr learned my addr 172.21.15.42:0/182203233 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:50.337 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.335+0000 7fe265b86700 1 -- 172.21.15.42:0/182203233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25c00b200 con 0x7fe2600fab30 2024-02-15T18:17:50.337 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.335+0000 7fe265b86700 1 --2- 172.21.15.42:0/182203233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2600fab30 0x7fe26010cbc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fe25c00e040 tx=0x7fe25c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:50.337 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.335+0000 7fe24e7fc700 1 -- 172.21.15.42:0/182203233 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe25c00d9e0 con 0x7fe2600fab30 2024-02-15T18:17:50.337 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.335+0000 7fe24e7fc700 1 -- 172.21.15.42:0/182203233 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe25c01b8e0 con 0x7fe2600fab30 2024-02-15T18:17:50.337 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.335+0000 7fe266b88700 1 -- 172.21.15.42:0/182203233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26010d2b0 con 0x7fe2600fab30 2024-02-15T18:17:50.338 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.336+0000 7fe266b88700 1 -- 172.21.15.42:0/182203233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26010d740 con 0x7fe2600fab30 2024-02-15T18:17:50.338 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.336+0000 7fe24e7fc700 1 -- 172.21.15.42:0/182203233 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe25c01a880 con 0x7fe2600fab30 2024-02-15T18:17:50.338 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.336+0000 7fe24e7fc700 1 -- 172.21.15.42:0/182203233 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7fe25c00f070 con 0x7fe2600fab30 2024-02-15T18:17:50.338 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.337+0000 7fe24e7fc700 1 --2- 172.21.15.42:0/182203233 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7fe24802ffb0 0x7fe248032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:50.338 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.337+0000 7fe24e7fc700 1 -- 172.21.15.42:0/182203233 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fe25c042470 con 0x7fe2600fab30 2024-02-15T18:17:50.338 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.337+0000 7fe266b88700 1 -- 172.21.15.42:0/182203233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe240004fa0 con 0x7fe2600fab30 2024-02-15T18:17:50.338 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.337+0000 7fe265385700 1 --2- 172.21.15.42:0/182203233 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7fe24802ffb0 0x7fe248032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:50.339 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.338+0000 7fe265385700 1 --2- 172.21.15.42:0/182203233 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7fe24802ffb0 0x7fe248032470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fe2500096a0 tx=0x7fe250006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:50.343 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.343+0000 7fe24e7fc700 1 -- 172.21.15.42:0/182203233 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe25c016070 con 0x7fe2600fab30 2024-02-15T18:17:50.550 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:50.550+0000 7fe266b88700 1 -- 172.21.15.42:0/182203233 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fe240005670 con 0x7fe2600fab30 2024-02-15T18:17:50.803 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:49.481249+0000 mon.smithi042 (mon.0) 84 : audit [INF] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:50.803 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:50.140785+0000 mon.smithi042 (mon.0) 85 : audit [INF] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:51.481 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:51.480+0000 7fe24e7fc700 1 -- 172.21.15.42:0/182203233 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 13) v1 ==== 36208+0+0 (secure 0 0 0) 0x7fe25c019d10 con 0x7fe2600fab30 2024-02-15T18:17:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:51.481+0000 7fe24e7fc700 1 -- 172.21.15.42:0/182203233 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v13) v1 ==== 88+0+0 (secure 0 0 0) 0x7fe25c019300 con 0x7fe2600fab30 2024-02-15T18:17:51.485 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:51.484+0000 7fe266b88700 1 -- 172.21.15.42:0/182203233 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7fe24802ffb0 msgr2=0x7fe248032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:51.486 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:51.484+0000 7fe266b88700 1 --2- 172.21.15.42:0/182203233 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7fe24802ffb0 0x7fe248032470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fe2500096a0 tx=0x7fe250006b40).stop 2024-02-15T18:17:51.486 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:51.485+0000 7fe266b88700 1 -- 172.21.15.42:0/182203233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2600fab30 msgr2=0x7fe26010cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:51.486 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:51.485+0000 7fe266b88700 1 --2- 172.21.15.42:0/182203233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2600fab30 0x7fe26010cbc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fe25c00e040 tx=0x7fe25c006ee0).stop 2024-02-15T18:17:51.486 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:51.485+0000 7fe266b88700 1 -- 172.21.15.42:0/182203233 shutdown_connections 2024-02-15T18:17:51.486 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:51.485+0000 7fe266b88700 1 --2- 172.21.15.42:0/182203233 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7fe24802ffb0 0x7fe248032470 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:51.486 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:51.485+0000 7fe266b88700 1 --2- 172.21.15.42:0/182203233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2600fab30 0x7fe26010cbc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:51.486 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:51.485+0000 7fe266b88700 1 -- 172.21.15.42:0/182203233 >> 172.21.15.42:0/182203233 conn(0x7fe2600f6390 msgr2=0x7fe260103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:51.486 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:51.485+0000 7fe266b88700 1 -- 172.21.15.42:0/182203233 shutdown_connections 2024-02-15T18:17:51.487 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:51.485+0000 7fe266b88700 1 -- 172.21.15.42:0/182203233 wait complete. 2024-02-15T18:17:51.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:50.551446+0000 mon.smithi042 (mon.0) 86 : audit [INF] from='client.? 172.21.15.42:0/182203233' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-15T18:17:51.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:51.178454+0000 mon.smithi042 (mon.0) 87 : audit [INF] from='mgr.14142 172.21.15.42:0/789696820' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:52.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:51.482746+0000 mon.smithi042 (mon.0) 88 : audit [INF] from='client.? 172.21.15.42:0/182203233' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-15T18:17:52.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:51.482801+0000 mon.smithi042 (mon.0) 89 : cluster [DBG] mgrmap e13: smithi042.kfoihb(active, since 11s) 2024-02-15T18:17:52.589 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr stat 2024-02-15T18:17:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.541+0000 7f1f2f472700 1 Processor -- start 2024-02-15T18:17:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.541+0000 7f1f2f472700 1 -- start start 2024-02-15T18:17:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.541+0000 7f1f2f472700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f280703b0 0x7f1f28100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.541+0000 7f1f2f472700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f280fc8c0 con 0x7f1f280703b0 2024-02-15T18:17:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.542+0000 7f1f2d20e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f280703b0 0x7f1f28100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.542+0000 7f1f2d20e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f280703b0 0x7f1f28100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56794/0 (socket says 172.21.15.42:56794) 2024-02-15T18:17:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.542+0000 7f1f2d20e700 1 -- 172.21.15.42:0/2601866998 learned_addr learned my addr 172.21.15.42:0/2601866998 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.542+0000 7f1f2d20e700 1 -- 172.21.15.42:0/2601866998 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f281013c0 con 0x7f1f280703b0 2024-02-15T18:17:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.542+0000 7f1f2d20e700 1 --2- 172.21.15.42:0/2601866998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f280703b0 0x7f1f28100ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f1f2400b5f0 tx=0x7f1f24006ee0).ready entity=mon.0 client_cookie=deb19f4b43df233e server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.542+0000 7f1f17fff700 1 -- 172.21.15.42:0/2601866998 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f2400d600 con 0x7f1f280703b0 2024-02-15T18:17:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.542+0000 7f1f17fff700 1 -- 172.21.15.42:0/2601866998 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f2400dc30 con 0x7f1f280703b0 2024-02-15T18:17:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.542+0000 7f1f17fff700 1 -- 172.21.15.42:0/2601866998 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f24011d30 con 0x7f1f280703b0 2024-02-15T18:17:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.543+0000 7f1f2f472700 1 -- 172.21.15.42:0/2601866998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f280703b0 msgr2=0x7f1f28100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.543+0000 7f1f2f472700 1 --2- 172.21.15.42:0/2601866998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f280703b0 0x7f1f28100ed0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f1f2400b5f0 tx=0x7f1f24006ee0).stop 2024-02-15T18:17:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.543+0000 7f1f2f472700 1 -- 172.21.15.42:0/2601866998 shutdown_connections 2024-02-15T18:17:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.543+0000 7f1f2f472700 1 --2- 172.21.15.42:0/2601866998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f280703b0 0x7f1f28100ed0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.543+0000 7f1f2f472700 1 -- 172.21.15.42:0/2601866998 >> 172.21.15.42:0/2601866998 conn(0x7f1f280f6530 msgr2=0x7f1f280f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.543+0000 7f1f2f472700 1 -- 172.21.15.42:0/2601866998 shutdown_connections 2024-02-15T18:17:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.543+0000 7f1f2f472700 1 -- 172.21.15.42:0/2601866998 wait complete. 2024-02-15T18:17:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.543+0000 7f1f2f472700 1 Processor -- start 2024-02-15T18:17:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.543+0000 7f1f2f472700 1 -- start start 2024-02-15T18:17:53.565 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.544+0000 7f1f2f472700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f280703b0 0x7f1f2810cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:53.565 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.544+0000 7f1f2f472700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2810d290 con 0x7f1f280703b0 2024-02-15T18:17:53.565 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.544+0000 7f1f2d20e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f280703b0 0x7f1f2810cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:53.565 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.544+0000 7f1f2d20e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f280703b0 0x7f1f2810cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56810/0 (socket says 172.21.15.42:56810) 2024-02-15T18:17:53.565 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.544+0000 7f1f2d20e700 1 -- 172.21.15.42:0/1715371165 learned_addr learned my addr 172.21.15.42:0/1715371165 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:53.565 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.544+0000 7f1f2d20e700 1 -- 172.21.15.42:0/1715371165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2400b200 con 0x7f1f280703b0 2024-02-15T18:17:53.565 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.544+0000 7f1f2d20e700 1 --2- 172.21.15.42:0/1715371165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f280703b0 0x7f1f2810cda0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f1f2400e040 tx=0x7f1f24006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:53.565 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.544+0000 7f1f15ffb700 1 -- 172.21.15.42:0/1715371165 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f2400da60 con 0x7f1f280703b0 2024-02-15T18:17:53.565 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.544+0000 7f1f15ffb700 1 -- 172.21.15.42:0/1715371165 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f2401b8e0 con 0x7f1f280703b0 2024-02-15T18:17:53.565 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.544+0000 7f1f15ffb700 1 -- 172.21.15.42:0/1715371165 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f2401a900 con 0x7f1f280703b0 2024-02-15T18:17:53.565 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.544+0000 7f1f2f472700 1 -- 172.21.15.42:0/1715371165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2810d490 con 0x7f1f280703b0 2024-02-15T18:17:53.565 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.545+0000 7f1f2f472700 1 -- 172.21.15.42:0/1715371165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2810e0d0 con 0x7f1f280703b0 2024-02-15T18:17:53.565 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.545+0000 7f1f15ffb700 1 -- 172.21.15.42:0/1715371165 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 36208+0+0 (secure 0 0 0) 0x7f1f2400f070 con 0x7f1f280703b0 2024-02-15T18:17:53.565 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.545+0000 7f1f15ffb700 1 --2- 172.21.15.42:0/1715371165 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f1f1002ff70 0x7f1f10032430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:53.566 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.545+0000 7f1f15ffb700 1 -- 172.21.15.42:0/1715371165 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f1f24041450 con 0x7f1f280703b0 2024-02-15T18:17:53.566 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.545+0000 7f1f2ca0d700 1 -- 172.21.15.42:0/1715371165 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f1f1002ff70 msgr2=0x7f1f10032430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/481027993 2024-02-15T18:17:53.566 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.545+0000 7f1f2ca0d700 1 --2- 172.21.15.42:0/1715371165 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f1f1002ff70 0x7f1f10032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:17:53.566 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.546+0000 7f1f2f472700 1 -- 172.21.15.42:0/1715371165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2810d920 con 0x7f1f280703b0 2024-02-15T18:17:53.566 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.550+0000 7f1f15ffb700 1 -- 172.21.15.42:0/1715371165 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1f24016070 con 0x7f1f280703b0 2024-02-15T18:17:53.747 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.746+0000 7f1f2ca0d700 1 -- 172.21.15.42:0/1715371165 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f1f1002ff70 msgr2=0x7f1f10032430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/481027993 2024-02-15T18:17:53.747 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.746+0000 7f1f2ca0d700 1 --2- 172.21.15.42:0/1715371165 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f1f1002ff70 0x7f1f10032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-15T18:17:53.750 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.749+0000 7f1f2f472700 1 -- 172.21.15.42:0/1715371165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f1f28102150 con 0x7f1f280703b0 2024-02-15T18:17:53.751 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.750+0000 7f1f15ffb700 1 -- 172.21.15.42:0/1715371165 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v13) v1 ==== 56+0+104 (secure 0 0 0) 0x7f1f240224f0 con 0x7f1f280703b0 2024-02-15T18:17:53.751 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: { 2024-02-15T18:17:53.752 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "epoch": 13, 2024-02-15T18:17:53.752 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "available": true, 2024-02-15T18:17:53.752 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "active_name": "smithi042.kfoihb", 2024-02-15T18:17:53.752 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-15T18:17:53.752 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: } 2024-02-15T18:17:53.754 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.752+0000 7f1f2f472700 1 -- 172.21.15.42:0/1715371165 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f1f1002ff70 msgr2=0x7f1f10032430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:17:53.754 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.752+0000 7f1f2f472700 1 --2- 172.21.15.42:0/1715371165 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f1f1002ff70 0x7f1f10032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:17:53.754 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.752+0000 7f1f2f472700 1 -- 172.21.15.42:0/1715371165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f280703b0 msgr2=0x7f1f2810cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:53.754 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.752+0000 7f1f2f472700 1 --2- 172.21.15.42:0/1715371165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f280703b0 0x7f1f2810cda0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f1f2400e040 tx=0x7f1f24006ee0).stop 2024-02-15T18:17:53.755 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.752+0000 7f1f2f472700 1 -- 172.21.15.42:0/1715371165 shutdown_connections 2024-02-15T18:17:53.755 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.752+0000 7f1f2f472700 1 --2- 172.21.15.42:0/1715371165 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f1f1002ff70 0x7f1f10032430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:17:53.755 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.752+0000 7f1f2f472700 1 --2- 172.21.15.42:0/1715371165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f280703b0 0x7f1f2810cda0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:53.755 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.752+0000 7f1f2f472700 1 -- 172.21.15.42:0/1715371165 >> 172.21.15.42:0/1715371165 conn(0x7f1f280f6530 msgr2=0x7f1f280f7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:53.755 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.752+0000 7f1f2f472700 1 -- 172.21.15.42:0/1715371165 shutdown_connections 2024-02-15T18:17:53.755 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:53.752+0000 7f1f2f472700 1 -- 172.21.15.42:0/1715371165 wait complete. 2024-02-15T18:17:54.062 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:53.752037+0000 mon.smithi042 (mon.0) 90 : audit [DBG] from='client.? 172.21.15.42:0/1715371165' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-15T18:17:54.534 INFO:teuthology.orchestra.run.smithi042.stderr:Waiting for the mgr to restart... 2024-02-15T18:17:54.548 INFO:teuthology.orchestra.run.smithi042.stderr:Waiting for mgr epoch 13... 2024-02-15T18:17:54.548 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 tell mgr mgr_status 2024-02-15T18:17:55.171 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.136+0000 7f33441a8700 1 Processor -- start 2024-02-15T18:17:55.171 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.137+0000 7f33441a8700 1 -- start start 2024-02-15T18:17:55.171 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.137+0000 7f33441a8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c0703d0 0x7f333c100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.137+0000 7f33441a8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c0fc8a0 con 0x7f333c0703d0 2024-02-15T18:17:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.137+0000 7f3341f44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c0703d0 0x7f333c100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.137+0000 7f3341f44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c0703d0 0x7f333c100eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56824/0 (socket says 172.21.15.42:56824) 2024-02-15T18:17:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.137+0000 7f3341f44700 1 -- 172.21.15.42:0/2516356998 learned_addr learned my addr 172.21.15.42:0/2516356998 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.137+0000 7f3341f44700 1 -- 172.21.15.42:0/2516356998 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c1013a0 con 0x7f333c0703d0 2024-02-15T18:17:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.137+0000 7f3341f44700 1 --2- 172.21.15.42:0/2516356998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c0703d0 0x7f333c100eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f333800b5f0 tx=0x7f3338006ee0).ready entity=mon.0 client_cookie=e93e1f0435d7c48a server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.137+0000 7f3340f42700 1 -- 172.21.15.42:0/2516356998 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f333800d600 con 0x7f333c0703d0 2024-02-15T18:17:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.137+0000 7f3340f42700 1 -- 172.21.15.42:0/2516356998 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f333800dc30 con 0x7f333c0703d0 2024-02-15T18:17:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.137+0000 7f3340f42700 1 -- 172.21.15.42:0/2516356998 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3338011d30 con 0x7f333c0703d0 2024-02-15T18:17:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.138+0000 7f33441a8700 1 -- 172.21.15.42:0/2516356998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c0703d0 msgr2=0x7f333c100eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.138+0000 7f33441a8700 1 --2- 172.21.15.42:0/2516356998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c0703d0 0x7f333c100eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f333800b5f0 tx=0x7f3338006ee0).stop 2024-02-15T18:17:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.138+0000 7f33441a8700 1 -- 172.21.15.42:0/2516356998 shutdown_connections 2024-02-15T18:17:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.138+0000 7f33441a8700 1 --2- 172.21.15.42:0/2516356998 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c0703d0 0x7f333c100eb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:55.172 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.138+0000 7f33441a8700 1 -- 172.21.15.42:0/2516356998 >> 172.21.15.42:0/2516356998 conn(0x7f333c0f6530 msgr2=0x7f333c0f8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.138+0000 7f33441a8700 1 -- 172.21.15.42:0/2516356998 shutdown_connections 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.138+0000 7f33441a8700 1 -- 172.21.15.42:0/2516356998 wait complete. 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.139+0000 7f33441a8700 1 Processor -- start 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.139+0000 7f33441a8700 1 -- start start 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.139+0000 7f33441a8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c0703d0 0x7f333c10ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.139+0000 7f33441a8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333c10d330 con 0x7f333c0703d0 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.139+0000 7f3341f44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c0703d0 0x7f333c10ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.139+0000 7f3341f44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c0703d0 0x7f333c10ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56834/0 (socket says 172.21.15.42:56834) 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.139+0000 7f3341f44700 1 -- 172.21.15.42:0/4122863757 learned_addr learned my addr 172.21.15.42:0/4122863757 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.139+0000 7f3341f44700 1 -- 172.21.15.42:0/4122863757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333800b200 con 0x7f333c0703d0 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.139+0000 7f3341f44700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c0703d0 0x7f333c10ce40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f333800c010 tx=0x7f3338006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.140+0000 7f332a7fc700 1 -- 172.21.15.42:0/4122863757 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f333800da60 con 0x7f333c0703d0 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.140+0000 7f33441a8700 1 -- 172.21.15.42:0/4122863757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333c10d530 con 0x7f333c0703d0 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.140+0000 7f332a7fc700 1 -- 172.21.15.42:0/4122863757 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f333801b8e0 con 0x7f333c0703d0 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.140+0000 7f332a7fc700 1 -- 172.21.15.42:0/4122863757 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f333801a990 con 0x7f333c0703d0 2024-02-15T18:17:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.140+0000 7f33441a8700 1 -- 172.21.15.42:0/4122863757 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333c10e140 con 0x7f333c0703d0 2024-02-15T18:17:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.140+0000 7f332a7fc700 1 -- 172.21.15.42:0/4122863757 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 36208+0+0 (secure 0 0 0) 0x7f333800f070 con 0x7f333c0703d0 2024-02-15T18:17:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.141+0000 7f332a7fc700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f332402ff70 0x7f3324032430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.141+0000 7f33441a8700 1 -- 172.21.15.42:0/4122863757 --> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f333c059100 con 0x7f332402ff70 2024-02-15T18:17:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.141+0000 7f332a7fc700 1 -- 172.21.15.42:0/4122863757 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f3338041420 con 0x7f333c0703d0 2024-02-15T18:17:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.141+0000 7f3341743700 1 -- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f332402ff70 msgr2=0x7f3324032430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/481027993 2024-02-15T18:17:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.141+0000 7f3341743700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f332402ff70 0x7f3324032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:17:55.342 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.342+0000 7f3341743700 1 -- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f332402ff70 msgr2=0x7f3324032430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/481027993 2024-02-15T18:17:55.342 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.342+0000 7f3341743700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f332402ff70 0x7f3324032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-15T18:17:55.743 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.743+0000 7f3341743700 1 -- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f332402ff70 msgr2=0x7f3324032430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/481027993 2024-02-15T18:17:55.743 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:55.743+0000 7f3341743700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f332402ff70 0x7f3324032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-15T18:17:56.545 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:56.544+0000 7f3341743700 1 -- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f332402ff70 msgr2=0x7f3324032430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/481027993 2024-02-15T18:17:56.545 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:56.544+0000 7f3341743700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f332402ff70 0x7f3324032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-15T18:17:56.756 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:56.755+0000 7f332a7fc700 1 -- 172.21.15.42:0/4122863757 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mgrmap(e 14) v1 ==== 35997+0+0 (secure 0 0 0) 0x7f333801bba0 con 0x7f333c0703d0 2024-02-15T18:17:56.756 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:56.755+0000 7f332a7fc700 1 -- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f332402ff70 msgr2=0x7f3324032430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:17:56.756 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:56.755+0000 7f332a7fc700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f332402ff70 0x7f3324032430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:17:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.757+0000 7f332a7fc700 1 -- 172.21.15.42:0/4122863757 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 15) v1 ==== 36067+0+0 (secure 0 0 0) 0x7f3338025460 con 0x7f333c0703d0 2024-02-15T18:17:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.757+0000 7f332a7fc700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3324033a50 0x7f3324035e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.757+0000 7f332a7fc700 1 -- 172.21.15.42:0/4122863757 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f333c059100 con 0x7f3324033a50 2024-02-15T18:17:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.758+0000 7f3341743700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3324033a50 0x7f3324035e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.758+0000 7f3341743700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3324033a50 0x7f3324035e40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f332c0036d0 tx=0x7f332c006ca0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:57.759 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.759+0000 7f332a7fc700 1 -- 172.21.15.42:0/4122863757 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7f333c059100 con 0x7f3324033a50 2024-02-15T18:17:57.767 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.765+0000 7f33441a8700 1 -- 172.21.15.42:0/4122863757 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f333c10d9c0 con 0x7f3324033a50 2024-02-15T18:17:57.767 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.765+0000 7f332a7fc700 1 -- 172.21.15.42:0/4122863757 <== mgr.14162 v2:172.21.15.42:6800/1691818213 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f333c10d9c0 con 0x7f3324033a50 2024-02-15T18:17:57.767 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: { 2024-02-15T18:17:57.767 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "mgrmap_epoch": 15, 2024-02-15T18:17:57.767 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: "initialized": true 2024-02-15T18:17:57.767 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: } 2024-02-15T18:17:57.767 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.765+0000 7f33441a8700 1 -- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3324033a50 msgr2=0x7f3324035e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:57.767 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.765+0000 7f33441a8700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3324033a50 0x7f3324035e40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f332c0036d0 tx=0x7f332c006ca0).stop 2024-02-15T18:17:57.767 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.765+0000 7f33441a8700 1 -- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c0703d0 msgr2=0x7f333c10ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:57.767 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.765+0000 7f33441a8700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c0703d0 0x7f333c10ce40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f333800c010 tx=0x7f3338006ee0).stop 2024-02-15T18:17:57.768 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.766+0000 7f33441a8700 1 -- 172.21.15.42:0/4122863757 shutdown_connections 2024-02-15T18:17:57.768 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.766+0000 7f33441a8700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3324033a50 0x7f3324035e40 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:57.768 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.766+0000 7f33441a8700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f333c0703d0 0x7f333c10ce40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:57.768 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.766+0000 7f33441a8700 1 --2- 172.21.15.42:0/4122863757 >> [v2:172.21.15.42:6800/481027993,v1:172.21.15.42:6801/481027993] conn(0x7f332402ff70 0x7f3324032430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:17:57.768 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.766+0000 7f33441a8700 1 -- 172.21.15.42:0/4122863757 >> 172.21.15.42:0/4122863757 conn(0x7f333c0f6530 msgr2=0x7f333c0f7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:57.768 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.766+0000 7f33441a8700 1 -- 172.21.15.42:0/4122863757 shutdown_connections 2024-02-15T18:17:57.768 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:57.766+0000 7f33441a8700 1 -- 172.21.15.42:0/4122863757 wait complete. 2024-02-15T18:17:58.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:56.704975+0000 mon.smithi042 (mon.0) 91 : cluster [INF] Active manager daemon smithi042.kfoihb restarted 2024-02-15T18:17:58.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:56.705366+0000 mon.smithi042 (mon.0) 92 : cluster [INF] Activating manager daemon smithi042.kfoihb 2024-02-15T18:17:58.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:56.707060+0000 mon.smithi042 (mon.0) 93 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-15T18:17:58.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:56.757838+0000 mon.smithi042 (mon.0) 94 : cluster [DBG] mgrmap e14: smithi042.kfoihb(active, starting, since 0.0525567s) 2024-02-15T18:17:58.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:56.759058+0000 mon.smithi042 (mon.0) 95 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-02-15T18:17:58.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:56.759589+0000 mon.smithi042 (mon.0) 96 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr metadata", "who": "smithi042.kfoihb", "id": "smithi042.kfoihb"}]: dispatch 2024-02-15T18:17:58.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:56.760267+0000 mon.smithi042 (mon.0) 97 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:17:58.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:56.760511+0000 mon.smithi042 (mon.0) 98 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:17:58.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:56.760734+0000 mon.smithi042 (mon.0) 99 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:17:58.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:56.774816+0000 mon.smithi042 (mon.0) 100 : cluster [INF] Manager daemon smithi042.kfoihb is now available 2024-02-15T18:17:58.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:56.878467+0000 mon.smithi042 (mon.0) 101 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:17:58.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:56.880599+0000 mon.smithi042 (mon.0) 102 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:17:58.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:56.898108+0000 mon.smithi042 (mon.0) 103 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:17:58.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:56.907166+0000 mon.smithi042 (mon.0) 104 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:17:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:mgr epoch 13 is available 2024-02-15T18:17:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:Generating a dashboard self-signed certificate... 2024-02-15T18:17:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 dashboard create-self-signed-cert 2024-02-15T18:17:58.873 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:57.759728+0000 mgr.smithi042.kfoihb (mgr.14162) 1 : audit [DBG] from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-15T18:17:58.873 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:57.760135+0000 mon.smithi042 (mon.0) 105 : cluster [DBG] mgrmap e15: smithi042.kfoihb(active, since 1.05485s) 2024-02-15T18:17:58.873 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:57.766473+0000 mgr.smithi042.kfoihb (mgr.14162) 2 : audit [DBG] from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-15T18:17:58.873 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:17:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:58.409514+0000 mon.smithi042 (mon.0) 106 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.616+0000 7fa9124f0700 1 Processor -- start 2024-02-15T18:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.616+0000 7fa9124f0700 1 -- start start 2024-02-15T18:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.616+0000 7fa9124f0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c0703f0 0x7fa90c100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.616+0000 7fa9124f0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c0fc6c0 con 0x7fa90c0703f0 2024-02-15T18:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.616+0000 7fa9114ee700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c0703f0 0x7fa90c100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.616+0000 7fa9114ee700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c0703f0 0x7fa90c100cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56892/0 (socket says 172.21.15.42:56892) 2024-02-15T18:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.616+0000 7fa9114ee700 1 -- 172.21.15.42:0/4086784724 learned_addr learned my addr 172.21.15.42:0/4086784724 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.617+0000 7fa9114ee700 1 -- 172.21.15.42:0/4086784724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c1011c0 con 0x7fa90c0703f0 2024-02-15T18:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.617+0000 7fa9114ee700 1 --2- 172.21.15.42:0/4086784724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c0703f0 0x7fa90c100cd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fa90800b5f0 tx=0x7fa908006ee0).ready entity=mon.0 client_cookie=33bdffc52ece7651 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.617+0000 7fa8fbfff700 1 -- 172.21.15.42:0/4086784724 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa90800d600 con 0x7fa90c0703f0 2024-02-15T18:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.617+0000 7fa8fbfff700 1 -- 172.21.15.42:0/4086784724 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa90800dc30 con 0x7fa90c0703f0 2024-02-15T18:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.617+0000 7fa8fbfff700 1 -- 172.21.15.42:0/4086784724 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa908011d60 con 0x7fa90c0703f0 2024-02-15T18:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.617+0000 7fa9124f0700 1 -- 172.21.15.42:0/4086784724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c0703f0 msgr2=0x7fa90c100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.617+0000 7fa9124f0700 1 --2- 172.21.15.42:0/4086784724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c0703f0 0x7fa90c100cd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fa90800b5f0 tx=0x7fa908006ee0).stop 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.617+0000 7fa9124f0700 1 -- 172.21.15.42:0/4086784724 shutdown_connections 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.617+0000 7fa9124f0700 1 --2- 172.21.15.42:0/4086784724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c0703f0 0x7fa90c100cd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.617+0000 7fa9124f0700 1 -- 172.21.15.42:0/4086784724 >> 172.21.15.42:0/4086784724 conn(0x7fa90c0f6390 msgr2=0x7fa90c0f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.618+0000 7fa9124f0700 1 -- 172.21.15.42:0/4086784724 shutdown_connections 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.618+0000 7fa9124f0700 1 -- 172.21.15.42:0/4086784724 wait complete. 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.618+0000 7fa9124f0700 1 Processor -- start 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.618+0000 7fa9124f0700 1 -- start start 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.618+0000 7fa9124f0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c0703f0 0x7fa90c10cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.618+0000 7fa9124f0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c10d0c0 con 0x7fa90c0703f0 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.618+0000 7fa9114ee700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c0703f0 0x7fa90c10cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.618+0000 7fa9114ee700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c0703f0 0x7fa90c10cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56902/0 (socket says 172.21.15.42:56902) 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.618+0000 7fa9114ee700 1 -- 172.21.15.42:0/1233436875 learned_addr learned my addr 172.21.15.42:0/1233436875 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.619+0000 7fa9114ee700 1 -- 172.21.15.42:0/1233436875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90800b200 con 0x7fa90c0703f0 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.619+0000 7fa9114ee700 1 --2- 172.21.15.42:0/1233436875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c0703f0 0x7fa90c10cbd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fa908006b80 tx=0x7fa908006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.619+0000 7fa8f9ffb700 1 -- 172.21.15.42:0/1233436875 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa908011e10 con 0x7fa90c0703f0 2024-02-15T18:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.619+0000 7fa9124f0700 1 -- 172.21.15.42:0/1233436875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90c10d2c0 con 0x7fa90c0703f0 2024-02-15T18:17:59.631 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.619+0000 7fa8f9ffb700 1 -- 172.21.15.42:0/1233436875 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa908011610 con 0x7fa90c0703f0 2024-02-15T18:17:59.631 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.619+0000 7fa8f9ffb700 1 -- 172.21.15.42:0/1233436875 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa90801a7b0 con 0x7fa90c0703f0 2024-02-15T18:17:59.631 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.619+0000 7fa9124f0700 1 -- 172.21.15.42:0/1233436875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90c10d750 con 0x7fa90c0703f0 2024-02-15T18:17:59.631 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.620+0000 7fa8f9ffb700 1 -- 172.21.15.42:0/1233436875 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 36208+0+0 (secure 0 0 0) 0x7fa90800f070 con 0x7fa90c0703f0 2024-02-15T18:17:59.631 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.620+0000 7fa8f9ffb700 1 --2- 172.21.15.42:0/1233436875 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa8f402ffc0 0x7fa8f4032480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:17:59.631 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.620+0000 7fa8f9ffb700 1 -- 172.21.15.42:0/1233436875 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fa908041120 con 0x7fa90c0703f0 2024-02-15T18:17:59.631 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.620+0000 7fa9124f0700 1 -- 172.21.15.42:0/1233436875 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90c045ad0 con 0x7fa90c0703f0 2024-02-15T18:17:59.631 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.620+0000 7fa910ced700 1 --2- 172.21.15.42:0/1233436875 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa8f402ffc0 0x7fa8f4032480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:17:59.631 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.621+0000 7fa910ced700 1 --2- 172.21.15.42:0/1233436875 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa8f402ffc0 0x7fa8f4032480 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fa8fc0096a0 tx=0x7fa8fc006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:17:59.631 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.625+0000 7fa8f9ffb700 1 -- 172.21.15.42:0/1233436875 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa908016070 con 0x7fa90c0703f0 2024-02-15T18:17:59.790 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.789+0000 7fa9124f0700 1 -- 172.21.15.42:0/1233436875 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fa90c10dfb0 con 0x7fa8f402ffc0 2024-02-15T18:17:59.829 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.828+0000 7fa8f9ffb700 1 -- 172.21.15.42:0/1233436875 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fa90c10dfb0 con 0x7fa8f402ffc0 2024-02-15T18:17:59.829 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: Self-signed certificate created 2024-02-15T18:17:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.830+0000 7fa9124f0700 1 -- 172.21.15.42:0/1233436875 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa8f402ffc0 msgr2=0x7fa8f4032480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.830+0000 7fa9124f0700 1 --2- 172.21.15.42:0/1233436875 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa8f402ffc0 0x7fa8f4032480 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fa8fc0096a0 tx=0x7fa8fc006b40).stop 2024-02-15T18:17:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.830+0000 7fa9124f0700 1 -- 172.21.15.42:0/1233436875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c0703f0 msgr2=0x7fa90c10cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:17:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.830+0000 7fa9124f0700 1 --2- 172.21.15.42:0/1233436875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c0703f0 0x7fa90c10cbd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fa908006b80 tx=0x7fa908006ee0).stop 2024-02-15T18:17:59.831 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.830+0000 7fa9124f0700 1 -- 172.21.15.42:0/1233436875 shutdown_connections 2024-02-15T18:17:59.832 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.830+0000 7fa9124f0700 1 --2- 172.21.15.42:0/1233436875 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa8f402ffc0 0x7fa8f4032480 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:59.832 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.830+0000 7fa9124f0700 1 --2- 172.21.15.42:0/1233436875 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa90c0703f0 0x7fa90c10cbd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:17:59.832 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.830+0000 7fa9124f0700 1 -- 172.21.15.42:0/1233436875 >> 172.21.15.42:0/1233436875 conn(0x7fa90c0f6390 msgr2=0x7fa90c0f7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:17:59.832 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.830+0000 7fa9124f0700 1 -- 172.21.15.42:0/1233436875 shutdown_connections 2024-02-15T18:17:59.832 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:17:59.830+0000 7fa9124f0700 1 -- 172.21.15.42:0/1233436875 wait complete. 2024-02-15T18:18:00.679 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:17:59.410957+0000 mon.smithi042 (mon.0) 107 : cluster [DBG] mgrmap e16: smithi042.kfoihb(active, since 2s) 2024-02-15T18:18:00.679 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:59.827596+0000 mon.smithi042 (mon.0) 108 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:18:00.679 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:59.829551+0000 mon.smithi042 (mon.0) 109 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:18:01.429 INFO:teuthology.orchestra.run.smithi042.stderr:Creating initial admin user... 2024-02-15T18:18:01.429 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmp9y779xjy:/tmp/dashboard.pw:z docker.io/ceph/ceph:v16.2.4 dashboard ac-user-create admin -i /tmp/dashboard.pw administrator --force-password --pwd-update-required 2024-02-15T18:18:01.687 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:17:59.791174+0000 mgr.smithi042.kfoihb (mgr.14162) 3 : audit [DBG] from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.001+0000 7f9ebc9a2700 1 Processor -- start 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.001+0000 7f9ebc9a2700 1 -- start start 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.001+0000 7f9ebc9a2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb80fab30 0x7f9eb80faf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.001+0000 7f9ebc9a2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80fb440 con 0x7f9eb80fab30 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.001+0000 7f9eb6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb80fab30 0x7f9eb80faf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.001+0000 7f9eb6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb80fab30 0x7f9eb80faf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53110/0 (socket says 172.21.15.42:53110) 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.001+0000 7f9eb6d9d700 1 -- 172.21.15.42:0/119142213 learned_addr learned my addr 172.21.15.42:0/119142213 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.002+0000 7f9eb6d9d700 1 -- 172.21.15.42:0/119142213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb80fb580 con 0x7f9eb80fab30 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.002+0000 7f9eb6d9d700 1 --2- 172.21.15.42:0/119142213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb80fab30 0x7f9eb80faf50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f9ea800b5f0 tx=0x7f9ea8006ee0).ready entity=mon.0 client_cookie=8e7011d685884927 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.002+0000 7f9eb5d9b700 1 -- 172.21.15.42:0/119142213 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ea800d600 con 0x7f9eb80fab30 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.002+0000 7f9eb5d9b700 1 -- 172.21.15.42:0/119142213 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ea800dc30 con 0x7f9eb80fab30 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.002+0000 7f9eb5d9b700 1 -- 172.21.15.42:0/119142213 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ea8011d30 con 0x7f9eb80fab30 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.003+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/119142213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb80fab30 msgr2=0x7f9eb80faf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.003+0000 7f9ebc9a2700 1 --2- 172.21.15.42:0/119142213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb80fab30 0x7f9eb80faf50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f9ea800b5f0 tx=0x7f9ea8006ee0).stop 2024-02-15T18:18:02.019 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.003+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/119142213 shutdown_connections 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.003+0000 7f9ebc9a2700 1 --2- 172.21.15.42:0/119142213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb80fab30 0x7f9eb80faf50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.003+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/119142213 >> 172.21.15.42:0/119142213 conn(0x7f9eb80f6390 msgr2=0x7f9eb80f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.003+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/119142213 shutdown_connections 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.003+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/119142213 wait complete. 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.003+0000 7f9ebc9a2700 1 Processor -- start 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.003+0000 7f9ebc9a2700 1 -- start start 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.004+0000 7f9ebc9a2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb80fab30 0x7f9eb810d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.004+0000 7f9ebc9a2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80704b0 con 0x7f9eb80fab30 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.004+0000 7f9eb6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb80fab30 0x7f9eb810d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.004+0000 7f9eb6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb80fab30 0x7f9eb810d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53112/0 (socket says 172.21.15.42:53112) 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.004+0000 7f9eb6d9d700 1 -- 172.21.15.42:0/945851397 learned_addr learned my addr 172.21.15.42:0/945851397 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.004+0000 7f9eb6d9d700 1 -- 172.21.15.42:0/945851397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea800b200 con 0x7f9eb80fab30 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.004+0000 7f9eb6d9d700 1 --2- 172.21.15.42:0/945851397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb80fab30 0x7f9eb810d700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f9ea800e040 tx=0x7f9ea8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.004+0000 7f9e9f7fe700 1 -- 172.21.15.42:0/945851397 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ea800d9e0 con 0x7f9eb80fab30 2024-02-15T18:18:02.020 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.004+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/945851397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb80706b0 con 0x7f9eb80fab30 2024-02-15T18:18:02.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.004+0000 7f9e9f7fe700 1 -- 172.21.15.42:0/945851397 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ea801b8e0 con 0x7f9eb80fab30 2024-02-15T18:18:02.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.004+0000 7f9e9f7fe700 1 -- 172.21.15.42:0/945851397 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ea801a880 con 0x7f9eb80fab30 2024-02-15T18:18:02.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.004+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/945851397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb810de30 con 0x7f9eb80fab30 2024-02-15T18:18:02.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.005+0000 7f9e9f7fe700 1 -- 172.21.15.42:0/945851397 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 36208+0+0 (secure 0 0 0) 0x7f9ea800f070 con 0x7f9eb80fab30 2024-02-15T18:18:02.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.005+0000 7f9e9f7fe700 1 --2- 172.21.15.42:0/945851397 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9ea402ff70 0x7f9ea4032430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:02.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.005+0000 7f9e9f7fe700 1 -- 172.21.15.42:0/945851397 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f9ea80424d0 con 0x7f9eb80fab30 2024-02-15T18:18:02.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.005+0000 7f9eb659c700 1 --2- 172.21.15.42:0/945851397 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9ea402ff70 0x7f9ea4032430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:02.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.006+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/945851397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb8045ad0 con 0x7f9eb80fab30 2024-02-15T18:18:02.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.006+0000 7f9eb659c700 1 --2- 172.21.15.42:0/945851397 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9ea402ff70 0x7f9ea4032430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f9eac0096a0 tx=0x7f9eac006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:02.021 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.010+0000 7f9e9f7fe700 1 -- 172.21.15.42:0/945851397 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9ea8016070 con 0x7f9eb80fab30 2024-02-15T18:18:02.186 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.185+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/945851397 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f9eb80fca10 con 0x7f9ea402ff70 2024-02-15T18:18:02.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.431+0000 7f9e9f7fe700 1 -- 172.21.15.42:0/945851397 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f9eb80fca10 con 0x7f9ea402ff70 2024-02-15T18:18:02.431 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$WNVWJhmoHp0N.VJnlUBsbuXwF8FooebcD4g4LkbZAnGIQ.uFs5j22", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708021082, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-15T18:18:02.433 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.432+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/945851397 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9ea402ff70 msgr2=0x7f9ea4032430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.432+0000 7f9ebc9a2700 1 --2- 172.21.15.42:0/945851397 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9ea402ff70 0x7f9ea4032430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f9eac0096a0 tx=0x7f9eac006b40).stop 2024-02-15T18:18:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.432+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/945851397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb80fab30 msgr2=0x7f9eb810d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.432+0000 7f9ebc9a2700 1 --2- 172.21.15.42:0/945851397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb80fab30 0x7f9eb810d700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f9ea800e040 tx=0x7f9ea8006ee0).stop 2024-02-15T18:18:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.433+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/945851397 shutdown_connections 2024-02-15T18:18:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.433+0000 7f9ebc9a2700 1 --2- 172.21.15.42:0/945851397 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9ea402ff70 0x7f9ea4032430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.433+0000 7f9ebc9a2700 1 --2- 172.21.15.42:0/945851397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb80fab30 0x7f9eb810d700 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.433+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/945851397 >> 172.21.15.42:0/945851397 conn(0x7f9eb80f6390 msgr2=0x7f9eb80f7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.433+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/945851397 shutdown_connections 2024-02-15T18:18:02.434 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:02.433+0000 7f9ebc9a2700 1 -- 172.21.15.42:0/945851397 wait complete. 2024-02-15T18:18:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:Fetching dashboard port number... 2024-02-15T18:18:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi042 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715:/var/log/ceph:z -v /tmp/ceph-tmpgycmxaec:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpu6woidlw:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config get mgr mgr/dashboard/ssl_server_port 2024-02-15T18:18:03.682 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.680+0000 7f1dc98cf700 1 Processor -- start 2024-02-15T18:18:03.682 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.680+0000 7f1dc98cf700 1 -- start start 2024-02-15T18:18:03.682 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.680+0000 7f1dc98cf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dc40703b0 0x7f1dc4100d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:03.682 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.680+0000 7f1dc98cf700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc40fc700 con 0x7f1dc40703b0 2024-02-15T18:18:03.682 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.680+0000 7f1dc88cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dc40703b0 0x7f1dc4100d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:03.683 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.680+0000 7f1dc88cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dc40703b0 0x7f1dc4100d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53116/0 (socket says 172.21.15.42:53116) 2024-02-15T18:18:03.683 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.680+0000 7f1dc88cd700 1 -- 172.21.15.42:0/1337512092 learned_addr learned my addr 172.21.15.42:0/1337512092 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:18:03.683 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.681+0000 7f1dc88cd700 1 -- 172.21.15.42:0/1337512092 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc4101200 con 0x7f1dc40703b0 2024-02-15T18:18:03.683 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.681+0000 7f1dc88cd700 1 --2- 172.21.15.42:0/1337512092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dc40703b0 0x7f1dc4100d10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f1db800b5f0 tx=0x7f1db8006ee0).ready entity=mon.0 client_cookie=5619a54a830045f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:03.683 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.681+0000 7f1dc37fe700 1 -- 172.21.15.42:0/1337512092 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1db800d600 con 0x7f1dc40703b0 2024-02-15T18:18:03.683 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.681+0000 7f1dc37fe700 1 -- 172.21.15.42:0/1337512092 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1db800dc30 con 0x7f1dc40703b0 2024-02-15T18:18:03.683 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.681+0000 7f1dc37fe700 1 -- 172.21.15.42:0/1337512092 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1db8011d30 con 0x7f1dc40703b0 2024-02-15T18:18:03.683 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.681+0000 7f1dc98cf700 1 -- 172.21.15.42:0/1337512092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dc40703b0 msgr2=0x7f1dc4100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:03.683 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.681+0000 7f1dc98cf700 1 --2- 172.21.15.42:0/1337512092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dc40703b0 0x7f1dc4100d10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f1db800b5f0 tx=0x7f1db8006ee0).stop 2024-02-15T18:18:03.683 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.682+0000 7f1dc98cf700 1 -- 172.21.15.42:0/1337512092 shutdown_connections 2024-02-15T18:18:03.683 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.682+0000 7f1dc98cf700 1 --2- 172.21.15.42:0/1337512092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dc40703b0 0x7f1dc4100d10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:03.686 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.682+0000 7f1dc98cf700 1 -- 172.21.15.42:0/1337512092 >> 172.21.15.42:0/1337512092 conn(0x7f1dc40f6390 msgr2=0x7f1dc40f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:03.686 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.682+0000 7f1dc98cf700 1 -- 172.21.15.42:0/1337512092 shutdown_connections 2024-02-15T18:18:03.686 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.682+0000 7f1dc98cf700 1 -- 172.21.15.42:0/1337512092 wait complete. 2024-02-15T18:18:03.686 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.682+0000 7f1dc98cf700 1 Processor -- start 2024-02-15T18:18:03.686 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.682+0000 7f1dc98cf700 1 -- start start 2024-02-15T18:18:03.686 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.683+0000 7f1dc98cf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dc40703b0 0x7f1dc410cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:03.686 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.683+0000 7f1dc98cf700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc410d0b0 con 0x7f1dc40703b0 2024-02-15T18:18:03.686 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.683+0000 7f1dc88cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dc40703b0 0x7f1dc410cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:03.686 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.683+0000 7f1dc88cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dc40703b0 0x7f1dc410cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53120/0 (socket says 172.21.15.42:53120) 2024-02-15T18:18:03.686 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.683+0000 7f1dc88cd700 1 -- 172.21.15.42:0/2037096467 learned_addr learned my addr 172.21.15.42:0/2037096467 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:18:03.686 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.683+0000 7f1dc88cd700 1 -- 172.21.15.42:0/2037096467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db800b200 con 0x7f1dc40703b0 2024-02-15T18:18:03.687 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.683+0000 7f1dc88cd700 1 --2- 172.21.15.42:0/2037096467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dc40703b0 0x7f1dc410cbc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f1db800e040 tx=0x7f1db8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:03.687 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.683+0000 7f1dc17fa700 1 -- 172.21.15.42:0/2037096467 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1db800da60 con 0x7f1dc40703b0 2024-02-15T18:18:03.687 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.683+0000 7f1dc17fa700 1 -- 172.21.15.42:0/2037096467 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1db801b8e0 con 0x7f1dc40703b0 2024-02-15T18:18:03.687 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.683+0000 7f1dc98cf700 1 -- 172.21.15.42:0/2037096467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc410d2b0 con 0x7f1dc40703b0 2024-02-15T18:18:03.687 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.683+0000 7f1dc17fa700 1 -- 172.21.15.42:0/2037096467 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1db801a900 con 0x7f1dc40703b0 2024-02-15T18:18:03.687 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.683+0000 7f1dc98cf700 1 -- 172.21.15.42:0/2037096467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc410d740 con 0x7f1dc40703b0 2024-02-15T18:18:03.687 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.684+0000 7f1dc17fa700 1 -- 172.21.15.42:0/2037096467 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f1db800f070 con 0x7f1dc40703b0 2024-02-15T18:18:03.687 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.684+0000 7f1dc17fa700 1 --2- 172.21.15.42:0/2037096467 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f1da8030020 0x7f1da80324e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:03.687 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.684+0000 7f1dc17fa700 1 -- 172.21.15.42:0/2037096467 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f1db8041910 con 0x7f1dc40703b0 2024-02-15T18:18:03.687 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.684+0000 7f1dc98cf700 1 -- 172.21.15.42:0/2037096467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc4045ad0 con 0x7f1dc40703b0 2024-02-15T18:18:03.687 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.684+0000 7f1dc3fff700 1 --2- 172.21.15.42:0/2037096467 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f1da8030020 0x7f1da80324e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:03.687 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.685+0000 7f1dc3fff700 1 --2- 172.21.15.42:0/2037096467 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f1da8030020 0x7f1da80324e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f1db00096a0 tx=0x7f1db0006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:03.689 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.689+0000 7f1dc17fa700 1 -- 172.21.15.42:0/2037096467 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1db8016020 con 0x7f1dc40703b0 2024-02-15T18:18:03.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:02.187565+0000 mgr.smithi042.kfoihb (mgr.14162) 4 : audit [DBG] from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:18:03.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:02.431699+0000 mon.smithi042 (mon.0) 110 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:18:03.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.846+0000 7f1dc98cf700 1 -- 172.21.15.42:0/2037096467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f1dc4101ef0 con 0x7f1dc40703b0 2024-02-15T18:18:03.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.847+0000 7f1dc17fa700 1 -- 172.21.15.42:0/2037096467 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v10) v1 ==== 112+0+5 (secure 0 0 0) 0x7f1db8022b90 con 0x7f1dc40703b0 2024-02-15T18:18:03.847 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 8443 2024-02-15T18:18:03.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.848+0000 7f1dc98cf700 1 -- 172.21.15.42:0/2037096467 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f1da8030020 msgr2=0x7f1da80324e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:03.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.848+0000 7f1dc98cf700 1 --2- 172.21.15.42:0/2037096467 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f1da8030020 0x7f1da80324e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f1db00096a0 tx=0x7f1db0006b40).stop 2024-02-15T18:18:03.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.848+0000 7f1dc98cf700 1 -- 172.21.15.42:0/2037096467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dc40703b0 msgr2=0x7f1dc410cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:03.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.848+0000 7f1dc98cf700 1 --2- 172.21.15.42:0/2037096467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dc40703b0 0x7f1dc410cbc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f1db800e040 tx=0x7f1db8006ee0).stop 2024-02-15T18:18:03.852 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.848+0000 7f1dc98cf700 1 -- 172.21.15.42:0/2037096467 shutdown_connections 2024-02-15T18:18:03.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.848+0000 7f1dc98cf700 1 --2- 172.21.15.42:0/2037096467 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f1da8030020 0x7f1da80324e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:03.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.848+0000 7f1dc98cf700 1 --2- 172.21.15.42:0/2037096467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1dc40703b0 0x7f1dc410cbc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:03.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.848+0000 7f1dc98cf700 1 -- 172.21.15.42:0/2037096467 >> 172.21.15.42:0/2037096467 conn(0x7f1dc40f6390 msgr2=0x7f1dc40f7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:03.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.848+0000 7f1dc98cf700 1 -- 172.21.15.42:0/2037096467 shutdown_connections 2024-02-15T18:18:03.853 INFO:teuthology.orchestra.run.smithi042.stderr:/usr/bin/ceph: 2024-02-15T18:18:03.848+0000 7f1dc98cf700 1 -- 172.21.15.42:0/2037096467 wait complete. 2024-02-15T18:18:04.519 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-enabled firewalld.service 2024-02-15T18:18:04.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:03.434971+0000 mon.smithi042 (mon.0) 111 : cluster [DBG] mgrmap e17: smithi042.kfoihb(active, since 6s) 2024-02-15T18:18:04.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:03.847953+0000 mon.smithi042 (mon.0) 112 : audit [DBG] from='client.? 172.21.15.42:0/2037096467' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-15T18:18:04.530 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: disabled 2024-02-15T18:18:04.531 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: systemctl is-active firewalld.service 2024-02-15T18:18:04.540 INFO:teuthology.orchestra.run.smithi042.stderr:systemctl: inactive 2024-02-15T18:18:04.540 INFO:teuthology.orchestra.run.smithi042.stderr:firewalld.service is not enabled 2024-02-15T18:18:04.540 INFO:teuthology.orchestra.run.smithi042.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-15T18:18:04.543 INFO:teuthology.orchestra.run.smithi042.stderr:Ceph Dashboard is now available at: 2024-02-15T18:18:04.544 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-02-15T18:18:04.544 INFO:teuthology.orchestra.run.smithi042.stderr: URL: https://smithi042.front.sepia.ceph.com:8443/ 2024-02-15T18:18:04.544 INFO:teuthology.orchestra.run.smithi042.stderr: User: admin 2024-02-15T18:18:04.544 INFO:teuthology.orchestra.run.smithi042.stderr: Password: zomo82mk9k 2024-02-15T18:18:04.544 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-02-15T18:18:04.545 INFO:teuthology.orchestra.run.smithi042.stderr:You can access the Ceph CLI with: 2024-02-15T18:18:04.545 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-02-15T18:18:04.545 INFO:teuthology.orchestra.run.smithi042.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-15T18:18:04.545 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-02-15T18:18:04.545 INFO:teuthology.orchestra.run.smithi042.stderr:Please consider enabling telemetry to help improve Ceph: 2024-02-15T18:18:04.545 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-02-15T18:18:04.545 INFO:teuthology.orchestra.run.smithi042.stderr: ceph telemetry on 2024-02-15T18:18:04.545 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-02-15T18:18:04.545 INFO:teuthology.orchestra.run.smithi042.stderr:For more information see: 2024-02-15T18:18:04.545 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-02-15T18:18:04.545 INFO:teuthology.orchestra.run.smithi042.stderr: https://docs.ceph.com/docs/pacific/mgr/telemetry/ 2024-02-15T18:18:04.546 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-02-15T18:18:04.546 INFO:teuthology.orchestra.run.smithi042.stderr:Bootstrap complete. 2024-02-15T18:18:04.546 INFO:teuthology.orchestra.run.smithi042.stderr:Releasing lock 139730620135912 on /run/cephadm/51458a44-cc2e-11ee-95ba-87774f69a715.lock 2024-02-15T18:18:04.546 INFO:teuthology.orchestra.run.smithi042.stderr:Lock 139730620135912 released on /run/cephadm/51458a44-cc2e-11ee-95ba-87774f69a715.lock 2024-02-15T18:18:04.586 INFO:tasks.cephadm:Fetching config... 2024-02-15T18:18:04.586 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:18:04.586 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-15T18:18:04.645 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-15T18:18:04.646 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:18:04.646 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-15T18:18:04.705 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-15T18:18:04.705 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:18:04.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/keyring of=/dev/stdout 2024-02-15T18:18:04.774 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-15T18:18:04.775 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:18:04.775 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-15T18:18:04.833 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-15T18:18:04.834 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB57nZ/+QBwSmgjDC+P/l2s8YhtFjUbiG7YsOzwK9ID0UDwGlrBEP008ijmSXa6yJseQNacBFhFTdu3M6qoI+A+lKu2DPqPV/E0fCppCE5cdqd12ptfRxVYxoPhoVx9odHVbyQWGI3ATjLsXZIz6uE2AvJx1ZjDiigVYRC6LxFRJTaN/+xWW/NNSwPIo+8784rt9fpu9xQbEpeaSsp2xK66XkD++J/sdS84IOqG3mN7b+KbjC9eRWUuFZcgqf3cI3I+6F11wrvlBIEl5ZyFbWf6IzYNlu1V5dJAnJik+tG2Sy4dZ7VW4ouy1TnaIURlXs7hA+WMLmvMdVPCXTapbapwzRhOFV3mS5j7VW3M8+UjqYMTY7Fxl8xkVxbNN40xlbgOGxn/9gqxGqRQoEaOffU/L9DVT50fpoqXwxIvsXtghjpDuN/j/lznc2NgpOqfFerAqlUIygJRVF4kxFIoyLmXl1MEfeTurFtbwMloCdxo27NAyxmRSAoKDrNP5yL3k= ceph-51458a44-cc2e-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-15T18:18:04.912 INFO:teuthology.orchestra.run.smithi042.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB57nZ/+QBwSmgjDC+P/l2s8YhtFjUbiG7YsOzwK9ID0UDwGlrBEP008ijmSXa6yJseQNacBFhFTdu3M6qoI+A+lKu2DPqPV/E0fCppCE5cdqd12ptfRxVYxoPhoVx9odHVbyQWGI3ATjLsXZIz6uE2AvJx1ZjDiigVYRC6LxFRJTaN/+xWW/NNSwPIo+8784rt9fpu9xQbEpeaSsp2xK66XkD++J/sdS84IOqG3mN7b+KbjC9eRWUuFZcgqf3cI3I+6F11wrvlBIEl5ZyFbWf6IzYNlu1V5dJAnJik+tG2Sy4dZ7VW4ouy1TnaIURlXs7hA+WMLmvMdVPCXTapbapwzRhOFV3mS5j7VW3M8+UjqYMTY7Fxl8xkVxbNN40xlbgOGxn/9gqxGqRQoEaOffU/L9DVT50fpoqXwxIvsXtghjpDuN/j/lznc2NgpOqfFerAqlUIygJRVF4kxFIoyLmXl1MEfeTurFtbwMloCdxo27NAyxmRSAoKDrNP5yL3k= ceph-51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:18:04.923 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB57nZ/+QBwSmgjDC+P/l2s8YhtFjUbiG7YsOzwK9ID0UDwGlrBEP008ijmSXa6yJseQNacBFhFTdu3M6qoI+A+lKu2DPqPV/E0fCppCE5cdqd12ptfRxVYxoPhoVx9odHVbyQWGI3ATjLsXZIz6uE2AvJx1ZjDiigVYRC6LxFRJTaN/+xWW/NNSwPIo+8784rt9fpu9xQbEpeaSsp2xK66XkD++J/sdS84IOqG3mN7b+KbjC9eRWUuFZcgqf3cI3I+6F11wrvlBIEl5ZyFbWf6IzYNlu1V5dJAnJik+tG2Sy4dZ7VW4ouy1TnaIURlXs7hA+WMLmvMdVPCXTapbapwzRhOFV3mS5j7VW3M8+UjqYMTY7Fxl8xkVxbNN40xlbgOGxn/9gqxGqRQoEaOffU/L9DVT50fpoqXwxIvsXtghjpDuN/j/lznc2NgpOqfFerAqlUIygJRVF4kxFIoyLmXl1MEfeTurFtbwMloCdxo27NAyxmRSAoKDrNP5yL3k= ceph-51458a44-cc2e-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-15T18:18:04.969 INFO:teuthology.orchestra.run.smithi191.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB57nZ/+QBwSmgjDC+P/l2s8YhtFjUbiG7YsOzwK9ID0UDwGlrBEP008ijmSXa6yJseQNacBFhFTdu3M6qoI+A+lKu2DPqPV/E0fCppCE5cdqd12ptfRxVYxoPhoVx9odHVbyQWGI3ATjLsXZIz6uE2AvJx1ZjDiigVYRC6LxFRJTaN/+xWW/NNSwPIo+8784rt9fpu9xQbEpeaSsp2xK66XkD++J/sdS84IOqG3mN7b+KbjC9eRWUuFZcgqf3cI3I+6F11wrvlBIEl5ZyFbWf6IzYNlu1V5dJAnJik+tG2Sy4dZ7VW4ouy1TnaIURlXs7hA+WMLmvMdVPCXTapbapwzRhOFV3mS5j7VW3M8+UjqYMTY7Fxl8xkVxbNN40xlbgOGxn/9gqxGqRQoEaOffU/L9DVT50fpoqXwxIvsXtghjpDuN/j/lznc2NgpOqfFerAqlUIygJRVF4kxFIoyLmXl1MEfeTurFtbwMloCdxo27NAyxmRSAoKDrNP5yL3k= ceph-51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:18:04.981 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-15T18:18:08.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.969+0000 7f514ef9d700 1 -- 172.21.15.42:0/3117270558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51480f68f0 msgr2=0x7f51480f6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:08.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.969+0000 7f514ef9d700 1 --2- 172.21.15.42:0/3117270558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51480f68f0 0x7f51480f6d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f5144009fb0 tx=0x7f5144009b30).stop 2024-02-15T18:18:08.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.969+0000 7f514ef9d700 1 -- 172.21.15.42:0/3117270558 shutdown_connections 2024-02-15T18:18:08.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.969+0000 7f514ef9d700 1 --2- 172.21.15.42:0/3117270558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51480f68f0 0x7f51480f6d10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.969+0000 7f514ef9d700 1 -- 172.21.15.42:0/3117270558 >> 172.21.15.42:0/3117270558 conn(0x7f51480f0120 msgr2=0x7f51480f2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.970+0000 7f514ef9d700 1 -- 172.21.15.42:0/3117270558 shutdown_connections 2024-02-15T18:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.970+0000 7f514ef9d700 1 -- 172.21.15.42:0/3117270558 wait complete. 2024-02-15T18:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.970+0000 7f514ef9d700 1 Processor -- start 2024-02-15T18:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.970+0000 7f514ef9d700 1 -- start start 2024-02-15T18:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.970+0000 7f514ef9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51480f68f0 0x7f5148108760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.970+0000 7f514ef9d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5148108c50 con 0x7f51480f68f0 2024-02-15T18:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.970+0000 7f514df9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51480f68f0 0x7f5148108760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.970+0000 7f514df9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51480f68f0 0x7f5148108760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53138/0 (socket says 172.21.15.42:53138) 2024-02-15T18:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.970+0000 7f514df9b700 1 -- 172.21.15.42:0/123359539 learned_addr learned my addr 172.21.15.42:0/123359539 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.971+0000 7f514df9b700 1 -- 172.21.15.42:0/123359539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5144005040 con 0x7f51480f68f0 2024-02-15T18:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.971+0000 7f514df9b700 1 --2- 172.21.15.42:0/123359539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51480f68f0 0x7f5148108760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f5144009830 tx=0x7f514400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.971+0000 7f513e7fc700 1 -- 172.21.15.42:0/123359539 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5144009320 con 0x7f51480f68f0 2024-02-15T18:18:08.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.971+0000 7f513e7fc700 1 -- 172.21.15.42:0/123359539 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5144013380 con 0x7f51480f68f0 2024-02-15T18:18:08.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.971+0000 7f514ef9d700 1 -- 172.21.15.42:0/123359539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5148108e50 con 0x7f51480f68f0 2024-02-15T18:18:08.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.971+0000 7f513e7fc700 1 -- 172.21.15.42:0/123359539 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5144014420 con 0x7f51480f68f0 2024-02-15T18:18:08.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.971+0000 7f514ef9d700 1 -- 172.21.15.42:0/123359539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51481092e0 con 0x7f51480f68f0 2024-02-15T18:18:08.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.972+0000 7f513e7fc700 1 -- 172.21.15.42:0/123359539 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f5144014580 con 0x7f51480f68f0 2024-02-15T18:18:08.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.972+0000 7f513e7fc700 1 --2- 172.21.15.42:0/123359539 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f513402ff60 0x7f5134032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:08.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.972+0000 7f513e7fc700 1 -- 172.21.15.42:0/123359539 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f51440095f0 con 0x7f51480f68f0 2024-02-15T18:18:08.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.972+0000 7f514d79a700 1 --2- 172.21.15.42:0/123359539 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f513402ff60 0x7f5134032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:08.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.972+0000 7f514ef9d700 1 -- 172.21.15.42:0/123359539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5148059100 con 0x7f51480f68f0 2024-02-15T18:18:08.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.972+0000 7f514d79a700 1 --2- 172.21.15.42:0/123359539 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f513402ff60 0x7f5134032420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f51380096a0 tx=0x7f5138006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:08.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:07.976+0000 7f513e7fc700 1 -- 172.21.15.42:0/123359539 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f51440273f0 con 0x7f51480f68f0 2024-02-15T18:18:08.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:08.138+0000 7f514ef9d700 1 -- 172.21.15.42:0/123359539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f5148045ad0 con 0x7f51480f68f0 2024-02-15T18:18:08.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:08.142+0000 7f513e7fc700 1 -- 172.21.15.42:0/123359539 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v11)=0 v11) v1 ==== 125+0+0 (secure 0 0 0) 0x7f514401c890 con 0x7f51480f68f0 2024-02-15T18:18:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:08.144+0000 7f514ef9d700 1 -- 172.21.15.42:0/123359539 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f513402ff60 msgr2=0x7f5134032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:08.144+0000 7f514ef9d700 1 --2- 172.21.15.42:0/123359539 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f513402ff60 0x7f5134032420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f51380096a0 tx=0x7f5138006b40).stop 2024-02-15T18:18:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:08.144+0000 7f514ef9d700 1 -- 172.21.15.42:0/123359539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51480f68f0 msgr2=0x7f5148108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:08.144+0000 7f514ef9d700 1 --2- 172.21.15.42:0/123359539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51480f68f0 0x7f5148108760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f5144009830 tx=0x7f514400b040).stop 2024-02-15T18:18:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:08.144+0000 7f514ef9d700 1 -- 172.21.15.42:0/123359539 shutdown_connections 2024-02-15T18:18:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:08.144+0000 7f514ef9d700 1 --2- 172.21.15.42:0/123359539 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f513402ff60 0x7f5134032420 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:08.144+0000 7f514ef9d700 1 --2- 172.21.15.42:0/123359539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51480f68f0 0x7f5148108760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:08.144+0000 7f514ef9d700 1 -- 172.21.15.42:0/123359539 >> 172.21.15.42:0/123359539 conn(0x7f51480f0120 msgr2=0x7f51480f0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:08.145+0000 7f514ef9d700 1 -- 172.21.15.42:0/123359539 shutdown_connections 2024-02-15T18:18:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:08.145+0000 7f514ef9d700 1 -- 172.21.15.42:0/123359539 wait complete. 2024-02-15T18:18:08.828 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi191 2024-02-15T18:18:08.828 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-15T18:18:08.828 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/etc/ceph/ceph.conf 2024-02-15T18:18:08.850 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-15T18:18:08.850 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-15T18:18:08.911 INFO:tasks.cephadm:Adding host smithi191 to orchestrator... 2024-02-15T18:18:08.911 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph orch host add smithi191 2024-02-15T18:18:09.396 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:08.143517+0000 mon.smithi042 (mon.0) 113 : audit [INF] from='client.? 172.21.15.42:0/123359539' entity='client.admin' 2024-02-15T18:18:11.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:09.633618+0000 mon.smithi042 (mon.0) 114 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:18:11.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:09.909246+0000 mon.smithi042 (mon.0) 115 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:18:11.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:18:09.915489+0000 mgr.smithi042.kfoihb (mgr.14162) 5 : cephadm [INF] Deploying daemon alertmanager.smithi042 on smithi042 2024-02-15T18:18:18.062 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:16.761554+0000 mgr.smithi042.kfoihb (mgr.14162) 6 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:20.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:18.761787+0000 mgr.smithi042.kfoihb (mgr.14162) 7 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:22.401 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:20.762169+0000 mgr.smithi042.kfoihb (mgr.14162) 8 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:24.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:22.762452+0000 mgr.smithi042.kfoihb (mgr.14162) 9 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:24.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:24.170462+0000 mon.smithi042 (mon.0) 116 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:18:24.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:24.171385+0000 mon.smithi042 (mon.0) 117 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi042", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:18:24.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:24.172879+0000 mon.smithi042 (mon.0) 118 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi042", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-15T18:18:24.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:24.173474+0000 mon.smithi042 (mon.0) 119 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:18:25.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:18:24.174062+0000 mgr.smithi042.kfoihb (mgr.14162) 10 : cephadm [INF] Deploying daemon crash.smithi042 on smithi042 2024-02-15T18:18:26.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:24.762731+0000 mgr.smithi042.kfoihb (mgr.14162) 11 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:27.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:26.507921+0000 mon.smithi042 (mon.0) 120 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:18:27.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:26.824872+0000 mon.smithi042 (mon.0) 121 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:18:27.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:26.826618+0000 mon.smithi042 (mon.0) 122 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:18:27.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:26.827009+0000 mon.smithi042 (mon.0) 123 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-15T18:18:27.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:26.829643+0000 mon.smithi042 (mon.0) 124 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:18:28.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:26.762972+0000 mgr.smithi042.kfoihb (mgr.14162) 12 : cluster [DBG] pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:28.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:26.827426+0000 mgr.smithi042.kfoihb (mgr.14162) 13 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-15T18:18:28.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:18:26.832581+0000 mgr.smithi042.kfoihb (mgr.14162) 14 : cephadm [INF] Deploying daemon grafana.smithi042 on smithi042 2024-02-15T18:18:30.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:28.763279+0000 mgr.smithi042.kfoihb (mgr.14162) 15 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:32.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:30.763653+0000 mgr.smithi042.kfoihb (mgr.14162) 16 : cluster [DBG] pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:34.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:32.763948+0000 mgr.smithi042.kfoihb (mgr.14162) 17 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:36.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:34.764173+0000 mgr.smithi042.kfoihb (mgr.14162) 18 : cluster [DBG] pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:37.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.701+0000 7f91fdecc700 1 -- 172.21.15.191:0/86037689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91f80f4800 msgr2=0x7f91f80f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:37.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.701+0000 7f91fdecc700 1 --2- 172.21.15.191:0/86037689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91f80f4800 0x7f91f80f4c20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f91e0009fb0 tx=0x7f91e0009b30).stop 2024-02-15T18:18:37.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.701+0000 7f91fdecc700 1 -- 172.21.15.191:0/86037689 shutdown_connections 2024-02-15T18:18:37.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.701+0000 7f91fdecc700 1 --2- 172.21.15.191:0/86037689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91f80f4800 0x7f91f80f4c20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:37.703 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.701+0000 7f91fdecc700 1 -- 172.21.15.191:0/86037689 >> 172.21.15.191:0/86037689 conn(0x7f91f80f0260 msgr2=0x7f91f80f2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:37.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.701+0000 7f91fdecc700 1 -- 172.21.15.191:0/86037689 shutdown_connections 2024-02-15T18:18:37.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.701+0000 7f91fdecc700 1 -- 172.21.15.191:0/86037689 wait complete. 2024-02-15T18:18:37.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.701+0000 7f91fdecc700 1 Processor -- start 2024-02-15T18:18:37.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.702+0000 7f91fdecc700 1 -- start start 2024-02-15T18:18:37.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.702+0000 7f91fdecc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91f80f4800 0x7f91f80f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:37.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.702+0000 7f91fdecc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f80f5870 con 0x7f91f80f4800 2024-02-15T18:18:37.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.702+0000 7f91f77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91f80f4800 0x7f91f80f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:37.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.702+0000 7f91f77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91f80f4800 0x7f91f80f73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.191:58206/0 (socket says 172.21.15.191:58206) 2024-02-15T18:18:37.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.702+0000 7f91f77fe700 1 -- 172.21.15.191:0/443563659 learned_addr learned my addr 172.21.15.191:0/443563659 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:18:37.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.702+0000 7f91f77fe700 1 -- 172.21.15.191:0/443563659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e0005040 con 0x7f91f80f4800 2024-02-15T18:18:37.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.702+0000 7f91f77fe700 1 --2- 172.21.15.191:0/443563659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91f80f4800 0x7f91f80f73f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f91e00089f0 tx=0x7f91e000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:37.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.703+0000 7f91effff700 1 -- 172.21.15.191:0/443563659 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91e0009320 con 0x7f91f80f4800 2024-02-15T18:18:37.704 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.703+0000 7f91effff700 1 -- 172.21.15.191:0/443563659 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91e0013380 con 0x7f91f80f4800 2024-02-15T18:18:37.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.703+0000 7f91fdecc700 1 -- 172.21.15.191:0/443563659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f80f5a10 con 0x7f91f80f4800 2024-02-15T18:18:37.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.703+0000 7f91effff700 1 -- 172.21.15.191:0/443563659 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91e0014e20 con 0x7f91f80f4800 2024-02-15T18:18:37.705 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.703+0000 7f91fdecc700 1 -- 172.21.15.191:0/443563659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f80f5ea0 con 0x7f91f80f4800 2024-02-15T18:18:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.703+0000 7f91effff700 1 -- 172.21.15.191:0/443563659 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f91e00263a0 con 0x7f91f80f4800 2024-02-15T18:18:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.704+0000 7f91effff700 1 --2- 172.21.15.191:0/443563659 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f91e402ff10 0x7f91e40323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.704+0000 7f91fdecc700 1 -- 172.21.15.191:0/443563659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f8059100 con 0x7f91f80f4800 2024-02-15T18:18:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.704+0000 7f91effff700 1 -- 172.21.15.191:0/443563659 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f91e00411b0 con 0x7f91f80f4800 2024-02-15T18:18:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.704+0000 7f91f6ffd700 1 --2- 172.21.15.191:0/443563659 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f91e402ff10 0x7f91e40323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:37.706 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.704+0000 7f91f6ffd700 1 --2- 172.21.15.191:0/443563659 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f91e402ff10 0x7f91e40323d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f91e80096a0 tx=0x7f91e8006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:37.709 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.709+0000 7f91effff700 1 -- 172.21.15.191:0/443563659 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f91e0012920 con 0x7f91f80f4800 2024-02-15T18:18:37.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:37.878+0000 7f91fdecc700 1 -- 172.21.15.191:0/443563659 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}) v1 -- 0x7f91f80fc200 con 0x7f91e402ff10 2024-02-15T18:18:38.496 INFO:teuthology.orchestra.run.smithi191.stdout:Added host 'smithi191' 2024-02-15T18:18:38.496 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:38.494+0000 7f91effff700 1 -- 172.21.15.191:0/443563659 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+23 (secure 0 0 0) 0x7f91f80fc200 con 0x7f91e402ff10 2024-02-15T18:18:38.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:38.497+0000 7f91fdecc700 1 -- 172.21.15.191:0/443563659 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f91e402ff10 msgr2=0x7f91e40323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:38.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:38.497+0000 7f91fdecc700 1 --2- 172.21.15.191:0/443563659 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f91e402ff10 0x7f91e40323d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f91e80096a0 tx=0x7f91e8006b40).stop 2024-02-15T18:18:38.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:38.497+0000 7f91fdecc700 1 -- 172.21.15.191:0/443563659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91f80f4800 msgr2=0x7f91f80f73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:38.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:38.497+0000 7f91fdecc700 1 --2- 172.21.15.191:0/443563659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91f80f4800 0x7f91f80f73f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f91e00089f0 tx=0x7f91e000b040).stop 2024-02-15T18:18:38.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:38.497+0000 7f91fdecc700 1 -- 172.21.15.191:0/443563659 shutdown_connections 2024-02-15T18:18:38.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:38.497+0000 7f91fdecc700 1 --2- 172.21.15.191:0/443563659 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f91e402ff10 0x7f91e40323d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:38.497 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:38.497+0000 7f91fdecc700 1 --2- 172.21.15.191:0/443563659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91f80f4800 0x7f91f80f73f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:38.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:38.497+0000 7f91fdecc700 1 -- 172.21.15.191:0/443563659 >> 172.21.15.191:0/443563659 conn(0x7f91f80f0260 msgr2=0x7f91f80f0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:38.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:38.497+0000 7f91fdecc700 1 -- 172.21.15.191:0/443563659 shutdown_connections 2024-02-15T18:18:38.498 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:38.497+0000 7f91fdecc700 1 -- 172.21.15.191:0/443563659 wait complete. 2024-02-15T18:18:38.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:36.764363+0000 mgr.smithi042.kfoihb (mgr.14162) 19 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:38.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:38.494421+0000 mon.smithi042 (mon.0) 125 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:18:39.079 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph orch host ls --format=json 2024-02-15T18:18:39.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:37.880087+0000 mgr.smithi042.kfoihb (mgr.14162) 20 : audit [DBG] from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:18:39.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:18:38.494771+0000 mgr.smithi042.kfoihb (mgr.14162) 21 : cephadm [INF] Added host smithi191 2024-02-15T18:18:40.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:38.764612+0000 mgr.smithi042.kfoihb (mgr.14162) 22 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:40.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.919+0000 7f2d58556700 1 -- 172.21.15.191:0/2107854629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d500f6510 msgr2=0x7f2d500f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:40.921 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.919+0000 7f2d58556700 1 --2- 172.21.15.191:0/2107854629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d500f6510 0x7f2d500f6930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f2d4c009fb0 tx=0x7f2d4c009b30).stop 2024-02-15T18:18:40.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.919+0000 7f2d58556700 1 -- 172.21.15.191:0/2107854629 shutdown_connections 2024-02-15T18:18:40.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.919+0000 7f2d58556700 1 --2- 172.21.15.191:0/2107854629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d500f6510 0x7f2d500f6930 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:40.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.919+0000 7f2d58556700 1 -- 172.21.15.191:0/2107854629 >> 172.21.15.191:0/2107854629 conn(0x7f2d500f1c70 msgr2=0x7f2d500f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:40.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.919+0000 7f2d58556700 1 -- 172.21.15.191:0/2107854629 shutdown_connections 2024-02-15T18:18:40.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.919+0000 7f2d58556700 1 -- 172.21.15.191:0/2107854629 wait complete. 2024-02-15T18:18:40.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.919+0000 7f2d58556700 1 Processor -- start 2024-02-15T18:18:40.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.919+0000 7f2d58556700 1 -- start start 2024-02-15T18:18:40.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.919+0000 7f2d58556700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d500f6510 0x7f2d5006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:40.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.919+0000 7f2d58556700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d50069e20 con 0x7f2d500f6510 2024-02-15T18:18:40.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.920+0000 7f2d562f2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d500f6510 0x7f2d5006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:40.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.920+0000 7f2d562f2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d500f6510 0x7f2d5006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.191:58230/0 (socket says 172.21.15.191:58230) 2024-02-15T18:18:40.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.920+0000 7f2d562f2700 1 -- 172.21.15.191:0/3333335504 learned_addr learned my addr 172.21.15.191:0/3333335504 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:18:40.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.920+0000 7f2d562f2700 1 -- 172.21.15.191:0/3333335504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c005040 con 0x7f2d500f6510 2024-02-15T18:18:40.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.920+0000 7f2d562f2700 1 --2- 172.21.15.191:0/3333335504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d500f6510 0x7f2d5006b9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f2d4c005010 tx=0x7f2d4c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:40.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.920+0000 7f2d46ffd700 1 -- 172.21.15.191:0/3333335504 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d4c009320 con 0x7f2d500f6510 2024-02-15T18:18:40.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.920+0000 7f2d46ffd700 1 -- 172.21.15.191:0/3333335504 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d4c013380 con 0x7f2d500f6510 2024-02-15T18:18:40.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.920+0000 7f2d58556700 1 -- 172.21.15.191:0/3333335504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d50069fc0 con 0x7f2d500f6510 2024-02-15T18:18:40.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.920+0000 7f2d46ffd700 1 -- 172.21.15.191:0/3333335504 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d4c014e50 con 0x7f2d500f6510 2024-02-15T18:18:40.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.920+0000 7f2d58556700 1 -- 172.21.15.191:0/3333335504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5006a450 con 0x7f2d500f6510 2024-02-15T18:18:40.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.921+0000 7f2d46ffd700 1 -- 172.21.15.191:0/3333335504 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f2d4c0263a0 con 0x7f2d500f6510 2024-02-15T18:18:40.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.921+0000 7f2d58556700 1 -- 172.21.15.191:0/3333335504 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d50059100 con 0x7f2d500f6510 2024-02-15T18:18:40.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.921+0000 7f2d46ffd700 1 --2- 172.21.15.191:0/3333335504 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2d3c02ff10 0x7f2d3c0323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:40.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.921+0000 7f2d46ffd700 1 -- 172.21.15.191:0/3333335504 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f2d4c041140 con 0x7f2d500f6510 2024-02-15T18:18:40.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.921+0000 7f2d55af1700 1 --2- 172.21.15.191:0/3333335504 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2d3c02ff10 0x7f2d3c0323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:40.924 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.922+0000 7f2d55af1700 1 --2- 172.21.15.191:0/3333335504 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2d3c02ff10 0x7f2d3c0323d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f2d400096a0 tx=0x7f2d40006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:40.927 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:40.926+0000 7f2d46ffd700 1 -- 172.21.15.191:0/3333335504 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f2d4c0126f0 con 0x7f2d500f6510 2024-02-15T18:18:41.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:41.096+0000 7f2d58556700 1 -- 172.21.15.191:0/3333335504 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d5006aca0 con 0x7f2d3c02ff10 2024-02-15T18:18:41.099 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:41.098+0000 7f2d46ffd700 1 -- 172.21.15.191:0/3333335504 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+156 (secure 0 0 0) 0x7f2d5006aca0 con 0x7f2d3c02ff10 2024-02-15T18:18:41.099 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:18:41.099 INFO:teuthology.orchestra.run.smithi191.stdout:[{"addr": "172.21.15.42", "hostname": "smithi042", "labels": [], "status": ""}, {"addr": "smithi191", "hostname": "smithi191", "labels": [], "status": ""}] 2024-02-15T18:18:41.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:41.099+0000 7f2d58556700 1 -- 172.21.15.191:0/3333335504 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2d3c02ff10 msgr2=0x7f2d3c0323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:41.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:41.099+0000 7f2d58556700 1 --2- 172.21.15.191:0/3333335504 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2d3c02ff10 0x7f2d3c0323d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f2d400096a0 tx=0x7f2d40006b40).stop 2024-02-15T18:18:41.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:41.099+0000 7f2d58556700 1 -- 172.21.15.191:0/3333335504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d500f6510 msgr2=0x7f2d5006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:41.100 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:41.099+0000 7f2d58556700 1 --2- 172.21.15.191:0/3333335504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d500f6510 0x7f2d5006b9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f2d4c005010 tx=0x7f2d4c00b040).stop 2024-02-15T18:18:41.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:41.099+0000 7f2d58556700 1 -- 172.21.15.191:0/3333335504 shutdown_connections 2024-02-15T18:18:41.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:41.099+0000 7f2d58556700 1 --2- 172.21.15.191:0/3333335504 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2d3c02ff10 0x7f2d3c0323d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:41.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:41.099+0000 7f2d58556700 1 --2- 172.21.15.191:0/3333335504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d500f6510 0x7f2d5006b9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:41.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:41.099+0000 7f2d58556700 1 -- 172.21.15.191:0/3333335504 >> 172.21.15.191:0/3333335504 conn(0x7f2d500f1c70 msgr2=0x7f2d500f26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:41.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:41.099+0000 7f2d58556700 1 -- 172.21.15.191:0/3333335504 shutdown_connections 2024-02-15T18:18:41.101 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:41.099+0000 7f2d58556700 1 -- 172.21.15.191:0/3333335504 wait complete. 2024-02-15T18:18:41.734 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-15T18:18:41.734 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph osd crush tunables default 2024-02-15T18:18:42.794 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:40.764849+0000 mgr.smithi042.kfoihb (mgr.14162) 23 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:42.795 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:41.098920+0000 mgr.smithi042.kfoihb (mgr.14162) 24 : audit [DBG] from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-15T18:18:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:42.765070+0000 mgr.smithi042.kfoihb (mgr.14162) 25 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.529+0000 7f81b7e4b700 1 -- 172.21.15.42:0/880291072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81b00f5410 msgr2=0x7f81b00f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.529+0000 7f81b7e4b700 1 --2- 172.21.15.42:0/880291072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81b00f5410 0x7f81b00f5830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f81ac009fb0 tx=0x7f81ac009b30).stop 2024-02-15T18:18:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.529+0000 7f81b7e4b700 1 -- 172.21.15.42:0/880291072 shutdown_connections 2024-02-15T18:18:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.529+0000 7f81b7e4b700 1 --2- 172.21.15.42:0/880291072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81b00f5410 0x7f81b00f5830 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.529+0000 7f81b7e4b700 1 -- 172.21.15.42:0/880291072 >> 172.21.15.42:0/880291072 conn(0x7f81b00f0b70 msgr2=0x7f81b00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.529+0000 7f81b7e4b700 1 -- 172.21.15.42:0/880291072 shutdown_connections 2024-02-15T18:18:45.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.529+0000 7f81b7e4b700 1 -- 172.21.15.42:0/880291072 wait complete. 2024-02-15T18:18:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.530+0000 7f81b7e4b700 1 Processor -- start 2024-02-15T18:18:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.530+0000 7f81b7e4b700 1 -- start start 2024-02-15T18:18:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.531+0000 7f81b7e4b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81b00f5410 0x7f81b010ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.531+0000 7f81b7e4b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b010d340 con 0x7f81b00f5410 2024-02-15T18:18:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.531+0000 7f81b5be7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81b00f5410 0x7f81b010ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.531+0000 7f81b5be7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81b00f5410 0x7f81b010ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58338/0 (socket says 172.21.15.42:58338) 2024-02-15T18:18:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.531+0000 7f81b5be7700 1 -- 172.21.15.42:0/1181844566 learned_addr learned my addr 172.21.15.42:0/1181844566 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:18:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.531+0000 7f81b5be7700 1 -- 172.21.15.42:0/1181844566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac005040 con 0x7f81b00f5410 2024-02-15T18:18:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.531+0000 7f81b5be7700 1 --2- 172.21.15.42:0/1181844566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81b00f5410 0x7f81b010ce50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f81ac005010 tx=0x7f81ac00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:45.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.531+0000 7f81a67fc700 1 -- 172.21.15.42:0/1181844566 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81ac009320 con 0x7f81b00f5410 2024-02-15T18:18:45.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.531+0000 7f81a67fc700 1 -- 172.21.15.42:0/1181844566 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81ac013380 con 0x7f81b00f5410 2024-02-15T18:18:45.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.531+0000 7f81b7e4b700 1 -- 172.21.15.42:0/1181844566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b010d540 con 0x7f81b00f5410 2024-02-15T18:18:45.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.531+0000 7f81a67fc700 1 -- 172.21.15.42:0/1181844566 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81ac014e50 con 0x7f81b00f5410 2024-02-15T18:18:45.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.531+0000 7f81b7e4b700 1 -- 172.21.15.42:0/1181844566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b010e150 con 0x7f81b00f5410 2024-02-15T18:18:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.532+0000 7f81a67fc700 1 -- 172.21.15.42:0/1181844566 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f81ac014420 con 0x7f81b00f5410 2024-02-15T18:18:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.532+0000 7f81a67fc700 1 --2- 172.21.15.42:0/1181844566 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f819c02ff10 0x7f819c0323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.532+0000 7f81b7e4b700 1 -- 172.21.15.42:0/1181844566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b010d9d0 con 0x7f81b00f5410 2024-02-15T18:18:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.532+0000 7f81a67fc700 1 -- 172.21.15.42:0/1181844566 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f81ac040e30 con 0x7f81b00f5410 2024-02-15T18:18:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.532+0000 7f81b53e6700 1 --2- 172.21.15.42:0/1181844566 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f819c02ff10 0x7f819c0323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.533+0000 7f81b53e6700 1 --2- 172.21.15.42:0/1181844566 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f819c02ff10 0x7f819c0323d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f81a00096a0 tx=0x7f81a0006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.537+0000 7f81a67fc700 1 -- 172.21.15.42:0/1181844566 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f81b010d9d0 con 0x7f81b00f5410 2024-02-15T18:18:45.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:45.702+0000 7f81b7e4b700 1 -- 172.21.15.42:0/1181844566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f81b010d9d0 con 0x7f81b00f5410 2024-02-15T18:18:46.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:46.523+0000 7f81a67fc700 1 -- 172.21.15.42:0/1181844566 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v5) v1 ==== 124+0+0 (secure 0 0 0) 0x7f81b010d9d0 con 0x7f81b00f5410 2024-02-15T18:18:46.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:46.525+0000 7f81b7e4b700 1 -- 172.21.15.42:0/1181844566 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f819c02ff10 msgr2=0x7f819c0323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:46.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:46.525+0000 7f81b7e4b700 1 --2- 172.21.15.42:0/1181844566 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f819c02ff10 0x7f819c0323d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f81a00096a0 tx=0x7f81a0006b40).stop 2024-02-15T18:18:46.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:46.526+0000 7f81b7e4b700 1 -- 172.21.15.42:0/1181844566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81b00f5410 msgr2=0x7f81b010ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:46.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:46.526+0000 7f81b7e4b700 1 --2- 172.21.15.42:0/1181844566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81b00f5410 0x7f81b010ce50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f81ac005010 tx=0x7f81ac00b040).stop 2024-02-15T18:18:46.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:46.526+0000 7f81b7e4b700 1 -- 172.21.15.42:0/1181844566 shutdown_connections 2024-02-15T18:18:46.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:46.526+0000 7f81b7e4b700 1 --2- 172.21.15.42:0/1181844566 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f819c02ff10 0x7f819c0323d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:46.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:46.526+0000 7f81b7e4b700 1 --2- 172.21.15.42:0/1181844566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81b00f5410 0x7f81b010ce50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:46.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:46.526+0000 7f81b7e4b700 1 -- 172.21.15.42:0/1181844566 >> 172.21.15.42:0/1181844566 conn(0x7f81b00f0b70 msgr2=0x7f81b00f1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:46.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:46.526+0000 7f81b7e4b700 1 -- 172.21.15.42:0/1181844566 shutdown_connections 2024-02-15T18:18:46.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:18:46.526+0000 7f81b7e4b700 1 -- 172.21.15.42:0/1181844566 wait complete. 2024-02-15T18:18:46.527 INFO:teuthology.orchestra.run.smithi042.stderr:adjusted tunables profile to default 2024-02-15T18:18:46.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:44.765433+0000 mgr.smithi042.kfoihb (mgr.14162) 26 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:46.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:45.704019+0000 mon.smithi042 (mon.0) 126 : audit [INF] from='client.? 172.21.15.42:0/1181844566' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-15T18:18:47.503 INFO:tasks.cephadm:Adding mon.smithi042 on smithi042 2024-02-15T18:18:47.503 INFO:tasks.cephadm:Adding mon.smithi191 on smithi191 2024-02-15T18:18:47.504 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph orch apply mon '2;smithi042:172.21.15.42=smithi042;smithi191:172.21.15.191=smithi191' 2024-02-15T18:18:47.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:46.524248+0000 mon.smithi042 (mon.0) 127 : audit [INF] from='client.? 172.21.15.42:0/1181844566' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-15T18:18:47.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:46.524300+0000 mon.smithi042 (mon.0) 128 : cluster [DBG] osdmap e5: 0 total, 0 up, 0 in 2024-02-15T18:18:48.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:46.765781+0000 mgr.smithi042.kfoihb (mgr.14162) 27 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:48.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:47.677651+0000 mon.smithi042 (mon.0) 129 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:18:49.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.241+0000 7f69563c6700 1 -- 172.21.15.191:0/525882597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500f47c0 msgr2=0x7f69500f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:49.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.241+0000 7f69563c6700 1 --2- 172.21.15.191:0/525882597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500f47c0 0x7f69500f4be0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f6940009fb0 tx=0x7f6940009b30).stop 2024-02-15T18:18:49.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.241+0000 7f69563c6700 1 -- 172.21.15.191:0/525882597 shutdown_connections 2024-02-15T18:18:49.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.241+0000 7f69563c6700 1 --2- 172.21.15.191:0/525882597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500f47c0 0x7f69500f4be0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:49.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.241+0000 7f69563c6700 1 -- 172.21.15.191:0/525882597 >> 172.21.15.191:0/525882597 conn(0x7f69500f02e0 msgr2=0x7f69500f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:49.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.241+0000 7f69563c6700 1 -- 172.21.15.191:0/525882597 shutdown_connections 2024-02-15T18:18:49.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.241+0000 7f69563c6700 1 -- 172.21.15.191:0/525882597 wait complete. 2024-02-15T18:18:49.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.241+0000 7f69563c6700 1 Processor -- start 2024-02-15T18:18:49.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.242+0000 7f69563c6700 1 -- start start 2024-02-15T18:18:49.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.242+0000 7f69563c6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500f47c0 0x7f69500f7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:49.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.242+0000 7f69563c6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69500f57f0 con 0x7f69500f47c0 2024-02-15T18:18:49.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.242+0000 7f694ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500f47c0 0x7f69500f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:49.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.242+0000 7f694ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500f47c0 0x7f69500f7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.191:43564/0 (socket says 172.21.15.191:43564) 2024-02-15T18:18:49.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.242+0000 7f694ffff700 1 -- 172.21.15.191:0/2834842968 learned_addr learned my addr 172.21.15.191:0/2834842968 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:18:49.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.242+0000 7f694ffff700 1 -- 172.21.15.191:0/2834842968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6940005040 con 0x7f69500f47c0 2024-02-15T18:18:49.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.242+0000 7f694ffff700 1 --2- 172.21.15.191:0/2834842968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500f47c0 0x7f69500f7370 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f69400089f0 tx=0x7f694000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:49.244 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.242+0000 7f694cff9700 1 -- 172.21.15.191:0/2834842968 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6940009320 con 0x7f69500f47c0 2024-02-15T18:18:49.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.242+0000 7f69563c6700 1 -- 172.21.15.191:0/2834842968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69500f5990 con 0x7f69500f47c0 2024-02-15T18:18:49.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.242+0000 7f694cff9700 1 -- 172.21.15.191:0/2834842968 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6940013380 con 0x7f69500f47c0 2024-02-15T18:18:49.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.242+0000 7f694cff9700 1 -- 172.21.15.191:0/2834842968 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f694001ed80 con 0x7f69500f47c0 2024-02-15T18:18:49.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.243+0000 7f69563c6700 1 -- 172.21.15.191:0/2834842968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69500f5e20 con 0x7f69500f47c0 2024-02-15T18:18:49.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.243+0000 7f694cff9700 1 -- 172.21.15.191:0/2834842968 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f6940014420 con 0x7f69500f47c0 2024-02-15T18:18:49.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.243+0000 7f694cff9700 1 --2- 172.21.15.191:0/2834842968 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f693802ff60 0x7f6938032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:49.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.243+0000 7f694cff9700 1 -- 172.21.15.191:0/2834842968 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f69400411c0 con 0x7f69500f47c0 2024-02-15T18:18:49.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.244+0000 7f69563c6700 1 -- 172.21.15.191:0/2834842968 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6950059100 con 0x7f69500f47c0 2024-02-15T18:18:49.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.244+0000 7f694f7fe700 1 --2- 172.21.15.191:0/2834842968 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f693802ff60 0x7f6938032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:49.245 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.244+0000 7f694f7fe700 1 --2- 172.21.15.191:0/2834842968 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f693802ff60 0x7f6938032420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f69440096a0 tx=0x7f6944006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:49.248 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.248+0000 7f694cff9700 1 -- 172.21.15.191:0/2834842968 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f694001c910 con 0x7f69500f47c0 2024-02-15T18:18:49.421 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.421+0000 7f69563c6700 1 -- 172.21.15.191:0/2834842968 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi042:172.21.15.42=smithi042;smithi191:172.21.15.191=smithi191", "target": ["mon-mgr", ""]}) v1 -- 0x7f69500f6640 con 0x7f693802ff60 2024-02-15T18:18:49.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.427+0000 7f694cff9700 1 -- 172.21.15.191:0/2834842968 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f69500f6640 con 0x7f693802ff60 2024-02-15T18:18:49.427 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mon update... 2024-02-15T18:18:49.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.428+0000 7f69563c6700 1 -- 172.21.15.191:0/2834842968 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f693802ff60 msgr2=0x7f6938032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:49.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.428+0000 7f69563c6700 1 --2- 172.21.15.191:0/2834842968 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f693802ff60 0x7f6938032420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f69440096a0 tx=0x7f6944006b40).stop 2024-02-15T18:18:49.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.428+0000 7f69563c6700 1 -- 172.21.15.191:0/2834842968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500f47c0 msgr2=0x7f69500f7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:49.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.428+0000 7f69563c6700 1 --2- 172.21.15.191:0/2834842968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500f47c0 0x7f69500f7370 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f69400089f0 tx=0x7f694000b040).stop 2024-02-15T18:18:49.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.428+0000 7f69563c6700 1 -- 172.21.15.191:0/2834842968 shutdown_connections 2024-02-15T18:18:49.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.428+0000 7f69563c6700 1 --2- 172.21.15.191:0/2834842968 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f693802ff60 0x7f6938032420 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:49.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.428+0000 7f69563c6700 1 --2- 172.21.15.191:0/2834842968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69500f47c0 0x7f69500f7370 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:49.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.428+0000 7f69563c6700 1 -- 172.21.15.191:0/2834842968 >> 172.21.15.191:0/2834842968 conn(0x7f69500f02e0 msgr2=0x7f69500f0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:49.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.428+0000 7f69563c6700 1 -- 172.21.15.191:0/2834842968 shutdown_connections 2024-02-15T18:18:49.429 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:49.428+0000 7f69563c6700 1 -- 172.21.15.191:0/2834842968 wait complete. 2024-02-15T18:18:49.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:18:47.678946+0000 mgr.smithi042.kfoihb (mgr.14162) 28 : cephadm [INF] Deploying daemon node-exporter.smithi042 on smithi042 2024-02-15T18:18:49.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:49.427227+0000 mon.smithi042 (mon.0) 130 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:18:50.107 DEBUG:teuthology.orchestra.run.smithi191:mon.smithi191> sudo journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi191.service 2024-02-15T18:18:50.110 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:18:50.110 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:18:50.139 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:-- Logs begin at Thu 2024-02-15 17:54:16 UTC. -- 2024-02-15T18:18:50.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:48.766074+0000 mgr.smithi042.kfoihb (mgr.14162) 29 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:50.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:49.423110+0000 mgr.smithi042.kfoihb (mgr.14162) 30 : audit [DBG] from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi042:172.21.15.42=smithi042;smithi191:172.21.15.191=smithi191", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:18:50.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:18:49.424576+0000 mgr.smithi042.kfoihb (mgr.14162) 31 : cephadm [INF] Saving service mon spec with placement smithi042:172.21.15.42=smithi042;smithi191:172.21.15.191=smithi191;count:2 2024-02-15T18:18:51.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:50.766387+0000 mgr.smithi042.kfoihb (mgr.14162) 32 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:54.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:52.766752+0000 mgr.smithi042.kfoihb (mgr.14162) 33 : cluster [DBG] pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:54.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.331+0000 7f26fe8b8700 1 -- 172.21.15.191:0/2363191293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f80f5410 msgr2=0x7f26f80f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:54.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.331+0000 7f26fe8b8700 1 --2- 172.21.15.191:0/2363191293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f80f5410 0x7f26f80f5830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f26ec009fb0 tx=0x7f26ec009b30).stop 2024-02-15T18:18:54.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.331+0000 7f26fe8b8700 1 -- 172.21.15.191:0/2363191293 shutdown_connections 2024-02-15T18:18:54.331 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.331+0000 7f26fe8b8700 1 --2- 172.21.15.191:0/2363191293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f80f5410 0x7f26f80f5830 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:54.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.331+0000 7f26fe8b8700 1 -- 172.21.15.191:0/2363191293 >> 172.21.15.191:0/2363191293 conn(0x7f26f80f0b70 msgr2=0x7f26f80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:54.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.331+0000 7f26fe8b8700 1 -- 172.21.15.191:0/2363191293 shutdown_connections 2024-02-15T18:18:54.332 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.331+0000 7f26fe8b8700 1 -- 172.21.15.191:0/2363191293 wait complete. 2024-02-15T18:18:54.333 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.332+0000 7f26fe8b8700 1 Processor -- start 2024-02-15T18:18:54.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.332+0000 7f26fe8b8700 1 -- start start 2024-02-15T18:18:54.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.332+0000 7f26fe8b8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f80f5410 0x7f26f81094e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:54.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.332+0000 7f26fe8b8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f8060240 con 0x7f26f80f5410 2024-02-15T18:18:54.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.333+0000 7f26f7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f80f5410 0x7f26f81094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:54.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.333+0000 7f26f7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f80f5410 0x7f26f81094e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.191:43586/0 (socket says 172.21.15.191:43586) 2024-02-15T18:18:54.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.333+0000 7f26f7fff700 1 -- 172.21.15.191:0/3912994902 learned_addr learned my addr 172.21.15.191:0/3912994902 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:18:54.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.333+0000 7f26f7fff700 1 -- 172.21.15.191:0/3912994902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec005040 con 0x7f26f80f5410 2024-02-15T18:18:54.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.333+0000 7f26f7fff700 1 --2- 172.21.15.191:0/3912994902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f80f5410 0x7f26f81094e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f26ec005010 tx=0x7f26ec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:54.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.333+0000 7f26f4ff9700 1 -- 172.21.15.191:0/3912994902 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26ec009320 con 0x7f26f80f5410 2024-02-15T18:18:54.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.333+0000 7f26f4ff9700 1 -- 172.21.15.191:0/3912994902 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26ec013380 con 0x7f26f80f5410 2024-02-15T18:18:54.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.333+0000 7f26f4ff9700 1 -- 172.21.15.191:0/3912994902 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26ec014e50 con 0x7f26f80f5410 2024-02-15T18:18:54.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.333+0000 7f26fe8b8700 1 -- 172.21.15.191:0/3912994902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f8060440 con 0x7f26f80f5410 2024-02-15T18:18:54.334 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.333+0000 7f26fe8b8700 1 -- 172.21.15.191:0/3912994902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f8109b90 con 0x7f26f80f5410 2024-02-15T18:18:54.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.334+0000 7f26f4ff9700 1 -- 172.21.15.191:0/3912994902 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f26ec014420 con 0x7f26f80f5410 2024-02-15T18:18:54.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.334+0000 7f26fe8b8700 1 -- 172.21.15.191:0/3912994902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f8045ad0 con 0x7f26f80f5410 2024-02-15T18:18:54.335 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.334+0000 7f26f4ff9700 1 --2- 172.21.15.191:0/3912994902 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f26e002ff10 0x7f26e00323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:54.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.334+0000 7f26f4ff9700 1 -- 172.21.15.191:0/3912994902 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f26ec040e50 con 0x7f26f80f5410 2024-02-15T18:18:54.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.335+0000 7f26f77fe700 1 --2- 172.21.15.191:0/3912994902 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f26e002ff10 0x7f26e00323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:54.336 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.335+0000 7f26f77fe700 1 --2- 172.21.15.191:0/3912994902 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f26e002ff10 0x7f26e00323d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f26e80096a0 tx=0x7f26e8006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:54.339 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.339+0000 7f26f4ff9700 1 -- 172.21.15.191:0/3912994902 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f26ec01dd50 con 0x7f26f80f5410 2024-02-15T18:18:54.542 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.542+0000 7f26fe8b8700 1 -- 172.21.15.191:0/3912994902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f26f810a3b0 con 0x7f26f80f5410 2024-02-15T18:18:54.543 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.543+0000 7f26f4ff9700 1 -- 172.21.15.191:0/3912994902 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f26ec01db70 con 0x7f26f80f5410 2024-02-15T18:18:54.543 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:18:54.543 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":1,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","modified":"2024-02-15T18:16:50.575606Z","created":"2024-02-15T18:16:50.575606Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:18:54.544 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.544+0000 7f26fe8b8700 1 -- 172.21.15.191:0/3912994902 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f26e002ff10 msgr2=0x7f26e00323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:54.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.544+0000 7f26fe8b8700 1 --2- 172.21.15.191:0/3912994902 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f26e002ff10 0x7f26e00323d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f26e80096a0 tx=0x7f26e8006b40).stop 2024-02-15T18:18:54.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.544+0000 7f26fe8b8700 1 -- 172.21.15.191:0/3912994902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f80f5410 msgr2=0x7f26f81094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:54.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.544+0000 7f26fe8b8700 1 --2- 172.21.15.191:0/3912994902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f80f5410 0x7f26f81094e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f26ec005010 tx=0x7f26ec00b040).stop 2024-02-15T18:18:54.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.544+0000 7f26fe8b8700 1 -- 172.21.15.191:0/3912994902 shutdown_connections 2024-02-15T18:18:54.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.544+0000 7f26fe8b8700 1 --2- 172.21.15.191:0/3912994902 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f26e002ff10 0x7f26e00323d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:54.545 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.544+0000 7f26fe8b8700 1 --2- 172.21.15.191:0/3912994902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f26f80f5410 0x7f26f81094e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:54.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.544+0000 7f26fe8b8700 1 -- 172.21.15.191:0/3912994902 >> 172.21.15.191:0/3912994902 conn(0x7f26f80f0b70 msgr2=0x7f26f80f1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:54.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.544+0000 7f26fe8b8700 1 -- 172.21.15.191:0/3912994902 shutdown_connections 2024-02-15T18:18:54.546 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:54.544+0000 7f26fe8b8700 1 -- 172.21.15.191:0/3912994902 wait complete. 2024-02-15T18:18:54.546 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 1 2024-02-15T18:18:55.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:54.543697+0000 mon.smithi042 (mon.0) 131 : audit [DBG] from='client.? 172.21.15.191:0/3912994902' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:18:56.176 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:18:56.177 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:18:56.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:54.767190+0000 mgr.smithi042.kfoihb (mgr.14162) 34 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:57.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:56.899909+0000 mon.smithi042 (mon.0) 132 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:18:57.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:56.909302+0000 mon.smithi042 (mon.0) 133 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:18:58.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.177+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1668209596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c980f62a0 msgr2=0x7f3c980f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:58.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.177+0000 7f3c9eda2700 1 --2- 172.21.15.191:0/1668209596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c980f62a0 0x7f3c980f66c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f3c88009fb0 tx=0x7f3c88009b30).stop 2024-02-15T18:18:58.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.177+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1668209596 shutdown_connections 2024-02-15T18:18:58.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.177+0000 7f3c9eda2700 1 --2- 172.21.15.191:0/1668209596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c980f62a0 0x7f3c980f66c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:58.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.177+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1668209596 >> 172.21.15.191:0/1668209596 conn(0x7f3c980f1aa0 msgr2=0x7f3c980f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:58.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.177+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1668209596 shutdown_connections 2024-02-15T18:18:58.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.177+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1668209596 wait complete. 2024-02-15T18:18:58.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.178+0000 7f3c9eda2700 1 Processor -- start 2024-02-15T18:18:58.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.178+0000 7f3c9eda2700 1 -- start start 2024-02-15T18:18:58.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.178+0000 7f3c9eda2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c980f62a0 0x7f3c9806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:58.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.178+0000 7f3c9eda2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c98069e20 con 0x7f3c980f62a0 2024-02-15T18:18:58.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.178+0000 7f3c9dda0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c980f62a0 0x7f3c9806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:58.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.178+0000 7f3c9dda0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c980f62a0 0x7f3c9806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.191:55302/0 (socket says 172.21.15.191:55302) 2024-02-15T18:18:58.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.178+0000 7f3c9dda0700 1 -- 172.21.15.191:0/1542694262 learned_addr learned my addr 172.21.15.191:0/1542694262 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:18:58.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.178+0000 7f3c9dda0700 1 -- 172.21.15.191:0/1542694262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c88005040 con 0x7f3c980f62a0 2024-02-15T18:18:58.207 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.178+0000 7f3c9dda0700 1 --2- 172.21.15.191:0/1542694262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c980f62a0 0x7f3c9806b9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f3c880089f0 tx=0x7f3c8800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:58.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.178+0000 7f3c8e7fc700 1 -- 172.21.15.191:0/1542694262 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c88009320 con 0x7f3c980f62a0 2024-02-15T18:18:58.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.178+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1542694262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c98069fc0 con 0x7f3c980f62a0 2024-02-15T18:18:58.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.179+0000 7f3c8e7fc700 1 -- 172.21.15.191:0/1542694262 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c88013380 con 0x7f3c980f62a0 2024-02-15T18:18:58.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.179+0000 7f3c8e7fc700 1 -- 172.21.15.191:0/1542694262 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3c88014e20 con 0x7f3c980f62a0 2024-02-15T18:18:58.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.179+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1542694262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9806a450 con 0x7f3c980f62a0 2024-02-15T18:18:58.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.179+0000 7f3c8e7fc700 1 -- 172.21.15.191:0/1542694262 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f3c88014420 con 0x7f3c980f62a0 2024-02-15T18:18:58.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.179+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1542694262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c98059100 con 0x7f3c980f62a0 2024-02-15T18:18:58.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.180+0000 7f3c8e7fc700 1 --2- 172.21.15.191:0/1542694262 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3c8402ff10 0x7f3c840323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:18:58.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.180+0000 7f3c8e7fc700 1 -- 172.21.15.191:0/1542694262 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f3c88040ef0 con 0x7f3c980f62a0 2024-02-15T18:18:58.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.180+0000 7f3c9d59f700 1 --2- 172.21.15.191:0/1542694262 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3c8402ff10 0x7f3c840323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:18:58.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.180+0000 7f3c9d59f700 1 --2- 172.21.15.191:0/1542694262 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3c8402ff10 0x7f3c840323d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f3c940096a0 tx=0x7f3c94006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:18:58.208 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.184+0000 7f3c8e7fc700 1 -- 172.21.15.191:0/1542694262 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f3c88014790 con 0x7f3c980f62a0 2024-02-15T18:18:58.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.392+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1542694262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3c98045ad0 con 0x7f3c980f62a0 2024-02-15T18:18:58.392 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.392+0000 7f3c8e7fc700 1 -- 172.21.15.191:0/1542694262 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f3c8801d7f0 con 0x7f3c980f62a0 2024-02-15T18:18:58.393 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:18:58.393 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":1,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","modified":"2024-02-15T18:16:50.575606Z","created":"2024-02-15T18:16:50.575606Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:18:58.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.394+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1542694262 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3c8402ff10 msgr2=0x7f3c840323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:58.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.394+0000 7f3c9eda2700 1 --2- 172.21.15.191:0/1542694262 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3c8402ff10 0x7f3c840323d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f3c940096a0 tx=0x7f3c94006b40).stop 2024-02-15T18:18:58.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.394+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1542694262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c980f62a0 msgr2=0x7f3c9806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:18:58.394 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.394+0000 7f3c9eda2700 1 --2- 172.21.15.191:0/1542694262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c980f62a0 0x7f3c9806b9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f3c880089f0 tx=0x7f3c8800b040).stop 2024-02-15T18:18:58.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.394+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1542694262 shutdown_connections 2024-02-15T18:18:58.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.394+0000 7f3c9eda2700 1 --2- 172.21.15.191:0/1542694262 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3c8402ff10 0x7f3c840323d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:58.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.394+0000 7f3c9eda2700 1 --2- 172.21.15.191:0/1542694262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3c980f62a0 0x7f3c9806b9a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:18:58.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.394+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1542694262 >> 172.21.15.191:0/1542694262 conn(0x7f3c980f1aa0 msgr2=0x7f3c980f2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:18:58.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.394+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1542694262 shutdown_connections 2024-02-15T18:18:58.395 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:18:58.394+0000 7f3c9eda2700 1 -- 172.21.15.191:0/1542694262 wait complete. 2024-02-15T18:18:58.395 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 1 2024-02-15T18:18:58.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:56.767417+0000 mgr.smithi042.kfoihb (mgr.14162) 35 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:18:58.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:57.980546+0000 mon.smithi042 (mon.0) 134 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:18:59.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:18:57.986243+0000 mgr.smithi042.kfoihb (mgr.14162) 36 : cephadm [INF] Deploying daemon prometheus.smithi042 on smithi042 2024-02-15T18:18:59.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:18:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:18:58.393427+0000 mon.smithi042 (mon.0) 135 : audit [DBG] from='client.? 172.21.15.191:0/1542694262' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:19:00.133 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:19:00.134 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:19:00.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:18:58.767735+0000 mgr.smithi042.kfoihb (mgr.14162) 37 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:01.761 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.760+0000 7feebf59e700 1 -- 172.21.15.191:0/3453488733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00f51a0 msgr2=0x7feec00f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:01.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.760+0000 7feebf59e700 1 --2- 172.21.15.191:0/3453488733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00f51a0 0x7feec00f55c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7feeb4009fb0 tx=0x7feeb4009b30).stop 2024-02-15T18:19:01.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.760+0000 7feebf59e700 1 -- 172.21.15.191:0/3453488733 shutdown_connections 2024-02-15T18:19:01.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.760+0000 7feebf59e700 1 --2- 172.21.15.191:0/3453488733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00f51a0 0x7feec00f55c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:01.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.760+0000 7feebf59e700 1 -- 172.21.15.191:0/3453488733 >> 172.21.15.191:0/3453488733 conn(0x7feec00f09a0 msgr2=0x7feec00f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:01.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.760+0000 7feebf59e700 1 -- 172.21.15.191:0/3453488733 shutdown_connections 2024-02-15T18:19:01.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.760+0000 7feebf59e700 1 -- 172.21.15.191:0/3453488733 wait complete. 2024-02-15T18:19:01.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.760+0000 7feebf59e700 1 Processor -- start 2024-02-15T18:19:01.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.760+0000 7feebf59e700 1 -- start start 2024-02-15T18:19:01.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.760+0000 7feebf59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00f51a0 0x7feec01065d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:01.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.761+0000 7feebf59e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec0106ac0 con 0x7feec00f51a0 2024-02-15T18:19:01.762 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.761+0000 7feebe59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00f51a0 0x7feec01065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:01.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.761+0000 7feebe59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00f51a0 0x7feec01065d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.191:55322/0 (socket says 172.21.15.191:55322) 2024-02-15T18:19:01.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.761+0000 7feebe59c700 1 -- 172.21.15.191:0/3981483807 learned_addr learned my addr 172.21.15.191:0/3981483807 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:19:01.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.761+0000 7feebe59c700 1 -- 172.21.15.191:0/3981483807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb4005040 con 0x7feec00f51a0 2024-02-15T18:19:01.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.761+0000 7feebe59c700 1 --2- 172.21.15.191:0/3981483807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00f51a0 0x7feec01065d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7feeb40089f0 tx=0x7feeb400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:01.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.761+0000 7feeaeffd700 1 -- 172.21.15.191:0/3981483807 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feeb4009320 con 0x7feec00f51a0 2024-02-15T18:19:01.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.761+0000 7feebf59e700 1 -- 172.21.15.191:0/3981483807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec0106cc0 con 0x7feec00f51a0 2024-02-15T18:19:01.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.761+0000 7feeaeffd700 1 -- 172.21.15.191:0/3981483807 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feeb4013380 con 0x7feec00f51a0 2024-02-15T18:19:01.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.761+0000 7feeaeffd700 1 -- 172.21.15.191:0/3981483807 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feeb4014e20 con 0x7feec00f51a0 2024-02-15T18:19:01.763 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.761+0000 7feebf59e700 1 -- 172.21.15.191:0/3981483807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec0107150 con 0x7feec00f51a0 2024-02-15T18:19:01.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.762+0000 7feebf59e700 1 -- 172.21.15.191:0/3981483807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec0045ad0 con 0x7feec00f51a0 2024-02-15T18:19:01.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.762+0000 7feeaeffd700 1 -- 172.21.15.191:0/3981483807 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7feeb40095f0 con 0x7feec00f51a0 2024-02-15T18:19:01.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.762+0000 7feeaeffd700 1 --2- 172.21.15.191:0/3981483807 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7feea802ff60 0x7feea8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:01.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.762+0000 7feeaeffd700 1 -- 172.21.15.191:0/3981483807 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7feeb4040ec0 con 0x7feec00f51a0 2024-02-15T18:19:01.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.763+0000 7feebdd9b700 1 --2- 172.21.15.191:0/3981483807 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7feea802ff60 0x7feea8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:01.764 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.763+0000 7feebdd9b700 1 --2- 172.21.15.191:0/3981483807 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7feea802ff60 0x7feea8032420 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7feeb00096a0 tx=0x7feeb0006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:01.767 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.767+0000 7feeaeffd700 1 -- 172.21.15.191:0/3981483807 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7feeb4014420 con 0x7feec00f51a0 2024-02-15T18:19:01.972 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.972+0000 7feebf59e700 1 -- 172.21.15.191:0/3981483807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7feec00fad30 con 0x7feec00f51a0 2024-02-15T18:19:01.973 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.973+0000 7feeaeffd700 1 -- 172.21.15.191:0/3981483807 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7feeb401d980 con 0x7feec00f51a0 2024-02-15T18:19:01.973 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:19:01.973 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":1,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","modified":"2024-02-15T18:16:50.575606Z","created":"2024-02-15T18:16:50.575606Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:19:01.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.975+0000 7feebf59e700 1 -- 172.21.15.191:0/3981483807 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7feea802ff60 msgr2=0x7feea8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:01.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.975+0000 7feebf59e700 1 --2- 172.21.15.191:0/3981483807 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7feea802ff60 0x7feea8032420 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7feeb00096a0 tx=0x7feeb0006b40).stop 2024-02-15T18:19:01.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.975+0000 7feebf59e700 1 -- 172.21.15.191:0/3981483807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00f51a0 msgr2=0x7feec01065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:01.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.975+0000 7feebf59e700 1 --2- 172.21.15.191:0/3981483807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00f51a0 0x7feec01065d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7feeb40089f0 tx=0x7feeb400b040).stop 2024-02-15T18:19:01.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.975+0000 7feebf59e700 1 -- 172.21.15.191:0/3981483807 shutdown_connections 2024-02-15T18:19:01.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.975+0000 7feebf59e700 1 --2- 172.21.15.191:0/3981483807 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7feea802ff60 0x7feea8032420 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:01.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.975+0000 7feebf59e700 1 --2- 172.21.15.191:0/3981483807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feec00f51a0 0x7feec01065d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:01.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.975+0000 7feebf59e700 1 -- 172.21.15.191:0/3981483807 >> 172.21.15.191:0/3981483807 conn(0x7feec00f09a0 msgr2=0x7feec005fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:01.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.975+0000 7feebf59e700 1 -- 172.21.15.191:0/3981483807 shutdown_connections 2024-02-15T18:19:01.975 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:01.975+0000 7feebf59e700 1 -- 172.21.15.191:0/3981483807 wait complete. 2024-02-15T18:19:01.976 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 1 2024-02-15T18:19:02.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:00.768114+0000 mgr.smithi042.kfoihb (mgr.14162) 38 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:02.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:01.974106+0000 mon.smithi042 (mon.0) 136 : audit [DBG] from='client.? 172.21.15.191:0/3981483807' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:19:03.609 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:19:03.610 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:19:04.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:02.768539+0000 mgr.smithi042.kfoihb (mgr.14162) 39 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:06.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:04.768944+0000 mgr.smithi042.kfoihb (mgr.14162) 40 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:08.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:06.769233+0000 mgr.smithi042.kfoihb (mgr.14162) 41 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:10.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:08.769645+0000 mgr.smithi042.kfoihb (mgr.14162) 42 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:12.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:10.770123+0000 mgr.smithi042.kfoihb (mgr.14162) 43 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:13.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.372+0000 7f2488c25700 1 -- 172.21.15.191:0/836726755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2484060280 msgr2=0x7f24840f90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:13.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.372+0000 7f2488c25700 1 --2- 172.21.15.191:0/836726755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2484060280 0x7f24840f90f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f246c009fb0 tx=0x7f246c009b30).stop 2024-02-15T18:19:13.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.372+0000 7f2488c25700 1 -- 172.21.15.191:0/836726755 shutdown_connections 2024-02-15T18:19:13.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.372+0000 7f2488c25700 1 --2- 172.21.15.191:0/836726755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2484060280 0x7f24840f90f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:13.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.372+0000 7f2488c25700 1 -- 172.21.15.191:0/836726755 >> 172.21.15.191:0/836726755 conn(0x7f24840f0b70 msgr2=0x7f24840f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:13.670 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.372+0000 7f2488c25700 1 -- 172.21.15.191:0/836726755 shutdown_connections 2024-02-15T18:19:13.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.372+0000 7f2488c25700 1 -- 172.21.15.191:0/836726755 wait complete. 2024-02-15T18:19:13.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.373+0000 7f2488c25700 1 Processor -- start 2024-02-15T18:19:13.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.373+0000 7f2488c25700 1 -- start start 2024-02-15T18:19:13.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.373+0000 7f2488c25700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2484060280 0x7f2484106780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:13.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.373+0000 7f2488c25700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2484106c70 con 0x7f2484060280 2024-02-15T18:19:13.671 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.373+0000 7f248259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2484060280 0x7f2484106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:13.671 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:19:13.671 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":1,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","modified":"2024-02-15T18:16:50.575606Z","created":"2024-02-15T18:16:50.575606Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:19:13.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.373+0000 7f248259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2484060280 0x7f2484106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.191:33742/0 (socket says 172.21.15.191:33742) 2024-02-15T18:19:13.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.373+0000 7f248259c700 1 -- 172.21.15.191:0/3747200936 learned_addr learned my addr 172.21.15.191:0/3747200936 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:19:13.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.374+0000 7f248259c700 1 -- 172.21.15.191:0/3747200936 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246c005040 con 0x7f2484060280 2024-02-15T18:19:13.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.374+0000 7f248259c700 1 --2- 172.21.15.191:0/3747200936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2484060280 0x7f2484106780 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f246c0089f0 tx=0x7f246c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:13.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.374+0000 7f247affd700 1 -- 172.21.15.191:0/3747200936 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f246c009320 con 0x7f2484060280 2024-02-15T18:19:13.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.374+0000 7f2488c25700 1 -- 172.21.15.191:0/3747200936 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2484106e70 con 0x7f2484060280 2024-02-15T18:19:13.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.374+0000 7f247affd700 1 -- 172.21.15.191:0/3747200936 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f246c013380 con 0x7f2484060280 2024-02-15T18:19:13.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.374+0000 7f247affd700 1 -- 172.21.15.191:0/3747200936 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f246c014e20 con 0x7f2484060280 2024-02-15T18:19:13.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.374+0000 7f2488c25700 1 -- 172.21.15.191:0/3747200936 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2484107300 con 0x7f2484060280 2024-02-15T18:19:13.672 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.375+0000 7f2488c25700 1 -- 172.21.15.191:0/3747200936 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2484045ad0 con 0x7f2484060280 2024-02-15T18:19:13.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.375+0000 7f247affd700 1 -- 172.21.15.191:0/3747200936 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f246c0095f0 con 0x7f2484060280 2024-02-15T18:19:13.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.375+0000 7f247affd700 1 --2- 172.21.15.191:0/3747200936 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f247002ff10 0x7f24700323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:13.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.375+0000 7f247affd700 1 -- 172.21.15.191:0/3747200936 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f246c040e90 con 0x7f2484060280 2024-02-15T18:19:13.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.375+0000 7f2481d9b700 1 --2- 172.21.15.191:0/3747200936 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f247002ff10 0x7f24700323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:13.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.375+0000 7f2481d9b700 1 --2- 172.21.15.191:0/3747200936 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f247002ff10 0x7f24700323d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f24740096a0 tx=0x7f2474006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:13.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.380+0000 7f247affd700 1 -- 172.21.15.191:0/3747200936 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f246c014420 con 0x7f2484060280 2024-02-15T18:19:13.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.584+0000 7f2488c25700 1 -- 172.21.15.191:0/3747200936 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f24841081b0 con 0x7f2484060280 2024-02-15T18:19:13.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.585+0000 7f247affd700 1 -- 172.21.15.191:0/3747200936 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f246c01d980 con 0x7f2484060280 2024-02-15T18:19:13.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.586+0000 7f2488c25700 1 -- 172.21.15.191:0/3747200936 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f247002ff10 msgr2=0x7f24700323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:13.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.586+0000 7f2488c25700 1 --2- 172.21.15.191:0/3747200936 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f247002ff10 0x7f24700323d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f24740096a0 tx=0x7f2474006b40).stop 2024-02-15T18:19:13.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.586+0000 7f2488c25700 1 -- 172.21.15.191:0/3747200936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2484060280 msgr2=0x7f2484106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:13.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.586+0000 7f2488c25700 1 --2- 172.21.15.191:0/3747200936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2484060280 0x7f2484106780 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f246c0089f0 tx=0x7f246c00b040).stop 2024-02-15T18:19:13.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.586+0000 7f2488c25700 1 -- 172.21.15.191:0/3747200936 shutdown_connections 2024-02-15T18:19:13.673 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.586+0000 7f2488c25700 1 --2- 172.21.15.191:0/3747200936 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f247002ff10 0x7f24700323d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:13.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.586+0000 7f2488c25700 1 --2- 172.21.15.191:0/3747200936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2484060280 0x7f2484106780 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:13.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.586+0000 7f2488c25700 1 -- 172.21.15.191:0/3747200936 >> 172.21.15.191:0/3747200936 conn(0x7f24840f0b70 msgr2=0x7f24840f15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:13.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.586+0000 7f2488c25700 1 -- 172.21.15.191:0/3747200936 shutdown_connections 2024-02-15T18:19:13.674 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:13.586+0000 7f2488c25700 1 -- 172.21.15.191:0/3747200936 wait complete. 2024-02-15T18:19:13.674 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 1 2024-02-15T18:19:14.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:12.770387+0000 mgr.smithi042.kfoihb (mgr.14162) 44 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:14.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:13.585724+0000 mon.smithi042 (mon.0) 137 : audit [DBG] from='client.? 172.21.15.191:0/3747200936' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:19:15.199 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:19:15.200 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:19:16.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:14.770751+0000 mgr.smithi042.kfoihb (mgr.14162) 45 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:17.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.028+0000 7f98344fa700 1 -- 172.21.15.191:0/2181465186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f982c0f6510 msgr2=0x7f982c0f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:17.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.028+0000 7f98344fa700 1 --2- 172.21.15.191:0/2181465186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f982c0f6510 0x7f982c0f6930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f981c009fb0 tx=0x7f981c009b30).stop 2024-02-15T18:19:17.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.029+0000 7f98344fa700 1 -- 172.21.15.191:0/2181465186 shutdown_connections 2024-02-15T18:19:17.029 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.029+0000 7f98344fa700 1 --2- 172.21.15.191:0/2181465186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f982c0f6510 0x7f982c0f6930 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:17.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.029+0000 7f98344fa700 1 -- 172.21.15.191:0/2181465186 >> 172.21.15.191:0/2181465186 conn(0x7f982c0f1c70 msgr2=0x7f982c0f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:17.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.029+0000 7f98344fa700 1 -- 172.21.15.191:0/2181465186 shutdown_connections 2024-02-15T18:19:17.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.029+0000 7f98344fa700 1 -- 172.21.15.191:0/2181465186 wait complete. 2024-02-15T18:19:17.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.029+0000 7f98344fa700 1 Processor -- start 2024-02-15T18:19:17.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.029+0000 7f98344fa700 1 -- start start 2024-02-15T18:19:17.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.029+0000 7f98344fa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f982c0f6510 0x7f982c1067e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:17.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.029+0000 7f98344fa700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982c106cd0 con 0x7f982c0f6510 2024-02-15T18:19:17.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.029+0000 7f9832296700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f982c0f6510 0x7f982c1067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:17.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.029+0000 7f9832296700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f982c0f6510 0x7f982c1067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.191:50384/0 (socket says 172.21.15.191:50384) 2024-02-15T18:19:17.030 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.029+0000 7f9832296700 1 -- 172.21.15.191:0/314184548 learned_addr learned my addr 172.21.15.191:0/314184548 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:19:17.031 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.030+0000 7f9832296700 1 -- 172.21.15.191:0/314184548 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c005040 con 0x7f982c0f6510 2024-02-15T18:19:17.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.030+0000 7f9832296700 1 --2- 172.21.15.191:0/314184548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f982c0f6510 0x7f982c1067e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f981c005010 tx=0x7f981c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:17.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.030+0000 7f9822ffd700 1 -- 172.21.15.191:0/314184548 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f981c009320 con 0x7f982c0f6510 2024-02-15T18:19:17.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.030+0000 7f98344fa700 1 -- 172.21.15.191:0/314184548 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f982c106ed0 con 0x7f982c0f6510 2024-02-15T18:19:17.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.030+0000 7f9822ffd700 1 -- 172.21.15.191:0/314184548 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f981c013380 con 0x7f982c0f6510 2024-02-15T18:19:17.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.030+0000 7f9822ffd700 1 -- 172.21.15.191:0/314184548 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f981c014e50 con 0x7f982c0f6510 2024-02-15T18:19:17.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.030+0000 7f98344fa700 1 -- 172.21.15.191:0/314184548 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f982c107360 con 0x7f982c0f6510 2024-02-15T18:19:17.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.031+0000 7f98344fa700 1 -- 172.21.15.191:0/314184548 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f982c045ad0 con 0x7f982c0f6510 2024-02-15T18:19:17.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.031+0000 7f9822ffd700 1 -- 172.21.15.191:0/314184548 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f981c014420 con 0x7f982c0f6510 2024-02-15T18:19:17.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.031+0000 7f9822ffd700 1 --2- 172.21.15.191:0/314184548 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f981802ff10 0x7f98180323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:17.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.031+0000 7f9822ffd700 1 -- 172.21.15.191:0/314184548 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f981c040f30 con 0x7f982c0f6510 2024-02-15T18:19:17.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.031+0000 7f9831a95700 1 --2- 172.21.15.191:0/314184548 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f981802ff10 0x7f98180323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:17.032 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.032+0000 7f9831a95700 1 --2- 172.21.15.191:0/314184548 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f981802ff10 0x7f98180323d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f98280096a0 tx=0x7f9828006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:17.036 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.036+0000 7f9822ffd700 1 -- 172.21.15.191:0/314184548 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f981c0095f0 con 0x7f982c0f6510 2024-02-15T18:19:17.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.239+0000 7f98344fa700 1 -- 172.21.15.191:0/314184548 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f982c1081b0 con 0x7f982c0f6510 2024-02-15T18:19:17.239 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.239+0000 7f9822ffd700 1 -- 172.21.15.191:0/314184548 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f981c01db70 con 0x7f982c0f6510 2024-02-15T18:19:17.240 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:19:17.240 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":1,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","modified":"2024-02-15T18:16:50.575606Z","created":"2024-02-15T18:16:50.575606Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:19:17.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.241+0000 7f98344fa700 1 -- 172.21.15.191:0/314184548 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f981802ff10 msgr2=0x7f98180323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:17.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.241+0000 7f98344fa700 1 --2- 172.21.15.191:0/314184548 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f981802ff10 0x7f98180323d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f98280096a0 tx=0x7f9828006b40).stop 2024-02-15T18:19:17.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.241+0000 7f98344fa700 1 -- 172.21.15.191:0/314184548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f982c0f6510 msgr2=0x7f982c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:17.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.241+0000 7f98344fa700 1 --2- 172.21.15.191:0/314184548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f982c0f6510 0x7f982c1067e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f981c005010 tx=0x7f981c00b040).stop 2024-02-15T18:19:17.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.241+0000 7f98344fa700 1 -- 172.21.15.191:0/314184548 shutdown_connections 2024-02-15T18:19:17.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.241+0000 7f98344fa700 1 --2- 172.21.15.191:0/314184548 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f981802ff10 0x7f98180323d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:17.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.241+0000 7f98344fa700 1 --2- 172.21.15.191:0/314184548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f982c0f6510 0x7f982c1067e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:17.241 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.241+0000 7f98344fa700 1 -- 172.21.15.191:0/314184548 >> 172.21.15.191:0/314184548 conn(0x7f982c0f1c70 msgr2=0x7f982c0f2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:17.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.241+0000 7f98344fa700 1 -- 172.21.15.191:0/314184548 shutdown_connections 2024-02-15T18:19:17.242 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:17.241+0000 7f98344fa700 1 -- 172.21.15.191:0/314184548 wait complete. 2024-02-15T18:19:17.242 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 1 2024-02-15T18:19:18.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:16.771167+0000 mgr.smithi042.kfoihb (mgr.14162) 46 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:18.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:17.240467+0000 mon.smithi042 (mon.0) 138 : audit [DBG] from='client.? 172.21.15.191:0/314184548' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:19:18.854 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:19:18.854 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:19:20.270 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:18.771629+0000 mgr.smithi042.kfoihb (mgr.14162) 47 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:20.270 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:18.921687+0000 mon.smithi042 (mon.0) 139 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:20.270 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:18.922312+0000 mon.smithi042 (mon.0) 140 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:19:20.270 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:19.164327+0000 mon.smithi042 (mon.0) 141 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:20.270 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:19.821695+0000 mon.smithi042 (mon.0) 142 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:20.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.876+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/3300812023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dd00f51a0 msgr2=0x7f0dd00f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:20.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.876+0000 7f0dd4c9d700 1 --2- 172.21.15.191:0/3300812023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dd00f51a0 0x7f0dd00f55c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f0dc0009fb0 tx=0x7f0dc0009b30).stop 2024-02-15T18:19:20.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.876+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/3300812023 shutdown_connections 2024-02-15T18:19:20.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.876+0000 7f0dd4c9d700 1 --2- 172.21.15.191:0/3300812023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dd00f51a0 0x7f0dd00f55c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:20.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.876+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/3300812023 >> 172.21.15.191:0/3300812023 conn(0x7f0dd00f09a0 msgr2=0x7f0dd00f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:20.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.876+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/3300812023 shutdown_connections 2024-02-15T18:19:20.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.876+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/3300812023 wait complete. 2024-02-15T18:19:20.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.877+0000 7f0dd4c9d700 1 Processor -- start 2024-02-15T18:19:20.877 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.877+0000 7f0dd4c9d700 1 -- start start 2024-02-15T18:19:20.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.877+0000 7f0dd4c9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dd00f51a0 0x7f0dd01065d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:20.878 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.877+0000 7f0dd4c9d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd0106ac0 con 0x7f0dd00f51a0 2024-02-15T18:19:20.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.877+0000 7f0dcf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dd00f51a0 0x7f0dd01065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:20.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.877+0000 7f0dcf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dd00f51a0 0x7f0dd01065d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.191:50404/0 (socket says 172.21.15.191:50404) 2024-02-15T18:19:20.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.877+0000 7f0dcf7fe700 1 -- 172.21.15.191:0/1739621958 learned_addr learned my addr 172.21.15.191:0/1739621958 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:19:20.879 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.878+0000 7f0dcf7fe700 1 -- 172.21.15.191:0/1739621958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc0005040 con 0x7f0dd00f51a0 2024-02-15T18:19:20.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.878+0000 7f0dcf7fe700 1 --2- 172.21.15.191:0/1739621958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dd00f51a0 0x7f0dd01065d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f0dc00089f0 tx=0x7f0dc000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:20.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.878+0000 7f0db7fff700 1 -- 172.21.15.191:0/1739621958 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0dc0009320 con 0x7f0dd00f51a0 2024-02-15T18:19:20.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.878+0000 7f0db7fff700 1 -- 172.21.15.191:0/1739621958 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0dc0013380 con 0x7f0dd00f51a0 2024-02-15T18:19:20.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.878+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/1739621958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd0106cc0 con 0x7f0dd00f51a0 2024-02-15T18:19:20.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.878+0000 7f0db7fff700 1 -- 172.21.15.191:0/1739621958 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0dc0014e20 con 0x7f0dd00f51a0 2024-02-15T18:19:20.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.878+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/1739621958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd0107150 con 0x7f0dd00f51a0 2024-02-15T18:19:20.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.879+0000 7f0db7fff700 1 -- 172.21.15.191:0/1739621958 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f0dc00263a0 con 0x7f0dd00f51a0 2024-02-15T18:19:20.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.879+0000 7f0db7fff700 1 --2- 172.21.15.191:0/1739621958 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0db802ff60 0x7f0db8032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:20.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.879+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/1739621958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd0045ad0 con 0x7f0dd00f51a0 2024-02-15T18:19:20.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.879+0000 7f0db7fff700 1 -- 172.21.15.191:0/1739621958 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f0dc0040f70 con 0x7f0dd00f51a0 2024-02-15T18:19:20.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.879+0000 7f0dceffd700 1 --2- 172.21.15.191:0/1739621958 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0db802ff60 0x7f0db8032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:20.880 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.879+0000 7f0dceffd700 1 --2- 172.21.15.191:0/1739621958 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0db802ff60 0x7f0db8032420 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f0dc40096a0 tx=0x7f0dc4006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:20.883 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:20.883+0000 7f0db7fff700 1 -- 172.21.15.191:0/1739621958 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0dc001ddc0 con 0x7f0dd00f51a0 2024-02-15T18:19:21.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:21.094+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/1739621958 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0dd00fad30 con 0x7f0dd00f51a0 2024-02-15T18:19:21.095 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:21.095+0000 7f0db7fff700 1 -- 172.21.15.191:0/1739621958 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f0dc001c320 con 0x7f0dd00f51a0 2024-02-15T18:19:21.095 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:19:21.095 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":1,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","modified":"2024-02-15T18:16:50.575606Z","created":"2024-02-15T18:16:50.575606Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:19:21.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:21.097+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/1739621958 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0db802ff60 msgr2=0x7f0db8032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:21.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:21.097+0000 7f0dd4c9d700 1 --2- 172.21.15.191:0/1739621958 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0db802ff60 0x7f0db8032420 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f0dc40096a0 tx=0x7f0dc4006b40).stop 2024-02-15T18:19:21.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:21.097+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/1739621958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dd00f51a0 msgr2=0x7f0dd01065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:21.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:21.097+0000 7f0dd4c9d700 1 --2- 172.21.15.191:0/1739621958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dd00f51a0 0x7f0dd01065d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f0dc00089f0 tx=0x7f0dc000b040).stop 2024-02-15T18:19:21.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:21.097+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/1739621958 shutdown_connections 2024-02-15T18:19:21.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:21.097+0000 7f0dd4c9d700 1 --2- 172.21.15.191:0/1739621958 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0db802ff60 0x7f0db8032420 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:21.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:21.097+0000 7f0dd4c9d700 1 --2- 172.21.15.191:0/1739621958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dd00f51a0 0x7f0dd01065d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:21.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:21.097+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/1739621958 >> 172.21.15.191:0/1739621958 conn(0x7f0dd00f09a0 msgr2=0x7f0dd005fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:21.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:21.097+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/1739621958 shutdown_connections 2024-02-15T18:19:21.097 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:21.097+0000 7f0dd4c9d700 1 -- 172.21.15.191:0/1739621958 wait complete. 2024-02-15T18:19:21.098 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 1 2024-02-15T18:19:22.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:20.771929+0000 mgr.smithi042.kfoihb (mgr.14162) 48 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:22.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:21.095949+0000 mon.smithi042 (mon.0) 143 : audit [DBG] from='client.? 172.21.15.191:0/1739621958' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:19:23.974 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:19:23.974 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:19:24.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:22.772297+0000 mgr.smithi042.kfoihb (mgr.14162) 49 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:24.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:23.045815+0000 mon.smithi042 (mon.0) 144 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:24.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:23.969379+0000 mon.smithi042 (mon.0) 145 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:25.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:24.495792+0000 mon.smithi042 (mon.0) 146 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:25.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:24.498535+0000 mon.smithi042 (mon.0) 147 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:25.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:24.499817+0000 mon.smithi042 (mon.0) 148 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi191", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:19:25.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:24.501859+0000 mon.smithi042 (mon.0) 149 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi191", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-15T18:19:25.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:24.502692+0000 mon.smithi042 (mon.0) 150 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:19:25.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:19:24.503533+0000 mgr.smithi042.kfoihb (mgr.14162) 50 : cephadm [INF] Deploying daemon crash.smithi191 on smithi191 2024-02-15T18:19:25.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.918+0000 7f833e8ba700 1 -- 172.21.15.191:0/3235218316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f833806b940 msgr2=0x7f8338069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:25.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.918+0000 7f833e8ba700 1 --2- 172.21.15.191:0/3235218316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f833806b940 0x7f8338069e40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f8328009fb0 tx=0x7f8328009b30).stop 2024-02-15T18:19:25.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.918+0000 7f833e8ba700 1 -- 172.21.15.191:0/3235218316 shutdown_connections 2024-02-15T18:19:25.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.918+0000 7f833e8ba700 1 --2- 172.21.15.191:0/3235218316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f833806b940 0x7f8338069e40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:25.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.918+0000 7f833e8ba700 1 -- 172.21.15.191:0/3235218316 >> 172.21.15.191:0/3235218316 conn(0x7f83380f1c70 msgr2=0x7f83380f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:25.919 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.918+0000 7f833e8ba700 1 -- 172.21.15.191:0/3235218316 shutdown_connections 2024-02-15T18:19:25.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.918+0000 7f833e8ba700 1 -- 172.21.15.191:0/3235218316 wait complete. 2024-02-15T18:19:25.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.919+0000 7f833e8ba700 1 Processor -- start 2024-02-15T18:19:25.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.919+0000 7f833e8ba700 1 -- start start 2024-02-15T18:19:25.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.919+0000 7f833e8ba700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f833806b940 0x7f83381089e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:25.920 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.919+0000 7f833e8ba700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8338108ed0 con 0x7f833806b940 2024-02-15T18:19:25.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.919+0000 7f8337fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f833806b940 0x7f83381089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:25.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.919+0000 7f8337fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f833806b940 0x7f83381089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.191:57514/0 (socket says 172.21.15.191:57514) 2024-02-15T18:19:25.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.919+0000 7f8337fff700 1 -- 172.21.15.191:0/4220211151 learned_addr learned my addr 172.21.15.191:0/4220211151 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:19:25.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.920+0000 7f8337fff700 1 -- 172.21.15.191:0/4220211151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8328005040 con 0x7f833806b940 2024-02-15T18:19:25.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.920+0000 7f8337fff700 1 --2- 172.21.15.191:0/4220211151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f833806b940 0x7f83381089e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f83280089f0 tx=0x7f832800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:25.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.920+0000 7f8334ff9700 1 -- 172.21.15.191:0/4220211151 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8328009320 con 0x7f833806b940 2024-02-15T18:19:25.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.920+0000 7f833e8ba700 1 -- 172.21.15.191:0/4220211151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83381090d0 con 0x7f833806b940 2024-02-15T18:19:25.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.920+0000 7f8334ff9700 1 -- 172.21.15.191:0/4220211151 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8328013380 con 0x7f833806b940 2024-02-15T18:19:25.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.920+0000 7f8334ff9700 1 -- 172.21.15.191:0/4220211151 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f832801ecb0 con 0x7f833806b940 2024-02-15T18:19:25.922 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.920+0000 7f833e8ba700 1 -- 172.21.15.191:0/4220211151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8338109560 con 0x7f833806b940 2024-02-15T18:19:25.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.921+0000 7f8334ff9700 1 -- 172.21.15.191:0/4220211151 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f8328014420 con 0x7f833806b940 2024-02-15T18:19:25.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.921+0000 7f8334ff9700 1 --2- 172.21.15.191:0/4220211151 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f832002ffb0 0x7f8320032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:25.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.921+0000 7f833e8ba700 1 -- 172.21.15.191:0/4220211151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8338045ad0 con 0x7f833806b940 2024-02-15T18:19:25.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.921+0000 7f8334ff9700 1 -- 172.21.15.191:0/4220211151 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f8328040e40 con 0x7f833806b940 2024-02-15T18:19:25.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.921+0000 7f83377fe700 1 --2- 172.21.15.191:0/4220211151 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f832002ffb0 0x7f8320032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:25.923 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.922+0000 7f83377fe700 1 --2- 172.21.15.191:0/4220211151 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f832002ffb0 0x7f8320032470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f832c0096a0 tx=0x7f832c006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:25.926 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:25.926+0000 7f8334ff9700 1 -- 172.21.15.191:0/4220211151 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f832801c8f0 con 0x7f833806b940 2024-02-15T18:19:26.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:26.135+0000 7f833e8ba700 1 -- 172.21.15.191:0/4220211151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f83380fe290 con 0x7f833806b940 2024-02-15T18:19:26.135 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:26.136+0000 7f8334ff9700 1 -- 172.21.15.191:0/4220211151 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f832801a230 con 0x7f833806b940 2024-02-15T18:19:26.136 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:19:26.136 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":1,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","modified":"2024-02-15T18:16:50.575606Z","created":"2024-02-15T18:16:50.575606Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:19:26.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:26.137+0000 7f833e8ba700 1 -- 172.21.15.191:0/4220211151 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f832002ffb0 msgr2=0x7f8320032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:26.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:26.137+0000 7f833e8ba700 1 --2- 172.21.15.191:0/4220211151 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f832002ffb0 0x7f8320032470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f832c0096a0 tx=0x7f832c006b40).stop 2024-02-15T18:19:26.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:26.137+0000 7f833e8ba700 1 -- 172.21.15.191:0/4220211151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f833806b940 msgr2=0x7f83381089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:26.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:26.137+0000 7f833e8ba700 1 --2- 172.21.15.191:0/4220211151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f833806b940 0x7f83381089e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f83280089f0 tx=0x7f832800b040).stop 2024-02-15T18:19:26.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:26.138+0000 7f833e8ba700 1 -- 172.21.15.191:0/4220211151 shutdown_connections 2024-02-15T18:19:26.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:26.138+0000 7f833e8ba700 1 --2- 172.21.15.191:0/4220211151 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f832002ffb0 0x7f8320032470 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:26.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:26.138+0000 7f833e8ba700 1 --2- 172.21.15.191:0/4220211151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f833806b940 0x7f83381089e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:26.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:26.138+0000 7f833e8ba700 1 -- 172.21.15.191:0/4220211151 >> 172.21.15.191:0/4220211151 conn(0x7f83380f1c70 msgr2=0x7f83380f2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:26.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:26.138+0000 7f833e8ba700 1 -- 172.21.15.191:0/4220211151 shutdown_connections 2024-02-15T18:19:26.138 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:26.138+0000 7f833e8ba700 1 -- 172.21.15.191:0/4220211151 wait complete. 2024-02-15T18:19:26.139 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 1 2024-02-15T18:19:26.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:24.772557+0000 mgr.smithi042.kfoihb (mgr.14162) 51 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:26.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:26.136671+0000 mon.smithi042 (mon.0) 151 : audit [DBG] from='client.? 172.21.15.191:0/4220211151' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:19:27.766 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:19:27.766 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:19:28.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:26.772887+0000 mgr.smithi042.kfoihb (mgr.14162) 52 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:30.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.193+0000 7fc3d89f8700 1 -- 172.21.15.191:0/3845242790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3d406b8c0 msgr2=0x7fc3d4069e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:30.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.193+0000 7fc3d89f8700 1 --2- 172.21.15.191:0/3845242790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3d406b8c0 0x7fc3d4069e60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fc3c4009fb0 tx=0x7fc3c4009b30).stop 2024-02-15T18:19:30.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.194+0000 7fc3d89f8700 1 -- 172.21.15.191:0/3845242790 shutdown_connections 2024-02-15T18:19:30.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.194+0000 7fc3d89f8700 1 --2- 172.21.15.191:0/3845242790 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3d406b8c0 0x7fc3d4069e60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:30.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.194+0000 7fc3d89f8700 1 -- 172.21.15.191:0/3845242790 >> 172.21.15.191:0/3845242790 conn(0x7fc3d40f1aa0 msgr2=0x7fc3d40f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:30.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.194+0000 7fc3d89f8700 1 -- 172.21.15.191:0/3845242790 shutdown_connections 2024-02-15T18:19:30.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.194+0000 7fc3d89f8700 1 -- 172.21.15.191:0/3845242790 wait complete. 2024-02-15T18:19:30.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.195+0000 7fc3d89f8700 1 Processor -- start 2024-02-15T18:19:30.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.195+0000 7fc3d89f8700 1 -- start start 2024-02-15T18:19:30.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.195+0000 7fc3d89f8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3d406b8c0 0x7fc3d41057a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:30.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.195+0000 7fc3d89f8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d4106c70 con 0x7fc3d406b8c0 2024-02-15T18:19:30.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.195+0000 7fc3d2d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3d406b8c0 0x7fc3d41057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:30.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.195+0000 7fc3d2d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3d406b8c0 0x7fc3d41057a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.191:57542/0 (socket says 172.21.15.191:57542) 2024-02-15T18:19:30.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.195+0000 7fc3d2d9d700 1 -- 172.21.15.191:0/2037147305 learned_addr learned my addr 172.21.15.191:0/2037147305 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:19:30.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.195+0000 7fc3d2d9d700 1 -- 172.21.15.191:0/2037147305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c4005040 con 0x7fc3d406b8c0 2024-02-15T18:19:30.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.196+0000 7fc3d2d9d700 1 --2- 172.21.15.191:0/2037147305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3d406b8c0 0x7fc3d41057a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fc3c4005010 tx=0x7fc3c400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:30.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.196+0000 7fc3bb7fe700 1 -- 172.21.15.191:0/2037147305 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3c4009320 con 0x7fc3d406b8c0 2024-02-15T18:19:30.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.196+0000 7fc3d89f8700 1 -- 172.21.15.191:0/2037147305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d4106e70 con 0x7fc3d406b8c0 2024-02-15T18:19:30.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.196+0000 7fc3d89f8700 1 -- 172.21.15.191:0/2037147305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d4105ed0 con 0x7fc3d406b8c0 2024-02-15T18:19:30.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.196+0000 7fc3bb7fe700 1 -- 172.21.15.191:0/2037147305 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3c4013380 con 0x7fc3d406b8c0 2024-02-15T18:19:30.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.196+0000 7fc3bb7fe700 1 -- 172.21.15.191:0/2037147305 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc3c4014ce0 con 0x7fc3d406b8c0 2024-02-15T18:19:30.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.197+0000 7fc3bb7fe700 1 -- 172.21.15.191:0/2037147305 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fc3c40263a0 con 0x7fc3d406b8c0 2024-02-15T18:19:30.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.197+0000 7fc3d89f8700 1 -- 172.21.15.191:0/2037147305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b4004fa0 con 0x7fc3d406b8c0 2024-02-15T18:19:30.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.197+0000 7fc3bb7fe700 1 --2- 172.21.15.191:0/2037147305 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc3bc02ff60 0x7fc3bc032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:30.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.197+0000 7fc3bb7fe700 1 -- 172.21.15.191:0/2037147305 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fc3c4040fa0 con 0x7fc3d406b8c0 2024-02-15T18:19:30.197 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.197+0000 7fc3d259c700 1 --2- 172.21.15.191:0/2037147305 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc3bc02ff60 0x7fc3bc032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:30.198 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.198+0000 7fc3d259c700 1 --2- 172.21.15.191:0/2037147305 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc3bc02ff60 0x7fc3bc032420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fc3c80096a0 tx=0x7fc3c8006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:30.201 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.202+0000 7fc3bb7fe700 1 -- 172.21.15.191:0/2037147305 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc3c40126f0 con 0x7fc3d406b8c0 2024-02-15T18:19:30.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:28.773215+0000 mgr.smithi042.kfoihb (mgr.14162) 53 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:30.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:28.962288+0000 mon.smithi042 (mon.0) 152 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:30.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:28.963631+0000 mon.smithi042 (mon.0) 153 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi191.gftdzo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:19:30.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:28.965763+0000 mon.smithi042 (mon.0) 154 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi191.gftdzo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-15T18:19:30.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:28.966590+0000 mon.smithi042 (mon.0) 155 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:19:30.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:28.967579+0000 mon.smithi042 (mon.0) 156 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:19:30.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:19:28.968275+0000 mgr.smithi042.kfoihb (mgr.14162) 54 : cephadm [INF] Deploying daemon mgr.smithi191.gftdzo on smithi191 2024-02-15T18:19:30.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.423+0000 7fc3d89f8700 1 -- 172.21.15.191:0/2037147305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc3b4005670 con 0x7fc3d406b8c0 2024-02-15T18:19:30.424 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.423+0000 7fc3bb7fe700 1 -- 172.21.15.191:0/2037147305 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7fc3c40095f0 con 0x7fc3d406b8c0 2024-02-15T18:19:30.424 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:19:30.424 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":1,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","modified":"2024-02-15T18:16:50.575606Z","created":"2024-02-15T18:16:50.575606Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:19:30.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.425+0000 7fc3d89f8700 1 -- 172.21.15.191:0/2037147305 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc3bc02ff60 msgr2=0x7fc3bc032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:30.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.425+0000 7fc3d89f8700 1 --2- 172.21.15.191:0/2037147305 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc3bc02ff60 0x7fc3bc032420 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fc3c80096a0 tx=0x7fc3c8006b40).stop 2024-02-15T18:19:30.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.425+0000 7fc3d89f8700 1 -- 172.21.15.191:0/2037147305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3d406b8c0 msgr2=0x7fc3d41057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:30.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.425+0000 7fc3d89f8700 1 --2- 172.21.15.191:0/2037147305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3d406b8c0 0x7fc3d41057a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fc3c4005010 tx=0x7fc3c400b040).stop 2024-02-15T18:19:30.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.425+0000 7fc3d89f8700 1 -- 172.21.15.191:0/2037147305 shutdown_connections 2024-02-15T18:19:30.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.425+0000 7fc3d89f8700 1 --2- 172.21.15.191:0/2037147305 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc3bc02ff60 0x7fc3bc032420 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:30.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.425+0000 7fc3d89f8700 1 --2- 172.21.15.191:0/2037147305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3d406b8c0 0x7fc3d41057a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:30.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.425+0000 7fc3d89f8700 1 -- 172.21.15.191:0/2037147305 >> 172.21.15.191:0/2037147305 conn(0x7fc3d40f1aa0 msgr2=0x7fc3d40f24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:30.425 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.425+0000 7fc3d89f8700 1 -- 172.21.15.191:0/2037147305 shutdown_connections 2024-02-15T18:19:30.426 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:30.425+0000 7fc3d89f8700 1 -- 172.21.15.191:0/2037147305 wait complete. 2024-02-15T18:19:30.426 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 1 2024-02-15T18:19:31.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:30.424559+0000 mon.smithi042 (mon.0) 157 : audit [DBG] from='client.? 172.21.15.191:0/2037147305' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:19:32.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:30.773529+0000 mgr.smithi042.kfoihb (mgr.14162) 55 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:32.933 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:19:32.934 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:19:34.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:32.610043+0000 mon.smithi042 (mon.0) 158 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:34.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:32.611536+0000 mon.smithi042 (mon.0) 159 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:19:34.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:32.612405+0000 mon.smithi042 (mon.0) 160 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:19:34.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:19:32.613234+0000 mgr.smithi042.kfoihb (mgr.14162) 56 : cephadm [INF] Deploying daemon mon.smithi191 on smithi191 2024-02-15T18:19:34.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:32.773782+0000 mgr.smithi042.kfoihb (mgr.14162) 57 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:36.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:34.774139+0000 mgr.smithi042.kfoihb (mgr.14162) 58 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:36.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.660+0000 7f36f48bb700 1 -- 172.21.15.191:0/3688846574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36f0060180 msgr2=0x7f36f00f8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:36.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.660+0000 7f36f48bb700 1 --2- 172.21.15.191:0/3688846574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36f0060180 0x7f36f00f8f80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f36e0009fb0 tx=0x7f36e0009b30).stop 2024-02-15T18:19:36.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.660+0000 7f36f48bb700 1 -- 172.21.15.191:0/3688846574 shutdown_connections 2024-02-15T18:19:36.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.660+0000 7f36f48bb700 1 --2- 172.21.15.191:0/3688846574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36f0060180 0x7f36f00f8f80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:36.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.660+0000 7f36f48bb700 1 -- 172.21.15.191:0/3688846574 >> 172.21.15.191:0/3688846574 conn(0x7f36f00f09a0 msgr2=0x7f36f00f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:36.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.660+0000 7f36f48bb700 1 -- 172.21.15.191:0/3688846574 shutdown_connections 2024-02-15T18:19:36.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.660+0000 7f36f48bb700 1 -- 172.21.15.191:0/3688846574 wait complete. 2024-02-15T18:19:36.663 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.660+0000 7f36f48bb700 1 Processor -- start 2024-02-15T18:19:36.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.661+0000 7f36f48bb700 1 -- start start 2024-02-15T18:19:36.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.661+0000 7f36f48bb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36f0060180 0x7f36f0106610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:36.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.661+0000 7f36f48bb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36f0106b00 con 0x7f36f0060180 2024-02-15T18:19:36.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.661+0000 7f36ef7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36f0060180 0x7f36f0106610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:36.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.661+0000 7f36ef7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36f0060180 0x7f36f0106610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.191:42048/0 (socket says 172.21.15.191:42048) 2024-02-15T18:19:36.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.661+0000 7f36ef7fe700 1 -- 172.21.15.191:0/3010322358 learned_addr learned my addr 172.21.15.191:0/3010322358 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:19:36.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.661+0000 7f36ef7fe700 1 -- 172.21.15.191:0/3010322358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e0005040 con 0x7f36f0060180 2024-02-15T18:19:36.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.661+0000 7f36ef7fe700 1 --2- 172.21.15.191:0/3010322358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36f0060180 0x7f36f0106610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f36e0005010 tx=0x7f36e000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:36.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.661+0000 7f36d7fff700 1 -- 172.21.15.191:0/3010322358 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36e0009320 con 0x7f36f0060180 2024-02-15T18:19:36.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.661+0000 7f36f48bb700 1 -- 172.21.15.191:0/3010322358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f0106d00 con 0x7f36f0060180 2024-02-15T18:19:36.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.661+0000 7f36d7fff700 1 -- 172.21.15.191:0/3010322358 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36e0013380 con 0x7f36f0060180 2024-02-15T18:19:36.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.662+0000 7f36d7fff700 1 -- 172.21.15.191:0/3010322358 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f36e0014e50 con 0x7f36f0060180 2024-02-15T18:19:36.664 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.662+0000 7f36f48bb700 1 -- 172.21.15.191:0/3010322358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f0107190 con 0x7f36f0060180 2024-02-15T18:19:36.665 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.662+0000 7f36d7fff700 1 -- 172.21.15.191:0/3010322358 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f36e00263a0 con 0x7f36f0060180 2024-02-15T18:19:36.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.662+0000 7f36f48bb700 1 -- 172.21.15.191:0/3010322358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36f0045ad0 con 0x7f36f0060180 2024-02-15T18:19:36.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.663+0000 7f36d7fff700 1 --2- 172.21.15.191:0/3010322358 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f36d802ff10 0x7f36d80323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:36.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.663+0000 7f36d7fff700 1 -- 172.21.15.191:0/3010322358 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f36e0040f80 con 0x7f36f0060180 2024-02-15T18:19:36.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.663+0000 7f36eeffd700 1 --2- 172.21.15.191:0/3010322358 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f36d802ff10 0x7f36d80323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:36.666 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.663+0000 7f36eeffd700 1 --2- 172.21.15.191:0/3010322358 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f36d802ff10 0x7f36d80323d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f36e40096a0 tx=0x7f36e4006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:36.668 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.667+0000 7f36d7fff700 1 -- 172.21.15.191:0/3010322358 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f36e00126f0 con 0x7f36f0060180 2024-02-15T18:19:36.886 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.885+0000 7f36f48bb700 1 -- 172.21.15.191:0/3010322358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f36f0108040 con 0x7f36f0060180 2024-02-15T18:19:36.887 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.886+0000 7f36d7fff700 1 -- 172.21.15.191:0/3010322358 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f36e00126f0 con 0x7f36f0060180 2024-02-15T18:19:36.887 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:19:36.887 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":1,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","modified":"2024-02-15T18:16:50.575606Z","created":"2024-02-15T18:16:50.575606Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-15T18:19:36.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.888+0000 7f36f48bb700 1 -- 172.21.15.191:0/3010322358 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f36d802ff10 msgr2=0x7f36d80323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:36.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.888+0000 7f36f48bb700 1 --2- 172.21.15.191:0/3010322358 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f36d802ff10 0x7f36d80323d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f36e40096a0 tx=0x7f36e4006b40).stop 2024-02-15T18:19:36.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.888+0000 7f36f48bb700 1 -- 172.21.15.191:0/3010322358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36f0060180 msgr2=0x7f36f0106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:36.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.888+0000 7f36f48bb700 1 --2- 172.21.15.191:0/3010322358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36f0060180 0x7f36f0106610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f36e0005010 tx=0x7f36e000b040).stop 2024-02-15T18:19:36.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.888+0000 7f36f48bb700 1 -- 172.21.15.191:0/3010322358 shutdown_connections 2024-02-15T18:19:36.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.888+0000 7f36f48bb700 1 --2- 172.21.15.191:0/3010322358 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f36d802ff10 0x7f36d80323d0 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0x7f36e40096a0 tx=0x7f36e4006b40).stop 2024-02-15T18:19:36.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.888+0000 7f36f48bb700 1 --2- 172.21.15.191:0/3010322358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36f0060180 0x7f36f0106610 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:36.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.888+0000 7f36f48bb700 1 -- 172.21.15.191:0/3010322358 >> 172.21.15.191:0/3010322358 conn(0x7f36f00f09a0 msgr2=0x7f36f00f1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:36.889 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.888+0000 7f36f48bb700 1 -- 172.21.15.191:0/3010322358 shutdown_connections 2024-02-15T18:19:36.890 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:36.888+0000 7f36f48bb700 1 -- 172.21.15.191:0/3010322358 wait complete. 2024-02-15T18:19:36.890 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 1 2024-02-15T18:19:37.544 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:37 smithi191 bash[40556]: c3dd4902e2a2ad5cadce5b6754bbc3279d0930be87d122ea3fe58d0fc909b519 2024-02-15T18:19:37.544 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:37 smithi191 systemd[1]: Started Ceph mon.smithi191 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:19:37.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:36.887932+0000 mon.smithi042 (mon.0) 161 : audit [DBG] from='client.? 172.21.15.191:0/3010322358' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:19:38.895 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-15T18:19:38.895 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-15T18:19:42.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:36.774464+0000 mgr.smithi042.kfoihb (mgr.14162) 59 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:42.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:37.301033+0000 mon.smithi042 (mon.0) 163 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-02-15T18:19:42.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:37.301233+0000 mon.smithi042 (mon.0) 164 : cluster [INF] mon.smithi042 calling monitor election 2024-02-15T18:19:42.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:37.313803+0000 mon.smithi042 (mon.0) 165 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:19:42.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:37.595675+0000 mon.smithi042 (mon.0) 166 : audit [DBG] from='mgr.? 172.21.15.191:0/3688280866' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/crt"}]: dispatch 2024-02-15T18:19:42.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:38.298421+0000 mon.smithi042 (mon.0) 167 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:19:42.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:38.774710+0000 mgr.smithi042.kfoihb (mgr.14162) 60 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:42.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:39.298653+0000 mon.smithi042 (mon.0) 168 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:19:42.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:39.299898+0000 mon.smithi191 (mon.1) 1 : cluster [INF] mon.smithi191 calling monitor election 2024-02-15T18:19:42.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:40.298688+0000 mon.smithi042 (mon.0) 169 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:19:42.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:40.775032+0000 mgr.smithi042.kfoihb (mgr.14162) 61 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:42.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:41.298678+0000 mon.smithi042 (mon.0) 170 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:19:42.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:42.298777+0000 mon.smithi042 (mon.0) 171 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:19:42.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:42.328111+0000 mon.smithi042 (mon.0) 172 : cluster [INF] mon.smithi042 is new leader, mons smithi042,smithi191 in quorum (ranks 0,1) 2024-02-15T18:19:42.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:42.331230+0000 mon.smithi042 (mon.0) 173 : cluster [DBG] monmap e2: 2 mons at {smithi042=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],smithi191=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} 2024-02-15T18:19:42.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:42.331278+0000 mon.smithi042 (mon.0) 174 : cluster [DBG] fsmap 2024-02-15T18:19:42.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:42.331307+0000 mon.smithi042 (mon.0) 175 : cluster [DBG] osdmap e5: 0 total, 0 up, 0 in 2024-02-15T18:19:42.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:42.331526+0000 mon.smithi042 (mon.0) 176 : cluster [DBG] mgrmap e17: smithi042.kfoihb(active, since 105s) 2024-02-15T18:19:42.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:42.332214+0000 mon.smithi042 (mon.0) 177 : cluster [DBG] Standby manager daemon smithi191.gftdzo started 2024-02-15T18:19:42.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:42.333485+0000 mon.smithi042 (mon.0) 178 : audit [DBG] from='mgr.? 172.21.15.191:0/3688280866' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:19:42.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:42.335287+0000 mon.smithi042 (mon.0) 179 : cluster [INF] overall HEALTH_OK 2024-02-15T18:19:42.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:42.335732+0000 mon.smithi042 (mon.0) 180 : audit [DBG] from='mgr.? 172.21.15.191:0/3688280866' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/key"}]: dispatch 2024-02-15T18:19:42.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:42.337549+0000 mon.smithi042 (mon.0) 181 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:42.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:42.337759+0000 mon.smithi042 (mon.0) 182 : audit [DBG] from='mgr.? 172.21.15.191:0/3688280866' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:19:42.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:36.774464+0000 mgr.smithi042.kfoihb (mgr.14162) 59 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:42.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:37.301033+0000 mon.smithi042 (mon.0) 163 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-02-15T18:19:42.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:37.301233+0000 mon.smithi042 (mon.0) 164 : cluster [INF] mon.smithi042 calling monitor election 2024-02-15T18:19:42.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:37.313803+0000 mon.smithi042 (mon.0) 165 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:19:42.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:37.595675+0000 mon.smithi042 (mon.0) 166 : audit [DBG] from='mgr.? 172.21.15.191:0/3688280866' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/crt"}]: dispatch 2024-02-15T18:19:42.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:38.298421+0000 mon.smithi042 (mon.0) 167 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:19:42.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:38.774710+0000 mgr.smithi042.kfoihb (mgr.14162) 60 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:42.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:39.298653+0000 mon.smithi042 (mon.0) 168 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:19:42.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:39.299898+0000 mon.smithi191 (mon.1) 1 : cluster [INF] mon.smithi191 calling monitor election 2024-02-15T18:19:42.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:40.298688+0000 mon.smithi042 (mon.0) 169 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:19:42.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:40.775032+0000 mgr.smithi042.kfoihb (mgr.14162) 61 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:42.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:41.298678+0000 mon.smithi042 (mon.0) 170 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:19:42.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:42.298777+0000 mon.smithi042 (mon.0) 171 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:19:42.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:42.328111+0000 mon.smithi042 (mon.0) 172 : cluster [INF] mon.smithi042 is new leader, mons smithi042,smithi191 in quorum (ranks 0,1) 2024-02-15T18:19:42.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:42.331230+0000 mon.smithi042 (mon.0) 173 : cluster [DBG] monmap e2: 2 mons at {smithi042=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],smithi191=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} 2024-02-15T18:19:42.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:42.331278+0000 mon.smithi042 (mon.0) 174 : cluster [DBG] fsmap 2024-02-15T18:19:42.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:42.331307+0000 mon.smithi042 (mon.0) 175 : cluster [DBG] osdmap e5: 0 total, 0 up, 0 in 2024-02-15T18:19:42.815 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:42.331526+0000 mon.smithi042 (mon.0) 176 : cluster [DBG] mgrmap e17: smithi042.kfoihb(active, since 105s) 2024-02-15T18:19:42.815 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:42.332214+0000 mon.smithi042 (mon.0) 177 : cluster [DBG] Standby manager daemon smithi191.gftdzo started 2024-02-15T18:19:42.815 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:42.333485+0000 mon.smithi042 (mon.0) 178 : audit [DBG] from='mgr.? 172.21.15.191:0/3688280866' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:19:42.815 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:42.335287+0000 mon.smithi042 (mon.0) 179 : cluster [INF] overall HEALTH_OK 2024-02-15T18:19:42.815 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:42.335732+0000 mon.smithi042 (mon.0) 180 : audit [DBG] from='mgr.? 172.21.15.191:0/3688280866' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/key"}]: dispatch 2024-02-15T18:19:42.815 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:42.337549+0000 mon.smithi042 (mon.0) 181 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:42.815 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:42.337759+0000 mon.smithi042 (mon.0) 182 : audit [DBG] from='mgr.? 172.21.15.191:0/3688280866' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:19:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.621+0000 7f2c4298f700 1 -- 172.21.15.191:0/416404846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c24003250 msgr2=0x7f2c240056e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.621+0000 7f2c4298f700 1 --2- 172.21.15.191:0/416404846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c24003250 0x7f2c240056e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f2c3c0f61c0 tx=0x7f2c2c00b040).stop 2024-02-15T18:19:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.621+0000 7f2c4298f700 1 -- 172.21.15.191:0/416404846 shutdown_connections 2024-02-15T18:19:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.621+0000 7f2c4298f700 1 --2- 172.21.15.191:0/416404846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c24003250 0x7f2c240056e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.621+0000 7f2c4298f700 1 -- 172.21.15.191:0/416404846 >> 172.21.15.191:0/416404846 conn(0x7f2c3c0f0b70 msgr2=0x7f2c3c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.621+0000 7f2c4298f700 1 -- 172.21.15.191:0/416404846 shutdown_connections 2024-02-15T18:19:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.621+0000 7f2c4298f700 1 -- 172.21.15.191:0/416404846 wait complete. 2024-02-15T18:19:43.625 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.623+0000 7f2c4298f700 1 Processor -- start 2024-02-15T18:19:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.623+0000 7f2c4298f700 1 -- start start 2024-02-15T18:19:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.623+0000 7f2c4298f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c24003250 0x7f2c3c1067a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.623+0000 7f2c4298f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c106c90 0x7f2c3c107b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.623+0000 7f2c4298f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c1071f0 con 0x7f2c24003250 2024-02-15T18:19:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.623+0000 7f2c4298f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c107330 con 0x7f2c3c106c90 2024-02-15T18:19:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.623+0000 7f2c3b7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c106c90 0x7f2c3c107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.623+0000 7f2c3b7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c106c90 0x7f2c3c107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:33980/0 (socket says 172.21.15.191:33980) 2024-02-15T18:19:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.623+0000 7f2c3b7fe700 1 -- 172.21.15.191:0/2969809452 learned_addr learned my addr 172.21.15.191:0/2969809452 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:19:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.624+0000 7f2c3b7fe700 1 -- 172.21.15.191:0/2969809452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c106c90 msgr2=0x7f2c3c107b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-15T18:19:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.624+0000 7f2c3b7fe700 1 -- 172.21.15.191:0/2969809452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c106c90 msgr2=0x7f2c3c107b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-15T18:19:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.624+0000 7f2c3b7fe700 1 --2- 172.21.15.191:0/2969809452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c106c90 0x7f2c3c107b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-15T18:19:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.624+0000 7f2c3bfff700 1 --2- 172.21.15.191:0/2969809452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c24003250 0x7f2c3c1067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.624+0000 7f2c3b7fe700 1 --2- 172.21.15.191:0/2969809452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c106c90 0x7f2c3c107b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-02-15T18:19:43.626 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.624+0000 7f2c3bfff700 1 -- 172.21.15.191:0/2969809452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c106c90 msgr2=0x7f2c3c107b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:19:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.624+0000 7f2c3bfff700 1 --2- 172.21.15.191:0/2969809452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c106c90 0x7f2c3c107b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.624+0000 7f2c3bfff700 1 -- 172.21.15.191:0/2969809452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c005040 con 0x7f2c24003250 2024-02-15T18:19:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.624+0000 7f2c3bfff700 1 --2- 172.21.15.191:0/2969809452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c24003250 0x7f2c3c1067a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f2c2c000c00 tx=0x7f2c2c002040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.624+0000 7f2c38ff9700 1 -- 172.21.15.191:0/2969809452 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c2c004910 con 0x7f2c24003250 2024-02-15T18:19:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.624+0000 7f2c4298f700 1 -- 172.21.15.191:0/2969809452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3c108130 con 0x7f2c24003250 2024-02-15T18:19:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.624+0000 7f2c38ff9700 1 -- 172.21.15.191:0/2969809452 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c2c020440 con 0x7f2c24003250 2024-02-15T18:19:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.624+0000 7f2c38ff9700 1 -- 172.21.15.191:0/2969809452 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c2c01f5c0 con 0x7f2c24003250 2024-02-15T18:19:43.627 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.625+0000 7f2c4298f700 1 -- 172.21.15.191:0/2969809452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3c1086f0 con 0x7f2c24003250 2024-02-15T18:19:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.625+0000 7f2c4298f700 1 -- 172.21.15.191:0/2969809452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3c045ad0 con 0x7f2c24003250 2024-02-15T18:19:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.626+0000 7f2c38ff9700 1 -- 172.21.15.191:0/2969809452 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f2c2c004a70 con 0x7f2c24003250 2024-02-15T18:19:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.626+0000 7f2c38ff9700 1 --2- 172.21.15.191:0/2969809452 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2c2805c3b0 0x7f2c2805e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.626+0000 7f2c38ff9700 1 -- 172.21.15.191:0/2969809452 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f2c2c077130 con 0x7f2c24003250 2024-02-15T18:19:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.627+0000 7f2c3b7fe700 1 --2- 172.21.15.191:0/2969809452 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2c2805c3b0 0x7f2c2805e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:43.628 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.627+0000 7f2c3b7fe700 1 --2- 172.21.15.191:0/2969809452 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2c2805c3b0 0x7f2c2805e870 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f2c30006cf0 tx=0x7f2c30008040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:43.631 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.631+0000 7f2c38ff9700 1 -- 172.21.15.191:0/2969809452 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f2c2c04ad80 con 0x7f2c24003250 2024-02-15T18:19:43.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:19:42.338557+0000 mgr.smithi042.kfoihb (mgr.14162) 62 : cephadm [INF] Deploying daemon node-exporter.smithi191 on smithi191 2024-02-15T18:19:43.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:42.392931+0000 mon.smithi042 (mon.0) 183 : cluster [DBG] mgrmap e18: smithi042.kfoihb(active, since 105s), standbys: smithi191.gftdzo 2024-02-15T18:19:43.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:42.393105+0000 mon.smithi042 (mon.0) 184 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr metadata", "who": "smithi191.gftdzo", "id": "smithi191.gftdzo"}]: dispatch 2024-02-15T18:19:43.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:43.299316+0000 mon.smithi042 (mon.0) 185 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:19:43.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:19:42.338557+0000 mgr.smithi042.kfoihb (mgr.14162) 62 : cephadm [INF] Deploying daemon node-exporter.smithi191 on smithi191 2024-02-15T18:19:43.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:42.392931+0000 mon.smithi042 (mon.0) 183 : cluster [DBG] mgrmap e18: smithi042.kfoihb(active, since 105s), standbys: smithi191.gftdzo 2024-02-15T18:19:43.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:42.393105+0000 mon.smithi042 (mon.0) 184 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr metadata", "who": "smithi191.gftdzo", "id": "smithi191.gftdzo"}]: dispatch 2024-02-15T18:19:43.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:43.299316+0000 mon.smithi042 (mon.0) 185 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:19:43.839 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.839+0000 7f2c4298f700 1 -- 172.21.15.191:0/2969809452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2c3c108960 con 0x7f2c24003250 2024-02-15T18:19:43.840 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.840+0000 7f2c38ff9700 1 -- 172.21.15.191:0/2969809452 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+968 (secure 0 0 0) 0x7f2c2c01b070 con 0x7f2c24003250 2024-02-15T18:19:43.840 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-15T18:19:43.840 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":2,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","modified":"2024-02-15T18:19:37.298263Z","created":"2024-02-15T18:16:50.575606Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi191","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-15T18:19:43.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.841+0000 7f2c4298f700 1 -- 172.21.15.191:0/2969809452 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2c2805c3b0 msgr2=0x7f2c2805e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:43.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.841+0000 7f2c4298f700 1 --2- 172.21.15.191:0/2969809452 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2c2805c3b0 0x7f2c2805e870 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f2c30006cf0 tx=0x7f2c30008040).stop 2024-02-15T18:19:43.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.841+0000 7f2c4298f700 1 -- 172.21.15.191:0/2969809452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c24003250 msgr2=0x7f2c3c1067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:43.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.841+0000 7f2c4298f700 1 --2- 172.21.15.191:0/2969809452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c24003250 0x7f2c3c1067a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f2c2c000c00 tx=0x7f2c2c002040).stop 2024-02-15T18:19:43.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.841+0000 7f2c4298f700 1 -- 172.21.15.191:0/2969809452 shutdown_connections 2024-02-15T18:19:43.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.841+0000 7f2c4298f700 1 --2- 172.21.15.191:0/2969809452 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2c2805c3b0 0x7f2c2805e870 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:43.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.841+0000 7f2c4298f700 1 --2- 172.21.15.191:0/2969809452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c24003250 0x7f2c3c1067a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:43.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.841+0000 7f2c4298f700 1 --2- 172.21.15.191:0/2969809452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c106c90 0x7f2c3c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:43.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.841+0000 7f2c4298f700 1 -- 172.21.15.191:0/2969809452 >> 172.21.15.191:0/2969809452 conn(0x7f2c3c0f0b70 msgr2=0x7f2c3c0f2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:43.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.841+0000 7f2c4298f700 1 -- 172.21.15.191:0/2969809452 shutdown_connections 2024-02-15T18:19:43.842 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:19:43.841+0000 7f2c4298f700 1 -- 172.21.15.191:0/2969809452 wait complete. 2024-02-15T18:19:43.843 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 2 2024-02-15T18:19:44.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:42.775270+0000 mgr.smithi042.kfoihb (mgr.14162) 63 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:44.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:43.840799+0000 mon.smithi042 (mon.0) 186 : audit [DBG] from='client.? 172.21.15.191:0/2969809452' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:19:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:42.775270+0000 mgr.smithi042.kfoihb (mgr.14162) 63 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:43.840799+0000 mon.smithi042 (mon.0) 186 : audit [DBG] from='client.? 172.21.15.191:0/2969809452' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-15T18:19:45.888 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-15T18:19:45.888 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph config generate-minimal-conf 2024-02-15T18:19:46.598 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:44.775616+0000 mgr.smithi042.kfoihb (mgr.14162) 64 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:46.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:46 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:44.775616+0000 mgr.smithi042.kfoihb (mgr.14162) 64 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:47.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.751+0000 7fd3092ed700 1 -- 172.21.15.42:0/2282454975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3040f5410 msgr2=0x7fd3040f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:47.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.751+0000 7fd3092ed700 1 --2- 172.21.15.42:0/2282454975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3040f5410 0x7fd3040f5830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fd2f4009fb0 tx=0x7fd2f4009b30).stop 2024-02-15T18:19:47.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.751+0000 7fd3092ed700 1 -- 172.21.15.42:0/2282454975 shutdown_connections 2024-02-15T18:19:47.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.751+0000 7fd3092ed700 1 --2- 172.21.15.42:0/2282454975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3040f5410 0x7fd3040f5830 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:47.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.751+0000 7fd3092ed700 1 -- 172.21.15.42:0/2282454975 >> 172.21.15.42:0/2282454975 conn(0x7fd3040f0b70 msgr2=0x7fd3040f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:47.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.751+0000 7fd3092ed700 1 -- 172.21.15.42:0/2282454975 shutdown_connections 2024-02-15T18:19:47.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.751+0000 7fd3092ed700 1 -- 172.21.15.42:0/2282454975 wait complete. 2024-02-15T18:19:47.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.752+0000 7fd3092ed700 1 Processor -- start 2024-02-15T18:19:47.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.752+0000 7fd3092ed700 1 -- start start 2024-02-15T18:19:47.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.752+0000 7fd3092ed700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3040f5410 0x7fd304106880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:47.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.752+0000 7fd3092ed700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd304106d70 0x7fd304107c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:47.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.752+0000 7fd3092ed700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3041072d0 con 0x7fd304106d70 2024-02-15T18:19:47.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.752+0000 7fd3092ed700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd304107410 con 0x7fd3040f5410 2024-02-15T18:19:47.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.752+0000 7fd3027fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd304106d70 0x7fd304107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:47.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.752+0000 7fd3027fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd304106d70 0x7fd304107c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42216/0 (socket says 172.21.15.42:42216) 2024-02-15T18:19:47.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.752+0000 7fd3027fc700 1 -- 172.21.15.42:0/2629547110 learned_addr learned my addr 172.21.15.42:0/2629547110 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:19:47.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.752+0000 7fd3027fc700 1 -- 172.21.15.42:0/2629547110 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3040f5410 msgr2=0x7fd304106880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:47.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.752+0000 7fd3027fc700 1 --2- 172.21.15.42:0/2629547110 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3040f5410 0x7fd304106880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:19:47.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.752+0000 7fd3027fc700 1 -- 172.21.15.42:0/2629547110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4005040 con 0x7fd304106d70 2024-02-15T18:19:47.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.752+0000 7fd3027fc700 1 --2- 172.21.15.42:0/2629547110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd304106d70 0x7fd304107c60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fd2f80093c0 tx=0x7fd2f8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:47.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.753+0000 7fd2eb7fe700 1 -- 172.21.15.42:0/2629547110 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2f800e650 con 0x7fd304106d70 2024-02-15T18:19:47.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.753+0000 7fd2eb7fe700 1 -- 172.21.15.42:0/2629547110 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2f8009e70 con 0x7fd304106d70 2024-02-15T18:19:47.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.753+0000 7fd3092ed700 1 -- 172.21.15.42:0/2629547110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd304108270 con 0x7fd304106d70 2024-02-15T18:19:47.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.753+0000 7fd2eb7fe700 1 -- 172.21.15.42:0/2629547110 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2f8011ce0 con 0x7fd304106d70 2024-02-15T18:19:47.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.753+0000 7fd3092ed700 1 -- 172.21.15.42:0/2629547110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd304108830 con 0x7fd304106d70 2024-02-15T18:19:47.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.754+0000 7fd3092ed700 1 -- 172.21.15.42:0/2629547110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd304059100 con 0x7fd304106d70 2024-02-15T18:19:47.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.754+0000 7fd2eb7fe700 1 -- 172.21.15.42:0/2629547110 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fd2f8018030 con 0x7fd304106d70 2024-02-15T18:19:47.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.754+0000 7fd2eb7fe700 1 --2- 172.21.15.42:0/2629547110 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd2ec05c3b0 0x7fd2ec05e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:47.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.754+0000 7fd2eb7fe700 1 -- 172.21.15.42:0/2629547110 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7fd2f80749a0 con 0x7fd304106d70 2024-02-15T18:19:47.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.754+0000 7fd302ffd700 1 --2- 172.21.15.42:0/2629547110 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd2ec05c3b0 0x7fd2ec05e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:47.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.755+0000 7fd302ffd700 1 --2- 172.21.15.42:0/2629547110 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd2ec05c3b0 0x7fd2ec05e870 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fd2f4009880 tx=0x7fd2f400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:47.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.758+0000 7fd2eb7fe700 1 -- 172.21.15.42:0/2629547110 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd2f80452b0 con 0x7fd304106d70 2024-02-15T18:19:47.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.916+0000 7fd3092ed700 1 -- 172.21.15.42:0/2629547110 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd304045ad0 con 0x7fd304106d70 2024-02-15T18:19:47.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.916+0000 7fd2eb7fe700 1 -- 172.21.15.42:0/2629547110 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v12) v1 ==== 76+0+225 (secure 0 0 0) 0x7fd2f8016020 con 0x7fd304106d70 2024-02-15T18:19:47.916 INFO:teuthology.orchestra.run.smithi042.stdout:# minimal ceph.conf for 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:19:47.917 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-02-15T18:19:47.917 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:19:47.917 INFO:teuthology.orchestra.run.smithi042.stdout: mon_host = [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] 2024-02-15T18:19:47.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.918+0000 7fd3092ed700 1 -- 172.21.15.42:0/2629547110 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd2ec05c3b0 msgr2=0x7fd2ec05e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:47.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.918+0000 7fd3092ed700 1 --2- 172.21.15.42:0/2629547110 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd2ec05c3b0 0x7fd2ec05e870 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fd2f4009880 tx=0x7fd2f400b040).stop 2024-02-15T18:19:47.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.918+0000 7fd3092ed700 1 -- 172.21.15.42:0/2629547110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd304106d70 msgr2=0x7fd304107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:47.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.918+0000 7fd3092ed700 1 --2- 172.21.15.42:0/2629547110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd304106d70 0x7fd304107c60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fd2f80093c0 tx=0x7fd2f8006ee0).stop 2024-02-15T18:19:47.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.918+0000 7fd3092ed700 1 -- 172.21.15.42:0/2629547110 shutdown_connections 2024-02-15T18:19:47.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.918+0000 7fd3092ed700 1 --2- 172.21.15.42:0/2629547110 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd2ec05c3b0 0x7fd2ec05e870 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:47.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.918+0000 7fd3092ed700 1 --2- 172.21.15.42:0/2629547110 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3040f5410 0x7fd304106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:19:47.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.918+0000 7fd3092ed700 1 --2- 172.21.15.42:0/2629547110 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd304106d70 0x7fd304107c60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:47.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.918+0000 7fd3092ed700 1 -- 172.21.15.42:0/2629547110 >> 172.21.15.42:0/2629547110 conn(0x7fd3040f0b70 msgr2=0x7fd30405fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:47.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.918+0000 7fd3092ed700 1 -- 172.21.15.42:0/2629547110 shutdown_connections 2024-02-15T18:19:47.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:47.918+0000 7fd3092ed700 1 -- 172.21.15.42:0/2629547110 wait complete. 2024-02-15T18:19:48.548 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:46.775982+0000 mgr.smithi042.kfoihb (mgr.14162) 65 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:48.548 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:47.917430+0000 mon.smithi042 (mon.0) 187 : audit [DBG] from='client.? 172.21.15.42:0/2629547110' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:19:48.572 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-15T18:19:48.572 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:19:48.572 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.conf 2024-02-15T18:19:48.606 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:19:48.606 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-15T18:19:48.677 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-15T18:19:48.677 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.conf 2024-02-15T18:19:48.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:46.775982+0000 mgr.smithi042.kfoihb (mgr.14162) 65 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:48.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:47.917430+0000 mon.smithi042 (mon.0) 187 : audit [DBG] from='client.? 172.21.15.42:0/2629547110' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:19:48.716 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-15T18:19:48.716 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-15T18:19:48.786 INFO:tasks.cephadm:Deploying OSDs... 2024-02-15T18:19:48.786 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:19:48.786 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-02-15T18:19:48.805 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-15T18:19:48.805 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-02-15T18:19:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-15T18:19:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:19:48.866 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 162530 Links: 1 2024-02-15T18:19:48.866 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:19:48.866 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:19:48.866 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-02-15 18:19:17.390278247 +0000 2024-02-15T18:19:48.866 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-02-15 18:01:07.497107429 +0000 2024-02-15T18:19:48.866 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-02-15 18:01:07.497107429 +0000 2024-02-15T18:19:48.866 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-02-15T18:19:48.866 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-15T18:19:48.934 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-02-15T18:19:48.934 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-02-15T18:19:48.935 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000110815 s, 4.6 MB/s 2024-02-15T18:19:48.935 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-15T18:19:48.997 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-02-15T18:19:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-15T18:19:49.057 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:19:49.057 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 165714 Links: 1 2024-02-15T18:19:49.057 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:19:49.057 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:19:49.057 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-02-15 18:19:17.390278247 +0000 2024-02-15T18:19:49.057 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-02-15 18:01:07.845104482 +0000 2024-02-15T18:19:49.058 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-02-15 18:01:07.845104482 +0000 2024-02-15T18:19:49.058 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-02-15T18:19:49.058 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-15T18:19:49.125 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-02-15T18:19:49.125 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-02-15T18:19:49.125 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 9.394e-05 s, 5.5 MB/s 2024-02-15T18:19:49.126 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-15T18:19:49.186 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-02-15T18:19:49.245 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-15T18:19:49.245 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:19:49.245 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 162643 Links: 1 2024-02-15T18:19:49.245 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:19:49.245 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:19:49.245 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-02-15 18:19:17.390278247 +0000 2024-02-15T18:19:49.245 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-02-15 18:01:08.178101661 +0000 2024-02-15T18:19:49.245 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-02-15 18:01:08.178101661 +0000 2024-02-15T18:19:49.245 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-02-15T18:19:49.246 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-15T18:19:49.312 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-02-15T18:19:49.313 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-02-15T18:19:49.313 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000103516 s, 4.9 MB/s 2024-02-15T18:19:49.314 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-15T18:19:49.374 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-02-15T18:19:49.434 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-15T18:19:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:19:49.435 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 168396 Links: 1 2024-02-15T18:19:49.435 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:19:49.435 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:19:49.435 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-02-15 18:19:17.390278247 +0000 2024-02-15T18:19:49.435 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-02-15 18:01:08.520098764 +0000 2024-02-15T18:19:49.435 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-02-15 18:01:08.520098764 +0000 2024-02-15T18:19:49.435 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-02-15T18:19:49.435 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-15T18:19:49.502 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-02-15T18:19:49.502 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-02-15T18:19:49.502 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 8.8552e-05 s, 5.8 MB/s 2024-02-15T18:19:49.503 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-15T18:19:49.563 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-15T18:19:49.563 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2024-02-15T18:19:49.585 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-15T18:19:49.585 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2024-02-15T18:19:49.646 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-15T18:19:49.646 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:19:49.646 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 169110 Links: 1 2024-02-15T18:19:49.646 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:19:49.646 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:19:49.647 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-15 18:19:42.769868836 +0000 2024-02-15T18:19:49.647 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-15 18:02:24.889242394 +0000 2024-02-15T18:19:49.647 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-15 18:02:24.889242394 +0000 2024-02-15T18:19:49.647 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-02-15T18:19:49.647 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-15T18:19:49.715 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-15T18:19:49.715 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-15T18:19:49.716 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000242701 s, 2.1 MB/s 2024-02-15T18:19:49.718 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-15T18:19:49.777 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2024-02-15T18:19:49.836 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-15T18:19:49.837 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:19:49.837 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 169157 Links: 1 2024-02-15T18:19:49.837 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:19:49.837 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:19:49.837 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-15 18:19:42.769868836 +0000 2024-02-15T18:19:49.837 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-15 18:02:25.212236690 +0000 2024-02-15T18:19:49.837 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-15 18:02:25.212236690 +0000 2024-02-15T18:19:49.837 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-02-15T18:19:49.837 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-15T18:19:49.905 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-15T18:19:49.905 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-15T18:19:49.905 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000230457 s, 2.2 MB/s 2024-02-15T18:19:49.906 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-15T18:19:49.965 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2024-02-15T18:19:50.023 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-15T18:19:50.023 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:19:50.023 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 169229 Links: 1 2024-02-15T18:19:50.023 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:19:50.023 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:19:50.023 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-15 18:19:42.769868836 +0000 2024-02-15T18:19:50.023 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-15 18:02:25.547230773 +0000 2024-02-15T18:19:50.023 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-15 18:02:25.547230773 +0000 2024-02-15T18:19:50.023 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-02-15T18:19:50.024 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-15T18:19:50.088 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-15T18:19:50.088 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-15T18:19:50.088 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000195487 s, 2.6 MB/s 2024-02-15T18:19:50.089 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-15T18:19:50.148 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2024-02-15T18:19:50.206 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-15T18:19:50.206 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-15T18:19:50.206 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 168298 Links: 1 2024-02-15T18:19:50.206 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:19:50.206 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-15T18:19:50.206 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-15 18:19:42.770868818 +0000 2024-02-15T18:19:50.206 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-15 18:02:25.869225086 +0000 2024-02-15T18:19:50.206 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-15 18:02:25.869225086 +0000 2024-02-15T18:19:50.207 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2024-02-15T18:19:50.207 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-15T18:19:50.271 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-15T18:19:50.272 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-15T18:19:50.272 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000253755 s, 2.0 MB/s 2024-02-15T18:19:50.273 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-15T18:19:50.331 INFO:tasks.cephadm:Deploying osd.0 on smithi042 with /dev/vg_nvme/lv_4... 2024-02-15T18:19:50.331 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-15T18:19:50.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:48.776260+0000 mgr.smithi042.kfoihb (mgr.14162) 66 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:50.733 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:48.776260+0000 mgr.smithi042.kfoihb (mgr.14162) 66 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:52.604 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-15T18:19:52.604 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-15T18:19:52.604 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-02-15T18:19:52.604 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-02-15T18:19:52.604 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00795112 s, 1.3 GB/s 2024-02-15T18:19:52.605 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-02-15T18:19:52.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:50.776510+0000 mgr.smithi042.kfoihb (mgr.14162) 67 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:52.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:51.890339+0000 mon.smithi042 (mon.0) 188 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:52.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:51.891261+0000 mon.smithi042 (mon.0) 189 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:19:52.738 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:50.776510+0000 mgr.smithi042.kfoihb (mgr.14162) 67 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:52.739 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:51.890339+0000 mon.smithi042 (mon.0) 188 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:52.739 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:51.891261+0000 mon.smithi042 (mon.0) 189 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:19:53.380 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi042:vg_nvme/lv_4 2024-02-15T18:19:54.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:52.776713+0000 mgr.smithi042.kfoihb (mgr.14162) 68 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:54.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:53.740179+0000 mon.smithi042 (mon.0) 190 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:54.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:52.776713+0000 mgr.smithi042.kfoihb (mgr.14162) 68 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:54.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:53.740179+0000 mon.smithi042 (mon.0) 190 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:19:56.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.001+0000 7f28c30f2700 1 -- 172.21.15.42:0/1346181405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28bc0f7b30 msgr2=0x7f28bc0f7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:56.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.001+0000 7f28c30f2700 1 --2- 172.21.15.42:0/1346181405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28bc0f7b30 0x7f28bc0f7f90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f28a8004660 tx=0x7f28a8009b30).stop 2024-02-15T18:19:56.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.001+0000 7f28c30f2700 1 -- 172.21.15.42:0/1346181405 shutdown_connections 2024-02-15T18:19:56.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.001+0000 7f28c30f2700 1 --2- 172.21.15.42:0/1346181405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28bc0f7b30 0x7f28bc0f7f90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:56.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.001+0000 7f28c30f2700 1 --2- 172.21.15.42:0/1346181405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28bc0f6890 0x7f28bc0f6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:56.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.001+0000 7f28c30f2700 1 -- 172.21.15.42:0/1346181405 >> 172.21.15.42:0/1346181405 conn(0x7f28bc0f1ff0 msgr2=0x7f28bc0f4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:19:56.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.001+0000 7f28c30f2700 1 -- 172.21.15.42:0/1346181405 shutdown_connections 2024-02-15T18:19:56.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.001+0000 7f28c30f2700 1 -- 172.21.15.42:0/1346181405 wait complete. 2024-02-15T18:19:56.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.002+0000 7f28c30f2700 1 Processor -- start 2024-02-15T18:19:56.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.002+0000 7f28c30f2700 1 -- start start 2024-02-15T18:19:56.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.002+0000 7f28c30f2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28bc0f6890 0x7f28bc107120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:56.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.002+0000 7f28c30f2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28bc0f7b30 0x7f28bc107610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:56.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.002+0000 7f28c30f2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc107bc0 con 0x7f28bc0f6890 2024-02-15T18:19:56.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.002+0000 7f28c30f2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc107d00 con 0x7f28bc0f7b30 2024-02-15T18:19:56.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.002+0000 7f28c0e8e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28bc0f6890 0x7f28bc107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:56.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.002+0000 7f28c0e8e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28bc0f6890 0x7f28bc107120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53252/0 (socket says 172.21.15.42:53252) 2024-02-15T18:19:56.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.003+0000 7f28c0e8e700 1 -- 172.21.15.42:0/2032225997 learned_addr learned my addr 172.21.15.42:0/2032225997 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:19:56.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.003+0000 7f28bbfff700 1 --2- 172.21.15.42:0/2032225997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28bc0f7b30 0x7f28bc107610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:56.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.003+0000 7f28c0e8e700 1 -- 172.21.15.42:0/2032225997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28bc0f7b30 msgr2=0x7f28bc107610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:19:56.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.003+0000 7f28c0e8e700 1 --2- 172.21.15.42:0/2032225997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28bc0f7b30 0x7f28bc107610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:19:56.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.003+0000 7f28c0e8e700 1 -- 172.21.15.42:0/2032225997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a8005040 con 0x7f28bc0f6890 2024-02-15T18:19:56.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.003+0000 7f28c0e8e700 1 --2- 172.21.15.42:0/2032225997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28bc0f6890 0x7f28bc107120 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f28b000e7c0 tx=0x7f28b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:56.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.003+0000 7f28b97fa700 1 -- 172.21.15.42:0/2032225997 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28b00092e0 con 0x7f28bc0f6890 2024-02-15T18:19:56.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.003+0000 7f28c30f2700 1 -- 172.21.15.42:0/2032225997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28bc1086a0 con 0x7f28bc0f6890 2024-02-15T18:19:56.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.003+0000 7f28b97fa700 1 -- 172.21.15.42:0/2032225997 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28b000f040 con 0x7f28bc0f6890 2024-02-15T18:19:56.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.003+0000 7f28b97fa700 1 -- 172.21.15.42:0/2032225997 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28b00147e0 con 0x7f28bc0f6890 2024-02-15T18:19:56.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.003+0000 7f28c30f2700 1 -- 172.21.15.42:0/2032225997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28bc108c60 con 0x7f28bc0f6890 2024-02-15T18:19:56.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.004+0000 7f28c30f2700 1 -- 172.21.15.42:0/2032225997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28bc045ad0 con 0x7f28bc0f6890 2024-02-15T18:19:56.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.004+0000 7f28b97fa700 1 -- 172.21.15.42:0/2032225997 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f28b001b030 con 0x7f28bc0f6890 2024-02-15T18:19:56.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.005+0000 7f28b97fa700 1 --2- 172.21.15.42:0/2032225997 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f28ac05c300 0x7f28ac05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:19:56.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.005+0000 7f28b97fa700 1 -- 172.21.15.42:0/2032225997 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1270+0+0 (secure 0 0 0) 0x7f28b0074f60 con 0x7f28bc0f6890 2024-02-15T18:19:56.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.005+0000 7f28bbfff700 1 --2- 172.21.15.42:0/2032225997 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f28ac05c300 0x7f28ac05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:19:56.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.005+0000 7f28bbfff700 1 --2- 172.21.15.42:0/2032225997 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f28ac05c300 0x7f28ac05e7c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f28a80043d0 tx=0x7f28a800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:19:56.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.009+0000 7f28b97fa700 1 -- 172.21.15.42:0/2032225997 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f28b00a12b0 con 0x7f28bc0f6890 2024-02-15T18:19:56.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:19:56.196+0000 7f28c30f2700 1 -- 172.21.15.42:0/2032225997 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f28bc0ff160 con 0x7f28ac05c300 2024-02-15T18:19:56.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:54.776948+0000 mgr.smithi042.kfoihb (mgr.14162) 69 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:56.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:56.199372+0000 mon.smithi042 (mon.0) 191 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:19:56.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:56.201237+0000 mon.smithi042 (mon.0) 192 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:19:56.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:56.201871+0000 mon.smithi042 (mon.0) 193 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:19:56.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:54.776948+0000 mgr.smithi042.kfoihb (mgr.14162) 69 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:56.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:56.199372+0000 mon.smithi042 (mon.0) 191 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:19:56.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:56.201237+0000 mon.smithi042 (mon.0) 192 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:19:56.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:56.201871+0000 mon.smithi042 (mon.0) 193 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:19:57.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:56.198276+0000 mgr.smithi042.kfoihb (mgr.14162) 70 : audit [DBG] from='client.14218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:19:57.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:19:56.200960+0000 mgr.smithi042.kfoihb (mgr.14162) 71 : cephadm [INF] Applying service osd.None on host smithi042... 2024-02-15T18:19:57.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:56.901526+0000 mon.smithi042 (mon.0) 194 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:19:57.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:56.910239+0000 mon.smithi042 (mon.0) 195 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:19:57.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:56.198276+0000 mgr.smithi042.kfoihb (mgr.14162) 70 : audit [DBG] from='client.14218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:19:57.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:19:56.200960+0000 mgr.smithi042.kfoihb (mgr.14162) 71 : cephadm [INF] Applying service osd.None on host smithi042... 2024-02-15T18:19:57.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:56.901526+0000 mon.smithi042 (mon.0) 194 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:19:57.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:56.910239+0000 mon.smithi042 (mon.0) 195 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:19:58.691 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:19:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:56.777279+0000 mgr.smithi042.kfoihb (mgr.14162) 72 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:19:58.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:19:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:56.777279+0000 mgr.smithi042.kfoihb (mgr.14162) 72 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:00.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:58.777597+0000 mgr.smithi042.kfoihb (mgr.14162) 73 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:00.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:59.459126+0000 mon.smithi042 (mon.0) 196 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:00.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:59.464182+0000 mon.smithi042 (mon.0) 197 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:00.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:19:59.465794+0000 mgr.smithi042.kfoihb (mgr.14162) 74 : cephadm [INF] Reconfiguring mon.smithi042 (unknown last config time)... 2024-02-15T18:20:00.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:59.466025+0000 mon.smithi042 (mon.0) 198 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:20:00.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:59.466618+0000 mon.smithi042 (mon.0) 199 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:20:00.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:59.467178+0000 mon.smithi042 (mon.0) 200 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:00.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:19:59.467641+0000 mgr.smithi042.kfoihb (mgr.14162) 75 : cephadm [INF] Reconfiguring daemon mon.smithi042 on smithi042 2024-02-15T18:20:00.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:59.681902+0000 mon.smithi042 (mon.0) 201 : audit [INF] from='client.? 172.21.15.42:0/1804311093' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42281c08-af07-43a5-bcfd-d476bc745782"}]: dispatch 2024-02-15T18:20:00.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:59.685368+0000 mon.smithi042 (mon.0) 202 : audit [INF] from='client.? 172.21.15.42:0/1804311093' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42281c08-af07-43a5-bcfd-d476bc745782"}]': finished 2024-02-15T18:20:00.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:19:59.685415+0000 mon.smithi042 (mon.0) 203 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-15T18:20:00.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:19:59.685505+0000 mon.smithi042 (mon.0) 204 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:20:00.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:00.000109+0000 mon.smithi042 (mon.0) 205 : cluster [INF] overall HEALTH_OK 2024-02-15T18:20:00.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:00.108939+0000 mon.smithi042 (mon.0) 206 : audit [DBG] from='client.? 172.21.15.42:0/1340434768' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:20:00.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:58.777597+0000 mgr.smithi042.kfoihb (mgr.14162) 73 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:00.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:59.459126+0000 mon.smithi042 (mon.0) 196 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:00.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:59.464182+0000 mon.smithi042 (mon.0) 197 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:00.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:19:59.465794+0000 mgr.smithi042.kfoihb (mgr.14162) 74 : cephadm [INF] Reconfiguring mon.smithi042 (unknown last config time)... 2024-02-15T18:20:00.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:59.466025+0000 mon.smithi042 (mon.0) 198 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:20:00.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:59.466618+0000 mon.smithi042 (mon.0) 199 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:20:00.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:59.467178+0000 mon.smithi042 (mon.0) 200 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:00.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:19:59.467641+0000 mgr.smithi042.kfoihb (mgr.14162) 75 : cephadm [INF] Reconfiguring daemon mon.smithi042 on smithi042 2024-02-15T18:20:00.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:59.681902+0000 mon.smithi042 (mon.0) 201 : audit [INF] from='client.? 172.21.15.42:0/1804311093' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "42281c08-af07-43a5-bcfd-d476bc745782"}]: dispatch 2024-02-15T18:20:00.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:59.685368+0000 mon.smithi042 (mon.0) 202 : audit [INF] from='client.? 172.21.15.42:0/1804311093' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42281c08-af07-43a5-bcfd-d476bc745782"}]': finished 2024-02-15T18:20:00.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:19:59.685415+0000 mon.smithi042 (mon.0) 203 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-15T18:20:00.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:19:59.685505+0000 mon.smithi042 (mon.0) 204 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:20:00.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:00.000109+0000 mon.smithi042 (mon.0) 205 : cluster [INF] overall HEALTH_OK 2024-02-15T18:20:00.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:00.108939+0000 mon.smithi042 (mon.0) 206 : audit [DBG] from='client.? 172.21.15.42:0/1340434768' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:20:02.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:00.777909+0000 mgr.smithi042.kfoihb (mgr.14162) 76 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:02.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:00.777909+0000 mgr.smithi042.kfoihb (mgr.14162) 76 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:04.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:02.778129+0000 mgr.smithi042.kfoihb (mgr.14162) 77 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:04.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:02.778129+0000 mgr.smithi042.kfoihb (mgr.14162) 77 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:06.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:04.778356+0000 mgr.smithi042.kfoihb (mgr.14162) 78 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:06.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:05.585055+0000 mon.smithi042 (mon.0) 207 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:06.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:05.585866+0000 mon.smithi042 (mon.0) 208 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi042.kfoihb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:20:06.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:05.586531+0000 mon.smithi042 (mon.0) 209 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:20:06.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:05.587073+0000 mon.smithi042 (mon.0) 210 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:06.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:04.778356+0000 mgr.smithi042.kfoihb (mgr.14162) 78 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:06.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:05.585055+0000 mon.smithi042 (mon.0) 207 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:06.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:05.585866+0000 mon.smithi042 (mon.0) 208 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi042.kfoihb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:20:06.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:05.586531+0000 mon.smithi042 (mon.0) 209 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:20:06.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:05.587073+0000 mon.smithi042 (mon.0) 210 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:07.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:05.585570+0000 mgr.smithi042.kfoihb (mgr.14162) 79 : cephadm [INF] Reconfiguring mgr.smithi042.kfoihb (unknown last config time)... 2024-02-15T18:20:07.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:05.587543+0000 mgr.smithi042.kfoihb (mgr.14162) 80 : cephadm [INF] Reconfiguring daemon mgr.smithi042.kfoihb on smithi042 2024-02-15T18:20:07.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:05.585570+0000 mgr.smithi042.kfoihb (mgr.14162) 79 : cephadm [INF] Reconfiguring mgr.smithi042.kfoihb (unknown last config time)... 2024-02-15T18:20:07.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:05.587543+0000 mgr.smithi042.kfoihb (mgr.14162) 80 : cephadm [INF] Reconfiguring daemon mgr.smithi042.kfoihb on smithi042 2024-02-15T18:20:08.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:06.778570+0000 mgr.smithi042.kfoihb (mgr.14162) 81 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:08.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:07.551112+0000 mon.smithi042 (mon.0) 211 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-15T18:20:08.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:07.551789+0000 mon.smithi042 (mon.0) 212 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:08.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:06.778570+0000 mgr.smithi042.kfoihb (mgr.14162) 81 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:08.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:07.551112+0000 mon.smithi042 (mon.0) 211 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-15T18:20:08.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:07.551789+0000 mon.smithi042 (mon.0) 212 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:09.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:07.552328+0000 mgr.smithi042.kfoihb (mgr.14162) 82 : cephadm [INF] Deploying daemon osd.0 on smithi042 2024-02-15T18:20:09.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:09.058449+0000 mon.smithi042 (mon.0) 213 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:09.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:07.552328+0000 mgr.smithi042.kfoihb (mgr.14162) 82 : cephadm [INF] Deploying daemon osd.0 on smithi042 2024-02-15T18:20:09.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:09.058449+0000 mon.smithi042 (mon.0) 213 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:10.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:08.778916+0000 mgr.smithi042.kfoihb (mgr.14162) 83 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:10.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:09.059016+0000 mgr.smithi042.kfoihb (mgr.14162) 84 : cephadm [INF] Reconfiguring alertmanager.smithi042 (dependencies changed)... 2024-02-15T18:20:10.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:09.061124+0000 mgr.smithi042.kfoihb (mgr.14162) 85 : cephadm [INF] Reconfiguring daemon alertmanager.smithi042 on smithi042 2024-02-15T18:20:10.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:08.778916+0000 mgr.smithi042.kfoihb (mgr.14162) 83 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:10.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:09.059016+0000 mgr.smithi042.kfoihb (mgr.14162) 84 : cephadm [INF] Reconfiguring alertmanager.smithi042 (dependencies changed)... 2024-02-15T18:20:10.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:09.061124+0000 mgr.smithi042.kfoihb (mgr.14162) 85 : cephadm [INF] Reconfiguring daemon alertmanager.smithi042 on smithi042 2024-02-15T18:20:12.632 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 0 on host 'smithi042' 2024-02-15T18:20:12.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:12.630+0000 7f28b97fa700 1 -- 172.21.15.42:0/2032225997 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f28bc0ff160 con 0x7f28ac05c300 2024-02-15T18:20:12.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:12.634+0000 7f28c30f2700 1 -- 172.21.15.42:0/2032225997 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f28ac05c300 msgr2=0x7f28ac05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:20:12.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:12.634+0000 7f28c30f2700 1 --2- 172.21.15.42:0/2032225997 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f28ac05c300 0x7f28ac05e7c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f28a80043d0 tx=0x7f28a800b040).stop 2024-02-15T18:20:12.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:12.634+0000 7f28c30f2700 1 -- 172.21.15.42:0/2032225997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28bc0f6890 msgr2=0x7f28bc107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:20:12.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:12.634+0000 7f28c30f2700 1 --2- 172.21.15.42:0/2032225997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28bc0f6890 0x7f28bc107120 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f28b000e7c0 tx=0x7f28b000b040).stop 2024-02-15T18:20:12.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:12.634+0000 7f28c30f2700 1 -- 172.21.15.42:0/2032225997 shutdown_connections 2024-02-15T18:20:12.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:12.634+0000 7f28c30f2700 1 --2- 172.21.15.42:0/2032225997 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f28ac05c300 0x7f28ac05e7c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:20:12.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:12.634+0000 7f28c30f2700 1 --2- 172.21.15.42:0/2032225997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28bc0f6890 0x7f28bc107120 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:20:12.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:12.634+0000 7f28c30f2700 1 --2- 172.21.15.42:0/2032225997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28bc0f7b30 0x7f28bc107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:20:12.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:12.634+0000 7f28c30f2700 1 -- 172.21.15.42:0/2032225997 >> 172.21.15.42:0/2032225997 conn(0x7f28bc0f1ff0 msgr2=0x7f28bc0fb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:20:12.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:12.634+0000 7f28c30f2700 1 -- 172.21.15.42:0/2032225997 shutdown_connections 2024-02-15T18:20:12.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:12.634+0000 7f28c30f2700 1 -- 172.21.15.42:0/2032225997 wait complete. 2024-02-15T18:20:12.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:10.779157+0000 mgr.smithi042.kfoihb (mgr.14162) 86 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:12.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:10.779157+0000 mgr.smithi042.kfoihb (mgr.14162) 86 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:13.787 DEBUG:teuthology.orchestra.run.smithi042:osd.0> sudo journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.0.service 2024-02-15T18:20:13.790 INFO:tasks.cephadm:Deploying osd.1 on smithi042 with /dev/vg_nvme/lv_3... 2024-02-15T18:20:13.790 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-15T18:20:13.826 INFO:journalctl@ceph.osd.0.smithi042.stdout:-- Logs begin at Thu 2024-02-15 17:54:32 UTC. -- 2024-02-15T18:20:13.851 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:12.625364+0000 mon.smithi042 (mon.0) 214 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:13.852 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:12.630806+0000 mon.smithi042 (mon.0) 215 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:13.852 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:12.779424+0000 mgr.smithi042.kfoihb (mgr.14162) 87 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:13.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:12.625364+0000 mon.smithi042 (mon.0) 214 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:13.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:12.630806+0000 mon.smithi042 (mon.0) 215 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:13.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:12.779424+0000 mgr.smithi042.kfoihb (mgr.14162) 87 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:16.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:14.779707+0000 mgr.smithi042.kfoihb (mgr.14162) 88 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:16.563 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:20:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.0[48500]: debug 2024-02-15T18:20:16.480+0000 7f3e6fe5a080 -1 osd.0 0 log_to_monitors {default=true} 2024-02-15T18:20:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:14.779707+0000 mgr.smithi042.kfoihb (mgr.14162) 88 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:17.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:16.484852+0000 mon.smithi042 (mon.0) 216 : audit [INF] from='osd.0 [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-15T18:20:17.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:16.484852+0000 mon.smithi042 (mon.0) 216 : audit [INF] from='osd.0 [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-15T18:20:18.506 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-15T18:20:18.507 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-15T18:20:18.507 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-02-15T18:20:18.507 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-02-15T18:20:18.507 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00985125 s, 1.1 GB/s 2024-02-15T18:20:18.507 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-02-15T18:20:18.563 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:20:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.0[48500]: debug 2024-02-15T18:20:18.247+0000 7f3e663cb700 -1 osd.0 0 waiting for initial osdmap 2024-02-15T18:20:18.563 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:20:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.0[48500]: debug 2024-02-15T18:20:18.250+0000 7f3e61b45700 -1 osd.0 8 set_numa_affinity unable to identify public interface 'ens1f0' numa node: (0) Success 2024-02-15T18:20:18.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:16.780113+0000 mgr.smithi042.kfoihb (mgr.14162) 89 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:18.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:17.243080+0000 mon.smithi042 (mon.0) 217 : audit [INF] from='osd.0 [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-15T18:20:18.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:17.243181+0000 mon.smithi042 (mon.0) 218 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-15T18:20:18.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:17.243354+0000 mon.smithi042 (mon.0) 219 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:20:18.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:17.243803+0000 mon.smithi042 (mon.0) 220 : audit [INF] from='osd.0 [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-02-15T18:20:18.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:16.780113+0000 mgr.smithi042.kfoihb (mgr.14162) 89 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:18.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:17.243080+0000 mon.smithi042 (mon.0) 217 : audit [INF] from='osd.0 [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-15T18:20:18.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:17.243181+0000 mon.smithi042 (mon.0) 218 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-15T18:20:18.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:17.243354+0000 mon.smithi042 (mon.0) 219 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:20:18.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:17.243803+0000 mon.smithi042 (mon.0) 220 : audit [INF] from='osd.0 [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-02-15T18:20:19.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:18.244155+0000 mon.smithi042 (mon.0) 221 : audit [INF] from='osd.0 [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-02-15T18:20:19.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:18.244231+0000 mon.smithi042 (mon.0) 222 : cluster [DBG] osdmap e8: 1 total, 0 up, 1 in 2024-02-15T18:20:19.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:18.244549+0000 mon.smithi042 (mon.0) 223 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:20:19.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:18.248388+0000 mon.smithi042 (mon.0) 224 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:20:19.517 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi042:vg_nvme/lv_3 2024-02-15T18:20:19.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:18.244155+0000 mon.smithi042 (mon.0) 221 : audit [INF] from='osd.0 [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-02-15T18:20:19.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:18.244231+0000 mon.smithi042 (mon.0) 222 : cluster [DBG] osdmap e8: 1 total, 0 up, 1 in 2024-02-15T18:20:19.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:18.244549+0000 mon.smithi042 (mon.0) 223 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:20:19.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:18.248388+0000 mon.smithi042 (mon.0) 224 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:20:20.482 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:17.479795+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:20:20.482 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:17.479918+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:20:20.482 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:18.780484+0000 mgr.smithi042.kfoihb (mgr.14162) 90 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:20.482 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:19.248852+0000 mon.smithi042 (mon.0) 225 : cluster [INF] osd.0 [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361] boot 2024-02-15T18:20:20.482 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:19.248879+0000 mon.smithi042 (mon.0) 226 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-15T18:20:20.483 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:19.249290+0000 mon.smithi042 (mon.0) 227 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:20:20.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:17.479795+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:20:20.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:17.479918+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:20:20.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:18.780484+0000 mgr.smithi042.kfoihb (mgr.14162) 90 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-15T18:20:20.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:19.248852+0000 mon.smithi042 (mon.0) 225 : cluster [INF] osd.0 [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361] boot 2024-02-15T18:20:20.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:19.248879+0000 mon.smithi042 (mon.0) 226 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-15T18:20:20.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:19.249290+0000 mon.smithi042 (mon.0) 227 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:20:21.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:20.255706+0000 mon.smithi042 (mon.0) 228 : cluster [DBG] osdmap e10: 1 total, 1 up, 1 in 2024-02-15T18:20:21.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:20.329401+0000 mon.smithi042 (mon.0) 229 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:21.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:20.329895+0000 mgr.smithi042.kfoihb (mgr.14162) 91 : cephadm [INF] Reconfiguring crash.smithi042 (monmap changed)... 2024-02-15T18:20:21.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:20.330110+0000 mon.smithi042 (mon.0) 230 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi042", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:20:21.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:20.330773+0000 mon.smithi042 (mon.0) 231 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:21.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:20.331298+0000 mgr.smithi042.kfoihb (mgr.14162) 92 : cephadm [INF] Reconfiguring daemon crash.smithi042 on smithi042 2024-02-15T18:20:21.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:20.255706+0000 mon.smithi042 (mon.0) 228 : cluster [DBG] osdmap e10: 1 total, 1 up, 1 in 2024-02-15T18:20:21.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:20.329401+0000 mon.smithi042 (mon.0) 229 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:21.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:20.329895+0000 mgr.smithi042.kfoihb (mgr.14162) 91 : cephadm [INF] Reconfiguring crash.smithi042 (monmap changed)... 2024-02-15T18:20:21.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:20.330110+0000 mon.smithi042 (mon.0) 230 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi042", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:20:21.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:20.330773+0000 mon.smithi042 (mon.0) 231 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:21.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:20.331298+0000 mgr.smithi042.kfoihb (mgr.14162) 92 : cephadm [INF] Reconfiguring daemon crash.smithi042 on smithi042 2024-02-15T18:20:22.435 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:20.780918+0000 mgr.smithi042.kfoihb (mgr.14162) 93 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:22.435 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:21.809190+0000 mon.smithi042 (mon.0) 232 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:22.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:20.780918+0000 mgr.smithi042.kfoihb (mgr.14162) 93 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:22.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:21.809190+0000 mon.smithi042 (mon.0) 232 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:23.599 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:21.809750+0000 mgr.smithi042.kfoihb (mgr.14162) 94 : cephadm [INF] Reconfiguring grafana.smithi042 (dependencies changed)... 2024-02-15T18:20:23.600 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:21.814085+0000 mgr.smithi042.kfoihb (mgr.14162) 95 : cephadm [INF] Reconfiguring daemon grafana.smithi042 on smithi042 2024-02-15T18:20:23.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.175+0000 7f5cb0d06700 1 -- 172.21.15.42:0/3793929928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cac0f66e0 msgr2=0x7f5cac0f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:20:23.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.175+0000 7f5cb0d06700 1 --2- 172.21.15.42:0/3793929928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cac0f66e0 0x7f5cac0f6b40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f5c9c004660 tx=0x7f5c9c009b30).stop 2024-02-15T18:20:23.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.175+0000 7f5cb0d06700 1 -- 172.21.15.42:0/3793929928 shutdown_connections 2024-02-15T18:20:23.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.175+0000 7f5cb0d06700 1 --2- 172.21.15.42:0/3793929928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cac0f66e0 0x7f5cac0f6b40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:20:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.175+0000 7f5cb0d06700 1 --2- 172.21.15.42:0/3793929928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cac0f5440 0x7f5cac0f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:20:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.175+0000 7f5cb0d06700 1 -- 172.21.15.42:0/3793929928 >> 172.21.15.42:0/3793929928 conn(0x7f5cac0f0ba0 msgr2=0x7f5cac0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:20:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.175+0000 7f5cb0d06700 1 -- 172.21.15.42:0/3793929928 shutdown_connections 2024-02-15T18:20:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.175+0000 7f5cb0d06700 1 -- 172.21.15.42:0/3793929928 wait complete. 2024-02-15T18:20:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.176+0000 7f5cb0d06700 1 Processor -- start 2024-02-15T18:20:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.176+0000 7f5cb0d06700 1 -- start start 2024-02-15T18:20:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.176+0000 7f5cb0d06700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cac0f5440 0x7f5cac105d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:20:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.176+0000 7f5cb0d06700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cac0f66e0 0x7f5cac106250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:20:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.176+0000 7f5cb0d06700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac106850 con 0x7f5cac0f66e0 2024-02-15T18:20:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.176+0000 7f5cb0d06700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac107060 con 0x7f5cac0f5440 2024-02-15T18:20:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.176+0000 7f5caaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cac0f66e0 0x7f5cac106250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:20:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.176+0000 7f5caaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cac0f66e0 0x7f5cac106250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45326/0 (socket says 172.21.15.42:45326) 2024-02-15T18:20:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.176+0000 7f5caaffd700 1 -- 172.21.15.42:0/1164224039 learned_addr learned my addr 172.21.15.42:0/1164224039 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:20:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.176+0000 7f5cab7fe700 1 --2- 172.21.15.42:0/1164224039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cac0f5440 0x7f5cac105d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:20:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.177+0000 7f5caaffd700 1 -- 172.21.15.42:0/1164224039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cac0f5440 msgr2=0x7f5cac105d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:20:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.177+0000 7f5caaffd700 1 --2- 172.21.15.42:0/1164224039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cac0f5440 0x7f5cac105d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:20:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.177+0000 7f5caaffd700 1 -- 172.21.15.42:0/1164224039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c005040 con 0x7f5cac0f66e0 2024-02-15T18:20:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.177+0000 7f5cab7fe700 1 --2- 172.21.15.42:0/1164224039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cac0f5440 0x7f5cac105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:20:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.177+0000 7f5caaffd700 1 --2- 172.21.15.42:0/1164224039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cac0f66e0 0x7f5cac106250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f5c9c009830 tx=0x7f5c9c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:20:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.177+0000 7f5c93fff700 1 -- 172.21.15.42:0/1164224039 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c9c018070 con 0x7f5cac0f66e0 2024-02-15T18:20:23.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.177+0000 7f5cb0d06700 1 -- 172.21.15.42:0/1164224039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac107280 con 0x7f5cac0f66e0 2024-02-15T18:20:23.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.177+0000 7f5c93fff700 1 -- 172.21.15.42:0/1164224039 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c9c007d70 con 0x7f5cac0f66e0 2024-02-15T18:20:23.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.177+0000 7f5c93fff700 1 -- 172.21.15.42:0/1164224039 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c9c01c620 con 0x7f5cac0f66e0 2024-02-15T18:20:23.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.177+0000 7f5cb0d06700 1 -- 172.21.15.42:0/1164224039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac1077e0 con 0x7f5cac0f66e0 2024-02-15T18:20:23.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.178+0000 7f5cb0d06700 1 -- 172.21.15.42:0/1164224039 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cac059100 con 0x7f5cac0f66e0 2024-02-15T18:20:23.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.178+0000 7f5c93fff700 1 -- 172.21.15.42:0/1164224039 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f5c9c01d980 con 0x7f5cac0f66e0 2024-02-15T18:20:23.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.179+0000 7f5c93fff700 1 --2- 172.21.15.42:0/1164224039 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5c9405c300 0x7f5c9405e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:20:23.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.179+0000 7f5c93fff700 1 -- 172.21.15.42:0/1164224039 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1882+0+0 (secure 0 0 0) 0x7f5c9c077060 con 0x7f5cac0f66e0 2024-02-15T18:20:23.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.179+0000 7f5cab7fe700 1 --2- 172.21.15.42:0/1164224039 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5c9405c300 0x7f5c9405e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:20:23.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.179+0000 7f5cab7fe700 1 --2- 172.21.15.42:0/1164224039 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5c9405c300 0x7f5c9405e7c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f5ca0007fd0 tx=0x7f5ca0009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:20:23.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.184+0000 7f5c93fff700 1 -- 172.21.15.42:0/1164224039 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5c9c04aa70 con 0x7f5cac0f66e0 2024-02-15T18:20:23.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:23.381+0000 7f5cb0d06700 1 -- 172.21.15.42:0/1164224039 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cac0008d0 con 0x7f5c9405c300 2024-02-15T18:20:23.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:21.809750+0000 mgr.smithi042.kfoihb (mgr.14162) 94 : cephadm [INF] Reconfiguring grafana.smithi042 (dependencies changed)... 2024-02-15T18:20:23.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:21.814085+0000 mgr.smithi042.kfoihb (mgr.14162) 95 : cephadm [INF] Reconfiguring daemon grafana.smithi042 on smithi042 2024-02-15T18:20:24.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:22.781190+0000 mgr.smithi042.kfoihb (mgr.14162) 96 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:24.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:23.382768+0000 mgr.smithi042.kfoihb (mgr.14162) 97 : audit [DBG] from='client.14236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:20:24.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:23.383501+0000 mon.smithi042 (mon.0) 233 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:20:24.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:23.384574+0000 mgr.smithi042.kfoihb (mgr.14162) 98 : cephadm [INF] Applying service osd.None on host smithi042... 2024-02-15T18:20:24.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:23.384781+0000 mon.smithi042 (mon.0) 234 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:20:24.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:23.385312+0000 mon.smithi042 (mon.0) 235 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:24.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:22.781190+0000 mgr.smithi042.kfoihb (mgr.14162) 96 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:24.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:23.382768+0000 mgr.smithi042.kfoihb (mgr.14162) 97 : audit [DBG] from='client.14236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:20:24.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:23.383501+0000 mon.smithi042 (mon.0) 233 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:20:24.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:23.384574+0000 mgr.smithi042.kfoihb (mgr.14162) 98 : cephadm [INF] Applying service osd.None on host smithi042... 2024-02-15T18:20:24.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:23.384781+0000 mon.smithi042 (mon.0) 234 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:20:24.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:23.385312+0000 mon.smithi042 (mon.0) 235 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:26.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:24.781460+0000 mgr.smithi042.kfoihb (mgr.14162) 99 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:26.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:24.781460+0000 mgr.smithi042.kfoihb (mgr.14162) 99 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:28.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:26.781785+0000 mgr.smithi042.kfoihb (mgr.14162) 100 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:28.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:27.496836+0000 mon.smithi042 (mon.0) 236 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:28.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:27.497461+0000 mgr.smithi042.kfoihb (mgr.14162) 101 : cephadm [INF] Reconfiguring prometheus.smithi042 (dependencies changed)... 2024-02-15T18:20:28.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:27.499426+0000 mgr.smithi042.kfoihb (mgr.14162) 102 : cephadm [INF] Reconfiguring daemon prometheus.smithi042 on smithi042 2024-02-15T18:20:28.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:26.781785+0000 mgr.smithi042.kfoihb (mgr.14162) 100 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:28.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:27.496836+0000 mon.smithi042 (mon.0) 236 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:28.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:27.497461+0000 mgr.smithi042.kfoihb (mgr.14162) 101 : cephadm [INF] Reconfiguring prometheus.smithi042 (dependencies changed)... 2024-02-15T18:20:28.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:27.499426+0000 mgr.smithi042.kfoihb (mgr.14162) 102 : cephadm [INF] Reconfiguring daemon prometheus.smithi042 on smithi042 2024-02-15T18:20:29.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:28.543626+0000 mon.smithi042 (mon.0) 237 : audit [INF] from='client.? 172.21.15.42:0/213236204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f67d329-7d44-4361-be02-f8b56b3e5713"}]: dispatch 2024-02-15T18:20:29.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:28.547101+0000 mon.smithi042 (mon.0) 238 : audit [INF] from='client.? 172.21.15.42:0/213236204' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f67d329-7d44-4361-be02-f8b56b3e5713"}]': finished 2024-02-15T18:20:29.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:28.547142+0000 mon.smithi042 (mon.0) 239 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-15T18:20:29.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:28.547237+0000 mon.smithi042 (mon.0) 240 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:20:29.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:28.960514+0000 mon.smithi042 (mon.0) 241 : audit [DBG] from='client.? 172.21.15.42:0/950695950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:20:29.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:28.543626+0000 mon.smithi042 (mon.0) 237 : audit [INF] from='client.? 172.21.15.42:0/213236204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f67d329-7d44-4361-be02-f8b56b3e5713"}]: dispatch 2024-02-15T18:20:29.952 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:28.547101+0000 mon.smithi042 (mon.0) 238 : audit [INF] from='client.? 172.21.15.42:0/213236204' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f67d329-7d44-4361-be02-f8b56b3e5713"}]': finished 2024-02-15T18:20:29.952 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:28.547142+0000 mon.smithi042 (mon.0) 239 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-15T18:20:29.952 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:28.547237+0000 mon.smithi042 (mon.0) 240 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:20:29.952 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:28.960514+0000 mon.smithi042 (mon.0) 241 : audit [DBG] from='client.? 172.21.15.42:0/950695950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:20:30.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:28.782040+0000 mgr.smithi042.kfoihb (mgr.14162) 103 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:30.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:28.782040+0000 mgr.smithi042.kfoihb (mgr.14162) 103 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:32.667 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:30.782386+0000 mgr.smithi042.kfoihb (mgr.14162) 104 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:32.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:30.782386+0000 mgr.smithi042.kfoihb (mgr.14162) 104 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:34.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:32.782634+0000 mgr.smithi042.kfoihb (mgr.14162) 105 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:34.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:32.782634+0000 mgr.smithi042.kfoihb (mgr.14162) 105 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:36.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:34.783067+0000 mgr.smithi042.kfoihb (mgr.14162) 106 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:36.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:34.783067+0000 mgr.smithi042.kfoihb (mgr.14162) 106 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:38.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:36.783460+0000 mgr.smithi042.kfoihb (mgr.14162) 107 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:38.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:38.022124+0000 mon.smithi042 (mon.0) 242 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:38.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:38.023118+0000 mon.smithi042 (mon.0) 243 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi191", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:20:38.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:38.023926+0000 mon.smithi042 (mon.0) 244 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:38.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:36.783460+0000 mgr.smithi042.kfoihb (mgr.14162) 107 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:38.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:38.022124+0000 mon.smithi042 (mon.0) 242 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:38.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:38.023118+0000 mon.smithi042 (mon.0) 243 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi191", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:20:38.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:38.023926+0000 mon.smithi042 (mon.0) 244 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:39.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:38.022805+0000 mgr.smithi042.kfoihb (mgr.14162) 108 : cephadm [INF] Reconfiguring crash.smithi191 (monmap changed)... 2024-02-15T18:20:39.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:38.024495+0000 mgr.smithi042.kfoihb (mgr.14162) 109 : cephadm [INF] Reconfiguring daemon crash.smithi191 on smithi191 2024-02-15T18:20:39.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:39.291071+0000 mon.smithi042 (mon.0) 245 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:39.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:39.291845+0000 mon.smithi042 (mon.0) 246 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi191.gftdzo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:20:39.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:39.292583+0000 mon.smithi042 (mon.0) 247 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:20:39.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:39.293150+0000 mon.smithi042 (mon.0) 248 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:39.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:39.424185+0000 mon.smithi042 (mon.0) 249 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-15T18:20:39.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:39.424886+0000 mon.smithi042 (mon.0) 250 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:39.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:38.022805+0000 mgr.smithi042.kfoihb (mgr.14162) 108 : cephadm [INF] Reconfiguring crash.smithi191 (monmap changed)... 2024-02-15T18:20:39.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:38.024495+0000 mgr.smithi042.kfoihb (mgr.14162) 109 : cephadm [INF] Reconfiguring daemon crash.smithi191 on smithi191 2024-02-15T18:20:39.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:39.291071+0000 mon.smithi042 (mon.0) 245 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:39.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:39.291845+0000 mon.smithi042 (mon.0) 246 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi191.gftdzo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:20:39.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:39.292583+0000 mon.smithi042 (mon.0) 247 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:20:39.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:39.293150+0000 mon.smithi042 (mon.0) 248 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:39.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:39.424185+0000 mon.smithi042 (mon.0) 249 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-15T18:20:39.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:39.424886+0000 mon.smithi042 (mon.0) 250 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:40.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:38.783742+0000 mgr.smithi042.kfoihb (mgr.14162) 110 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:40.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:39.291588+0000 mgr.smithi042.kfoihb (mgr.14162) 111 : cephadm [INF] Reconfiguring mgr.smithi191.gftdzo (monmap changed)... 2024-02-15T18:20:40.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:39.293619+0000 mgr.smithi042.kfoihb (mgr.14162) 112 : cephadm [INF] Reconfiguring daemon mgr.smithi191.gftdzo on smithi191 2024-02-15T18:20:40.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:39.425468+0000 mgr.smithi042.kfoihb (mgr.14162) 113 : cephadm [INF] Deploying daemon osd.1 on smithi042 2024-02-15T18:20:40.794 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:38.783742+0000 mgr.smithi042.kfoihb (mgr.14162) 110 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:40.794 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:39.291588+0000 mgr.smithi042.kfoihb (mgr.14162) 111 : cephadm [INF] Reconfiguring mgr.smithi191.gftdzo (monmap changed)... 2024-02-15T18:20:40.794 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:39.293619+0000 mgr.smithi042.kfoihb (mgr.14162) 112 : cephadm [INF] Reconfiguring daemon mgr.smithi191.gftdzo on smithi191 2024-02-15T18:20:40.794 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:39.425468+0000 mgr.smithi042.kfoihb (mgr.14162) 113 : cephadm [INF] Deploying daemon osd.1 on smithi042 2024-02-15T18:20:42.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:40.767391+0000 mon.smithi042 (mon.0) 251 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:42.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:40.767962+0000 mgr.smithi042.kfoihb (mgr.14162) 114 : cephadm [INF] Reconfiguring mon.smithi191 (monmap changed)... 2024-02-15T18:20:42.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:40.768274+0000 mon.smithi042 (mon.0) 252 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:20:42.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:40.769057+0000 mon.smithi042 (mon.0) 253 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:20:42.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:40.769814+0000 mon.smithi042 (mon.0) 254 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:42.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:40.770431+0000 mgr.smithi042.kfoihb (mgr.14162) 115 : cephadm [INF] Reconfiguring daemon mon.smithi191 on smithi191 2024-02-15T18:20:42.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:40.784169+0000 mgr.smithi042.kfoihb (mgr.14162) 116 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:42.077 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:40.767391+0000 mon.smithi042 (mon.0) 251 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:42.077 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:40.767962+0000 mgr.smithi042.kfoihb (mgr.14162) 114 : cephadm [INF] Reconfiguring mon.smithi191 (monmap changed)... 2024-02-15T18:20:42.077 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:40.768274+0000 mon.smithi042 (mon.0) 252 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:20:42.077 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:40.769057+0000 mon.smithi042 (mon.0) 253 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:20:42.077 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:40.769814+0000 mon.smithi042 (mon.0) 254 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:42.077 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:40.770431+0000 mgr.smithi042.kfoihb (mgr.14162) 115 : cephadm [INF] Reconfiguring daemon mon.smithi191 on smithi191 2024-02-15T18:20:42.077 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:40.784169+0000 mgr.smithi042.kfoihb (mgr.14162) 116 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:43.042 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 1 on host 'smithi042' 2024-02-15T18:20:43.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:43.040+0000 7f5c93fff700 1 -- 172.21.15.42:0/1164224039 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5cac0008d0 con 0x7f5c9405c300 2024-02-15T18:20:43.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:43.043+0000 7f5cb0d06700 1 -- 172.21.15.42:0/1164224039 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5c9405c300 msgr2=0x7f5c9405e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:20:43.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:43.043+0000 7f5cb0d06700 1 --2- 172.21.15.42:0/1164224039 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5c9405c300 0x7f5c9405e7c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f5ca0007fd0 tx=0x7f5ca0009040).stop 2024-02-15T18:20:43.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:43.043+0000 7f5cb0d06700 1 -- 172.21.15.42:0/1164224039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cac0f66e0 msgr2=0x7f5cac106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:20:43.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:43.043+0000 7f5cb0d06700 1 --2- 172.21.15.42:0/1164224039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cac0f66e0 0x7f5cac106250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f5c9c009830 tx=0x7f5c9c009b30).stop 2024-02-15T18:20:43.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:43.043+0000 7f5cb0d06700 1 -- 172.21.15.42:0/1164224039 shutdown_connections 2024-02-15T18:20:43.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:43.043+0000 7f5cb0d06700 1 --2- 172.21.15.42:0/1164224039 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5c9405c300 0x7f5c9405e7c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:20:43.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:43.043+0000 7f5cb0d06700 1 --2- 172.21.15.42:0/1164224039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cac0f5440 0x7f5cac105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:20:43.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:43.043+0000 7f5cb0d06700 1 --2- 172.21.15.42:0/1164224039 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cac0f66e0 0x7f5cac106250 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:20:43.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:43.043+0000 7f5cb0d06700 1 -- 172.21.15.42:0/1164224039 >> 172.21.15.42:0/1164224039 conn(0x7f5cac0f0ba0 msgr2=0x7f5cac0fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:20:43.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:43.043+0000 7f5cb0d06700 1 -- 172.21.15.42:0/1164224039 shutdown_connections 2024-02-15T18:20:43.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:43.043+0000 7f5cb0d06700 1 -- 172.21.15.42:0/1164224039 wait complete. 2024-02-15T18:20:43.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:42.117165+0000 mon.smithi042 (mon.0) 255 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:43.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:42.119389+0000 mon.smithi042 (mon.0) 256 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:20:43.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:42.119684+0000 mgr.smithi042.kfoihb (mgr.14162) 117 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:20:43.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.037053+0000 mon.smithi042 (mon.0) 257 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:43.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.040565+0000 mon.smithi042 (mon.0) 258 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:43.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.052077+0000 mon.smithi042 (mon.0) 259 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.42:9093"}]: dispatch 2024-02-15T18:20:43.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.056864+0000 mon.smithi042 (mon.0) 260 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:43.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.060272+0000 mon.smithi042 (mon.0) 261 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:20:43.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.065115+0000 mon.smithi042 (mon.0) 262 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.42:3000"}]: dispatch 2024-02-15T18:20:43.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.069261+0000 mon.smithi042 (mon.0) 263 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:43.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.072573+0000 mon.smithi042 (mon.0) 264 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:20:43.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.076664+0000 mon.smithi042 (mon.0) 265 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.42:9095"}]: dispatch 2024-02-15T18:20:43.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.080408+0000 mon.smithi042 (mon.0) 266 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:43.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.082472+0000 mon.smithi042 (mon.0) 267 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:20:43.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:42.117165+0000 mon.smithi042 (mon.0) 255 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:43.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:42.119389+0000 mon.smithi042 (mon.0) 256 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:20:43.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:42.119684+0000 mgr.smithi042.kfoihb (mgr.14162) 117 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:20:43.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.037053+0000 mon.smithi042 (mon.0) 257 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:43.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.040565+0000 mon.smithi042 (mon.0) 258 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:43.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.052077+0000 mon.smithi042 (mon.0) 259 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.42:9093"}]: dispatch 2024-02-15T18:20:43.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.056864+0000 mon.smithi042 (mon.0) 260 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:43.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.060272+0000 mon.smithi042 (mon.0) 261 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:20:43.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.065115+0000 mon.smithi042 (mon.0) 262 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.42:3000"}]: dispatch 2024-02-15T18:20:43.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.069261+0000 mon.smithi042 (mon.0) 263 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:43.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.072573+0000 mon.smithi042 (mon.0) 264 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:20:43.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.076664+0000 mon.smithi042 (mon.0) 265 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.42:9095"}]: dispatch 2024-02-15T18:20:43.451 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.080408+0000 mon.smithi042 (mon.0) 266 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:43.451 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.082472+0000 mon.smithi042 (mon.0) 267 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:20:43.653 DEBUG:teuthology.orchestra.run.smithi042:osd.1> sudo journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.1.service 2024-02-15T18:20:43.655 INFO:tasks.cephadm:Deploying osd.2 on smithi042 with /dev/vg_nvme/lv_2... 2024-02-15T18:20:43.655 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-15T18:20:43.687 INFO:journalctl@ceph.osd.1.smithi042.stdout:-- Logs begin at Thu 2024-02-15 17:54:32 UTC. -- 2024-02-15T18:20:44.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:42.784480+0000 mgr.smithi042.kfoihb (mgr.14162) 118 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:44.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.052374+0000 mgr.smithi042.kfoihb (mgr.14162) 119 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.42:9093"}]: dispatch 2024-02-15T18:20:44.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.060726+0000 mgr.smithi042.kfoihb (mgr.14162) 120 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:20:44.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.065424+0000 mgr.smithi042.kfoihb (mgr.14162) 121 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.42:3000"}]: dispatch 2024-02-15T18:20:44.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.072894+0000 mgr.smithi042.kfoihb (mgr.14162) 122 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:20:44.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:43.076928+0000 mgr.smithi042.kfoihb (mgr.14162) 123 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.42:9095"}]: dispatch 2024-02-15T18:20:44.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:42.784480+0000 mgr.smithi042.kfoihb (mgr.14162) 118 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:44.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.052374+0000 mgr.smithi042.kfoihb (mgr.14162) 119 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.42:9093"}]: dispatch 2024-02-15T18:20:44.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.060726+0000 mgr.smithi042.kfoihb (mgr.14162) 120 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:20:44.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.065424+0000 mgr.smithi042.kfoihb (mgr.14162) 121 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.42:3000"}]: dispatch 2024-02-15T18:20:44.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.072894+0000 mgr.smithi042.kfoihb (mgr.14162) 122 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:20:44.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:43.076928+0000 mgr.smithi042.kfoihb (mgr.14162) 123 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.42:9095"}]: dispatch 2024-02-15T18:20:46.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:44.784769+0000 mgr.smithi042.kfoihb (mgr.14162) 124 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:46.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:44.922725+0000 mon.smithi042 (mon.0) 268 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:46.291 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:44.784769+0000 mgr.smithi042.kfoihb (mgr.14162) 124 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:46.292 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:44.922725+0000 mon.smithi042 (mon.0) 268 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:46.370 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-15T18:20:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-15T18:20:46.371 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-02-15T18:20:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-02-15T18:20:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0102667 s, 1.0 GB/s 2024-02-15T18:20:46.371 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 2024-02-15T18:20:46.371 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-02-15T18:20:47.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:46 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:46.916696+0000 mon.smithi042 (mon.0) 269 : audit [INF] from='osd.1 [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-15T18:20:47.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:46.916696+0000 mon.smithi042 (mon.0) 269 : audit [INF] from='osd.1 [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-15T18:20:47.313 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:20:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.1[52563]: debug 2024-02-15T18:20:46.911+0000 7f06228a8080 -1 osd.1 0 log_to_monitors {default=true} 2024-02-15T18:20:47.796 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi042:vg_nvme/lv_2 2024-02-15T18:20:48.063 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:20:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.1[52563]: debug 2024-02-15T18:20:47.933+0000 7f0618e19700 -1 osd.1 0 waiting for initial osdmap 2024-02-15T18:20:48.063 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:20:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.1[52563]: debug 2024-02-15T18:20:47.937+0000 7f0614d94700 -1 osd.1 13 set_numa_affinity unable to identify public interface 'ens1f0' numa node: (0) Success 2024-02-15T18:20:48.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:46.785051+0000 mgr.smithi042.kfoihb (mgr.14162) 125 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:48.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:46.929531+0000 mon.smithi042 (mon.0) 270 : audit [INF] from='osd.1 [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-15T18:20:48.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:46.929574+0000 mon.smithi042 (mon.0) 271 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-15T18:20:48.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:46.929684+0000 mon.smithi042 (mon.0) 272 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:20:48.064 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:46.929984+0000 mon.smithi042 (mon.0) 273 : audit [INF] from='osd.1 [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-02-15T18:20:48.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:46.785051+0000 mgr.smithi042.kfoihb (mgr.14162) 125 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:48.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:46.929531+0000 mon.smithi042 (mon.0) 270 : audit [INF] from='osd.1 [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-15T18:20:48.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:46.929574+0000 mon.smithi042 (mon.0) 271 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-15T18:20:48.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:46.929684+0000 mon.smithi042 (mon.0) 272 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:20:48.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:46.929984+0000 mon.smithi042 (mon.0) 273 : audit [INF] from='osd.1 [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-02-15T18:20:49.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:47.931049+0000 mon.smithi042 (mon.0) 274 : audit [INF] from='osd.1 [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-02-15T18:20:49.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:47.931123+0000 mon.smithi042 (mon.0) 275 : cluster [DBG] osdmap e13: 2 total, 1 up, 2 in 2024-02-15T18:20:49.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:47.931435+0000 mon.smithi042 (mon.0) 276 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:20:49.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:47.934983+0000 mon.smithi042 (mon.0) 277 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:20:49.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:47.931049+0000 mon.smithi042 (mon.0) 274 : audit [INF] from='osd.1 [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-02-15T18:20:49.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:47.931123+0000 mon.smithi042 (mon.0) 275 : cluster [DBG] osdmap e13: 2 total, 1 up, 2 in 2024-02-15T18:20:49.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:47.931435+0000 mon.smithi042 (mon.0) 276 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:20:49.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:47.934983+0000 mon.smithi042 (mon.0) 277 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:20:50.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:47.951750+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:20:50.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:47.951850+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:20:50.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:48.785308+0000 mgr.smithi042.kfoihb (mgr.14162) 126 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:50.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:48.935880+0000 mon.smithi042 (mon.0) 278 : cluster [INF] osd.1 [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965] boot 2024-02-15T18:20:50.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:48.935911+0000 mon.smithi042 (mon.0) 279 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-15T18:20:50.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:48.936304+0000 mon.smithi042 (mon.0) 280 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:20:50.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:47.951750+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:20:50.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:47.951850+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:20:50.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:48.785308+0000 mgr.smithi042.kfoihb (mgr.14162) 126 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-02-15T18:20:50.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:48.935880+0000 mon.smithi042 (mon.0) 278 : cluster [INF] osd.1 [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965] boot 2024-02-15T18:20:50.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:48.935911+0000 mon.smithi042 (mon.0) 279 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-15T18:20:50.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:48.936304+0000 mon.smithi042 (mon.0) 280 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:20:50.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.583+0000 7f7fb632d700 1 -- 172.21.15.42:0/622992350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fb00f66e0 msgr2=0x7f7fb00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:20:50.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.583+0000 7f7fb632d700 1 --2- 172.21.15.42:0/622992350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fb00f66e0 0x7f7fb00f6b00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f7fa80099e0 tx=0x7f7fa8004d10).stop 2024-02-15T18:20:50.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.583+0000 7f7fb632d700 1 -- 172.21.15.42:0/622992350 shutdown_connections 2024-02-15T18:20:50.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.583+0000 7f7fb632d700 1 --2- 172.21.15.42:0/622992350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fb00f7980 0x7f7fb00f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:20:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.583+0000 7f7fb632d700 1 --2- 172.21.15.42:0/622992350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fb00f66e0 0x7f7fb00f6b00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:20:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.583+0000 7f7fb632d700 1 -- 172.21.15.42:0/622992350 >> 172.21.15.42:0/622992350 conn(0x7f7fb00f1e40 msgr2=0x7f7fb00f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:20:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.583+0000 7f7fb632d700 1 -- 172.21.15.42:0/622992350 shutdown_connections 2024-02-15T18:20:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.583+0000 7f7fb632d700 1 -- 172.21.15.42:0/622992350 wait complete. 2024-02-15T18:20:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.584+0000 7f7fb632d700 1 Processor -- start 2024-02-15T18:20:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.584+0000 7f7fb632d700 1 -- start start 2024-02-15T18:20:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.584+0000 7f7fb632d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fb00f66e0 0x7f7fb0107220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:20:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.584+0000 7f7fb632d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fb00f7980 0x7f7fb0107710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:20:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.584+0000 7f7fb632d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb0107d50 con 0x7f7fb00f7980 2024-02-15T18:20:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.584+0000 7f7fb632d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb0108600 con 0x7f7fb00f66e0 2024-02-15T18:20:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.584+0000 7f7faf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fb00f7980 0x7f7fb0107710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:20:50.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.584+0000 7f7faf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fb00f7980 0x7f7fb0107710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48052/0 (socket says 172.21.15.42:48052) 2024-02-15T18:20:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.584+0000 7f7faf7fe700 1 -- 172.21.15.42:0/1269561394 learned_addr learned my addr 172.21.15.42:0/1269561394 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:20:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.584+0000 7f7faffff700 1 --2- 172.21.15.42:0/1269561394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fb00f66e0 0x7f7fb0107220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:20:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.585+0000 7f7faf7fe700 1 -- 172.21.15.42:0/1269561394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fb00f66e0 msgr2=0x7f7fb0107220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:20:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.585+0000 7f7faf7fe700 1 --2- 172.21.15.42:0/1269561394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fb00f66e0 0x7f7fb0107220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:20:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.585+0000 7f7faf7fe700 1 -- 172.21.15.42:0/1269561394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa8009640 con 0x7f7fb00f7980 2024-02-15T18:20:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.585+0000 7f7faf7fe700 1 --2- 172.21.15.42:0/1269561394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fb00f7980 0x7f7fb0107710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f7fa000b410 tx=0x7f7fa0006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:20:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.585+0000 7f7facff9700 1 -- 172.21.15.42:0/1269561394 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa00125f0 con 0x7f7fb00f7980 2024-02-15T18:20:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.585+0000 7f7facff9700 1 -- 172.21.15.42:0/1269561394 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fa000be30 con 0x7f7fb00f7980 2024-02-15T18:20:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.585+0000 7f7fb632d700 1 -- 172.21.15.42:0/1269561394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb01088e0 con 0x7f7fb00f7980 2024-02-15T18:20:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.585+0000 7f7facff9700 1 -- 172.21.15.42:0/1269561394 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa0002a90 con 0x7f7fb00f7980 2024-02-15T18:20:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.585+0000 7f7fb632d700 1 -- 172.21.15.42:0/1269561394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb0108e20 con 0x7f7fb00f7980 2024-02-15T18:20:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.586+0000 7f7fb632d700 1 -- 172.21.15.42:0/1269561394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb00f5ad0 con 0x7f7fb00f7980 2024-02-15T18:20:50.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.586+0000 7f7facff9700 1 -- 172.21.15.42:0/1269561394 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f7fa000f030 con 0x7f7fb00f7980 2024-02-15T18:20:50.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.587+0000 7f7facff9700 1 --2- 172.21.15.42:0/1269561394 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7f9c05c2b0 0x7f7f9c05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:20:50.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.587+0000 7f7facff9700 1 -- 172.21.15.42:0/1269561394 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2314+0+0 (secure 0 0 0) 0x7f7fa00756b0 con 0x7f7fb00f7980 2024-02-15T18:20:50.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.587+0000 7f7faffff700 1 --2- 172.21.15.42:0/1269561394 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7f9c05c2b0 0x7f7f9c05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:20:50.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.587+0000 7f7faffff700 1 --2- 172.21.15.42:0/1269561394 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7f9c05c2b0 0x7f7f9c05e770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7fa8003380 tx=0x7f7fa8004d10).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:20:50.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.591+0000 7f7facff9700 1 -- 172.21.15.42:0/1269561394 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7fa0048f70 con 0x7f7fb00f7980 2024-02-15T18:20:50.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:20:50.783+0000 7f7fb632d700 1 -- 172.21.15.42:0/1269561394 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fb00ff100 con 0x7f7f9c05c2b0 2024-02-15T18:20:51.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:50.245949+0000 mon.smithi042 (mon.0) 281 : cluster [DBG] osdmap e15: 2 total, 2 up, 2 in 2024-02-15T18:20:51.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:50.785734+0000 mon.smithi042 (mon.0) 282 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:20:51.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:50.786970+0000 mon.smithi042 (mon.0) 283 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:20:51.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:50.787502+0000 mon.smithi042 (mon.0) 284 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:51.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:50.245949+0000 mon.smithi042 (mon.0) 281 : cluster [DBG] osdmap e15: 2 total, 2 up, 2 in 2024-02-15T18:20:51.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:50.785734+0000 mon.smithi042 (mon.0) 282 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:20:51.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:50.786970+0000 mon.smithi042 (mon.0) 283 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:20:51.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:50.787502+0000 mon.smithi042 (mon.0) 284 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:20:52.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:50.785011+0000 mgr.smithi042.kfoihb (mgr.14162) 127 : audit [DBG] from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:20:52.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:50.785561+0000 mgr.smithi042.kfoihb (mgr.14162) 128 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:20:52.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:20:50.786788+0000 mgr.smithi042.kfoihb (mgr.14162) 129 : cephadm [INF] Applying service osd.None on host smithi042... 2024-02-15T18:20:52.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:51.426147+0000 mon.smithi042 (mon.0) 285 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:52.742 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:50.785011+0000 mgr.smithi042.kfoihb (mgr.14162) 127 : audit [DBG] from='client.14254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:20:52.742 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:50.785561+0000 mgr.smithi042.kfoihb (mgr.14162) 128 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:20:52.742 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:20:50.786788+0000 mgr.smithi042.kfoihb (mgr.14162) 129 : cephadm [INF] Applying service osd.None on host smithi042... 2024-02-15T18:20:52.742 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:51.426147+0000 mon.smithi042 (mon.0) 285 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:20:54.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:52.785831+0000 mgr.smithi042.kfoihb (mgr.14162) 130 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:20:54.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:53.573547+0000 mon.smithi042 (mon.0) 286 : audit [INF] from='client.? 172.21.15.42:0/3296479781' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "680f6437-b450-4353-a761-b50d1388bcd8"}]: dispatch 2024-02-15T18:20:54.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:53.577279+0000 mon.smithi042 (mon.0) 287 : audit [INF] from='client.? 172.21.15.42:0/3296479781' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "680f6437-b450-4353-a761-b50d1388bcd8"}]': finished 2024-02-15T18:20:54.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:53.577315+0000 mon.smithi042 (mon.0) 288 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-15T18:20:54.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:53.577396+0000 mon.smithi042 (mon.0) 289 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:20:54.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:53.995551+0000 mon.smithi042 (mon.0) 290 : audit [DBG] from='client.? 172.21.15.42:0/3999431949' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:20:54.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:52.785831+0000 mgr.smithi042.kfoihb (mgr.14162) 130 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:20:54.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:53.573547+0000 mon.smithi042 (mon.0) 286 : audit [INF] from='client.? 172.21.15.42:0/3296479781' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "680f6437-b450-4353-a761-b50d1388bcd8"}]: dispatch 2024-02-15T18:20:54.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:53.577279+0000 mon.smithi042 (mon.0) 287 : audit [INF] from='client.? 172.21.15.42:0/3296479781' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "680f6437-b450-4353-a761-b50d1388bcd8"}]': finished 2024-02-15T18:20:54.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:53.577315+0000 mon.smithi042 (mon.0) 288 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-15T18:20:54.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:53.577396+0000 mon.smithi042 (mon.0) 289 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:20:54.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:53.995551+0000 mon.smithi042 (mon.0) 290 : audit [DBG] from='client.? 172.21.15.42:0/3999431949' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:20:56.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:54.786197+0000 mgr.smithi042.kfoihb (mgr.14162) 131 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:20:56.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:54.786197+0000 mgr.smithi042.kfoihb (mgr.14162) 131 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:20:57.649 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:56.907296+0000 mon.smithi042 (mon.0) 291 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:20:57.649 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:20:56.910971+0000 mon.smithi042 (mon.0) 292 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:20:57.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:56.907296+0000 mon.smithi042 (mon.0) 291 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:20:57.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:20:56.910971+0000 mon.smithi042 (mon.0) 292 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:20:58.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:20:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:56.786512+0000 mgr.smithi042.kfoihb (mgr.14162) 132 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:20:58.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:20:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:56.786512+0000 mgr.smithi042.kfoihb (mgr.14162) 132 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:00.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:20:58.786913+0000 mgr.smithi042.kfoihb (mgr.14162) 133 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:00.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:00.215309+0000 mon.smithi042 (mon.0) 293 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:00.722 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:20:58.786913+0000 mgr.smithi042.kfoihb (mgr.14162) 133 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:00.722 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:00.215309+0000 mon.smithi042 (mon.0) 293 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:01.720 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:21:00.210560+0000 mgr.smithi042.kfoihb (mgr.14162) 134 : cephadm [INF] Detected new or changed devices on smithi042 2024-02-15T18:21:01.720 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:00.547175+0000 mon.smithi042 (mon.0) 294 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:01.720 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:00.550910+0000 mon.smithi042 (mon.0) 295 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:21:01.720 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:01.407979+0000 mon.smithi042 (mon.0) 296 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-15T18:21:01.721 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:01.408656+0000 mon.smithi042 (mon.0) 297 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:21:01.907 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:21:00.210560+0000 mgr.smithi042.kfoihb (mgr.14162) 134 : cephadm [INF] Detected new or changed devices on smithi042 2024-02-15T18:21:01.907 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:00.547175+0000 mon.smithi042 (mon.0) 294 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:01.908 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:00.550910+0000 mon.smithi042 (mon.0) 295 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:21:01.908 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:01.407979+0000 mon.smithi042 (mon.0) 296 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-15T18:21:01.908 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:01.408656+0000 mon.smithi042 (mon.0) 297 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:21:02.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:00.787363+0000 mgr.smithi042.kfoihb (mgr.14162) 135 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:02.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:21:01.409264+0000 mgr.smithi042.kfoihb (mgr.14162) 136 : cephadm [INF] Deploying daemon osd.2 on smithi042 2024-02-15T18:21:02.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:02.338431+0000 mon.smithi042 (mon.0) 298 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:02.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:00.787363+0000 mgr.smithi042.kfoihb (mgr.14162) 135 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:02.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:21:01.409264+0000 mgr.smithi042.kfoihb (mgr.14162) 136 : cephadm [INF] Deploying daemon osd.2 on smithi042 2024-02-15T18:21:02.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:02.338431+0000 mon.smithi042 (mon.0) 298 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:03.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:02.787759+0000 mgr.smithi042.kfoihb (mgr.14162) 137 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:03.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:02.787759+0000 mgr.smithi042.kfoihb (mgr.14162) 137 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:06.532 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:04.788117+0000 mgr.smithi042.kfoihb (mgr.14162) 138 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:06.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:04.788117+0000 mgr.smithi042.kfoihb (mgr.14162) 138 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:07.074 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 2 on host 'smithi042' 2024-02-15T18:21:07.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:21:07.073+0000 7f7facff9700 1 -- 172.21.15.42:0/1269561394 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7fb00ff100 con 0x7f7f9c05c2b0 2024-02-15T18:21:07.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:21:07.075+0000 7f7fb632d700 1 -- 172.21.15.42:0/1269561394 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7f9c05c2b0 msgr2=0x7f7f9c05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:21:07.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:21:07.075+0000 7f7fb632d700 1 --2- 172.21.15.42:0/1269561394 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7f9c05c2b0 0x7f7f9c05e770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7fa8003380 tx=0x7f7fa8004d10).stop 2024-02-15T18:21:07.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:21:07.075+0000 7f7fb632d700 1 -- 172.21.15.42:0/1269561394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fb00f7980 msgr2=0x7f7fb0107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:21:07.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:21:07.075+0000 7f7fb632d700 1 --2- 172.21.15.42:0/1269561394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fb00f7980 0x7f7fb0107710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f7fa000b410 tx=0x7f7fa0006ef0).stop 2024-02-15T18:21:07.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:21:07.076+0000 7f7fb632d700 1 -- 172.21.15.42:0/1269561394 shutdown_connections 2024-02-15T18:21:07.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:21:07.076+0000 7f7fb632d700 1 --2- 172.21.15.42:0/1269561394 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7f9c05c2b0 0x7f7f9c05e770 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:07.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:21:07.076+0000 7f7fb632d700 1 --2- 172.21.15.42:0/1269561394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fb00f66e0 0x7f7fb0107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:07.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:21:07.076+0000 7f7fb632d700 1 --2- 172.21.15.42:0/1269561394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fb00f7980 0x7f7fb0107710 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:07.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:21:07.076+0000 7f7fb632d700 1 -- 172.21.15.42:0/1269561394 >> 172.21.15.42:0/1269561394 conn(0x7f7fb00f1e40 msgr2=0x7f7fb00fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:21:07.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:21:07.076+0000 7f7fb632d700 1 -- 172.21.15.42:0/1269561394 shutdown_connections 2024-02-15T18:21:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:21:07.076+0000 7f7fb632d700 1 -- 172.21.15.42:0/1269561394 wait complete. 2024-02-15T18:21:08.097 DEBUG:teuthology.orchestra.run.smithi042:osd.2> sudo journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.2.service 2024-02-15T18:21:08.099 INFO:tasks.cephadm:Deploying osd.3 on smithi191 with /dev/vg_nvme/lv_4... 2024-02-15T18:21:08.100 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-15T18:21:08.130 INFO:journalctl@ceph.osd.2.smithi042.stdout:-- Logs begin at Thu 2024-02-15 17:54:32 UTC. -- 2024-02-15T18:21:08.380 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:06.788451+0000 mgr.smithi042.kfoihb (mgr.14162) 139 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:08.380 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:07.070077+0000 mon.smithi042 (mon.0) 299 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:08.380 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:07.073968+0000 mon.smithi042 (mon.0) 300 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:08.380 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:07.638715+0000 mon.smithi042 (mon.0) 301 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:08.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:06.788451+0000 mgr.smithi042.kfoihb (mgr.14162) 139 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:08.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:07.070077+0000 mon.smithi042 (mon.0) 299 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:08.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:07.073968+0000 mon.smithi042 (mon.0) 300 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:08.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:07.638715+0000 mon.smithi042 (mon.0) 301 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:10.240 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-15T18:21:10.240 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-15T18:21:10.240 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-02-15T18:21:10.240 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-02-15T18:21:10.240 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110808 s, 946 MB/s 2024-02-15T18:21:10.240 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-02-15T18:21:10.537 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:08.788743+0000 mgr.smithi042.kfoihb (mgr.14162) 140 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:10.579 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:08.788743+0000 mgr.smithi042.kfoihb (mgr.14162) 140 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:10.908 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_4 2024-02-15T18:21:11.214 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:21:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.2[56906]: debug 2024-02-15T18:21:10.876+0000 7f4a89692080 -1 osd.2 0 log_to_monitors {default=true} 2024-02-15T18:21:11.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:10.880545+0000 mon.smithi042 (mon.0) 302 : audit [INF] from='osd.2 [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-15T18:21:11.608 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:10.880545+0000 mon.smithi042 (mon.0) 302 : audit [INF] from='osd.2 [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-15T18:21:12.516 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:10.789192+0000 mgr.smithi042.kfoihb (mgr.14162) 141 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:12.517 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:11.255764+0000 mon.smithi042 (mon.0) 303 : audit [INF] from='osd.2 [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-15T18:21:12.517 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:11.255840+0000 mon.smithi042 (mon.0) 304 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-15T18:21:12.517 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:11.255969+0000 mon.smithi042 (mon.0) 305 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:21:12.517 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:11.256489+0000 mon.smithi042 (mon.0) 306 : audit [INF] from='osd.2 [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-02-15T18:21:12.517 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:11.454755+0000 mon.smithi042 (mon.0) 307 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:12.517 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:11.457376+0000 mon.smithi042 (mon.0) 308 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:12.517 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:11.459642+0000 mon.smithi042 (mon.0) 309 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:21:12.563 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:21:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.2[56906]: debug 2024-02-15T18:21:12.258+0000 7f4a7fc03700 -1 osd.2 0 waiting for initial osdmap 2024-02-15T18:21:12.563 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:21:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.2[56906]: debug 2024-02-15T18:21:12.264+0000 7f4a7a37b700 -1 osd.2 18 set_numa_affinity unable to identify public interface 'ens1f0' numa node: (0) Success 2024-02-15T18:21:12.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:10.789192+0000 mgr.smithi042.kfoihb (mgr.14162) 141 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:12.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:11.255764+0000 mon.smithi042 (mon.0) 303 : audit [INF] from='osd.2 [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-15T18:21:12.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:11.255840+0000 mon.smithi042 (mon.0) 304 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-15T18:21:12.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:11.255969+0000 mon.smithi042 (mon.0) 305 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:21:12.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:11.256489+0000 mon.smithi042 (mon.0) 306 : audit [INF] from='osd.2 [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-02-15T18:21:12.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:11.454755+0000 mon.smithi042 (mon.0) 307 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:12.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:11.457376+0000 mon.smithi042 (mon.0) 308 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:12.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:11.459642+0000 mon.smithi042 (mon.0) 309 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:21:12.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.802+0000 7f9bb19ef700 1 -- 172.21.15.191:0/4220743049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bac0f5430 msgr2=0x7f9bac0f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:21:12.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.802+0000 7f9bb19ef700 1 --2- 172.21.15.191:0/4220743049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bac0f5430 0x7f9bac0f5850 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f9ba0004660 tx=0x7f9ba0009b30).stop 2024-02-15T18:21:12.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.802+0000 7f9bb19ef700 1 -- 172.21.15.191:0/4220743049 shutdown_connections 2024-02-15T18:21:12.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.802+0000 7f9bb19ef700 1 --2- 172.21.15.191:0/4220743049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bac0f66d0 0x7f9bac0f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:12.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.802+0000 7f9bb19ef700 1 --2- 172.21.15.191:0/4220743049 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bac0f5430 0x7f9bac0f5850 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:12.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.802+0000 7f9bb19ef700 1 -- 172.21.15.191:0/4220743049 >> 172.21.15.191:0/4220743049 conn(0x7f9bac0f0b90 msgr2=0x7f9bac0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:21:12.824 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.802+0000 7f9bb19ef700 1 -- 172.21.15.191:0/4220743049 shutdown_connections 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.802+0000 7f9bb19ef700 1 -- 172.21.15.191:0/4220743049 wait complete. 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.802+0000 7f9bb19ef700 1 Processor -- start 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.802+0000 7f9bb19ef700 1 -- start start 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.802+0000 7f9bb19ef700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bac0f5430 0x7f9bac109b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.802+0000 7f9bb19ef700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bac0f66d0 0x7f9bac10c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.802+0000 7f9bb19ef700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0603e0 con 0x7f9bac0f5430 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.803+0000 7f9bb19ef700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac060520 con 0x7f9bac0f66d0 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.803+0000 7f9babfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bac0f66d0 0x7f9bac10c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.803+0000 7f9babfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bac0f66d0 0x7f9bac10c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:49896/0 (socket says 172.21.15.191:49896) 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.803+0000 7f9babfff700 1 -- 172.21.15.191:0/376402237 learned_addr learned my addr 172.21.15.191:0/376402237 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.803+0000 7f9bb09ed700 1 --2- 172.21.15.191:0/376402237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bac0f5430 0x7f9bac109b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.803+0000 7f9babfff700 1 -- 172.21.15.191:0/376402237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bac0f5430 msgr2=0x7f9bac109b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.803+0000 7f9babfff700 1 --2- 172.21.15.191:0/376402237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bac0f5430 0x7f9bac109b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.803+0000 7f9babfff700 1 -- 172.21.15.191:0/376402237 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba0005040 con 0x7f9bac0f66d0 2024-02-15T18:21:12.825 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.803+0000 7f9babfff700 1 --2- 172.21.15.191:0/376402237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bac0f66d0 0x7f9bac10c070 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f9b9800c530 tx=0x7f9b98007e20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:21:12.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.803+0000 7f9ba97fa700 1 -- 172.21.15.191:0/376402237 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b9800f650 con 0x7f9bac0f66d0 2024-02-15T18:21:12.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.803+0000 7f9ba97fa700 1 -- 172.21.15.191:0/376402237 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b9800fc80 con 0x7f9bac0f66d0 2024-02-15T18:21:12.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.803+0000 7f9bb19ef700 1 -- 172.21.15.191:0/376402237 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bac10c620 con 0x7f9bac0f66d0 2024-02-15T18:21:12.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.803+0000 7f9ba97fa700 1 -- 172.21.15.191:0/376402237 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b98012d30 con 0x7f9bac0f66d0 2024-02-15T18:21:12.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.803+0000 7f9bb19ef700 1 -- 172.21.15.191:0/376402237 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bac10cbe0 con 0x7f9bac0f66d0 2024-02-15T18:21:12.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.804+0000 7f9bb19ef700 1 -- 172.21.15.191:0/376402237 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bac045ad0 con 0x7f9bac0f66d0 2024-02-15T18:21:12.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.804+0000 7f9ba97fa700 1 -- 172.21.15.191:0/376402237 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f9b98010070 con 0x7f9bac0f66d0 2024-02-15T18:21:12.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.805+0000 7f9ba97fa700 1 --2- 172.21.15.191:0/376402237 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9b9c05c300 0x7f9b9c05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:21:12.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.805+0000 7f9ba97fa700 1 -- 172.21.15.191:0/376402237 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2466+0+0 (secure 0 0 0) 0x7f9b98074f70 con 0x7f9bac0f66d0 2024-02-15T18:21:12.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.805+0000 7f9bb09ed700 1 --2- 172.21.15.191:0/376402237 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9b9c05c300 0x7f9b9c05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:21:12.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.805+0000 7f9bb09ed700 1 --2- 172.21.15.191:0/376402237 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9b9c05c300 0x7f9b9c05e7c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f9bac060060 tx=0x7f9ba0009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:21:12.826 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.809+0000 7f9ba97fa700 1 -- 172.21.15.191:0/376402237 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9b980452b0 con 0x7f9bac0f66d0 2024-02-15T18:21:12.994 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:12.994+0000 7f9bb19ef700 1 -- 172.21.15.191:0/376402237 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bac057f10 con 0x7f9b9c05c300 2024-02-15T18:21:13.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:12.256560+0000 mon.smithi042 (mon.0) 310 : audit [INF] from='osd.2 [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-02-15T18:21:13.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:12.256641+0000 mon.smithi042 (mon.0) 311 : cluster [DBG] osdmap e18: 3 total, 2 up, 3 in 2024-02-15T18:21:13.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:12.256921+0000 mon.smithi042 (mon.0) 312 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:21:13.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:12.259906+0000 mon.smithi042 (mon.0) 313 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:21:13.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:12.996450+0000 mon.smithi042 (mon.0) 314 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:21:13.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:12.997957+0000 mon.smithi042 (mon.0) 315 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:21:13.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:12.998496+0000 mon.smithi042 (mon.0) 316 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:21:13.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:13.259615+0000 mon.smithi042 (mon.0) 317 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:21:13.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:12.256560+0000 mon.smithi042 (mon.0) 310 : audit [INF] from='osd.2 [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-02-15T18:21:13.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:12.256641+0000 mon.smithi042 (mon.0) 311 : cluster [DBG] osdmap e18: 3 total, 2 up, 3 in 2024-02-15T18:21:13.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:12.256921+0000 mon.smithi042 (mon.0) 312 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:21:13.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:12.259906+0000 mon.smithi042 (mon.0) 313 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:21:13.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:12.996450+0000 mon.smithi042 (mon.0) 314 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:21:13.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:12.997957+0000 mon.smithi042 (mon.0) 315 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:21:13.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:12.998496+0000 mon.smithi042 (mon.0) 316 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:21:13.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:13.259615+0000 mon.smithi042 (mon.0) 317 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:21:14.421 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:11.894728+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:21:14.421 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:11.894828+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:21:14.421 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:12.789446+0000 mgr.smithi042.kfoihb (mgr.14162) 142 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:14.422 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:12.995675+0000 mgr.smithi042.kfoihb (mgr.14162) 143 : audit [DBG] from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:21:14.422 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:21:12.997723+0000 mgr.smithi042.kfoihb (mgr.14162) 144 : cephadm [INF] Applying service osd.None on host smithi191... 2024-02-15T18:21:14.422 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:13.264181+0000 mon.smithi042 (mon.0) 318 : cluster [INF] osd.2 [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670] boot 2024-02-15T18:21:14.422 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:13.264206+0000 mon.smithi042 (mon.0) 319 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-15T18:21:14.422 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:13.264505+0000 mon.smithi042 (mon.0) 320 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:21:14.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:11.894728+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:21:14.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:11.894828+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:21:14.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:12.789446+0000 mgr.smithi042.kfoihb (mgr.14162) 142 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-02-15T18:21:14.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:12.995675+0000 mgr.smithi042.kfoihb (mgr.14162) 143 : audit [DBG] from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:21:14.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:21:12.997723+0000 mgr.smithi042.kfoihb (mgr.14162) 144 : cephadm [INF] Applying service osd.None on host smithi191... 2024-02-15T18:21:14.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:13.264181+0000 mon.smithi042 (mon.0) 318 : cluster [INF] osd.2 [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670] boot 2024-02-15T18:21:14.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:13.264206+0000 mon.smithi042 (mon.0) 319 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-15T18:21:14.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:13.264505+0000 mon.smithi042 (mon.0) 320 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:21:15.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:14.273203+0000 mon.smithi042 (mon.0) 321 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-15T18:21:15.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:14.273203+0000 mon.smithi042 (mon.0) 321 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-15T18:21:16.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:14.789840+0000 mgr.smithi042.kfoihb (mgr.14162) 145 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:16.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:15.676413+0000 mon.smithi042 (mon.0) 322 : audit [INF] from='client.? 172.21.15.191:0/549023544' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6aeea641-3735-4f11-b95f-40c45d4dcd07"}]: dispatch 2024-02-15T18:21:16.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:15.680271+0000 mon.smithi042 (mon.0) 323 : audit [INF] from='client.? 172.21.15.191:0/549023544' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6aeea641-3735-4f11-b95f-40c45d4dcd07"}]': finished 2024-02-15T18:21:16.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:15.680336+0000 mon.smithi042 (mon.0) 324 : cluster [DBG] osdmap e21: 4 total, 3 up, 4 in 2024-02-15T18:21:16.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:15.680434+0000 mon.smithi042 (mon.0) 325 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:16.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:15.765386+0000 mon.smithi042 (mon.0) 326 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:16.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:15.960561+0000 mon.smithi042 (mon.0) 327 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:16.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:16.084327+0000 mon.smithi191 (mon.1) 2 : audit [DBG] from='client.? 172.21.15.191:0/2436936846' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:21:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:14.789840+0000 mgr.smithi042.kfoihb (mgr.14162) 145 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:15.676413+0000 mon.smithi042 (mon.0) 322 : audit [INF] from='client.? 172.21.15.191:0/549023544' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6aeea641-3735-4f11-b95f-40c45d4dcd07"}]: dispatch 2024-02-15T18:21:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:15.680271+0000 mon.smithi042 (mon.0) 323 : audit [INF] from='client.? 172.21.15.191:0/549023544' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6aeea641-3735-4f11-b95f-40c45d4dcd07"}]': finished 2024-02-15T18:21:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:15.680336+0000 mon.smithi042 (mon.0) 324 : cluster [DBG] osdmap e21: 4 total, 3 up, 4 in 2024-02-15T18:21:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:15.680434+0000 mon.smithi042 (mon.0) 325 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:15.765386+0000 mon.smithi042 (mon.0) 326 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:15.960561+0000 mon.smithi042 (mon.0) 327 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:16.084327+0000 mon.smithi191 (mon.1) 2 : audit [DBG] from='client.? 172.21.15.191:0/2436936846' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:21:18.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:16.790258+0000 mgr.smithi042.kfoihb (mgr.14162) 146 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:18.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:16.790258+0000 mgr.smithi042.kfoihb (mgr.14162) 146 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:20.388 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:18.790735+0000 mgr.smithi042.kfoihb (mgr.14162) 147 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:20.388 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:19.173711+0000 mon.smithi042 (mon.0) 328 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:20.388 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:19.176665+0000 mon.smithi042 (mon.0) 329 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:20.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:18.790735+0000 mgr.smithi042.kfoihb (mgr.14162) 147 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:20.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:19.173711+0000 mon.smithi042 (mon.0) 328 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:20.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:19.176665+0000 mon.smithi042 (mon.0) 329 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:22.399 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:20.791165+0000 mgr.smithi042.kfoihb (mgr.14162) 148 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:22.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:20.791165+0000 mgr.smithi042.kfoihb (mgr.14162) 148 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:23.399 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:22.716588+0000 mon.smithi042 (mon.0) 330 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-15T18:21:23.399 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:22.717433+0000 mon.smithi042 (mon.0) 331 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:21:23.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:22.716588+0000 mon.smithi042 (mon.0) 330 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-15T18:21:23.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:22.717433+0000 mon.smithi042 (mon.0) 331 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:21:24.318 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:21:22.718118+0000 mgr.smithi042.kfoihb (mgr.14162) 149 : cephadm [INF] Deploying daemon osd.3 on smithi191 2024-02-15T18:21:24.318 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:22.791575+0000 mgr.smithi042.kfoihb (mgr.14162) 150 : cluster [DBG] pgmap v113: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:24.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:21:22.718118+0000 mgr.smithi042.kfoihb (mgr.14162) 149 : cephadm [INF] Deploying daemon osd.3 on smithi191 2024-02-15T18:21:24.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:22.791575+0000 mgr.smithi042.kfoihb (mgr.14162) 150 : cluster [DBG] pgmap v113: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:26.194 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 3 on host 'smithi191' 2024-02-15T18:21:26.194 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:26.192+0000 7f9ba97fa700 1 -- 172.21.15.191:0/376402237 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9bac057f10 con 0x7f9b9c05c300 2024-02-15T18:21:26.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:26.195+0000 7f9bb19ef700 1 -- 172.21.15.191:0/376402237 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9b9c05c300 msgr2=0x7f9b9c05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:21:26.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:26.195+0000 7f9bb19ef700 1 --2- 172.21.15.191:0/376402237 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9b9c05c300 0x7f9b9c05e7c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f9bac060060 tx=0x7f9ba0009b30).stop 2024-02-15T18:21:26.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:26.195+0000 7f9bb19ef700 1 -- 172.21.15.191:0/376402237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bac0f66d0 msgr2=0x7f9bac10c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:21:26.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:26.195+0000 7f9bb19ef700 1 --2- 172.21.15.191:0/376402237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bac0f66d0 0x7f9bac10c070 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f9b9800c530 tx=0x7f9b98007e20).stop 2024-02-15T18:21:26.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:26.195+0000 7f9bb19ef700 1 -- 172.21.15.191:0/376402237 shutdown_connections 2024-02-15T18:21:26.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:26.195+0000 7f9bb19ef700 1 --2- 172.21.15.191:0/376402237 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9b9c05c300 0x7f9b9c05e7c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:26.195 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:26.195+0000 7f9bb19ef700 1 --2- 172.21.15.191:0/376402237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9bac0f5430 0x7f9bac109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:26.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:26.195+0000 7f9bb19ef700 1 --2- 172.21.15.191:0/376402237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bac0f66d0 0x7f9bac10c070 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:26.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:26.195+0000 7f9bb19ef700 1 -- 172.21.15.191:0/376402237 >> 172.21.15.191:0/376402237 conn(0x7f9bac0f0b90 msgr2=0x7f9bac0fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:21:26.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:26.195+0000 7f9bb19ef700 1 -- 172.21.15.191:0/376402237 shutdown_connections 2024-02-15T18:21:26.196 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:26.195+0000 7f9bb19ef700 1 -- 172.21.15.191:0/376402237 wait complete. 2024-02-15T18:21:26.755 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:24.791948+0000 mgr.smithi042.kfoihb (mgr.14162) 151 : cluster [DBG] pgmap v114: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:26.756 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:26.184020+0000 mon.smithi042 (mon.0) 332 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:21:26.756 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:26.187377+0000 mon.smithi042 (mon.0) 333 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:26.756 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:26.192824+0000 mon.smithi042 (mon.0) 334 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:26.779 DEBUG:teuthology.orchestra.run.smithi191:osd.3> sudo journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.3.service 2024-02-15T18:21:26.782 INFO:tasks.cephadm:Deploying osd.4 on smithi191 with /dev/vg_nvme/lv_3... 2024-02-15T18:21:26.782 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-15T18:21:26.813 INFO:journalctl@ceph.osd.3.smithi191.stdout:-- Logs begin at Thu 2024-02-15 17:54:16 UTC. -- 2024-02-15T18:21:27.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:24.791948+0000 mgr.smithi042.kfoihb (mgr.14162) 151 : cluster [DBG] pgmap v114: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:27.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:26.184020+0000 mon.smithi042 (mon.0) 332 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:21:27.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:26.187377+0000 mon.smithi042 (mon.0) 333 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:27.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:26.192824+0000 mon.smithi042 (mon.0) 334 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:27.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:26.195382+0000 mon.smithi042 (mon.0) 335 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]: dispatch 2024-02-15T18:21:27.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:26.792236+0000 mgr.smithi042.kfoihb (mgr.14162) 152 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:27.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:27.193551+0000 mon.smithi042 (mon.0) 336 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]': finished 2024-02-15T18:21:27.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:27.193604+0000 mon.smithi042 (mon.0) 337 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-15T18:21:27.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:27.193751+0000 mon.smithi042 (mon.0) 338 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:27.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:27.194526+0000 mon.smithi042 (mon.0) 339 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:21:27.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:26.195382+0000 mon.smithi042 (mon.0) 335 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]: dispatch 2024-02-15T18:21:27.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:26.792236+0000 mgr.smithi042.kfoihb (mgr.14162) 152 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:27.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:27.193551+0000 mon.smithi042 (mon.0) 336 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]': finished 2024-02-15T18:21:27.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:27.193604+0000 mon.smithi042 (mon.0) 337 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-15T18:21:27.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:27.193751+0000 mon.smithi042 (mon.0) 338 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:27.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:27.194526+0000 mon.smithi042 (mon.0) 339 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:21:29.227 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:28.195193+0000 mon.smithi042 (mon.0) 340 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-15T18:21:29.228 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:28.195270+0000 mon.smithi042 (mon.0) 341 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-15T18:21:29.228 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:28.195389+0000 mon.smithi042 (mon.0) 342 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:29.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:28.195193+0000 mon.smithi042 (mon.0) 340 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-15T18:21:29.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:28.195270+0000 mon.smithi042 (mon.0) 341 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-15T18:21:29.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:28.195389+0000 mon.smithi042 (mon.0) 342 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:29.720 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-15T18:21:29.720 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-15T18:21:29.720 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-02-15T18:21:29.721 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-02-15T18:21:29.721 INFO:teuthology.orchestra.run.smithi191.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0150152 s, 698 MB/s 2024-02-15T18:21:29.721 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-02-15T18:21:30.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:28.792572+0000 mgr.smithi042.kfoihb (mgr.14162) 153 : cluster [DBG] pgmap v118: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:30.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:29.199375+0000 mon.smithi042 (mon.0) 343 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-15T18:21:30.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:29.199511+0000 mon.smithi042 (mon.0) 344 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:30.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:30.036675+0000 mon.smithi191 (mon.1) 3 : audit [INF] from='osd.3 [v2:172.21.15.191:6800/3223849184,v1:172.21.15.191:6801/3223849184]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:21:30.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:30.037141+0000 mon.smithi042 (mon.0) 345 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:21:30.450 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:21:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.3[44935]: debug 2024-02-15T18:21:30.031+0000 7f2acf406080 -1 osd.3 0 log_to_monitors {default=true} 2024-02-15T18:21:30.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:28.792572+0000 mgr.smithi042.kfoihb (mgr.14162) 153 : cluster [DBG] pgmap v118: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:30.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:29.199375+0000 mon.smithi042 (mon.0) 343 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-15T18:21:30.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:29.199511+0000 mon.smithi042 (mon.0) 344 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:30.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:30.036675+0000 mon.smithi191 (mon.1) 3 : audit [INF] from='osd.3 [v2:172.21.15.191:6800/3223849184,v1:172.21.15.191:6801/3223849184]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:21:30.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:30.037141+0000 mon.smithi042 (mon.0) 345 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:21:30.912 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_3 2024-02-15T18:21:31.438 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:21:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.3[44935]: debug 2024-02-15T18:21:31.260+0000 7f2ac717a700 -1 osd.3 0 waiting for initial osdmap 2024-02-15T18:21:31.438 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:21:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.3[44935]: debug 2024-02-15T18:21:31.266+0000 7f2ac00ef700 -1 osd.3 26 set_numa_affinity unable to identify public interface 'enp3s0f1' numa node: (0) Success 2024-02-15T18:21:31.439 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:30.255325+0000 mon.smithi042 (mon.0) 346 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-15T18:21:31.439 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:30.255392+0000 mon.smithi042 (mon.0) 347 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-02-15T18:21:31.439 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:30.255543+0000 mon.smithi042 (mon.0) 348 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:31.439 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:30.255959+0000 mon.smithi191 (mon.1) 4 : audit [INF] from='osd.3 [v2:172.21.15.191:6800/3223849184,v1:172.21.15.191:6801/3223849184]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:21:31.439 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:30.256223+0000 mon.smithi042 (mon.0) 349 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:21:31.439 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:30.603082+0000 mon.smithi042 (mon.0) 350 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:31.439 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:30.611337+0000 mon.smithi042 (mon.0) 351 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:31.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:30.255325+0000 mon.smithi042 (mon.0) 346 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-15T18:21:31.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:30.255392+0000 mon.smithi042 (mon.0) 347 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2024-02-15T18:21:31.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:30.255543+0000 mon.smithi042 (mon.0) 348 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:31.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:30.255959+0000 mon.smithi191 (mon.1) 4 : audit [INF] from='osd.3 [v2:172.21.15.191:6800/3223849184,v1:172.21.15.191:6801/3223849184]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:21:31.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:30.256223+0000 mon.smithi042 (mon.0) 349 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:21:31.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:30.603082+0000 mon.smithi042 (mon.0) 350 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:31.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:30.611337+0000 mon.smithi042 (mon.0) 351 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:32.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:30.793045+0000 mgr.smithi042.kfoihb (mgr.14162) 154 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:32.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:31.256981+0000 mon.smithi042 (mon.0) 352 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-15T18:21:32.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:31.257106+0000 mon.smithi042 (mon.0) 353 : cluster [DBG] osdmap e26: 4 total, 3 up, 4 in 2024-02-15T18:21:32.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:31.257489+0000 mon.smithi042 (mon.0) 354 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:32.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:31.262249+0000 mon.smithi042 (mon.0) 355 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:32.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:30.793045+0000 mgr.smithi042.kfoihb (mgr.14162) 154 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:32.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:31.256981+0000 mon.smithi042 (mon.0) 352 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-15T18:21:32.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:31.257106+0000 mon.smithi042 (mon.0) 353 : cluster [DBG] osdmap e26: 4 total, 3 up, 4 in 2024-02-15T18:21:32.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:31.257489+0000 mon.smithi042 (mon.0) 354 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:32.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:31.262249+0000 mon.smithi042 (mon.0) 355 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:33.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.247+0000 7fd6f9752700 1 -- 172.21.15.191:0/3354218130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6f4060310 msgr2=0x7fd6f40f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:21:33.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.247+0000 7fd6f9752700 1 --2- 172.21.15.191:0/3354218130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6f4060310 0x7fd6f40f9110 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fd6e8004660 tx=0x7fd6e8009b30).stop 2024-02-15T18:21:33.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.247+0000 7fd6f9752700 1 -- 172.21.15.191:0/3354218130 shutdown_connections 2024-02-15T18:21:33.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.247+0000 7fd6f9752700 1 --2- 172.21.15.191:0/3354218130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6f40f9690 0x7fd6f40fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:21:33.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.247+0000 7fd6f9752700 1 --2- 172.21.15.191:0/3354218130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6f4060310 0x7fd6f40f9110 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:33.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.247+0000 7fd6f9752700 1 -- 172.21.15.191:0/3354218130 >> 172.21.15.191:0/3354218130 conn(0x7fd6f40f0b90 msgr2=0x7fd6f40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:21:33.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.247+0000 7fd6f9752700 1 -- 172.21.15.191:0/3354218130 shutdown_connections 2024-02-15T18:21:33.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.247+0000 7fd6f9752700 1 -- 172.21.15.191:0/3354218130 wait complete. 2024-02-15T18:21:33.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.248+0000 7fd6f9752700 1 Processor -- start 2024-02-15T18:21:33.249 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.248+0000 7fd6f9752700 1 -- start start 2024-02-15T18:21:33.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.248+0000 7fd6f9752700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6f4060310 0x7fd6f4106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:21:33.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.248+0000 7fd6f9752700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6f40f9690 0x7fd6f41073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:21:33.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.248+0000 7fd6f9752700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f4107a40 con 0x7fd6f40f9690 2024-02-15T18:21:33.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.248+0000 7fd6f9752700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f41081c0 con 0x7fd6f4060310 2024-02-15T18:21:33.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.248+0000 7fd6f3fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6f4060310 0x7fd6f4106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:21:33.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.248+0000 7fd6f3fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6f4060310 0x7fd6f4106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:34588/0 (socket says 172.21.15.191:34588) 2024-02-15T18:21:33.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.248+0000 7fd6f3fff700 1 -- 172.21.15.191:0/3579518814 learned_addr learned my addr 172.21.15.191:0/3579518814 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:21:33.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.248+0000 7fd6f37fe700 1 --2- 172.21.15.191:0/3579518814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6f40f9690 0x7fd6f41073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:21:33.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.248+0000 7fd6f3fff700 1 -- 172.21.15.191:0/3579518814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6f40f9690 msgr2=0x7fd6f41073b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:21:33.251 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.248+0000 7fd6f3fff700 1 --2- 172.21.15.191:0/3579518814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6f40f9690 0x7fd6f41073b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:33.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.248+0000 7fd6f3fff700 1 -- 172.21.15.191:0/3579518814 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8005040 con 0x7fd6f4060310 2024-02-15T18:21:33.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.249+0000 7fd6f37fe700 1 --2- 172.21.15.191:0/3579518814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6f40f9690 0x7fd6f41073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:21:33.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.249+0000 7fd6f3fff700 1 --2- 172.21.15.191:0/3579518814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6f4060310 0x7fd6f4106ec0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fd6e8009830 tx=0x7fd6e8009b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:21:33.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.249+0000 7fd6f0ff9700 1 -- 172.21.15.191:0/3579518814 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6e8018070 con 0x7fd6f4060310 2024-02-15T18:21:33.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.249+0000 7fd6f0ff9700 1 -- 172.21.15.191:0/3579518814 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6e8007d70 con 0x7fd6f4060310 2024-02-15T18:21:33.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.249+0000 7fd6f0ff9700 1 -- 172.21.15.191:0/3579518814 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6e801c620 con 0x7fd6f4060310 2024-02-15T18:21:33.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.249+0000 7fd6f9752700 1 -- 172.21.15.191:0/3579518814 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f41083e0 con 0x7fd6f4060310 2024-02-15T18:21:33.252 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.249+0000 7fd6f9752700 1 -- 172.21.15.191:0/3579518814 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f4108940 con 0x7fd6f4060310 2024-02-15T18:21:33.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.250+0000 7fd6f0ff9700 1 -- 172.21.15.191:0/3579518814 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fd6e801d980 con 0x7fd6f4060310 2024-02-15T18:21:33.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.250+0000 7fd6f9752700 1 -- 172.21.15.191:0/3579518814 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f4045ad0 con 0x7fd6f4060310 2024-02-15T18:21:33.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.251+0000 7fd6f0ff9700 1 --2- 172.21.15.191:0/3579518814 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd6dc05c300 0x7fd6dc05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:21:33.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.251+0000 7fd6f0ff9700 1 -- 172.21.15.191:0/3579518814 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3721+0+0 (secure 0 0 0) 0x7fd6e8077660 con 0x7fd6f4060310 2024-02-15T18:21:33.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.251+0000 7fd6f37fe700 1 --2- 172.21.15.191:0/3579518814 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd6dc05c300 0x7fd6dc05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:21:33.253 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.251+0000 7fd6f37fe700 1 --2- 172.21.15.191:0/3579518814 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd6dc05c300 0x7fd6dc05e7c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fd6f40600e0 tx=0x7fd6e4008040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:21:33.256 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.255+0000 7fd6f0ff9700 1 -- 172.21.15.191:0/3579518814 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd6e804a9c0 con 0x7fd6f4060310 2024-02-15T18:21:33.436 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:33.434+0000 7fd6f9752700 1 -- 172.21.15.191:0/3579518814 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6f40ff050 con 0x7fd6dc05c300 2024-02-15T18:21:33.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:31.013655+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:21:33.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:31.013754+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:21:33.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:32.262567+0000 mon.smithi042 (mon.0) 356 : cluster [INF] osd.3 [v2:172.21.15.191:6800/3223849184,v1:172.21.15.191:6801/3223849184] boot 2024-02-15T18:21:33.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:32.262623+0000 mon.smithi042 (mon.0) 357 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-02-15T18:21:33.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:32.263165+0000 mon.smithi042 (mon.0) 358 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:33.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:33.263908+0000 mon.smithi042 (mon.0) 359 : cluster [DBG] osdmap e28: 4 total, 4 up, 4 in 2024-02-15T18:21:33.654 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:31.013655+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:21:33.654 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:31.013754+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:21:33.654 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:32.262567+0000 mon.smithi042 (mon.0) 356 : cluster [INF] osd.3 [v2:172.21.15.191:6800/3223849184,v1:172.21.15.191:6801/3223849184] boot 2024-02-15T18:21:33.654 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:32.262623+0000 mon.smithi042 (mon.0) 357 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2024-02-15T18:21:33.654 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:32.263165+0000 mon.smithi042 (mon.0) 358 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:21:33.654 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:33.263908+0000 mon.smithi042 (mon.0) 359 : cluster [DBG] osdmap e28: 4 total, 4 up, 4 in 2024-02-15T18:21:34.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:32.793494+0000 mgr.smithi042.kfoihb (mgr.14162) 155 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:34.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:33.437905+0000 mgr.smithi042.kfoihb (mgr.14162) 156 : audit [DBG] from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:21:34.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:33.439228+0000 mon.smithi042 (mon.0) 360 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:21:34.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:21:33.440641+0000 mgr.smithi042.kfoihb (mgr.14162) 157 : cephadm [INF] Applying service osd.None on host smithi191... 2024-02-15T18:21:34.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:33.440879+0000 mon.smithi042 (mon.0) 361 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:21:34.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:33.441545+0000 mon.smithi042 (mon.0) 362 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:21:34.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:34.264035+0000 mon.smithi042 (mon.0) 363 : cluster [DBG] osdmap e29: 4 total, 4 up, 4 in 2024-02-15T18:21:34.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:32.793494+0000 mgr.smithi042.kfoihb (mgr.14162) 155 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-02-15T18:21:34.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:33.437905+0000 mgr.smithi042.kfoihb (mgr.14162) 156 : audit [DBG] from='client.24153 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:21:34.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:33.439228+0000 mon.smithi042 (mon.0) 360 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:21:34.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:21:33.440641+0000 mgr.smithi042.kfoihb (mgr.14162) 157 : cephadm [INF] Applying service osd.None on host smithi191... 2024-02-15T18:21:34.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:33.440879+0000 mon.smithi042 (mon.0) 361 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:21:34.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:33.441545+0000 mon.smithi042 (mon.0) 362 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:21:34.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:34.264035+0000 mon.smithi042 (mon.0) 363 : cluster [DBG] osdmap e29: 4 total, 4 up, 4 in 2024-02-15T18:21:36.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:34.793975+0000 mgr.smithi042.kfoihb (mgr.14162) 158 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:36.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:34.793975+0000 mgr.smithi042.kfoihb (mgr.14162) 158 : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:37.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:21:36.273592+0000 mgr.smithi042.kfoihb (mgr.14162) 159 : cephadm [INF] Detected new or changed devices on smithi191 2024-02-15T18:21:37.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:36.278794+0000 mon.smithi042 (mon.0) 364 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:37.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:36.282324+0000 mon.smithi042 (mon.0) 365 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:37.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:36.285282+0000 mon.smithi042 (mon.0) 366 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:21:37.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:36.967342+0000 mon.smithi191 (mon.1) 5 : audit [INF] from='client.? 172.21.15.191:0/651592931' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d9839d2-bd60-4bae-a16f-ebf56f204acc"}]: dispatch 2024-02-15T18:21:37.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:36.967901+0000 mon.smithi042 (mon.0) 367 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d9839d2-bd60-4bae-a16f-ebf56f204acc"}]: dispatch 2024-02-15T18:21:37.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:36.971843+0000 mon.smithi042 (mon.0) 368 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d9839d2-bd60-4bae-a16f-ebf56f204acc"}]': finished 2024-02-15T18:21:37.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:36.971900+0000 mon.smithi042 (mon.0) 369 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-15T18:21:37.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:36.972012+0000 mon.smithi042 (mon.0) 370 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:21:37.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:21:36.273592+0000 mgr.smithi042.kfoihb (mgr.14162) 159 : cephadm [INF] Detected new or changed devices on smithi191 2024-02-15T18:21:37.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:36.278794+0000 mon.smithi042 (mon.0) 364 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:37.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:36.282324+0000 mon.smithi042 (mon.0) 365 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:37.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:36.285282+0000 mon.smithi042 (mon.0) 366 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:21:37.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:36.967342+0000 mon.smithi191 (mon.1) 5 : audit [INF] from='client.? 172.21.15.191:0/651592931' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d9839d2-bd60-4bae-a16f-ebf56f204acc"}]: dispatch 2024-02-15T18:21:37.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:36.967901+0000 mon.smithi042 (mon.0) 367 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d9839d2-bd60-4bae-a16f-ebf56f204acc"}]: dispatch 2024-02-15T18:21:37.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:36.971843+0000 mon.smithi042 (mon.0) 368 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d9839d2-bd60-4bae-a16f-ebf56f204acc"}]': finished 2024-02-15T18:21:37.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:36.971900+0000 mon.smithi042 (mon.0) 369 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-15T18:21:37.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:36.972012+0000 mon.smithi042 (mon.0) 370 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:21:38.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:36.794268+0000 mgr.smithi042.kfoihb (mgr.14162) 160 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:38.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:37.387763+0000 mon.smithi191 (mon.1) 6 : audit [DBG] from='client.? 172.21.15.191:0/332528180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:21:38.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:36.794268+0000 mgr.smithi042.kfoihb (mgr.14162) 160 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:38.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:37.387763+0000 mon.smithi191 (mon.1) 6 : audit [DBG] from='client.? 172.21.15.191:0/332528180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:21:40.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:38.794585+0000 mgr.smithi042.kfoihb (mgr.14162) 161 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:40.646 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:38.794585+0000 mgr.smithi042.kfoihb (mgr.14162) 161 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:40.550879+0000 mon.smithi042 (mon.0) 371 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:40.553595+0000 mon.smithi042 (mon.0) 372 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:41.900 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:40.550879+0000 mon.smithi042 (mon.0) 371 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:41.900 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:40.553595+0000 mon.smithi042 (mon.0) 372 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:42.658 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:40.795047+0000 mgr.smithi042.kfoihb (mgr.14162) 162 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:42.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:40.795047+0000 mgr.smithi042.kfoihb (mgr.14162) 162 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:44.812 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:42.795448+0000 mgr.smithi042.kfoihb (mgr.14162) 163 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:44.812 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:44.096193+0000 mon.smithi042 (mon.0) 373 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-15T18:21:44.812 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:44.097056+0000 mon.smithi042 (mon.0) 374 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:21:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:42.795448+0000 mgr.smithi042.kfoihb (mgr.14162) 163 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:44.096193+0000 mon.smithi042 (mon.0) 373 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-15T18:21:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:44.097056+0000 mon.smithi042 (mon.0) 374 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:21:45.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:21:44.097757+0000 mgr.smithi042.kfoihb (mgr.14162) 164 : cephadm [INF] Deploying daemon osd.4 on smithi191 2024-02-15T18:21:45.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:44.795906+0000 mgr.smithi042.kfoihb (mgr.14162) 165 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:45.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:21:44.097757+0000 mgr.smithi042.kfoihb (mgr.14162) 164 : cephadm [INF] Deploying daemon osd.4 on smithi191 2024-02-15T18:21:45.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:44.795906+0000 mgr.smithi042.kfoihb (mgr.14162) 165 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:47.688 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 4 on host 'smithi191' 2024-02-15T18:21:47.688 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:47.686+0000 7fd6f0ff9700 1 -- 172.21.15.191:0/3579518814 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd6e400b410 con 0x7fd6dc05c300 2024-02-15T18:21:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:47.689+0000 7fd6f9752700 1 -- 172.21.15.191:0/3579518814 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd6dc05c300 msgr2=0x7fd6dc05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:21:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:47.689+0000 7fd6f9752700 1 --2- 172.21.15.191:0/3579518814 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd6dc05c300 0x7fd6dc05e7c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fd6f40600e0 tx=0x7fd6e4008040).stop 2024-02-15T18:21:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:47.689+0000 7fd6f9752700 1 -- 172.21.15.191:0/3579518814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6f4060310 msgr2=0x7fd6f4106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:21:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:47.689+0000 7fd6f9752700 1 --2- 172.21.15.191:0/3579518814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6f4060310 0x7fd6f4106ec0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fd6e8009830 tx=0x7fd6e8009b30).stop 2024-02-15T18:21:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:47.689+0000 7fd6f9752700 1 -- 172.21.15.191:0/3579518814 shutdown_connections 2024-02-15T18:21:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:47.689+0000 7fd6f9752700 1 --2- 172.21.15.191:0/3579518814 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd6dc05c300 0x7fd6dc05e7c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:47.689+0000 7fd6f9752700 1 --2- 172.21.15.191:0/3579518814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6f4060310 0x7fd6f4106ec0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:47.689+0000 7fd6f9752700 1 --2- 172.21.15.191:0/3579518814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6f40f9690 0x7fd6f41073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:47.689 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:47.689+0000 7fd6f9752700 1 -- 172.21.15.191:0/3579518814 >> 172.21.15.191:0/3579518814 conn(0x7fd6f40f0b90 msgr2=0x7fd6f40f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:21:47.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:47.689+0000 7fd6f9752700 1 -- 172.21.15.191:0/3579518814 shutdown_connections 2024-02-15T18:21:47.690 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:47.689+0000 7fd6f9752700 1 -- 172.21.15.191:0/3579518814 wait complete. 2024-02-15T18:21:47.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:46.796447+0000 mgr.smithi042.kfoihb (mgr.14162) 166 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:47.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:47.676305+0000 mon.smithi042 (mon.0) 375 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:21:47.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:47.681219+0000 mon.smithi042 (mon.0) 376 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:47.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:47.686254+0000 mon.smithi042 (mon.0) 377 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:48.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:46.796447+0000 mgr.smithi042.kfoihb (mgr.14162) 166 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:48.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:47.676305+0000 mon.smithi042 (mon.0) 375 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:21:48.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:47.681219+0000 mon.smithi042 (mon.0) 376 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:48.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:47.686254+0000 mon.smithi042 (mon.0) 377 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:48.322 DEBUG:teuthology.orchestra.run.smithi191:osd.4> sudo journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.4.service 2024-02-15T18:21:48.324 INFO:tasks.cephadm:Deploying osd.5 on smithi191 with /dev/vg_nvme/lv_2... 2024-02-15T18:21:48.325 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-15T18:21:48.354 INFO:journalctl@ceph.osd.4.smithi191.stdout:-- Logs begin at Thu 2024-02-15 17:54:16 UTC. -- 2024-02-15T18:21:50.305 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:48.796714+0000 mgr.smithi042.kfoihb (mgr.14162) 167 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:50.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:48.796714+0000 mgr.smithi042.kfoihb (mgr.14162) 167 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:51.116 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-15T18:21:51.116 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-15T18:21:51.116 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-02-15T18:21:51.116 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-02-15T18:21:51.117 INFO:teuthology.orchestra.run.smithi191.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122382 s, 857 MB/s 2024-02-15T18:21:51.117 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 2024-02-15T18:21:51.117 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-02-15T18:21:51.864 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:21:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.4[48536]: debug 2024-02-15T18:21:51.494+0000 7f50a061b080 -1 osd.4 0 log_to_monitors {default=true} 2024-02-15T18:21:51.883 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi191:vg_nvme/lv_2 2024-02-15T18:21:52.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:50.797074+0000 mgr.smithi042.kfoihb (mgr.14162) 168 : cluster [DBG] pgmap v136: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:52.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:51.498149+0000 mon.smithi191 (mon.1) 7 : audit [INF] from='osd.4 [v2:172.21.15.191:6808/424560222,v1:172.21.15.191:6809/424560222]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:21:52.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:51.498404+0000 mon.smithi042 (mon.0) 378 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:21:52.604 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:50.797074+0000 mgr.smithi042.kfoihb (mgr.14162) 168 : cluster [DBG] pgmap v136: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:52.604 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:51.498149+0000 mon.smithi191 (mon.1) 7 : audit [INF] from='osd.4 [v2:172.21.15.191:6808/424560222,v1:172.21.15.191:6809/424560222]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:21:52.604 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:51.498404+0000 mon.smithi042 (mon.0) 378 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:21:53.561 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:21:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.4[48536]: debug 2024-02-15T18:21:53.270+0000 7f5096b8c700 -1 osd.4 0 waiting for initial osdmap 2024-02-15T18:21:53.561 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:21:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.4[48536]: debug 2024-02-15T18:21:53.276+0000 7f508f300700 -1 osd.4 32 set_numa_affinity unable to identify public interface 'enp3s0f1' numa node: (0) Success 2024-02-15T18:21:53.561 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:52.264412+0000 mon.smithi042 (mon.0) 379 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-15T18:21:53.561 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:52.264486+0000 mon.smithi042 (mon.0) 380 : cluster [DBG] osdmap e31: 5 total, 4 up, 5 in 2024-02-15T18:21:53.561 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:52.264610+0000 mon.smithi042 (mon.0) 381 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:21:53.561 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:52.265015+0000 mon.smithi191 (mon.1) 8 : audit [INF] from='osd.4 [v2:172.21.15.191:6808/424560222,v1:172.21.15.191:6809/424560222]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:21:53.561 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:52.265236+0000 mon.smithi042 (mon.0) 382 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:21:53.562 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:52.531649+0000 mon.smithi042 (mon.0) 383 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:53.562 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:52.729118+0000 mon.smithi042 (mon.0) 384 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:53.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:52.264412+0000 mon.smithi042 (mon.0) 379 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-15T18:21:53.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:52.264486+0000 mon.smithi042 (mon.0) 380 : cluster [DBG] osdmap e31: 5 total, 4 up, 5 in 2024-02-15T18:21:53.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:52.264610+0000 mon.smithi042 (mon.0) 381 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:21:53.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:52.265015+0000 mon.smithi191 (mon.1) 8 : audit [INF] from='osd.4 [v2:172.21.15.191:6808/424560222,v1:172.21.15.191:6809/424560222]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:21:53.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:52.265236+0000 mon.smithi042 (mon.0) 382 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:21:53.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:52.531649+0000 mon.smithi042 (mon.0) 383 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:53.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:52.729118+0000 mon.smithi042 (mon.0) 384 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:54.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:52.797606+0000 mgr.smithi042.kfoihb (mgr.14162) 169 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:54.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:53.266430+0000 mon.smithi042 (mon.0) 385 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-15T18:21:54.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:53.266555+0000 mon.smithi042 (mon.0) 386 : cluster [DBG] osdmap e32: 5 total, 4 up, 5 in 2024-02-15T18:21:54.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:53.267037+0000 mon.smithi042 (mon.0) 387 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:21:54.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:53.271199+0000 mon.smithi042 (mon.0) 388 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:21:54.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:54.267189+0000 mon.smithi042 (mon.0) 389 : cluster [INF] osd.4 [v2:172.21.15.191:6808/424560222,v1:172.21.15.191:6809/424560222] boot 2024-02-15T18:21:54.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:54.267263+0000 mon.smithi042 (mon.0) 390 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-15T18:21:54.618 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:52.797606+0000 mgr.smithi042.kfoihb (mgr.14162) 169 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-02-15T18:21:54.618 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:53.266430+0000 mon.smithi042 (mon.0) 385 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-15T18:21:54.618 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:53.266555+0000 mon.smithi042 (mon.0) 386 : cluster [DBG] osdmap e32: 5 total, 4 up, 5 in 2024-02-15T18:21:54.618 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:53.267037+0000 mon.smithi042 (mon.0) 387 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:21:54.618 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:53.271199+0000 mon.smithi042 (mon.0) 388 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:21:54.618 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:54.267189+0000 mon.smithi042 (mon.0) 389 : cluster [INF] osd.4 [v2:172.21.15.191:6808/424560222,v1:172.21.15.191:6809/424560222] boot 2024-02-15T18:21:54.618 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:54.267263+0000 mon.smithi042 (mon.0) 390 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-15T18:21:55.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.164+0000 7f55346ec700 1 -- 172.21.15.191:0/4173932313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f552c0f6860 msgr2=0x7f552c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:21:55.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.164+0000 7f55346ec700 1 --2- 172.21.15.191:0/4173932313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f552c0f6860 0x7f552c0f6cc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f551c004660 tx=0x7f551c009b30).stop 2024-02-15T18:21:55.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.165+0000 7f55346ec700 1 -- 172.21.15.191:0/4173932313 shutdown_connections 2024-02-15T18:21:55.165 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.165+0000 7f55346ec700 1 --2- 172.21.15.191:0/4173932313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f552c0f6860 0x7f552c0f6cc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:55.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.165+0000 7f55346ec700 1 --2- 172.21.15.191:0/4173932313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f552c0f55c0 0x7f552c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:21:55.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.165+0000 7f55346ec700 1 -- 172.21.15.191:0/4173932313 >> 172.21.15.191:0/4173932313 conn(0x7f552c0f0d40 msgr2=0x7f552c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:21:55.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.165+0000 7f55346ec700 1 -- 172.21.15.191:0/4173932313 shutdown_connections 2024-02-15T18:21:55.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.165+0000 7f55346ec700 1 -- 172.21.15.191:0/4173932313 wait complete. 2024-02-15T18:21:55.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.165+0000 7f55346ec700 1 Processor -- start 2024-02-15T18:21:55.166 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.165+0000 7f55346ec700 1 -- start start 2024-02-15T18:21:55.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f55346ec700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f552c0f55c0 0x7f552c109d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:21:55.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f55346ec700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f552c0f6860 0x7f552c10c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:21:55.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f55346ec700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c060450 con 0x7f552c0f6860 2024-02-15T18:21:55.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f55346ec700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c060590 con 0x7f552c0f55c0 2024-02-15T18:21:55.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f5532488700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f552c0f55c0 0x7f552c109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:21:55.168 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f5532488700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f552c0f55c0 0x7f552c109d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:44418/0 (socket says 172.21.15.191:44418) 2024-02-15T18:21:55.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f5532488700 1 -- 172.21.15.191:0/3030800930 learned_addr learned my addr 172.21.15.191:0/3030800930 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:21:55.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f5531c87700 1 --2- 172.21.15.191:0/3030800930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f552c0f6860 0x7f552c10c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:21:55.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f5532488700 1 -- 172.21.15.191:0/3030800930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f552c0f6860 msgr2=0x7f552c10c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:21:55.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f5532488700 1 --2- 172.21.15.191:0/3030800930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f552c0f6860 0x7f552c10c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:21:55.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f5532488700 1 -- 172.21.15.191:0/3030800930 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c005040 con 0x7f552c0f55c0 2024-02-15T18:21:55.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f5532488700 1 --2- 172.21.15.191:0/3030800930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f552c0f55c0 0x7f552c109d90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f55280093f0 tx=0x7f5528006ec0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:21:55.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f5522ffd700 1 -- 172.21.15.191:0/3030800930 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5528013640 con 0x7f552c0f55c0 2024-02-15T18:21:55.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f55346ec700 1 -- 172.21.15.191:0/3030800930 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552c10c840 con 0x7f552c0f55c0 2024-02-15T18:21:55.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f5522ffd700 1 -- 172.21.15.191:0/3030800930 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5528013c70 con 0x7f552c0f55c0 2024-02-15T18:21:55.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f5522ffd700 1 -- 172.21.15.191:0/3030800930 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5528012d80 con 0x7f552c0f55c0 2024-02-15T18:21:55.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.166+0000 7f55346ec700 1 -- 172.21.15.191:0/3030800930 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552c10ce00 con 0x7f552c0f55c0 2024-02-15T18:21:55.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.167+0000 7f55346ec700 1 -- 172.21.15.191:0/3030800930 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552c045ad0 con 0x7f552c0f55c0 2024-02-15T18:21:55.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.167+0000 7f5522ffd700 1 -- 172.21.15.191:0/3030800930 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f552801a060 con 0x7f552c0f55c0 2024-02-15T18:21:55.169 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.168+0000 7f5522ffd700 1 --2- 172.21.15.191:0/3030800930 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f551805c2b0 0x7f551805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:21:55.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.168+0000 7f5522ffd700 1 -- 172.21.15.191:0/3030800930 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4120+0+0 (secure 0 0 0) 0x7f55280759e0 con 0x7f552c0f55c0 2024-02-15T18:21:55.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.168+0000 7f5531c87700 1 --2- 172.21.15.191:0/3030800930 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f551805c2b0 0x7f551805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:21:55.170 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.168+0000 7f5531c87700 1 --2- 172.21.15.191:0/3030800930 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f551805c2b0 0x7f551805e770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f551c009f50 tx=0x7f551c009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:21:55.171 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.172+0000 7f5522ffd700 1 -- 172.21.15.191:0/3030800930 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5528048b90 con 0x7f552c0f55c0 2024-02-15T18:21:55.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:21:55.355+0000 7f55346ec700 1 -- 172.21.15.191:0/3030800930 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f552c057f10 con 0x7f551805c2b0 2024-02-15T18:21:55.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:52.493553+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:21:55.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:52.493754+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:21:55.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:54.267401+0000 mon.smithi042 (mon.0) 391 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:21:55.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:52.493553+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:21:55.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:52.493754+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:21:55.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:54.267401+0000 mon.smithi042 (mon.0) 391 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:21:56.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:54.798029+0000 mgr.smithi042.kfoihb (mgr.14162) 170 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:21:56.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:55.276353+0000 mon.smithi042 (mon.0) 392 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2024-02-15T18:21:56.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:55.357298+0000 mgr.smithi042.kfoihb (mgr.14162) 171 : audit [DBG] from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:21:56.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:55.358198+0000 mon.smithi042 (mon.0) 393 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:21:56.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:21:55.359604+0000 mgr.smithi042.kfoihb (mgr.14162) 172 : cephadm [INF] Applying service osd.None on host smithi191... 2024-02-15T18:21:56.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:55.359904+0000 mon.smithi042 (mon.0) 394 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:21:56.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:55.360657+0000 mon.smithi042 (mon.0) 395 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:21:56.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:54.798029+0000 mgr.smithi042.kfoihb (mgr.14162) 170 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:21:56.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:55.276353+0000 mon.smithi042 (mon.0) 392 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2024-02-15T18:21:56.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:55.357298+0000 mgr.smithi042.kfoihb (mgr.14162) 171 : audit [DBG] from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:21:56.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:55.358198+0000 mon.smithi042 (mon.0) 393 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-15T18:21:56.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:21:55.359604+0000 mgr.smithi042.kfoihb (mgr.14162) 172 : cephadm [INF] Applying service osd.None on host smithi191... 2024-02-15T18:21:56.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:55.359904+0000 mon.smithi042 (mon.0) 394 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-15T18:21:56.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:55.360657+0000 mon.smithi042 (mon.0) 395 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:21:57.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:56.912850+0000 mon.smithi042 (mon.0) 396 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:21:57.450 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:56.913101+0000 mon.smithi042 (mon.0) 397 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:21:57.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:56.912850+0000 mon.smithi042 (mon.0) 396 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:21:57.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:56.913101+0000 mon.smithi042 (mon.0) 397 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:21:58.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:56.798367+0000 mgr.smithi042.kfoihb (mgr.14162) 173 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:21:58.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:57.885903+0000 mon.smithi042 (mon.0) 398 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:58.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:58.194259+0000 mon.smithi042 (mon.0) 399 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:58.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:58.197232+0000 mon.smithi042 (mon.0) 400 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:21:58.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:56.798367+0000 mgr.smithi042.kfoihb (mgr.14162) 173 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:21:58.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:57.885903+0000 mon.smithi042 (mon.0) 398 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:58.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:58.194259+0000 mon.smithi042 (mon.0) 399 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:21:58.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:58.197232+0000 mon.smithi042 (mon.0) 400 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:21:59.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:21:57.881026+0000 mgr.smithi042.kfoihb (mgr.14162) 174 : cephadm [INF] Detected new or changed devices on smithi191 2024-02-15T18:21:59.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:58.583256+0000 mon.smithi191 (mon.1) 9 : audit [INF] from='client.? 172.21.15.191:0/232467553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a75bb280-bd87-4961-918b-18c426f31c97"}]: dispatch 2024-02-15T18:21:59.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:58.583642+0000 mon.smithi042 (mon.0) 401 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a75bb280-bd87-4961-918b-18c426f31c97"}]: dispatch 2024-02-15T18:21:59.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:58.587398+0000 mon.smithi042 (mon.0) 402 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a75bb280-bd87-4961-918b-18c426f31c97"}]': finished 2024-02-15T18:21:59.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:58.587452+0000 mon.smithi042 (mon.0) 403 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-15T18:21:59.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:58.587556+0000 mon.smithi042 (mon.0) 404 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:21:59.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:21:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:21:58.994580+0000 mon.smithi191 (mon.1) 10 : audit [DBG] from='client.? 172.21.15.191:0/425116447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:21:59.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:21:57.881026+0000 mgr.smithi042.kfoihb (mgr.14162) 174 : cephadm [INF] Detected new or changed devices on smithi191 2024-02-15T18:21:59.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:58.583256+0000 mon.smithi191 (mon.1) 9 : audit [INF] from='client.? 172.21.15.191:0/232467553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a75bb280-bd87-4961-918b-18c426f31c97"}]: dispatch 2024-02-15T18:21:59.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:58.583642+0000 mon.smithi042 (mon.0) 401 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a75bb280-bd87-4961-918b-18c426f31c97"}]: dispatch 2024-02-15T18:21:59.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:58.587398+0000 mon.smithi042 (mon.0) 402 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a75bb280-bd87-4961-918b-18c426f31c97"}]': finished 2024-02-15T18:21:59.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:58.587452+0000 mon.smithi042 (mon.0) 403 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-15T18:21:59.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:58.587556+0000 mon.smithi042 (mon.0) 404 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:21:59.701 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:21:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:21:58.994580+0000 mon.smithi191 (mon.1) 10 : audit [DBG] from='client.? 172.21.15.191:0/425116447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-15T18:22:00.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:21:58.798677+0000 mgr.smithi042.kfoihb (mgr.14162) 175 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:00.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:21:58.798677+0000 mgr.smithi042.kfoihb (mgr.14162) 175 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:02.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:00.799097+0000 mgr.smithi042.kfoihb (mgr.14162) 176 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:02.590 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:00.799097+0000 mgr.smithi042.kfoihb (mgr.14162) 176 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:03.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:02.545750+0000 mon.smithi042 (mon.0) 405 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:03.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:02.809598+0000 mon.smithi042 (mon.0) 406 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:03.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:02.545750+0000 mon.smithi042 (mon.0) 405 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:03.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:02.809598+0000 mon.smithi042 (mon.0) 406 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:04.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:02.799382+0000 mgr.smithi042.kfoihb (mgr.14162) 177 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:04.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:02.799382+0000 mgr.smithi042.kfoihb (mgr.14162) 177 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:06.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:04.799859+0000 mgr.smithi042.kfoihb (mgr.14162) 178 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:06.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:05.825270+0000 mon.smithi042 (mon.0) 407 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-15T18:22:06.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:05.826088+0000 mon.smithi042 (mon.0) 408 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:22:06.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:04.799859+0000 mgr.smithi042.kfoihb (mgr.14162) 178 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:06.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:05.825270+0000 mon.smithi042 (mon.0) 407 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-15T18:22:06.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:05.826088+0000 mon.smithi042 (mon.0) 408 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:22:07.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:22:05.826738+0000 mgr.smithi042.kfoihb (mgr.14162) 179 : cephadm [INF] Deploying daemon osd.5 on smithi191 2024-02-15T18:22:07.845 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:22:05.826738+0000 mgr.smithi042.kfoihb (mgr.14162) 179 : cephadm [INF] Deploying daemon osd.5 on smithi191 2024-02-15T18:22:08.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:06.800296+0000 mgr.smithi042.kfoihb (mgr.14162) 180 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:08.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:06.800296+0000 mgr.smithi042.kfoihb (mgr.14162) 180 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:09.567 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 5 on host 'smithi191' 2024-02-15T18:22:09.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:09.565+0000 7f5522ffd700 1 -- 172.21.15.191:0/3030800930 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f552c057f10 con 0x7f551805c2b0 2024-02-15T18:22:09.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:09.567+0000 7f55346ec700 1 -- 172.21.15.191:0/3030800930 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f551805c2b0 msgr2=0x7f551805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:09.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:09.567+0000 7f55346ec700 1 --2- 172.21.15.191:0/3030800930 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f551805c2b0 0x7f551805e770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f551c009f50 tx=0x7f551c009b30).stop 2024-02-15T18:22:09.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:09.567+0000 7f55346ec700 1 -- 172.21.15.191:0/3030800930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f552c0f55c0 msgr2=0x7f552c109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:09.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:09.567+0000 7f55346ec700 1 --2- 172.21.15.191:0/3030800930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f552c0f55c0 0x7f552c109d90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f55280093f0 tx=0x7f5528006ec0).stop 2024-02-15T18:22:09.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:09.567+0000 7f55346ec700 1 -- 172.21.15.191:0/3030800930 shutdown_connections 2024-02-15T18:22:09.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:09.567+0000 7f55346ec700 1 --2- 172.21.15.191:0/3030800930 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f551805c2b0 0x7f551805e770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:09.568 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:09.567+0000 7f55346ec700 1 --2- 172.21.15.191:0/3030800930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f552c0f55c0 0x7f552c109d90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:09.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:09.567+0000 7f55346ec700 1 --2- 172.21.15.191:0/3030800930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f552c0f6860 0x7f552c10c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:09.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:09.567+0000 7f55346ec700 1 -- 172.21.15.191:0/3030800930 >> 172.21.15.191:0/3030800930 conn(0x7f552c0f0d40 msgr2=0x7f552c0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:09.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:09.568+0000 7f55346ec700 1 -- 172.21.15.191:0/3030800930 shutdown_connections 2024-02-15T18:22:09.569 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:09.568+0000 7f55346ec700 1 -- 172.21.15.191:0/3030800930 wait complete. 2024-02-15T18:22:10.234 DEBUG:teuthology.orchestra.run.smithi191:osd.5> sudo journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.5.service 2024-02-15T18:22:10.236 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-02-15T18:22:10.237 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-15T18:22:10.268 INFO:journalctl@ceph.osd.5.smithi191.stdout:-- Logs begin at Thu 2024-02-15 17:54:16 UTC. -- 2024-02-15T18:22:10.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:08.800785+0000 mgr.smithi042.kfoihb (mgr.14162) 181 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:10.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:09.553600+0000 mon.smithi042 (mon.0) 409 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:22:10.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:09.558378+0000 mon.smithi042 (mon.0) 410 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:10.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:09.565566+0000 mon.smithi042 (mon.0) 411 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:10.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:08.800785+0000 mgr.smithi042.kfoihb (mgr.14162) 181 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:10.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:09.553600+0000 mon.smithi042 (mon.0) 409 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:22:10.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:09.558378+0000 mon.smithi042 (mon.0) 410 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:10.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:09.565566+0000 mon.smithi042 (mon.0) 411 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:12.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:10.801111+0000 mgr.smithi042.kfoihb (mgr.14162) 182 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:12.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:12.185820+0000 mon.smithi042 (mon.0) 412 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:12.636 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:10.801111+0000 mgr.smithi042.kfoihb (mgr.14162) 182 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:12.636 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:12.185820+0000 mon.smithi042 (mon.0) 412 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:12.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.892+0000 7f601c607700 1 -- 172.21.15.42:0/4237030804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60140f7980 msgr2=0x7f60140f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:12.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.892+0000 7f601c607700 1 --2- 172.21.15.42:0/4237030804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60140f7980 0x7f60140f7de0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f6008004660 tx=0x7f6008009b30).stop 2024-02-15T18:22:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.892+0000 7f601c607700 1 -- 172.21.15.42:0/4237030804 shutdown_connections 2024-02-15T18:22:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.892+0000 7f601c607700 1 --2- 172.21.15.42:0/4237030804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60140f7980 0x7f60140f7de0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.892+0000 7f601c607700 1 --2- 172.21.15.42:0/4237030804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60140f66e0 0x7f60140f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.892+0000 7f601c607700 1 -- 172.21.15.42:0/4237030804 >> 172.21.15.42:0/4237030804 conn(0x7f60140f1e40 msgr2=0x7f60140f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.892+0000 7f601c607700 1 -- 172.21.15.42:0/4237030804 shutdown_connections 2024-02-15T18:22:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.892+0000 7f601c607700 1 -- 172.21.15.42:0/4237030804 wait complete. 2024-02-15T18:22:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.893+0000 7f601c607700 1 Processor -- start 2024-02-15T18:22:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.893+0000 7f601c607700 1 -- start start 2024-02-15T18:22:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.893+0000 7f601c607700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60140f66e0 0x7f601406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.893+0000 7f601c607700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60140f7980 0x7f6014069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.893+0000 7f601c607700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601406a380 con 0x7f60140f7980 2024-02-15T18:22:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.893+0000 7f601c607700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601406a4c0 con 0x7f60140f66e0 2024-02-15T18:22:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.893+0000 7f6019ba2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60140f7980 0x7f6014069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.893+0000 7f6019ba2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60140f7980 0x7f6014069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51906/0 (socket says 172.21.15.42:51906) 2024-02-15T18:22:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.893+0000 7f6019ba2700 1 -- 172.21.15.42:0/3306877246 learned_addr learned my addr 172.21.15.42:0/3306877246 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.893+0000 7f601a3a3700 1 --2- 172.21.15.42:0/3306877246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60140f66e0 0x7f601406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.893+0000 7f6019ba2700 1 -- 172.21.15.42:0/3306877246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60140f66e0 msgr2=0x7f601406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.893+0000 7f6019ba2700 1 --2- 172.21.15.42:0/3306877246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60140f66e0 0x7f601406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.893+0000 7f6019ba2700 1 -- 172.21.15.42:0/3306877246 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6008005040 con 0x7f60140f7980 2024-02-15T18:22:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.893+0000 7f601a3a3700 1 --2- 172.21.15.42:0/3306877246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60140f66e0 0x7f601406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:22:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.894+0000 7f6019ba2700 1 --2- 172.21.15.42:0/3306877246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60140f7980 0x7f6014069e20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f6008009830 tx=0x7f6008009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.894+0000 7f6006ffd700 1 -- 172.21.15.42:0/3306877246 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6008018070 con 0x7f60140f7980 2024-02-15T18:22:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.894+0000 7f601c607700 1 -- 172.21.15.42:0/3306877246 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601406a740 con 0x7f60140f7980 2024-02-15T18:22:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.894+0000 7f6006ffd700 1 -- 172.21.15.42:0/3306877246 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6008007d70 con 0x7f60140f7980 2024-02-15T18:22:12.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.894+0000 7f6006ffd700 1 -- 172.21.15.42:0/3306877246 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f600801c620 con 0x7f60140f7980 2024-02-15T18:22:12.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.894+0000 7f601c607700 1 -- 172.21.15.42:0/3306877246 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601406aca0 con 0x7f60140f7980 2024-02-15T18:22:12.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.895+0000 7f601c607700 1 -- 172.21.15.42:0/3306877246 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6014045ad0 con 0x7f60140f7980 2024-02-15T18:22:12.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.895+0000 7f6006ffd700 1 -- 172.21.15.42:0/3306877246 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f600801d980 con 0x7f60140f7980 2024-02-15T18:22:12.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.896+0000 7f6006ffd700 1 --2- 172.21.15.42:0/3306877246 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f600005c2b0 0x7f600005e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:12.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.896+0000 7f6006ffd700 1 -- 172.21.15.42:0/3306877246 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4235+0+0 (secure 0 0 0) 0x7f6008077b70 con 0x7f60140f7980 2024-02-15T18:22:12.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.896+0000 7f601a3a3700 1 --2- 172.21.15.42:0/3306877246 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f600005c2b0 0x7f600005e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:12.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.896+0000 7f601a3a3700 1 --2- 172.21.15.42:0/3306877246 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f600005c2b0 0x7f600005e770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f6010007fd0 tx=0x7f6010009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:12.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:12.900+0000 7f6006ffd700 1 -- 172.21.15.42:0/3306877246 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f600804ada0 con 0x7f60140f7980 2024-02-15T18:22:13.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:13.061+0000 7f601c607700 1 -- 172.21.15.42:0/3306877246 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f6014027a00 con 0x7f60140f7980 2024-02-15T18:22:13.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:13.061+0000 7f6006ffd700 1 -- 172.21.15.42:0/3306877246 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v35) v1 ==== 74+0+130 (secure 0 0 0) 0x7f600804e3c0 con 0x7f60140f7980 2024-02-15T18:22:13.061 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:22:13.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:13.063+0000 7f601c607700 1 -- 172.21.15.42:0/3306877246 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f600005c2b0 msgr2=0x7f600005e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:13.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:13.063+0000 7f601c607700 1 --2- 172.21.15.42:0/3306877246 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f600005c2b0 0x7f600005e770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f6010007fd0 tx=0x7f6010009040).stop 2024-02-15T18:22:13.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:13.063+0000 7f601c607700 1 -- 172.21.15.42:0/3306877246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60140f7980 msgr2=0x7f6014069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:13.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:13.063+0000 7f601c607700 1 --2- 172.21.15.42:0/3306877246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60140f7980 0x7f6014069e20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f6008009830 tx=0x7f6008009b30).stop 2024-02-15T18:22:13.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:13.063+0000 7f601c607700 1 -- 172.21.15.42:0/3306877246 shutdown_connections 2024-02-15T18:22:13.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:13.063+0000 7f601c607700 1 --2- 172.21.15.42:0/3306877246 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f600005c2b0 0x7f600005e770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:13.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:13.063+0000 7f601c607700 1 --2- 172.21.15.42:0/3306877246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60140f66e0 0x7f601406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:13.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:13.063+0000 7f601c607700 1 --2- 172.21.15.42:0/3306877246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60140f7980 0x7f6014069e20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:13.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:13.063+0000 7f601c607700 1 -- 172.21.15.42:0/3306877246 >> 172.21.15.42:0/3306877246 conn(0x7f60140f1e40 msgr2=0x7f60140fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:13.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:13.063+0000 7f601c607700 1 -- 172.21.15.42:0/3306877246 shutdown_connections 2024-02-15T18:22:13.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:13.063+0000 7f601c607700 1 -- 172.21.15.42:0/3306877246 wait complete. 2024-02-15T18:22:13.559 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:13.062274+0000 mon.smithi042 (mon.0) 413 : audit [DBG] from='client.? 172.21.15.42:0/3306877246' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-15T18:22:13.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:13.062274+0000 mon.smithi042 (mon.0) 413 : audit [DBG] from='client.? 172.21.15.42:0/3306877246' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-15T18:22:13.700 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:22:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.5[52193]: debug 2024-02-15T18:22:13.427+0000 7fea0219d080 -1 osd.5 0 log_to_monitors {default=true} 2024-02-15T18:22:13.869 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":35,"num_osds":6,"num_up_osds":5,"osd_up_since":1708021314,"num_in_osds":6,"osd_in_since":1708021318,"num_remapped_pgs":0} 2024-02-15T18:22:14.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:12.801423+0000 mgr.smithi042.kfoihb (mgr.14162) 183 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:14.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:13.431597+0000 mon.smithi191 (mon.1) 11 : audit [INF] from='osd.5 [v2:172.21.15.191:6816/2014313522,v1:172.21.15.191:6817/2014313522]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:22:14.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:13.431910+0000 mon.smithi042 (mon.0) 414 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:22:14.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:12.801423+0000 mgr.smithi042.kfoihb (mgr.14162) 183 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:14.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:13.431597+0000 mon.smithi191 (mon.1) 11 : audit [INF] from='osd.5 [v2:172.21.15.191:6816/2014313522,v1:172.21.15.191:6817/2014313522]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:22:14.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:13.431910+0000 mon.smithi042 (mon.0) 414 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:22:14.870 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-15T18:22:15.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:14.274080+0000 mon.smithi042 (mon.0) 415 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-15T18:22:15.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:14.274149+0000 mon.smithi042 (mon.0) 416 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-15T18:22:15.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:14.274273+0000 mon.smithi042 (mon.0) 417 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:22:15.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:14.274744+0000 mon.smithi191 (mon.1) 12 : audit [INF] from='osd.5 [v2:172.21.15.191:6816/2014313522,v1:172.21.15.191:6817/2014313522]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:22:15.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:14.275012+0000 mon.smithi042 (mon.0) 418 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:22:15.576 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:22:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.5[52193]: debug 2024-02-15T18:22:15.278+0000 7fe9f870e700 -1 osd.5 0 waiting for initial osdmap 2024-02-15T18:22:15.576 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:22:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.5[52193]: debug 2024-02-15T18:22:15.285+0000 7fe9f2e86700 -1 osd.5 37 set_numa_affinity unable to identify public interface 'enp3s0f1' numa node: (0) Success 2024-02-15T18:22:15.576 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:14.274080+0000 mon.smithi042 (mon.0) 415 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-15T18:22:15.576 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:14.274149+0000 mon.smithi042 (mon.0) 416 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-15T18:22:15.576 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:14.274273+0000 mon.smithi042 (mon.0) 417 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:22:15.576 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:14.274744+0000 mon.smithi191 (mon.1) 12 : audit [INF] from='osd.5 [v2:172.21.15.191:6816/2014313522,v1:172.21.15.191:6817/2014313522]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:22:15.576 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:14.275012+0000 mon.smithi042 (mon.0) 418 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:22:16.475 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:14.801800+0000 mgr.smithi042.kfoihb (mgr.14162) 184 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:16.476 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:15.275652+0000 mon.smithi042 (mon.0) 419 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-15T18:22:16.476 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:15.275708+0000 mon.smithi042 (mon.0) 420 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2024-02-15T18:22:16.476 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:15.276051+0000 mon.smithi042 (mon.0) 421 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:22:16.476 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:15.279150+0000 mon.smithi042 (mon.0) 422 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:22:16.476 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:15.800874+0000 mon.smithi042 (mon.0) 423 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:14.801800+0000 mgr.smithi042.kfoihb (mgr.14162) 184 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:15.275652+0000 mon.smithi042 (mon.0) 419 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-15T18:22:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:15.275708+0000 mon.smithi042 (mon.0) 420 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2024-02-15T18:22:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:15.276051+0000 mon.smithi042 (mon.0) 421 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:22:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:15.279150+0000 mon.smithi042 (mon.0) 422 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:22:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:15.800874+0000 mon.smithi042 (mon.0) 423 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:17.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:14.397601+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:22:17.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:14.397698+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:22:17.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:22:15.796594+0000 mgr.smithi042.kfoihb (mgr.14162) 185 : cephadm [INF] Detected new or changed devices on smithi191 2024-02-15T18:22:17.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:16.281390+0000 mon.smithi042 (mon.0) 424 : cluster [INF] osd.5 [v2:172.21.15.191:6816/2014313522,v1:172.21.15.191:6817/2014313522] boot 2024-02-15T18:22:17.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:16.281443+0000 mon.smithi042 (mon.0) 425 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-02-15T18:22:17.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:16.281797+0000 mon.smithi042 (mon.0) 426 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:22:17.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.319+0000 7feff759e700 1 -- 172.21.15.42:0/3673053975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feff80f5440 msgr2=0x7feff80f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:17.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.319+0000 7feff759e700 1 --2- 172.21.15.42:0/3673053975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feff80f5440 0x7feff80f5860 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fefe8004660 tx=0x7fefe8009b30).stop 2024-02-15T18:22:17.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.319+0000 7feff759e700 1 -- 172.21.15.42:0/3673053975 shutdown_connections 2024-02-15T18:22:17.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.319+0000 7feff759e700 1 --2- 172.21.15.42:0/3673053975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff80f66e0 0x7feff80f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:17.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.319+0000 7feff759e700 1 --2- 172.21.15.42:0/3673053975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feff80f5440 0x7feff80f5860 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:17.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.319+0000 7feff759e700 1 -- 172.21.15.42:0/3673053975 >> 172.21.15.42:0/3673053975 conn(0x7feff80f0ba0 msgr2=0x7feff80f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:17.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.319+0000 7feff759e700 1 -- 172.21.15.42:0/3673053975 shutdown_connections 2024-02-15T18:22:17.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.319+0000 7feff759e700 1 -- 172.21.15.42:0/3673053975 wait complete. 2024-02-15T18:22:17.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.319+0000 7feff759e700 1 Processor -- start 2024-02-15T18:22:17.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.319+0000 7feff759e700 1 -- start start 2024-02-15T18:22:17.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.319+0000 7feff759e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feff80f5440 0x7feff8106ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:17.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.319+0000 7feff759e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff80f66e0 0x7feff8107390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:17.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.319+0000 7feff759e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff8107940 con 0x7feff80f5440 2024-02-15T18:22:17.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.319+0000 7feff759e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff8108140 con 0x7feff80f66e0 2024-02-15T18:22:17.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.320+0000 7feff659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feff80f5440 0x7feff8106ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:17.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.320+0000 7feff659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feff80f5440 0x7feff8106ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51916/0 (socket says 172.21.15.42:51916) 2024-02-15T18:22:17.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.320+0000 7feff659c700 1 -- 172.21.15.42:0/1293345811 learned_addr learned my addr 172.21.15.42:0/1293345811 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:17.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.320+0000 7feff5d9b700 1 --2- 172.21.15.42:0/1293345811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff80f66e0 0x7feff8107390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:17.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.320+0000 7feff659c700 1 -- 172.21.15.42:0/1293345811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff80f66e0 msgr2=0x7feff8107390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:17.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.320+0000 7feff659c700 1 --2- 172.21.15.42:0/1293345811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff80f66e0 0x7feff8107390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:17.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.320+0000 7feff659c700 1 -- 172.21.15.42:0/1293345811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe8005040 con 0x7feff80f5440 2024-02-15T18:22:17.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.320+0000 7feff659c700 1 --2- 172.21.15.42:0/1293345811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feff80f5440 0x7feff8106ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fefe8002040 tx=0x7fefe800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:17.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.320+0000 7fefe6ffd700 1 -- 172.21.15.42:0/1293345811 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefe8019070 con 0x7feff80f5440 2024-02-15T18:22:17.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.320+0000 7feff759e700 1 -- 172.21.15.42:0/1293345811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff81083c0 con 0x7feff80f5440 2024-02-15T18:22:17.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.320+0000 7fefe6ffd700 1 -- 172.21.15.42:0/1293345811 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fefe8007e90 con 0x7feff80f5440 2024-02-15T18:22:17.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.320+0000 7fefe6ffd700 1 -- 172.21.15.42:0/1293345811 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefe8014d20 con 0x7feff80f5440 2024-02-15T18:22:17.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.320+0000 7feff759e700 1 -- 172.21.15.42:0/1293345811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff8108920 con 0x7feff80f5440 2024-02-15T18:22:17.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.321+0000 7feff759e700 1 -- 172.21.15.42:0/1293345811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff8059100 con 0x7feff80f5440 2024-02-15T18:22:17.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.322+0000 7fefe6ffd700 1 -- 172.21.15.42:0/1293345811 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fefe8025070 con 0x7feff80f5440 2024-02-15T18:22:17.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.322+0000 7fefe6ffd700 1 --2- 172.21.15.42:0/1293345811 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fefe005c350 0x7fefe005e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:17.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.322+0000 7fefe6ffd700 1 -- 172.21.15.42:0/1293345811 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fefe8076b60 con 0x7feff80f5440 2024-02-15T18:22:17.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.325+0000 7feff5d9b700 1 --2- 172.21.15.42:0/1293345811 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fefe005c350 0x7fefe005e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:17.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.325+0000 7feff5d9b700 1 --2- 172.21.15.42:0/1293345811 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fefe005c350 0x7fefe005e810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fefec009fd0 tx=0x7fefec00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:17.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.327+0000 7fefe6ffd700 1 -- 172.21.15.42:0/1293345811 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fefe8049aa0 con 0x7feff80f5440 2024-02-15T18:22:17.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.494+0000 7feff759e700 1 -- 172.21.15.42:0/1293345811 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7feff810d380 con 0x7feff80f5440 2024-02-15T18:22:17.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.495+0000 7fefe6ffd700 1 -- 172.21.15.42:0/1293345811 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7fefe804d0c0 con 0x7feff80f5440 2024-02-15T18:22:17.495 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:22:17.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.496+0000 7feff759e700 1 -- 172.21.15.42:0/1293345811 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fefe005c350 msgr2=0x7fefe005e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:17.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.497+0000 7feff759e700 1 --2- 172.21.15.42:0/1293345811 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fefe005c350 0x7fefe005e810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fefec009fd0 tx=0x7fefec00b040).stop 2024-02-15T18:22:17.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.497+0000 7feff759e700 1 -- 172.21.15.42:0/1293345811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feff80f5440 msgr2=0x7feff8106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:17.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.497+0000 7feff759e700 1 --2- 172.21.15.42:0/1293345811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feff80f5440 0x7feff8106ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fefe8002040 tx=0x7fefe800b040).stop 2024-02-15T18:22:17.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.497+0000 7feff759e700 1 -- 172.21.15.42:0/1293345811 shutdown_connections 2024-02-15T18:22:17.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.497+0000 7feff759e700 1 --2- 172.21.15.42:0/1293345811 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fefe005c350 0x7fefe005e810 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:17.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.497+0000 7feff759e700 1 --2- 172.21.15.42:0/1293345811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feff80f5440 0x7feff8106ea0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:17.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.497+0000 7feff759e700 1 --2- 172.21.15.42:0/1293345811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feff80f66e0 0x7feff8107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:17.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.497+0000 7feff759e700 1 -- 172.21.15.42:0/1293345811 >> 172.21.15.42:0/1293345811 conn(0x7feff80f0ba0 msgr2=0x7feff805f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:17.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.497+0000 7feff759e700 1 -- 172.21.15.42:0/1293345811 shutdown_connections 2024-02-15T18:22:17.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:17.497+0000 7feff759e700 1 -- 172.21.15.42:0/1293345811 wait complete. 2024-02-15T18:22:17.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:14.397601+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-15T18:22:17.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:14.397698+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-15T18:22:17.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:22:15.796594+0000 mgr.smithi042.kfoihb (mgr.14162) 185 : cephadm [INF] Detected new or changed devices on smithi191 2024-02-15T18:22:17.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:16.281390+0000 mon.smithi042 (mon.0) 424 : cluster [INF] osd.5 [v2:172.21.15.191:6816/2014313522,v1:172.21.15.191:6817/2014313522] boot 2024-02-15T18:22:17.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:16.281443+0000 mon.smithi042 (mon.0) 425 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-02-15T18:22:17.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:16.281797+0000 mon.smithi042 (mon.0) 426 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:22:18.272 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":39,"num_osds":6,"num_up_osds":6,"osd_up_since":1708021336,"num_in_osds":6,"osd_in_since":1708021318,"num_remapped_pgs":0} 2024-02-15T18:22:18.273 INFO:tasks.cephadm:Setting up client nodes... 2024-02-15T18:22:18.273 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-15T18:22:18.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:16.802183+0000 mgr.smithi042.kfoihb (mgr.14162) 186 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:18.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:17.291268+0000 mon.smithi042 (mon.0) 427 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-15T18:22:18.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:17.406703+0000 mon.smithi042 (mon.0) 428 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:18.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:17.410497+0000 mon.smithi042 (mon.0) 429 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:18.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:17.412792+0000 mon.smithi042 (mon.0) 430 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:22:18.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:17.496111+0000 mon.smithi042 (mon.0) 431 : audit [DBG] from='client.? 172.21.15.42:0/1293345811' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-15T18:22:18.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:16.802183+0000 mgr.smithi042.kfoihb (mgr.14162) 186 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-02-15T18:22:18.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:17.291268+0000 mon.smithi042 (mon.0) 427 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-15T18:22:18.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:17.406703+0000 mon.smithi042 (mon.0) 428 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:18.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:17.410497+0000 mon.smithi042 (mon.0) 429 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:18.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:17.412792+0000 mon.smithi042 (mon.0) 430 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:22:18.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:17.496111+0000 mon.smithi042 (mon.0) 431 : audit [DBG] from='client.? 172.21.15.42:0/1293345811' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-15T18:22:20.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:18.802553+0000 mgr.smithi042.kfoihb (mgr.14162) 187 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:20.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:18.802553+0000 mgr.smithi042.kfoihb (mgr.14162) 187 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:21.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.161+0000 7f50a6958700 1 -- 172.21.15.42:0/2304303532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a0060270 msgr2=0x7f50a00f9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:21.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.161+0000 7f50a6958700 1 --2- 172.21.15.42:0/2304303532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a0060270 0x7f50a00f9330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f5090004660 tx=0x7f5090009b30).stop 2024-02-15T18:22:21.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.161+0000 7f50a6958700 1 -- 172.21.15.42:0/2304303532 shutdown_connections 2024-02-15T18:22:21.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.161+0000 7f50a6958700 1 --2- 172.21.15.42:0/2304303532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50a00f9820 0x7f50a00fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:22:21.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.161+0000 7f50a6958700 1 --2- 172.21.15.42:0/2304303532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a0060270 0x7f50a00f9330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:21.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.161+0000 7f50a6958700 1 -- 172.21.15.42:0/2304303532 >> 172.21.15.42:0/2304303532 conn(0x7f50a00f0d10 msgr2=0x7f50a00f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:21.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.161+0000 7f50a6958700 1 -- 172.21.15.42:0/2304303532 shutdown_connections 2024-02-15T18:22:21.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.162+0000 7f50a6958700 1 -- 172.21.15.42:0/2304303532 wait complete. 2024-02-15T18:22:21.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.162+0000 7f50a6958700 1 Processor -- start 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.162+0000 7f50a6958700 1 -- start start 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f50a6958700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50a0060270 0x7f50a0107040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f50a6958700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a00f9820 0x7f50a0107530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f50a6958700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a0107b70 con 0x7f50a00f9820 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f50a6958700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a0108420 con 0x7f50a0060270 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f509f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a00f9820 0x7f50a0107530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f509f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a00f9820 0x7f50a0107530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51936/0 (socket says 172.21.15.42:51936) 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f509f7fe700 1 -- 172.21.15.42:0/3197895236 learned_addr learned my addr 172.21.15.42:0/3197895236 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f509ffff700 1 --2- 172.21.15.42:0/3197895236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50a0060270 0x7f50a0107040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f509f7fe700 1 -- 172.21.15.42:0/3197895236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50a0060270 msgr2=0x7f50a0107040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f509f7fe700 1 --2- 172.21.15.42:0/3197895236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50a0060270 0x7f50a0107040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f509f7fe700 1 -- 172.21.15.42:0/3197895236 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5090005040 con 0x7f50a00f9820 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f509f7fe700 1 --2- 172.21.15.42:0/3197895236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a00f9820 0x7f50a0107530 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f50940093f0 tx=0x7f5094006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f509cff9700 1 -- 172.21.15.42:0/3197895236 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5094009ea0 con 0x7f50a00f9820 2024-02-15T18:22:21.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f509cff9700 1 -- 172.21.15.42:0/3197895236 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5094013b10 con 0x7f50a00f9820 2024-02-15T18:22:21.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f50a6958700 1 -- 172.21.15.42:0/3197895236 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a0108700 con 0x7f50a00f9820 2024-02-15T18:22:21.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.163+0000 7f509cff9700 1 -- 172.21.15.42:0/3197895236 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f509400dbc0 con 0x7f50a00f9820 2024-02-15T18:22:21.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.164+0000 7f50a6958700 1 -- 172.21.15.42:0/3197895236 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a0108cc0 con 0x7f50a00f9820 2024-02-15T18:22:21.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.165+0000 7f50a6958700 1 -- 172.21.15.42:0/3197895236 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a00f49a0 con 0x7f50a00f9820 2024-02-15T18:22:21.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.165+0000 7f509cff9700 1 -- 172.21.15.42:0/3197895236 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f509401a060 con 0x7f50a00f9820 2024-02-15T18:22:21.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.165+0000 7f509cff9700 1 --2- 172.21.15.42:0/3197895236 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f508805c2b0 0x7f508805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:21.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.165+0000 7f509cff9700 1 -- 172.21.15.42:0/3197895236 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f5094075c30 con 0x7f50a00f9820 2024-02-15T18:22:21.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.165+0000 7f509ffff700 1 --2- 172.21.15.42:0/3197895236 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f508805c2b0 0x7f508805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:21.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.166+0000 7f509ffff700 1 --2- 172.21.15.42:0/3197895236 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f508805c2b0 0x7f508805e770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f5090004630 tx=0x7f5090009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:21.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.169+0000 7f509cff9700 1 -- 172.21.15.42:0/3197895236 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f509404c600 con 0x7f50a00f9820 2024-02-15T18:22:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.387+0000 7f50a6958700 1 -- 172.21.15.42:0/3197895236 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f50a00ff030 con 0x7f50a00f9820 2024-02-15T18:22:21.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.390+0000 7f509cff9700 1 -- 172.21.15.42:0/3197895236 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f509404c1f0 con 0x7f50a00f9820 2024-02-15T18:22:21.391 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-02-15T18:22:21.391 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQBdVs5l6UEqFxAAhcNZ0B0rrHOyUvcGKEviVA== 2024-02-15T18:22:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.392+0000 7f50a6958700 1 -- 172.21.15.42:0/3197895236 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f508805c2b0 msgr2=0x7f508805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.392+0000 7f50a6958700 1 --2- 172.21.15.42:0/3197895236 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f508805c2b0 0x7f508805e770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f5090004630 tx=0x7f5090009b30).stop 2024-02-15T18:22:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.392+0000 7f50a6958700 1 -- 172.21.15.42:0/3197895236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a00f9820 msgr2=0x7f50a0107530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.392+0000 7f50a6958700 1 --2- 172.21.15.42:0/3197895236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a00f9820 0x7f50a0107530 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f50940093f0 tx=0x7f5094006ec0).stop 2024-02-15T18:22:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.392+0000 7f50a6958700 1 -- 172.21.15.42:0/3197895236 shutdown_connections 2024-02-15T18:22:21.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.392+0000 7f50a6958700 1 --2- 172.21.15.42:0/3197895236 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f508805c2b0 0x7f508805e770 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:21.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.392+0000 7f50a6958700 1 --2- 172.21.15.42:0/3197895236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f50a0060270 0x7f50a0107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:21.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.392+0000 7f50a6958700 1 --2- 172.21.15.42:0/3197895236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a00f9820 0x7f50a0107530 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:21.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.392+0000 7f50a6958700 1 -- 172.21.15.42:0/3197895236 >> 172.21.15.42:0/3197895236 conn(0x7f50a00f0d10 msgr2=0x7f50a00f3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:21.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.392+0000 7f50a6958700 1 -- 172.21.15.42:0/3197895236 shutdown_connections 2024-02-15T18:22:21.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:21.392+0000 7f50a6958700 1 -- 172.21.15.42:0/3197895236 wait complete. 2024-02-15T18:22:22.195 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:22:22.195 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-15T18:22:22.195 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-15T18:22:22.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-15T18:22:22.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:20.803052+0000 mgr.smithi042.kfoihb (mgr.14162) 188 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:22.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:21.388499+0000 mon.smithi042 (mon.0) 432 : audit [INF] from='client.? 172.21.15.42:0/3197895236' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-15T18:22:22.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:21.391581+0000 mon.smithi042 (mon.0) 433 : audit [INF] from='client.? 172.21.15.42:0/3197895236' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-15T18:22:22.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:20.803052+0000 mgr.smithi042.kfoihb (mgr.14162) 188 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:22.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:21.388499+0000 mon.smithi042 (mon.0) 432 : audit [INF] from='client.? 172.21.15.42:0/3197895236' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-15T18:22:22.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:21.391581+0000 mon.smithi042 (mon.0) 433 : audit [INF] from='client.? 172.21.15.42:0/3197895236' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-15T18:22:24.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.197+0000 7f17c6667700 1 -- 172.21.15.191:0/874931459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17c00f5ed0 msgr2=0x7f17c00f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:24.199 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.197+0000 7f17c6667700 1 --2- 172.21.15.191:0/874931459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17c00f5ed0 0x7f17c00f6900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f17b0004660 tx=0x7f17b0009b30).stop 2024-02-15T18:22:24.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.198+0000 7f17c6667700 1 -- 172.21.15.191:0/874931459 shutdown_connections 2024-02-15T18:22:24.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.198+0000 7f17c6667700 1 --2- 172.21.15.191:0/874931459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17c00f5ed0 0x7f17c00f6900 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:24.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.198+0000 7f17c6667700 1 --2- 172.21.15.191:0/874931459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17c00f55c0 0x7f17c00f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:24.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.198+0000 7f17c6667700 1 -- 172.21.15.191:0/874931459 >> 172.21.15.191:0/874931459 conn(0x7f17c00f0de0 msgr2=0x7f17c00f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:24.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.198+0000 7f17c6667700 1 -- 172.21.15.191:0/874931459 shutdown_connections 2024-02-15T18:22:24.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.198+0000 7f17c6667700 1 -- 172.21.15.191:0/874931459 wait complete. 2024-02-15T18:22:24.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.198+0000 7f17c6667700 1 Processor -- start 2024-02-15T18:22:24.200 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.198+0000 7f17c6667700 1 -- start start 2024-02-15T18:22:24.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17c6667700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17c00f55c0 0x7f17c0105f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:24.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17c6667700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17c00f5ed0 0x7f17c0106450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:24.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17c6667700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0106a00 con 0x7f17c00f55c0 2024-02-15T18:22:24.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17c6667700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0106b40 con 0x7f17c00f5ed0 2024-02-15T18:22:24.202 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17bf7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17c00f5ed0 0x7f17c0106450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:24.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17bf7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17c00f5ed0 0x7f17c0106450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:38536/0 (socket says 172.21.15.191:38536) 2024-02-15T18:22:24.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17bf7fe700 1 -- 172.21.15.191:0/3622865584 learned_addr learned my addr 172.21.15.191:0/3622865584 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:22:24.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17bffff700 1 --2- 172.21.15.191:0/3622865584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17c00f55c0 0x7f17c0105f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:24.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17bf7fe700 1 -- 172.21.15.191:0/3622865584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17c00f55c0 msgr2=0x7f17c0105f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:24.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17bf7fe700 1 --2- 172.21.15.191:0/3622865584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17c00f55c0 0x7f17c0105f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:24.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17bf7fe700 1 -- 172.21.15.191:0/3622865584 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b0005040 con 0x7f17c00f5ed0 2024-02-15T18:22:24.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17bf7fe700 1 --2- 172.21.15.191:0/3622865584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17c00f5ed0 0x7f17c0106450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f17b0009830 tx=0x7f17b0009b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:24.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17bcff9700 1 -- 172.21.15.191:0/3622865584 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17b0018070 con 0x7f17c00f5ed0 2024-02-15T18:22:24.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17c6667700 1 -- 172.21.15.191:0/3622865584 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c0107480 con 0x7f17c00f5ed0 2024-02-15T18:22:24.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17bcff9700 1 -- 172.21.15.191:0/3622865584 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17b0007c70 con 0x7f17c00f5ed0 2024-02-15T18:22:24.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17bcff9700 1 -- 172.21.15.191:0/3622865584 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17b001c620 con 0x7f17c00f5ed0 2024-02-15T18:22:24.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.199+0000 7f17c6667700 1 -- 172.21.15.191:0/3622865584 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c01079e0 con 0x7f17c00f5ed0 2024-02-15T18:22:24.203 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.200+0000 7f17c6667700 1 -- 172.21.15.191:0/3622865584 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c0045ad0 con 0x7f17c00f5ed0 2024-02-15T18:22:24.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.201+0000 7f17bcff9700 1 -- 172.21.15.191:0/3622865584 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f17b001d420 con 0x7f17c00f5ed0 2024-02-15T18:22:24.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.201+0000 7f17bcff9700 1 --2- 172.21.15.191:0/3622865584 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f17ac05c2b0 0x7f17ac05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:24.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.201+0000 7f17bcff9700 1 -- 172.21.15.191:0/3622865584 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f17b00779c0 con 0x7f17c00f5ed0 2024-02-15T18:22:24.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.201+0000 7f17bffff700 1 --2- 172.21.15.191:0/3622865584 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f17ac05c2b0 0x7f17ac05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:24.204 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.202+0000 7f17bffff700 1 --2- 172.21.15.191:0/3622865584 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f17ac05c2b0 0x7f17ac05e770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f17a80041d0 tx=0x7f17a8009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:24.206 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.205+0000 7f17bcff9700 1 -- 172.21.15.191:0/3622865584 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f17b004a9c0 con 0x7f17c00f5ed0 2024-02-15T18:22:24.412 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.411+0000 7f17c6667700 1 -- 172.21.15.191:0/3622865584 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f17c0027a00 con 0x7f17c00f5ed0 2024-02-15T18:22:24.416 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.415+0000 7f17bcff9700 1 -- 172.21.15.191:0/3622865584 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f17b004dfe0 con 0x7f17c00f5ed0 2024-02-15T18:22:24.416 INFO:teuthology.orchestra.run.smithi191.stdout:[client.1] 2024-02-15T18:22:24.416 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQBgVs5ln/SrGBAAoKS3t6mFoP5hD6R/l1+oLA== 2024-02-15T18:22:24.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.417+0000 7f17c6667700 1 -- 172.21.15.191:0/3622865584 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f17ac05c2b0 msgr2=0x7f17ac05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:24.418 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.417+0000 7f17c6667700 1 --2- 172.21.15.191:0/3622865584 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f17ac05c2b0 0x7f17ac05e770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f17a80041d0 tx=0x7f17a8009040).stop 2024-02-15T18:22:24.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.417+0000 7f17c6667700 1 -- 172.21.15.191:0/3622865584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17c00f5ed0 msgr2=0x7f17c0106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:24.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.417+0000 7f17c6667700 1 --2- 172.21.15.191:0/3622865584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17c00f5ed0 0x7f17c0106450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f17b0009830 tx=0x7f17b0009b30).stop 2024-02-15T18:22:24.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.417+0000 7f17c6667700 1 -- 172.21.15.191:0/3622865584 shutdown_connections 2024-02-15T18:22:24.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.417+0000 7f17c6667700 1 --2- 172.21.15.191:0/3622865584 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f17ac05c2b0 0x7f17ac05e770 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:24.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.417+0000 7f17c6667700 1 --2- 172.21.15.191:0/3622865584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17c00f55c0 0x7f17c0105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:24.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.417+0000 7f17c6667700 1 --2- 172.21.15.191:0/3622865584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17c00f5ed0 0x7f17c0106450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:24.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.417+0000 7f17c6667700 1 -- 172.21.15.191:0/3622865584 >> 172.21.15.191:0/3622865584 conn(0x7f17c00f0de0 msgr2=0x7f17c00fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:24.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.417+0000 7f17c6667700 1 -- 172.21.15.191:0/3622865584 shutdown_connections 2024-02-15T18:22:24.419 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:22:24.417+0000 7f17c6667700 1 -- 172.21.15.191:0/3622865584 wait complete. 2024-02-15T18:22:24.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:22.803497+0000 mgr.smithi042.kfoihb (mgr.14162) 189 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:24.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:22.803497+0000 mgr.smithi042.kfoihb (mgr.14162) 189 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:25.029 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-15T18:22:25.029 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-15T18:22:25.029 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-15T18:22:25.068 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-15T18:22:25.304 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:22:25.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:24.413153+0000 mon.smithi191 (mon.1) 13 : audit [INF] from='client.? 172.21.15.191:0/3622865584' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-15T18:22:25.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:24.413744+0000 mon.smithi042 (mon.0) 434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-15T18:22:25.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:24.417154+0000 mon.smithi042 (mon.0) 435 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-15T18:22:25.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:24.725144+0000 mon.smithi042 (mon.0) 436 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:25.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:24.727939+0000 mon.smithi042 (mon.0) 437 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:25.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:24.413153+0000 mon.smithi191 (mon.1) 13 : audit [INF] from='client.? 172.21.15.191:0/3622865584' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-15T18:22:25.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:24.413744+0000 mon.smithi042 (mon.0) 434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-15T18:22:25.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:24.417154+0000 mon.smithi042 (mon.0) 435 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-15T18:22:25.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:24.725144+0000 mon.smithi042 (mon.0) 436 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:25.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:24.727939+0000 mon.smithi042 (mon.0) 437 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:22:26.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:24.803846+0000 mgr.smithi042.kfoihb (mgr.14162) 190 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:26.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:24.803846+0000 mgr.smithi042.kfoihb (mgr.14162) 190 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.122+0000 7f5bb1dec700 1 -- 172.21.15.42:0/1974993785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bac0f67d0 msgr2=0x7f5bac0f6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.122+0000 7f5bb1dec700 1 --2- 172.21.15.42:0/1974993785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bac0f67d0 0x7f5bac0f6bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f5b9c004660 tx=0x7f5b9c009b30).stop 2024-02-15T18:22:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.122+0000 7f5bb1dec700 1 -- 172.21.15.42:0/1974993785 shutdown_connections 2024-02-15T18:22:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.122+0000 7f5bb1dec700 1 --2- 172.21.15.42:0/1974993785 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bac0f7a70 0x7f5bac0f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.122+0000 7f5bb1dec700 1 --2- 172.21.15.42:0/1974993785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bac0f67d0 0x7f5bac0f6bf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.122+0000 7f5bb1dec700 1 -- 172.21.15.42:0/1974993785 >> 172.21.15.42:0/1974993785 conn(0x7f5bac0f1f50 msgr2=0x7f5bac0f4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.122+0000 7f5bb1dec700 1 -- 172.21.15.42:0/1974993785 shutdown_connections 2024-02-15T18:22:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.122+0000 7f5bb1dec700 1 -- 172.21.15.42:0/1974993785 wait complete. 2024-02-15T18:22:27.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.123+0000 7f5bb1dec700 1 Processor -- start 2024-02-15T18:22:27.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.123+0000 7f5bb1dec700 1 -- start start 2024-02-15T18:22:27.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.123+0000 7f5bb1dec700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bac0f67d0 0x7f5bac107130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:27.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.123+0000 7f5bb1dec700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bac0f7a70 0x7f5bac107620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:27.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.123+0000 7f5bb1dec700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac107bd0 con 0x7f5bac0f7a70 2024-02-15T18:22:27.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.123+0000 7f5bb1dec700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac107d10 con 0x7f5bac0f67d0 2024-02-15T18:22:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.123+0000 7f5baaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bac0f7a70 0x7f5bac107620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.123+0000 7f5baaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bac0f7a70 0x7f5bac107620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57810/0 (socket says 172.21.15.42:57810) 2024-02-15T18:22:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.123+0000 7f5baaffd700 1 -- 172.21.15.42:0/2107319724 learned_addr learned my addr 172.21.15.42:0/2107319724 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.123+0000 7f5bab7fe700 1 --2- 172.21.15.42:0/2107319724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bac0f67d0 0x7f5bac107130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.124+0000 7f5baaffd700 1 -- 172.21.15.42:0/2107319724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bac0f67d0 msgr2=0x7f5bac107130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.124+0000 7f5baaffd700 1 --2- 172.21.15.42:0/2107319724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bac0f67d0 0x7f5bac107130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.124+0000 7f5baaffd700 1 -- 172.21.15.42:0/2107319724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c005040 con 0x7f5bac0f7a70 2024-02-15T18:22:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.124+0000 7f5baaffd700 1 --2- 172.21.15.42:0/2107319724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bac0f7a70 0x7f5bac107620 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f5b9400e7f0 tx=0x7f5b9400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.124+0000 7f5ba3fff700 1 -- 172.21.15.42:0/2107319724 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b940092e0 con 0x7f5bac0f7a70 2024-02-15T18:22:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.124+0000 7f5bb1dec700 1 -- 172.21.15.42:0/2107319724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bac1086b0 con 0x7f5bac0f7a70 2024-02-15T18:22:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.124+0000 7f5ba3fff700 1 -- 172.21.15.42:0/2107319724 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b9400f040 con 0x7f5bac0f7a70 2024-02-15T18:22:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.124+0000 7f5ba3fff700 1 -- 172.21.15.42:0/2107319724 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b940147e0 con 0x7f5bac0f7a70 2024-02-15T18:22:27.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.124+0000 7f5bb1dec700 1 -- 172.21.15.42:0/2107319724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bac108c70 con 0x7f5bac0f7a70 2024-02-15T18:22:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.125+0000 7f5bb1dec700 1 -- 172.21.15.42:0/2107319724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bac045ad0 con 0x7f5bac0f7a70 2024-02-15T18:22:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.125+0000 7f5ba3fff700 1 -- 172.21.15.42:0/2107319724 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f5b9401b030 con 0x7f5bac0f7a70 2024-02-15T18:22:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.126+0000 7f5ba3fff700 1 --2- 172.21.15.42:0/2107319724 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5b9805c2a0 0x7f5b9805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:27.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.126+0000 7f5ba3fff700 1 -- 172.21.15.42:0/2107319724 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f5b94075990 con 0x7f5bac0f7a70 2024-02-15T18:22:27.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.126+0000 7f5bab7fe700 1 --2- 172.21.15.42:0/2107319724 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5b9805c2a0 0x7f5b9805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:27.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.126+0000 7f5bab7fe700 1 --2- 172.21.15.42:0/2107319724 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5b9805c2a0 0x7f5b9805e760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f5b9c009720 tx=0x7f5b9c016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:27.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.130+0000 7f5ba3fff700 1 -- 172.21.15.42:0/2107319724 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5b94048a40 con 0x7f5bac0f7a70 2024-02-15T18:22:27.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.290+0000 7f5bb1dec700 1 -- 172.21.15.42:0/2107319724 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bac027a00 con 0x7f5bac0f7a70 2024-02-15T18:22:27.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.290+0000 7f5ba3fff700 1 -- 172.21.15.42:0/2107319724 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+10907 (secure 0 0 0) 0x7f5b94019030 con 0x7f5bac0f7a70 2024-02-15T18:22:27.291 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:22:27.291 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":39,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","created":"2024-02-15T18:16:53.406642+0000","modified":"2024-02-15T18:22:17.282788+0000","last_up_change":"2024-02-15T18:22:16.276777+0000","last_in_change":"2024-02-15T18:21:58.584131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-15T18:21:26.655770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"42281c08-af07-43a5-bcfd-d476bc745782","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6803","nonce":3129297361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6805","nonce":3129297361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6809","nonce":3129297361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6807","nonce":3129297361}]},"public_addr":"172.21.15.42:6803/3129297361","cluster_addr":"172.21.15.42:6805/3129297361","heartbeat_back_addr":"172.21.15.42:6809/3129297361","heartbeat_front_addr":"172.21.15.42:6807/3129297361","state":["exists","up"]},{"osd":1,"uuid":"8f67d329-7d44-4361-be02-f8b56b3e5713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6811","nonce":563363965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6813","nonce":563363965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6817","nonce":563363965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6815","nonce":563363965}]},"public_addr":"172.21.15.42:6811/563363965","cluster_addr":"172.21.15.42:6813/563363965","heartbeat_back_addr":"172.21.15.42:6817/563363965","heartbeat_front_addr":"172.21.15.42:6815/563363965","state":["exists","up"]},{"osd":2,"uuid":"680f6437-b450-4353-a761-b50d1388bcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6819","nonce":2706708670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6821","nonce":2706708670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6825","nonce":2706708670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6823","nonce":2706708670}]},"public_addr":"172.21.15.42:6819/2706708670","cluster_addr":"172.21.15.42:6821/2706708670","heartbeat_back_addr":"172.21.15.42:6825/2706708670","heartbeat_front_addr":"172.21.15.42:6823/2706708670","state":["exists","up"]},{"osd":3,"uuid":"6aeea641-3735-4f11-b95f-40c45d4dcd07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6801","nonce":3223849184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6803","nonce":3223849184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6807","nonce":3223849184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6805","nonce":3223849184}]},"public_addr":"172.21.15.191:6801/3223849184","cluster_addr":"172.21.15.191:6803/3223849184","heartbeat_back_addr":"172.21.15.191:6807/3223849184","heartbeat_front_addr":"172.21.15.191:6805/3223849184","state":["exists","up"]},{"osd":4,"uuid":"7d9839d2-bd60-4bae-a16f-ebf56f204acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6809","nonce":424560222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6811","nonce":424560222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6815","nonce":424560222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6813","nonce":424560222}]},"public_addr":"172.21.15.191:6809/424560222","cluster_addr":"172.21.15.191:6811/424560222","heartbeat_back_addr":"172.21.15.191:6815/424560222","heartbeat_front_addr":"172.21.15.191:6813/424560222","state":["exists","up"]},{"osd":5,"uuid":"a75bb280-bd87-4961-918b-18c426f31c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6817","nonce":2014313522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6819","nonce":2014313522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6823","nonce":2014313522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6821","nonce":2014313522}]},"public_addr":"172.21.15.191:6817/2014313522","cluster_addr":"172.21.15.191:6819/2014313522","heartbeat_back_addr":"172.21.15.191:6823/2014313522","heartbeat_front_addr":"172.21.15.191:6821/2014313522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:20:17.479921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:20:47.951852+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:11.894830+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:31.013756+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:52.493759+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:22:14.397700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1703009783":"2024-02-16T18:17:56.705000+0000","172.21.15.42:6801/481027993":"2024-02-16T18:17:56.705000+0000","172.21.15.42:0/1629344114":"2024-02-16T18:17:56.705000+0000","172.21.15.42:0/3157909680":"2024-02-16T18:17:40.441172+0000","172.21.15.42:6801/3051492708":"2024-02-16T18:17:40.441172+0000","172.21.15.42:0/1046171692":"2024-02-16T18:17:18.618949+0000","172.21.15.42:6800/481027993":"2024-02-16T18:17:56.705000+0000","172.21.15.42:6800/3051492708":"2024-02-16T18:17:40.441172+0000","172.21.15.42:6800/1746226634":"2024-02-16T18:17:18.618949+0000","172.21.15.42:0/4044100":"2024-02-16T18:17:18.618949+0000","172.21.15.42:0/4136985555":"2024-02-16T18:17:40.441172+0000","172.21.15.42:6801/1746226634":"2024-02-16T18:17:18.618949+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-15T18:22:27.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.292+0000 7f5bb1dec700 1 -- 172.21.15.42:0/2107319724 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5b9805c2a0 msgr2=0x7f5b9805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:27.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.292+0000 7f5bb1dec700 1 --2- 172.21.15.42:0/2107319724 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5b9805c2a0 0x7f5b9805e760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f5b9c009720 tx=0x7f5b9c016040).stop 2024-02-15T18:22:27.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.292+0000 7f5bb1dec700 1 -- 172.21.15.42:0/2107319724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bac0f7a70 msgr2=0x7f5bac107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:27.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.292+0000 7f5bb1dec700 1 --2- 172.21.15.42:0/2107319724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bac0f7a70 0x7f5bac107620 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f5b9400e7f0 tx=0x7f5b9400b040).stop 2024-02-15T18:22:27.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.292+0000 7f5bb1dec700 1 -- 172.21.15.42:0/2107319724 shutdown_connections 2024-02-15T18:22:27.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.292+0000 7f5bb1dec700 1 --2- 172.21.15.42:0/2107319724 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5b9805c2a0 0x7f5b9805e760 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:27.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.292+0000 7f5bb1dec700 1 --2- 172.21.15.42:0/2107319724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5bac0f67d0 0x7f5bac107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:27.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.292+0000 7f5bb1dec700 1 --2- 172.21.15.42:0/2107319724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bac0f7a70 0x7f5bac107620 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:27.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.292+0000 7f5bb1dec700 1 -- 172.21.15.42:0/2107319724 >> 172.21.15.42:0/2107319724 conn(0x7f5bac0f1f50 msgr2=0x7f5bac0fb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:27.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.292+0000 7f5bb1dec700 1 -- 172.21.15.42:0/2107319724 shutdown_connections 2024-02-15T18:22:27.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:27.292+0000 7f5bb1dec700 1 -- 172.21.15.42:0/2107319724 wait complete. 2024-02-15T18:22:27.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:27.291461+0000 mon.smithi042 (mon.0) 438 : audit [DBG] from='client.? 172.21.15.42:0/2107319724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:22:27.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:27.291461+0000 mon.smithi042 (mon.0) 438 : audit [DBG] from='client.? 172.21.15.42:0/2107319724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:22:28.067 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-15T18:21:26.655770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}] 2024-02-15T18:22:28.067 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph osd pool get device_health_metrics pg_num 2024-02-15T18:22:28.309 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:22:28.558 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:26.804183+0000 mgr.smithi042.kfoihb (mgr.14162) 191 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:28.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:26.804183+0000 mgr.smithi042.kfoihb (mgr.14162) 191 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:30.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.156+0000 7fc834eaa700 1 -- 172.21.15.42:0/2869499799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8300f6970 msgr2=0x7fc8300f6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:30.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.156+0000 7fc834eaa700 1 --2- 172.21.15.42:0/2869499799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8300f6970 0x7fc8300f6dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fc824004660 tx=0x7fc824009b30).stop 2024-02-15T18:22:30.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.156+0000 7fc834eaa700 1 -- 172.21.15.42:0/2869499799 shutdown_connections 2024-02-15T18:22:30.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.156+0000 7fc834eaa700 1 --2- 172.21.15.42:0/2869499799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8300f6970 0x7fc8300f6dd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:30.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.156+0000 7fc834eaa700 1 --2- 172.21.15.42:0/2869499799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8300f56d0 0x7fc8300f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:30.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.156+0000 7fc834eaa700 1 -- 172.21.15.42:0/2869499799 >> 172.21.15.42:0/2869499799 conn(0x7fc8300f0e50 msgr2=0x7fc8300f3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:30.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.156+0000 7fc834eaa700 1 -- 172.21.15.42:0/2869499799 shutdown_connections 2024-02-15T18:22:30.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.156+0000 7fc834eaa700 1 -- 172.21.15.42:0/2869499799 wait complete. 2024-02-15T18:22:30.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.156+0000 7fc834eaa700 1 Processor -- start 2024-02-15T18:22:30.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.157+0000 7fc834eaa700 1 -- start start 2024-02-15T18:22:30.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.157+0000 7fc834eaa700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8300f56d0 0x7fc830109eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:30.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.157+0000 7fc834eaa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8300f6970 0x7fc83010c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:30.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.157+0000 7fc834eaa700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc830060940 con 0x7fc8300f6970 2024-02-15T18:22:30.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.157+0000 7fc834eaa700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc830060a80 con 0x7fc8300f56d0 2024-02-15T18:22:30.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.157+0000 7fc82dd9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8300f6970 0x7fc83010c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:30.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.157+0000 7fc82dd9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8300f6970 0x7fc83010c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57836/0 (socket says 172.21.15.42:57836) 2024-02-15T18:22:30.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.157+0000 7fc82dd9b700 1 -- 172.21.15.42:0/3448697388 learned_addr learned my addr 172.21.15.42:0/3448697388 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:30.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.157+0000 7fc82dd9b700 1 -- 172.21.15.42:0/3448697388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8300f56d0 msgr2=0x7fc830109eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:30.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.157+0000 7fc82dd9b700 1 --2- 172.21.15.42:0/3448697388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8300f56d0 0x7fc830109eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:22:30.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.157+0000 7fc82dd9b700 1 -- 172.21.15.42:0/3448697388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc824005040 con 0x7fc8300f6970 2024-02-15T18:22:30.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.157+0000 7fc82dd9b700 1 --2- 172.21.15.42:0/3448697388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8300f6970 0x7fc83010c3b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fc824009830 tx=0x7fc824009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:30.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.157+0000 7fc81effd700 1 -- 172.21.15.42:0/3448697388 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc824018070 con 0x7fc8300f6970 2024-02-15T18:22:30.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.158+0000 7fc834eaa700 1 -- 172.21.15.42:0/3448697388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83010c900 con 0x7fc8300f6970 2024-02-15T18:22:30.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.158+0000 7fc81effd700 1 -- 172.21.15.42:0/3448697388 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc824007d70 con 0x7fc8300f6970 2024-02-15T18:22:30.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.158+0000 7fc81effd700 1 -- 172.21.15.42:0/3448697388 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc82401c620 con 0x7fc8300f6970 2024-02-15T18:22:30.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.158+0000 7fc834eaa700 1 -- 172.21.15.42:0/3448697388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83010ce60 con 0x7fc8300f6970 2024-02-15T18:22:30.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.158+0000 7fc834eaa700 1 -- 172.21.15.42:0/3448697388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc830045ad0 con 0x7fc8300f6970 2024-02-15T18:22:30.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.159+0000 7fc81effd700 1 -- 172.21.15.42:0/3448697388 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fc82401d980 con 0x7fc8300f6970 2024-02-15T18:22:30.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.159+0000 7fc81effd700 1 --2- 172.21.15.42:0/3448697388 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc81805c2a0 0x7fc81805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:30.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.159+0000 7fc81effd700 1 -- 172.21.15.42:0/3448697388 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fc824077a70 con 0x7fc8300f6970 2024-02-15T18:22:30.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.159+0000 7fc82e59c700 1 --2- 172.21.15.42:0/3448697388 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc81805c2a0 0x7fc81805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:30.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.160+0000 7fc82e59c700 1 --2- 172.21.15.42:0/3448697388 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc81805c2a0 0x7fc81805e760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fc8300605c0 tx=0x7fc820007a60).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:30.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.163+0000 7fc81effd700 1 -- 172.21.15.42:0/3448697388 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc82404aa10 con 0x7fc8300f6970 2024-02-15T18:22:30.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.322+0000 7fc834eaa700 1 -- 172.21.15.42:0/3448697388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc8300fe0e0 con 0x7fc8300f6970 2024-02-15T18:22:30.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.322+0000 7fc81effd700 1 -- 172.21.15.42:0/3448697388 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v39) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc824077e60 con 0x7fc8300f6970 2024-02-15T18:22:30.323 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-02-15T18:22:30.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.324+0000 7fc834eaa700 1 -- 172.21.15.42:0/3448697388 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc81805c2a0 msgr2=0x7fc81805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:30.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.324+0000 7fc834eaa700 1 --2- 172.21.15.42:0/3448697388 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc81805c2a0 0x7fc81805e760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fc8300605c0 tx=0x7fc820007a60).stop 2024-02-15T18:22:30.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.324+0000 7fc834eaa700 1 -- 172.21.15.42:0/3448697388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8300f6970 msgr2=0x7fc83010c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:30.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.324+0000 7fc834eaa700 1 --2- 172.21.15.42:0/3448697388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8300f6970 0x7fc83010c3b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fc824009830 tx=0x7fc824009b30).stop 2024-02-15T18:22:30.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.324+0000 7fc834eaa700 1 -- 172.21.15.42:0/3448697388 shutdown_connections 2024-02-15T18:22:30.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.324+0000 7fc834eaa700 1 --2- 172.21.15.42:0/3448697388 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc81805c2a0 0x7fc81805e760 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:30.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.324+0000 7fc834eaa700 1 --2- 172.21.15.42:0/3448697388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8300f56d0 0x7fc830109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:22:30.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.324+0000 7fc834eaa700 1 --2- 172.21.15.42:0/3448697388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8300f6970 0x7fc83010c3b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:30.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.324+0000 7fc834eaa700 1 -- 172.21.15.42:0/3448697388 >> 172.21.15.42:0/3448697388 conn(0x7fc8300f0e50 msgr2=0x7fc8300fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:30.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.324+0000 7fc834eaa700 1 -- 172.21.15.42:0/3448697388 shutdown_connections 2024-02-15T18:22:30.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:30.324+0000 7fc834eaa700 1 -- 172.21.15.42:0/3448697388 wait complete. 2024-02-15T18:22:30.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:28.804584+0000 mgr.smithi042.kfoihb (mgr.14162) 192 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:30.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:28.804584+0000 mgr.smithi042.kfoihb (mgr.14162) 192 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:31.171 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-15T18:22:31.171 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-15T18:22:31.172 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph mgr dump --format=json 2024-02-15T18:22:31.412 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:22:31.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:30.323673+0000 mon.smithi042 (mon.0) 439 : audit [DBG] from='client.? 172.21.15.42:0/3448697388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-15T18:22:31.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:30.323673+0000 mon.smithi042 (mon.0) 439 : audit [DBG] from='client.? 172.21.15.42:0/3448697388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-15T18:22:32.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:30.805057+0000 mgr.smithi042.kfoihb (mgr.14162) 193 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:32.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:30.805057+0000 mgr.smithi042.kfoihb (mgr.14162) 193 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:33.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.202+0000 7f48b7cd9700 1 -- 172.21.15.42:0/640329471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48b00f67c0 msgr2=0x7f48b00f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:33.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.202+0000 7f48b7cd9700 1 --2- 172.21.15.42:0/640329471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48b00f67c0 0x7f48b00f6be0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f48ac004660 tx=0x7f48ac009b30).stop 2024-02-15T18:22:33.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.202+0000 7f48b7cd9700 1 -- 172.21.15.42:0/640329471 shutdown_connections 2024-02-15T18:22:33.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.202+0000 7f48b7cd9700 1 --2- 172.21.15.42:0/640329471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b00f7a60 0x7f48b00f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:33.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.202+0000 7f48b7cd9700 1 --2- 172.21.15.42:0/640329471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48b00f67c0 0x7f48b00f6be0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:33.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.202+0000 7f48b7cd9700 1 -- 172.21.15.42:0/640329471 >> 172.21.15.42:0/640329471 conn(0x7f48b00f1f20 msgr2=0x7f48b00f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:33.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.202+0000 7f48b7cd9700 1 -- 172.21.15.42:0/640329471 shutdown_connections 2024-02-15T18:22:33.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.202+0000 7f48b7cd9700 1 -- 172.21.15.42:0/640329471 wait complete. 2024-02-15T18:22:33.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.202+0000 7f48b7cd9700 1 Processor -- start 2024-02-15T18:22:33.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b7cd9700 1 -- start start 2024-02-15T18:22:33.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b7cd9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48b00f67c0 0x7f48b01071a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:33.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b7cd9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b00f7a60 0x7f48b0107690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:33.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b7cd9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b0107c40 con 0x7f48b00f67c0 2024-02-15T18:22:33.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b7cd9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b0107d80 con 0x7f48b00f7a60 2024-02-15T18:22:33.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b5a75700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48b00f67c0 0x7f48b01071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:33.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b5a75700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48b00f67c0 0x7f48b01071a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34020/0 (socket says 172.21.15.42:34020) 2024-02-15T18:22:33.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b5a75700 1 -- 172.21.15.42:0/908516577 learned_addr learned my addr 172.21.15.42:0/908516577 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:33.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b5274700 1 --2- 172.21.15.42:0/908516577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b00f7a60 0x7f48b0107690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:33.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b5a75700 1 -- 172.21.15.42:0/908516577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b00f7a60 msgr2=0x7f48b0107690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:33.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b5a75700 1 --2- 172.21.15.42:0/908516577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b00f7a60 0x7f48b0107690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:33.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b5a75700 1 -- 172.21.15.42:0/908516577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac005040 con 0x7f48b00f67c0 2024-02-15T18:22:33.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b5274700 1 --2- 172.21.15.42:0/908516577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b00f7a60 0x7f48b0107690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:22:33.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b5a75700 1 --2- 172.21.15.42:0/908516577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48b00f67c0 0x7f48b01071a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f48ac004e30 tx=0x7f48ac009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:33.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48a67fc700 1 -- 172.21.15.42:0/908516577 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48ac018070 con 0x7f48b00f67c0 2024-02-15T18:22:33.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48b7cd9700 1 -- 172.21.15.42:0/908516577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b01086c0 con 0x7f48b00f67c0 2024-02-15T18:22:33.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48a67fc700 1 -- 172.21.15.42:0/908516577 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48ac007ba0 con 0x7f48b00f67c0 2024-02-15T18:22:33.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.203+0000 7f48a67fc700 1 -- 172.21.15.42:0/908516577 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48ac01c620 con 0x7f48b00f67c0 2024-02-15T18:22:33.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.204+0000 7f48b7cd9700 1 -- 172.21.15.42:0/908516577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b0108c20 con 0x7f48b00f67c0 2024-02-15T18:22:33.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.204+0000 7f48b7cd9700 1 -- 172.21.15.42:0/908516577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b0045ad0 con 0x7f48b00f67c0 2024-02-15T18:22:33.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.205+0000 7f48a67fc700 1 -- 172.21.15.42:0/908516577 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f48ac007d10 con 0x7f48b00f67c0 2024-02-15T18:22:33.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.205+0000 7f48a67fc700 1 --2- 172.21.15.42:0/908516577 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f489c05c2a0 0x7f489c05e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:33.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.205+0000 7f48a67fc700 1 -- 172.21.15.42:0/908516577 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f48ac0778a0 con 0x7f48b00f67c0 2024-02-15T18:22:33.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.205+0000 7f48b5274700 1 --2- 172.21.15.42:0/908516577 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f489c05c2a0 0x7f489c05e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:33.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.206+0000 7f48b5274700 1 --2- 172.21.15.42:0/908516577 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f489c05c2a0 0x7f489c05e760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f48a0009740 tx=0x7f48a0009be0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:33.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.209+0000 7f48a67fc700 1 -- 172.21.15.42:0/908516577 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f48ac04a8a0 con 0x7f48b00f67c0 2024-02-15T18:22:33.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.407+0000 7f48b7cd9700 1 -- 172.21.15.42:0/908516577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f48b0027a00 con 0x7f48b00f67c0 2024-02-15T18:22:33.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.410+0000 7f48a67fc700 1 -- 172.21.15.42:0/908516577 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v18) v1 ==== 74+0+143500 (secure 0 0 0) 0x7f48ac04dec0 con 0x7f48b00f67c0 2024-02-15T18:22:33.413 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:22:33.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.412+0000 7f48b7cd9700 1 -- 172.21.15.42:0/908516577 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f489c05c2a0 msgr2=0x7f489c05e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:33.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.412+0000 7f48b7cd9700 1 --2- 172.21.15.42:0/908516577 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f489c05c2a0 0x7f489c05e760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f48a0009740 tx=0x7f48a0009be0).stop 2024-02-15T18:22:33.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.412+0000 7f48b7cd9700 1 -- 172.21.15.42:0/908516577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48b00f67c0 msgr2=0x7f48b01071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:33.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.412+0000 7f48b7cd9700 1 --2- 172.21.15.42:0/908516577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48b00f67c0 0x7f48b01071a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f48ac004e30 tx=0x7f48ac009b30).stop 2024-02-15T18:22:33.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.412+0000 7f48b7cd9700 1 -- 172.21.15.42:0/908516577 shutdown_connections 2024-02-15T18:22:33.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.412+0000 7f48b7cd9700 1 --2- 172.21.15.42:0/908516577 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f489c05c2a0 0x7f489c05e760 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:33.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.412+0000 7f48b7cd9700 1 --2- 172.21.15.42:0/908516577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48b00f67c0 0x7f48b01071a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:33.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.412+0000 7f48b7cd9700 1 --2- 172.21.15.42:0/908516577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48b00f7a60 0x7f48b0107690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:33.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.412+0000 7f48b7cd9700 1 -- 172.21.15.42:0/908516577 >> 172.21.15.42:0/908516577 conn(0x7f48b00f1f20 msgr2=0x7f48b00fb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:33.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.412+0000 7f48b7cd9700 1 -- 172.21.15.42:0/908516577 shutdown_connections 2024-02-15T18:22:33.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:33.412+0000 7f48b7cd9700 1 -- 172.21.15.42:0/908516577 wait complete. 2024-02-15T18:22:34.180 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":18,"active_gid":14162,"active_name":"smithi042.kfoihb","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":1691818213},{"type":"v1","addr":"172.21.15.42:6801","nonce":1691818213}]},"active_addr":"172.21.15.42:6801/1691818213","active_change":"2024-02-15T18:17:56.705271+0000","active_mgr_features":4540138297136906239,"available":true,"standbys":[{"gid":14204,"name":"smithi191.gftdzo","mgr_features":4540138297136906239,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Registry to which we should normalize unqualified image names","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Registry to which we should normalize unqualified image names","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://smithi042.front.sepia.ceph.com:8443/","prometheus":"http://smithi042.front.sepia.ceph.com:9283/"},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":4,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":1489061810}]},{"addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":4019907068}]}]}} 2024-02-15T18:22:34.185 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-15T18:22:34.186 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-15T18:22:34.186 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-15T18:22:34.429 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:22:34.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:32.805406+0000 mgr.smithi042.kfoihb (mgr.14162) 194 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:34.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:33.409108+0000 mon.smithi042 (mon.0) 440 : audit [DBG] from='client.? 172.21.15.42:0/908516577' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-15T18:22:34.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:32.805406+0000 mgr.smithi042.kfoihb (mgr.14162) 194 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:34.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:33.409108+0000 mon.smithi042 (mon.0) 440 : audit [DBG] from='client.? 172.21.15.42:0/908516577' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-15T18:22:36.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.292+0000 7ffb65a85700 1 -- 172.21.15.42:0/2016579859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb600f56c0 msgr2=0x7ffb600f5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:36.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.292+0000 7ffb65a85700 1 --2- 172.21.15.42:0/2016579859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb600f56c0 0x7ffb600f5ae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7ffb500099e0 tx=0x7ffb50004d10).stop 2024-02-15T18:22:36.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.292+0000 7ffb65a85700 1 -- 172.21.15.42:0/2016579859 shutdown_connections 2024-02-15T18:22:36.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.292+0000 7ffb65a85700 1 --2- 172.21.15.42:0/2016579859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb600f6960 0x7ffb600f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:36.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.292+0000 7ffb65a85700 1 --2- 172.21.15.42:0/2016579859 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb600f56c0 0x7ffb600f5ae0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:36.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.292+0000 7ffb65a85700 1 -- 172.21.15.42:0/2016579859 >> 172.21.15.42:0/2016579859 conn(0x7ffb600f0e20 msgr2=0x7ffb600f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:36.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.292+0000 7ffb65a85700 1 -- 172.21.15.42:0/2016579859 shutdown_connections 2024-02-15T18:22:36.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.292+0000 7ffb65a85700 1 -- 172.21.15.42:0/2016579859 wait complete. 2024-02-15T18:22:36.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.293+0000 7ffb65a85700 1 Processor -- start 2024-02-15T18:22:36.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.293+0000 7ffb65a85700 1 -- start start 2024-02-15T18:22:36.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.293+0000 7ffb65a85700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb600f56c0 0x7ffb601061d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:36.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.293+0000 7ffb65a85700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb600f6960 0x7ffb601066c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:36.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.293+0000 7ffb65a85700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb60106c70 con 0x7ffb600f56c0 2024-02-15T18:22:36.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.293+0000 7ffb65a85700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb60106db0 con 0x7ffb600f6960 2024-02-15T18:22:36.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.294+0000 7ffb5f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb600f56c0 0x7ffb601061d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:36.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.294+0000 7ffb5f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb600f56c0 0x7ffb601061d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34036/0 (socket says 172.21.15.42:34036) 2024-02-15T18:22:36.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.294+0000 7ffb5f7fe700 1 -- 172.21.15.42:0/3345946484 learned_addr learned my addr 172.21.15.42:0/3345946484 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:36.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.294+0000 7ffb5f7fe700 1 -- 172.21.15.42:0/3345946484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb600f6960 msgr2=0x7ffb601066c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:36.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.294+0000 7ffb5effd700 1 --2- 172.21.15.42:0/3345946484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb600f6960 0x7ffb601066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:36.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.294+0000 7ffb5f7fe700 1 --2- 172.21.15.42:0/3345946484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb600f6960 0x7ffb601066c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:36.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.294+0000 7ffb5f7fe700 1 -- 172.21.15.42:0/3345946484 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb50009640 con 0x7ffb600f56c0 2024-02-15T18:22:36.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.294+0000 7ffb5f7fe700 1 --2- 172.21.15.42:0/3345946484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb600f56c0 0x7ffb601061d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7ffb50003380 tx=0x7ffb50004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:36.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.294+0000 7ffb57fff700 1 -- 172.21.15.42:0/3345946484 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb5000e070 con 0x7ffb600f56c0 2024-02-15T18:22:36.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.294+0000 7ffb65a85700 1 -- 172.21.15.42:0/3345946484 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb601076f0 con 0x7ffb600f56c0 2024-02-15T18:22:36.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.294+0000 7ffb57fff700 1 -- 172.21.15.42:0/3345946484 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb5001d8f0 con 0x7ffb600f56c0 2024-02-15T18:22:36.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.294+0000 7ffb57fff700 1 -- 172.21.15.42:0/3345946484 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb50018070 con 0x7ffb600f56c0 2024-02-15T18:22:36.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.294+0000 7ffb65a85700 1 -- 172.21.15.42:0/3345946484 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb60107bd0 con 0x7ffb600f56c0 2024-02-15T18:22:36.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.295+0000 7ffb65a85700 1 -- 172.21.15.42:0/3345946484 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb60045ad0 con 0x7ffb600f56c0 2024-02-15T18:22:36.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.295+0000 7ffb57fff700 1 -- 172.21.15.42:0/3345946484 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7ffb50024060 con 0x7ffb600f56c0 2024-02-15T18:22:36.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.296+0000 7ffb57fff700 1 --2- 172.21.15.42:0/3345946484 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ffb4c05c2a0 0x7ffb4c05e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:36.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.296+0000 7ffb57fff700 1 -- 172.21.15.42:0/3345946484 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7ffb50077520 con 0x7ffb600f56c0 2024-02-15T18:22:36.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.296+0000 7ffb5effd700 1 --2- 172.21.15.42:0/3345946484 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ffb4c05c2a0 0x7ffb4c05e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:36.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.296+0000 7ffb5effd700 1 --2- 172.21.15.42:0/3345946484 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ffb4c05c2a0 0x7ffb4c05e760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7ffb480041d0 tx=0x7ffb48009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:36.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.300+0000 7ffb57fff700 1 -- 172.21.15.42:0/3345946484 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ffb5004a4c0 con 0x7ffb600f56c0 2024-02-15T18:22:36.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.462+0000 7ffb65a85700 1 -- 172.21.15.42:0/3345946484 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb600fdf90 con 0x7ffb600f56c0 2024-02-15T18:22:36.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.463+0000 7ffb57fff700 1 -- 172.21.15.42:0/3345946484 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+10907 (secure 0 0 0) 0x7ffb5004dae0 con 0x7ffb600f56c0 2024-02-15T18:22:36.463 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:22:36.463 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":39,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","created":"2024-02-15T18:16:53.406642+0000","modified":"2024-02-15T18:22:17.282788+0000","last_up_change":"2024-02-15T18:22:16.276777+0000","last_in_change":"2024-02-15T18:21:58.584131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-15T18:21:26.655770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"42281c08-af07-43a5-bcfd-d476bc745782","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6803","nonce":3129297361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6805","nonce":3129297361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6809","nonce":3129297361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6807","nonce":3129297361}]},"public_addr":"172.21.15.42:6803/3129297361","cluster_addr":"172.21.15.42:6805/3129297361","heartbeat_back_addr":"172.21.15.42:6809/3129297361","heartbeat_front_addr":"172.21.15.42:6807/3129297361","state":["exists","up"]},{"osd":1,"uuid":"8f67d329-7d44-4361-be02-f8b56b3e5713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6811","nonce":563363965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6813","nonce":563363965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6817","nonce":563363965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6815","nonce":563363965}]},"public_addr":"172.21.15.42:6811/563363965","cluster_addr":"172.21.15.42:6813/563363965","heartbeat_back_addr":"172.21.15.42:6817/563363965","heartbeat_front_addr":"172.21.15.42:6815/563363965","state":["exists","up"]},{"osd":2,"uuid":"680f6437-b450-4353-a761-b50d1388bcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6819","nonce":2706708670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6821","nonce":2706708670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6825","nonce":2706708670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6823","nonce":2706708670}]},"public_addr":"172.21.15.42:6819/2706708670","cluster_addr":"172.21.15.42:6821/2706708670","heartbeat_back_addr":"172.21.15.42:6825/2706708670","heartbeat_front_addr":"172.21.15.42:6823/2706708670","state":["exists","up"]},{"osd":3,"uuid":"6aeea641-3735-4f11-b95f-40c45d4dcd07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6801","nonce":3223849184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6803","nonce":3223849184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6807","nonce":3223849184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6805","nonce":3223849184}]},"public_addr":"172.21.15.191:6801/3223849184","cluster_addr":"172.21.15.191:6803/3223849184","heartbeat_back_addr":"172.21.15.191:6807/3223849184","heartbeat_front_addr":"172.21.15.191:6805/3223849184","state":["exists","up"]},{"osd":4,"uuid":"7d9839d2-bd60-4bae-a16f-ebf56f204acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6809","nonce":424560222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6811","nonce":424560222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6815","nonce":424560222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6813","nonce":424560222}]},"public_addr":"172.21.15.191:6809/424560222","cluster_addr":"172.21.15.191:6811/424560222","heartbeat_back_addr":"172.21.15.191:6815/424560222","heartbeat_front_addr":"172.21.15.191:6813/424560222","state":["exists","up"]},{"osd":5,"uuid":"a75bb280-bd87-4961-918b-18c426f31c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6817","nonce":2014313522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6819","nonce":2014313522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6823","nonce":2014313522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6821","nonce":2014313522}]},"public_addr":"172.21.15.191:6817/2014313522","cluster_addr":"172.21.15.191:6819/2014313522","heartbeat_back_addr":"172.21.15.191:6823/2014313522","heartbeat_front_addr":"172.21.15.191:6821/2014313522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:20:17.479921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:20:47.951852+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:11.894830+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:31.013756+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:52.493759+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:22:14.397700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1703009783":"2024-02-16T18:17:56.705000+0000","172.21.15.42:6801/481027993":"2024-02-16T18:17:56.705000+0000","172.21.15.42:0/1629344114":"2024-02-16T18:17:56.705000+0000","172.21.15.42:0/3157909680":"2024-02-16T18:17:40.441172+0000","172.21.15.42:6801/3051492708":"2024-02-16T18:17:40.441172+0000","172.21.15.42:0/1046171692":"2024-02-16T18:17:18.618949+0000","172.21.15.42:6800/481027993":"2024-02-16T18:17:56.705000+0000","172.21.15.42:6800/3051492708":"2024-02-16T18:17:40.441172+0000","172.21.15.42:6800/1746226634":"2024-02-16T18:17:18.618949+0000","172.21.15.42:0/4044100":"2024-02-16T18:17:18.618949+0000","172.21.15.42:0/4136985555":"2024-02-16T18:17:40.441172+0000","172.21.15.42:6801/1746226634":"2024-02-16T18:17:18.618949+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-15T18:22:36.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.465+0000 7ffb65a85700 1 -- 172.21.15.42:0/3345946484 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ffb4c05c2a0 msgr2=0x7ffb4c05e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:36.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.465+0000 7ffb65a85700 1 --2- 172.21.15.42:0/3345946484 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ffb4c05c2a0 0x7ffb4c05e760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7ffb480041d0 tx=0x7ffb48009040).stop 2024-02-15T18:22:36.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.465+0000 7ffb65a85700 1 -- 172.21.15.42:0/3345946484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb600f56c0 msgr2=0x7ffb601061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:36.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.465+0000 7ffb65a85700 1 --2- 172.21.15.42:0/3345946484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb600f56c0 0x7ffb601061d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7ffb50003380 tx=0x7ffb50004d10).stop 2024-02-15T18:22:36.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.465+0000 7ffb65a85700 1 -- 172.21.15.42:0/3345946484 shutdown_connections 2024-02-15T18:22:36.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.465+0000 7ffb65a85700 1 --2- 172.21.15.42:0/3345946484 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ffb4c05c2a0 0x7ffb4c05e760 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:36.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.465+0000 7ffb65a85700 1 --2- 172.21.15.42:0/3345946484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb600f56c0 0x7ffb601061d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:36.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.465+0000 7ffb65a85700 1 --2- 172.21.15.42:0/3345946484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb600f6960 0x7ffb601066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:36.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.465+0000 7ffb65a85700 1 -- 172.21.15.42:0/3345946484 >> 172.21.15.42:0/3345946484 conn(0x7ffb600f0e20 msgr2=0x7ffb600fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:36.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.465+0000 7ffb65a85700 1 -- 172.21.15.42:0/3345946484 shutdown_connections 2024-02-15T18:22:36.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:36.465+0000 7ffb65a85700 1 -- 172.21.15.42:0/3345946484 wait complete. 2024-02-15T18:22:36.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:34.805876+0000 mgr.smithi042.kfoihb (mgr.14162) 195 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:36.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:34.805876+0000 mgr.smithi042.kfoihb (mgr.14162) 195 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:37.161 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-15T18:22:37.161 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-15T18:22:37.401 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:22:37.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:36.463809+0000 mon.smithi042 (mon.0) 441 : audit [DBG] from='client.? 172.21.15.42:0/3345946484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:22:37.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:36.463809+0000 mon.smithi042 (mon.0) 441 : audit [DBG] from='client.? 172.21.15.42:0/3345946484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:22:38.487 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:36.806292+0000 mgr.smithi042.kfoihb (mgr.14162) 196 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:38.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:36.806292+0000 mgr.smithi042.kfoihb (mgr.14162) 196 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:39.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.004+0000 7f13eb867700 1 -- 172.21.15.42:0/1233855740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13e40607d0 msgr2=0x7f13e40f9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.004+0000 7f13eb867700 1 --2- 172.21.15.42:0/1233855740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13e40607d0 0x7f13e40f9440 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f13e0004660 tx=0x7f13e0009b30).stop 2024-02-15T18:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.004+0000 7f13eb867700 1 -- 172.21.15.42:0/1233855740 shutdown_connections 2024-02-15T18:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.004+0000 7f13eb867700 1 --2- 172.21.15.42:0/1233855740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13e40f9930 0x7f13e40fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.004+0000 7f13eb867700 1 --2- 172.21.15.42:0/1233855740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13e40607d0 0x7f13e40f9440 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.004+0000 7f13eb867700 1 -- 172.21.15.42:0/1233855740 >> 172.21.15.42:0/1233855740 conn(0x7f13e40f0e20 msgr2=0x7f13e40f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.004+0000 7f13eb867700 1 -- 172.21.15.42:0/1233855740 shutdown_connections 2024-02-15T18:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.004+0000 7f13eb867700 1 -- 172.21.15.42:0/1233855740 wait complete. 2024-02-15T18:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.004+0000 7f13eb867700 1 Processor -- start 2024-02-15T18:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.004+0000 7f13eb867700 1 -- start start 2024-02-15T18:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13eb867700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13e40607d0 0x7f13e41071d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13eb867700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13e40f9930 0x7f13e41076c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13eb867700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e4107c70 con 0x7f13e40607d0 2024-02-15T18:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13eb867700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e4107db0 con 0x7f13e40f9930 2024-02-15T18:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13e9603700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13e40607d0 0x7f13e41071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13e9603700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13e40607d0 0x7f13e41071d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34066/0 (socket says 172.21.15.42:34066) 2024-02-15T18:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13e9603700 1 -- 172.21.15.42:0/401288536 learned_addr learned my addr 172.21.15.42:0/401288536 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13e8e02700 1 --2- 172.21.15.42:0/401288536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13e40f9930 0x7f13e41076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13e9603700 1 -- 172.21.15.42:0/401288536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13e40f9930 msgr2=0x7f13e41076c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13e9603700 1 --2- 172.21.15.42:0/401288536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13e40f9930 0x7f13e41076c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13e9603700 1 -- 172.21.15.42:0/401288536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e0005040 con 0x7f13e40607d0 2024-02-15T18:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13e9603700 1 --2- 172.21.15.42:0/401288536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13e40607d0 0x7f13e41071d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f13e0009830 tx=0x7f13e0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13d5ffb700 1 -- 172.21.15.42:0/401288536 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13e0018070 con 0x7f13e40607d0 2024-02-15T18:22:39.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13d5ffb700 1 -- 172.21.15.42:0/401288536 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13e0007ba0 con 0x7f13e40607d0 2024-02-15T18:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13eb867700 1 -- 172.21.15.42:0/401288536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e41086f0 con 0x7f13e40607d0 2024-02-15T18:22:39.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.005+0000 7f13d5ffb700 1 -- 172.21.15.42:0/401288536 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13e001c3c0 con 0x7f13e40607d0 2024-02-15T18:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.006+0000 7f13eb867700 1 -- 172.21.15.42:0/401288536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e4108c50 con 0x7f13e40607d0 2024-02-15T18:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.007+0000 7f13eb867700 1 -- 172.21.15.42:0/401288536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e4045ad0 con 0x7f13e40607d0 2024-02-15T18:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.007+0000 7f13d5ffb700 1 -- 172.21.15.42:0/401288536 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f13e0007d10 con 0x7f13e40607d0 2024-02-15T18:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.007+0000 7f13d5ffb700 1 --2- 172.21.15.42:0/401288536 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f13d005c2a0 0x7f13d005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.007+0000 7f13d5ffb700 1 -- 172.21.15.42:0/401288536 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f13e0077880 con 0x7f13e40607d0 2024-02-15T18:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.007+0000 7f13e8e02700 1 --2- 172.21.15.42:0/401288536 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f13d005c2a0 0x7f13d005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:39.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.008+0000 7f13e8e02700 1 --2- 172.21.15.42:0/401288536 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f13d005c2a0 0x7f13d005e760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f13e40605a0 tx=0x7f13d800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.011+0000 7f13d5ffb700 1 -- 172.21.15.42:0/401288536 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f13e004a820 con 0x7f13e40607d0 2024-02-15T18:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.168+0000 7f13eb867700 1 -- 172.21.15.42:0/401288536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13e4027a00 con 0x7f13e40607d0 2024-02-15T18:22:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.168+0000 7f13d5ffb700 1 -- 172.21.15.42:0/401288536 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+10907 (secure 0 0 0) 0x7f13e004de40 con 0x7f13e40607d0 2024-02-15T18:22:39.169 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:22:39.169 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":39,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","created":"2024-02-15T18:16:53.406642+0000","modified":"2024-02-15T18:22:17.282788+0000","last_up_change":"2024-02-15T18:22:16.276777+0000","last_in_change":"2024-02-15T18:21:58.584131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-15T18:21:26.655770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"42281c08-af07-43a5-bcfd-d476bc745782","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6803","nonce":3129297361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6805","nonce":3129297361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6809","nonce":3129297361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6807","nonce":3129297361}]},"public_addr":"172.21.15.42:6803/3129297361","cluster_addr":"172.21.15.42:6805/3129297361","heartbeat_back_addr":"172.21.15.42:6809/3129297361","heartbeat_front_addr":"172.21.15.42:6807/3129297361","state":["exists","up"]},{"osd":1,"uuid":"8f67d329-7d44-4361-be02-f8b56b3e5713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6811","nonce":563363965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6813","nonce":563363965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6817","nonce":563363965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6815","nonce":563363965}]},"public_addr":"172.21.15.42:6811/563363965","cluster_addr":"172.21.15.42:6813/563363965","heartbeat_back_addr":"172.21.15.42:6817/563363965","heartbeat_front_addr":"172.21.15.42:6815/563363965","state":["exists","up"]},{"osd":2,"uuid":"680f6437-b450-4353-a761-b50d1388bcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6819","nonce":2706708670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6821","nonce":2706708670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6825","nonce":2706708670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6823","nonce":2706708670}]},"public_addr":"172.21.15.42:6819/2706708670","cluster_addr":"172.21.15.42:6821/2706708670","heartbeat_back_addr":"172.21.15.42:6825/2706708670","heartbeat_front_addr":"172.21.15.42:6823/2706708670","state":["exists","up"]},{"osd":3,"uuid":"6aeea641-3735-4f11-b95f-40c45d4dcd07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6801","nonce":3223849184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6803","nonce":3223849184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6807","nonce":3223849184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6805","nonce":3223849184}]},"public_addr":"172.21.15.191:6801/3223849184","cluster_addr":"172.21.15.191:6803/3223849184","heartbeat_back_addr":"172.21.15.191:6807/3223849184","heartbeat_front_addr":"172.21.15.191:6805/3223849184","state":["exists","up"]},{"osd":4,"uuid":"7d9839d2-bd60-4bae-a16f-ebf56f204acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6809","nonce":424560222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6811","nonce":424560222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6815","nonce":424560222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6813","nonce":424560222}]},"public_addr":"172.21.15.191:6809/424560222","cluster_addr":"172.21.15.191:6811/424560222","heartbeat_back_addr":"172.21.15.191:6815/424560222","heartbeat_front_addr":"172.21.15.191:6813/424560222","state":["exists","up"]},{"osd":5,"uuid":"a75bb280-bd87-4961-918b-18c426f31c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6817","nonce":2014313522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6819","nonce":2014313522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6823","nonce":2014313522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6821","nonce":2014313522}]},"public_addr":"172.21.15.191:6817/2014313522","cluster_addr":"172.21.15.191:6819/2014313522","heartbeat_back_addr":"172.21.15.191:6823/2014313522","heartbeat_front_addr":"172.21.15.191:6821/2014313522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:20:17.479921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:20:47.951852+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:11.894830+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:31.013756+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:52.493759+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:22:14.397700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1703009783":"2024-02-16T18:17:56.705000+0000","172.21.15.42:6801/481027993":"2024-02-16T18:17:56.705000+0000","172.21.15.42:0/1629344114":"2024-02-16T18:17:56.705000+0000","172.21.15.42:0/3157909680":"2024-02-16T18:17:40.441172+0000","172.21.15.42:6801/3051492708":"2024-02-16T18:17:40.441172+0000","172.21.15.42:0/1046171692":"2024-02-16T18:17:18.618949+0000","172.21.15.42:6800/481027993":"2024-02-16T18:17:56.705000+0000","172.21.15.42:6800/3051492708":"2024-02-16T18:17:40.441172+0000","172.21.15.42:6800/1746226634":"2024-02-16T18:17:18.618949+0000","172.21.15.42:0/4044100":"2024-02-16T18:17:18.618949+0000","172.21.15.42:0/4136985555":"2024-02-16T18:17:40.441172+0000","172.21.15.42:6801/1746226634":"2024-02-16T18:17:18.618949+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-15T18:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.170+0000 7f13eb867700 1 -- 172.21.15.42:0/401288536 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f13d005c2a0 msgr2=0x7f13d005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.170+0000 7f13eb867700 1 --2- 172.21.15.42:0/401288536 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f13d005c2a0 0x7f13d005e760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f13e40605a0 tx=0x7f13d800b040).stop 2024-02-15T18:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.170+0000 7f13eb867700 1 -- 172.21.15.42:0/401288536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13e40607d0 msgr2=0x7f13e41071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.170+0000 7f13eb867700 1 --2- 172.21.15.42:0/401288536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13e40607d0 0x7f13e41071d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f13e0009830 tx=0x7f13e0009b30).stop 2024-02-15T18:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.170+0000 7f13eb867700 1 -- 172.21.15.42:0/401288536 shutdown_connections 2024-02-15T18:22:39.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.170+0000 7f13eb867700 1 --2- 172.21.15.42:0/401288536 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f13d005c2a0 0x7f13d005e760 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.170+0000 7f13eb867700 1 --2- 172.21.15.42:0/401288536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13e40607d0 0x7f13e41071d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.170+0000 7f13eb867700 1 --2- 172.21.15.42:0/401288536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13e40f9930 0x7f13e41076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.170+0000 7f13eb867700 1 -- 172.21.15.42:0/401288536 >> 172.21.15.42:0/401288536 conn(0x7f13e40f0e20 msgr2=0x7f13e40f3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.170+0000 7f13eb867700 1 -- 172.21.15.42:0/401288536 shutdown_connections 2024-02-15T18:22:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:39.170+0000 7f13eb867700 1 -- 172.21.15.42:0/401288536 wait complete. 2024-02-15T18:22:39.511 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:39.169271+0000 mon.smithi042 (mon.0) 442 : audit [DBG] from='client.? 172.21.15.42:0/401288536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:22:39.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:39.169271+0000 mon.smithi042 (mon.0) 442 : audit [DBG] from='client.? 172.21.15.42:0/401288536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:22:39.848 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-15T18:22:40.084 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:22:40.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:38.806711+0000 mgr.smithi042.kfoihb (mgr.14162) 197 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:40.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:38.806711+0000 mgr.smithi042.kfoihb (mgr.14162) 197 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:41.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.880+0000 7f764e99c700 1 -- 172.21.15.42:0/1602974671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76480f67e0 msgr2=0x7f76480f6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:41.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.880+0000 7f764e99c700 1 --2- 172.21.15.42:0/1602974671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76480f67e0 0x7f76480f6c00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f7638004660 tx=0x7f7638009b30).stop 2024-02-15T18:22:41.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.880+0000 7f764e99c700 1 -- 172.21.15.42:0/1602974671 shutdown_connections 2024-02-15T18:22:41.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.880+0000 7f764e99c700 1 --2- 172.21.15.42:0/1602974671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76480f7a80 0x7f76480f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:22:41.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.880+0000 7f764e99c700 1 --2- 172.21.15.42:0/1602974671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76480f67e0 0x7f76480f6c00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:41.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.880+0000 7f764e99c700 1 -- 172.21.15.42:0/1602974671 >> 172.21.15.42:0/1602974671 conn(0x7f76480f1f80 msgr2=0x7f76480f43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:41.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.880+0000 7f764e99c700 1 -- 172.21.15.42:0/1602974671 shutdown_connections 2024-02-15T18:22:41.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.880+0000 7f764e99c700 1 -- 172.21.15.42:0/1602974671 wait complete. 2024-02-15T18:22:41.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.881+0000 7f764e99c700 1 Processor -- start 2024-02-15T18:22:41.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.881+0000 7f764e99c700 1 -- start start 2024-02-15T18:22:41.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.881+0000 7f764e99c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76480f67e0 0x7f7648107160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:41.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.881+0000 7f764e99c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76480f7a80 0x7f7648107650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.881+0000 7f764e99c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7648107c90 con 0x7f76480f67e0 2024-02-15T18:22:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.881+0000 7f764e99c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7648108540 con 0x7f76480f7a80 2024-02-15T18:22:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.882+0000 7f7647fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76480f67e0 0x7f7648107160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.882+0000 7f7647fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76480f67e0 0x7f7648107160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44762/0 (socket says 172.21.15.42:44762) 2024-02-15T18:22:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.882+0000 7f7647fff700 1 -- 172.21.15.42:0/123702177 learned_addr learned my addr 172.21.15.42:0/123702177 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.882+0000 7f76477fe700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76480f7a80 0x7f7648107650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.882+0000 7f7647fff700 1 -- 172.21.15.42:0/123702177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76480f7a80 msgr2=0x7f7648107650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.882+0000 7f7647fff700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76480f7a80 0x7f7648107650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.882+0000 7f7647fff700 1 -- 172.21.15.42:0/123702177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7638005040 con 0x7f76480f67e0 2024-02-15T18:22:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.882+0000 7f76477fe700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76480f7a80 0x7f7648107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:22:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.882+0000 7f7647fff700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76480f67e0 0x7f7648107160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f7638009f50 tx=0x7f7638009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.882+0000 7f7644ff9700 1 -- 172.21.15.42:0/123702177 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7638018070 con 0x7f76480f67e0 2024-02-15T18:22:41.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.882+0000 7f764e99c700 1 -- 172.21.15.42:0/123702177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76481087c0 con 0x7f76480f67e0 2024-02-15T18:22:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.882+0000 7f7644ff9700 1 -- 172.21.15.42:0/123702177 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7638007d70 con 0x7f76480f67e0 2024-02-15T18:22:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.882+0000 7f7644ff9700 1 -- 172.21.15.42:0/123702177 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f763801c620 con 0x7f76480f67e0 2024-02-15T18:22:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.882+0000 7f764e99c700 1 -- 172.21.15.42:0/123702177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7648108d20 con 0x7f76480f67e0 2024-02-15T18:22:41.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.883+0000 7f764e99c700 1 -- 172.21.15.42:0/123702177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f764805dcf0 con 0x7f76480f67e0 2024-02-15T18:22:41.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.884+0000 7f7644ff9700 1 -- 172.21.15.42:0/123702177 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f763801c8e0 con 0x7f76480f67e0 2024-02-15T18:22:41.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.884+0000 7f7644ff9700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f763005c2a0 0x7f763005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:41.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.884+0000 7f7644ff9700 1 -- 172.21.15.42:0/123702177 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f7638077a40 con 0x7f76480f67e0 2024-02-15T18:22:41.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.884+0000 7f76477fe700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f763005c2a0 0x7f763005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:41.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.884+0000 7f7644ff9700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361] conn(0x7f7630062030 0x7f7630064490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:41.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.884+0000 7f7644ff9700 1 -- 172.21.15.42:0/123702177 --> [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7630064b10 con 0x7f7630062030 2024-02-15T18:22:41.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.884+0000 7f7644ff9700 1 -- 172.21.15.42:0/123702177 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f763807c220 con 0x7f76480f67e0 2024-02-15T18:22:41.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.884+0000 7f764cf39700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361] conn(0x7f7630062030 0x7f7630064490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:41.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.885+0000 7f76477fe700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f763005c2a0 0x7f763005e760 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f763c00b830 tx=0x7f763c008040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:41.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.885+0000 7f764cf39700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361] conn(0x7f7630062030 0x7f7630064490 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:41.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.885+0000 7f7644ff9700 1 -- 172.21.15.42:0/123702177 <== osd.0 v2:172.21.15.42:6802/3129297361 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f7630064b10 con 0x7f7630062030 2024-02-15T18:22:41.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.900+0000 7f764e99c700 1 -- 172.21.15.42:0/123702177 --> [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f76481092c0 con 0x7f7630062030 2024-02-15T18:22:41.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f7644ff9700 1 -- 172.21.15.42:0/123702177 <== osd.0 v2:172.21.15.42:6802/3129297361 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f76481092c0 con 0x7f7630062030 2024-02-15T18:22:41.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f764e99c700 1 -- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361] conn(0x7f7630062030 msgr2=0x7f7630064490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f764e99c700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361] conn(0x7f7630062030 0x7f7630064490 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f764e99c700 1 -- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f763005c2a0 msgr2=0x7f763005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f764e99c700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f763005c2a0 0x7f763005e760 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f763c00b830 tx=0x7f763c008040).stop 2024-02-15T18:22:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f764e99c700 1 -- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76480f67e0 msgr2=0x7f7648107160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f764e99c700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76480f67e0 0x7f7648107160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f7638009f50 tx=0x7f7638009b30).stop 2024-02-15T18:22:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f764e99c700 1 -- 172.21.15.42:0/123702177 shutdown_connections 2024-02-15T18:22:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f764e99c700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:6802/3129297361,v1:172.21.15.42:6803/3129297361] conn(0x7f7630062030 0x7f7630064490 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f764e99c700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f763005c2a0 0x7f763005e760 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f764e99c700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76480f67e0 0x7f7648107160 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f764e99c700 1 --2- 172.21.15.42:0/123702177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76480f7a80 0x7f7648107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f764e99c700 1 -- 172.21.15.42:0/123702177 >> 172.21.15.42:0/123702177 conn(0x7f76480f1f80 msgr2=0x7f76480fb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f764e99c700 1 -- 172.21.15.42:0/123702177 shutdown_connections 2024-02-15T18:22:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:41.901+0000 7f764e99c700 1 -- 172.21.15.42:0/123702177 wait complete. 2024-02-15T18:22:42.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:40.807060+0000 mgr.smithi042.kfoihb (mgr.14162) 198 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:42.681 INFO:teuthology.orchestra.run.smithi042.stdout:38654705695 2024-02-15T18:22:42.681 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-15T18:22:42.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:40.807060+0000 mgr.smithi042.kfoihb (mgr.14162) 198 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:42.925 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:22:44.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.631+0000 7f212c538700 1 -- 172.21.15.42:0/1983654887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21240f6960 msgr2=0x7f21240f6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:44.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.631+0000 7f212c538700 1 --2- 172.21.15.42:0/1983654887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21240f6960 0x7f21240f6dc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f2114004660 tx=0x7f2114009b30).stop 2024-02-15T18:22:44.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.631+0000 7f212c538700 1 -- 172.21.15.42:0/1983654887 shutdown_connections 2024-02-15T18:22:44.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.631+0000 7f212c538700 1 --2- 172.21.15.42:0/1983654887 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21240f6960 0x7f21240f6dc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:44.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.631+0000 7f212c538700 1 --2- 172.21.15.42:0/1983654887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21240f56c0 0x7f21240f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:44.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.631+0000 7f212c538700 1 -- 172.21.15.42:0/1983654887 >> 172.21.15.42:0/1983654887 conn(0x7f21240f0e20 msgr2=0x7f21240f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:44.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.631+0000 7f212c538700 1 -- 172.21.15.42:0/1983654887 shutdown_connections 2024-02-15T18:22:44.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.631+0000 7f212c538700 1 -- 172.21.15.42:0/1983654887 wait complete. 2024-02-15T18:22:44.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.631+0000 7f212c538700 1 Processor -- start 2024-02-15T18:22:44.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.632+0000 7f212c538700 1 -- start start 2024-02-15T18:22:44.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.632+0000 7f212c538700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21240f56c0 0x7f2124109f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:44.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.632+0000 7f212c538700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21240f6960 0x7f212410c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:44.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.632+0000 7f212c538700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2124060920 con 0x7f21240f6960 2024-02-15T18:22:44.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.632+0000 7f212c538700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2124060a60 con 0x7f21240f56c0 2024-02-15T18:22:44.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.632+0000 7f2129ad3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21240f6960 0x7f212410c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:44.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.632+0000 7f2129ad3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21240f6960 0x7f212410c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44792/0 (socket says 172.21.15.42:44792) 2024-02-15T18:22:44.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.632+0000 7f2129ad3700 1 -- 172.21.15.42:0/1607544556 learned_addr learned my addr 172.21.15.42:0/1607544556 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:44.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.632+0000 7f212a2d4700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21240f56c0 0x7f2124109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:44.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.632+0000 7f2129ad3700 1 -- 172.21.15.42:0/1607544556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21240f56c0 msgr2=0x7f2124109f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:44.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.632+0000 7f2129ad3700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21240f56c0 0x7f2124109f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:44.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.632+0000 7f2129ad3700 1 -- 172.21.15.42:0/1607544556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2114005040 con 0x7f21240f6960 2024-02-15T18:22:44.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.632+0000 7f2129ad3700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21240f6960 0x7f212410c400 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f2114009830 tx=0x7f2114009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:44.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.632+0000 7f211affd700 1 -- 172.21.15.42:0/1607544556 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2114018070 con 0x7f21240f6960 2024-02-15T18:22:44.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.633+0000 7f211affd700 1 -- 172.21.15.42:0/1607544556 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2114007ba0 con 0x7f21240f6960 2024-02-15T18:22:44.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.633+0000 7f211affd700 1 -- 172.21.15.42:0/1607544556 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f211401c3c0 con 0x7f21240f6960 2024-02-15T18:22:44.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.633+0000 7f212c538700 1 -- 172.21.15.42:0/1607544556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212410c950 con 0x7f21240f6960 2024-02-15T18:22:44.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.633+0000 7f212c538700 1 -- 172.21.15.42:0/1607544556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212410ce80 con 0x7f21240f6960 2024-02-15T18:22:44.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.633+0000 7f212c538700 1 -- 172.21.15.42:0/1607544556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f212405dd30 con 0x7f21240f6960 2024-02-15T18:22:44.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.634+0000 7f211affd700 1 -- 172.21.15.42:0/1607544556 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f2114007d10 con 0x7f21240f6960 2024-02-15T18:22:44.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.634+0000 7f211affd700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f211005c2a0 0x7f211005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:44.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.634+0000 7f211affd700 1 -- 172.21.15.42:0/1607544556 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f2114077a60 con 0x7f21240f6960 2024-02-15T18:22:44.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.635+0000 7f212a2d4700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f211005c2a0 0x7f211005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:44.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.635+0000 7f211affd700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965] conn(0x7f2110062080 0x7f21100644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:44.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.635+0000 7f211affd700 1 -- 172.21.15.42:0/1607544556 --> [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2110064b60 con 0x7f2110062080 2024-02-15T18:22:44.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.635+0000 7f211affd700 1 -- 172.21.15.42:0/1607544556 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2114077e60 con 0x7f21240f6960 2024-02-15T18:22:44.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.635+0000 7f212aad5700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965] conn(0x7f2110062080 0x7f21100644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:44.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.635+0000 7f212a2d4700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f211005c2a0 0x7f211005e760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f21240605a0 tx=0x7f212000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:44.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.635+0000 7f212aad5700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965] conn(0x7f2110062080 0x7f21100644e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:44.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.636+0000 7f211affd700 1 -- 172.21.15.42:0/1607544556 <== osd.1 v2:172.21.15.42:6810/563363965 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f2110064b60 con 0x7f2110062080 2024-02-15T18:22:44.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.650+0000 7f212c538700 1 -- 172.21.15.42:0/1607544556 --> [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f21240581d0 con 0x7f2110062080 2024-02-15T18:22:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f211affd700 1 -- 172.21.15.42:0/1607544556 <== osd.1 v2:172.21.15.42:6810/563363965 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f21240581d0 con 0x7f2110062080 2024-02-15T18:22:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f212c538700 1 -- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965] conn(0x7f2110062080 msgr2=0x7f21100644e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f212c538700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965] conn(0x7f2110062080 0x7f21100644e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f212c538700 1 -- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f211005c2a0 msgr2=0x7f211005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f212c538700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f211005c2a0 0x7f211005e760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f21240605a0 tx=0x7f212000b040).stop 2024-02-15T18:22:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f212c538700 1 -- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21240f6960 msgr2=0x7f212410c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f212c538700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21240f6960 0x7f212410c400 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f2114009830 tx=0x7f2114009b30).stop 2024-02-15T18:22:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f212c538700 1 -- 172.21.15.42:0/1607544556 shutdown_connections 2024-02-15T18:22:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f212c538700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f211005c2a0 0x7f211005e760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f212c538700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:6810/563363965,v1:172.21.15.42:6811/563363965] conn(0x7f2110062080 0x7f21100644e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f212c538700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f21240f56c0 0x7f2124109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f212c538700 1 --2- 172.21.15.42:0/1607544556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21240f6960 0x7f212410c400 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f212c538700 1 -- 172.21.15.42:0/1607544556 >> 172.21.15.42:0/1607544556 conn(0x7f21240f0e20 msgr2=0x7f21240fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f212c538700 1 -- 172.21.15.42:0/1607544556 shutdown_connections 2024-02-15T18:22:44.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:44.651+0000 7f212c538700 1 -- 172.21.15.42:0/1607544556 wait complete. 2024-02-15T18:22:44.679 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:42.807384+0000 mgr.smithi042.kfoihb (mgr.14162) 199 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:44.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:42.807384+0000 mgr.smithi042.kfoihb (mgr.14162) 199 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:45.416 INFO:teuthology.orchestra.run.smithi042.stdout:60129542168 2024-02-15T18:22:45.416 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-15T18:22:45.661 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:22:46.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:46 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:44.807776+0000 mgr.smithi042.kfoihb (mgr.14162) 200 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:46.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:44.807776+0000 mgr.smithi042.kfoihb (mgr.14162) 200 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:47.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.375+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/3564347666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b340f56c0 msgr2=0x7f9b340f5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:47.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.375+0000 7f9b3ae3a700 1 --2- 172.21.15.42:0/3564347666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b340f56c0 0x7f9b340f5ae0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f9b28004660 tx=0x7f9b28009b30).stop 2024-02-15T18:22:47.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.375+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/3564347666 shutdown_connections 2024-02-15T18:22:47.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.375+0000 7f9b3ae3a700 1 --2- 172.21.15.42:0/3564347666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b340f6960 0x7f9b340f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:47.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.375+0000 7f9b3ae3a700 1 --2- 172.21.15.42:0/3564347666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b340f56c0 0x7f9b340f5ae0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:47.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.375+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/3564347666 >> 172.21.15.42:0/3564347666 conn(0x7f9b340f0e20 msgr2=0x7f9b340f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:47.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.375+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/3564347666 shutdown_connections 2024-02-15T18:22:47.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.375+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/3564347666 wait complete. 2024-02-15T18:22:47.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.376+0000 7f9b3ae3a700 1 Processor -- start 2024-02-15T18:22:47.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.376+0000 7f9b3ae3a700 1 -- start start 2024-02-15T18:22:47.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.376+0000 7f9b3ae3a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b340f56c0 0x7f9b341060f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:47.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.376+0000 7f9b3ae3a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b340f6960 0x7f9b341065e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:47.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.376+0000 7f9b3ae3a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b34106b90 con 0x7f9b340f6960 2024-02-15T18:22:47.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.376+0000 7f9b3ae3a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b34106cd0 con 0x7f9b340f56c0 2024-02-15T18:22:47.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.376+0000 7f9b33fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b340f6960 0x7f9b341065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:47.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.376+0000 7f9b33fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b340f6960 0x7f9b341065e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44818/0 (socket says 172.21.15.42:44818) 2024-02-15T18:22:47.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.376+0000 7f9b33fff700 1 -- 172.21.15.42:0/534647939 learned_addr learned my addr 172.21.15.42:0/534647939 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:47.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.376+0000 7f9b38bd6700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b340f56c0 0x7f9b341060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:47.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.377+0000 7f9b33fff700 1 -- 172.21.15.42:0/534647939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b340f56c0 msgr2=0x7f9b341060f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:47.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.377+0000 7f9b33fff700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b340f56c0 0x7f9b341060f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:47.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.377+0000 7f9b33fff700 1 -- 172.21.15.42:0/534647939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b28005040 con 0x7f9b340f6960 2024-02-15T18:22:47.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.377+0000 7f9b33fff700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b340f6960 0x7f9b341065e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f9b200026d0 tx=0x7f9b20009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:47.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.377+0000 7f9b317fa700 1 -- 172.21.15.42:0/534647939 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b200079c0 con 0x7f9b340f6960 2024-02-15T18:22:47.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.377+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/534647939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b34107670 con 0x7f9b340f6960 2024-02-15T18:22:47.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.377+0000 7f9b317fa700 1 -- 172.21.15.42:0/534647939 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b2000f040 con 0x7f9b340f6960 2024-02-15T18:22:47.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.377+0000 7f9b317fa700 1 -- 172.21.15.42:0/534647939 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b20014980 con 0x7f9b340f6960 2024-02-15T18:22:47.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.377+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/534647939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b34107c30 con 0x7f9b340f6960 2024-02-15T18:22:47.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.378+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/534647939 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9b3405dd30 con 0x7f9b340f6960 2024-02-15T18:22:47.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.378+0000 7f9b317fa700 1 -- 172.21.15.42:0/534647939 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f9b200090d0 con 0x7f9b340f6960 2024-02-15T18:22:47.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.379+0000 7f9b317fa700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9b2405c2a0 0x7f9b2405e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:47.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.379+0000 7f9b317fa700 1 -- 172.21.15.42:0/534647939 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f9b20075fe0 con 0x7f9b340f6960 2024-02-15T18:22:47.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.379+0000 7f9b38bd6700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9b2405c2a0 0x7f9b2405e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:47.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.379+0000 7f9b317fa700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670] conn(0x7f9b24062030 0x7f9b24064490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:47.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.379+0000 7f9b317fa700 1 -- 172.21.15.42:0/534647939 --> [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b24064b10 con 0x7f9b24062030 2024-02-15T18:22:47.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.379+0000 7f9b317fa700 1 -- 172.21.15.42:0/534647939 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b200763e0 con 0x7f9b340f6960 2024-02-15T18:22:47.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.379+0000 7f9b393d7700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670] conn(0x7f9b24062030 0x7f9b24064490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:47.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.379+0000 7f9b38bd6700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9b2405c2a0 0x7f9b2405e760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f9b28004630 tx=0x7f9b28009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:47.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.379+0000 7f9b393d7700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670] conn(0x7f9b24062030 0x7f9b24064490 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:47.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.380+0000 7f9b317fa700 1 -- 172.21.15.42:0/534647939 <== osd.2 v2:172.21.15.42:6818/2706708670 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f9b24064b10 con 0x7f9b24062030 2024-02-15T18:22:47.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.394+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/534647939 --> [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9b341080f0 con 0x7f9b24062030 2024-02-15T18:22:47.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b317fa700 1 -- 172.21.15.42:0/534647939 <== osd.2 v2:172.21.15.42:6818/2706708670 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9b341080f0 con 0x7f9b24062030 2024-02-15T18:22:47.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670] conn(0x7f9b24062030 msgr2=0x7f9b24064490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:47.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b3ae3a700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670] conn(0x7f9b24062030 0x7f9b24064490 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:47.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9b2405c2a0 msgr2=0x7f9b2405e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:47.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b3ae3a700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9b2405c2a0 0x7f9b2405e760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f9b28004630 tx=0x7f9b28009b30).stop 2024-02-15T18:22:47.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b340f6960 msgr2=0x7f9b341065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:47.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b3ae3a700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b340f6960 0x7f9b341065e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f9b200026d0 tx=0x7f9b20009de0).stop 2024-02-15T18:22:47.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/534647939 shutdown_connections 2024-02-15T18:22:47.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b3ae3a700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9b2405c2a0 0x7f9b2405e760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:47.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b3ae3a700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:6818/2706708670,v1:172.21.15.42:6819/2706708670] conn(0x7f9b24062030 0x7f9b24064490 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:47.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b3ae3a700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9b340f56c0 0x7f9b341060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:47.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b3ae3a700 1 --2- 172.21.15.42:0/534647939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9b340f6960 0x7f9b341065e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:47.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/534647939 >> 172.21.15.42:0/534647939 conn(0x7f9b340f0e20 msgr2=0x7f9b340fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:47.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/534647939 shutdown_connections 2024-02-15T18:22:47.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:47.395+0000 7f9b3ae3a700 1 -- 172.21.15.42:0/534647939 wait complete. 2024-02-15T18:22:48.008 INFO:teuthology.orchestra.run.smithi042.stdout:81604378646 2024-02-15T18:22:48.009 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-15T18:22:48.244 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:22:48.688 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:46.808187+0000 mgr.smithi042.kfoihb (mgr.14162) 201 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:48.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:46.808187+0000 mgr.smithi042.kfoihb (mgr.14162) 201 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:50.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.057+0000 7fb25b999700 1 -- 172.21.15.42:0/871106172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2540f9950 msgr2=0x7fb2540fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:50.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.057+0000 7fb25b999700 1 --2- 172.21.15.42:0/871106172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2540f9950 0x7fb2540fbd80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fb248004660 tx=0x7fb248009b30).stop 2024-02-15T18:22:50.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.057+0000 7fb25b999700 1 -- 172.21.15.42:0/871106172 shutdown_connections 2024-02-15T18:22:50.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.057+0000 7fb25b999700 1 --2- 172.21.15.42:0/871106172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2540f9950 0x7fb2540fbd80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:50.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.057+0000 7fb25b999700 1 --2- 172.21.15.42:0/871106172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2540607b0 0x7fb2540f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:22:50.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.057+0000 7fb25b999700 1 -- 172.21.15.42:0/871106172 >> 172.21.15.42:0/871106172 conn(0x7fb2540f0e20 msgr2=0x7fb2540f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:50.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.057+0000 7fb25b999700 1 -- 172.21.15.42:0/871106172 shutdown_connections 2024-02-15T18:22:50.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.057+0000 7fb25b999700 1 -- 172.21.15.42:0/871106172 wait complete. 2024-02-15T18:22:50.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.058+0000 7fb25b999700 1 Processor -- start 2024-02-15T18:22:50.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.058+0000 7fb25b999700 1 -- start start 2024-02-15T18:22:50.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.058+0000 7fb25b999700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2540607b0 0x7fb2541071d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:50.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.058+0000 7fb25b999700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2540f9950 0x7fb2541076c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:50.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.058+0000 7fb25b999700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb254107d00 con 0x7fb2540607b0 2024-02-15T18:22:50.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.058+0000 7fb25b999700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2541085b0 con 0x7fb2540f9950 2024-02-15T18:22:50.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.058+0000 7fb259735700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2540607b0 0x7fb2541071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:50.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.058+0000 7fb259735700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2540607b0 0x7fb2541071d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44836/0 (socket says 172.21.15.42:44836) 2024-02-15T18:22:50.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.058+0000 7fb259735700 1 -- 172.21.15.42:0/2682059742 learned_addr learned my addr 172.21.15.42:0/2682059742 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:50.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.058+0000 7fb259735700 1 -- 172.21.15.42:0/2682059742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2540f9950 msgr2=0x7fb2541076c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:50.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.058+0000 7fb259735700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2540f9950 0x7fb2541076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:22:50.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.058+0000 7fb259735700 1 -- 172.21.15.42:0/2682059742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb248005040 con 0x7fb2540607b0 2024-02-15T18:22:50.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.058+0000 7fb259735700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2540607b0 0x7fb2541071d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fb25000b410 tx=0x7fb250006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:50.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.059+0000 7fb245ffb700 1 -- 172.21.15.42:0/2682059742 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2500125f0 con 0x7fb2540607b0 2024-02-15T18:22:50.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.059+0000 7fb25b999700 1 -- 172.21.15.42:0/2682059742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb254108890 con 0x7fb2540607b0 2024-02-15T18:22:50.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.059+0000 7fb245ffb700 1 -- 172.21.15.42:0/2682059742 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb250012c20 con 0x7fb2540607b0 2024-02-15T18:22:50.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.059+0000 7fb245ffb700 1 -- 172.21.15.42:0/2682059742 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb250002b30 con 0x7fb2540607b0 2024-02-15T18:22:50.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.059+0000 7fb25b999700 1 -- 172.21.15.42:0/2682059742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb254108e50 con 0x7fb2540607b0 2024-02-15T18:22:50.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.060+0000 7fb25b999700 1 -- 172.21.15.42:0/2682059742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb25405dd30 con 0x7fb2540607b0 2024-02-15T18:22:50.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.060+0000 7fb245ffb700 1 -- 172.21.15.42:0/2682059742 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fb25000f030 con 0x7fb2540607b0 2024-02-15T18:22:50.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.060+0000 7fb245ffb700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fb24005c2a0 0x7fb24005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:50.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.060+0000 7fb245ffb700 1 -- 172.21.15.42:0/2682059742 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fb250076060 con 0x7fb2540607b0 2024-02-15T18:22:50.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.060+0000 7fb258f34700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fb24005c2a0 0x7fb24005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:50.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.061+0000 7fb245ffb700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.191:6800/3223849184,v1:172.21.15.191:6801/3223849184] conn(0x7fb240062030 0x7fb240064490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:50.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.061+0000 7fb245ffb700 1 -- 172.21.15.42:0/2682059742 --> [v2:172.21.15.191:6800/3223849184,v1:172.21.15.191:6801/3223849184] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb240064b10 con 0x7fb240062030 2024-02-15T18:22:50.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.061+0000 7fb245ffb700 1 -- 172.21.15.42:0/2682059742 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2500786d0 con 0x7fb2540607b0 2024-02-15T18:22:50.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.061+0000 7fb258f34700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fb24005c2a0 0x7fb24005e760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fb248009f50 tx=0x7fb248009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:50.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.061+0000 7fb259f36700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.191:6800/3223849184,v1:172.21.15.191:6801/3223849184] conn(0x7fb240062030 0x7fb240064490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:50.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.061+0000 7fb259f36700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.191:6800/3223849184,v1:172.21.15.191:6801/3223849184] conn(0x7fb240062030 0x7fb240064490 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:50.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.062+0000 7fb245ffb700 1 -- 172.21.15.42:0/2682059742 <== osd.3 v2:172.21.15.191:6800/3223849184 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fb240064b10 con 0x7fb240062030 2024-02-15T18:22:50.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.076+0000 7fb25b999700 1 -- 172.21.15.42:0/2682059742 --> [v2:172.21.15.191:6800/3223849184,v1:172.21.15.191:6801/3223849184] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb254109330 con 0x7fb240062030 2024-02-15T18:22:50.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb245ffb700 1 -- 172.21.15.42:0/2682059742 <== osd.3 v2:172.21.15.191:6800/3223849184 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb254109330 con 0x7fb240062030 2024-02-15T18:22:50.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb25b999700 1 -- 172.21.15.42:0/2682059742 >> [v2:172.21.15.191:6800/3223849184,v1:172.21.15.191:6801/3223849184] conn(0x7fb240062030 msgr2=0x7fb240064490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:50.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb25b999700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.191:6800/3223849184,v1:172.21.15.191:6801/3223849184] conn(0x7fb240062030 0x7fb240064490 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:50.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb25b999700 1 -- 172.21.15.42:0/2682059742 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fb24005c2a0 msgr2=0x7fb24005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:50.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb25b999700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fb24005c2a0 0x7fb24005e760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fb248009f50 tx=0x7fb248009b30).stop 2024-02-15T18:22:50.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb25b999700 1 -- 172.21.15.42:0/2682059742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2540607b0 msgr2=0x7fb2541071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:50.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb25b999700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2540607b0 0x7fb2541071d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fb25000b410 tx=0x7fb250006ef0).stop 2024-02-15T18:22:50.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb25b999700 1 -- 172.21.15.42:0/2682059742 shutdown_connections 2024-02-15T18:22:50.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb25b999700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.191:6800/3223849184,v1:172.21.15.191:6801/3223849184] conn(0x7fb240062030 0x7fb240064490 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:50.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb25b999700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fb24005c2a0 0x7fb24005e760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:50.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb25b999700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2540607b0 0x7fb2541071d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:50.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb25b999700 1 --2- 172.21.15.42:0/2682059742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2540f9950 0x7fb2541076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:22:50.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb25b999700 1 -- 172.21.15.42:0/2682059742 >> 172.21.15.42:0/2682059742 conn(0x7fb2540f0e20 msgr2=0x7fb2540f3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:50.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb25b999700 1 -- 172.21.15.42:0/2682059742 shutdown_connections 2024-02-15T18:22:50.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:50.077+0000 7fb25b999700 1 -- 172.21.15.42:0/2682059742 wait complete. 2024-02-15T18:22:50.469 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:48.808659+0000 mgr.smithi042.kfoihb (mgr.14162) 202 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:50.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:48.808659+0000 mgr.smithi042.kfoihb (mgr.14162) 202 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:50.899 INFO:teuthology.orchestra.run.smithi042.stdout:115964117011 2024-02-15T18:22:50.899 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-15T18:22:51.138 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:22:52.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:50.809233+0000 mgr.smithi042.kfoihb (mgr.14162) 203 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:52.717 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:50.809233+0000 mgr.smithi042.kfoihb (mgr.14162) 203 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.970+0000 7fe2c66e8700 1 -- 172.21.15.42:0/312240013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2c00f7a70 msgr2=0x7fe2c00f7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.970+0000 7fe2c66e8700 1 --2- 172.21.15.42:0/312240013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2c00f7a70 0x7fe2c00f7ed0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fe2b4004660 tx=0x7fe2b4009b30).stop 2024-02-15T18:22:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.970+0000 7fe2c66e8700 1 -- 172.21.15.42:0/312240013 shutdown_connections 2024-02-15T18:22:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.970+0000 7fe2c66e8700 1 --2- 172.21.15.42:0/312240013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2c00f7a70 0x7fe2c00f7ed0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.970+0000 7fe2c66e8700 1 --2- 172.21.15.42:0/312240013 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2c00f67d0 0x7fe2c00f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:22:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.970+0000 7fe2c66e8700 1 -- 172.21.15.42:0/312240013 >> 172.21.15.42:0/312240013 conn(0x7fe2c00f1f50 msgr2=0x7fe2c00f4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.970+0000 7fe2c66e8700 1 -- 172.21.15.42:0/312240013 shutdown_connections 2024-02-15T18:22:52.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.970+0000 7fe2c66e8700 1 -- 172.21.15.42:0/312240013 wait complete. 2024-02-15T18:22:52.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.971+0000 7fe2c66e8700 1 Processor -- start 2024-02-15T18:22:52.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.971+0000 7fe2c66e8700 1 -- start start 2024-02-15T18:22:52.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2c66e8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2c00f67d0 0x7fe2c0107190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:52.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2c66e8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2c00f7a70 0x7fe2c0107680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:52.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2c66e8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c0107cc0 con 0x7fe2c00f67d0 2024-02-15T18:22:52.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2c66e8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c0108570 con 0x7fe2c00f7a70 2024-02-15T18:22:52.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2bffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2c00f67d0 0x7fe2c0107190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:52.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2bffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2c00f67d0 0x7fe2c0107190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34910/0 (socket says 172.21.15.42:34910) 2024-02-15T18:22:52.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2bffff700 1 -- 172.21.15.42:0/100012305 learned_addr learned my addr 172.21.15.42:0/100012305 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:52.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2bffff700 1 -- 172.21.15.42:0/100012305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2c00f7a70 msgr2=0x7fe2c0107680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:52.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2bffff700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2c00f7a70 0x7fe2c0107680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:22:52.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2bffff700 1 -- 172.21.15.42:0/100012305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b4005040 con 0x7fe2c00f67d0 2024-02-15T18:22:52.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2bffff700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2c00f67d0 0x7fe2c0107190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fe2b00093f0 tx=0x7fe2b0006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:52.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2bcff9700 1 -- 172.21.15.42:0/100012305 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2b0009ea0 con 0x7fe2c00f67d0 2024-02-15T18:22:52.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2bcff9700 1 -- 172.21.15.42:0/100012305 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2b0013b10 con 0x7fe2c00f67d0 2024-02-15T18:22:52.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2c66e8700 1 -- 172.21.15.42:0/100012305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c0108850 con 0x7fe2c00f67d0 2024-02-15T18:22:52.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2bcff9700 1 -- 172.21.15.42:0/100012305 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2b000dbc0 con 0x7fe2c00f67d0 2024-02-15T18:22:52.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.972+0000 7fe2c66e8700 1 -- 172.21.15.42:0/100012305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c0108e10 con 0x7fe2c00f67d0 2024-02-15T18:22:52.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.973+0000 7fe2c66e8700 1 -- 172.21.15.42:0/100012305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe2c005dd10 con 0x7fe2c00f67d0 2024-02-15T18:22:52.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.974+0000 7fe2bcff9700 1 -- 172.21.15.42:0/100012305 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fe2b001a060 con 0x7fe2c00f67d0 2024-02-15T18:22:52.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.974+0000 7fe2bcff9700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe2a805c2a0 0x7fe2a805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:52.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.974+0000 7fe2bcff9700 1 -- 172.21.15.42:0/100012305 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fe2b0075c10 con 0x7fe2c00f67d0 2024-02-15T18:22:52.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.974+0000 7fe2bf7fe700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe2a805c2a0 0x7fe2a805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:52.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.974+0000 7fe2bcff9700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.191:6808/424560222,v1:172.21.15.191:6809/424560222] conn(0x7fe2a8062030 0x7fe2a8064490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:52.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.974+0000 7fe2bcff9700 1 -- 172.21.15.42:0/100012305 --> [v2:172.21.15.191:6808/424560222,v1:172.21.15.191:6809/424560222] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2a8064b10 con 0x7fe2a8062030 2024-02-15T18:22:52.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.974+0000 7fe2bcff9700 1 -- 172.21.15.42:0/100012305 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2b007b070 con 0x7fe2c00f67d0 2024-02-15T18:22:52.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.974+0000 7fe2bf7fe700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe2a805c2a0 0x7fe2a805e760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fe2b4009f50 tx=0x7fe2b4009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:52.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.974+0000 7fe2c4c85700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.191:6808/424560222,v1:172.21.15.191:6809/424560222] conn(0x7fe2a8062030 0x7fe2a8064490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:52.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.975+0000 7fe2c4c85700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.191:6808/424560222,v1:172.21.15.191:6809/424560222] conn(0x7fe2a8062030 0x7fe2a8064490 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:52.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.976+0000 7fe2bcff9700 1 -- 172.21.15.42:0/100012305 <== osd.4 v2:172.21.15.191:6808/424560222 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fe2a8064b10 con 0x7fe2a8062030 2024-02-15T18:22:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.990+0000 7fe2c66e8700 1 -- 172.21.15.42:0/100012305 --> [v2:172.21.15.191:6808/424560222,v1:172.21.15.191:6809/424560222] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe2c0109350 con 0x7fe2a8062030 2024-02-15T18:22:52.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.990+0000 7fe2bcff9700 1 -- 172.21.15.42:0/100012305 <== osd.4 v2:172.21.15.191:6808/424560222 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe2c0109350 con 0x7fe2a8062030 2024-02-15T18:22:52.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.991+0000 7fe2c66e8700 1 -- 172.21.15.42:0/100012305 >> [v2:172.21.15.191:6808/424560222,v1:172.21.15.191:6809/424560222] conn(0x7fe2a8062030 msgr2=0x7fe2a8064490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.991+0000 7fe2c66e8700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.191:6808/424560222,v1:172.21.15.191:6809/424560222] conn(0x7fe2a8062030 0x7fe2a8064490 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.991+0000 7fe2c66e8700 1 -- 172.21.15.42:0/100012305 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe2a805c2a0 msgr2=0x7fe2a805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.991+0000 7fe2c66e8700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe2a805c2a0 0x7fe2a805e760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fe2b4009f50 tx=0x7fe2b4009b30).stop 2024-02-15T18:22:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.991+0000 7fe2c66e8700 1 -- 172.21.15.42:0/100012305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2c00f67d0 msgr2=0x7fe2c0107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.991+0000 7fe2c66e8700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2c00f67d0 0x7fe2c0107190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fe2b00093f0 tx=0x7fe2b0006ec0).stop 2024-02-15T18:22:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.991+0000 7fe2c66e8700 1 -- 172.21.15.42:0/100012305 shutdown_connections 2024-02-15T18:22:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.991+0000 7fe2c66e8700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.191:6808/424560222,v1:172.21.15.191:6809/424560222] conn(0x7fe2a8062030 0x7fe2a8064490 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.991+0000 7fe2c66e8700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe2a805c2a0 0x7fe2a805e760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.991+0000 7fe2c66e8700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2c00f67d0 0x7fe2c0107190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:52.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.991+0000 7fe2c66e8700 1 --2- 172.21.15.42:0/100012305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2c00f7a70 0x7fe2c0107680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:22:52.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.991+0000 7fe2c66e8700 1 -- 172.21.15.42:0/100012305 >> 172.21.15.42:0/100012305 conn(0x7fe2c00f1f50 msgr2=0x7fe2c00fb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:52.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.991+0000 7fe2c66e8700 1 -- 172.21.15.42:0/100012305 shutdown_connections 2024-02-15T18:22:52.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:52.991+0000 7fe2c66e8700 1 -- 172.21.15.42:0/100012305 wait complete. 2024-02-15T18:22:53.722 INFO:teuthology.orchestra.run.smithi042.stdout:141733920782 2024-02-15T18:22:53.722 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-15T18:22:53.961 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:22:54.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:52.809518+0000 mgr.smithi042.kfoihb (mgr.14162) 204 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:54.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:52.809518+0000 mgr.smithi042.kfoihb (mgr.14162) 204 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:55.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.628+0000 7fbc5c31f700 1 -- 172.21.15.42:0/2340770011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f56c0 msgr2=0x7fbc540f5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:55.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.628+0000 7fbc5c31f700 1 --2- 172.21.15.42:0/2340770011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f56c0 0x7fbc540f5ae0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fbc50004660 tx=0x7fbc50009b30).stop 2024-02-15T18:22:55.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.628+0000 7fbc5c31f700 1 -- 172.21.15.42:0/2340770011 shutdown_connections 2024-02-15T18:22:55.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.628+0000 7fbc5c31f700 1 --2- 172.21.15.42:0/2340770011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc540f6960 0x7fbc540f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:22:55.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.628+0000 7fbc5c31f700 1 --2- 172.21.15.42:0/2340770011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f56c0 0x7fbc540f5ae0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:55.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.628+0000 7fbc5c31f700 1 -- 172.21.15.42:0/2340770011 >> 172.21.15.42:0/2340770011 conn(0x7fbc540f0e20 msgr2=0x7fbc540f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:55.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.628+0000 7fbc5c31f700 1 -- 172.21.15.42:0/2340770011 shutdown_connections 2024-02-15T18:22:55.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.628+0000 7fbc5c31f700 1 -- 172.21.15.42:0/2340770011 wait complete. 2024-02-15T18:22:55.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.628+0000 7fbc5c31f700 1 Processor -- start 2024-02-15T18:22:55.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.629+0000 7fbc5c31f700 1 -- start start 2024-02-15T18:22:55.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.629+0000 7fbc5c31f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc540f56c0 0x7fbc54109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:55.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.629+0000 7fbc5c31f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6960 0x7fbc5410c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:55.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.629+0000 7fbc5c31f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540609b0 con 0x7fbc540f6960 2024-02-15T18:22:55.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.629+0000 7fbc5c31f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54060af0 con 0x7fbc540f56c0 2024-02-15T18:22:55.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.629+0000 7fbc598ba700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6960 0x7fbc5410c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:55.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.629+0000 7fbc598ba700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6960 0x7fbc5410c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34918/0 (socket says 172.21.15.42:34918) 2024-02-15T18:22:55.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.629+0000 7fbc598ba700 1 -- 172.21.15.42:0/3110312878 learned_addr learned my addr 172.21.15.42:0/3110312878 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:55.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.629+0000 7fbc5a0bb700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc540f56c0 0x7fbc54109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:55.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.629+0000 7fbc598ba700 1 -- 172.21.15.42:0/3110312878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc540f56c0 msgr2=0x7fbc54109ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:55.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.629+0000 7fbc598ba700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc540f56c0 0x7fbc54109ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:55.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.629+0000 7fbc598ba700 1 -- 172.21.15.42:0/3110312878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc50005040 con 0x7fbc540f6960 2024-02-15T18:22:55.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.629+0000 7fbc598ba700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6960 0x7fbc5410c3f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fbc4400b410 tx=0x7fbc44006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:55.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.630+0000 7fbc4affd700 1 -- 172.21.15.42:0/3110312878 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc440125f0 con 0x7fbc540f6960 2024-02-15T18:22:55.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.630+0000 7fbc5c31f700 1 -- 172.21.15.42:0/3110312878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5410ca00 con 0x7fbc540f6960 2024-02-15T18:22:55.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.630+0000 7fbc4affd700 1 -- 172.21.15.42:0/3110312878 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc44012c20 con 0x7fbc540f6960 2024-02-15T18:22:55.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.630+0000 7fbc4affd700 1 -- 172.21.15.42:0/3110312878 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc44011d70 con 0x7fbc540f6960 2024-02-15T18:22:55.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.630+0000 7fbc5c31f700 1 -- 172.21.15.42:0/3110312878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5410cfc0 con 0x7fbc540f6960 2024-02-15T18:22:55.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.630+0000 7fbc5c31f700 1 -- 172.21.15.42:0/3110312878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbc5405dd30 con 0x7fbc540f6960 2024-02-15T18:22:55.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.631+0000 7fbc4affd700 1 -- 172.21.15.42:0/3110312878 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fbc4400f030 con 0x7fbc540f6960 2024-02-15T18:22:55.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.631+0000 7fbc4affd700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fbc4005c2a0 0x7fbc4005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:55.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.631+0000 7fbc4affd700 1 -- 172.21.15.42:0/3110312878 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fbc44075f40 con 0x7fbc540f6960 2024-02-15T18:22:55.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.631+0000 7fbc5a0bb700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fbc4005c2a0 0x7fbc4005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:55.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.631+0000 7fbc4affd700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.191:6816/2014313522,v1:172.21.15.191:6817/2014313522] conn(0x7fbc40062030 0x7fbc40064490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:55.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.631+0000 7fbc4affd700 1 -- 172.21.15.42:0/3110312878 --> [v2:172.21.15.191:6816/2014313522,v1:172.21.15.191:6817/2014313522] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbc40064b10 con 0x7fbc40062030 2024-02-15T18:22:55.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.631+0000 7fbc4affd700 1 -- 172.21.15.42:0/3110312878 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc44078550 con 0x7fbc540f6960 2024-02-15T18:22:55.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.632+0000 7fbc5a0bb700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fbc4005c2a0 0x7fbc4005e760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fbc540605a0 tx=0x7fbc50009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:55.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.632+0000 7fbc5a8bc700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.191:6816/2014313522,v1:172.21.15.191:6817/2014313522] conn(0x7fbc40062030 0x7fbc40064490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:55.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.632+0000 7fbc5a8bc700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.191:6816/2014313522,v1:172.21.15.191:6817/2014313522] conn(0x7fbc40062030 0x7fbc40064490 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:55.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.633+0000 7fbc4affd700 1 -- 172.21.15.42:0/3110312878 <== osd.5 v2:172.21.15.191:6816/2014313522 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fbc40064b10 con 0x7fbc40062030 2024-02-15T18:22:55.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.647+0000 7fbc5c31f700 1 -- 172.21.15.42:0/3110312878 --> [v2:172.21.15.191:6816/2014313522,v1:172.21.15.191:6817/2014313522] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbc54058150 con 0x7fbc40062030 2024-02-15T18:22:55.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc4affd700 1 -- 172.21.15.42:0/3110312878 <== osd.5 v2:172.21.15.191:6816/2014313522 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbc54058150 con 0x7fbc40062030 2024-02-15T18:22:55.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc5c31f700 1 -- 172.21.15.42:0/3110312878 >> [v2:172.21.15.191:6816/2014313522,v1:172.21.15.191:6817/2014313522] conn(0x7fbc40062030 msgr2=0x7fbc40064490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:55.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc5c31f700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.191:6816/2014313522,v1:172.21.15.191:6817/2014313522] conn(0x7fbc40062030 0x7fbc40064490 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:55.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc5c31f700 1 -- 172.21.15.42:0/3110312878 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fbc4005c2a0 msgr2=0x7fbc4005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:55.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc5c31f700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fbc4005c2a0 0x7fbc4005e760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fbc540605a0 tx=0x7fbc50009b30).stop 2024-02-15T18:22:55.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc5c31f700 1 -- 172.21.15.42:0/3110312878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6960 msgr2=0x7fbc5410c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:55.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc5c31f700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6960 0x7fbc5410c3f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fbc4400b410 tx=0x7fbc44006ef0).stop 2024-02-15T18:22:55.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc5c31f700 1 -- 172.21.15.42:0/3110312878 shutdown_connections 2024-02-15T18:22:55.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc5c31f700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.191:6816/2014313522,v1:172.21.15.191:6817/2014313522] conn(0x7fbc40062030 0x7fbc40064490 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:55.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc5c31f700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fbc4005c2a0 0x7fbc4005e760 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:55.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc5c31f700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc540f56c0 0x7fbc54109ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:55.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc5c31f700 1 --2- 172.21.15.42:0/3110312878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6960 0x7fbc5410c3f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:55.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc5c31f700 1 -- 172.21.15.42:0/3110312878 >> 172.21.15.42:0/3110312878 conn(0x7fbc540f0e20 msgr2=0x7fbc540fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:55.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc5c31f700 1 -- 172.21.15.42:0/3110312878 shutdown_connections 2024-02-15T18:22:55.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:55.648+0000 7fbc5c31f700 1 -- 172.21.15.42:0/3110312878 wait complete. 2024-02-15T18:22:56.311 INFO:teuthology.orchestra.run.smithi042.stdout:163208757259 2024-02-15T18:22:56.311 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-15T18:22:56.551 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:22:56.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:54.810029+0000 mgr.smithi042.kfoihb (mgr.14162) 205 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:56.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:54.810029+0000 mgr.smithi042.kfoihb (mgr.14162) 205 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:57.668 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:56.914222+0000 mon.smithi042 (mon.0) 443 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:22:57.668 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:56.914406+0000 mon.smithi042 (mon.0) 444 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:22:57.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:56.914222+0000 mon.smithi042 (mon.0) 443 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:22:57.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:56.914406+0000 mon.smithi042 (mon.0) 444 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:22:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.143+0000 7fdd901a1700 1 -- 172.21.15.42:0/1500315431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd880f4a70 msgr2=0x7fdd880f4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.143+0000 7fdd901a1700 1 --2- 172.21.15.42:0/1500315431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd880f4a70 0x7fdd880f4e90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fdd84004660 tx=0x7fdd84009b30).stop 2024-02-15T18:22:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.143+0000 7fdd901a1700 1 -- 172.21.15.42:0/1500315431 shutdown_connections 2024-02-15T18:22:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.143+0000 7fdd901a1700 1 --2- 172.21.15.42:0/1500315431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd880f5410 0x7fdd880f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:22:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.143+0000 7fdd901a1700 1 --2- 172.21.15.42:0/1500315431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd880f4a70 0x7fdd880f4e90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.143+0000 7fdd901a1700 1 -- 172.21.15.42:0/1500315431 >> 172.21.15.42:0/1500315431 conn(0x7fdd880f0590 msgr2=0x7fdd880f29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.144+0000 7fdd901a1700 1 -- 172.21.15.42:0/1500315431 shutdown_connections 2024-02-15T18:22:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.144+0000 7fdd901a1700 1 -- 172.21.15.42:0/1500315431 wait complete. 2024-02-15T18:22:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.144+0000 7fdd901a1700 1 Processor -- start 2024-02-15T18:22:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.144+0000 7fdd901a1700 1 -- start start 2024-02-15T18:22:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.144+0000 7fdd901a1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd880f4a70 0x7fdd88107200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.144+0000 7fdd901a1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd880f5410 0x7fdd881076f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.144+0000 7fdd901a1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd88107d30 con 0x7fdd880f5410 2024-02-15T18:22:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.144+0000 7fdd901a1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd881085e0 con 0x7fdd880f4a70 2024-02-15T18:22:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.144+0000 7fdd8d73c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd880f5410 0x7fdd881076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.144+0000 7fdd8d73c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd880f5410 0x7fdd881076f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34936/0 (socket says 172.21.15.42:34936) 2024-02-15T18:22:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.144+0000 7fdd8d73c700 1 -- 172.21.15.42:0/4035830072 learned_addr learned my addr 172.21.15.42:0/4035830072 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:22:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.145+0000 7fdd8df3d700 1 --2- 172.21.15.42:0/4035830072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd880f4a70 0x7fdd88107200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.145+0000 7fdd8d73c700 1 -- 172.21.15.42:0/4035830072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd880f4a70 msgr2=0x7fdd88107200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.145+0000 7fdd8d73c700 1 --2- 172.21.15.42:0/4035830072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd880f4a70 0x7fdd88107200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.145+0000 7fdd8d73c700 1 -- 172.21.15.42:0/4035830072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84005040 con 0x7fdd880f5410 2024-02-15T18:22:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.145+0000 7fdd8d73c700 1 --2- 172.21.15.42:0/4035830072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd880f5410 0x7fdd881076f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fdd7c0093f0 tx=0x7fdd7c006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:58.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.145+0000 7fdd7a7fc700 1 -- 172.21.15.42:0/4035830072 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd7c013640 con 0x7fdd880f5410 2024-02-15T18:22:58.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.145+0000 7fdd7a7fc700 1 -- 172.21.15.42:0/4035830072 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd7c013c70 con 0x7fdd880f5410 2024-02-15T18:22:58.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.145+0000 7fdd901a1700 1 -- 172.21.15.42:0/4035830072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd881088c0 con 0x7fdd880f5410 2024-02-15T18:22:58.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.145+0000 7fdd7a7fc700 1 -- 172.21.15.42:0/4035830072 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd7c012d80 con 0x7fdd880f5410 2024-02-15T18:22:58.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.145+0000 7fdd901a1700 1 -- 172.21.15.42:0/4035830072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd88108e80 con 0x7fdd880f5410 2024-02-15T18:22:58.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.146+0000 7fdd901a1700 1 -- 172.21.15.42:0/4035830072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd88059100 con 0x7fdd880f5410 2024-02-15T18:22:58.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.146+0000 7fdd7a7fc700 1 -- 172.21.15.42:0/4035830072 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fdd7c01a060 con 0x7fdd880f5410 2024-02-15T18:22:58.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.147+0000 7fdd7a7fc700 1 --2- 172.21.15.42:0/4035830072 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fdd7405c2a0 0x7fdd7405e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:22:58.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.147+0000 7fdd7a7fc700 1 -- 172.21.15.42:0/4035830072 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fdd7c075a70 con 0x7fdd880f5410 2024-02-15T18:22:58.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.147+0000 7fdd8df3d700 1 --2- 172.21.15.42:0/4035830072 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fdd7405c2a0 0x7fdd7405e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:22:58.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.147+0000 7fdd8df3d700 1 --2- 172.21.15.42:0/4035830072 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fdd7405c2a0 0x7fdd7405e760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fdd84009f50 tx=0x7fdd84009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:22:58.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.151+0000 7fdd7a7fc700 1 -- 172.21.15.42:0/4035830072 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fdd7c048af0 con 0x7fdd880f5410 2024-02-15T18:22:58.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.308+0000 7fdd901a1700 1 -- 172.21.15.42:0/4035830072 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdd880ff140 con 0x7fdd880f5410 2024-02-15T18:22:58.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.309+0000 7fdd7a7fc700 1 -- 172.21.15.42:0/4035830072 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdd7c075e60 con 0x7fdd880f5410 2024-02-15T18:22:58.309 INFO:teuthology.orchestra.run.smithi042.stdout:38654705698 2024-02-15T18:22:58.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.310+0000 7fdd901a1700 1 -- 172.21.15.42:0/4035830072 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fdd7405c2a0 msgr2=0x7fdd7405e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:58.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.310+0000 7fdd901a1700 1 --2- 172.21.15.42:0/4035830072 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fdd7405c2a0 0x7fdd7405e760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fdd84009f50 tx=0x7fdd84009b30).stop 2024-02-15T18:22:58.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.310+0000 7fdd901a1700 1 -- 172.21.15.42:0/4035830072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd880f5410 msgr2=0x7fdd881076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:22:58.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.310+0000 7fdd901a1700 1 --2- 172.21.15.42:0/4035830072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd880f5410 0x7fdd881076f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fdd7c0093f0 tx=0x7fdd7c006ec0).stop 2024-02-15T18:22:58.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.310+0000 7fdd901a1700 1 -- 172.21.15.42:0/4035830072 shutdown_connections 2024-02-15T18:22:58.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.310+0000 7fdd901a1700 1 --2- 172.21.15.42:0/4035830072 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fdd7405c2a0 0x7fdd7405e760 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:58.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.310+0000 7fdd901a1700 1 --2- 172.21.15.42:0/4035830072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd880f4a70 0x7fdd88107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:58.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.310+0000 7fdd901a1700 1 --2- 172.21.15.42:0/4035830072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd880f5410 0x7fdd881076f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:22:58.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.310+0000 7fdd901a1700 1 -- 172.21.15.42:0/4035830072 >> 172.21.15.42:0/4035830072 conn(0x7fdd880f0590 msgr2=0x7fdd880fb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:22:58.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.310+0000 7fdd901a1700 1 -- 172.21.15.42:0/4035830072 shutdown_connections 2024-02-15T18:22:58.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:22:58.310+0000 7fdd901a1700 1 -- 172.21.15.42:0/4035830072 wait complete. 2024-02-15T18:22:58.669 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:56.810307+0000 mgr.smithi042.kfoihb (mgr.14162) 206 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:58.669 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:22:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:22:58.310082+0000 mon.smithi042 (mon.0) 445 : audit [DBG] from='client.? 172.21.15.42:0/4035830072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-15T18:22:58.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:56.810307+0000 mgr.smithi042.kfoihb (mgr.14162) 206 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:22:58.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:22:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:22:58.310082+0000 mon.smithi042 (mon.0) 445 : audit [DBG] from='client.? 172.21.15.42:0/4035830072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-15T18:22:59.057 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705695 got 38654705698 for osd.0 2024-02-15T18:22:59.058 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-15T18:22:59.297 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:23:00.491 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:22:58.810626+0000 mgr.smithi042.kfoihb (mgr.14162) 207 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:00.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:22:58.810626+0000 mgr.smithi042.kfoihb (mgr.14162) 207 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:00.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.986+0000 7f14d574c700 1 -- 172.21.15.42:0/1449454350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d0060870 msgr2=0x7f14d00f93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:00.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.986+0000 7f14d574c700 1 --2- 172.21.15.42:0/1449454350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d0060870 0x7f14d00f93c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f14c4004660 tx=0x7f14c4009b30).stop 2024-02-15T18:23:00.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.986+0000 7f14d574c700 1 -- 172.21.15.42:0/1449454350 shutdown_connections 2024-02-15T18:23:00.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.986+0000 7f14d574c700 1 --2- 172.21.15.42:0/1449454350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14d00f9940 0x7f14d00fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:00.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.986+0000 7f14d574c700 1 --2- 172.21.15.42:0/1449454350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d0060870 0x7f14d00f93c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:00.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.986+0000 7f14d574c700 1 -- 172.21.15.42:0/1449454350 >> 172.21.15.42:0/1449454350 conn(0x7f14d00f0e20 msgr2=0x7f14d00f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:00.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.986+0000 7f14d574c700 1 -- 172.21.15.42:0/1449454350 shutdown_connections 2024-02-15T18:23:00.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.986+0000 7f14d574c700 1 -- 172.21.15.42:0/1449454350 wait complete. 2024-02-15T18:23:00.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.986+0000 7f14d574c700 1 Processor -- start 2024-02-15T18:23:00.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.986+0000 7f14d574c700 1 -- start start 2024-02-15T18:23:00.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14d574c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d0060870 0x7f14d00f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:00.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14d574c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14d00f9940 0x7f14d00f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:00.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14d574c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00f5820 con 0x7f14d0060870 2024-02-15T18:23:00.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14d574c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00f5960 con 0x7f14d00f9940 2024-02-15T18:23:00.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14ceffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d0060870 0x7f14d00f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14ceffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d0060870 0x7f14d00f6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34962/0 (socket says 172.21.15.42:34962) 2024-02-15T18:23:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14ceffd700 1 -- 172.21.15.42:0/1024089954 learned_addr learned my addr 172.21.15.42:0/1024089954 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14ce7fc700 1 --2- 172.21.15.42:0/1024089954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14d00f9940 0x7f14d00f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14ceffd700 1 -- 172.21.15.42:0/1024089954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14d00f9940 msgr2=0x7f14d00f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14ceffd700 1 --2- 172.21.15.42:0/1024089954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14d00f9940 0x7f14d00f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14ceffd700 1 -- 172.21.15.42:0/1024089954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c4005040 con 0x7f14d0060870 2024-02-15T18:23:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14ce7fc700 1 --2- 172.21.15.42:0/1024089954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14d00f9940 0x7f14d00f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:23:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14ceffd700 1 --2- 172.21.15.42:0/1024089954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d0060870 0x7f14d00f6e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f14c4009830 tx=0x7f14c4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14b77fe700 1 -- 172.21.15.42:0/1024089954 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14c4018070 con 0x7f14d0060870 2024-02-15T18:23:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14d574c700 1 -- 172.21.15.42:0/1024089954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d00f5be0 con 0x7f14d0060870 2024-02-15T18:23:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14b77fe700 1 -- 172.21.15.42:0/1024089954 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f14c4007d70 con 0x7f14d0060870 2024-02-15T18:23:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14b77fe700 1 -- 172.21.15.42:0/1024089954 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14c401c620 con 0x7f14d0060870 2024-02-15T18:23:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.987+0000 7f14d574c700 1 -- 172.21.15.42:0/1024089954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d00f6140 con 0x7f14d0060870 2024-02-15T18:23:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.988+0000 7f14d574c700 1 -- 172.21.15.42:0/1024089954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d0045ad0 con 0x7f14d0060870 2024-02-15T18:23:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.989+0000 7f14b77fe700 1 -- 172.21.15.42:0/1024089954 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f14c401d980 con 0x7f14d0060870 2024-02-15T18:23:00.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.989+0000 7f14b77fe700 1 --2- 172.21.15.42:0/1024089954 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f14b805c2a0 0x7f14b805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:00.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.989+0000 7f14b77fe700 1 -- 172.21.15.42:0/1024089954 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f14c4077a70 con 0x7f14d0060870 2024-02-15T18:23:00.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.989+0000 7f14ce7fc700 1 --2- 172.21.15.42:0/1024089954 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f14b805c2a0 0x7f14b805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:00.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.990+0000 7f14ce7fc700 1 --2- 172.21.15.42:0/1024089954 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f14b805c2a0 0x7f14b805e760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f14d0060640 tx=0x7f14c0009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:00.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:00.993+0000 7f14b77fe700 1 -- 172.21.15.42:0/1024089954 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f14c404aa10 con 0x7f14d0060870 2024-02-15T18:23:01.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:01.152+0000 7f14d574c700 1 -- 172.21.15.42:0/1024089954 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f14d0027a00 con 0x7f14d0060870 2024-02-15T18:23:01.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:01.152+0000 7f14b77fe700 1 -- 172.21.15.42:0/1024089954 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14c4077e60 con 0x7f14d0060870 2024-02-15T18:23:01.153 INFO:teuthology.orchestra.run.smithi042.stdout:60129542171 2024-02-15T18:23:01.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:01.154+0000 7f14d574c700 1 -- 172.21.15.42:0/1024089954 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f14b805c2a0 msgr2=0x7f14b805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:01.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:01.154+0000 7f14d574c700 1 --2- 172.21.15.42:0/1024089954 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f14b805c2a0 0x7f14b805e760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f14d0060640 tx=0x7f14c0009040).stop 2024-02-15T18:23:01.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:01.154+0000 7f14d574c700 1 -- 172.21.15.42:0/1024089954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d0060870 msgr2=0x7f14d00f6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:01.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:01.154+0000 7f14d574c700 1 --2- 172.21.15.42:0/1024089954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d0060870 0x7f14d00f6e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f14c4009830 tx=0x7f14c4009b30).stop 2024-02-15T18:23:01.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:01.154+0000 7f14d574c700 1 -- 172.21.15.42:0/1024089954 shutdown_connections 2024-02-15T18:23:01.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:01.154+0000 7f14d574c700 1 --2- 172.21.15.42:0/1024089954 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f14b805c2a0 0x7f14b805e760 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:01.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:01.154+0000 7f14d574c700 1 --2- 172.21.15.42:0/1024089954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d0060870 0x7f14d00f6e40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:01.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:01.154+0000 7f14d574c700 1 --2- 172.21.15.42:0/1024089954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14d00f9940 0x7f14d00f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:01.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:01.154+0000 7f14d574c700 1 -- 172.21.15.42:0/1024089954 >> 172.21.15.42:0/1024089954 conn(0x7f14d00f0e20 msgr2=0x7f14d00f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:01.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:01.154+0000 7f14d574c700 1 -- 172.21.15.42:0/1024089954 shutdown_connections 2024-02-15T18:23:01.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:01.154+0000 7f14d574c700 1 -- 172.21.15.42:0/1024089954 wait complete. 2024-02-15T18:23:01.484 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:01.153323+0000 mon.smithi042 (mon.0) 446 : audit [DBG] from='client.? 172.21.15.42:0/1024089954' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-15T18:23:01.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:01.153323+0000 mon.smithi042 (mon.0) 446 : audit [DBG] from='client.? 172.21.15.42:0/1024089954' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-15T18:23:01.798 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542168 got 60129542171 for osd.1 2024-02-15T18:23:01.798 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-15T18:23:02.035 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:23:02.504 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:00.811117+0000 mgr.smithi042.kfoihb (mgr.14162) 208 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:02.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:00.811117+0000 mgr.smithi042.kfoihb (mgr.14162) 208 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:03.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.804+0000 7fa86205b700 1 -- 172.21.15.42:0/2387021938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c0f67e0 msgr2=0x7fa85c0f6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.804+0000 7fa86205b700 1 --2- 172.21.15.42:0/2387021938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c0f67e0 0x7fa85c0f6c00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fa84c004660 tx=0x7fa84c009b30).stop 2024-02-15T18:23:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.804+0000 7fa86205b700 1 -- 172.21.15.42:0/2387021938 shutdown_connections 2024-02-15T18:23:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.804+0000 7fa86205b700 1 --2- 172.21.15.42:0/2387021938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa85c0f7a80 0x7fa85c0f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.804+0000 7fa86205b700 1 --2- 172.21.15.42:0/2387021938 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c0f67e0 0x7fa85c0f6c00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.804+0000 7fa86205b700 1 -- 172.21.15.42:0/2387021938 >> 172.21.15.42:0/2387021938 conn(0x7fa85c0f1f40 msgr2=0x7fa85c0f4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.804+0000 7fa86205b700 1 -- 172.21.15.42:0/2387021938 shutdown_connections 2024-02-15T18:23:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.804+0000 7fa86205b700 1 -- 172.21.15.42:0/2387021938 wait complete. 2024-02-15T18:23:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.805+0000 7fa86205b700 1 Processor -- start 2024-02-15T18:23:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.805+0000 7fa86205b700 1 -- start start 2024-02-15T18:23:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.805+0000 7fa86205b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c0f67e0 0x7fa85c1071e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.805+0000 7fa86205b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa85c0f7a80 0x7fa85c1076d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.805+0000 7fa86205b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c107c80 con 0x7fa85c0f67e0 2024-02-15T18:23:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.805+0000 7fa86205b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c107dc0 con 0x7fa85c0f7a80 2024-02-15T18:23:03.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.805+0000 7fa85b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c0f67e0 0x7fa85c1071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:03.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.805+0000 7fa85b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c0f67e0 0x7fa85c1071e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44188/0 (socket says 172.21.15.42:44188) 2024-02-15T18:23:03.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.805+0000 7fa85b7fe700 1 -- 172.21.15.42:0/2907475662 learned_addr learned my addr 172.21.15.42:0/2907475662 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:03.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.806+0000 7fa85b7fe700 1 -- 172.21.15.42:0/2907475662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa85c0f7a80 msgr2=0x7fa85c1076d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:03.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.806+0000 7fa85affd700 1 --2- 172.21.15.42:0/2907475662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa85c0f7a80 0x7fa85c1076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:03.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.806+0000 7fa85b7fe700 1 --2- 172.21.15.42:0/2907475662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa85c0f7a80 0x7fa85c1076d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:03.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.806+0000 7fa85b7fe700 1 -- 172.21.15.42:0/2907475662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c005040 con 0x7fa85c0f67e0 2024-02-15T18:23:03.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.806+0000 7fa85b7fe700 1 --2- 172.21.15.42:0/2907475662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c0f67e0 0x7fa85c1071e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fa84c009f50 tx=0x7fa84c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:03.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.806+0000 7fa843fff700 1 -- 172.21.15.42:0/2907475662 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa84c018070 con 0x7fa85c0f67e0 2024-02-15T18:23:03.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.806+0000 7fa86205b700 1 -- 172.21.15.42:0/2907475662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85c108700 con 0x7fa85c0f67e0 2024-02-15T18:23:03.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.806+0000 7fa843fff700 1 -- 172.21.15.42:0/2907475662 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa84c007d70 con 0x7fa85c0f67e0 2024-02-15T18:23:03.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.806+0000 7fa843fff700 1 -- 172.21.15.42:0/2907475662 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa84c01c620 con 0x7fa85c0f67e0 2024-02-15T18:23:03.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.806+0000 7fa86205b700 1 -- 172.21.15.42:0/2907475662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85c108c60 con 0x7fa85c0f67e0 2024-02-15T18:23:03.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.807+0000 7fa86205b700 1 -- 172.21.15.42:0/2907475662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85c045ad0 con 0x7fa85c0f67e0 2024-02-15T18:23:03.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.807+0000 7fa843fff700 1 -- 172.21.15.42:0/2907475662 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fa84c01c8e0 con 0x7fa85c0f67e0 2024-02-15T18:23:03.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.808+0000 7fa843fff700 1 --2- 172.21.15.42:0/2907475662 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa84405c2a0 0x7fa84405e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:03.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.808+0000 7fa843fff700 1 -- 172.21.15.42:0/2907475662 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fa84c077a40 con 0x7fa85c0f67e0 2024-02-15T18:23:03.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.808+0000 7fa85affd700 1 --2- 172.21.15.42:0/2907475662 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa84405c2a0 0x7fa84405e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:03.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.808+0000 7fa85affd700 1 --2- 172.21.15.42:0/2907475662 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa84405c2a0 0x7fa84405e760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fa8500041d0 tx=0x7fa850009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:03.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.812+0000 7fa843fff700 1 -- 172.21.15.42:0/2907475662 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa84c04aa40 con 0x7fa85c0f67e0 2024-02-15T18:23:03.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.969+0000 7fa86205b700 1 -- 172.21.15.42:0/2907475662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa85c027a00 con 0x7fa85c0f67e0 2024-02-15T18:23:03.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.970+0000 7fa843fff700 1 -- 172.21.15.42:0/2907475662 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa84c04e060 con 0x7fa85c0f67e0 2024-02-15T18:23:03.970 INFO:teuthology.orchestra.run.smithi042.stdout:81604378649 2024-02-15T18:23:03.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.971+0000 7fa86205b700 1 -- 172.21.15.42:0/2907475662 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa84405c2a0 msgr2=0x7fa84405e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:03.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.971+0000 7fa86205b700 1 --2- 172.21.15.42:0/2907475662 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa84405c2a0 0x7fa84405e760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fa8500041d0 tx=0x7fa850009040).stop 2024-02-15T18:23:03.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.971+0000 7fa86205b700 1 -- 172.21.15.42:0/2907475662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c0f67e0 msgr2=0x7fa85c1071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:03.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.971+0000 7fa86205b700 1 --2- 172.21.15.42:0/2907475662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c0f67e0 0x7fa85c1071e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fa84c009f50 tx=0x7fa84c009b30).stop 2024-02-15T18:23:03.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.971+0000 7fa86205b700 1 -- 172.21.15.42:0/2907475662 shutdown_connections 2024-02-15T18:23:03.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.971+0000 7fa86205b700 1 --2- 172.21.15.42:0/2907475662 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa84405c2a0 0x7fa84405e760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:03.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.971+0000 7fa86205b700 1 --2- 172.21.15.42:0/2907475662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa85c0f67e0 0x7fa85c1071e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:03.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.971+0000 7fa86205b700 1 --2- 172.21.15.42:0/2907475662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa85c0f7a80 0x7fa85c1076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:03.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.971+0000 7fa86205b700 1 -- 172.21.15.42:0/2907475662 >> 172.21.15.42:0/2907475662 conn(0x7fa85c0f1f40 msgr2=0x7fa85c0fb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:03.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.972+0000 7fa86205b700 1 -- 172.21.15.42:0/2907475662 shutdown_connections 2024-02-15T18:23:03.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:03.972+0000 7fa86205b700 1 -- 172.21.15.42:0/2907475662 wait complete. 2024-02-15T18:23:04.646 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:02.811571+0000 mgr.smithi042.kfoihb (mgr.14162) 209 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:04.646 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:03.971091+0000 mon.smithi042 (mon.0) 447 : audit [DBG] from='client.? 172.21.15.42:0/2907475662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-15T18:23:04.670 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378646 got 81604378649 for osd.2 2024-02-15T18:23:04.670 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-15T18:23:04.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:02.811571+0000 mgr.smithi042.kfoihb (mgr.14162) 209 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:04.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:03.971091+0000 mon.smithi042 (mon.0) 447 : audit [DBG] from='client.? 172.21.15.42:0/2907475662' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-15T18:23:04.910 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:23:06.540 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:04.811884+0000 mgr.smithi042.kfoihb (mgr.14162) 210 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:06.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:04.811884+0000 mgr.smithi042.kfoihb (mgr.14162) 210 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.766+0000 7fe47a261700 1 -- 172.21.15.42:0/2624159423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4740f9930 msgr2=0x7fe4740fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.766+0000 7fe47a261700 1 --2- 172.21.15.42:0/2624159423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4740f9930 0x7fe4740fbd80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fe464004660 tx=0x7fe464009b30).stop 2024-02-15T18:23:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.766+0000 7fe47a261700 1 -- 172.21.15.42:0/2624159423 shutdown_connections 2024-02-15T18:23:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.766+0000 7fe47a261700 1 --2- 172.21.15.42:0/2624159423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4740f9930 0x7fe4740fbd80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.766+0000 7fe47a261700 1 --2- 172.21.15.42:0/2624159423 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4740607d0 0x7fe4740f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.766+0000 7fe47a261700 1 -- 172.21.15.42:0/2624159423 >> 172.21.15.42:0/2624159423 conn(0x7fe4740f0e20 msgr2=0x7fe4740f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:06.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.766+0000 7fe47a261700 1 -- 172.21.15.42:0/2624159423 shutdown_connections 2024-02-15T18:23:06.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.766+0000 7fe47a261700 1 -- 172.21.15.42:0/2624159423 wait complete. 2024-02-15T18:23:06.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.767+0000 7fe47a261700 1 Processor -- start 2024-02-15T18:23:06.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.767+0000 7fe47a261700 1 -- start start 2024-02-15T18:23:06.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.767+0000 7fe47a261700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4740607d0 0x7fe4740f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.767+0000 7fe47a261700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4740f9930 0x7fe4740f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.767+0000 7fe47a261700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740f58a0 con 0x7fe4740f9930 2024-02-15T18:23:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.767+0000 7fe47a261700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740f59e0 con 0x7fe4740607d0 2024-02-15T18:23:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.767+0000 7fe472ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4740f9930 0x7fe4740f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.768+0000 7fe472ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4740f9930 0x7fe4740f5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44198/0 (socket says 172.21.15.42:44198) 2024-02-15T18:23:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.768+0000 7fe472ffd700 1 -- 172.21.15.42:0/1709046612 learned_addr learned my addr 172.21.15.42:0/1709046612 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.768+0000 7fe4737fe700 1 --2- 172.21.15.42:0/1709046612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4740607d0 0x7fe4740f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.768+0000 7fe472ffd700 1 -- 172.21.15.42:0/1709046612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4740607d0 msgr2=0x7fe4740f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.768+0000 7fe472ffd700 1 --2- 172.21.15.42:0/1709046612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4740607d0 0x7fe4740f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.768+0000 7fe472ffd700 1 -- 172.21.15.42:0/1709046612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe464005040 con 0x7fe4740f9930 2024-02-15T18:23:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.768+0000 7fe472ffd700 1 --2- 172.21.15.42:0/1709046612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4740f9930 0x7fe4740f5340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fe464004e30 tx=0x7fe464009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.768+0000 7fe46bfff700 1 -- 172.21.15.42:0/1709046612 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe464018070 con 0x7fe4740f9930 2024-02-15T18:23:06.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.768+0000 7fe46bfff700 1 -- 172.21.15.42:0/1709046612 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe464007ba0 con 0x7fe4740f9930 2024-02-15T18:23:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.768+0000 7fe47a261700 1 -- 172.21.15.42:0/1709046612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4740f5c60 con 0x7fe4740f9930 2024-02-15T18:23:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.768+0000 7fe46bfff700 1 -- 172.21.15.42:0/1709046612 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe46401c3c0 con 0x7fe4740f9930 2024-02-15T18:23:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.768+0000 7fe47a261700 1 -- 172.21.15.42:0/1709046612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4740f61c0 con 0x7fe4740f9930 2024-02-15T18:23:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.769+0000 7fe47a261700 1 -- 172.21.15.42:0/1709046612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe474045ad0 con 0x7fe4740f9930 2024-02-15T18:23:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.770+0000 7fe46bfff700 1 -- 172.21.15.42:0/1709046612 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fe464007d10 con 0x7fe4740f9930 2024-02-15T18:23:06.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.770+0000 7fe46bfff700 1 --2- 172.21.15.42:0/1709046612 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe46005c2a0 0x7fe46005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.770+0000 7fe46bfff700 1 -- 172.21.15.42:0/1709046612 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fe464077750 con 0x7fe4740f9930 2024-02-15T18:23:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.770+0000 7fe4737fe700 1 --2- 172.21.15.42:0/1709046612 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe46005c2a0 0x7fe46005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:06.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.770+0000 7fe4737fe700 1 --2- 172.21.15.42:0/1709046612 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe46005c2a0 0x7fe46005e760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fe4740f6c50 tx=0x7fe45c00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:06.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.774+0000 7fe46bfff700 1 -- 172.21.15.42:0/1709046612 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe46404a7d0 con 0x7fe4740f9930 2024-02-15T18:23:06.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.933+0000 7fe47a261700 1 -- 172.21.15.42:0/1709046612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe474027a00 con 0x7fe4740f9930 2024-02-15T18:23:06.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.934+0000 7fe46bfff700 1 -- 172.21.15.42:0/1709046612 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe46404ddf0 con 0x7fe4740f9930 2024-02-15T18:23:06.934 INFO:teuthology.orchestra.run.smithi042.stdout:115964117014 2024-02-15T18:23:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.935+0000 7fe47a261700 1 -- 172.21.15.42:0/1709046612 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe46005c2a0 msgr2=0x7fe46005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.935+0000 7fe47a261700 1 --2- 172.21.15.42:0/1709046612 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe46005c2a0 0x7fe46005e760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fe4740f6c50 tx=0x7fe45c00b040).stop 2024-02-15T18:23:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.935+0000 7fe47a261700 1 -- 172.21.15.42:0/1709046612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4740f9930 msgr2=0x7fe4740f5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.935+0000 7fe47a261700 1 --2- 172.21.15.42:0/1709046612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4740f9930 0x7fe4740f5340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fe464004e30 tx=0x7fe464009b30).stop 2024-02-15T18:23:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.935+0000 7fe47a261700 1 -- 172.21.15.42:0/1709046612 shutdown_connections 2024-02-15T18:23:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.935+0000 7fe47a261700 1 --2- 172.21.15.42:0/1709046612 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe46005c2a0 0x7fe46005e760 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.935+0000 7fe47a261700 1 --2- 172.21.15.42:0/1709046612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4740607d0 0x7fe4740f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.935+0000 7fe47a261700 1 --2- 172.21.15.42:0/1709046612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4740f9930 0x7fe4740f5340 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.935+0000 7fe47a261700 1 -- 172.21.15.42:0/1709046612 >> 172.21.15.42:0/1709046612 conn(0x7fe4740f0e20 msgr2=0x7fe4740f3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.936+0000 7fe47a261700 1 -- 172.21.15.42:0/1709046612 shutdown_connections 2024-02-15T18:23:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:06.936+0000 7fe47a261700 1 -- 172.21.15.42:0/1709046612 wait complete. 2024-02-15T18:23:07.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:06.935113+0000 mon.smithi042 (mon.0) 448 : audit [DBG] from='client.? 172.21.15.42:0/1709046612' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-15T18:23:07.745 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:06.935113+0000 mon.smithi042 (mon.0) 448 : audit [DBG] from='client.? 172.21.15.42:0/1709046612' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-15T18:23:07.768 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117011 got 115964117014 for osd.3 2024-02-15T18:23:07.768 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-15T18:23:08.008 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:23:08.498 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:06.812155+0000 mgr.smithi042.kfoihb (mgr.14162) 211 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:08.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:06.812155+0000 mgr.smithi042.kfoihb (mgr.14162) 211 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:09.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.879+0000 7f3cdd29e700 1 -- 172.21.15.42:0/2316515742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cd80f6980 msgr2=0x7f3cd80f6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.879+0000 7f3cdd29e700 1 --2- 172.21.15.42:0/2316515742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cd80f6980 0x7f3cd80f6de0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f3cc0004660 tx=0x7f3cc0009b30).stop 2024-02-15T18:23:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.879+0000 7f3cdd29e700 1 -- 172.21.15.42:0/2316515742 shutdown_connections 2024-02-15T18:23:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.879+0000 7f3cdd29e700 1 --2- 172.21.15.42:0/2316515742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cd80f6980 0x7f3cd80f6de0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.879+0000 7f3cdd29e700 1 --2- 172.21.15.42:0/2316515742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd80f56e0 0x7f3cd80f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.879+0000 7f3cdd29e700 1 -- 172.21.15.42:0/2316515742 >> 172.21.15.42:0/2316515742 conn(0x7f3cd80f0e20 msgr2=0x7f3cd80f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.879+0000 7f3cdd29e700 1 -- 172.21.15.42:0/2316515742 shutdown_connections 2024-02-15T18:23:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.879+0000 7f3cdd29e700 1 -- 172.21.15.42:0/2316515742 wait complete. 2024-02-15T18:23:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.880+0000 7f3cdd29e700 1 Processor -- start 2024-02-15T18:23:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.880+0000 7f3cdd29e700 1 -- start start 2024-02-15T18:23:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.880+0000 7f3cdd29e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cd80f56e0 0x7f3cd8107190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.880+0000 7f3cdd29e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd80f6980 0x7f3cd8107680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.880+0000 7f3cdd29e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd8107c30 con 0x7f3cd80f56e0 2024-02-15T18:23:09.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.880+0000 7f3cdd29e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd8107d70 con 0x7f3cd80f6980 2024-02-15T18:23:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.880+0000 7f3cd6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cd80f56e0 0x7f3cd8107190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.880+0000 7f3cd6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cd80f56e0 0x7f3cd8107190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44220/0 (socket says 172.21.15.42:44220) 2024-02-15T18:23:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.880+0000 7f3cd6ffd700 1 -- 172.21.15.42:0/3785724942 learned_addr learned my addr 172.21.15.42:0/3785724942 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.881+0000 7f3cd67fc700 1 --2- 172.21.15.42:0/3785724942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd80f6980 0x7f3cd8107680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.881+0000 7f3cd6ffd700 1 -- 172.21.15.42:0/3785724942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd80f6980 msgr2=0x7f3cd8107680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.881+0000 7f3cd6ffd700 1 --2- 172.21.15.42:0/3785724942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd80f6980 0x7f3cd8107680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.881+0000 7f3cd6ffd700 1 -- 172.21.15.42:0/3785724942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc0005040 con 0x7f3cd80f56e0 2024-02-15T18:23:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.881+0000 7f3cd6ffd700 1 --2- 172.21.15.42:0/3785724942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cd80f56e0 0x7f3cd8107190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f3cc800c790 tx=0x7f3cc8007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.881+0000 7f3ccf7fe700 1 -- 172.21.15.42:0/3785724942 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cc800e680 con 0x7f3cd80f56e0 2024-02-15T18:23:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.881+0000 7f3ccf7fe700 1 -- 172.21.15.42:0/3785724942 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cc800ecb0 con 0x7f3cd80f56e0 2024-02-15T18:23:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.881+0000 7f3cdd29e700 1 -- 172.21.15.42:0/3785724942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd8108710 con 0x7f3cd80f56e0 2024-02-15T18:23:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.881+0000 7f3ccf7fe700 1 -- 172.21.15.42:0/3785724942 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cc8012db0 con 0x7f3cd80f56e0 2024-02-15T18:23:09.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.881+0000 7f3cdd29e700 1 -- 172.21.15.42:0/3785724942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd8108cd0 con 0x7f3cd80f56e0 2024-02-15T18:23:09.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.882+0000 7f3cdd29e700 1 -- 172.21.15.42:0/3785724942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd8045ad0 con 0x7f3cd80f56e0 2024-02-15T18:23:09.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.882+0000 7f3ccf7fe700 1 -- 172.21.15.42:0/3785724942 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f3cc8010070 con 0x7f3cd80f56e0 2024-02-15T18:23:09.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.883+0000 7f3ccf7fe700 1 --2- 172.21.15.42:0/3785724942 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3cc405c2a0 0x7f3cc405e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:09.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.883+0000 7f3ccf7fe700 1 -- 172.21.15.42:0/3785724942 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f3cc8075b60 con 0x7f3cd80f56e0 2024-02-15T18:23:09.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.883+0000 7f3cd67fc700 1 --2- 172.21.15.42:0/3785724942 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3cc405c2a0 0x7f3cc405e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:09.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.883+0000 7f3cd67fc700 1 --2- 172.21.15.42:0/3785724942 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3cc405c2a0 0x7f3cc405e760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f3cc0004e30 tx=0x7f3cc0009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:09.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:09.887+0000 7f3ccf7fe700 1 -- 172.21.15.42:0/3785724942 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f3cc8048be0 con 0x7f3cd80f56e0 2024-02-15T18:23:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:10.046+0000 7f3cdd29e700 1 -- 172.21.15.42:0/3785724942 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3cd8109360 con 0x7f3cd80f56e0 2024-02-15T18:23:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:10.046+0000 7f3ccf7fe700 1 -- 172.21.15.42:0/3785724942 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3cc804c200 con 0x7f3cd80f56e0 2024-02-15T18:23:10.046 INFO:teuthology.orchestra.run.smithi042.stdout:141733920785 2024-02-15T18:23:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:10.047+0000 7f3cdd29e700 1 -- 172.21.15.42:0/3785724942 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3cc405c2a0 msgr2=0x7f3cc405e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:10.047+0000 7f3cdd29e700 1 --2- 172.21.15.42:0/3785724942 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3cc405c2a0 0x7f3cc405e760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f3cc0004e30 tx=0x7f3cc0009b30).stop 2024-02-15T18:23:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:10.047+0000 7f3cdd29e700 1 -- 172.21.15.42:0/3785724942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cd80f56e0 msgr2=0x7f3cd8107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:10.047+0000 7f3cdd29e700 1 --2- 172.21.15.42:0/3785724942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cd80f56e0 0x7f3cd8107190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f3cc800c790 tx=0x7f3cc8007d90).stop 2024-02-15T18:23:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:10.048+0000 7f3cdd29e700 1 -- 172.21.15.42:0/3785724942 shutdown_connections 2024-02-15T18:23:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:10.048+0000 7f3cdd29e700 1 --2- 172.21.15.42:0/3785724942 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3cc405c2a0 0x7f3cc405e760 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:10.048+0000 7f3cdd29e700 1 --2- 172.21.15.42:0/3785724942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cd80f56e0 0x7f3cd8107190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:10.048+0000 7f3cdd29e700 1 --2- 172.21.15.42:0/3785724942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cd80f6980 0x7f3cd8107680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:10.048+0000 7f3cdd29e700 1 -- 172.21.15.42:0/3785724942 >> 172.21.15.42:0/3785724942 conn(0x7f3cd80f0e20 msgr2=0x7f3cd80603a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:10.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:10.048+0000 7f3cdd29e700 1 -- 172.21.15.42:0/3785724942 shutdown_connections 2024-02-15T18:23:10.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:10.048+0000 7f3cdd29e700 1 -- 172.21.15.42:0/3785724942 wait complete. 2024-02-15T18:23:10.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:08.812620+0000 mgr.smithi042.kfoihb (mgr.14162) 212 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:10.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:10.047359+0000 mon.smithi042 (mon.0) 449 : audit [DBG] from='client.? 172.21.15.42:0/3785724942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-15T18:23:10.725 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:08.812620+0000 mgr.smithi042.kfoihb (mgr.14162) 212 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:10.725 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:10.047359+0000 mon.smithi042 (mon.0) 449 : audit [DBG] from='client.? 172.21.15.42:0/3785724942' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-15T18:23:10.749 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920782 got 141733920785 for osd.4 2024-02-15T18:23:10.749 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-15T18:23:10.988 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:23:12.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:10.812974+0000 mgr.smithi042.kfoihb (mgr.14162) 213 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.671+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2206151582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140f56c0 msgr2=0x7f5f140f5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.671+0000 7f5f1ad23700 1 --2- 172.21.15.42:0/2206151582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140f56c0 0x7f5f140f5ae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f5f08004660 tx=0x7f5f08009b30).stop 2024-02-15T18:23:12.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.671+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2206151582 shutdown_connections 2024-02-15T18:23:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.671+0000 7f5f1ad23700 1 --2- 172.21.15.42:0/2206151582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f140f6960 0x7f5f140f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.671+0000 7f5f1ad23700 1 --2- 172.21.15.42:0/2206151582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140f56c0 0x7f5f140f5ae0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.671+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2206151582 >> 172.21.15.42:0/2206151582 conn(0x7f5f140f0e20 msgr2=0x7f5f140f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.671+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2206151582 shutdown_connections 2024-02-15T18:23:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.671+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2206151582 wait complete. 2024-02-15T18:23:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.672+0000 7f5f1ad23700 1 Processor -- start 2024-02-15T18:23:12.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.672+0000 7f5f1ad23700 1 -- start start 2024-02-15T18:23:12.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f1ad23700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140f56c0 0x7f5f14106060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:12.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f1ad23700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f140f6960 0x7f5f14106550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:12.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f1ad23700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f14106b00 con 0x7f5f140f56c0 2024-02-15T18:23:12.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f1ad23700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f14106c40 con 0x7f5f140f6960 2024-02-15T18:23:12.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f18abf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140f56c0 0x7f5f14106060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:12.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f18abf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140f56c0 0x7f5f14106060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38658/0 (socket says 172.21.15.42:38658) 2024-02-15T18:23:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f18abf700 1 -- 172.21.15.42:0/2994795522 learned_addr learned my addr 172.21.15.42:0/2994795522 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f13fff700 1 --2- 172.21.15.42:0/2994795522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f140f6960 0x7f5f14106550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f18abf700 1 -- 172.21.15.42:0/2994795522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f140f6960 msgr2=0x7f5f14106550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f18abf700 1 --2- 172.21.15.42:0/2994795522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f140f6960 0x7f5f14106550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f18abf700 1 -- 172.21.15.42:0/2994795522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f08005040 con 0x7f5f140f56c0 2024-02-15T18:23:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f18abf700 1 --2- 172.21.15.42:0/2994795522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140f56c0 0x7f5f14106060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f5f08002040 tx=0x7f5f0800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f117fa700 1 -- 172.21.15.42:0/2994795522 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f08019070 con 0x7f5f140f56c0 2024-02-15T18:23:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f117fa700 1 -- 172.21.15.42:0/2994795522 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f08007e90 con 0x7f5f140f56c0 2024-02-15T18:23:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.673+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2994795522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f14107580 con 0x7f5f140f56c0 2024-02-15T18:23:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.674+0000 7f5f117fa700 1 -- 172.21.15.42:0/2994795522 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f08014d90 con 0x7f5f140f56c0 2024-02-15T18:23:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.674+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2994795522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f14107ae0 con 0x7f5f140f56c0 2024-02-15T18:23:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.675+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2994795522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f14045ad0 con 0x7f5f140f56c0 2024-02-15T18:23:12.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.675+0000 7f5f117fa700 1 -- 172.21.15.42:0/2994795522 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f5f08025070 con 0x7f5f140f56c0 2024-02-15T18:23:12.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.675+0000 7f5f117fa700 1 --2- 172.21.15.42:0/2994795522 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5f0405c2a0 0x7f5f0405e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:12.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.675+0000 7f5f117fa700 1 -- 172.21.15.42:0/2994795522 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f5f08076aa0 con 0x7f5f140f56c0 2024-02-15T18:23:12.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.675+0000 7f5f13fff700 1 --2- 172.21.15.42:0/2994795522 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5f0405c2a0 0x7f5f0405e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:12.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.676+0000 7f5f13fff700 1 --2- 172.21.15.42:0/2994795522 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5f0405c2a0 0x7f5f0405e760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f5f00007f80 tx=0x7f5f0000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:12.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.679+0000 7f5f117fa700 1 -- 172.21.15.42:0/2994795522 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5f08049aa0 con 0x7f5f140f56c0 2024-02-15T18:23:12.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:10.812974+0000 mgr.smithi042.kfoihb (mgr.14162) 213 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:12.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.841+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2994795522 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5f140fe110 con 0x7f5f140f56c0 2024-02-15T18:23:12.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.841+0000 7f5f117fa700 1 -- 172.21.15.42:0/2994795522 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5f0804d0c0 con 0x7f5f140f56c0 2024-02-15T18:23:12.842 INFO:teuthology.orchestra.run.smithi042.stdout:163208757262 2024-02-15T18:23:12.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.843+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2994795522 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5f0405c2a0 msgr2=0x7f5f0405e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:12.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.843+0000 7f5f1ad23700 1 --2- 172.21.15.42:0/2994795522 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5f0405c2a0 0x7f5f0405e760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f5f00007f80 tx=0x7f5f0000b040).stop 2024-02-15T18:23:12.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.843+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2994795522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140f56c0 msgr2=0x7f5f14106060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:12.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.843+0000 7f5f1ad23700 1 --2- 172.21.15.42:0/2994795522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140f56c0 0x7f5f14106060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f5f08002040 tx=0x7f5f0800b040).stop 2024-02-15T18:23:12.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.843+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2994795522 shutdown_connections 2024-02-15T18:23:12.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.843+0000 7f5f1ad23700 1 --2- 172.21.15.42:0/2994795522 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5f0405c2a0 0x7f5f0405e760 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:12.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.843+0000 7f5f1ad23700 1 --2- 172.21.15.42:0/2994795522 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f140f56c0 0x7f5f14106060 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:12.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.843+0000 7f5f1ad23700 1 --2- 172.21.15.42:0/2994795522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f140f6960 0x7f5f14106550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:12.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.843+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2994795522 >> 172.21.15.42:0/2994795522 conn(0x7f5f140f0e20 msgr2=0x7f5f140fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:12.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.843+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2994795522 shutdown_connections 2024-02-15T18:23:12.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:12.843+0000 7f5f1ad23700 1 -- 172.21.15.42:0/2994795522 wait complete. 2024-02-15T18:23:13.497 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:12.842725+0000 mon.smithi042 (mon.0) 450 : audit [DBG] from='client.? 172.21.15.42:0/2994795522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-15T18:23:13.519 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757259 got 163208757262 for osd.5 2024-02-15T18:23:13.519 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-15T18:23:13.519 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-15T18:23:13.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:12.842725+0000 mon.smithi042 (mon.0) 450 : audit [DBG] from='client.? 172.21.15.42:0/2994795522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-15T18:23:13.755 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:23:14.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:12.813295+0000 mgr.smithi042.kfoihb (mgr.14162) 214 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:14.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:12.813295+0000 mgr.smithi042.kfoihb (mgr.14162) 214 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:15.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.428+0000 7fe4f52cd700 1 -- 172.21.15.42:0/3006772587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4f00f56e0 msgr2=0x7fe4f00f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:15.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.428+0000 7fe4f52cd700 1 --2- 172.21.15.42:0/3006772587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4f00f56e0 0x7fe4f00f5b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fe4e0004660 tx=0x7fe4e0009b30).stop 2024-02-15T18:23:15.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.428+0000 7fe4f52cd700 1 -- 172.21.15.42:0/3006772587 shutdown_connections 2024-02-15T18:23:15.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.428+0000 7fe4f52cd700 1 --2- 172.21.15.42:0/3006772587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f00f6980 0x7fe4f00f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:15.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.428+0000 7fe4f52cd700 1 --2- 172.21.15.42:0/3006772587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4f00f56e0 0x7fe4f00f5b00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:15.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.428+0000 7fe4f52cd700 1 -- 172.21.15.42:0/3006772587 >> 172.21.15.42:0/3006772587 conn(0x7fe4f00f0e20 msgr2=0x7fe4f00f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:15.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.428+0000 7fe4f52cd700 1 -- 172.21.15.42:0/3006772587 shutdown_connections 2024-02-15T18:23:15.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.428+0000 7fe4f52cd700 1 -- 172.21.15.42:0/3006772587 wait complete. 2024-02-15T18:23:15.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.429+0000 7fe4f52cd700 1 Processor -- start 2024-02-15T18:23:15.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.429+0000 7fe4f52cd700 1 -- start start 2024-02-15T18:23:15.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.429+0000 7fe4f52cd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f00f56e0 0x7fe4f0109ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:15.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.429+0000 7fe4f52cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4f00f6980 0x7fe4f010c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:15.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.429+0000 7fe4f52cd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f0060920 con 0x7fe4f00f6980 2024-02-15T18:23:15.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.429+0000 7fe4f52cd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f0060a60 con 0x7fe4f00f56e0 2024-02-15T18:23:15.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.429+0000 7fe4ee7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4f00f6980 0x7fe4f010c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:15.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.429+0000 7fe4ee7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4f00f6980 0x7fe4f010c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38672/0 (socket says 172.21.15.42:38672) 2024-02-15T18:23:15.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.430+0000 7fe4ee7fc700 1 -- 172.21.15.42:0/2743718366 learned_addr learned my addr 172.21.15.42:0/2743718366 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:15.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.430+0000 7fe4eeffd700 1 --2- 172.21.15.42:0/2743718366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f00f56e0 0x7fe4f0109ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:15.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.430+0000 7fe4ee7fc700 1 -- 172.21.15.42:0/2743718366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f00f56e0 msgr2=0x7fe4f0109ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:15.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.430+0000 7fe4ee7fc700 1 --2- 172.21.15.42:0/2743718366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f00f56e0 0x7fe4f0109ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:15.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.430+0000 7fe4ee7fc700 1 -- 172.21.15.42:0/2743718366 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e0005040 con 0x7fe4f00f6980 2024-02-15T18:23:15.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.430+0000 7fe4ee7fc700 1 --2- 172.21.15.42:0/2743718366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4f00f6980 0x7fe4f010c3e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fe4e400c790 tx=0x7fe4e4007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:15.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.431+0000 7fe4d77fe700 1 -- 172.21.15.42:0/2743718366 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4e4002590 con 0x7fe4f00f6980 2024-02-15T18:23:15.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.431+0000 7fe4d77fe700 1 -- 172.21.15.42:0/2743718366 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4e4002bc0 con 0x7fe4f00f6980 2024-02-15T18:23:15.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.431+0000 7fe4f52cd700 1 -- 172.21.15.42:0/2743718366 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f010c990 con 0x7fe4f00f6980 2024-02-15T18:23:15.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.431+0000 7fe4d77fe700 1 -- 172.21.15.42:0/2743718366 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4e4013e40 con 0x7fe4f00f6980 2024-02-15T18:23:15.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.431+0000 7fe4f52cd700 1 -- 172.21.15.42:0/2743718366 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f010cf20 con 0x7fe4f00f6980 2024-02-15T18:23:15.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.432+0000 7fe4f52cd700 1 -- 172.21.15.42:0/2743718366 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f0059100 con 0x7fe4f00f6980 2024-02-15T18:23:15.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.433+0000 7fe4d77fe700 1 -- 172.21.15.42:0/2743718366 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fe4e401a060 con 0x7fe4f00f6980 2024-02-15T18:23:15.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.433+0000 7fe4d77fe700 1 --2- 172.21.15.42:0/2743718366 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe4d805c2f0 0x7fe4d805e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:15.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.433+0000 7fe4d77fe700 1 -- 172.21.15.42:0/2743718366 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fe4e4075db0 con 0x7fe4f00f6980 2024-02-15T18:23:15.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.434+0000 7fe4eeffd700 1 --2- 172.21.15.42:0/2743718366 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe4d805c2f0 0x7fe4d805e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:15.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.434+0000 7fe4eeffd700 1 --2- 172.21.15.42:0/2743718366 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe4d805c2f0 0x7fe4d805e7b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fe4f00605a0 tx=0x7fe4e0009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:15.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.440+0000 7fe4d77fe700 1 -- 172.21.15.42:0/2743718366 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe4e404c860 con 0x7fe4f00f6980 2024-02-15T18:23:15.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.594+0000 7fe4f52cd700 1 -- 172.21.15.42:0/2743718366 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4f00fe080 con 0x7fe4d805c2f0 2024-02-15T18:23:15.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.596+0000 7fe4d77fe700 1 -- 172.21.15.42:0/2743718366 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27683 (secure 0 0 0) 0x7fe4f00fe080 con 0x7fe4d805c2f0 2024-02-15T18:23:15.596 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:23:15.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.597+0000 7fe4f52cd700 1 -- 172.21.15.42:0/2743718366 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe4d805c2f0 msgr2=0x7fe4d805e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:15.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.597+0000 7fe4f52cd700 1 --2- 172.21.15.42:0/2743718366 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe4d805c2f0 0x7fe4d805e7b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fe4f00605a0 tx=0x7fe4e0009b30).stop 2024-02-15T18:23:15.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.597+0000 7fe4f52cd700 1 -- 172.21.15.42:0/2743718366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4f00f6980 msgr2=0x7fe4f010c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:15.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.597+0000 7fe4f52cd700 1 --2- 172.21.15.42:0/2743718366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4f00f6980 0x7fe4f010c3e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fe4e400c790 tx=0x7fe4e4007d70).stop 2024-02-15T18:23:15.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.597+0000 7fe4f52cd700 1 -- 172.21.15.42:0/2743718366 shutdown_connections 2024-02-15T18:23:15.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.597+0000 7fe4f52cd700 1 --2- 172.21.15.42:0/2743718366 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe4d805c2f0 0x7fe4d805e7b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:15.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.597+0000 7fe4f52cd700 1 --2- 172.21.15.42:0/2743718366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4f00f56e0 0x7fe4f0109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:15.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.597+0000 7fe4f52cd700 1 --2- 172.21.15.42:0/2743718366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4f00f6980 0x7fe4f010c3e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:15.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.597+0000 7fe4f52cd700 1 -- 172.21.15.42:0/2743718366 >> 172.21.15.42:0/2743718366 conn(0x7fe4f00f0e20 msgr2=0x7fe4f00fa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:15.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.598+0000 7fe4f52cd700 1 -- 172.21.15.42:0/2743718366 shutdown_connections 2024-02-15T18:23:15.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:15.598+0000 7fe4f52cd700 1 -- 172.21.15.42:0/2743718366 wait complete. 2024-02-15T18:23:15.599 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-02-15T18:23:16.243 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-02-15T18:23:14.813520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":31208,"kb_used_data":2040,"kb_used_omap":0,"kb_used_meta":29120,"kb_avail":562439704,"statfs":{"total":575970213888,"available":575938256896,"internally_reserved":0,"allocated":2088960,"data_stored":786132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29818880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":"35","reported_epoch":"38","state":"active+clean","last_fresh":"2024-02-15T18:22:17.324755+0000","last_change":"2024-02-15T18:21:54.440698+0000","last_active":"2024-02-15T18:22:17.324755+0000","last_peered":"2024-02-15T18:22:17.324755+0000","last_clean":"2024-02-15T18:22:17.324755+0000","last_became_active":"2024-02-15T18:21:54.440346+0000","last_became_peered":"2024-02-15T18:21:54.440346+0000","last_unstale":"2024-02-15T18:22:17.324755+0000","last_undegraded":"2024-02-15T18:22:17.324755+0000","last_fullsized":"2024-02-15T18:22:17.324755+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-15T18:21:27.190224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-15T18:21:27.190224+0000","last_clean_scrub_stamp":"2024-02-15T18:21:27.190224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":38,"seq":163208757263,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5084,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":93740068,"statfs":{"total":95995035648,"available":95989829632,"internally_reserved":0,"allocated":348160,"data_stored":131022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920787,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5084,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":93740068,"statfs":{"total":95995035648,"available":95989829632,"internally_reserved":0,"allocated":348160,"data_stored":131022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:22:56 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.76300000000000001}]},{"osd":1,"last update":"Thu Feb 15 18:22:56 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Feb 15 18:22:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.622}]},{"osd":3,"last update":"Thu Feb 15 18:22:56 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.502},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.49399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5212,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":93739940,"statfs":{"total":95995035648,"available":95989698560,"internally_reserved":0,"allocated":348160,"data_stored":131022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Feb 15 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.5},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Feb 15 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.53800000000000003}]},{"osd":4,"last update":"Thu Feb 15 18:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5212,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":93739940,"statfs":{"total":95995035648,"available":95989698560,"internally_reserved":0,"allocated":348160,"data_stored":131022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.27700000000000002}]},{"osd":1,"last update":"Thu Feb 15 18:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.23000000000000001}]},{"osd":3,"last update":"Thu Feb 15 18:22:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Feb 15 18:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705701,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5340,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739812,"statfs":{"total":95995035648,"available":95989567488,"internally_reserved":0,"allocated":348160,"data_stored":131022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 15 18:22:54 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.35999999999999999,"15min":0.35799999999999998},"min":{"1min":0.22800000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.35299999999999998,"15min":0.34599999999999997},"min":{"1min":0.255,"5min":0.22,"15min":0.22},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Feb 15 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.28599999999999998}]},{"osd":3,"last update":"Thu Feb 15 18:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.628}]},{"osd":4,"last update":"Thu Feb 15 18:22:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5276,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":93739876,"statfs":{"total":95995035648,"available":95989633024,"internally_reserved":0,"allocated":348160,"data_stored":131022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33200000000000002,"15min":0.33100000000000002},"min":{"1min":0.23000000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.33900000000000002},"min":{"1min":0.255,"5min":0.218,"15min":0.218},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.29999999999999999}]},{"osd":2,"last update":"Thu Feb 15 18:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.27700000000000002}]},{"osd":3,"last update":"Thu Feb 15 18:22:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Feb 15 18:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-15T18:23:16.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-15T18:23:16.483 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:23:16.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:14.813683+0000 mgr.smithi042.kfoihb (mgr.14162) 215 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:16.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:14.813683+0000 mgr.smithi042.kfoihb (mgr.14162) 215 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:17.652 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:15.596007+0000 mgr.smithi042.kfoihb (mgr.14162) 216 : audit [DBG] from='client.14396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-15T18:23:17.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:15.596007+0000 mgr.smithi042.kfoihb (mgr.14162) 216 : audit [DBG] from='client.14396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-15T18:23:18.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.158+0000 7fc236999700 1 -- 172.21.15.42:0/1906192408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc230060870 msgr2=0x7fc2300f93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:18.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.158+0000 7fc236999700 1 --2- 172.21.15.42:0/1906192408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc230060870 0x7fc2300f93a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fc220004660 tx=0x7fc220009b30).stop 2024-02-15T18:23:18.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.158+0000 7fc236999700 1 -- 172.21.15.42:0/1906192408 shutdown_connections 2024-02-15T18:23:18.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.158+0000 7fc236999700 1 --2- 172.21.15.42:0/1906192408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2300f9920 0x7fc2300fbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:18.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.158+0000 7fc236999700 1 --2- 172.21.15.42:0/1906192408 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc230060870 0x7fc2300f93a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:18.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.158+0000 7fc236999700 1 -- 172.21.15.42:0/1906192408 >> 172.21.15.42:0/1906192408 conn(0x7fc2300f0e20 msgr2=0x7fc2300f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:18.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.158+0000 7fc236999700 1 -- 172.21.15.42:0/1906192408 shutdown_connections 2024-02-15T18:23:18.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.158+0000 7fc236999700 1 -- 172.21.15.42:0/1906192408 wait complete. 2024-02-15T18:23:18.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.158+0000 7fc236999700 1 Processor -- start 2024-02-15T18:23:18.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.158+0000 7fc236999700 1 -- start start 2024-02-15T18:23:18.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc236999700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc230060870 0x7fc2301092d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:18.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc236999700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2300f9920 0x7fc2301097c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:18.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc236999700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc230109d70 con 0x7fc230060870 2024-02-15T18:23:18.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc236999700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc230109eb0 con 0x7fc2300f9920 2024-02-15T18:23:18.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc22ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc230060870 0x7fc2301092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:18.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc22ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc230060870 0x7fc2301092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38690/0 (socket says 172.21.15.42:38690) 2024-02-15T18:23:18.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc22ffff700 1 -- 172.21.15.42:0/121360780 learned_addr learned my addr 172.21.15.42:0/121360780 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:18.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc22f7fe700 1 --2- 172.21.15.42:0/121360780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2300f9920 0x7fc2301097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:18.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc22ffff700 1 -- 172.21.15.42:0/121360780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2300f9920 msgr2=0x7fc2301097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:18.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc22ffff700 1 --2- 172.21.15.42:0/121360780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2300f9920 0x7fc2301097c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:18.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc22ffff700 1 -- 172.21.15.42:0/121360780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc220005040 con 0x7fc230060870 2024-02-15T18:23:18.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc22ffff700 1 --2- 172.21.15.42:0/121360780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc230060870 0x7fc2301092d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fc220009830 tx=0x7fc220009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:18.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc22cff9700 1 -- 172.21.15.42:0/121360780 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc220018070 con 0x7fc230060870 2024-02-15T18:23:18.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc22cff9700 1 -- 172.21.15.42:0/121360780 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc220007ba0 con 0x7fc230060870 2024-02-15T18:23:18.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc236999700 1 -- 172.21.15.42:0/121360780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23010a7f0 con 0x7fc230060870 2024-02-15T18:23:18.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc22cff9700 1 -- 172.21.15.42:0/121360780 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc22001c3c0 con 0x7fc230060870 2024-02-15T18:23:18.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.159+0000 7fc236999700 1 -- 172.21.15.42:0/121360780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23010ad50 con 0x7fc230060870 2024-02-15T18:23:18.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.160+0000 7fc236999700 1 -- 172.21.15.42:0/121360780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc230045ad0 con 0x7fc230060870 2024-02-15T18:23:18.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.161+0000 7fc22cff9700 1 -- 172.21.15.42:0/121360780 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fc220007d10 con 0x7fc230060870 2024-02-15T18:23:18.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.161+0000 7fc22cff9700 1 --2- 172.21.15.42:0/121360780 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc21805c2a0 0x7fc21805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:18.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.161+0000 7fc22cff9700 1 -- 172.21.15.42:0/121360780 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fc2200777c0 con 0x7fc230060870 2024-02-15T18:23:18.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.161+0000 7fc22f7fe700 1 --2- 172.21.15.42:0/121360780 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc21805c2a0 0x7fc21805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:18.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.162+0000 7fc22f7fe700 1 --2- 172.21.15.42:0/121360780 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc21805c2a0 0x7fc21805e760 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fc230060640 tx=0x7fc22400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:18.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.165+0000 7fc22cff9700 1 -- 172.21.15.42:0/121360780 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc22004a760 con 0x7fc230060870 2024-02-15T18:23:18.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.320+0000 7fc236999700 1 -- 172.21.15.42:0/121360780 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc2300fa7f0 con 0x7fc21805c2a0 2024-02-15T18:23:18.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.321+0000 7fc22cff9700 1 -- 172.21.15.42:0/121360780 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27669 (secure 0 0 0) 0x7fc2300fa7f0 con 0x7fc21805c2a0 2024-02-15T18:23:18.322 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:23:18.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.323+0000 7fc236999700 1 -- 172.21.15.42:0/121360780 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc21805c2a0 msgr2=0x7fc21805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:18.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.323+0000 7fc236999700 1 --2- 172.21.15.42:0/121360780 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc21805c2a0 0x7fc21805e760 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fc230060640 tx=0x7fc22400b040).stop 2024-02-15T18:23:18.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.323+0000 7fc236999700 1 -- 172.21.15.42:0/121360780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc230060870 msgr2=0x7fc2301092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:18.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.323+0000 7fc236999700 1 --2- 172.21.15.42:0/121360780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc230060870 0x7fc2301092d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fc220009830 tx=0x7fc220009b30).stop 2024-02-15T18:23:18.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.323+0000 7fc236999700 1 -- 172.21.15.42:0/121360780 shutdown_connections 2024-02-15T18:23:18.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.323+0000 7fc236999700 1 --2- 172.21.15.42:0/121360780 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc21805c2a0 0x7fc21805e760 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:18.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.323+0000 7fc236999700 1 --2- 172.21.15.42:0/121360780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc230060870 0x7fc2301092d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:18.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.323+0000 7fc236999700 1 --2- 172.21.15.42:0/121360780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2300f9920 0x7fc2301097c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:18.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.323+0000 7fc236999700 1 -- 172.21.15.42:0/121360780 >> 172.21.15.42:0/121360780 conn(0x7fc2300f0e20 msgr2=0x7fc2300fa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:18.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.323+0000 7fc236999700 1 -- 172.21.15.42:0/121360780 shutdown_connections 2024-02-15T18:23:18.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:18.323+0000 7fc236999700 1 -- 172.21.15.42:0/121360780 wait complete. 2024-02-15T18:23:18.324 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-02-15T18:23:18.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:16.813995+0000 mgr.smithi042.kfoihb (mgr.14162) 217 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:18.729 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:16.813995+0000 mgr.smithi042.kfoihb (mgr.14162) 217 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:19.076 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-02-15T18:23:16.813846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":31208,"kb_used_data":2040,"kb_used_omap":0,"kb_used_meta":29120,"kb_avail":562439704,"statfs":{"total":575970213888,"available":575938256896,"internally_reserved":0,"allocated":2088960,"data_stored":786132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29818880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":"35","reported_epoch":"38","state":"active+clean","last_fresh":"2024-02-15T18:22:17.324755+0000","last_change":"2024-02-15T18:21:54.440698+0000","last_active":"2024-02-15T18:22:17.324755+0000","last_peered":"2024-02-15T18:22:17.324755+0000","last_clean":"2024-02-15T18:22:17.324755+0000","last_became_active":"2024-02-15T18:21:54.440346+0000","last_became_peered":"2024-02-15T18:21:54.440346+0000","last_unstale":"2024-02-15T18:22:17.324755+0000","last_undegraded":"2024-02-15T18:22:17.324755+0000","last_fullsized":"2024-02-15T18:22:17.324755+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-15T18:21:27.190224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-15T18:21:27.190224+0000","last_clean_scrub_stamp":"2024-02-15T18:21:27.190224+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":38,"seq":163208757263,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5084,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":93740068,"statfs":{"total":95995035648,"available":95989829632,"internally_reserved":0,"allocated":348160,"data_stored":131022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920787,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5084,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":93740068,"statfs":{"total":95995035648,"available":95989829632,"internally_reserved":0,"allocated":348160,"data_stored":131022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:22:56 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.76300000000000001}]},{"osd":1,"last update":"Thu Feb 15 18:22:56 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Feb 15 18:22:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.622}]},{"osd":3,"last update":"Thu Feb 15 18:22:56 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.502},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.49399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5212,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":93739940,"statfs":{"total":95995035648,"available":95989698560,"internally_reserved":0,"allocated":348160,"data_stored":131022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51200000000000001}]},{"osd":1,"last update":"Thu Feb 15 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.5},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Feb 15 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.53800000000000003}]},{"osd":4,"last update":"Thu Feb 15 18:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5212,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":93739940,"statfs":{"total":95995035648,"available":95989698560,"internally_reserved":0,"allocated":348160,"data_stored":131022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.27700000000000002}]},{"osd":1,"last update":"Thu Feb 15 18:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.23000000000000001}]},{"osd":3,"last update":"Thu Feb 15 18:22:32 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Feb 15 18:22:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5340,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739812,"statfs":{"total":95995035648,"available":95989567488,"internally_reserved":0,"allocated":348160,"data_stored":131022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Feb 15 18:22:54 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.35999999999999999,"15min":0.35799999999999998},"min":{"1min":0.22800000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.35299999999999998,"15min":0.34599999999999997},"min":{"1min":0.255,"5min":0.22,"15min":0.22},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.436}]},{"osd":2,"last update":"Thu Feb 15 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Feb 15 18:22:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.55800000000000005}]},{"osd":4,"last update":"Thu Feb 15 18:22:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542175,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5276,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":93739876,"statfs":{"total":95995035648,"available":95989633024,"internally_reserved":0,"allocated":348160,"data_stored":131022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Feb 15 18:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33200000000000002,"15min":0.33100000000000002},"min":{"1min":0.23000000000000001,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.33900000000000002},"min":{"1min":0.255,"5min":0.218,"15min":0.218},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.32400000000000001}]},{"osd":2,"last update":"Thu Feb 15 18:22:17 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.372},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.36199999999999999}]},{"osd":3,"last update":"Thu Feb 15 18:22:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Feb 15 18:22:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.54100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-15T18:23:19.077 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-15T18:23:19.077 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-15T18:23:19.077 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-15T18:23:19.077 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- ceph health --format=json 2024-02-15T18:23:19.319 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/mon.smithi042/config 2024-02-15T18:23:19.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:18.321267+0000 mgr.smithi042.kfoihb (mgr.14162) 218 : audit [DBG] from='client.14400 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-15T18:23:19.804 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:18.321267+0000 mgr.smithi042.kfoihb (mgr.14162) 218 : audit [DBG] from='client.14400 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-15T18:23:20.518 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:18.814520+0000 mgr.smithi042.kfoihb (mgr.14162) 219 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:20.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:18.814520+0000 mgr.smithi042.kfoihb (mgr.14162) 219 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:21.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.021+0000 7f3b8b874700 1 -- 172.21.15.42:0/2192969967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b840f67d0 msgr2=0x7f3b840f6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:21.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.021+0000 7f3b8b874700 1 --2- 172.21.15.42:0/2192969967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b840f67d0 0x7f3b840f6bf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f3b80004660 tx=0x7f3b80009b30).stop 2024-02-15T18:23:21.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.021+0000 7f3b8b874700 1 -- 172.21.15.42:0/2192969967 shutdown_connections 2024-02-15T18:23:21.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.021+0000 7f3b8b874700 1 --2- 172.21.15.42:0/2192969967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b840f7a70 0x7f3b840f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:21.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.021+0000 7f3b8b874700 1 --2- 172.21.15.42:0/2192969967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b840f67d0 0x7f3b840f6bf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:21.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.021+0000 7f3b8b874700 1 -- 172.21.15.42:0/2192969967 >> 172.21.15.42:0/2192969967 conn(0x7f3b840f1f50 msgr2=0x7f3b840f4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:21.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.021+0000 7f3b8b874700 1 -- 172.21.15.42:0/2192969967 shutdown_connections 2024-02-15T18:23:21.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.021+0000 7f3b8b874700 1 -- 172.21.15.42:0/2192969967 wait complete. 2024-02-15T18:23:21.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.022+0000 7f3b8b874700 1 Processor -- start 2024-02-15T18:23:21.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.022+0000 7f3b8b874700 1 -- start start 2024-02-15T18:23:21.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.022+0000 7f3b8b874700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b840f67d0 0x7f3b841093b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:21.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.022+0000 7f3b8b874700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b840f7a70 0x7f3b841098a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:21.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.022+0000 7f3b8b874700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b84109e50 con 0x7f3b840f7a70 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.022+0000 7f3b8b874700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b84109f90 con 0x7f3b840f67d0 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.022+0000 7f3b88e0f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b840f7a70 0x7f3b841098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.022+0000 7f3b88e0f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b840f7a70 0x7f3b841098a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38700/0 (socket says 172.21.15.42:38700) 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.022+0000 7f3b88e0f700 1 -- 172.21.15.42:0/2506989248 learned_addr learned my addr 172.21.15.42:0/2506989248 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.022+0000 7f3b89610700 1 --2- 172.21.15.42:0/2506989248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b840f67d0 0x7f3b841093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.022+0000 7f3b88e0f700 1 -- 172.21.15.42:0/2506989248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b840f67d0 msgr2=0x7f3b841093b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.022+0000 7f3b88e0f700 1 --2- 172.21.15.42:0/2506989248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b840f67d0 0x7f3b841093b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.023+0000 7f3b88e0f700 1 -- 172.21.15.42:0/2506989248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b80005040 con 0x7f3b840f7a70 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.023+0000 7f3b89610700 1 --2- 172.21.15.42:0/2506989248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b840f67d0 0x7f3b841093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.023+0000 7f3b88e0f700 1 --2- 172.21.15.42:0/2506989248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b840f7a70 0x7f3b841098a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f3b7800c530 tx=0x7f3b78007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.023+0000 7f3b75ffb700 1 -- 172.21.15.42:0/2506989248 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b7800f650 con 0x7f3b840f7a70 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.023+0000 7f3b75ffb700 1 -- 172.21.15.42:0/2506989248 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b7800fc80 con 0x7f3b840f7a70 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.023+0000 7f3b8b874700 1 -- 172.21.15.42:0/2506989248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8410a930 con 0x7f3b840f7a70 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.023+0000 7f3b75ffb700 1 -- 172.21.15.42:0/2506989248 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b78012d80 con 0x7f3b840f7a70 2024-02-15T18:23:21.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.023+0000 7f3b8b874700 1 -- 172.21.15.42:0/2506989248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8410aef0 con 0x7f3b840f7a70 2024-02-15T18:23:21.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.024+0000 7f3b8b874700 1 -- 172.21.15.42:0/2506989248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b84045ad0 con 0x7f3b840f7a70 2024-02-15T18:23:21.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.024+0000 7f3b75ffb700 1 -- 172.21.15.42:0/2506989248 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f3b78010070 con 0x7f3b840f7a70 2024-02-15T18:23:21.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.025+0000 7f3b75ffb700 1 --2- 172.21.15.42:0/2506989248 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3b7005c2a0 0x7f3b7005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:21.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.025+0000 7f3b75ffb700 1 -- 172.21.15.42:0/2506989248 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f3b780758b0 con 0x7f3b840f7a70 2024-02-15T18:23:21.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.025+0000 7f3b89610700 1 --2- 172.21.15.42:0/2506989248 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3b7005c2a0 0x7f3b7005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:21.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.025+0000 7f3b89610700 1 --2- 172.21.15.42:0/2506989248 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3b7005c2a0 0x7f3b7005e760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f3b80004e30 tx=0x7f3b80009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:21.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.029+0000 7f3b75ffb700 1 -- 172.21.15.42:0/2506989248 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f3b780452b0 con 0x7f3b840f7a70 2024-02-15T18:23:21.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.247+0000 7f3b8b874700 1 -- 172.21.15.42:0/2506989248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3b84027a00 con 0x7f3b840f7a70 2024-02-15T18:23:21.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.247+0000 7f3b75ffb700 1 -- 172.21.15.42:0/2506989248 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f3b780488b0 con 0x7f3b840f7a70 2024-02-15T18:23:21.248 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.249+0000 7f3b8b874700 1 -- 172.21.15.42:0/2506989248 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3b7005c2a0 msgr2=0x7f3b7005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.249+0000 7f3b8b874700 1 --2- 172.21.15.42:0/2506989248 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3b7005c2a0 0x7f3b7005e760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f3b80004e30 tx=0x7f3b80009b30).stop 2024-02-15T18:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.249+0000 7f3b8b874700 1 -- 172.21.15.42:0/2506989248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b840f7a70 msgr2=0x7f3b841098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.249+0000 7f3b8b874700 1 --2- 172.21.15.42:0/2506989248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b840f7a70 0x7f3b841098a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f3b7800c530 tx=0x7f3b78007e20).stop 2024-02-15T18:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.249+0000 7f3b8b874700 1 -- 172.21.15.42:0/2506989248 shutdown_connections 2024-02-15T18:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.249+0000 7f3b8b874700 1 --2- 172.21.15.42:0/2506989248 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f3b7005c2a0 0x7f3b7005e760 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:21.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.249+0000 7f3b8b874700 1 --2- 172.21.15.42:0/2506989248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b840f67d0 0x7f3b841093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:21.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.249+0000 7f3b8b874700 1 --2- 172.21.15.42:0/2506989248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b840f7a70 0x7f3b841098a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:21.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.249+0000 7f3b8b874700 1 -- 172.21.15.42:0/2506989248 >> 172.21.15.42:0/2506989248 conn(0x7f3b840f1f50 msgr2=0x7f3b8406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:21.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.249+0000 7f3b8b874700 1 -- 172.21.15.42:0/2506989248 shutdown_connections 2024-02-15T18:23:21.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:21.249+0000 7f3b8b874700 1 -- 172.21.15.42:0/2506989248 wait complete. 2024-02-15T18:23:21.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:21.248595+0000 mon.smithi042 (mon.0) 451 : audit [DBG] from='client.? 172.21.15.42:0/2506989248' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-15T18:23:21.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:21.248595+0000 mon.smithi042 (mon.0) 451 : audit [DBG] from='client.? 172.21.15.42:0/2506989248' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-15T18:23:22.216 INFO:teuthology.orchestra.run.smithi042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-15T18:23:22.216 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-15T18:23:22.217 INFO:tasks.cephadm:Setup complete, yielding 2024-02-15T18:23:22.217 INFO:teuthology.run_tasks:Running task print... 2024-02-15T18:23:22.226 INFO:teuthology.task.print:**** done starting v16.2.4 2024-02-15T18:23:22.226 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-15T18:23:22.263 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-02-15T18:23:22.264 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- bash -c 'ceph orch status' 2024-02-15T18:23:22.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:20.814863+0000 mgr.smithi042.kfoihb (mgr.14162) 220 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:22.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:20.814863+0000 mgr.smithi042.kfoihb (mgr.14162) 220 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:24.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.254+0000 7f64f9f9d700 1 -- 172.21.15.42:0/4247458565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64f40f96a0 msgr2=0x7f64f40fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:24.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.254+0000 7f64f9f9d700 1 --2- 172.21.15.42:0/4247458565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64f40f96a0 0x7f64f40fbaf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f64e0004660 tx=0x7f64e0009b30).stop 2024-02-15T18:23:24.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.254+0000 7f64f9f9d700 1 -- 172.21.15.42:0/4247458565 shutdown_connections 2024-02-15T18:23:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.254+0000 7f64f9f9d700 1 --2- 172.21.15.42:0/4247458565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64f40f96a0 0x7f64f40fbaf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.254+0000 7f64f9f9d700 1 --2- 172.21.15.42:0/4247458565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64f4060310 0x7f64f40f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.254+0000 7f64f9f9d700 1 -- 172.21.15.42:0/4247458565 >> 172.21.15.42:0/4247458565 conn(0x7f64f40f0ba0 msgr2=0x7f64f40f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.254+0000 7f64f9f9d700 1 -- 172.21.15.42:0/4247458565 shutdown_connections 2024-02-15T18:23:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.254+0000 7f64f9f9d700 1 -- 172.21.15.42:0/4247458565 wait complete. 2024-02-15T18:23:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.254+0000 7f64f9f9d700 1 Processor -- start 2024-02-15T18:23:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.254+0000 7f64f9f9d700 1 -- start start 2024-02-15T18:23:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.255+0000 7f64f9f9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64f4060310 0x7f64f4106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.255+0000 7f64f9f9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64f41074e0 0x7f64f41083d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.255+0000 7f64f9f9d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f41079b0 con 0x7f64f4060310 2024-02-15T18:23:24.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.255+0000 7f64f9f9d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f4107af0 con 0x7f64f41074e0 2024-02-15T18:23:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.255+0000 7f64f8f9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64f4060310 0x7f64f4106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.255+0000 7f64f8f9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64f4060310 0x7f64f4106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41234/0 (socket says 172.21.15.42:41234) 2024-02-15T18:23:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.255+0000 7f64f8f9b700 1 -- 172.21.15.42:0/2636339459 learned_addr learned my addr 172.21.15.42:0/2636339459 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.255+0000 7f64f3fff700 1 --2- 172.21.15.42:0/2636339459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64f41074e0 0x7f64f41083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.255+0000 7f64f8f9b700 1 -- 172.21.15.42:0/2636339459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64f41074e0 msgr2=0x7f64f41083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.255+0000 7f64f8f9b700 1 --2- 172.21.15.42:0/2636339459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64f41074e0 0x7f64f41083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.255+0000 7f64f8f9b700 1 -- 172.21.15.42:0/2636339459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e0005040 con 0x7f64f4060310 2024-02-15T18:23:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.255+0000 7f64f8f9b700 1 --2- 172.21.15.42:0/2636339459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64f4060310 0x7f64f4106ff0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f64e80026d0 tx=0x7f64e8009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.256+0000 7f64f17fa700 1 -- 172.21.15.42:0/2636339459 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e80079c0 con 0x7f64f4060310 2024-02-15T18:23:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.256+0000 7f64f9f9d700 1 -- 172.21.15.42:0/2636339459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f4108980 con 0x7f64f4060310 2024-02-15T18:23:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.256+0000 7f64f17fa700 1 -- 172.21.15.42:0/2636339459 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64e800f040 con 0x7f64f4060310 2024-02-15T18:23:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.256+0000 7f64f17fa700 1 -- 172.21.15.42:0/2636339459 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64e8014980 con 0x7f64f4060310 2024-02-15T18:23:24.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.256+0000 7f64f9f9d700 1 -- 172.21.15.42:0/2636339459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f4108f40 con 0x7f64f4060310 2024-02-15T18:23:24.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.257+0000 7f64f9f9d700 1 -- 172.21.15.42:0/2636339459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f41091c0 con 0x7f64f4060310 2024-02-15T18:23:24.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.257+0000 7f64f17fa700 1 -- 172.21.15.42:0/2636339459 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f64e80090d0 con 0x7f64f4060310 2024-02-15T18:23:24.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.257+0000 7f64f17fa700 1 --2- 172.21.15.42:0/2636339459 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f64e405c300 0x7f64e405e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:24.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.257+0000 7f64f17fa700 1 -- 172.21.15.42:0/2636339459 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f64e8075b90 con 0x7f64f4060310 2024-02-15T18:23:24.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.257+0000 7f64f3fff700 1 --2- 172.21.15.42:0/2636339459 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f64e405c300 0x7f64e405e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:24.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.258+0000 7f64f3fff700 1 --2- 172.21.15.42:0/2636339459 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f64e405c300 0x7f64e405e7c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f64e0000c00 tx=0x7f64e0009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:24.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.261+0000 7f64f17fa700 1 -- 172.21.15.42:0/2636339459 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f64e8048b90 con 0x7f64f4060310 2024-02-15T18:23:24.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.453+0000 7f64f9f9d700 1 -- 172.21.15.42:0/2636339459 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64f40fee10 con 0x7f64e405c300 2024-02-15T18:23:24.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.454+0000 7f64f17fa700 1 -- 172.21.15.42:0/2636339459 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f64f40fee10 con 0x7f64e405c300 2024-02-15T18:23:24.454 INFO:teuthology.orchestra.run.smithi042.stdout:Backend: cephadm 2024-02-15T18:23:24.454 INFO:teuthology.orchestra.run.smithi042.stdout:Available: Yes 2024-02-15T18:23:24.454 INFO:teuthology.orchestra.run.smithi042.stdout:Paused: No 2024-02-15T18:23:24.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.455+0000 7f64f9f9d700 1 -- 172.21.15.42:0/2636339459 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f64e405c300 msgr2=0x7f64e405e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:24.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.455+0000 7f64f9f9d700 1 --2- 172.21.15.42:0/2636339459 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f64e405c300 0x7f64e405e7c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f64e0000c00 tx=0x7f64e0009b30).stop 2024-02-15T18:23:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.455+0000 7f64f9f9d700 1 -- 172.21.15.42:0/2636339459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64f4060310 msgr2=0x7f64f4106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.455+0000 7f64f9f9d700 1 --2- 172.21.15.42:0/2636339459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64f4060310 0x7f64f4106ff0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f64e80026d0 tx=0x7f64e8009de0).stop 2024-02-15T18:23:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.455+0000 7f64f9f9d700 1 -- 172.21.15.42:0/2636339459 shutdown_connections 2024-02-15T18:23:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.455+0000 7f64f9f9d700 1 --2- 172.21.15.42:0/2636339459 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f64e405c300 0x7f64e405e7c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.455+0000 7f64f9f9d700 1 --2- 172.21.15.42:0/2636339459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64f4060310 0x7f64f4106ff0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.455+0000 7f64f9f9d700 1 --2- 172.21.15.42:0/2636339459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64f41074e0 0x7f64f41083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.455+0000 7f64f9f9d700 1 -- 172.21.15.42:0/2636339459 >> 172.21.15.42:0/2636339459 conn(0x7f64f40f0ba0 msgr2=0x7f64f40f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.455+0000 7f64f9f9d700 1 -- 172.21.15.42:0/2636339459 shutdown_connections 2024-02-15T18:23:24.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:24.455+0000 7f64f9f9d700 1 -- 172.21.15.42:0/2636339459 wait complete. 2024-02-15T18:23:24.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:22.815324+0000 mgr.smithi042.kfoihb (mgr.14162) 221 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:24.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:22.815324+0000 mgr.smithi042.kfoihb (mgr.14162) 221 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:25.268 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-15T18:23:25.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:24.454746+0000 mgr.smithi042.kfoihb (mgr.14162) 222 : audit [DBG] from='client.14408 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:23:25.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:24.731085+0000 mon.smithi042 (mon.0) 452 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:23:25.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:24.454746+0000 mgr.smithi042.kfoihb (mgr.14162) 222 : audit [DBG] from='client.14408 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:23:25.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:24.731085+0000 mon.smithi042 (mon.0) 452 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:23:26.698 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:24.815653+0000 mgr.smithi042.kfoihb (mgr.14162) 223 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:26.698 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:25.425131+0000 mon.smithi042 (mon.0) 453 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:26.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:24.815653+0000 mgr.smithi042.kfoihb (mgr.14162) 223 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:26.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:25.425131+0000 mon.smithi042 (mon.0) 453 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:27.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.235+0000 7f7b0e17d700 1 -- 172.21.15.42:0/1642045661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b080f55c0 msgr2=0x7f7b080f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.235+0000 7f7b0e17d700 1 --2- 172.21.15.42:0/1642045661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b080f55c0 0x7f7b080f59e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f7af8004660 tx=0x7f7af8009b30).stop 2024-02-15T18:23:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.235+0000 7f7b0e17d700 1 -- 172.21.15.42:0/1642045661 shutdown_connections 2024-02-15T18:23:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.235+0000 7f7b0e17d700 1 --2- 172.21.15.42:0/1642045661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b080f6860 0x7f7b080f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.235+0000 7f7b0e17d700 1 --2- 172.21.15.42:0/1642045661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b080f55c0 0x7f7b080f59e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.235+0000 7f7b0e17d700 1 -- 172.21.15.42:0/1642045661 >> 172.21.15.42:0/1642045661 conn(0x7f7b080f0d40 msgr2=0x7f7b080f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.235+0000 7f7b0e17d700 1 -- 172.21.15.42:0/1642045661 shutdown_connections 2024-02-15T18:23:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.235+0000 7f7b0e17d700 1 -- 172.21.15.42:0/1642045661 wait complete. 2024-02-15T18:23:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.236+0000 7f7b0e17d700 1 Processor -- start 2024-02-15T18:23:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.236+0000 7f7b0e17d700 1 -- start start 2024-02-15T18:23:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.236+0000 7f7b0e17d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b080f55c0 0x7f7b080f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.236+0000 7f7b0e17d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b080f6860 0x7f7b080f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.236+0000 7f7b0e17d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080f8100 con 0x7f7b080f55c0 2024-02-15T18:23:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.236+0000 7f7b0e17d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080f8240 con 0x7f7b080f6860 2024-02-15T18:23:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.237+0000 7f7b077fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b080f55c0 0x7f7b080f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.237+0000 7f7b077fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b080f55c0 0x7f7b080f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41252/0 (socket says 172.21.15.42:41252) 2024-02-15T18:23:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.237+0000 7f7b077fe700 1 -- 172.21.15.42:0/869261597 learned_addr learned my addr 172.21.15.42:0/869261597 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.237+0000 7f7b06ffd700 1 --2- 172.21.15.42:0/869261597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b080f6860 0x7f7b080f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.237+0000 7f7b077fe700 1 -- 172.21.15.42:0/869261597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b080f6860 msgr2=0x7f7b080f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.237+0000 7f7b077fe700 1 --2- 172.21.15.42:0/869261597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b080f6860 0x7f7b080f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.237+0000 7f7b077fe700 1 -- 172.21.15.42:0/869261597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af8005040 con 0x7f7b080f55c0 2024-02-15T18:23:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.237+0000 7f7b077fe700 1 --2- 172.21.15.42:0/869261597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b080f55c0 0x7f7b080f9720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f7af8004e30 tx=0x7f7af8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.237+0000 7f7aeffff700 1 -- 172.21.15.42:0/869261597 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7af8018070 con 0x7f7b080f55c0 2024-02-15T18:23:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.237+0000 7f7aeffff700 1 -- 172.21.15.42:0/869261597 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7af8007ba0 con 0x7f7b080f55c0 2024-02-15T18:23:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.237+0000 7f7b0e17d700 1 -- 172.21.15.42:0/869261597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b080f84c0 con 0x7f7b080f55c0 2024-02-15T18:23:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.237+0000 7f7aeffff700 1 -- 172.21.15.42:0/869261597 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7af801c3c0 con 0x7f7b080f55c0 2024-02-15T18:23:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.237+0000 7f7b0e17d700 1 -- 172.21.15.42:0/869261597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b080f8a20 con 0x7f7b080f55c0 2024-02-15T18:23:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.238+0000 7f7b0e17d700 1 -- 172.21.15.42:0/869261597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b08045ad0 con 0x7f7b080f55c0 2024-02-15T18:23:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.239+0000 7f7aeffff700 1 -- 172.21.15.42:0/869261597 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f7af8007d10 con 0x7f7b080f55c0 2024-02-15T18:23:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.239+0000 7f7aeffff700 1 --2- 172.21.15.42:0/869261597 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7af005c2b0 0x7f7af005e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.239+0000 7f7aeffff700 1 -- 172.21.15.42:0/869261597 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f7af8077790 con 0x7f7b080f55c0 2024-02-15T18:23:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.239+0000 7f7b06ffd700 1 --2- 172.21.15.42:0/869261597 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7af005c2b0 0x7f7af005e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.239+0000 7f7b06ffd700 1 --2- 172.21.15.42:0/869261597 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7af005c2b0 0x7f7af005e770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f7afc004e20 tx=0x7f7afc00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:27.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.243+0000 7f7aeffff700 1 -- 172.21.15.42:0/869261597 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7af804a790 con 0x7f7b080f55c0 2024-02-15T18:23:27.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.428+0000 7f7b0e17d700 1 -- 172.21.15.42:0/869261597 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b080faca0 con 0x7f7af005c2b0 2024-02-15T18:23:27.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.435+0000 7f7aeffff700 1 -- 172.21.15.42:0/869261597 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2142 (secure 0 0 0) 0x7f7b080faca0 con 0x7f7af005c2b0 2024-02-15T18:23:27.435 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE VERSION IMAGE ID CONTAINER ID 2024-02-15T18:23:27.436 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (3m) 62s ago 5m 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:23:27.436 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (5m) 62s ago 5m 16.2.4 8d91d370c2b8 94d74c33c758 2024-02-15T18:23:27.436 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (3m) 75s ago 3m 16.2.4 8d91d370c2b8 ff0f56f9475a 2024-02-15T18:23:27.436 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (3m) 62s ago 4m 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:23:27.436 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:9283 running (6m) 62s ago 6m 16.2.4 8d91d370c2b8 a56a503c9572 2024-02-15T18:23:27.436 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (3m) 75s ago 3m 16.2.4 8d91d370c2b8 f2a90ea3fad1 2024-02-15T18:23:27.436 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (6m) 62s ago 6m 16.2.4 8d91d370c2b8 9dcbc5c49e05 2024-02-15T18:23:27.436 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (3m) 75s ago 3m 16.2.4 8d91d370c2b8 c3dd4902e2a2 2024-02-15T18:23:27.436 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (4m) 62s ago 4m 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:23:27.436 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (3m) 75s ago 3m 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:23:27.436 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (3m) 62s ago 3m 16.2.4 8d91d370c2b8 973745b31d5a 2024-02-15T18:23:27.436 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (2m) 62s ago 2m 16.2.4 8d91d370c2b8 2b3ae86111e8 2024-02-15T18:23:27.436 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (2m) 62s ago 2m 16.2.4 8d91d370c2b8 3706071e4698 2024-02-15T18:23:27.437 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (2m) 75s ago 2m 16.2.4 8d91d370c2b8 e6fe8e77c6fe 2024-02-15T18:23:27.437 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (100s) 75s ago 99s 16.2.4 8d91d370c2b8 24582012e234 2024-02-15T18:23:27.437 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (78s) 75s ago 77s 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:23:27.437 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (2m) 62s ago 4m 2.18.1 de242295e225 7fffb9a3be63 2024-02-15T18:23:27.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.437+0000 7f7b0e17d700 1 -- 172.21.15.42:0/869261597 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7af005c2b0 msgr2=0x7f7af005e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:27.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.437+0000 7f7b0e17d700 1 --2- 172.21.15.42:0/869261597 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7af005c2b0 0x7f7af005e770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f7afc004e20 tx=0x7f7afc00b040).stop 2024-02-15T18:23:27.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.437+0000 7f7b0e17d700 1 -- 172.21.15.42:0/869261597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b080f55c0 msgr2=0x7f7b080f9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.437+0000 7f7b0e17d700 1 --2- 172.21.15.42:0/869261597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b080f55c0 0x7f7b080f9720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f7af8004e30 tx=0x7f7af8009b30).stop 2024-02-15T18:23:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.437+0000 7f7b0e17d700 1 -- 172.21.15.42:0/869261597 shutdown_connections 2024-02-15T18:23:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.437+0000 7f7b0e17d700 1 --2- 172.21.15.42:0/869261597 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7af005c2b0 0x7f7af005e770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.437+0000 7f7b0e17d700 1 --2- 172.21.15.42:0/869261597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b080f55c0 0x7f7b080f9720 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.437+0000 7f7b0e17d700 1 --2- 172.21.15.42:0/869261597 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b080f6860 0x7f7b080f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.437+0000 7f7b0e17d700 1 -- 172.21.15.42:0/869261597 >> 172.21.15.42:0/869261597 conn(0x7f7b080f0d40 msgr2=0x7f7b0805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.437+0000 7f7b0e17d700 1 -- 172.21.15.42:0/869261597 shutdown_connections 2024-02-15T18:23:27.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:27.437+0000 7f7b0e17d700 1 -- 172.21.15.42:0/869261597 wait complete. 2024-02-15T18:23:28.084 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-15T18:23:28.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:26.815974+0000 mgr.smithi042.kfoihb (mgr.14162) 224 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:28.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:27.429954+0000 mgr.smithi042.kfoihb (mgr.14162) 225 : audit [DBG] from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:23:28.746 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:26.815974+0000 mgr.smithi042.kfoihb (mgr.14162) 224 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:28.746 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:27.429954+0000 mgr.smithi042.kfoihb (mgr.14162) 225 : audit [DBG] from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:23:29.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.991+0000 7ff3db49e700 1 -- 172.21.15.42:0/2037807535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3d40fba80 msgr2=0x7ff3d40fde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:29.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.991+0000 7ff3db49e700 1 --2- 172.21.15.42:0/2037807535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3d40fba80 0x7ff3d40fde70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7ff3c4004690 tx=0x7ff3c4009b30).stop 2024-02-15T18:23:29.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.991+0000 7ff3db49e700 1 -- 172.21.15.42:0/2037807535 shutdown_connections 2024-02-15T18:23:29.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.991+0000 7ff3db49e700 1 --2- 172.21.15.42:0/2037807535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3d40fba80 0x7ff3d40fde70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:29.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.991+0000 7ff3db49e700 1 --2- 172.21.15.42:0/2037807535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3d4060290 0x7ff3d40fb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:29.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.991+0000 7ff3db49e700 1 -- 172.21.15.42:0/2037807535 >> 172.21.15.42:0/2037807535 conn(0x7ff3d40f0b90 msgr2=0x7ff3d40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:29.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.991+0000 7ff3db49e700 1 -- 172.21.15.42:0/2037807535 shutdown_connections 2024-02-15T18:23:29.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.991+0000 7ff3db49e700 1 -- 172.21.15.42:0/2037807535 wait complete. 2024-02-15T18:23:29.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.991+0000 7ff3db49e700 1 Processor -- start 2024-02-15T18:23:29.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.991+0000 7ff3db49e700 1 -- start start 2024-02-15T18:23:29.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3db49e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3d4060290 0x7ff3d4108fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:29.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3db49e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3d40fba80 0x7ff3d41094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:29.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3db49e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d4109a70 con 0x7ff3d40fba80 2024-02-15T18:23:29.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3db49e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d4109bb0 con 0x7ff3d4060290 2024-02-15T18:23:29.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3d9c9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3d40fba80 0x7ff3d41094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:29.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3d9c9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3d40fba80 0x7ff3d41094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41284/0 (socket says 172.21.15.42:41284) 2024-02-15T18:23:29.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3d9c9b700 1 -- 172.21.15.42:0/1692916090 learned_addr learned my addr 172.21.15.42:0/1692916090 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:29.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3da49c700 1 --2- 172.21.15.42:0/1692916090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3d4060290 0x7ff3d4108fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:29.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3d9c9b700 1 -- 172.21.15.42:0/1692916090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3d4060290 msgr2=0x7ff3d4108fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:29.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3d9c9b700 1 --2- 172.21.15.42:0/1692916090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3d4060290 0x7ff3d4108fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:29.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3d9c9b700 1 -- 172.21.15.42:0/1692916090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c4005040 con 0x7ff3d40fba80 2024-02-15T18:23:29.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3d9c9b700 1 --2- 172.21.15.42:0/1692916090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3d40fba80 0x7ff3d41094c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7ff3c40043d0 tx=0x7ff3c4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:29.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3caffd700 1 -- 172.21.15.42:0/1692916090 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3c4018070 con 0x7ff3d40fba80 2024-02-15T18:23:29.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3caffd700 1 -- 172.21.15.42:0/1692916090 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3c4013410 con 0x7ff3d40fba80 2024-02-15T18:23:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3caffd700 1 -- 172.21.15.42:0/1692916090 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3c401c620 con 0x7ff3d40fba80 2024-02-15T18:23:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3db49e700 1 -- 172.21.15.42:0/1692916090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d410a4f0 con 0x7ff3d40fba80 2024-02-15T18:23:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.992+0000 7ff3db49e700 1 -- 172.21.15.42:0/1692916090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d410aa50 con 0x7ff3d40fba80 2024-02-15T18:23:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.994+0000 7ff3db49e700 1 -- 172.21.15.42:0/1692916090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d4045ad0 con 0x7ff3d40fba80 2024-02-15T18:23:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.994+0000 7ff3caffd700 1 -- 172.21.15.42:0/1692916090 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7ff3c401d420 con 0x7ff3d40fba80 2024-02-15T18:23:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.994+0000 7ff3caffd700 1 --2- 172.21.15.42:0/1692916090 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff3c005c300 0x7ff3c005e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.994+0000 7ff3caffd700 1 -- 172.21.15.42:0/1692916090 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7ff3c40777d0 con 0x7ff3d40fba80 2024-02-15T18:23:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.994+0000 7ff3da49c700 1 --2- 172.21.15.42:0/1692916090 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff3c005c300 0x7ff3c005e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.995+0000 7ff3da49c700 1 --2- 172.21.15.42:0/1692916090 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff3c005c300 0x7ff3c005e7c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7ff3d0004c40 tx=0x7ff3d000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:29.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:29.998+0000 7ff3caffd700 1 -- 172.21.15.42:0/1692916090 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff3c404a850 con 0x7ff3d40fba80 2024-02-15T18:23:30.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:30.183+0000 7ff3db49e700 1 -- 172.21.15.42:0/1692916090 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3d410acc0 con 0x7ff3c005c300 2024-02-15T18:23:30.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:30.187+0000 7ff3caffd700 1 -- 172.21.15.42:0/1692916090 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1116 (secure 0 0 0) 0x7ff3d410acc0 con 0x7ff3c005c300 2024-02-15T18:23:30.187 INFO:teuthology.orchestra.run.smithi042.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-15T18:23:30.187 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager 1/1 65s ago 5m count:1 2024-02-15T18:23:30.187 INFO:teuthology.orchestra.run.smithi042.stdout:crash 2/2 78s ago 5m * 2024-02-15T18:23:30.187 INFO:teuthology.orchestra.run.smithi042.stdout:grafana 1/1 65s ago 5m count:1 2024-02-15T18:23:30.188 INFO:teuthology.orchestra.run.smithi042.stdout:mgr 2/2 78s ago 5m count:2 2024-02-15T18:23:30.188 INFO:teuthology.orchestra.run.smithi042.stdout:mon 2/2 78s ago 4m smithi042:172.21.15.42=smithi042;smithi191:172.21.15.191=smithi191;count:2 2024-02-15T18:23:30.188 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter 2/2 78s ago 5m * 2024-02-15T18:23:30.188 INFO:teuthology.orchestra.run.smithi042.stdout:osd.unmanaged 6/6 78s ago - 2024-02-15T18:23:30.188 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus 1/1 65s ago 5m count:1 2024-02-15T18:23:30.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:30.188+0000 7ff3db49e700 1 -- 172.21.15.42:0/1692916090 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff3c005c300 msgr2=0x7ff3c005e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:30.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:30.188+0000 7ff3db49e700 1 --2- 172.21.15.42:0/1692916090 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff3c005c300 0x7ff3c005e7c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7ff3d0004c40 tx=0x7ff3d000b040).stop 2024-02-15T18:23:30.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:30.188+0000 7ff3db49e700 1 -- 172.21.15.42:0/1692916090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3d40fba80 msgr2=0x7ff3d41094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:30.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:30.188+0000 7ff3db49e700 1 --2- 172.21.15.42:0/1692916090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3d40fba80 0x7ff3d41094c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7ff3c40043d0 tx=0x7ff3c4009b30).stop 2024-02-15T18:23:30.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:30.188+0000 7ff3db49e700 1 -- 172.21.15.42:0/1692916090 shutdown_connections 2024-02-15T18:23:30.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:30.188+0000 7ff3db49e700 1 --2- 172.21.15.42:0/1692916090 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff3c005c300 0x7ff3c005e7c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:30.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:30.188+0000 7ff3db49e700 1 --2- 172.21.15.42:0/1692916090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3d4060290 0x7ff3d4108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:30.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:30.188+0000 7ff3db49e700 1 --2- 172.21.15.42:0/1692916090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3d40fba80 0x7ff3d41094c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:30.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:30.189+0000 7ff3db49e700 1 -- 172.21.15.42:0/1692916090 >> 172.21.15.42:0/1692916090 conn(0x7ff3d40f0b90 msgr2=0x7ff3d40f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:30.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:30.189+0000 7ff3db49e700 1 -- 172.21.15.42:0/1692916090 shutdown_connections 2024-02-15T18:23:30.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:30.189+0000 7ff3db49e700 1 -- 172.21.15.42:0/1692916090 wait complete. 2024-02-15T18:23:30.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:28.816280+0000 mgr.smithi042.kfoihb (mgr.14162) 226 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:30.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:28.816280+0000 mgr.smithi042.kfoihb (mgr.14162) 226 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:30.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-15T18:23:31.609 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:30.184938+0000 mgr.smithi042.kfoihb (mgr.14162) 227 : audit [DBG] from='client.14416 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:23:31.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:30.184938+0000 mgr.smithi042.kfoihb (mgr.14162) 227 : audit [DBG] from='client.14416 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:23:32.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:30.816606+0000 mgr.smithi042.kfoihb (mgr.14162) 228 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:32.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:30.816606+0000 mgr.smithi042.kfoihb (mgr.14162) 228 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.841+0000 7f9dad7a2700 1 -- 172.21.15.42:0/2941599434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da80602b0 msgr2=0x7f9da80f9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.841+0000 7f9dad7a2700 1 --2- 172.21.15.42:0/2941599434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da80602b0 0x7f9da80f9170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f9d98004660 tx=0x7f9d98009b30).stop 2024-02-15T18:23:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.841+0000 7f9dad7a2700 1 -- 172.21.15.42:0/2941599434 shutdown_connections 2024-02-15T18:23:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.841+0000 7f9dad7a2700 1 --2- 172.21.15.42:0/2941599434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9da80f96f0 0x7f9da80fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.841+0000 7f9dad7a2700 1 --2- 172.21.15.42:0/2941599434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da80602b0 0x7f9da80f9170 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.841+0000 7f9dad7a2700 1 -- 172.21.15.42:0/2941599434 >> 172.21.15.42:0/2941599434 conn(0x7f9da80f0b90 msgr2=0x7f9da80f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:32.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.841+0000 7f9dad7a2700 1 -- 172.21.15.42:0/2941599434 shutdown_connections 2024-02-15T18:23:32.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.841+0000 7f9dad7a2700 1 -- 172.21.15.42:0/2941599434 wait complete. 2024-02-15T18:23:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.841+0000 7f9dad7a2700 1 Processor -- start 2024-02-15T18:23:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.841+0000 7f9dad7a2700 1 -- start start 2024-02-15T18:23:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9dad7a2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9da80602b0 0x7f9da8106e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9dad7a2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da80f96f0 0x7f9da8107360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9dad7a2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da8107910 con 0x7f9da80f96f0 2024-02-15T18:23:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9dad7a2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da8107a50 con 0x7f9da80602b0 2024-02-15T18:23:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9da77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da80f96f0 0x7f9da8107360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9da77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da80f96f0 0x7f9da8107360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53076/0 (socket says 172.21.15.42:53076) 2024-02-15T18:23:32.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9da77fe700 1 -- 172.21.15.42:0/110498354 learned_addr learned my addr 172.21.15.42:0/110498354 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9da7fff700 1 --2- 172.21.15.42:0/110498354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9da80602b0 0x7f9da8106e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9da77fe700 1 -- 172.21.15.42:0/110498354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9da80602b0 msgr2=0x7f9da8106e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9da77fe700 1 --2- 172.21.15.42:0/110498354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9da80602b0 0x7f9da8106e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9da77fe700 1 -- 172.21.15.42:0/110498354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d98005040 con 0x7f9da80f96f0 2024-02-15T18:23:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9da77fe700 1 --2- 172.21.15.42:0/110498354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da80f96f0 0x7f9da8107360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f9d9c00c790 tx=0x7f9d9c007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9da4ff9700 1 -- 172.21.15.42:0/110498354 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d9c013630 con 0x7f9da80f96f0 2024-02-15T18:23:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9da4ff9700 1 -- 172.21.15.42:0/110498354 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d9c013c60 con 0x7f9da80f96f0 2024-02-15T18:23:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9dad7a2700 1 -- 172.21.15.42:0/110498354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da81083f0 con 0x7f9da80f96f0 2024-02-15T18:23:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.842+0000 7f9da4ff9700 1 -- 172.21.15.42:0/110498354 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d9c002c30 con 0x7f9da80f96f0 2024-02-15T18:23:32.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.843+0000 7f9dad7a2700 1 -- 172.21.15.42:0/110498354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da81089b0 con 0x7f9da80f96f0 2024-02-15T18:23:32.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.843+0000 7f9dad7a2700 1 -- 172.21.15.42:0/110498354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da8045ad0 con 0x7f9da80f96f0 2024-02-15T18:23:32.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.844+0000 7f9da4ff9700 1 -- 172.21.15.42:0/110498354 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f9d9c01a060 con 0x7f9da80f96f0 2024-02-15T18:23:32.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.844+0000 7f9da4ff9700 1 --2- 172.21.15.42:0/110498354 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9d9005c300 0x7f9d9005e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:32.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.844+0000 7f9da4ff9700 1 -- 172.21.15.42:0/110498354 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f9d9c075d00 con 0x7f9da80f96f0 2024-02-15T18:23:32.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.844+0000 7f9da7fff700 1 --2- 172.21.15.42:0/110498354 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9d9005c300 0x7f9d9005e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:32.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.845+0000 7f9da7fff700 1 --2- 172.21.15.42:0/110498354 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9d9005c300 0x7f9d9005e7c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f9d98009f50 tx=0x7f9d98009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:32.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:32.848+0000 7f9da4ff9700 1 -- 172.21.15.42:0/110498354 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9d9c079030 con 0x7f9da80f96f0 2024-02-15T18:23:33.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:33.029+0000 7f9dad7a2700 1 -- 172.21.15.42:0/110498354 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9da80fef90 con 0x7f9d9005c300 2024-02-15T18:23:33.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:33.030+0000 7f9da4ff9700 1 -- 172.21.15.42:0/110498354 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+126 (secure 0 0 0) 0x7f9da80fef90 con 0x7f9d9005c300 2024-02-15T18:23:33.030 INFO:teuthology.orchestra.run.smithi042.stdout:HOST ADDR LABELS STATUS 2024-02-15T18:23:33.031 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 172.21.15.42 2024-02-15T18:23:33.031 INFO:teuthology.orchestra.run.smithi042.stdout:smithi191 smithi191 2024-02-15T18:23:33.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:33.032+0000 7f9dad7a2700 1 -- 172.21.15.42:0/110498354 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9d9005c300 msgr2=0x7f9d9005e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:33.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:33.032+0000 7f9dad7a2700 1 --2- 172.21.15.42:0/110498354 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9d9005c300 0x7f9d9005e7c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f9d98009f50 tx=0x7f9d98009b30).stop 2024-02-15T18:23:33.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:33.032+0000 7f9dad7a2700 1 -- 172.21.15.42:0/110498354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da80f96f0 msgr2=0x7f9da8107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:33.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:33.032+0000 7f9dad7a2700 1 --2- 172.21.15.42:0/110498354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da80f96f0 0x7f9da8107360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f9d9c00c790 tx=0x7f9d9c007d40).stop 2024-02-15T18:23:33.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:33.032+0000 7f9dad7a2700 1 -- 172.21.15.42:0/110498354 shutdown_connections 2024-02-15T18:23:33.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:33.032+0000 7f9dad7a2700 1 --2- 172.21.15.42:0/110498354 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9d9005c300 0x7f9d9005e7c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:33.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:33.032+0000 7f9dad7a2700 1 --2- 172.21.15.42:0/110498354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9da80602b0 0x7f9da8106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:33.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:33.032+0000 7f9dad7a2700 1 --2- 172.21.15.42:0/110498354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9da80f96f0 0x7f9da8107360 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:33.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:33.032+0000 7f9dad7a2700 1 -- 172.21.15.42:0/110498354 >> 172.21.15.42:0/110498354 conn(0x7f9da80f0b90 msgr2=0x7f9da80f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:33.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:33.032+0000 7f9dad7a2700 1 -- 172.21.15.42:0/110498354 shutdown_connections 2024-02-15T18:23:33.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:33.032+0000 7f9dad7a2700 1 -- 172.21.15.42:0/110498354 wait complete. 2024-02-15T18:23:33.730 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-15T18:23:34.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:32.816901+0000 mgr.smithi042.kfoihb (mgr.14162) 229 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:34.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:33.031177+0000 mgr.smithi042.kfoihb (mgr.14162) 230 : audit [DBG] from='client.14420 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:23:34.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:32.816901+0000 mgr.smithi042.kfoihb (mgr.14162) 229 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:34.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:33.031177+0000 mgr.smithi042.kfoihb (mgr.14162) 230 : audit [DBG] from='client.14420 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:23:35.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.778+0000 7fe3e057f700 1 -- 172.21.15.42:0/424812610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3d80f7980 msgr2=0x7fe3d80f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:35.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.778+0000 7fe3e057f700 1 --2- 172.21.15.42:0/424812610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3d80f7980 0x7fe3d80f7de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fe3cc004660 tx=0x7fe3cc009b30).stop 2024-02-15T18:23:35.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.778+0000 7fe3e057f700 1 -- 172.21.15.42:0/424812610 shutdown_connections 2024-02-15T18:23:35.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.778+0000 7fe3e057f700 1 --2- 172.21.15.42:0/424812610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3d80f7980 0x7fe3d80f7de0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:35.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.778+0000 7fe3e057f700 1 --2- 172.21.15.42:0/424812610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d80f66e0 0x7fe3d80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:35.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.778+0000 7fe3e057f700 1 -- 172.21.15.42:0/424812610 >> 172.21.15.42:0/424812610 conn(0x7fe3d80f1e40 msgr2=0x7fe3d80f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:35.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.778+0000 7fe3e057f700 1 -- 172.21.15.42:0/424812610 shutdown_connections 2024-02-15T18:23:35.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.778+0000 7fe3e057f700 1 -- 172.21.15.42:0/424812610 wait complete. 2024-02-15T18:23:35.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.779+0000 7fe3e057f700 1 Processor -- start 2024-02-15T18:23:35.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.779+0000 7fe3e057f700 1 -- start start 2024-02-15T18:23:35.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.779+0000 7fe3e057f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d80f66e0 0x7fe3d8107090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:35.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.779+0000 7fe3e057f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3d80f7980 0x7fe3d8107580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:35.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.779+0000 7fe3e057f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d8107b30 con 0x7fe3d80f7980 2024-02-15T18:23:35.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.779+0000 7fe3e057f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d8107c70 con 0x7fe3d80f66e0 2024-02-15T18:23:35.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.779+0000 7fe3ddb1a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3d80f7980 0x7fe3d8107580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:35.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.779+0000 7fe3ddb1a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3d80f7980 0x7fe3d8107580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53096/0 (socket says 172.21.15.42:53096) 2024-02-15T18:23:35.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.779+0000 7fe3ddb1a700 1 -- 172.21.15.42:0/3211582792 learned_addr learned my addr 172.21.15.42:0/3211582792 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:35.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.779+0000 7fe3de31b700 1 --2- 172.21.15.42:0/3211582792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d80f66e0 0x7fe3d8107090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:35.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.779+0000 7fe3ddb1a700 1 -- 172.21.15.42:0/3211582792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d80f66e0 msgr2=0x7fe3d8107090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:35.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.779+0000 7fe3ddb1a700 1 --2- 172.21.15.42:0/3211582792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d80f66e0 0x7fe3d8107090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:35.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.779+0000 7fe3ddb1a700 1 -- 172.21.15.42:0/3211582792 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc005040 con 0x7fe3d80f7980 2024-02-15T18:23:35.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.779+0000 7fe3ddb1a700 1 --2- 172.21.15.42:0/3211582792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3d80f7980 0x7fe3d8107580 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fe3cc009830 tx=0x7fe3cc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:35.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.780+0000 7fe3caffd700 1 -- 172.21.15.42:0/3211582792 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3cc018070 con 0x7fe3d80f7980 2024-02-15T18:23:35.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.780+0000 7fe3e057f700 1 -- 172.21.15.42:0/3211582792 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d81085b0 con 0x7fe3d80f7980 2024-02-15T18:23:35.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.780+0000 7fe3caffd700 1 -- 172.21.15.42:0/3211582792 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3cc007ba0 con 0x7fe3d80f7980 2024-02-15T18:23:35.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.780+0000 7fe3caffd700 1 -- 172.21.15.42:0/3211582792 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3cc01c3c0 con 0x7fe3d80f7980 2024-02-15T18:23:35.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.780+0000 7fe3e057f700 1 -- 172.21.15.42:0/3211582792 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d8108b10 con 0x7fe3d80f7980 2024-02-15T18:23:35.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.781+0000 7fe3e057f700 1 -- 172.21.15.42:0/3211582792 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d8045ad0 con 0x7fe3d80f7980 2024-02-15T18:23:35.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.781+0000 7fe3caffd700 1 -- 172.21.15.42:0/3211582792 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fe3cc007d10 con 0x7fe3d80f7980 2024-02-15T18:23:35.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.781+0000 7fe3caffd700 1 --2- 172.21.15.42:0/3211582792 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe3c405c2b0 0x7fe3c405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:35.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.781+0000 7fe3caffd700 1 -- 172.21.15.42:0/3211582792 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7fe3cc0777c0 con 0x7fe3d80f7980 2024-02-15T18:23:35.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.781+0000 7fe3de31b700 1 --2- 172.21.15.42:0/3211582792 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe3c405c2b0 0x7fe3c405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:35.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.782+0000 7fe3de31b700 1 --2- 172.21.15.42:0/3211582792 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe3c405c2b0 0x7fe3c405e770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fe3d80f77e0 tx=0x7fe3d400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:35.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.785+0000 7fe3caffd700 1 -- 172.21.15.42:0/3211582792 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe3cc04a760 con 0x7fe3d80f7980 2024-02-15T18:23:35.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.967+0000 7fe3e057f700 1 -- 172.21.15.42:0/3211582792 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3d80ff0c0 con 0x7fe3c405c2b0 2024-02-15T18:23:35.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.968+0000 7fe3caffd700 1 -- 172.21.15.42:0/3211582792 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+279 (secure 0 0 0) 0x7fe3d80ff0c0 con 0x7fe3c405c2b0 2024-02-15T18:23:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:Hostname Path Type Serial Size Health Ident Fault Available 2024-02-15T18:23:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 /dev/nvme0n1 ssd PHMB75130021480DGN 480G Unknown N/A N/A No 2024-02-15T18:23:35.969 INFO:teuthology.orchestra.run.smithi042.stdout:smithi191 /dev/nvme0n1 ssd CVFT6303008M400BGN 400G Unknown N/A N/A No 2024-02-15T18:23:35.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.970+0000 7fe3e057f700 1 -- 172.21.15.42:0/3211582792 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe3c405c2b0 msgr2=0x7fe3c405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:35.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.970+0000 7fe3e057f700 1 --2- 172.21.15.42:0/3211582792 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe3c405c2b0 0x7fe3c405e770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fe3d80f77e0 tx=0x7fe3d400b040).stop 2024-02-15T18:23:35.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.970+0000 7fe3e057f700 1 -- 172.21.15.42:0/3211582792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3d80f7980 msgr2=0x7fe3d8107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:35.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.970+0000 7fe3e057f700 1 --2- 172.21.15.42:0/3211582792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3d80f7980 0x7fe3d8107580 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fe3cc009830 tx=0x7fe3cc009b30).stop 2024-02-15T18:23:35.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.970+0000 7fe3e057f700 1 -- 172.21.15.42:0/3211582792 shutdown_connections 2024-02-15T18:23:35.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.970+0000 7fe3e057f700 1 --2- 172.21.15.42:0/3211582792 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe3c405c2b0 0x7fe3c405e770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:35.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.970+0000 7fe3e057f700 1 --2- 172.21.15.42:0/3211582792 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3d80f66e0 0x7fe3d8107090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:35.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.970+0000 7fe3e057f700 1 --2- 172.21.15.42:0/3211582792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe3d80f7980 0x7fe3d8107580 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:35.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.970+0000 7fe3e057f700 1 -- 172.21.15.42:0/3211582792 >> 172.21.15.42:0/3211582792 conn(0x7fe3d80f1e40 msgr2=0x7fe3d80fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:35.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.970+0000 7fe3e057f700 1 -- 172.21.15.42:0/3211582792 shutdown_connections 2024-02-15T18:23:35.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:35.970+0000 7fe3e057f700 1 -- 172.21.15.42:0/3211582792 wait complete. 2024-02-15T18:23:36.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:34.817372+0000 mgr.smithi042.kfoihb (mgr.14162) 231 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:36.651 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-15T18:23:36.661 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-02-15T18:23:36.661 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-02-15T18:23:36.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:34.817372+0000 mgr.smithi042.kfoihb (mgr.14162) 231 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:37.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:35.968908+0000 mgr.smithi042.kfoihb (mgr.14162) 232 : audit [DBG] from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:23:37.731 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:35.968908+0000 mgr.smithi042.kfoihb (mgr.14162) 232 : audit [DBG] from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:23:38.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:36.817692+0000 mgr.smithi042.kfoihb (mgr.14162) 233 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:38.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.567+0000 7f6b6e078700 1 -- 172.21.15.42:0/2334218711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b680f55c0 msgr2=0x7f6b680f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:38.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.567+0000 7f6b6e078700 1 --2- 172.21.15.42:0/2334218711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b680f55c0 0x7f6b680f59e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f6b58004660 tx=0x7f6b58009b30).stop 2024-02-15T18:23:38.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.567+0000 7f6b6e078700 1 -- 172.21.15.42:0/2334218711 shutdown_connections 2024-02-15T18:23:38.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.567+0000 7f6b6e078700 1 --2- 172.21.15.42:0/2334218711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b680f6860 0x7f6b680f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:38.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.567+0000 7f6b6e078700 1 --2- 172.21.15.42:0/2334218711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b680f55c0 0x7f6b680f59e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:38.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.567+0000 7f6b6e078700 1 -- 172.21.15.42:0/2334218711 >> 172.21.15.42:0/2334218711 conn(0x7f6b680f0d40 msgr2=0x7f6b680f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:38.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.567+0000 7f6b6e078700 1 -- 172.21.15.42:0/2334218711 shutdown_connections 2024-02-15T18:23:38.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.567+0000 7f6b6e078700 1 -- 172.21.15.42:0/2334218711 wait complete. 2024-02-15T18:23:38.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.568+0000 7f6b6e078700 1 Processor -- start 2024-02-15T18:23:38.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.568+0000 7f6b6e078700 1 -- start start 2024-02-15T18:23:38.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.568+0000 7f6b6e078700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b680f55c0 0x7f6b681070a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:38.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.568+0000 7f6b6e078700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b680f6860 0x7f6b68107590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.568+0000 7f6b6e078700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68107b40 con 0x7f6b680f6860 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.568+0000 7f6b6e078700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68107c80 con 0x7f6b680f55c0 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.568+0000 7f6b5effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b680f6860 0x7f6b68107590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.568+0000 7f6b5effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b680f6860 0x7f6b68107590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53112/0 (socket says 172.21.15.42:53112) 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.568+0000 7f6b5effd700 1 -- 172.21.15.42:0/3680783248 learned_addr learned my addr 172.21.15.42:0/3680783248 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.568+0000 7f6b677fe700 1 --2- 172.21.15.42:0/3680783248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b680f55c0 0x7f6b681070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.568+0000 7f6b5effd700 1 -- 172.21.15.42:0/3680783248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b680f55c0 msgr2=0x7f6b681070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.568+0000 7f6b5effd700 1 --2- 172.21.15.42:0/3680783248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b680f55c0 0x7f6b681070a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.568+0000 7f6b5effd700 1 -- 172.21.15.42:0/3680783248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b58005040 con 0x7f6b680f6860 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.568+0000 7f6b5effd700 1 --2- 172.21.15.42:0/3680783248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b680f6860 0x7f6b68107590 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f6b5000c530 tx=0x7f6b50007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.569+0000 7f6b64ff9700 1 -- 172.21.15.42:0/3680783248 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b5000f650 con 0x7f6b680f6860 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.569+0000 7f6b64ff9700 1 -- 172.21.15.42:0/3680783248 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b5000fc80 con 0x7f6b680f6860 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.569+0000 7f6b6e078700 1 -- 172.21.15.42:0/3680783248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b68108620 con 0x7f6b680f6860 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.569+0000 7f6b64ff9700 1 -- 172.21.15.42:0/3680783248 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b50012d80 con 0x7f6b680f6860 2024-02-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.569+0000 7f6b6e078700 1 -- 172.21.15.42:0/3680783248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b68108be0 con 0x7f6b680f6860 2024-02-15T18:23:38.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.570+0000 7f6b6e078700 1 -- 172.21.15.42:0/3680783248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b68045ad0 con 0x7f6b680f6860 2024-02-15T18:23:38.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.570+0000 7f6b64ff9700 1 -- 172.21.15.42:0/3680783248 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f6b50010070 con 0x7f6b680f6860 2024-02-15T18:23:38.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.570+0000 7f6b64ff9700 1 --2- 172.21.15.42:0/3680783248 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f6b5405c2b0 0x7f6b5405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:38.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.570+0000 7f6b64ff9700 1 -- 172.21.15.42:0/3680783248 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4552+0+0 (secure 0 0 0) 0x7f6b500757f0 con 0x7f6b680f6860 2024-02-15T18:23:38.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.570+0000 7f6b677fe700 1 --2- 172.21.15.42:0/3680783248 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f6b5405c2b0 0x7f6b5405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:38.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.571+0000 7f6b677fe700 1 --2- 172.21.15.42:0/3680783248 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f6b5405c2b0 0x7f6b5405e770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f6b58009d60 tx=0x7f6b58009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:38.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.574+0000 7f6b64ff9700 1 -- 172.21.15.42:0/3680783248 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6b500452b0 con 0x7f6b680f6860 2024-02-15T18:23:38.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:36.817692+0000 mgr.smithi042.kfoihb (mgr.14162) 233 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:38.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:38.760+0000 7f6b6e078700 1 -- 172.21.15.42:0/3680783248 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b6805fd30 con 0x7f6b5405c2b0 2024-02-15T18:23:39.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:38.762196+0000 mon.smithi042 (mon.0) 454 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-15T18:23:39.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:38.762196+0000 mon.smithi042 (mon.0) 454 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-15T18:23:40.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:40.491+0000 7f6b64ff9700 1 -- 172.21.15.42:0/3680783248 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6b6805fd30 con 0x7f6b5405c2b0 2024-02-15T18:23:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:40.493+0000 7f6b6e078700 1 -- 172.21.15.42:0/3680783248 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f6b5405c2b0 msgr2=0x7f6b5405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:40.493+0000 7f6b6e078700 1 --2- 172.21.15.42:0/3680783248 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f6b5405c2b0 0x7f6b5405e770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f6b58009d60 tx=0x7f6b58009b30).stop 2024-02-15T18:23:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:40.493+0000 7f6b6e078700 1 -- 172.21.15.42:0/3680783248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b680f6860 msgr2=0x7f6b68107590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:40.493+0000 7f6b6e078700 1 --2- 172.21.15.42:0/3680783248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b680f6860 0x7f6b68107590 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f6b5000c530 tx=0x7f6b50007e20).stop 2024-02-15T18:23:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:40.493+0000 7f6b6e078700 1 -- 172.21.15.42:0/3680783248 shutdown_connections 2024-02-15T18:23:40.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:40.493+0000 7f6b6e078700 1 --2- 172.21.15.42:0/3680783248 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f6b5405c2b0 0x7f6b5405e770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:40.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:40.493+0000 7f6b6e078700 1 --2- 172.21.15.42:0/3680783248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b680f55c0 0x7f6b681070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:40.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:40.493+0000 7f6b6e078700 1 --2- 172.21.15.42:0/3680783248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b680f6860 0x7f6b68107590 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:40.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:40.493+0000 7f6b6e078700 1 -- 172.21.15.42:0/3680783248 >> 172.21.15.42:0/3680783248 conn(0x7f6b680f0d40 msgr2=0x7f6b6805f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:40.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:40.493+0000 7f6b6e078700 1 -- 172.21.15.42:0/3680783248 shutdown_connections 2024-02-15T18:23:40.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:40.493+0000 7f6b6e078700 1 -- 172.21.15.42:0/3680783248 wait complete. 2024-02-15T18:23:40.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:38.761718+0000 mgr.smithi042.kfoihb (mgr.14162) 234 : audit [DBG] from='client.14428 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:23:40.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:38.818086+0000 mgr.smithi042.kfoihb (mgr.14162) 235 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:40.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:39.457691+0000 mon.smithi042 (mon.0) 455 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-15T18:23:40.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:39.457732+0000 mon.smithi042 (mon.0) 456 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-02-15T18:23:40.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:39.458556+0000 mon.smithi042 (mon.0) 457 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-15T18:23:40.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:23:40.468+0000 7f0b5dcfc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-15T18:23:40.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:38.761718+0000 mgr.smithi042.kfoihb (mgr.14162) 234 : audit [DBG] from='client.14428 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:23:40.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:38.818086+0000 mgr.smithi042.kfoihb (mgr.14162) 235 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:40.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:39.457691+0000 mon.smithi042 (mon.0) 455 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-15T18:23:40.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:39.457732+0000 mon.smithi042 (mon.0) 456 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2024-02-15T18:23:40.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:39.458556+0000 mon.smithi042 (mon.0) 457 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-15T18:23:41.186 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-15T18:23:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:40.462462+0000 mon.smithi042 (mon.0) 458 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-15T18:23:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:40.462680+0000 mon.smithi042 (mon.0) 459 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-02-15T18:23:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:40.468821+0000 mon.smithi042 (mon.0) 460 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-15T18:23:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:40.469664+0000 mon.smithi042 (mon.0) 461 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-15T18:23:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:40.469689+0000 mon.smithi042 (mon.0) 462 : cluster [WRN] Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-15T18:23:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:40.483872+0000 mon.smithi042 (mon.0) 463 : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2024-02-15T18:23:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:40.484020+0000 mon.smithi042 (mon.0) 464 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-15T18:23:41.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:40.484084+0000 mon.smithi042 (mon.0) 465 : cluster [DBG] fsmap cephfs:0 2024-02-15T18:23:41.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:23:40.488363+0000 mgr.smithi042.kfoihb (mgr.14162) 236 : cephadm [INF] Saving service mds.cephfs spec with placement count:4 2024-02-15T18:23:41.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:40.491778+0000 mon.smithi042 (mon.0) 466 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:41.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:40.492427+0000 mon.smithi042 (mon.0) 467 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:23:41.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:40.499778+0000 mon.smithi042 (mon.0) 468 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:41.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:40.507002+0000 mon.smithi042 (mon.0) 469 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:41.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:40.507830+0000 mon.smithi042 (mon.0) 470 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi191.icwcnb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:23:41.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:40.510577+0000 mon.smithi042 (mon.0) 471 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi191.icwcnb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:23:41.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:40.511238+0000 mon.smithi042 (mon.0) 472 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:23:41.814 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:23:40.511789+0000 mgr.smithi042.kfoihb (mgr.14162) 237 : cephadm [INF] Deploying daemon mds.cephfs.smithi191.icwcnb on smithi191 2024-02-15T18:23:41.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:40.462462+0000 mon.smithi042 (mon.0) 458 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-15T18:23:41.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:40.462680+0000 mon.smithi042 (mon.0) 459 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2024-02-15T18:23:41.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:40.468821+0000 mon.smithi042 (mon.0) 460 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-15T18:23:41.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:40.469664+0000 mon.smithi042 (mon.0) 461 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-15T18:23:41.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:40.469689+0000 mon.smithi042 (mon.0) 462 : cluster [WRN] Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-15T18:23:41.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:40.483872+0000 mon.smithi042 (mon.0) 463 : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2024-02-15T18:23:41.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:40.484020+0000 mon.smithi042 (mon.0) 464 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-15T18:23:41.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:40.484084+0000 mon.smithi042 (mon.0) 465 : cluster [DBG] fsmap cephfs:0 2024-02-15T18:23:41.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:23:40.488363+0000 mgr.smithi042.kfoihb (mgr.14162) 236 : cephadm [INF] Saving service mds.cephfs spec with placement count:4 2024-02-15T18:23:41.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:40.491778+0000 mon.smithi042 (mon.0) 466 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:41.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:40.492427+0000 mon.smithi042 (mon.0) 467 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:23:41.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:40.499778+0000 mon.smithi042 (mon.0) 468 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:41.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:40.507002+0000 mon.smithi042 (mon.0) 469 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:41.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:40.507830+0000 mon.smithi042 (mon.0) 470 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi191.icwcnb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:23:41.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:40.510577+0000 mon.smithi042 (mon.0) 471 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi191.icwcnb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:23:41.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:40.511238+0000 mon.smithi042 (mon.0) 472 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:23:41.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:23:40.511789+0000 mgr.smithi042.kfoihb (mgr.14162) 237 : cephadm [INF] Deploying daemon mds.cephfs.smithi191.icwcnb on smithi191 2024-02-15T18:23:42.560 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:40.818548+0000 mgr.smithi042.kfoihb (mgr.14162) 238 : cluster [DBG] pgmap v203: 65 pgs: 10 creating+peering, 1 active+clean, 54 unknown; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:42.560 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:41.482974+0000 mon.smithi042 (mon.0) 473 : cluster [DBG] osdmap e43: 6 total, 6 up, 6 in 2024-02-15T18:23:42.634 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:40.818548+0000 mgr.smithi042.kfoihb (mgr.14162) 238 : cluster [DBG] pgmap v203: 65 pgs: 10 creating+peering, 1 active+clean, 54 unknown; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:42.634 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:41.482974+0000 mon.smithi042 (mon.0) 473 : cluster [DBG] osdmap e43: 6 total, 6 up, 6 in 2024-02-15T18:23:43.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.044+0000 7f49b566c700 1 -- 172.21.15.42:0/710047042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49b00f6860 msgr2=0x7f49b00f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:43.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.044+0000 7f49b566c700 1 --2- 172.21.15.42:0/710047042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49b00f6860 0x7f49b00f6cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f49a4004660 tx=0x7f49a4009b30).stop 2024-02-15T18:23:43.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.044+0000 7f49b566c700 1 -- 172.21.15.42:0/710047042 shutdown_connections 2024-02-15T18:23:43.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.044+0000 7f49b566c700 1 --2- 172.21.15.42:0/710047042 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49b00f6860 0x7f49b00f6cc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:43.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.044+0000 7f49b566c700 1 --2- 172.21.15.42:0/710047042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b00f55c0 0x7f49b00f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:43.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.044+0000 7f49b566c700 1 -- 172.21.15.42:0/710047042 >> 172.21.15.42:0/710047042 conn(0x7f49b00f0d40 msgr2=0x7f49b00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:43.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.044+0000 7f49b566c700 1 -- 172.21.15.42:0/710047042 shutdown_connections 2024-02-15T18:23:43.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.044+0000 7f49b566c700 1 -- 172.21.15.42:0/710047042 wait complete. 2024-02-15T18:23:43.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.045+0000 7f49b566c700 1 Processor -- start 2024-02-15T18:23:43.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.045+0000 7f49b566c700 1 -- start start 2024-02-15T18:23:43.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.045+0000 7f49b566c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b00f55c0 0x7f49b00f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:43.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.045+0000 7f49b566c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49b00f6860 0x7f49b00f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:43.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.045+0000 7f49b566c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b00f8100 con 0x7f49b00f6860 2024-02-15T18:23:43.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.045+0000 7f49b566c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b00f8240 con 0x7f49b00f55c0 2024-02-15T18:23:43.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.045+0000 7f49ae7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49b00f6860 0x7f49b00f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:43.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.045+0000 7f49ae7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49b00f6860 0x7f49b00f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34188/0 (socket says 172.21.15.42:34188) 2024-02-15T18:23:43.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.045+0000 7f49ae7fc700 1 -- 172.21.15.42:0/2503995324 learned_addr learned my addr 172.21.15.42:0/2503995324 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:43.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.045+0000 7f49aeffd700 1 --2- 172.21.15.42:0/2503995324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b00f55c0 0x7f49b00f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:43.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.045+0000 7f49ae7fc700 1 -- 172.21.15.42:0/2503995324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b00f55c0 msgr2=0x7f49b00f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:43.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.045+0000 7f49ae7fc700 1 --2- 172.21.15.42:0/2503995324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b00f55c0 0x7f49b00f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:43.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.045+0000 7f49ae7fc700 1 -- 172.21.15.42:0/2503995324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a4005040 con 0x7f49b00f6860 2024-02-15T18:23:43.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.046+0000 7f49ae7fc700 1 --2- 172.21.15.42:0/2503995324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49b00f6860 0x7f49b00f7ba0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f49a4000c00 tx=0x7f49a400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:43.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.046+0000 7f49977fe700 1 -- 172.21.15.42:0/2503995324 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49a4019070 con 0x7f49b00f6860 2024-02-15T18:23:43.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.046+0000 7f49977fe700 1 -- 172.21.15.42:0/2503995324 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49a4007e90 con 0x7f49b00f6860 2024-02-15T18:23:43.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.046+0000 7f49b566c700 1 -- 172.21.15.42:0/2503995324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b00f84c0 con 0x7f49b00f6860 2024-02-15T18:23:43.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.046+0000 7f49977fe700 1 -- 172.21.15.42:0/2503995324 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49a4014d50 con 0x7f49b00f6860 2024-02-15T18:23:43.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.046+0000 7f49b566c700 1 -- 172.21.15.42:0/2503995324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b00f8a20 con 0x7f49b00f6860 2024-02-15T18:23:43.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.047+0000 7f49b566c700 1 -- 172.21.15.42:0/2503995324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b0045ad0 con 0x7f49b00f6860 2024-02-15T18:23:43.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.047+0000 7f49977fe700 1 -- 172.21.15.42:0/2503995324 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f49a4025070 con 0x7f49b00f6860 2024-02-15T18:23:43.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.048+0000 7f49977fe700 1 --2- 172.21.15.42:0/2503995324 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f499805c2b0 0x7f499805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:43.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.048+0000 7f49977fe700 1 -- 172.21.15.42:0/2503995324 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f49a4076d60 con 0x7f49b00f6860 2024-02-15T18:23:43.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.048+0000 7f49aeffd700 1 --2- 172.21.15.42:0/2503995324 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f499805c2b0 0x7f499805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:43.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.048+0000 7f49aeffd700 1 --2- 172.21.15.42:0/2503995324 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f499805c2b0 0x7f499805e770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f49b00f94b0 tx=0x7f49a000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:43.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.052+0000 7f49977fe700 1 -- 172.21.15.42:0/2503995324 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f49a4049aa0 con 0x7f49b00f6860 2024-02-15T18:23:43.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.268+0000 7f49b566c700 1 -- 172.21.15.42:0/2503995324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f49b00fde90 con 0x7f49b00f6860 2024-02-15T18:23:43.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.268+0000 7f49977fe700 1 -- 172.21.15.42:0/2503995324 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1003 (secure 0 0 0) 0x7f49a404d0c0 con 0x7f49b00f6860 2024-02-15T18:23:43.269 INFO:teuthology.orchestra.run.smithi042.stdout:e2 2024-02-15T18:23:43.269 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:23:43.269 INFO:teuthology.orchestra.run.smithi042.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:23:43.269 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:23:43.269 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:23:43.269 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:23:43.269 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:23:43.269 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 2 2024-02-15T18:23:43.269 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:40.469572+0000 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:in 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-02-15T18:23:43.270 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:23:43.271 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:23:43.271 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:23:43.271 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:23:43.271 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:23:43.271 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:23:43.271 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:23:43.271 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 0 2024-02-15T18:23:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:23:43.271 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:23:43.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.270+0000 7f49b566c700 1 -- 172.21.15.42:0/2503995324 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f499805c2b0 msgr2=0x7f499805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:43.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.270+0000 7f49b566c700 1 --2- 172.21.15.42:0/2503995324 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f499805c2b0 0x7f499805e770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f49b00f94b0 tx=0x7f49a000b040).stop 2024-02-15T18:23:43.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.270+0000 7f49b566c700 1 -- 172.21.15.42:0/2503995324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49b00f6860 msgr2=0x7f49b00f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:43.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.270+0000 7f49b566c700 1 --2- 172.21.15.42:0/2503995324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49b00f6860 0x7f49b00f7ba0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f49a4000c00 tx=0x7f49a400b040).stop 2024-02-15T18:23:43.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.270+0000 7f49b566c700 1 -- 172.21.15.42:0/2503995324 shutdown_connections 2024-02-15T18:23:43.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.270+0000 7f49b566c700 1 --2- 172.21.15.42:0/2503995324 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f499805c2b0 0x7f499805e770 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:43.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.270+0000 7f49b566c700 1 --2- 172.21.15.42:0/2503995324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49b00f55c0 0x7f49b00f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:43.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.270+0000 7f49b566c700 1 --2- 172.21.15.42:0/2503995324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49b00f6860 0x7f49b00f7ba0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:43.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.270+0000 7f49b566c700 1 -- 172.21.15.42:0/2503995324 >> 172.21.15.42:0/2503995324 conn(0x7f49b00f0d40 msgr2=0x7f49b005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:43.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.270+0000 7f49b566c700 1 -- 172.21.15.42:0/2503995324 shutdown_connections 2024-02-15T18:23:43.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:43.270+0000 7f49b566c700 1 -- 172.21.15.42:0/2503995324 wait complete. 2024-02-15T18:23:43.273 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 2 2024-02-15T18:23:43.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:42.485099+0000 mon.smithi042 (mon.0) 474 : cluster [DBG] osdmap e44: 6 total, 6 up, 6 in 2024-02-15T18:23:43.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:42.999980+0000 mon.smithi042 (mon.0) 475 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:43.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:43.000807+0000 mon.smithi042 (mon.0) 476 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.vtwvlo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:23:43.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:43.004627+0000 mon.smithi042 (mon.0) 477 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.vtwvlo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:23:43.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:43.005426+0000 mon.smithi042 (mon.0) 478 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:23:43.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:43.269669+0000 mon.smithi042 (mon.0) 479 : audit [DBG] from='client.? 172.21.15.42:0/2503995324' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:23:43.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:42.485099+0000 mon.smithi042 (mon.0) 474 : cluster [DBG] osdmap e44: 6 total, 6 up, 6 in 2024-02-15T18:23:43.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:42.999980+0000 mon.smithi042 (mon.0) 475 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:43.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:43.000807+0000 mon.smithi042 (mon.0) 476 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.vtwvlo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:23:43.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:43.004627+0000 mon.smithi042 (mon.0) 477 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.vtwvlo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:23:43.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:43.005426+0000 mon.smithi042 (mon.0) 478 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:23:43.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:43.269669+0000 mon.smithi042 (mon.0) 479 : audit [DBG] from='client.? 172.21.15.42:0/2503995324' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:23:44.233 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-15T18:23:44.247 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-02-15T18:23:44.247 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-02-15T18:23:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:42.818874+0000 mgr.smithi042.kfoihb (mgr.14162) 239 : cluster [DBG] pgmap v206: 65 pgs: 10 creating+peering, 14 active+clean, 41 unknown; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:23:43.006190+0000 mgr.smithi042.kfoihb (mgr.14162) 240 : cephadm [INF] Deploying daemon mds.cephfs.smithi042.vtwvlo on smithi042 2024-02-15T18:23:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:43.492288+0000 mon.smithi042 (mon.0) 480 : cluster [DBG] mds.? [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] up:boot 2024-02-15T18:23:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:43.492354+0000 mon.smithi042 (mon.0) 481 : cluster [INF] daemon mds.cephfs.smithi191.icwcnb assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-15T18:23:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:43.492422+0000 mon.smithi042 (mon.0) 482 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-15T18:23:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:43.492448+0000 mon.smithi042 (mon.0) 483 : cluster [INF] Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-15T18:23:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:43.492461+0000 mon.smithi042 (mon.0) 484 : cluster [INF] Cluster is now healthy 2024-02-15T18:23:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:43.492591+0000 mon.smithi042 (mon.0) 485 : cluster [DBG] fsmap cephfs:0 1 up:standby 2024-02-15T18:23:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:43.492784+0000 mon.smithi042 (mon.0) 486 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.icwcnb"}]: dispatch 2024-02-15T18:23:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:43.494732+0000 mon.smithi042 (mon.0) 487 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:creating} 2024-02-15T18:23:44.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:43.509165+0000 mon.smithi042 (mon.0) 488 : cluster [INF] daemon mds.cephfs.smithi191.icwcnb is now active in filesystem cephfs as rank 0 2024-02-15T18:23:44.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:42.818874+0000 mgr.smithi042.kfoihb (mgr.14162) 239 : cluster [DBG] pgmap v206: 65 pgs: 10 creating+peering, 14 active+clean, 41 unknown; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:44.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:23:43.006190+0000 mgr.smithi042.kfoihb (mgr.14162) 240 : cephadm [INF] Deploying daemon mds.cephfs.smithi042.vtwvlo on smithi042 2024-02-15T18:23:44.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:43.492288+0000 mon.smithi042 (mon.0) 480 : cluster [DBG] mds.? [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] up:boot 2024-02-15T18:23:44.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:43.492354+0000 mon.smithi042 (mon.0) 481 : cluster [INF] daemon mds.cephfs.smithi191.icwcnb assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-15T18:23:44.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:43.492422+0000 mon.smithi042 (mon.0) 482 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-15T18:23:44.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:43.492448+0000 mon.smithi042 (mon.0) 483 : cluster [INF] Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-15T18:23:44.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:43.492461+0000 mon.smithi042 (mon.0) 484 : cluster [INF] Cluster is now healthy 2024-02-15T18:23:44.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:43.492591+0000 mon.smithi042 (mon.0) 485 : cluster [DBG] fsmap cephfs:0 1 up:standby 2024-02-15T18:23:44.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:43.492784+0000 mon.smithi042 (mon.0) 486 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.icwcnb"}]: dispatch 2024-02-15T18:23:44.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:43.494732+0000 mon.smithi042 (mon.0) 487 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:creating} 2024-02-15T18:23:44.951 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:43.509165+0000 mon.smithi042 (mon.0) 488 : cluster [INF] daemon mds.cephfs.smithi191.icwcnb is now active in filesystem cephfs as rank 0 2024-02-15T18:23:45.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:44.498123+0000 mon.smithi042 (mon.0) 489 : cluster [DBG] mds.? [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] up:active 2024-02-15T18:23:45.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:44.498170+0000 mon.smithi042 (mon.0) 490 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 2024-02-15T18:23:45.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:44.498123+0000 mon.smithi042 (mon.0) 489 : cluster [DBG] mds.? [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] up:active 2024-02-15T18:23:45.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:44.498170+0000 mon.smithi042 (mon.0) 490 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 2024-02-15T18:23:46.634 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:44.819294+0000 mgr.smithi042.kfoihb (mgr.14162) 241 : cluster [DBG] pgmap v207: 65 pgs: 10 creating+peering, 44 active+clean, 11 unknown; 22 B data, 32 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:46.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:46 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:44.819294+0000 mgr.smithi042.kfoihb (mgr.14162) 241 : cluster [DBG] pgmap v207: 65 pgs: 10 creating+peering, 44 active+clean, 11 unknown; 22 B data, 32 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:23:47.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.139+0000 7ff1169db700 1 -- 172.21.15.42:0/646756438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1100f5430 msgr2=0x7ff1100f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:47.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.139+0000 7ff1169db700 1 --2- 172.21.15.42:0/646756438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1100f5430 0x7ff1100f5850 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7ff104004660 tx=0x7ff104009b30).stop 2024-02-15T18:23:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.139+0000 7ff1169db700 1 -- 172.21.15.42:0/646756438 shutdown_connections 2024-02-15T18:23:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.139+0000 7ff1169db700 1 --2- 172.21.15.42:0/646756438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1100f66d0 0x7ff1100f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.139+0000 7ff1169db700 1 --2- 172.21.15.42:0/646756438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1100f5430 0x7ff1100f5850 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.139+0000 7ff1169db700 1 -- 172.21.15.42:0/646756438 >> 172.21.15.42:0/646756438 conn(0x7ff1100f0b90 msgr2=0x7ff1100f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.140+0000 7ff1169db700 1 -- 172.21.15.42:0/646756438 shutdown_connections 2024-02-15T18:23:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.140+0000 7ff1169db700 1 -- 172.21.15.42:0/646756438 wait complete. 2024-02-15T18:23:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.140+0000 7ff1169db700 1 Processor -- start 2024-02-15T18:23:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.140+0000 7ff1169db700 1 -- start start 2024-02-15T18:23:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.140+0000 7ff1169db700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1100f5430 0x7ff110109080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.140+0000 7ff1169db700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1100f66d0 0x7ff110109570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:47.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.140+0000 7ff1169db700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff110109b20 con 0x7ff1100f66d0 2024-02-15T18:23:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.140+0000 7ff1169db700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff110109c60 con 0x7ff1100f5430 2024-02-15T18:23:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.141+0000 7ff1151d8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1100f66d0 0x7ff110109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.141+0000 7ff1151d8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1100f66d0 0x7ff110109570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34200/0 (socket says 172.21.15.42:34200) 2024-02-15T18:23:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.141+0000 7ff1151d8700 1 -- 172.21.15.42:0/2617841984 learned_addr learned my addr 172.21.15.42:0/2617841984 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.141+0000 7ff1159d9700 1 --2- 172.21.15.42:0/2617841984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1100f5430 0x7ff110109080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.141+0000 7ff1151d8700 1 -- 172.21.15.42:0/2617841984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1100f5430 msgr2=0x7ff110109080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.141+0000 7ff1151d8700 1 --2- 172.21.15.42:0/2617841984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1100f5430 0x7ff110109080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.141+0000 7ff1151d8700 1 -- 172.21.15.42:0/2617841984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff104005040 con 0x7ff1100f66d0 2024-02-15T18:23:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.141+0000 7ff1151d8700 1 --2- 172.21.15.42:0/2617841984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1100f66d0 0x7ff110109570 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7ff10c00c790 tx=0x7ff10c007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.141+0000 7ff1027fc700 1 -- 172.21.15.42:0/2617841984 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff10c002590 con 0x7ff1100f66d0 2024-02-15T18:23:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.141+0000 7ff1169db700 1 -- 172.21.15.42:0/2617841984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11010a600 con 0x7ff1100f66d0 2024-02-15T18:23:47.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.141+0000 7ff1027fc700 1 -- 172.21.15.42:0/2617841984 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff10c002bc0 con 0x7ff1100f66d0 2024-02-15T18:23:47.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.141+0000 7ff1027fc700 1 -- 172.21.15.42:0/2617841984 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff10c013e40 con 0x7ff1100f66d0 2024-02-15T18:23:47.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.141+0000 7ff1169db700 1 -- 172.21.15.42:0/2617841984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11010abc0 con 0x7ff1100f66d0 2024-02-15T18:23:47.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.142+0000 7ff1169db700 1 -- 172.21.15.42:0/2617841984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff110045ad0 con 0x7ff1100f66d0 2024-02-15T18:23:47.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.143+0000 7ff1027fc700 1 -- 172.21.15.42:0/2617841984 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7ff10c01a060 con 0x7ff1100f66d0 2024-02-15T18:23:47.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.143+0000 7ff1027fc700 1 --2- 172.21.15.42:0/2617841984 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff0fc05c300 0x7ff0fc05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:47.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.143+0000 7ff1027fc700 1 -- 172.21.15.42:0/2617841984 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7ff10c076170 con 0x7ff1100f66d0 2024-02-15T18:23:47.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.143+0000 7ff1159d9700 1 --2- 172.21.15.42:0/2617841984 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff0fc05c300 0x7ff0fc05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:47.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.143+0000 7ff1159d9700 1 --2- 172.21.15.42:0/2617841984 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff0fc05c300 0x7ff0fc05e7c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7ff104009f50 tx=0x7ff104009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:47.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.147+0000 7ff1027fc700 1 -- 172.21.15.42:0/2617841984 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff10c048e30 con 0x7ff1100f66d0 2024-02-15T18:23:47.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.348+0000 7ff1169db700 1 -- 172.21.15.42:0/2617841984 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7ff1100fde90 con 0x7ff1100f66d0 2024-02-15T18:23:47.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.508+0000 7ff1027fc700 1 -- 172.21.15.42:0/2617841984 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7ff1100fde90 con 0x7ff1100f66d0 2024-02-15T18:23:47.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.510+0000 7ff1169db700 1 -- 172.21.15.42:0/2617841984 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff0fc05c300 msgr2=0x7ff0fc05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:47.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.510+0000 7ff1169db700 1 --2- 172.21.15.42:0/2617841984 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff0fc05c300 0x7ff0fc05e7c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7ff104009f50 tx=0x7ff104009b30).stop 2024-02-15T18:23:47.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.510+0000 7ff1169db700 1 -- 172.21.15.42:0/2617841984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1100f66d0 msgr2=0x7ff110109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:47.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.510+0000 7ff1169db700 1 --2- 172.21.15.42:0/2617841984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1100f66d0 0x7ff110109570 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7ff10c00c790 tx=0x7ff10c007d70).stop 2024-02-15T18:23:47.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.510+0000 7ff1169db700 1 -- 172.21.15.42:0/2617841984 shutdown_connections 2024-02-15T18:23:47.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.510+0000 7ff1169db700 1 --2- 172.21.15.42:0/2617841984 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff0fc05c300 0x7ff0fc05e7c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:47.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.510+0000 7ff1169db700 1 --2- 172.21.15.42:0/2617841984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1100f5430 0x7ff110109080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:47.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.510+0000 7ff1169db700 1 --2- 172.21.15.42:0/2617841984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1100f66d0 0x7ff110109570 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:47.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.510+0000 7ff1169db700 1 -- 172.21.15.42:0/2617841984 >> 172.21.15.42:0/2617841984 conn(0x7ff1100f0b90 msgr2=0x7ff11005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:47.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.511+0000 7ff1169db700 1 -- 172.21.15.42:0/2617841984 shutdown_connections 2024-02-15T18:23:47.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:47.511+0000 7ff1169db700 1 -- 172.21.15.42:0/2617841984 wait complete. 2024-02-15T18:23:47.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:47.349327+0000 mon.smithi042 (mon.0) 491 : audit [INF] from='client.? 172.21.15.42:0/2617841984' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-02-15T18:23:47.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:47.414017+0000 mon.smithi042 (mon.0) 492 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:47.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:47.414782+0000 mon.smithi042 (mon.0) 493 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi191.erlpxp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:23:47.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:47.417342+0000 mon.smithi042 (mon.0) 494 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi191.erlpxp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:23:47.563 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:47.417944+0000 mon.smithi042 (mon.0) 495 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:23:47.874 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:47.349327+0000 mon.smithi042 (mon.0) 491 : audit [INF] from='client.? 172.21.15.42:0/2617841984' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-02-15T18:23:47.875 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:47.414017+0000 mon.smithi042 (mon.0) 492 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:47.875 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:47.414782+0000 mon.smithi042 (mon.0) 493 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi191.erlpxp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:23:47.875 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:47.417342+0000 mon.smithi042 (mon.0) 494 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi191.erlpxp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:23:47.875 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:47.417944+0000 mon.smithi042 (mon.0) 495 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:23:48.165 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-15T18:23:48.175 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-02-15T18:23:48.175 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-02-15T18:23:48.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:46.819693+0000 mgr.smithi042.kfoihb (mgr.14162) 242 : cluster [DBG] pgmap v208: 65 pgs: 65 active+clean; 770 B data, 32 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s wr, 4 op/s 2024-02-15T18:23:48.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:23:47.418462+0000 mgr.smithi042.kfoihb (mgr.14162) 243 : cephadm [INF] Deploying daemon mds.cephfs.smithi191.erlpxp on smithi191 2024-02-15T18:23:48.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:47.509730+0000 mon.smithi042 (mon.0) 496 : audit [INF] from='client.? 172.21.15.42:0/2617841984' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-02-15T18:23:48.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:47.509812+0000 mon.smithi042 (mon.0) 497 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 2024-02-15T18:23:48.927 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:46.819693+0000 mgr.smithi042.kfoihb (mgr.14162) 242 : cluster [DBG] pgmap v208: 65 pgs: 65 active+clean; 770 B data, 32 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s wr, 4 op/s 2024-02-15T18:23:48.927 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:23:47.418462+0000 mgr.smithi042.kfoihb (mgr.14162) 243 : cephadm [INF] Deploying daemon mds.cephfs.smithi191.erlpxp on smithi191 2024-02-15T18:23:48.927 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:47.509730+0000 mon.smithi042 (mon.0) 496 : audit [INF] from='client.? 172.21.15.42:0/2617841984' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-02-15T18:23:48.927 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:47.509812+0000 mon.smithi042 (mon.0) 497 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 2024-02-15T18:23:50.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.105+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/3322652183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c280f7980 msgr2=0x7f6c280f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:50.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.105+0000 7f6c2d1ce700 1 --2- 172.21.15.42:0/3322652183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c280f7980 0x7f6c280f7de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f6c1c004660 tx=0x7f6c1c009b30).stop 2024-02-15T18:23:50.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.105+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/3322652183 shutdown_connections 2024-02-15T18:23:50.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.105+0000 7f6c2d1ce700 1 --2- 172.21.15.42:0/3322652183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c280f7980 0x7f6c280f7de0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:50.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.105+0000 7f6c2d1ce700 1 --2- 172.21.15.42:0/3322652183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c280f66e0 0x7f6c280f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:50.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.105+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/3322652183 >> 172.21.15.42:0/3322652183 conn(0x7f6c280f1e40 msgr2=0x7f6c280f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:50.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.105+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/3322652183 shutdown_connections 2024-02-15T18:23:50.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.105+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/3322652183 wait complete. 2024-02-15T18:23:50.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.106+0000 7f6c2d1ce700 1 Processor -- start 2024-02-15T18:23:50.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.106+0000 7f6c2d1ce700 1 -- start start 2024-02-15T18:23:50.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.106+0000 7f6c2d1ce700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c280f66e0 0x7f6c28107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:50.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.106+0000 7f6c2d1ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c280f7980 0x7f6c281074f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:50.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.106+0000 7f6c2d1ce700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c28107aa0 con 0x7f6c280f7980 2024-02-15T18:23:50.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.106+0000 7f6c2d1ce700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c28107be0 con 0x7f6c280f66e0 2024-02-15T18:23:50.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.106+0000 7f6c2659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c280f7980 0x7f6c281074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:50.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.107+0000 7f6c26d9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c280f66e0 0x7f6c28107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:50.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.107+0000 7f6c2659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c280f7980 0x7f6c281074f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34252/0 (socket says 172.21.15.42:34252) 2024-02-15T18:23:50.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.107+0000 7f6c2659c700 1 -- 172.21.15.42:0/2170954661 learned_addr learned my addr 172.21.15.42:0/2170954661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:50.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.107+0000 7f6c2659c700 1 -- 172.21.15.42:0/2170954661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c280f66e0 msgr2=0x7f6c28107000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:50.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.107+0000 7f6c2659c700 1 --2- 172.21.15.42:0/2170954661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c280f66e0 0x7f6c28107000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:50.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.107+0000 7f6c2659c700 1 -- 172.21.15.42:0/2170954661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c005040 con 0x7f6c280f7980 2024-02-15T18:23:50.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.107+0000 7f6c2659c700 1 --2- 172.21.15.42:0/2170954661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c280f7980 0x7f6c281074f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f6c1c000c00 tx=0x7f6c1c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:50.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.107+0000 7f6c0f7fe700 1 -- 172.21.15.42:0/2170954661 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c1c019070 con 0x7f6c280f7980 2024-02-15T18:23:50.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.107+0000 7f6c0f7fe700 1 -- 172.21.15.42:0/2170954661 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c1c007e90 con 0x7f6c280f7980 2024-02-15T18:23:50.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.107+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/2170954661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c28108520 con 0x7f6c280f7980 2024-02-15T18:23:50.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.107+0000 7f6c0f7fe700 1 -- 172.21.15.42:0/2170954661 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c1c014d50 con 0x7f6c280f7980 2024-02-15T18:23:50.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.107+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/2170954661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c28108a80 con 0x7f6c280f7980 2024-02-15T18:23:50.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.108+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/2170954661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c28045ad0 con 0x7f6c280f7980 2024-02-15T18:23:50.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.109+0000 7f6c0f7fe700 1 -- 172.21.15.42:0/2170954661 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f6c1c025070 con 0x7f6c280f7980 2024-02-15T18:23:50.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.109+0000 7f6c0f7fe700 1 --2- 172.21.15.42:0/2170954661 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f6c1005c2b0 0x7f6c1005e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:50.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.109+0000 7f6c0f7fe700 1 -- 172.21.15.42:0/2170954661 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f6c1c076de0 con 0x7f6c280f7980 2024-02-15T18:23:50.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.109+0000 7f6c26d9d700 1 --2- 172.21.15.42:0/2170954661 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f6c1005c2b0 0x7f6c1005e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:50.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.110+0000 7f6c26d9d700 1 --2- 172.21.15.42:0/2170954661 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f6c1005c2b0 0x7f6c1005e770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f6c18009fd0 tx=0x7f6c1800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:50.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.114+0000 7f6c0f7fe700 1 -- 172.21.15.42:0/2170954661 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6c1c049aa0 con 0x7f6c280f7980 2024-02-15T18:23:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.321+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/2170954661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f6c28027a00 con 0x7f6c280f7980 2024-02-15T18:23:50.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.512+0000 7f6c0f7fe700 1 -- 172.21.15.42:0/2170954661 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v7) v1 ==== 121+0+0 (secure 0 0 0) 0x7f6c1c04d0c0 con 0x7f6c280f7980 2024-02-15T18:23:50.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.514+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/2170954661 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f6c1005c2b0 msgr2=0x7f6c1005e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:50.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.514+0000 7f6c2d1ce700 1 --2- 172.21.15.42:0/2170954661 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f6c1005c2b0 0x7f6c1005e770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f6c18009fd0 tx=0x7f6c1800b040).stop 2024-02-15T18:23:50.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.514+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/2170954661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c280f7980 msgr2=0x7f6c281074f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:50.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.514+0000 7f6c2d1ce700 1 --2- 172.21.15.42:0/2170954661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c280f7980 0x7f6c281074f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f6c1c000c00 tx=0x7f6c1c00b040).stop 2024-02-15T18:23:50.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.514+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/2170954661 shutdown_connections 2024-02-15T18:23:50.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.514+0000 7f6c2d1ce700 1 --2- 172.21.15.42:0/2170954661 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f6c1005c2b0 0x7f6c1005e770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:50.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.515+0000 7f6c2d1ce700 1 --2- 172.21.15.42:0/2170954661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c280f66e0 0x7f6c28107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:50.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.515+0000 7f6c2d1ce700 1 --2- 172.21.15.42:0/2170954661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c280f7980 0x7f6c281074f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:50.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.515+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/2170954661 >> 172.21.15.42:0/2170954661 conn(0x7f6c280f1e40 msgr2=0x7f6c280fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:50.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.515+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/2170954661 shutdown_connections 2024-02-15T18:23:50.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:50.515+0000 7f6c2d1ce700 1 -- 172.21.15.42:0/2170954661 wait complete. 2024-02-15T18:23:50.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:48.820335+0000 mgr.smithi042.kfoihb (mgr.14162) 244 : cluster [DBG] pgmap v209: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 3.0 KiB/s wr, 5 op/s 2024-02-15T18:23:50.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:49.898046+0000 mon.smithi042 (mon.0) 498 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:50.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:49.898779+0000 mon.smithi042 (mon.0) 499 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.idnxkc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:23:50.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:49.901881+0000 mon.smithi042 (mon.0) 500 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.idnxkc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:23:50.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:49.902648+0000 mon.smithi042 (mon.0) 501 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:23:50.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:50.322262+0000 mon.smithi042 (mon.0) 502 : audit [INF] from='client.? 172.21.15.42:0/2170954661' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-15T18:23:50.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:48.820335+0000 mgr.smithi042.kfoihb (mgr.14162) 244 : cluster [DBG] pgmap v209: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 3.0 KiB/s wr, 5 op/s 2024-02-15T18:23:50.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:49.898046+0000 mon.smithi042 (mon.0) 498 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:50.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:49.898779+0000 mon.smithi042 (mon.0) 499 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.idnxkc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:23:50.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:49.901881+0000 mon.smithi042 (mon.0) 500 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.idnxkc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-15T18:23:50.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:49.902648+0000 mon.smithi042 (mon.0) 501 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:23:50.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:50.322262+0000 mon.smithi042 (mon.0) 502 : audit [INF] from='client.? 172.21.15.42:0/2170954661' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-15T18:23:51.660 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:23:49.903261+0000 mgr.smithi042.kfoihb (mgr.14162) 245 : cephadm [INF] Deploying daemon mds.cephfs.smithi042.idnxkc on smithi042 2024-02-15T18:23:51.660 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:50.513518+0000 mon.smithi042 (mon.0) 503 : cluster [DBG] mds.? [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] up:boot 2024-02-15T18:23:51.660 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:50.513574+0000 mon.smithi042 (mon.0) 504 : audit [INF] from='client.? 172.21.15.42:0/2170954661' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-15T18:23:51.660 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:50.513920+0000 mon.smithi042 (mon.0) 505 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby 2024-02-15T18:23:51.660 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:50.514050+0000 mon.smithi042 (mon.0) 506 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.erlpxp"}]: dispatch 2024-02-15T18:23:51.661 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:50.518472+0000 mon.smithi042 (mon.0) 507 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 2024-02-15T18:23:51.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:23:49.903261+0000 mgr.smithi042.kfoihb (mgr.14162) 245 : cephadm [INF] Deploying daemon mds.cephfs.smithi042.idnxkc on smithi042 2024-02-15T18:23:51.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:50.513518+0000 mon.smithi042 (mon.0) 503 : cluster [DBG] mds.? [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] up:boot 2024-02-15T18:23:51.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:50.513574+0000 mon.smithi042 (mon.0) 504 : audit [INF] from='client.? 172.21.15.42:0/2170954661' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-15T18:23:51.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:50.513920+0000 mon.smithi042 (mon.0) 505 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby 2024-02-15T18:23:51.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:50.514050+0000 mon.smithi042 (mon.0) 506 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.erlpxp"}]: dispatch 2024-02-15T18:23:51.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:50.518472+0000 mon.smithi042 (mon.0) 507 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 2024-02-15T18:23:52.355 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-15T18:23:52.365 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-02-15T18:23:52.366 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-02-15T18:23:52.693 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:50.820917+0000 mgr.smithi042.kfoihb (mgr.14162) 246 : cluster [DBG] pgmap v210: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s wr, 3 op/s 2024-02-15T18:23:52.694 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:51.522579+0000 mon.smithi042 (mon.0) 508 : cluster [DBG] mds.? [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] up:boot 2024-02-15T18:23:52.694 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:51.522640+0000 mon.smithi042 (mon.0) 509 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 1 up:standby 2024-02-15T18:23:52.694 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:51.522737+0000 mon.smithi042 (mon.0) 510 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.vtwvlo"}]: dispatch 2024-02-15T18:23:52.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:50.820917+0000 mgr.smithi042.kfoihb (mgr.14162) 246 : cluster [DBG] pgmap v210: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s wr, 3 op/s 2024-02-15T18:23:52.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:51.522579+0000 mon.smithi042 (mon.0) 508 : cluster [DBG] mds.? [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] up:boot 2024-02-15T18:23:52.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:51.522640+0000 mon.smithi042 (mon.0) 509 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 1 up:standby 2024-02-15T18:23:52.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:51.522737+0000 mon.smithi042 (mon.0) 510 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.vtwvlo"}]: dispatch 2024-02-15T18:23:54.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:52.821292+0000 mgr.smithi042.kfoihb (mgr.14162) 247 : cluster [DBG] pgmap v211: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 2 op/s 2024-02-15T18:23:54.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:53.027273+0000 mon.smithi042 (mon.0) 511 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:54.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:53.027951+0000 mon.smithi042 (mon.0) 512 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:23:54.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:52.821292+0000 mgr.smithi042.kfoihb (mgr.14162) 247 : cluster [DBG] pgmap v211: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s wr, 2 op/s 2024-02-15T18:23:54.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:53.027273+0000 mon.smithi042 (mon.0) 511 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:54.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:53.027951+0000 mon.smithi042 (mon.0) 512 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:23:54.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.564+0000 7f0b042f4700 1 -- 172.21.15.42:0/1719094374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0f7980 msgr2=0x7f0afc0f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:54.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.564+0000 7f0b042f4700 1 --2- 172.21.15.42:0/1719094374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0f7980 0x7f0afc0f7de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f0aec004660 tx=0x7f0aec009b30).stop 2024-02-15T18:23:54.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.564+0000 7f0b042f4700 1 -- 172.21.15.42:0/1719094374 shutdown_connections 2024-02-15T18:23:54.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.564+0000 7f0b042f4700 1 --2- 172.21.15.42:0/1719094374 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0f7980 0x7f0afc0f7de0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:54.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.564+0000 7f0b042f4700 1 --2- 172.21.15.42:0/1719094374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0afc0f66e0 0x7f0afc0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:54.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.564+0000 7f0b042f4700 1 -- 172.21.15.42:0/1719094374 >> 172.21.15.42:0/1719094374 conn(0x7f0afc0f1e40 msgr2=0x7f0afc0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:54.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.564+0000 7f0b042f4700 1 -- 172.21.15.42:0/1719094374 shutdown_connections 2024-02-15T18:23:54.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.564+0000 7f0b042f4700 1 -- 172.21.15.42:0/1719094374 wait complete. 2024-02-15T18:23:54.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.564+0000 7f0b042f4700 1 Processor -- start 2024-02-15T18:23:54.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.564+0000 7f0b042f4700 1 -- start start 2024-02-15T18:23:54.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.564+0000 7f0b042f4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0afc0f66e0 0x7f0afc107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:54.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.564+0000 7f0b042f4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0f7980 0x7f0afc1074f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:54.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.564+0000 7f0b042f4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc107aa0 con 0x7f0afc0f7980 2024-02-15T18:23:54.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.564+0000 7f0b042f4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc107be0 con 0x7f0afc0f66e0 2024-02-15T18:23:54.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.565+0000 7f0b0188f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0f7980 0x7f0afc1074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:54.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.565+0000 7f0b0188f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0f7980 0x7f0afc1074f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60300/0 (socket says 172.21.15.42:60300) 2024-02-15T18:23:54.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.565+0000 7f0b0188f700 1 -- 172.21.15.42:0/2181540860 learned_addr learned my addr 172.21.15.42:0/2181540860 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:54.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.565+0000 7f0b02090700 1 --2- 172.21.15.42:0/2181540860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0afc0f66e0 0x7f0afc107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:54.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.565+0000 7f0b0188f700 1 -- 172.21.15.42:0/2181540860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0afc0f66e0 msgr2=0x7f0afc107000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.565+0000 7f0b0188f700 1 --2- 172.21.15.42:0/2181540860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0afc0f66e0 0x7f0afc107000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.565+0000 7f0b0188f700 1 -- 172.21.15.42:0/2181540860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aec005040 con 0x7f0afc0f7980 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.565+0000 7f0b0188f700 1 --2- 172.21.15.42:0/2181540860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0f7980 0x7f0afc1074f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f0aec000c00 tx=0x7f0aec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.565+0000 7f0af2ffd700 1 -- 172.21.15.42:0/2181540860 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0aec019070 con 0x7f0afc0f7980 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.565+0000 7f0b042f4700 1 -- 172.21.15.42:0/2181540860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0afc108520 con 0x7f0afc0f7980 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.565+0000 7f0af2ffd700 1 -- 172.21.15.42:0/2181540860 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0aec007e90 con 0x7f0afc0f7980 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.565+0000 7f0af2ffd700 1 -- 172.21.15.42:0/2181540860 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0aec014d50 con 0x7f0afc0f7980 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.565+0000 7f0b042f4700 1 -- 172.21.15.42:0/2181540860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0afc108a80 con 0x7f0afc0f7980 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.566+0000 7f0b042f4700 1 -- 172.21.15.42:0/2181540860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0afc045ad0 con 0x7f0afc0f7980 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.567+0000 7f0af2ffd700 1 -- 172.21.15.42:0/2181540860 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f0aec025070 con 0x7f0afc0f7980 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.567+0000 7f0af2ffd700 1 --2- 172.21.15.42:0/2181540860 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0ae805c2b0 0x7f0ae805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.567+0000 7f0af2ffd700 1 -- 172.21.15.42:0/2181540860 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f0aec076e60 con 0x7f0afc0f7980 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.567+0000 7f0b02090700 1 --2- 172.21.15.42:0/2181540860 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0ae805c2b0 0x7f0ae805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.567+0000 7f0b02090700 1 --2- 172.21.15.42:0/2181540860 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0ae805c2b0 0x7f0ae805e770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f0af8009fd0 tx=0x7f0af800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:54.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.571+0000 7f0af2ffd700 1 -- 172.21.15.42:0/2181540860 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0aec049b20 con 0x7f0afc0f7980 2024-02-15T18:23:54.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:54.772+0000 7f0b042f4700 1 -- 172.21.15.42:0/2181540860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f0afc027a00 con 0x7f0afc0f7980 2024-02-15T18:23:55.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:55.528+0000 7f0af2ffd700 1 -- 172.21.15.42:0/2181540860 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v11) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0aec04d140 con 0x7f0afc0f7980 2024-02-15T18:23:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:55.530+0000 7f0b042f4700 1 -- 172.21.15.42:0/2181540860 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0ae805c2b0 msgr2=0x7f0ae805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:55.530+0000 7f0b042f4700 1 --2- 172.21.15.42:0/2181540860 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0ae805c2b0 0x7f0ae805e770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f0af8009fd0 tx=0x7f0af800b040).stop 2024-02-15T18:23:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:55.530+0000 7f0b042f4700 1 -- 172.21.15.42:0/2181540860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0f7980 msgr2=0x7f0afc1074f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:55.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:55.530+0000 7f0b042f4700 1 --2- 172.21.15.42:0/2181540860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0f7980 0x7f0afc1074f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f0aec000c00 tx=0x7f0aec00b040).stop 2024-02-15T18:23:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:55.530+0000 7f0b042f4700 1 -- 172.21.15.42:0/2181540860 shutdown_connections 2024-02-15T18:23:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:55.530+0000 7f0b042f4700 1 --2- 172.21.15.42:0/2181540860 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0ae805c2b0 0x7f0ae805e770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:55.530+0000 7f0b042f4700 1 --2- 172.21.15.42:0/2181540860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0afc0f66e0 0x7f0afc107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:55.530+0000 7f0b042f4700 1 --2- 172.21.15.42:0/2181540860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0f7980 0x7f0afc1074f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:55.530+0000 7f0b042f4700 1 -- 172.21.15.42:0/2181540860 >> 172.21.15.42:0/2181540860 conn(0x7f0afc0f1e40 msgr2=0x7f0afc0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:55.530+0000 7f0b042f4700 1 -- 172.21.15.42:0/2181540860 shutdown_connections 2024-02-15T18:23:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:55.530+0000 7f0b042f4700 1 -- 172.21.15.42:0/2181540860 wait complete. 2024-02-15T18:23:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:inline data disabled 2024-02-15T18:23:55.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:54.526295+0000 mon.smithi042 (mon.0) 513 : cluster [DBG] mds.? [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] up:standby-replay 2024-02-15T18:23:55.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:54.526366+0000 mon.smithi042 (mon.0) 514 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 1 up:standby 2024-02-15T18:23:55.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:54.774331+0000 mon.smithi042 (mon.0) 515 : audit [INF] from='client.? 172.21.15.42:0/2181540860' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-02-15T18:23:55.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:55.138500+0000 mon.smithi042 (mon.0) 516 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:55.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:54.526295+0000 mon.smithi042 (mon.0) 513 : cluster [DBG] mds.? [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] up:standby-replay 2024-02-15T18:23:55.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:54.526366+0000 mon.smithi042 (mon.0) 514 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 1 up:standby 2024-02-15T18:23:55.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:54.774331+0000 mon.smithi042 (mon.0) 515 : audit [INF] from='client.? 172.21.15.42:0/2181540860' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-02-15T18:23:55.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:55.138500+0000 mon.smithi042 (mon.0) 516 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:23:56.705 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-15T18:23:56.716 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-02-15T18:23:56.717 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-15T18:23:56.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:54.821897+0000 mgr.smithi042.kfoihb (mgr.14162) 248 : cluster [DBG] pgmap v212: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 818 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-15T18:23:56.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:55.530667+0000 mon.smithi042 (mon.0) 517 : audit [INF] from='client.? 172.21.15.42:0/2181540860' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-02-15T18:23:56.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:55.530738+0000 mon.smithi042 (mon.0) 518 : cluster [DBG] mds.? [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] up:standby 2024-02-15T18:23:56.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:55.530801+0000 mon.smithi042 (mon.0) 519 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 1 up:standby 2024-02-15T18:23:56.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:54.821897+0000 mgr.smithi042.kfoihb (mgr.14162) 248 : cluster [DBG] pgmap v212: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 818 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-15T18:23:56.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:55.530667+0000 mon.smithi042 (mon.0) 517 : audit [INF] from='client.? 172.21.15.42:0/2181540860' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-02-15T18:23:56.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:55.530738+0000 mon.smithi042 (mon.0) 518 : cluster [DBG] mds.? [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] up:standby 2024-02-15T18:23:56.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:55.530801+0000 mon.smithi042 (mon.0) 519 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 1 up:standby 2024-02-15T18:23:57.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:56.919799+0000 mon.smithi042 (mon.0) 520 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:23:57.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:56.932553+0000 mon.smithi042 (mon.0) 521 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:23:57.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:56.919799+0000 mon.smithi042 (mon.0) 520 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:23:57.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:56.932553+0000 mon.smithi042 (mon.0) 521 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:23:58.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:56.822252+0000 mgr.smithi042.kfoihb (mgr.14162) 249 : cluster [DBG] pgmap v213: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 1023 B/s wr, 3 op/s 2024-02-15T18:23:58.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:57.535921+0000 mon.smithi042 (mon.0) 522 : cluster [DBG] mds.? [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] up:boot 2024-02-15T18:23:58.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:57.536001+0000 mon.smithi042 (mon.0) 523 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 2 up:standby 2024-02-15T18:23:58.813 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:23:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:57.536250+0000 mon.smithi042 (mon.0) 524 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.idnxkc"}]: dispatch 2024-02-15T18:23:58.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:56.822252+0000 mgr.smithi042.kfoihb (mgr.14162) 249 : cluster [DBG] pgmap v213: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 1023 B/s wr, 3 op/s 2024-02-15T18:23:58.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:57.535921+0000 mon.smithi042 (mon.0) 522 : cluster [DBG] mds.? [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] up:boot 2024-02-15T18:23:58.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:57.536001+0000 mon.smithi042 (mon.0) 523 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 2 up:standby 2024-02-15T18:23:58.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:23:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:57.536250+0000 mon.smithi042 (mon.0) 524 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.idnxkc"}]: dispatch 2024-02-15T18:23:59.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.236+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1135553247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a480f6530 msgr2=0x7f5a480f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:59.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.236+0000 7f5a4e9bd700 1 --2- 172.21.15.42:0/1135553247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a480f6530 0x7f5a480f6950 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f5a44004660 tx=0x7f5a44009b30).stop 2024-02-15T18:23:59.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.236+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1135553247 shutdown_connections 2024-02-15T18:23:59.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.237+0000 7f5a4e9bd700 1 --2- 172.21.15.42:0/1135553247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a480f77d0 0x7f5a480f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:59.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.237+0000 7f5a4e9bd700 1 --2- 172.21.15.42:0/1135553247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a480f6530 0x7f5a480f6950 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:59.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.237+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1135553247 >> 172.21.15.42:0/1135553247 conn(0x7f5a480f1c90 msgr2=0x7f5a480f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:59.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.237+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1135553247 shutdown_connections 2024-02-15T18:23:59.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.237+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1135553247 wait complete. 2024-02-15T18:23:59.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.237+0000 7f5a4e9bd700 1 Processor -- start 2024-02-15T18:23:59.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.237+0000 7f5a4e9bd700 1 -- start start 2024-02-15T18:23:59.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.237+0000 7f5a4e9bd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a480f6530 0x7f5a48106ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:59.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.237+0000 7f5a4e9bd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a480f77d0 0x7f5a48107390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:59.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.237+0000 7f5a4e9bd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48107940 con 0x7f5a480f77d0 2024-02-15T18:23:59.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.237+0000 7f5a4e9bd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48107a80 con 0x7f5a480f6530 2024-02-15T18:23:59.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.237+0000 7f5a4d1ba700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a480f77d0 0x7f5a48107390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:59.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.238+0000 7f5a4d1ba700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a480f77d0 0x7f5a48107390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60330/0 (socket says 172.21.15.42:60330) 2024-02-15T18:23:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.238+0000 7f5a4d1ba700 1 -- 172.21.15.42:0/1101654660 learned_addr learned my addr 172.21.15.42:0/1101654660 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:23:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.238+0000 7f5a4d9bb700 1 --2- 172.21.15.42:0/1101654660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a480f6530 0x7f5a48106ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.238+0000 7f5a4d1ba700 1 -- 172.21.15.42:0/1101654660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a480f6530 msgr2=0x7f5a48106ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.238+0000 7f5a4d1ba700 1 --2- 172.21.15.42:0/1101654660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a480f6530 0x7f5a48106ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.238+0000 7f5a4d1ba700 1 -- 172.21.15.42:0/1101654660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a44005040 con 0x7f5a480f77d0 2024-02-15T18:23:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.238+0000 7f5a4d1ba700 1 --2- 172.21.15.42:0/1101654660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a480f77d0 0x7f5a48107390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f5a380026d0 tx=0x7f5a38009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.238+0000 7f5a3e7fc700 1 -- 172.21.15.42:0/1101654660 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a380079c0 con 0x7f5a480f77d0 2024-02-15T18:23:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.238+0000 7f5a3e7fc700 1 -- 172.21.15.42:0/1101654660 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a3800f040 con 0x7f5a480f77d0 2024-02-15T18:23:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.238+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1101654660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a48108420 con 0x7f5a480f77d0 2024-02-15T18:23:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.238+0000 7f5a3e7fc700 1 -- 172.21.15.42:0/1101654660 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a38014980 con 0x7f5a480f77d0 2024-02-15T18:23:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.238+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1101654660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a481089e0 con 0x7f5a480f77d0 2024-02-15T18:23:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.239+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1101654660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a48045ad0 con 0x7f5a480f77d0 2024-02-15T18:23:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.240+0000 7f5a3e7fc700 1 -- 172.21.15.42:0/1101654660 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f5a380090d0 con 0x7f5a480f77d0 2024-02-15T18:23:59.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.240+0000 7f5a3e7fc700 1 --2- 172.21.15.42:0/1101654660 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5a3405c300 0x7f5a3405e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:23:59.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.240+0000 7f5a3e7fc700 1 -- 172.21.15.42:0/1101654660 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f5a38075ed0 con 0x7f5a480f77d0 2024-02-15T18:23:59.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.240+0000 7f5a4d9bb700 1 --2- 172.21.15.42:0/1101654660 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5a3405c300 0x7f5a3405e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:23:59.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.240+0000 7f5a4d9bb700 1 --2- 172.21.15.42:0/1101654660 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5a3405c300 0x7f5a3405e7c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f5a44004630 tx=0x7f5a44009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:23:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.244+0000 7f5a3e7fc700 1 -- 172.21.15.42:0/1101654660 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5a38048b90 con 0x7f5a480f77d0 2024-02-15T18:23:59.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.443+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1101654660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5a48027a00 con 0x7f5a480f77d0 2024-02-15T18:23:59.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.443+0000 7f5a3e7fc700 1 -- 172.21.15.42:0/1101654660 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 12 v12) v1 ==== 76+0+1613 (secure 0 0 0) 0x7f5a3804c1b0 con 0x7f5a480f77d0 2024-02-15T18:23:59.443 INFO:teuthology.orchestra.run.smithi042.stdout:e12 2024-02-15T18:23:59.443 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:23:59.443 INFO:teuthology.orchestra.run.smithi042.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:23:59.443 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:23:59.443 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:23:59.443 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:23:59.443 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:23:59.443 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:23:59.444 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:23:59.444 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:23:59.444 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:23:59.444 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:23:59.444 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:23:59.444 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:23:59.444 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:23:59.444 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:23:59.444 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:23:59.444 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:23:59.444 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:23:59.444 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:23:59.444 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:23:59.444 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000]] 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509]] 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:23:59.445 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:23:59.446 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366]] 2024-02-15T18:23:59.446 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 2 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796]] 2024-02-15T18:23:59.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.445+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1101654660 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5a3405c300 msgr2=0x7f5a3405e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:59.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.445+0000 7f5a4e9bd700 1 --2- 172.21.15.42:0/1101654660 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5a3405c300 0x7f5a3405e7c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f5a44004630 tx=0x7f5a44009b30).stop 2024-02-15T18:23:59.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.445+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1101654660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a480f77d0 msgr2=0x7f5a48107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:23:59.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.445+0000 7f5a4e9bd700 1 --2- 172.21.15.42:0/1101654660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a480f77d0 0x7f5a48107390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f5a380026d0 tx=0x7f5a38009de0).stop 2024-02-15T18:23:59.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.445+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1101654660 shutdown_connections 2024-02-15T18:23:59.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.445+0000 7f5a4e9bd700 1 --2- 172.21.15.42:0/1101654660 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f5a3405c300 0x7f5a3405e7c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:59.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.445+0000 7f5a4e9bd700 1 --2- 172.21.15.42:0/1101654660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a480f6530 0x7f5a48106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:59.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.445+0000 7f5a4e9bd700 1 --2- 172.21.15.42:0/1101654660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a480f77d0 0x7f5a48107390 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:23:59.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.445+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1101654660 >> 172.21.15.42:0/1101654660 conn(0x7f5a480f1c90 msgr2=0x7f5a480fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:23:59.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.445+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1101654660 shutdown_connections 2024-02-15T18:23:59.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:23:59.445+0000 7f5a4e9bd700 1 -- 172.21.15.42:0/1101654660 wait complete. 2024-02-15T18:23:59.447 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 12 2024-02-15T18:24:00.429 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:23:58.822733+0000 mgr.smithi042.kfoihb (mgr.14162) 250 : cluster [DBG] pgmap v214: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 3 op/s 2024-02-15T18:24:00.429 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:23:59.444287+0000 mon.smithi042 (mon.0) 525 : audit [DBG] from='client.? 172.21.15.42:0/1101654660' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:24:00.454 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-02-15T18:24:00.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:23:58.822733+0000 mgr.smithi042.kfoihb (mgr.14162) 250 : cluster [DBG] pgmap v214: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 3 op/s 2024-02-15T18:24:00.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:23:59.444287+0000 mon.smithi042 (mon.0) 525 : audit [DBG] from='client.? 172.21.15.42:0/1101654660' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:24:02.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:00.823339+0000 mgr.smithi042.kfoihb (mgr.14162) 251 : cluster [DBG] pgmap v215: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-15T18:24:02.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:01.295923+0000 mon.smithi042 (mon.0) 526 : cluster [DBG] mds.? [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] up:standby 2024-02-15T18:24:02.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:01.295991+0000 mon.smithi042 (mon.0) 527 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 2 up:standby 2024-02-15T18:24:02.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:00.823339+0000 mgr.smithi042.kfoihb (mgr.14162) 251 : cluster [DBG] pgmap v215: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-15T18:24:02.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:01.295923+0000 mon.smithi042 (mon.0) 526 : cluster [DBG] mds.? [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] up:standby 2024-02-15T18:24:02.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:01.295991+0000 mon.smithi042 (mon.0) 527 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 2 up:standby 2024-02-15T18:24:02.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.885+0000 7f7d15465700 1 -- 172.21.15.42:0/3092676000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d100f6b70 msgr2=0x7f7d100f6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:02.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.885+0000 7f7d15465700 1 --2- 172.21.15.42:0/3092676000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d100f6b70 0x7f7d100f6f90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f7d00004660 tx=0x7f7d00009b30).stop 2024-02-15T18:24:02.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.885+0000 7f7d15465700 1 -- 172.21.15.42:0/3092676000 shutdown_connections 2024-02-15T18:24:02.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.885+0000 7f7d15465700 1 --2- 172.21.15.42:0/3092676000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d100f7510 0x7f7d100f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:02.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.885+0000 7f7d15465700 1 --2- 172.21.15.42:0/3092676000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d100f6b70 0x7f7d100f6f90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:02.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.885+0000 7f7d15465700 1 -- 172.21.15.42:0/3092676000 >> 172.21.15.42:0/3092676000 conn(0x7f7d100f0470 msgr2=0x7f7d100f2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:02.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.885+0000 7f7d15465700 1 -- 172.21.15.42:0/3092676000 shutdown_connections 2024-02-15T18:24:02.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.885+0000 7f7d15465700 1 -- 172.21.15.42:0/3092676000 wait complete. 2024-02-15T18:24:02.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.885+0000 7f7d15465700 1 Processor -- start 2024-02-15T18:24:02.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.885+0000 7f7d15465700 1 -- start start 2024-02-15T18:24:02.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d15465700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d100f6b70 0x7f7d1010c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:02.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d15465700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d100f7510 0x7f7d1010e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:02.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d15465700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100f54e0 con 0x7f7d100f7510 2024-02-15T18:24:02.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d15465700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100f5620 con 0x7f7d100f6b70 2024-02-15T18:24:02.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d0e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d100f7510 0x7f7d1010e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:02.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d0e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d100f7510 0x7f7d1010e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59806/0 (socket says 172.21.15.42:59806) 2024-02-15T18:24:02.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d0e7fc700 1 -- 172.21.15.42:0/194920117 learned_addr learned my addr 172.21.15.42:0/194920117 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:02.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d0effd700 1 --2- 172.21.15.42:0/194920117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d100f6b70 0x7f7d1010c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:02.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d0e7fc700 1 -- 172.21.15.42:0/194920117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d100f6b70 msgr2=0x7f7d1010c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:02.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d0e7fc700 1 --2- 172.21.15.42:0/194920117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d100f6b70 0x7f7d1010c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:02.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d0e7fc700 1 -- 172.21.15.42:0/194920117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d00005040 con 0x7f7d100f7510 2024-02-15T18:24:02.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d0e7fc700 1 --2- 172.21.15.42:0/194920117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d100f7510 0x7f7d1010e5e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f7cf800c510 tx=0x7f7cf8007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:02.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d077fe700 1 -- 172.21.15.42:0/194920117 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cf800f650 con 0x7f7d100f7510 2024-02-15T18:24:02.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d077fe700 1 -- 172.21.15.42:0/194920117 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cf800fc80 con 0x7f7d100f7510 2024-02-15T18:24:02.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d15465700 1 -- 172.21.15.42:0/194920117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1010ead0 con 0x7f7d100f7510 2024-02-15T18:24:02.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.886+0000 7f7d077fe700 1 -- 172.21.15.42:0/194920117 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cf8012d10 con 0x7f7d100f7510 2024-02-15T18:24:02.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.887+0000 7f7d15465700 1 -- 172.21.15.42:0/194920117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1010ef50 con 0x7f7d100f7510 2024-02-15T18:24:02.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.887+0000 7f7d15465700 1 -- 172.21.15.42:0/194920117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d10045ad0 con 0x7f7d100f7510 2024-02-15T18:24:02.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.888+0000 7f7d077fe700 1 -- 172.21.15.42:0/194920117 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f7cf8010070 con 0x7f7d100f7510 2024-02-15T18:24:02.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.888+0000 7f7d077fe700 1 --2- 172.21.15.42:0/194920117 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7cfc05c300 0x7f7cfc05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:02.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.888+0000 7f7d077fe700 1 -- 172.21.15.42:0/194920117 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f7cf8075b30 con 0x7f7d100f7510 2024-02-15T18:24:02.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.888+0000 7f7d0effd700 1 --2- 172.21.15.42:0/194920117 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7cfc05c300 0x7f7cfc05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:02.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.889+0000 7f7d0effd700 1 --2- 172.21.15.42:0/194920117 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7cfc05c300 0x7f7cfc05e7c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f7d100f5270 tx=0x7f7d00009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:02.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:02.892+0000 7f7d077fe700 1 -- 172.21.15.42:0/194920117 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7cf80452b0 con 0x7f7d100f7510 2024-02-15T18:24:03.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:03.102+0000 7f7d15465700 1 -- 172.21.15.42:0/194920117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d10027a00 con 0x7f7d100f7510 2024-02-15T18:24:03.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:03.103+0000 7f7d077fe700 1 -- 172.21.15.42:0/194920117 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2939 (secure 0 0 0) 0x7f7cf80487f0 con 0x7f7d100f7510 2024-02-15T18:24:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:03.104+0000 7f7d15465700 1 -- 172.21.15.42:0/194920117 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7cfc05c300 msgr2=0x7f7cfc05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:03.104+0000 7f7d15465700 1 --2- 172.21.15.42:0/194920117 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7cfc05c300 0x7f7cfc05e7c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f7d100f5270 tx=0x7f7d00009b30).stop 2024-02-15T18:24:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:03.104+0000 7f7d15465700 1 -- 172.21.15.42:0/194920117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d100f7510 msgr2=0x7f7d1010e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:03.104+0000 7f7d15465700 1 --2- 172.21.15.42:0/194920117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d100f7510 0x7f7d1010e5e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f7cf800c510 tx=0x7f7cf8007df0).stop 2024-02-15T18:24:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:03.104+0000 7f7d15465700 1 -- 172.21.15.42:0/194920117 shutdown_connections 2024-02-15T18:24:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:03.104+0000 7f7d15465700 1 --2- 172.21.15.42:0/194920117 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7cfc05c300 0x7f7cfc05e7c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:03.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:03.104+0000 7f7d15465700 1 --2- 172.21.15.42:0/194920117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d100f6b70 0x7f7d1010c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:03.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:03.104+0000 7f7d15465700 1 --2- 172.21.15.42:0/194920117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d100f7510 0x7f7d1010e5e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:03.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:03.104+0000 7f7d15465700 1 -- 172.21.15.42:0/194920117 >> 172.21.15.42:0/194920117 conn(0x7f7d100f0470 msgr2=0x7f7d100fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:03.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:03.104+0000 7f7d15465700 1 -- 172.21.15.42:0/194920117 shutdown_connections 2024-02-15T18:24:03.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:03.105+0000 7f7d15465700 1 -- 172.21.15.42:0/194920117 wait complete. 2024-02-15T18:24:03.105 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:24:03.114 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:24:03.796 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-02-15T18:24:04.062 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:02.823775+0000 mgr.smithi042.kfoihb (mgr.14162) 252 : cluster [DBG] pgmap v216: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-15T18:24:04.062 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:02.976905+0000 mon.smithi042 (mon.0) 528 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:04.062 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:02.980111+0000 mon.smithi042 (mon.0) 529 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:04.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:02.983055+0000 mon.smithi042 (mon.0) 530 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:24:04.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:03.103814+0000 mon.smithi042 (mon.0) 531 : audit [DBG] from='client.? 172.21.15.42:0/194920117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-15T18:24:04.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:02.823775+0000 mgr.smithi042.kfoihb (mgr.14162) 252 : cluster [DBG] pgmap v216: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-15T18:24:04.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:02.976905+0000 mon.smithi042 (mon.0) 528 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:04.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:02.980111+0000 mon.smithi042 (mon.0) 529 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:04.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:02.983055+0000 mon.smithi042 (mon.0) 530 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:24:04.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:03.103814+0000 mon.smithi042 (mon.0) 531 : audit [DBG] from='client.? 172.21.15.42:0/194920117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-15T18:24:06.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:04.824412+0000 mgr.smithi042.kfoihb (mgr.14162) 253 : cluster [DBG] pgmap v217: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-02-15T18:24:06.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:05.105046+0000 mon.smithi042 (mon.0) 532 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:06.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:04.824412+0000 mgr.smithi042.kfoihb (mgr.14162) 253 : cluster [DBG] pgmap v217: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-02-15T18:24:06.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:05.105046+0000 mon.smithi042 (mon.0) 532 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:06.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.470+0000 7ff858d18700 1 -- 172.21.15.42:0/2053470134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff854060310 msgr2=0x7ff8540f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:06.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.470+0000 7ff858d18700 1 --2- 172.21.15.42:0/2053470134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff854060310 0x7ff8540f9110 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7ff83c004660 tx=0x7ff83c009b30).stop 2024-02-15T18:24:06.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.470+0000 7ff858d18700 1 -- 172.21.15.42:0/2053470134 shutdown_connections 2024-02-15T18:24:06.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.470+0000 7ff858d18700 1 --2- 172.21.15.42:0/2053470134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8540f9690 0x7ff8540fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:06.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.470+0000 7ff858d18700 1 --2- 172.21.15.42:0/2053470134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff854060310 0x7ff8540f9110 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:06.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.470+0000 7ff858d18700 1 -- 172.21.15.42:0/2053470134 >> 172.21.15.42:0/2053470134 conn(0x7ff8540f0b90 msgr2=0x7ff8540f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:06.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.470+0000 7ff858d18700 1 -- 172.21.15.42:0/2053470134 shutdown_connections 2024-02-15T18:24:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.470+0000 7ff858d18700 1 -- 172.21.15.42:0/2053470134 wait complete. 2024-02-15T18:24:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.470+0000 7ff858d18700 1 Processor -- start 2024-02-15T18:24:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.470+0000 7ff858d18700 1 -- start start 2024-02-15T18:24:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff858d18700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff854060310 0x7ff854106eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff858d18700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8540f9690 0x7ff8541073a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff858d18700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8541079a0 con 0x7ff8540f9690 2024-02-15T18:24:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff858d18700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8541081b0 con 0x7ff854060310 2024-02-15T18:24:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff852ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8540f9690 0x7ff8541073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff852ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8540f9690 0x7ff8541073a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59832/0 (socket says 172.21.15.42:59832) 2024-02-15T18:24:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff852ffd700 1 -- 172.21.15.42:0/1948389402 learned_addr learned my addr 172.21.15.42:0/1948389402 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff8537fe700 1 --2- 172.21.15.42:0/1948389402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff854060310 0x7ff854106eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:06.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff852ffd700 1 -- 172.21.15.42:0/1948389402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff854060310 msgr2=0x7ff854106eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff852ffd700 1 --2- 172.21.15.42:0/1948389402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff854060310 0x7ff854106eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff852ffd700 1 -- 172.21.15.42:0/1948389402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83c005040 con 0x7ff8540f9690 2024-02-15T18:24:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff852ffd700 1 --2- 172.21.15.42:0/1948389402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8540f9690 0x7ff8541073a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7ff844002700 tx=0x7ff844009d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff84bfff700 1 -- 172.21.15.42:0/1948389402 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8440079c0 con 0x7ff8540f9690 2024-02-15T18:24:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff84bfff700 1 -- 172.21.15.42:0/1948389402 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff84400f040 con 0x7ff8540f9690 2024-02-15T18:24:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff858d18700 1 -- 172.21.15.42:0/1948389402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff854108430 con 0x7ff8540f9690 2024-02-15T18:24:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff84bfff700 1 -- 172.21.15.42:0/1948389402 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff844014840 con 0x7ff8540f9690 2024-02-15T18:24:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.471+0000 7ff858d18700 1 -- 172.21.15.42:0/1948389402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8541089f0 con 0x7ff8540f9690 2024-02-15T18:24:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.472+0000 7ff858d18700 1 -- 172.21.15.42:0/1948389402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff854059100 con 0x7ff8540f9690 2024-02-15T18:24:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.473+0000 7ff84bfff700 1 -- 172.21.15.42:0/1948389402 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7ff84401b060 con 0x7ff8540f9690 2024-02-15T18:24:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.473+0000 7ff84bfff700 1 --2- 172.21.15.42:0/1948389402 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff84005c300 0x7ff84005e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.473+0000 7ff84bfff700 1 -- 172.21.15.42:0/1948389402 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7ff844075d20 con 0x7ff8540f9690 2024-02-15T18:24:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.473+0000 7ff8537fe700 1 --2- 172.21.15.42:0/1948389402 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff84005c300 0x7ff84005e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:06.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.473+0000 7ff8537fe700 1 --2- 172.21.15.42:0/1948389402 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff84005c300 0x7ff84005e7c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7ff83c004630 tx=0x7ff83c009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:06.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.477+0000 7ff84bfff700 1 -- 172.21.15.42:0/1948389402 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff84404c3b0 con 0x7ff8540f9690 2024-02-15T18:24:06.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.696+0000 7ff858d18700 1 -- 172.21.15.42:0/1948389402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7ff8540fefb0 con 0x7ff8540f9690 2024-02-15T18:24:06.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.697+0000 7ff84bfff700 1 -- 172.21.15.42:0/1948389402 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v13) v1 ==== 78+0+86 (secure 0 0 0) 0x7ff8540fefb0 con 0x7ff8540f9690 2024-02-15T18:24:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.698+0000 7ff858d18700 1 -- 172.21.15.42:0/1948389402 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff84005c300 msgr2=0x7ff84005e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.698+0000 7ff858d18700 1 --2- 172.21.15.42:0/1948389402 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff84005c300 0x7ff84005e7c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7ff83c004630 tx=0x7ff83c009b30).stop 2024-02-15T18:24:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.698+0000 7ff858d18700 1 -- 172.21.15.42:0/1948389402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8540f9690 msgr2=0x7ff8541073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.698+0000 7ff858d18700 1 --2- 172.21.15.42:0/1948389402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8540f9690 0x7ff8541073a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7ff844002700 tx=0x7ff844009d60).stop 2024-02-15T18:24:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.698+0000 7ff858d18700 1 -- 172.21.15.42:0/1948389402 shutdown_connections 2024-02-15T18:24:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.698+0000 7ff858d18700 1 --2- 172.21.15.42:0/1948389402 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7ff84005c300 0x7ff84005e7c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.698+0000 7ff858d18700 1 --2- 172.21.15.42:0/1948389402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff854060310 0x7ff854106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.698+0000 7ff858d18700 1 --2- 172.21.15.42:0/1948389402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8540f9690 0x7ff8541073a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.698+0000 7ff858d18700 1 -- 172.21.15.42:0/1948389402 >> 172.21.15.42:0/1948389402 conn(0x7ff8540f0b90 msgr2=0x7ff8540f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.699+0000 7ff858d18700 1 -- 172.21.15.42:0/1948389402 shutdown_connections 2024-02-15T18:24:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:06.699+0000 7ff858d18700 1 -- 172.21.15.42:0/1948389402 wait complete. 2024-02-15T18:24:06.708 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:24:07.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:06.698088+0000 mon.smithi042 (mon.0) 533 : audit [DBG] from='client.? 172.21.15.42:0/1948389402' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-15T18:24:07.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:06.698088+0000 mon.smithi042 (mon.0) 533 : audit [DBG] from='client.? 172.21.15.42:0/1948389402' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-15T18:24:07.708 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-02-15T18:24:07.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-15T18:24:08.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.056+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/2919193566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd040f53b0 msgr2=0x7fbd040f57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:08.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.056+0000 7fbd0b2d6700 1 --2- 172.21.15.42:0/2919193566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd040f53b0 0x7fbd040f57d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fbd00004660 tx=0x7fbd00009b30).stop 2024-02-15T18:24:08.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.057+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/2919193566 shutdown_connections 2024-02-15T18:24:08.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.057+0000 7fbd0b2d6700 1 --2- 172.21.15.42:0/2919193566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd040f5cc0 0x7fbd040f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.057+0000 7fbd0b2d6700 1 --2- 172.21.15.42:0/2919193566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd040f53b0 0x7fbd040f57d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.057+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/2919193566 >> 172.21.15.42:0/2919193566 conn(0x7fbd040f0bd0 msgr2=0x7fbd040f2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:08.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.057+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/2919193566 shutdown_connections 2024-02-15T18:24:08.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.057+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/2919193566 wait complete. 2024-02-15T18:24:08.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.057+0000 7fbd0b2d6700 1 Processor -- start 2024-02-15T18:24:08.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.057+0000 7fbd0b2d6700 1 -- start start 2024-02-15T18:24:08.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbd0b2d6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd040f53b0 0x7fbd0405fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:08.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbd0b2d6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd040f5cc0 0x7fbd0410baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:08.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbd0b2d6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0410bf90 con 0x7fbd040f5cc0 2024-02-15T18:24:08.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbd0b2d6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0410c0d0 con 0x7fbd040f53b0 2024-02-15T18:24:08.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbd08871700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd040f5cc0 0x7fbd0410baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:08.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbd08871700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd040f5cc0 0x7fbd0410baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59850/0 (socket says 172.21.15.42:59850) 2024-02-15T18:24:08.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbd08871700 1 -- 172.21.15.42:0/3549829471 learned_addr learned my addr 172.21.15.42:0/3549829471 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:08.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbd09072700 1 --2- 172.21.15.42:0/3549829471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd040f53b0 0x7fbd0405fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:08.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbd08871700 1 -- 172.21.15.42:0/3549829471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd040f53b0 msgr2=0x7fbd0405fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:08.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbd08871700 1 --2- 172.21.15.42:0/3549829471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd040f53b0 0x7fbd0405fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbd08871700 1 -- 172.21.15.42:0/3549829471 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd00005040 con 0x7fbd040f5cc0 2024-02-15T18:24:08.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbd08871700 1 --2- 172.21.15.42:0/3549829471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd040f5cc0 0x7fbd0410baa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fbcf4002700 tx=0x7fbcf4009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:08.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbcf9ffb700 1 -- 172.21.15.42:0/3549829471 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcf4009320 con 0x7fbd040f5cc0 2024-02-15T18:24:08.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/3549829471 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0410c330 con 0x7fbd040f5cc0 2024-02-15T18:24:08.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbcf9ffb700 1 -- 172.21.15.42:0/3549829471 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbcf400f040 con 0x7fbd040f5cc0 2024-02-15T18:24:08.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.058+0000 7fbcf9ffb700 1 -- 172.21.15.42:0/3549829471 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcf4014910 con 0x7fbd040f5cc0 2024-02-15T18:24:08.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.059+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/3549829471 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0410c8f0 con 0x7fbd040f5cc0 2024-02-15T18:24:08.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.060+0000 7fbcf9ffb700 1 -- 172.21.15.42:0/3549829471 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fbcf40077b0 con 0x7fbd040f5cc0 2024-02-15T18:24:08.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.060+0000 7fbcf9ffb700 1 --2- 172.21.15.42:0/3549829471 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fbcf005c370 0x7fbcf005e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:08.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.060+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/3549829471 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd04027a00 con 0x7fbd040f5cc0 2024-02-15T18:24:08.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.061+0000 7fbcf9ffb700 1 -- 172.21.15.42:0/3549829471 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fbcf4076420 con 0x7fbd040f5cc0 2024-02-15T18:24:08.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.061+0000 7fbd09072700 1 --2- 172.21.15.42:0/3549829471 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fbcf005c370 0x7fbcf005e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:08.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.061+0000 7fbd09072700 1 --2- 172.21.15.42:0/3549829471 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fbcf005c370 0x7fbcf005e830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fbd00004630 tx=0x7fbd00009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:08.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.068+0000 7fbcf9ffb700 1 -- 172.21.15.42:0/3549829471 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fbcf40490e0 con 0x7fbd040f5cc0 2024-02-15T18:24:08.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.229+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/3549829471 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd040586e0 con 0x7fbd040f5cc0 2024-02-15T18:24:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.230+0000 7fbcf9ffb700 1 -- 172.21.15.42:0/3549829471 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+14082 (secure 0 0 0) 0x7fbcf4050090 con 0x7fbd040f5cc0 2024-02-15T18:24:08.230 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:24:08.230 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":44,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","created":"2024-02-15T18:16:53.406642+0000","modified":"2024-02-15T18:23:42.481057+0000","last_up_change":"2024-02-15T18:22:16.276777+0000","last_in_change":"2024-02-15T18:21:58.584131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-15T18:21:26.655770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-15T18:23:39.196684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-15T18:23:39.900559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42281c08-af07-43a5-bcfd-d476bc745782","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6803","nonce":3129297361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6805","nonce":3129297361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6809","nonce":3129297361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":3129297361},{"type":"v1","addr":"172.21.15.42:6807","nonce":3129297361}]},"public_addr":"172.21.15.42:6803/3129297361","cluster_addr":"172.21.15.42:6805/3129297361","heartbeat_back_addr":"172.21.15.42:6809/3129297361","heartbeat_front_addr":"172.21.15.42:6807/3129297361","state":["exists","up"]},{"osd":1,"uuid":"8f67d329-7d44-4361-be02-f8b56b3e5713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6811","nonce":563363965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6813","nonce":563363965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6817","nonce":563363965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":563363965},{"type":"v1","addr":"172.21.15.42:6815","nonce":563363965}]},"public_addr":"172.21.15.42:6811/563363965","cluster_addr":"172.21.15.42:6813/563363965","heartbeat_back_addr":"172.21.15.42:6817/563363965","heartbeat_front_addr":"172.21.15.42:6815/563363965","state":["exists","up"]},{"osd":2,"uuid":"680f6437-b450-4353-a761-b50d1388bcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6819","nonce":2706708670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6821","nonce":2706708670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6825","nonce":2706708670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2706708670},{"type":"v1","addr":"172.21.15.42:6823","nonce":2706708670}]},"public_addr":"172.21.15.42:6819/2706708670","cluster_addr":"172.21.15.42:6821/2706708670","heartbeat_back_addr":"172.21.15.42:6825/2706708670","heartbeat_front_addr":"172.21.15.42:6823/2706708670","state":["exists","up"]},{"osd":3,"uuid":"6aeea641-3735-4f11-b95f-40c45d4dcd07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6801","nonce":3223849184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6803","nonce":3223849184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6807","nonce":3223849184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":3223849184},{"type":"v1","addr":"172.21.15.191:6805","nonce":3223849184}]},"public_addr":"172.21.15.191:6801/3223849184","cluster_addr":"172.21.15.191:6803/3223849184","heartbeat_back_addr":"172.21.15.191:6807/3223849184","heartbeat_front_addr":"172.21.15.191:6805/3223849184","state":["exists","up"]},{"osd":4,"uuid":"7d9839d2-bd60-4bae-a16f-ebf56f204acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6809","nonce":424560222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6811","nonce":424560222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6815","nonce":424560222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":424560222},{"type":"v1","addr":"172.21.15.191:6813","nonce":424560222}]},"public_addr":"172.21.15.191:6809/424560222","cluster_addr":"172.21.15.191:6811/424560222","heartbeat_back_addr":"172.21.15.191:6815/424560222","heartbeat_front_addr":"172.21.15.191:6813/424560222","state":["exists","up"]},{"osd":5,"uuid":"a75bb280-bd87-4961-918b-18c426f31c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6817","nonce":2014313522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6819","nonce":2014313522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6823","nonce":2014313522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":2014313522},{"type":"v1","addr":"172.21.15.191:6821","nonce":2014313522}]},"public_addr":"172.21.15.191:6817/2014313522","cluster_addr":"172.21.15.191:6819/2014313522","heartbeat_back_addr":"172.21.15.191:6823/2014313522","heartbeat_front_addr":"172.21.15.191:6821/2014313522","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:20:17.479921+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:20:47.951852+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:11.894830+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:31.013756+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:52.493759+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:22:14.397700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1703009783":"2024-02-16T18:17:56.705000+0000","172.21.15.42:6801/481027993":"2024-02-16T18:17:56.705000+0000","172.21.15.42:0/1629344114":"2024-02-16T18:17:56.705000+0000","172.21.15.42:0/3157909680":"2024-02-16T18:17:40.441172+0000","172.21.15.42:6801/3051492708":"2024-02-16T18:17:40.441172+0000","172.21.15.42:0/1046171692":"2024-02-16T18:17:18.618949+0000","172.21.15.42:6800/481027993":"2024-02-16T18:17:56.705000+0000","172.21.15.42:6800/3051492708":"2024-02-16T18:17:40.441172+0000","172.21.15.42:6800/1746226634":"2024-02-16T18:17:18.618949+0000","172.21.15.42:0/4044100":"2024-02-16T18:17:18.618949+0000","172.21.15.42:0/4136985555":"2024-02-16T18:17:40.441172+0000","172.21.15.42:6801/1746226634":"2024-02-16T18:17:18.618949+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-15T18:24:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.232+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/3549829471 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fbcf005c370 msgr2=0x7fbcf005e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.232+0000 7fbd0b2d6700 1 --2- 172.21.15.42:0/3549829471 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fbcf005c370 0x7fbcf005e830 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fbd00004630 tx=0x7fbd00009b30).stop 2024-02-15T18:24:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.232+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/3549829471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd040f5cc0 msgr2=0x7fbd0410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.232+0000 7fbd0b2d6700 1 --2- 172.21.15.42:0/3549829471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd040f5cc0 0x7fbd0410baa0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fbcf4002700 tx=0x7fbcf4009de0).stop 2024-02-15T18:24:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.232+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/3549829471 shutdown_connections 2024-02-15T18:24:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.232+0000 7fbd0b2d6700 1 --2- 172.21.15.42:0/3549829471 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fbcf005c370 0x7fbcf005e830 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.232+0000 7fbd0b2d6700 1 --2- 172.21.15.42:0/3549829471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd040f53b0 0x7fbd0405fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.232+0000 7fbd0b2d6700 1 --2- 172.21.15.42:0/3549829471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd040f5cc0 0x7fbd0410baa0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.232+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/3549829471 >> 172.21.15.42:0/3549829471 conn(0x7fbd040f0bd0 msgr2=0x7fbd040fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.232+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/3549829471 shutdown_connections 2024-02-15T18:24:08.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.233+0000 7fbd0b2d6700 1 -- 172.21.15.42:0/3549829471 wait complete. 2024-02-15T18:24:08.244 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-15T18:21:26.655770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-15T18:23:39.196684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-15T18:23:39.900559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-15T18:24:08.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-02-15T18:24:08.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:06.824892+0000 mgr.smithi042.kfoihb (mgr.14162) 254 : cluster [DBG] pgmap v218: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-15T18:24:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.369+0000 7fb7f489a700 1 -- 172.21.15.42:0/974990857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7f00f64e0 msgr2=0x7fb7f00f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.369+0000 7fb7f489a700 1 --2- 172.21.15.42:0/974990857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7f00f64e0 0x7fb7f00f6900 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fb7e00045a0 tx=0x7fb7e0009b30).stop 2024-02-15T18:24:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.369+0000 7fb7f489a700 1 -- 172.21.15.42:0/974990857 shutdown_connections 2024-02-15T18:24:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.369+0000 7fb7f489a700 1 --2- 172.21.15.42:0/974990857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7f00f7780 0x7fb7f00f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.369+0000 7fb7f489a700 1 --2- 172.21.15.42:0/974990857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7f00f64e0 0x7fb7f00f6900 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.369+0000 7fb7f489a700 1 -- 172.21.15.42:0/974990857 >> 172.21.15.42:0/974990857 conn(0x7fb7f00f1c40 msgr2=0x7fb7f00f4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.369+0000 7fb7f489a700 1 -- 172.21.15.42:0/974990857 shutdown_connections 2024-02-15T18:24:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.369+0000 7fb7f489a700 1 -- 172.21.15.42:0/974990857 wait complete. 2024-02-15T18:24:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.370+0000 7fb7f489a700 1 Processor -- start 2024-02-15T18:24:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.370+0000 7fb7f489a700 1 -- start start 2024-02-15T18:24:08.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.370+0000 7fb7f489a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7f00f64e0 0x7fb7f0106d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.370+0000 7fb7f489a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7f00f7780 0x7fb7f0107200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.370+0000 7fb7f489a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f0107800 con 0x7fb7f00f7780 2024-02-15T18:24:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.370+0000 7fb7f489a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f0107940 con 0x7fb7f00f64e0 2024-02-15T18:24:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.370+0000 7fb7edd9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7f00f7780 0x7fb7f0107200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.370+0000 7fb7edd9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7f00f7780 0x7fb7f0107200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59862/0 (socket says 172.21.15.42:59862) 2024-02-15T18:24:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.370+0000 7fb7edd9b700 1 -- 172.21.15.42:0/1852569626 learned_addr learned my addr 172.21.15.42:0/1852569626 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.371+0000 7fb7ee59c700 1 --2- 172.21.15.42:0/1852569626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7f00f64e0 0x7fb7f0106d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.371+0000 7fb7edd9b700 1 -- 172.21.15.42:0/1852569626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7f00f64e0 msgr2=0x7fb7f0106d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.371+0000 7fb7edd9b700 1 --2- 172.21.15.42:0/1852569626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7f00f64e0 0x7fb7f0106d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.371+0000 7fb7edd9b700 1 -- 172.21.15.42:0/1852569626 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e4005040 con 0x7fb7f00f7780 2024-02-15T18:24:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.371+0000 7fb7edd9b700 1 --2- 172.21.15.42:0/1852569626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7f00f7780 0x7fb7f0107200 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fb7e4004960 tx=0x7fb7e4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.371+0000 7fb7deffd700 1 -- 172.21.15.42:0/1852569626 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7e400e030 con 0x7fb7f00f7780 2024-02-15T18:24:08.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.371+0000 7fb7deffd700 1 -- 172.21.15.42:0/1852569626 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7e400f040 con 0x7fb7f00f7780 2024-02-15T18:24:08.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.371+0000 7fb7f489a700 1 -- 172.21.15.42:0/1852569626 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e0005040 con 0x7fb7f00f7780 2024-02-15T18:24:08.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.371+0000 7fb7deffd700 1 -- 172.21.15.42:0/1852569626 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7e4014940 con 0x7fb7f00f7780 2024-02-15T18:24:08.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.371+0000 7fb7f489a700 1 -- 172.21.15.42:0/1852569626 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f010c700 con 0x7fb7f00f7780 2024-02-15T18:24:08.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.372+0000 7fb7f489a700 1 -- 172.21.15.42:0/1852569626 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f0027a00 con 0x7fb7f00f7780 2024-02-15T18:24:08.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.372+0000 7fb7deffd700 1 -- 172.21.15.42:0/1852569626 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fb7e4010500 con 0x7fb7f00f7780 2024-02-15T18:24:08.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.373+0000 7fb7deffd700 1 --2- 172.21.15.42:0/1852569626 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fb7d805c2b0 0x7fb7d805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:08.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.373+0000 7fb7deffd700 1 -- 172.21.15.42:0/1852569626 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fb7e4076420 con 0x7fb7f00f7780 2024-02-15T18:24:08.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.373+0000 7fb7ee59c700 1 --2- 172.21.15.42:0/1852569626 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fb7d805c2b0 0x7fb7d805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:08.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.373+0000 7fb7ee59c700 1 --2- 172.21.15.42:0/1852569626 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fb7d805c2b0 0x7fb7d805e770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fb7e0004570 tx=0x7fb7e0016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:08.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.376+0000 7fb7deffd700 1 -- 172.21.15.42:0/1852569626 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb7e4049160 con 0x7fb7f00f7780 2024-02-15T18:24:08.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:06.824892+0000 mgr.smithi042.kfoihb (mgr.14162) 254 : cluster [DBG] pgmap v218: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-15T18:24:08.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.536+0000 7fb7f489a700 1 -- 172.21.15.42:0/1852569626 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb7f00fedb0 con 0x7fb7f00f7780 2024-02-15T18:24:08.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.536+0000 7fb7deffd700 1 -- 172.21.15.42:0/1852569626 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v44) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb7e404c780 con 0x7fb7f00f7780 2024-02-15T18:24:08.536 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-02-15T18:24:08.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.538+0000 7fb7f489a700 1 -- 172.21.15.42:0/1852569626 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fb7d805c2b0 msgr2=0x7fb7d805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:08.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.538+0000 7fb7f489a700 1 --2- 172.21.15.42:0/1852569626 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fb7d805c2b0 0x7fb7d805e770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fb7e0004570 tx=0x7fb7e0016040).stop 2024-02-15T18:24:08.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.538+0000 7fb7f489a700 1 -- 172.21.15.42:0/1852569626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7f00f7780 msgr2=0x7fb7f0107200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:08.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.538+0000 7fb7f489a700 1 --2- 172.21.15.42:0/1852569626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7f00f7780 0x7fb7f0107200 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fb7e4004960 tx=0x7fb7e4009b30).stop 2024-02-15T18:24:08.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.538+0000 7fb7f489a700 1 -- 172.21.15.42:0/1852569626 shutdown_connections 2024-02-15T18:24:08.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.538+0000 7fb7f489a700 1 --2- 172.21.15.42:0/1852569626 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fb7d805c2b0 0x7fb7d805e770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.538+0000 7fb7f489a700 1 --2- 172.21.15.42:0/1852569626 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7f00f64e0 0x7fb7f0106d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.538+0000 7fb7f489a700 1 --2- 172.21.15.42:0/1852569626 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7f00f7780 0x7fb7f0107200 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.538+0000 7fb7f489a700 1 -- 172.21.15.42:0/1852569626 >> 172.21.15.42:0/1852569626 conn(0x7fb7f00f1c40 msgr2=0x7fb7f00fb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:08.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.538+0000 7fb7f489a700 1 -- 172.21.15.42:0/1852569626 shutdown_connections 2024-02-15T18:24:08.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.538+0000 7fb7f489a700 1 -- 172.21.15.42:0/1852569626 wait complete. 2024-02-15T18:24:08.552 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-15T18:24:08.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.678+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/1758285320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e00f7760 msgr2=0x7fa7e00f7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:08.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.678+0000 7fa7e7e1a700 1 --2- 172.21.15.42:0/1758285320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e00f7760 0x7fa7e00f7bc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fa7d4004660 tx=0x7fa7d4009b30).stop 2024-02-15T18:24:08.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.678+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/1758285320 shutdown_connections 2024-02-15T18:24:08.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.678+0000 7fa7e7e1a700 1 --2- 172.21.15.42:0/1758285320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e00f7760 0x7fa7e00f7bc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.678+0000 7fa7e7e1a700 1 --2- 172.21.15.42:0/1758285320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7e00f64c0 0x7fa7e00f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.678+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/1758285320 >> 172.21.15.42:0/1758285320 conn(0x7fa7e00f1c00 msgr2=0x7fa7e00f4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.678+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/1758285320 shutdown_connections 2024-02-15T18:24:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.678+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/1758285320 wait complete. 2024-02-15T18:24:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.679+0000 7fa7e7e1a700 1 Processor -- start 2024-02-15T18:24:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.679+0000 7fa7e7e1a700 1 -- start start 2024-02-15T18:24:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.679+0000 7fa7e7e1a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7e00f64c0 0x7fa7e0106cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.679+0000 7fa7e7e1a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e00f7760 0x7fa7e01071e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.679+0000 7fa7e7e1a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01077e0 con 0x7fa7e00f7760 2024-02-15T18:24:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.679+0000 7fa7e7e1a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e0107920 con 0x7fa7e00f64c0 2024-02-15T18:24:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.679+0000 7fa7e53b5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e00f7760 0x7fa7e01071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.679+0000 7fa7e53b5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e00f7760 0x7fa7e01071e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59876/0 (socket says 172.21.15.42:59876) 2024-02-15T18:24:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.679+0000 7fa7e53b5700 1 -- 172.21.15.42:0/2996421024 learned_addr learned my addr 172.21.15.42:0/2996421024 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.679+0000 7fa7e5bb6700 1 --2- 172.21.15.42:0/2996421024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7e00f64c0 0x7fa7e0106cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.679+0000 7fa7e53b5700 1 -- 172.21.15.42:0/2996421024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7e00f64c0 msgr2=0x7fa7e0106cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:08.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.679+0000 7fa7e53b5700 1 --2- 172.21.15.42:0/2996421024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7e00f64c0 0x7fa7e0106cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.679+0000 7fa7e53b5700 1 -- 172.21.15.42:0/2996421024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d4005040 con 0x7fa7e00f7760 2024-02-15T18:24:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.680+0000 7fa7e53b5700 1 --2- 172.21.15.42:0/2996421024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e00f7760 0x7fa7e01071e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fa7d4000c00 tx=0x7fa7d400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.680+0000 7fa7de7fc700 1 -- 172.21.15.42:0/2996421024 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7d4019070 con 0x7fa7e00f7760 2024-02-15T18:24:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.680+0000 7fa7de7fc700 1 -- 172.21.15.42:0/2996421024 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7d4007e90 con 0x7fa7e00f7760 2024-02-15T18:24:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.680+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/2996421024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e010c2d0 con 0x7fa7e00f7760 2024-02-15T18:24:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.680+0000 7fa7de7fc700 1 -- 172.21.15.42:0/2996421024 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7d4014d50 con 0x7fa7e00f7760 2024-02-15T18:24:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.680+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/2996421024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e010c830 con 0x7fa7e00f7760 2024-02-15T18:24:08.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.681+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/2996421024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e0027a00 con 0x7fa7e00f7760 2024-02-15T18:24:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.681+0000 7fa7de7fc700 1 -- 172.21.15.42:0/2996421024 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fa7d4025070 con 0x7fa7e00f7760 2024-02-15T18:24:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.682+0000 7fa7de7fc700 1 --2- 172.21.15.42:0/2996421024 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa7c805c2b0 0x7fa7c805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.682+0000 7fa7de7fc700 1 -- 172.21.15.42:0/2996421024 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fa7d4076d60 con 0x7fa7e00f7760 2024-02-15T18:24:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.682+0000 7fa7e5bb6700 1 --2- 172.21.15.42:0/2996421024 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa7c805c2b0 0x7fa7c805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:08.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.682+0000 7fa7e5bb6700 1 --2- 172.21.15.42:0/2996421024 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa7c805c2b0 0x7fa7c805e770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fa7e01081e0 tx=0x7fa7d800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:08.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.686+0000 7fa7de7fc700 1 -- 172.21.15.42:0/2996421024 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa7d4049aa0 con 0x7fa7e00f7760 2024-02-15T18:24:08.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.842+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/2996421024 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fa7e00fed90 con 0x7fa7e00f7760 2024-02-15T18:24:08.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.842+0000 7fa7de7fc700 1 -- 172.21.15.42:0/2996421024 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v44) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa7d404d0c0 con 0x7fa7e00f7760 2024-02-15T18:24:08.842 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-02-15T18:24:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.844+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/2996421024 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa7c805c2b0 msgr2=0x7fa7c805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.844+0000 7fa7e7e1a700 1 --2- 172.21.15.42:0/2996421024 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa7c805c2b0 0x7fa7c805e770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fa7e01081e0 tx=0x7fa7d800b040).stop 2024-02-15T18:24:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.844+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/2996421024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e00f7760 msgr2=0x7fa7e01071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.844+0000 7fa7e7e1a700 1 --2- 172.21.15.42:0/2996421024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e00f7760 0x7fa7e01071e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fa7d4000c00 tx=0x7fa7d400b040).stop 2024-02-15T18:24:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.844+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/2996421024 shutdown_connections 2024-02-15T18:24:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.844+0000 7fa7e7e1a700 1 --2- 172.21.15.42:0/2996421024 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fa7c805c2b0 0x7fa7c805e770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.844+0000 7fa7e7e1a700 1 --2- 172.21.15.42:0/2996421024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa7e00f64c0 0x7fa7e0106cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.844+0000 7fa7e7e1a700 1 --2- 172.21.15.42:0/2996421024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e00f7760 0x7fa7e01071e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.844+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/2996421024 >> 172.21.15.42:0/2996421024 conn(0x7fa7e00f1c00 msgr2=0x7fa7e00fb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.844+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/2996421024 shutdown_connections 2024-02-15T18:24:08.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.844+0000 7fa7e7e1a700 1 -- 172.21.15.42:0/2996421024 wait complete. 2024-02-15T18:24:08.856 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-15T18:24:08.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.981+0000 7f087b51f700 1 -- 172.21.15.42:0/2297047256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08740f5390 msgr2=0x7f08740f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:08.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.981+0000 7f087b51f700 1 --2- 172.21.15.42:0/2297047256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08740f5390 0x7f08740f57b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f0870004660 tx=0x7f0870009b30).stop 2024-02-15T18:24:08.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.981+0000 7f087b51f700 1 -- 172.21.15.42:0/2297047256 shutdown_connections 2024-02-15T18:24:08.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.981+0000 7f087b51f700 1 --2- 172.21.15.42:0/2297047256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08740f5ca0 0x7f08740f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.981+0000 7f087b51f700 1 --2- 172.21.15.42:0/2297047256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08740f5390 0x7f08740f57b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.981+0000 7f087b51f700 1 -- 172.21.15.42:0/2297047256 >> 172.21.15.42:0/2297047256 conn(0x7f08740f0b70 msgr2=0x7f08740f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:08.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.981+0000 7f087b51f700 1 -- 172.21.15.42:0/2297047256 shutdown_connections 2024-02-15T18:24:08.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.981+0000 7f087b51f700 1 -- 172.21.15.42:0/2297047256 wait complete. 2024-02-15T18:24:08.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.982+0000 7f087b51f700 1 Processor -- start 2024-02-15T18:24:08.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.982+0000 7f087b51f700 1 -- start start 2024-02-15T18:24:08.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.982+0000 7f087b51f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08740f5390 0x7f0874105bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:08.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.982+0000 7f087b51f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08740f5ca0 0x7f08741060c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:08.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.982+0000 7f087b51f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0874106670 con 0x7f08740f5390 2024-02-15T18:24:08.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.982+0000 7f087b51f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08741067b0 con 0x7f08740f5ca0 2024-02-15T18:24:08.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.983+0000 7f08792bb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08740f5390 0x7f0874105bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:08.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.983+0000 7f08792bb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08740f5390 0x7f0874105bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59894/0 (socket says 172.21.15.42:59894) 2024-02-15T18:24:08.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.983+0000 7f08792bb700 1 -- 172.21.15.42:0/699183154 learned_addr learned my addr 172.21.15.42:0/699183154 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:08.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.983+0000 7f0878aba700 1 --2- 172.21.15.42:0/699183154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08740f5ca0 0x7f08741060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:08.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.983+0000 7f08792bb700 1 -- 172.21.15.42:0/699183154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08740f5ca0 msgr2=0x7f08741060c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:08.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.983+0000 7f08792bb700 1 --2- 172.21.15.42:0/699183154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08740f5ca0 0x7f08741060c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:08.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.983+0000 7f08792bb700 1 -- 172.21.15.42:0/699183154 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0870005040 con 0x7f08740f5390 2024-02-15T18:24:08.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.983+0000 7f08792bb700 1 --2- 172.21.15.42:0/699183154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08740f5390 0x7f0874105bd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f0870009830 tx=0x7f0870009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:08.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.983+0000 7f0869ffb700 1 -- 172.21.15.42:0/699183154 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0870018070 con 0x7f08740f5390 2024-02-15T18:24:08.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.983+0000 7f0869ffb700 1 -- 172.21.15.42:0/699183154 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0870007ba0 con 0x7f08740f5390 2024-02-15T18:24:08.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.983+0000 7f0869ffb700 1 -- 172.21.15.42:0/699183154 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f087001c440 con 0x7f08740f5390 2024-02-15T18:24:08.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.983+0000 7f087b51f700 1 -- 172.21.15.42:0/699183154 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087410b1f0 con 0x7f08740f5390 2024-02-15T18:24:08.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.983+0000 7f087b51f700 1 -- 172.21.15.42:0/699183154 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087410b7b0 con 0x7f08740f5390 2024-02-15T18:24:08.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.984+0000 7f087b51f700 1 -- 172.21.15.42:0/699183154 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0874027a00 con 0x7f08740f5390 2024-02-15T18:24:08.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.984+0000 7f0869ffb700 1 -- 172.21.15.42:0/699183154 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f087001d420 con 0x7f08740f5390 2024-02-15T18:24:08.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.985+0000 7f0869ffb700 1 --2- 172.21.15.42:0/699183154 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f086005c2b0 0x7f086005e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:08.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.985+0000 7f0869ffb700 1 -- 172.21.15.42:0/699183154 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f0870077d60 con 0x7f08740f5390 2024-02-15T18:24:08.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.985+0000 7f0878aba700 1 --2- 172.21.15.42:0/699183154 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f086005c2b0 0x7f086005e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:08.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.985+0000 7f0878aba700 1 --2- 172.21.15.42:0/699183154 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f086005c2b0 0x7f086005e770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f0864009740 tx=0x7f0864009c60).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:08.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:08.989+0000 7f0869ffb700 1 -- 172.21.15.42:0/699183154 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f087004aa20 con 0x7f08740f5390 2024-02-15T18:24:09.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.149+0000 7f087b51f700 1 -- 172.21.15.42:0/699183154 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f08740fdc90 con 0x7f08740f5390 2024-02-15T18:24:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.150+0000 7f0869ffb700 1 -- 172.21.15.42:0/699183154 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v44) v1 ==== 107+0+11 (secure 0 0 0) 0x7f087004e040 con 0x7f08740f5390 2024-02-15T18:24:09.150 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-02-15T18:24:09.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.152+0000 7f087b51f700 1 -- 172.21.15.42:0/699183154 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f086005c2b0 msgr2=0x7f086005e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.152+0000 7f087b51f700 1 --2- 172.21.15.42:0/699183154 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f086005c2b0 0x7f086005e770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f0864009740 tx=0x7f0864009c60).stop 2024-02-15T18:24:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.152+0000 7f087b51f700 1 -- 172.21.15.42:0/699183154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08740f5390 msgr2=0x7f0874105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.152+0000 7f087b51f700 1 --2- 172.21.15.42:0/699183154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08740f5390 0x7f0874105bd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f0870009830 tx=0x7f0870009b30).stop 2024-02-15T18:24:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.152+0000 7f087b51f700 1 -- 172.21.15.42:0/699183154 shutdown_connections 2024-02-15T18:24:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.152+0000 7f087b51f700 1 --2- 172.21.15.42:0/699183154 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f086005c2b0 0x7f086005e770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.152+0000 7f087b51f700 1 --2- 172.21.15.42:0/699183154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08740f5390 0x7f0874105bd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.152+0000 7f087b51f700 1 --2- 172.21.15.42:0/699183154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08740f5ca0 0x7f08741060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.152+0000 7f087b51f700 1 -- 172.21.15.42:0/699183154 >> 172.21.15.42:0/699183154 conn(0x7f08740f0b70 msgr2=0x7f08740fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.152+0000 7f087b51f700 1 -- 172.21.15.42:0/699183154 shutdown_connections 2024-02-15T18:24:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.152+0000 7f087b51f700 1 -- 172.21.15.42:0/699183154 wait complete. 2024-02-15T18:24:09.163 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-15T18:24:09.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.284+0000 7f8016032700 1 -- 172.21.15.42:0/3705182328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80100f53b0 msgr2=0x7f80100f57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:09.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.284+0000 7f8016032700 1 --2- 172.21.15.42:0/3705182328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80100f53b0 0x7f80100f57d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f8000004660 tx=0x7f8000009b30).stop 2024-02-15T18:24:09.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.284+0000 7f8016032700 1 -- 172.21.15.42:0/3705182328 shutdown_connections 2024-02-15T18:24:09.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.284+0000 7f8016032700 1 --2- 172.21.15.42:0/3705182328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80100f5cc0 0x7f80100f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:09.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.284+0000 7f8016032700 1 --2- 172.21.15.42:0/3705182328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80100f53b0 0x7f80100f57d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:09.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.284+0000 7f8016032700 1 -- 172.21.15.42:0/3705182328 >> 172.21.15.42:0/3705182328 conn(0x7f80100f0bd0 msgr2=0x7f80100f2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:09.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.284+0000 7f8016032700 1 -- 172.21.15.42:0/3705182328 shutdown_connections 2024-02-15T18:24:09.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.284+0000 7f8016032700 1 -- 172.21.15.42:0/3705182328 wait complete. 2024-02-15T18:24:09.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.284+0000 7f8016032700 1 Processor -- start 2024-02-15T18:24:09.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.284+0000 7f8016032700 1 -- start start 2024-02-15T18:24:09.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f8016032700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80100f53b0 0x7f801005fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:09.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f8016032700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80100f5cc0 0x7f801010bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:09.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f8016032700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801010bfb0 con 0x7f80100f5cc0 2024-02-15T18:24:09.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f8016032700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801010c0f0 con 0x7f80100f53b0 2024-02-15T18:24:09.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f800effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80100f5cc0 0x7f801010bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:09.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f800effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80100f5cc0 0x7f801010bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59926/0 (socket says 172.21.15.42:59926) 2024-02-15T18:24:09.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f800effd700 1 -- 172.21.15.42:0/3003574015 learned_addr learned my addr 172.21.15.42:0/3003574015 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:09.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f800f7fe700 1 --2- 172.21.15.42:0/3003574015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80100f53b0 0x7f801005fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:09.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f800effd700 1 -- 172.21.15.42:0/3003574015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80100f53b0 msgr2=0x7f801005fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:09.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f800effd700 1 --2- 172.21.15.42:0/3003574015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80100f53b0 0x7f801005fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:09.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f800effd700 1 -- 172.21.15.42:0/3003574015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8000005040 con 0x7f80100f5cc0 2024-02-15T18:24:09.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f800effd700 1 --2- 172.21.15.42:0/3003574015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80100f5cc0 0x7f801010bac0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f80040026d0 tx=0x7f8004009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:09.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f7ff7fff700 1 -- 172.21.15.42:0/3003574015 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80040079c0 con 0x7f80100f5cc0 2024-02-15T18:24:09.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f8016032700 1 -- 172.21.15.42:0/3003574015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801010c350 con 0x7f80100f5cc0 2024-02-15T18:24:09.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f7ff7fff700 1 -- 172.21.15.42:0/3003574015 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f800400f040 con 0x7f80100f5cc0 2024-02-15T18:24:09.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.285+0000 7f7ff7fff700 1 -- 172.21.15.42:0/3003574015 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8004014980 con 0x7f80100f5cc0 2024-02-15T18:24:09.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.286+0000 7f8016032700 1 -- 172.21.15.42:0/3003574015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801010c8e0 con 0x7f80100f5cc0 2024-02-15T18:24:09.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.286+0000 7f8016032700 1 -- 172.21.15.42:0/3003574015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8010027a00 con 0x7f80100f5cc0 2024-02-15T18:24:09.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.287+0000 7f7ff7fff700 1 -- 172.21.15.42:0/3003574015 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f80040090d0 con 0x7f80100f5cc0 2024-02-15T18:24:09.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.288+0000 7f7ff7fff700 1 --2- 172.21.15.42:0/3003574015 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7ff805c370 0x7f7ff805e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:09.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.288+0000 7f7ff7fff700 1 -- 172.21.15.42:0/3003574015 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f8004076070 con 0x7f80100f5cc0 2024-02-15T18:24:09.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.288+0000 7f800f7fe700 1 --2- 172.21.15.42:0/3003574015 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7ff805c370 0x7f7ff805e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:09.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.288+0000 7f800f7fe700 1 --2- 172.21.15.42:0/3003574015 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7ff805c370 0x7f7ff805e830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f8000004630 tx=0x7f8000009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:09.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.291+0000 7f7ff7fff700 1 -- 172.21.15.42:0/3003574015 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8004048e20 con 0x7f80100f5cc0 2024-02-15T18:24:09.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:08.231240+0000 mon.smithi042 (mon.0) 534 : audit [DBG] from='client.? 172.21.15.42:0/3549829471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:24:09.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:08.537756+0000 mon.smithi042 (mon.0) 535 : audit [DBG] from='client.? 172.21.15.42:0/1852569626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-15T18:24:09.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:08.843667+0000 mon.smithi042 (mon.0) 536 : audit [DBG] from='client.? 172.21.15.42:0/2996421024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-15T18:24:09.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.495+0000 7f8016032700 1 -- 172.21.15.42:0/3003574015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80100fdcb0 con 0x7f80100f5cc0 2024-02-15T18:24:09.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.496+0000 7f7ff7fff700 1 -- 172.21.15.42:0/3003574015 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2939 (secure 0 0 0) 0x7f800404c440 con 0x7f80100f5cc0 2024-02-15T18:24:09.495 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:24:09.495 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"cephfs.smithi042.vtwvlo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.42:6827/1892158366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1892158366},{"type":"v1","addr":"172.21.15.42:6827","nonce":1892158366}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":11},{"gid":14450,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.42:6829/2580627796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2580627796},{"type":"v1","addr":"172.21.15.42:6829","nonce":2580627796}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":13}],"filesystems":[{"mdsmap":{"epoch":11,"flags":50,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:23:55.523717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24263":{"gid":24263,"name":"cephfs.smithi191.icwcnb","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.191:6825/1971151000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1971151000},{"type":"v1","addr":"172.21.15.191:6825","nonce":1971151000}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_24277":{"gid":24277,"name":"cephfs.smithi191.erlpxp","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.191:6827/126692509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":126692509},{"type":"v1","addr":"172.21.15.191:6827","nonce":126692509}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:24:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.497+0000 7f8016032700 1 -- 172.21.15.42:0/3003574015 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7ff805c370 msgr2=0x7f7ff805e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.497+0000 7f8016032700 1 --2- 172.21.15.42:0/3003574015 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7ff805c370 0x7f7ff805e830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f8000004630 tx=0x7f8000009b30).stop 2024-02-15T18:24:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.497+0000 7f8016032700 1 -- 172.21.15.42:0/3003574015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80100f5cc0 msgr2=0x7f801010bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.497+0000 7f8016032700 1 --2- 172.21.15.42:0/3003574015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80100f5cc0 0x7f801010bac0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f80040026d0 tx=0x7f8004009de0).stop 2024-02-15T18:24:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.497+0000 7f8016032700 1 -- 172.21.15.42:0/3003574015 shutdown_connections 2024-02-15T18:24:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.497+0000 7f8016032700 1 --2- 172.21.15.42:0/3003574015 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f7ff805c370 0x7f7ff805e830 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.497+0000 7f8016032700 1 --2- 172.21.15.42:0/3003574015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f80100f53b0 0x7f801005fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.497+0000 7f8016032700 1 --2- 172.21.15.42:0/3003574015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80100f5cc0 0x7f801010bac0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.497+0000 7f8016032700 1 -- 172.21.15.42:0/3003574015 >> 172.21.15.42:0/3003574015 conn(0x7f80100f0bd0 msgr2=0x7f80100fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.497+0000 7f8016032700 1 -- 172.21.15.42:0/3003574015 shutdown_connections 2024-02-15T18:24:09.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.497+0000 7f8016032700 1 -- 172.21.15.42:0/3003574015 wait complete. 2024-02-15T18:24:09.498 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:24:09.508 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:08.231240+0000 mon.smithi042 (mon.0) 534 : audit [DBG] from='client.? 172.21.15.42:0/3549829471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:24:09.508 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:08.537756+0000 mon.smithi042 (mon.0) 535 : audit [DBG] from='client.? 172.21.15.42:0/1852569626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-15T18:24:09.508 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:08.843667+0000 mon.smithi042 (mon.0) 536 : audit [DBG] from='client.? 172.21.15.42:0/2996421024' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-15T18:24:09.509 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 11, 'max_mds': 1, 'flags': 50} 2024-02-15T18:24:09.509 INFO:teuthology.run_tasks:Running task kclient... 2024-02-15T18:24:09.530 INFO:tasks.kclient:Mounting kernel clients... 2024-02-15T18:24:09.530 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-02-15T18:24:09.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:09.530 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-02-15T18:24:09.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:09.549 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-02-15T18:24:09.619 INFO:teuthology.orchestra.run.smithi042.stderr:Cannot find device "ceph-brx" 2024-02-15T18:24:09.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:24:09.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:09.621 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-02-15T18:24:09.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:09.645 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2024-02-15T18:24:09.724 INFO:teuthology.orchestra.run.smithi191.stderr:Cannot find device "ceph-brx" 2024-02-15T18:24:09.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:24:09.726 INFO:tasks.kclient:client.0 config is {} 2024-02-15T18:24:09.726 DEBUG:teuthology.orchestra.run.smithi042:> ceph fs ls 2024-02-15T18:24:09.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.836+0000 7f0d684d1700 1 -- 172.21.15.42:0/826629318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d600f96a0 msgr2=0x7f0d600fba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:09.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.836+0000 7f0d684d1700 1 --2- 172.21.15.42:0/826629318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d600f96a0 0x7f0d600fba90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f0d54004660 tx=0x7f0d54009b30).stop 2024-02-15T18:24:09.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.836+0000 7f0d684d1700 1 -- 172.21.15.42:0/826629318 shutdown_connections 2024-02-15T18:24:09.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.836+0000 7f0d684d1700 1 --2- 172.21.15.42:0/826629318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d600f96a0 0x7f0d600fba90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:09.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.836+0000 7f0d684d1700 1 --2- 172.21.15.42:0/826629318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d6005fec0 0x7f0d600f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:09.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.836+0000 7f0d684d1700 1 -- 172.21.15.42:0/826629318 >> 172.21.15.42:0/826629318 conn(0x7f0d600f0bd0 msgr2=0x7f0d600f2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:09.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.836+0000 7f0d684d1700 1 -- 172.21.15.42:0/826629318 shutdown_connections 2024-02-15T18:24:09.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.836+0000 7f0d684d1700 1 -- 172.21.15.42:0/826629318 wait complete. 2024-02-15T18:24:09.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.837+0000 7f0d684d1700 1 Processor -- start 2024-02-15T18:24:09.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.837+0000 7f0d684d1700 1 -- start start 2024-02-15T18:24:09.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.837+0000 7f0d684d1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6005fec0 0x7f0d60106ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:09.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.837+0000 7f0d684d1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d600f96a0 0x7f0d601071d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:09.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.837+0000 7f0d684d1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d601077d0 con 0x7f0d6005fec0 2024-02-15T18:24:09.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.837+0000 7f0d684d1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d60107910 con 0x7f0d600f96a0 2024-02-15T18:24:09.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.837+0000 7f0d6626d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6005fec0 0x7f0d60106ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:09.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.837+0000 7f0d6626d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6005fec0 0x7f0d60106ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59942/0 (socket says 172.21.15.42:59942) 2024-02-15T18:24:09.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.837+0000 7f0d6626d700 1 -- 172.21.15.42:0/2253206542 learned_addr learned my addr 172.21.15.42:0/2253206542 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:09.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.837+0000 7f0d65a6c700 1 --2- 172.21.15.42:0/2253206542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d600f96a0 0x7f0d601071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:09.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.837+0000 7f0d6626d700 1 -- 172.21.15.42:0/2253206542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d600f96a0 msgr2=0x7f0d601071d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:09.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.837+0000 7f0d6626d700 1 --2- 172.21.15.42:0/2253206542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d600f96a0 0x7f0d601071d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:09.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.837+0000 7f0d6626d700 1 -- 172.21.15.42:0/2253206542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d54005040 con 0x7f0d6005fec0 2024-02-15T18:24:09.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.837+0000 7f0d6626d700 1 --2- 172.21.15.42:0/2253206542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6005fec0 0x7f0d60106ce0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f0d5c00e7c0 tx=0x7f0d5c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:09.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.838+0000 7f0d52ffd700 1 -- 172.21.15.42:0/2253206542 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d5c0092e0 con 0x7f0d6005fec0 2024-02-15T18:24:09.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.838+0000 7f0d52ffd700 1 -- 172.21.15.42:0/2253206542 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d5c00f040 con 0x7f0d6005fec0 2024-02-15T18:24:09.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.838+0000 7f0d684d1700 1 -- 172.21.15.42:0/2253206542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d6010c320 con 0x7f0d6005fec0 2024-02-15T18:24:09.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.838+0000 7f0d52ffd700 1 -- 172.21.15.42:0/2253206542 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d5c014880 con 0x7f0d6005fec0 2024-02-15T18:24:09.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.838+0000 7f0d684d1700 1 -- 172.21.15.42:0/2253206542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6010c8e0 con 0x7f0d6005fec0 2024-02-15T18:24:09.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.839+0000 7f0d684d1700 1 -- 172.21.15.42:0/2253206542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d60027a00 con 0x7f0d6005fec0 2024-02-15T18:24:09.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.839+0000 7f0d52ffd700 1 -- 172.21.15.42:0/2253206542 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f0d5c01b030 con 0x7f0d6005fec0 2024-02-15T18:24:09.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.840+0000 7f0d52ffd700 1 --2- 172.21.15.42:0/2253206542 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0d4c05c2b0 0x7f0d4c05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:09.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.840+0000 7f0d52ffd700 1 -- 172.21.15.42:0/2253206542 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f0d5c076120 con 0x7f0d6005fec0 2024-02-15T18:24:09.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.840+0000 7f0d65a6c700 1 --2- 172.21.15.42:0/2253206542 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0d4c05c2b0 0x7f0d4c05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:09.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.840+0000 7f0d65a6c700 1 --2- 172.21.15.42:0/2253206542 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0d4c05c2b0 0x7f0d4c05e770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f0d54004630 tx=0x7f0d54016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:09.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:09.843+0000 7f0d52ffd700 1 -- 172.21.15.42:0/2253206542 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0d5c048e60 con 0x7f0d6005fec0 2024-02-15T18:24:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:10.044+0000 7f0d684d1700 1 -- 172.21.15.42:0/2253206542 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0d60058780 con 0x7f0d6005fec0 2024-02-15T18:24:10.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:10.045+0000 7f0d52ffd700 1 -- 172.21.15.42:0/2253206542 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+83 (secure 0 0 0) 0x7f0d5c019070 con 0x7f0d6005fec0 2024-02-15T18:24:10.045 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-15T18:24:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:10.046+0000 7f0d684d1700 1 -- 172.21.15.42:0/2253206542 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0d4c05c2b0 msgr2=0x7f0d4c05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:10.046+0000 7f0d684d1700 1 --2- 172.21.15.42:0/2253206542 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0d4c05c2b0 0x7f0d4c05e770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f0d54004630 tx=0x7f0d54016040).stop 2024-02-15T18:24:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:10.046+0000 7f0d684d1700 1 -- 172.21.15.42:0/2253206542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6005fec0 msgr2=0x7f0d60106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:10.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:10.046+0000 7f0d684d1700 1 --2- 172.21.15.42:0/2253206542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6005fec0 0x7f0d60106ce0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f0d5c00e7c0 tx=0x7f0d5c00b040).stop 2024-02-15T18:24:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:10.047+0000 7f0d684d1700 1 -- 172.21.15.42:0/2253206542 shutdown_connections 2024-02-15T18:24:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:10.047+0000 7f0d684d1700 1 --2- 172.21.15.42:0/2253206542 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0d4c05c2b0 0x7f0d4c05e770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:10.047+0000 7f0d684d1700 1 --2- 172.21.15.42:0/2253206542 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d6005fec0 0x7f0d60106ce0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:10.047+0000 7f0d684d1700 1 --2- 172.21.15.42:0/2253206542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d600f96a0 0x7f0d601071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:10.047+0000 7f0d684d1700 1 -- 172.21.15.42:0/2253206542 >> 172.21.15.42:0/2253206542 conn(0x7f0d600f0bd0 msgr2=0x7f0d600f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:10.047+0000 7f0d684d1700 1 -- 172.21.15.42:0/2253206542 shutdown_connections 2024-02-15T18:24:10.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:10.047+0000 7f0d684d1700 1 -- 172.21.15.42:0/2253206542 wait complete. 2024-02-15T18:24:10.057 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-15T18:24:10.057 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-15T18:24:10.057 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-02-15T18:24:10.057 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-15T18:24:10.057 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-15T18:24:10.057 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-15T18:24:10.057 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-15T18:24:10.057 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-15T18:24:10.057 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-15T18:24:10.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:10.057 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-02-15T18:24:10.076 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-15T18:24:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-15T18:24:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-02-15T18:24:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-02-15T18:24:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-02-15T18:24:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-02-15T18:24:10.076 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-15T18:24:10.076 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-02-15T18:24:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-02-15T18:24:10.077 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-15T18:24:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-02-15T18:24:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-02-15T18:24:10.077 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-15T18:24:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-02-15T18:24:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-02-15T18:24:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-15T18:24:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 41436sec preferred_lft 41436sec 2024-02-15T18:24:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2024-02-15T18:24:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-02-15T18:24:10.077 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-15T18:24:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-02-15T18:24:10.077 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-02-15T18:24:10.078 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-15T18:24:10.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:10.078 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2024-02-15T18:24:10.078 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-02-15T18:24:10.078 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add name ceph-brx type bridge 2024-02-15T18:24:10.078 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr flush dev ceph-brx 2024-02-15T18:24:10.078 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set ceph-brx up 2024-02-15T18:24:10.078 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-15T18:24:10.079 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-02-15T18:24:10.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:10.246 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-15T18:24:10.279 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-02-15T18:24:10.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:10.280 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-02-15T18:24:10.358 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-02-15T18:24:10.358 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-15T18:24:10.358 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-15T18:24:10.358 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-15T18:24:10.358 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-15T18:24:10.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:10.359 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2024-02-15T18:24:10.359 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-02-15T18:24:10.359 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-15T18:24:10.359 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-15T18:24:10.359 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-15T18:24:10.359 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-02-15T18:24:10.439 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:08.825361+0000 mgr.smithi042.kfoihb (mgr.14162) 255 : cluster [DBG] pgmap v219: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:24:10.439 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:09.151374+0000 mon.smithi042 (mon.0) 537 : audit [DBG] from='client.? 172.21.15.42:0/699183154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-15T18:24:10.439 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:09.496824+0000 mon.smithi042 (mon.0) 538 : audit [DBG] from='client.? 172.21.15.42:0/3003574015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-15T18:24:10.439 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:10.046065+0000 mon.smithi042 (mon.0) 539 : audit [DBG] from='client.? 172.21.15.42:0/2253206542' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-15T18:24:10.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:08.825361+0000 mgr.smithi042.kfoihb (mgr.14162) 255 : cluster [DBG] pgmap v219: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:24:10.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:09.151374+0000 mon.smithi042 (mon.0) 537 : audit [DBG] from='client.? 172.21.15.42:0/699183154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-15T18:24:10.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:09.496824+0000 mon.smithi042 (mon.0) 538 : audit [DBG] from='client.? 172.21.15.42:0/3003574015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-15T18:24:10.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:10.046065+0000 mon.smithi042 (mon.0) 539 : audit [DBG] from='client.? 172.21.15.42:0/2253206542' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-15T18:24:10.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:10.634 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-02-15T18:24:10.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:10.694 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list-id 2024-02-15T18:24:10.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:10.751 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2024-02-15T18:24:10.751 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-02-15T18:24:10.751 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-15T18:24:10.752 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-15T18:24:10.752 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-02-15T18:24:10.848 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-15T18:24:10.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:10.848 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2024-02-15T18:24:10.848 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-02-15T18:24:10.849 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-15T18:24:10.849 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-15T18:24:10.849 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-15T18:24:10.849 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-15T18:24:10.849 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-15T18:24:10.849 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-02-15T18:24:11.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:11.038 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2024-02-15T18:24:11.038 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-02-15T18:24:11.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-02-15T18:24:11.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set dev brx.0 master ceph-brx 2024-02-15T18:24:11.038 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-02-15T18:24:11.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:11.137 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p /home/ubuntu/cephtest/mnt.0 2024-02-15T18:24:11.154 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-02-15T18:24:11.154 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-02-15T18:24:11.245 INFO:teuthology.orchestra.run.smithi042.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-02-15T18:24:11.246 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-02-15T18:24:11.267 INFO:teuthology.orchestra.run.smithi042.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-02-15T18:24:11.267 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-02-15T18:24:11.270 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-15T18:24:11.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:11.270 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-15T18:24:11.298 INFO:tasks.kclient:client.1 config is {} 2024-02-15T18:24:11.298 DEBUG:teuthology.orchestra.run.smithi191:> ceph fs ls 2024-02-15T18:24:11.487 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.486+0000 7f9f0b14d700 1 -- 172.21.15.191:0/2513882257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f040f64e0 msgr2=0x7f9f040f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:11.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.486+0000 7f9f0b14d700 1 --2- 172.21.15.191:0/2513882257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f040f64e0 0x7f9f040f6900 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f9efc0099e0 tx=0x7f9efc004d10).stop 2024-02-15T18:24:11.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.486+0000 7f9f0b14d700 1 -- 172.21.15.191:0/2513882257 shutdown_connections 2024-02-15T18:24:11.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.486+0000 7f9f0b14d700 1 --2- 172.21.15.191:0/2513882257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f040f7780 0x7f9f040f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:11.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.486+0000 7f9f0b14d700 1 --2- 172.21.15.191:0/2513882257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f040f64e0 0x7f9f040f6900 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:11.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.486+0000 7f9f0b14d700 1 -- 172.21.15.191:0/2513882257 >> 172.21.15.191:0/2513882257 conn(0x7f9f040f1c40 msgr2=0x7f9f040f4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:11.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.486+0000 7f9f0b14d700 1 -- 172.21.15.191:0/2513882257 shutdown_connections 2024-02-15T18:24:11.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.486+0000 7f9f0b14d700 1 -- 172.21.15.191:0/2513882257 wait complete. 2024-02-15T18:24:11.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.487+0000 7f9f0b14d700 1 Processor -- start 2024-02-15T18:24:11.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.487+0000 7f9f0b14d700 1 -- start start 2024-02-15T18:24:11.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.487+0000 7f9f0b14d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f040f64e0 0x7f9f04106e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:11.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.487+0000 7f9f0b14d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f040f7780 0x7f9f04107330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:11.488 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.487+0000 7f9f0b14d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f041078e0 con 0x7f9f040f64e0 2024-02-15T18:24:11.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.487+0000 7f9f0b14d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f04107a20 con 0x7f9f040f7780 2024-02-15T18:24:11.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.487+0000 7f9f03fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f040f7780 0x7f9f04107330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:11.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.487+0000 7f9f03fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f040f7780 0x7f9f04107330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:55002/0 (socket says 172.21.15.191:55002) 2024-02-15T18:24:11.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.487+0000 7f9f03fff700 1 -- 172.21.15.191:0/3528438242 learned_addr learned my addr 172.21.15.191:0/3528438242 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-02-15T18:24:11.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.487+0000 7f9f08ee9700 1 --2- 172.21.15.191:0/3528438242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f040f64e0 0x7f9f04106e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:11.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.487+0000 7f9f03fff700 1 -- 172.21.15.191:0/3528438242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f040f64e0 msgr2=0x7f9f04106e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:11.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.487+0000 7f9f03fff700 1 --2- 172.21.15.191:0/3528438242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f040f64e0 0x7f9f04106e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:11.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.488+0000 7f9f03fff700 1 -- 172.21.15.191:0/3528438242 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc009640 con 0x7f9f040f7780 2024-02-15T18:24:11.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.488+0000 7f9f03fff700 1 --2- 172.21.15.191:0/3528438242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f040f7780 0x7f9f04107330 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f9ef800c790 tx=0x7f9ef8007d70).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:11.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.488+0000 7f9f017fa700 1 -- 172.21.15.191:0/3528438242 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ef8002590 con 0x7f9f040f7780 2024-02-15T18:24:11.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.488+0000 7f9f017fa700 1 -- 172.21.15.191:0/3528438242 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ef8002bc0 con 0x7f9f040f7780 2024-02-15T18:24:11.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.488+0000 7f9f0b14d700 1 -- 172.21.15.191:0/3528438242 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0410c4c0 con 0x7f9f040f7780 2024-02-15T18:24:11.489 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.488+0000 7f9f017fa700 1 -- 172.21.15.191:0/3528438242 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ef8013e40 con 0x7f9f040f7780 2024-02-15T18:24:11.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.488+0000 7f9f0b14d700 1 -- 172.21.15.191:0/3528438242 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0410ca00 con 0x7f9f040f7780 2024-02-15T18:24:11.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.489+0000 7f9f0b14d700 1 -- 172.21.15.191:0/3528438242 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f04027a00 con 0x7f9f040f7780 2024-02-15T18:24:11.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.489+0000 7f9f017fa700 1 -- 172.21.15.191:0/3528438242 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f9ef801a060 con 0x7f9f040f7780 2024-02-15T18:24:11.490 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.489+0000 7f9f017fa700 1 --2- 172.21.15.191:0/3528438242 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9eec05c2b0 0x7f9eec05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:11.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.489+0000 7f9f017fa700 1 -- 172.21.15.191:0/3528438242 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f9ef80760f0 con 0x7f9f040f7780 2024-02-15T18:24:11.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.489+0000 7f9f08ee9700 1 --2- 172.21.15.191:0/3528438242 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9eec05c2b0 0x7f9eec05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:11.491 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.490+0000 7f9f08ee9700 1 --2- 172.21.15.191:0/3528438242 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9eec05c2b0 0x7f9eec05e770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f9efc003380 tx=0x7f9efc004d10).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:11.492 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.493+0000 7f9f017fa700 1 -- 172.21.15.191:0/3528438242 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9ef804c860 con 0x7f9f040f7780 2024-02-15T18:24:11.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.678+0000 7f9f0b14d700 1 -- 172.21.15.191:0/3528438242 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9f040fedb0 con 0x7f9f040f7780 2024-02-15T18:24:11.677 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.678+0000 7f9f017fa700 1 -- 172.21.15.191:0/3528438242 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+83 (secure 0 0 0) 0x7f9ef804c450 con 0x7f9f040f7780 2024-02-15T18:24:11.677 INFO:teuthology.orchestra.run.smithi191.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-15T18:24:11.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.680+0000 7f9f0b14d700 1 -- 172.21.15.191:0/3528438242 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9eec05c2b0 msgr2=0x7f9eec05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:11.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.680+0000 7f9f0b14d700 1 --2- 172.21.15.191:0/3528438242 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9eec05c2b0 0x7f9eec05e770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f9efc003380 tx=0x7f9efc004d10).stop 2024-02-15T18:24:11.679 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.680+0000 7f9f0b14d700 1 -- 172.21.15.191:0/3528438242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f040f7780 msgr2=0x7f9f04107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:11.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.680+0000 7f9f0b14d700 1 --2- 172.21.15.191:0/3528438242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f040f7780 0x7f9f04107330 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f9ef800c790 tx=0x7f9ef8007d70).stop 2024-02-15T18:24:11.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.680+0000 7f9f0b14d700 1 -- 172.21.15.191:0/3528438242 shutdown_connections 2024-02-15T18:24:11.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.680+0000 7f9f0b14d700 1 --2- 172.21.15.191:0/3528438242 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f9eec05c2b0 0x7f9eec05e770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:11.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.680+0000 7f9f0b14d700 1 --2- 172.21.15.191:0/3528438242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f040f64e0 0x7f9f04106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:11.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.680+0000 7f9f0b14d700 1 --2- 172.21.15.191:0/3528438242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f040f7780 0x7f9f04107330 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:11.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.680+0000 7f9f0b14d700 1 -- 172.21.15.191:0/3528438242 >> 172.21.15.191:0/3528438242 conn(0x7f9f040f1c40 msgr2=0x7f9f040fb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:11.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.680+0000 7f9f0b14d700 1 -- 172.21.15.191:0/3528438242 shutdown_connections 2024-02-15T18:24:11.680 INFO:teuthology.orchestra.run.smithi191.stderr:2024-02-15T18:24:11.680+0000 7f9f0b14d700 1 -- 172.21.15.191:0/3528438242 wait complete. 2024-02-15T18:24:11.689 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-15T18:24:11.690 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-15T18:24:11.690 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2024-02-15T18:24:11.690 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-15T18:24:11.690 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-15T18:24:11.690 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-15T18:24:11.690 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-15T18:24:11.690 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-15T18:24:11.690 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-15T18:24:11.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:11.690 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2024-02-15T18:24:11.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:10.353358+0000 mon.smithi042 (mon.0) 540 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:11.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:10.362045+0000 mon.smithi042 (mon.0) 541 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:11.708 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-15T18:24:11.709 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-15T18:24:11.709 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2024-02-15T18:24:11.709 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-02-15T18:24:11.709 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2024-02-15T18:24:11.709 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-02-15T18:24:11.709 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-15T18:24:11.709 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2024-02-15T18:24:11.709 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2024-02-15T18:24:11.709 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-15T18:24:11.709 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2024-02-15T18:24:11.709 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2024-02-15T18:24:11.709 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-15T18:24:11.710 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2024-02-15T18:24:11.710 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-15T18:24:11.710 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2024-02-15T18:24:11.710 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-15T18:24:11.710 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 41418sec preferred_lft 41418sec 2024-02-15T18:24:11.710 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2024-02-15T18:24:11.710 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2024-02-15T18:24:11.710 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-15T18:24:11.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:11.710 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-02-15T18:24:11.710 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-02-15T18:24:11.710 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add name ceph-brx type bridge 2024-02-15T18:24:11.710 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip addr flush dev ceph-brx 2024-02-15T18:24:11.711 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set ceph-brx up 2024-02-15T18:24:11.711 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-15T18:24:11.711 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-02-15T18:24:11.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:10.353358+0000 mon.smithi042 (mon.0) 540 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:11.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:10.362045+0000 mon.smithi042 (mon.0) 541 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:11.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:11.875 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-15T18:24:11.904 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-02-15T18:24:11.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:11.906 DEBUG:teuthology.orchestra.run.smithi191:> route 2024-02-15T18:24:11.999 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2024-02-15T18:24:12.000 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-15T18:24:12.000 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-15T18:24:12.000 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-15T18:24:12.000 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-15T18:24:12.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:12.000 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-02-15T18:24:12.000 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-02-15T18:24:12.000 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-15T18:24:12.000 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-15T18:24:12.000 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-15T18:24:12.001 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-02-15T18:24:12.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:12.242 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-02-15T18:24:12.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:12.260 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list-id 2024-02-15T18:24:12.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:12.317 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-02-15T18:24:12.317 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-02-15T18:24:12.317 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-15T18:24:12.317 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-02-15T18:24:12.317 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-02-15T18:24:12.412 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-02-15T18:24:12.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:12.412 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-02-15T18:24:12.412 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-02-15T18:24:12.412 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-02-15T18:24:12.412 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-15T18:24:12.412 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-02-15T18:24:12.412 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-02-15T18:24:12.412 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-02-15T18:24:12.413 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-02-15T18:24:12.615 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:10.825775+0000 mgr.smithi042.kfoihb (mgr.14162) 256 : cluster [DBG] pgmap v220: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:24:12.615 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:11.679201+0000 mon.smithi191 (mon.1) 14 : audit [DBG] from='client.? 172.21.15.191:0/3528438242' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-15T18:24:12.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:12.617 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2024-02-15T18:24:12.617 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2024-02-15T18:24:12.617 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2024-02-15T18:24:12.617 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set dev brx.0 master ceph-brx 2024-02-15T18:24:12.617 DEBUG:teuthology.orchestra.run.smithi191:> ') 2024-02-15T18:24:12.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:12.714 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p /home/ubuntu/cephtest/mnt.1 2024-02-15T18:24:12.732 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-02-15T18:24:12.732 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-02-15T18:24:12.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:10.825775+0000 mgr.smithi042.kfoihb (mgr.14162) 256 : cluster [DBG] pgmap v220: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:24:12.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:11.679201+0000 mon.smithi191 (mon.1) 14 : audit [DBG] from='client.? 172.21.15.191:0/3528438242' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-15T18:24:12.842 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-02-15T18:24:12.842 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,name=1,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-02-15T18:24:12.862 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2024-02-15T18:24:12.862 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,name=1,mds_namespace=cephfs,norbytes" will pass to kernel. 2024-02-15T18:24:12.865 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-15T18:24:12.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:24:12.865 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-15T18:24:12.892 INFO:teuthology.run_tasks:Running task print... 2024-02-15T18:24:12.902 INFO:teuthology.task.print:**** done client 2024-02-15T18:24:12.902 INFO:teuthology.run_tasks:Running task parallel... 2024-02-15T18:24:12.945 INFO:teuthology.task.parallel:starting parallel... 2024-02-15T18:24:12.946 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-15T18:24:12.946 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-15T18:24:12.947 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-02-15T18:24:12.947 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-02-15T18:24:12.947 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-15T18:24:12.947 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-02-15T18:24:12.951 INFO:tasks.workunit:Pulling workunits from ref bedfc49c6a823fb0ff75911e476e1f3cb464ad3e 2024-02-15T18:24:12.951 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-15T18:24:12.951 INFO:tasks.workunit:timeout=3h 2024-02-15T18:24:12.951 INFO:tasks.workunit:cleanup=True 2024-02-15T18:24:12.951 DEBUG:teuthology.orchestra.run.smithi042:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-15T18:24:12.976 INFO:teuthology.orchestra.run.smithi042.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-15T18:24:12.976 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-02-15T18:24:12.976 INFO:teuthology.orchestra.run.smithi042.stdout:Device: a1h/161d Inode: 1 Links: 2 2024-02-15T18:24:12.977 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:24:12.977 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:cephfs_t:s0 2024-02-15T18:24:12.977 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-15T18:24:12.977 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-02-15 18:23:43.496455396 +0000 2024-02-15T18:24:12.977 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-02-15 18:24:11.297873837 +0000 2024-02-15T18:24:12.977 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-02-15 18:23:43.496455396 +0000 2024-02-15T18:24:12.977 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-15T18:24:12.977 DEBUG:teuthology.orchestra.run.smithi042:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-15T18:24:13.051 DEBUG:teuthology.orchestra.run.smithi191:> stat -- /home/ubuntu/cephtest/mnt.1 2024-02-15T18:24:13.069 INFO:teuthology.orchestra.run.smithi191.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-02-15T18:24:13.069 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-02-15T18:24:13.069 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 7dh/125d Inode: 1 Links: 3 2024-02-15T18:24:13.069 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-15T18:24:13.069 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:cephfs_t:s0 2024-02-15T18:24:13.069 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-15T18:24:13.069 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-15 18:24:13.047860042 +0000 2024-02-15T18:24:13.069 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-15 18:24:13.047860042 +0000 2024-02-15T18:24:13.069 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-15 18:23:43.496455396 +0000 2024-02-15T18:24:13.070 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-02-15T18:24:13.070 DEBUG:teuthology.orchestra.run.smithi191:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-02-15T18:24:13.139 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ljflores/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bedfc49c6a823fb0ff75911e476e1f3cb464ad3e 2024-02-15T18:24:13.139 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://github.com/ljflores/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout bedfc49c6a823fb0ff75911e476e1f3cb464ad3e 2024-02-15T18:24:13.159 INFO:tasks.workunit.client.0.smithi042.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-15T18:24:13.197 INFO:tasks.workunit.client.1.smithi191.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-02-15T18:24:14.658 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:12.826208+0000 mgr.smithi042.kfoihb (mgr.14162) 257 : cluster [DBG] pgmap v221: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:24:14.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:12.826208+0000 mgr.smithi042.kfoihb (mgr.14162) 257 : cluster [DBG] pgmap v221: 65 pgs: 65 active+clean; 2.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:24:14.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.913+0000 7f32916a3700 1 -- 172.21.15.42:0/392068156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328c0fba40 msgr2=0x7f328c0fde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:14.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.913+0000 7f32916a3700 1 --2- 172.21.15.42:0/392068156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328c0fba40 0x7f328c0fde30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f3280004660 tx=0x7f3280009b30).stop 2024-02-15T18:24:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.913+0000 7f32916a3700 1 -- 172.21.15.42:0/392068156 shutdown_connections 2024-02-15T18:24:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.913+0000 7f32916a3700 1 --2- 172.21.15.42:0/392068156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328c0fba40 0x7f328c0fde30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.913+0000 7f32916a3700 1 --2- 172.21.15.42:0/392068156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f328c0602d0 0x7f328c0fb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.913+0000 7f32916a3700 1 -- 172.21.15.42:0/392068156 >> 172.21.15.42:0/392068156 conn(0x7f328c0f0b90 msgr2=0x7f328c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.913+0000 7f32916a3700 1 -- 172.21.15.42:0/392068156 shutdown_connections 2024-02-15T18:24:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.913+0000 7f32916a3700 1 -- 172.21.15.42:0/392068156 wait complete. 2024-02-15T18:24:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.914+0000 7f32916a3700 1 Processor -- start 2024-02-15T18:24:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.914+0000 7f32916a3700 1 -- start start 2024-02-15T18:24:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.914+0000 7f32916a3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f328c0602d0 0x7f328c109060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.914+0000 7f32916a3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328c0fba40 0x7f328c109550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.914+0000 7f32916a3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c109b00 con 0x7f328c0fba40 2024-02-15T18:24:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.914+0000 7f32916a3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328c109c40 con 0x7f328c0602d0 2024-02-15T18:24:14.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.914+0000 7f328b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328c0fba40 0x7f328c109550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.914+0000 7f328b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328c0fba40 0x7f328c109550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48298/0 (socket says 172.21.15.42:48298) 2024-02-15T18:24:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.914+0000 7f328b7fe700 1 -- 172.21.15.42:0/3251125664 learned_addr learned my addr 172.21.15.42:0/3251125664 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.914+0000 7f328bfff700 1 --2- 172.21.15.42:0/3251125664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f328c0602d0 0x7f328c109060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.915+0000 7f328b7fe700 1 -- 172.21.15.42:0/3251125664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f328c0602d0 msgr2=0x7f328c109060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.915+0000 7f328b7fe700 1 --2- 172.21.15.42:0/3251125664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f328c0602d0 0x7f328c109060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.915+0000 7f328b7fe700 1 -- 172.21.15.42:0/3251125664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3280005040 con 0x7f328c0fba40 2024-02-15T18:24:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.915+0000 7f328b7fe700 1 --2- 172.21.15.42:0/3251125664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328c0fba40 0x7f328c109550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f3280004e30 tx=0x7f3280009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.915+0000 7f3288ff9700 1 -- 172.21.15.42:0/3251125664 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3280018070 con 0x7f328c0fba40 2024-02-15T18:24:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.915+0000 7f3288ff9700 1 -- 172.21.15.42:0/3251125664 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3280007ba0 con 0x7f328c0fba40 2024-02-15T18:24:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.915+0000 7f32916a3700 1 -- 172.21.15.42:0/3251125664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328c10a580 con 0x7f328c0fba40 2024-02-15T18:24:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.915+0000 7f3288ff9700 1 -- 172.21.15.42:0/3251125664 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f328001c3c0 con 0x7f328c0fba40 2024-02-15T18:24:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.915+0000 7f32916a3700 1 -- 172.21.15.42:0/3251125664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328c10aae0 con 0x7f328c0fba40 2024-02-15T18:24:14.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.916+0000 7f32916a3700 1 -- 172.21.15.42:0/3251125664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328c045ad0 con 0x7f328c0fba40 2024-02-15T18:24:14.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.917+0000 7f3288ff9700 1 -- 172.21.15.42:0/3251125664 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f3280007d10 con 0x7f328c0fba40 2024-02-15T18:24:14.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.917+0000 7f3288ff9700 1 --2- 172.21.15.42:0/3251125664 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f327405c300 0x7f327405e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:14.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.917+0000 7f3288ff9700 1 -- 172.21.15.42:0/3251125664 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f3280077aa0 con 0x7f328c0fba40 2024-02-15T18:24:14.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.917+0000 7f328bfff700 1 --2- 172.21.15.42:0/3251125664 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f327405c300 0x7f327405e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:14.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.917+0000 7f328bfff700 1 --2- 172.21.15.42:0/3251125664 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f327405c300 0x7f327405e7c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f327c004c40 tx=0x7f327c00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:14.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:14.921+0000 7f3288ff9700 1 -- 172.21.15.42:0/3251125664 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f328004a760 con 0x7f328c0fba40 2024-02-15T18:24:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:15.088+0000 7f32916a3700 1 -- 172.21.15.42:0/3251125664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f328c027a00 con 0x7f328c0fba40 2024-02-15T18:24:15.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:15.089+0000 7f3288ff9700 1 -- 172.21.15.42:0/3251125664 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v16)=0 v16) v1 ==== 155+0+0 (secure 0 0 0) 0x7f328004dd80 con 0x7f328c0fba40 2024-02-15T18:24:15.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:15.090+0000 7f32916a3700 1 -- 172.21.15.42:0/3251125664 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f327405c300 msgr2=0x7f327405e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:15.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:15.090+0000 7f32916a3700 1 --2- 172.21.15.42:0/3251125664 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f327405c300 0x7f327405e7c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f327c004c40 tx=0x7f327c00b040).stop 2024-02-15T18:24:15.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:15.090+0000 7f32916a3700 1 -- 172.21.15.42:0/3251125664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328c0fba40 msgr2=0x7f328c109550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:15.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:15.090+0000 7f32916a3700 1 --2- 172.21.15.42:0/3251125664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328c0fba40 0x7f328c109550 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f3280004e30 tx=0x7f3280009b30).stop 2024-02-15T18:24:15.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:15.091+0000 7f32916a3700 1 -- 172.21.15.42:0/3251125664 shutdown_connections 2024-02-15T18:24:15.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:15.091+0000 7f32916a3700 1 --2- 172.21.15.42:0/3251125664 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f327405c300 0x7f327405e7c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:15.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:15.091+0000 7f32916a3700 1 --2- 172.21.15.42:0/3251125664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f328c0602d0 0x7f328c109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:15.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:15.091+0000 7f32916a3700 1 --2- 172.21.15.42:0/3251125664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f328c0fba40 0x7f328c109550 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:15.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:15.091+0000 7f32916a3700 1 -- 172.21.15.42:0/3251125664 >> 172.21.15.42:0/3251125664 conn(0x7f328c0f0b90 msgr2=0x7f328c0f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:15.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:15.091+0000 7f32916a3700 1 -- 172.21.15.42:0/3251125664 shutdown_connections 2024-02-15T18:24:15.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:15.091+0000 7f32916a3700 1 -- 172.21.15.42:0/3251125664 wait complete. 2024-02-15T18:24:15.818 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-02-15T18:24:16.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:14.826599+0000 mgr.smithi042.kfoihb (mgr.14162) 258 : cluster [DBG] pgmap v222: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-02-15T18:24:16.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:14.826599+0000 mgr.smithi042.kfoihb (mgr.14162) 258 : cluster [DBG] pgmap v222: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-02-15T18:24:17.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.894+0000 7f85bf59e700 1 -- 172.21.15.42:0/480584606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85c00f96f0 msgr2=0x7f85c00fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:17.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.894+0000 7f85bf59e700 1 --2- 172.21.15.42:0/480584606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85c00f96f0 0x7f85c00fbae0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f85b4004660 tx=0x7f85b4009b30).stop 2024-02-15T18:24:17.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.894+0000 7f85bf59e700 1 -- 172.21.15.42:0/480584606 shutdown_connections 2024-02-15T18:24:17.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.894+0000 7f85bf59e700 1 --2- 172.21.15.42:0/480584606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85c00f96f0 0x7f85c00fbae0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:17.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.894+0000 7f85bf59e700 1 --2- 172.21.15.42:0/480584606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85c00602b0 0x7f85c00f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:17.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.894+0000 7f85bf59e700 1 -- 172.21.15.42:0/480584606 >> 172.21.15.42:0/480584606 conn(0x7f85c00f0b90 msgr2=0x7f85c00f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:17.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.894+0000 7f85bf59e700 1 -- 172.21.15.42:0/480584606 shutdown_connections 2024-02-15T18:24:17.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.894+0000 7f85bf59e700 1 -- 172.21.15.42:0/480584606 wait complete. 2024-02-15T18:24:17.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.895+0000 7f85bf59e700 1 Processor -- start 2024-02-15T18:24:17.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.895+0000 7f85bf59e700 1 -- start start 2024-02-15T18:24:17.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.895+0000 7f85bf59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85c00602b0 0x7f85c0106e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:17.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.895+0000 7f85bf59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85c00f96f0 0x7f85c0107310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:17.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.895+0000 7f85bf59e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c0107910 con 0x7f85c00602b0 2024-02-15T18:24:17.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.895+0000 7f85bf59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c0108120 con 0x7f85c00f96f0 2024-02-15T18:24:17.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.895+0000 7f85be59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85c00602b0 0x7f85c0106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:17.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.895+0000 7f85be59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85c00602b0 0x7f85c0106e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48320/0 (socket says 172.21.15.42:48320) 2024-02-15T18:24:17.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.895+0000 7f85be59c700 1 -- 172.21.15.42:0/185823205 learned_addr learned my addr 172.21.15.42:0/185823205 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:17.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.895+0000 7f85bdd9b700 1 --2- 172.21.15.42:0/185823205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85c00f96f0 0x7f85c0107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:17.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.895+0000 7f85be59c700 1 -- 172.21.15.42:0/185823205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85c00f96f0 msgr2=0x7f85c0107310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:17.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.895+0000 7f85be59c700 1 --2- 172.21.15.42:0/185823205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85c00f96f0 0x7f85c0107310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:17.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.895+0000 7f85be59c700 1 -- 172.21.15.42:0/185823205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b4005040 con 0x7f85c00602b0 2024-02-15T18:24:17.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.896+0000 7f85be59c700 1 --2- 172.21.15.42:0/185823205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85c00602b0 0x7f85c0106e20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f85b000e7f0 tx=0x7f85b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:17.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.896+0000 7f85aeffd700 1 -- 172.21.15.42:0/185823205 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85b00092e0 con 0x7f85c00602b0 2024-02-15T18:24:17.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.896+0000 7f85bf59e700 1 -- 172.21.15.42:0/185823205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c01083a0 con 0x7f85c00602b0 2024-02-15T18:24:17.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.896+0000 7f85aeffd700 1 -- 172.21.15.42:0/185823205 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85b000f040 con 0x7f85c00602b0 2024-02-15T18:24:17.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.896+0000 7f85aeffd700 1 -- 172.21.15.42:0/185823205 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85b00147e0 con 0x7f85c00602b0 2024-02-15T18:24:17.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.896+0000 7f85bf59e700 1 -- 172.21.15.42:0/185823205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c0108960 con 0x7f85c00602b0 2024-02-15T18:24:17.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.897+0000 7f85aeffd700 1 -- 172.21.15.42:0/185823205 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f85b001b030 con 0x7f85c00602b0 2024-02-15T18:24:17.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.897+0000 7f85bf59e700 1 -- 172.21.15.42:0/185823205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c0059100 con 0x7f85c00602b0 2024-02-15T18:24:17.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.897+0000 7f85aeffd700 1 --2- 172.21.15.42:0/185823205 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f85a805c300 0x7f85a805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:17.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.898+0000 7f85aeffd700 1 -- 172.21.15.42:0/185823205 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f85b0075d80 con 0x7f85c00602b0 2024-02-15T18:24:17.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.898+0000 7f85bdd9b700 1 --2- 172.21.15.42:0/185823205 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f85a805c300 0x7f85a805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:17.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.898+0000 7f85bdd9b700 1 --2- 172.21.15.42:0/185823205 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f85a805c300 0x7f85a805e7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f85b4009720 tx=0x7f85b4016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:17.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:17.901+0000 7f85aeffd700 1 -- 172.21.15.42:0/185823205 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f85b00489d0 con 0x7f85c00602b0 2024-02-15T18:24:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:18.066+0000 7f85bf59e700 1 -- 172.21.15.42:0/185823205 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f85c0045ad0 con 0x7f85c00602b0 2024-02-15T18:24:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:18.067+0000 7f85aeffd700 1 -- 172.21.15.42:0/185823205 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v16)=0 v16) v1 ==== 163+0+0 (secure 0 0 0) 0x7f85b0019070 con 0x7f85c00602b0 2024-02-15T18:24:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:18.068+0000 7f85bf59e700 1 -- 172.21.15.42:0/185823205 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f85a805c300 msgr2=0x7f85a805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:18.068+0000 7f85bf59e700 1 --2- 172.21.15.42:0/185823205 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f85a805c300 0x7f85a805e7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f85b4009720 tx=0x7f85b4016040).stop 2024-02-15T18:24:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:18.068+0000 7f85bf59e700 1 -- 172.21.15.42:0/185823205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85c00602b0 msgr2=0x7f85c0106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:18.068+0000 7f85bf59e700 1 --2- 172.21.15.42:0/185823205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85c00602b0 0x7f85c0106e20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f85b000e7f0 tx=0x7f85b000b040).stop 2024-02-15T18:24:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:18.068+0000 7f85bf59e700 1 -- 172.21.15.42:0/185823205 shutdown_connections 2024-02-15T18:24:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:18.068+0000 7f85bf59e700 1 --2- 172.21.15.42:0/185823205 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f85a805c300 0x7f85a805e7c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:18.068+0000 7f85bf59e700 1 --2- 172.21.15.42:0/185823205 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85c00602b0 0x7f85c0106e20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:18.068+0000 7f85bf59e700 1 --2- 172.21.15.42:0/185823205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85c00f96f0 0x7f85c0107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:18.068+0000 7f85bf59e700 1 -- 172.21.15.42:0/185823205 >> 172.21.15.42:0/185823205 conn(0x7f85c00f0b90 msgr2=0x7f85c00f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:18.068+0000 7f85bf59e700 1 -- 172.21.15.42:0/185823205 shutdown_connections 2024-02-15T18:24:18.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:18.068+0000 7f85bf59e700 1 -- 172.21.15.42:0/185823205 wait complete. 2024-02-15T18:24:18.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:16.826970+0000 mgr.smithi042.kfoihb (mgr.14162) 259 : cluster [DBG] pgmap v223: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 682 B/s wr, 2 op/s 2024-02-15T18:24:18.812 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:16.826970+0000 mgr.smithi042.kfoihb (mgr.14162) 259 : cluster [DBG] pgmap v223: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 682 B/s wr, 2 op/s 2024-02-15T18:24:18.852 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph config set global log_to_journald false --force' 2024-02-15T18:24:20.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:18.827540+0000 mgr.smithi042.kfoihb (mgr.14162) 260 : cluster [DBG] pgmap v224: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-02-15T18:24:20.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:18.827540+0000 mgr.smithi042.kfoihb (mgr.14162) 260 : cluster [DBG] pgmap v224: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-02-15T18:24:20.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.830+0000 7f1c18a02700 1 -- 172.21.15.42:0/556045576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c140f6530 msgr2=0x7f1c140f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:20.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.830+0000 7f1c18a02700 1 --2- 172.21.15.42:0/556045576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c140f6530 0x7f1c140f6950 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f1c04004660 tx=0x7f1c04009b30).stop 2024-02-15T18:24:20.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.830+0000 7f1c18a02700 1 -- 172.21.15.42:0/556045576 shutdown_connections 2024-02-15T18:24:20.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.830+0000 7f1c18a02700 1 --2- 172.21.15.42:0/556045576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c140f77d0 0x7f1c140f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:24:20.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.830+0000 7f1c18a02700 1 --2- 172.21.15.42:0/556045576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c140f6530 0x7f1c140f6950 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:20.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.830+0000 7f1c18a02700 1 -- 172.21.15.42:0/556045576 >> 172.21.15.42:0/556045576 conn(0x7f1c140f1c90 msgr2=0x7f1c140f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:20.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.830+0000 7f1c18a02700 1 -- 172.21.15.42:0/556045576 shutdown_connections 2024-02-15T18:24:20.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.830+0000 7f1c18a02700 1 -- 172.21.15.42:0/556045576 wait complete. 2024-02-15T18:24:20.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.831+0000 7f1c18a02700 1 Processor -- start 2024-02-15T18:24:20.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.831+0000 7f1c18a02700 1 -- start start 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.831+0000 7f1c18a02700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c140f6530 0x7f1c14106ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.831+0000 7f1c18a02700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c140f77d0 0x7f1c14107390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.831+0000 7f1c18a02700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c141079d0 con 0x7f1c140f77d0 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.831+0000 7f1c18a02700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c14108280 con 0x7f1c140f6530 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.831+0000 7f1c1259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c140f77d0 0x7f1c14107390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.831+0000 7f1c1259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c140f77d0 0x7f1c14107390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48340/0 (socket says 172.21.15.42:48340) 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.831+0000 7f1c1259c700 1 -- 172.21.15.42:0/1783648869 learned_addr learned my addr 172.21.15.42:0/1783648869 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.831+0000 7f1c12d9d700 1 --2- 172.21.15.42:0/1783648869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c140f6530 0x7f1c14106ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.832+0000 7f1c1259c700 1 -- 172.21.15.42:0/1783648869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c140f6530 msgr2=0x7f1c14106ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.832+0000 7f1c1259c700 1 --2- 172.21.15.42:0/1783648869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c140f6530 0x7f1c14106ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.832+0000 7f1c1259c700 1 -- 172.21.15.42:0/1783648869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c04005040 con 0x7f1c140f77d0 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.832+0000 7f1c1259c700 1 --2- 172.21.15.42:0/1783648869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c140f77d0 0x7f1c14107390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f1c0800b410 tx=0x7f1c08006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.832+0000 7f1bfb7fe700 1 -- 172.21.15.42:0/1783648869 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c080125f0 con 0x7f1c140f77d0 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.832+0000 7f1c18a02700 1 -- 172.21.15.42:0/1783648869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c14108560 con 0x7f1c140f77d0 2024-02-15T18:24:20.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.832+0000 7f1bfb7fe700 1 -- 172.21.15.42:0/1783648869 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c0800be30 con 0x7f1c140f77d0 2024-02-15T18:24:20.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.832+0000 7f1bfb7fe700 1 -- 172.21.15.42:0/1783648869 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c08002a90 con 0x7f1c140f77d0 2024-02-15T18:24:20.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.832+0000 7f1c18a02700 1 -- 172.21.15.42:0/1783648869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c14108b20 con 0x7f1c140f77d0 2024-02-15T18:24:20.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.833+0000 7f1c18a02700 1 -- 172.21.15.42:0/1783648869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c14059100 con 0x7f1c140f77d0 2024-02-15T18:24:20.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.833+0000 7f1bfb7fe700 1 -- 172.21.15.42:0/1783648869 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f1c0800f030 con 0x7f1c140f77d0 2024-02-15T18:24:20.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.834+0000 7f1bfb7fe700 1 --2- 172.21.15.42:0/1783648869 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f1bfc05c300 0x7f1bfc05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:20.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.834+0000 7f1bfb7fe700 1 -- 172.21.15.42:0/1783648869 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f1c08076320 con 0x7f1c140f77d0 2024-02-15T18:24:20.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.834+0000 7f1c12d9d700 1 --2- 172.21.15.42:0/1783648869 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f1bfc05c300 0x7f1bfc05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:20.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.834+0000 7f1c12d9d700 1 --2- 172.21.15.42:0/1783648869 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f1bfc05c300 0x7f1bfc05e7c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f1c04009f50 tx=0x7f1c04009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:20.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:20.838+0000 7f1bfb7fe700 1 -- 172.21.15.42:0/1783648869 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1c08048fe0 con 0x7f1c140f77d0 2024-02-15T18:24:21.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:21.003+0000 7f1c18a02700 1 -- 172.21.15.42:0/1783648869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f1c140fee80 con 0x7f1c140f77d0 2024-02-15T18:24:21.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:21.007+0000 7f1bfb7fe700 1 -- 172.21.15.42:0/1783648869 <== mon.0 v2:172.21.15.42:3300/0 7 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f1c08048fe0 con 0x7f1c140f77d0 2024-02-15T18:24:21.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:21.007+0000 7f1bfb7fe700 1 -- 172.21.15.42:0/1783648869 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v17)=0 v17) v1 ==== 135+0+0 (secure 0 0 0) 0x7f1c0804c600 con 0x7f1c140f77d0 2024-02-15T18:24:21.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:21.010+0000 7f1c18a02700 1 -- 172.21.15.42:0/1783648869 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f1bfc05c300 msgr2=0x7f1bfc05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:21.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:21.010+0000 7f1c18a02700 1 --2- 172.21.15.42:0/1783648869 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f1bfc05c300 0x7f1bfc05e7c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f1c04009f50 tx=0x7f1c04009b30).stop 2024-02-15T18:24:21.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:21.010+0000 7f1c18a02700 1 -- 172.21.15.42:0/1783648869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c140f77d0 msgr2=0x7f1c14107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:21.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:21.010+0000 7f1c18a02700 1 --2- 172.21.15.42:0/1783648869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c140f77d0 0x7f1c14107390 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f1c0800b410 tx=0x7f1c08006ef0).stop 2024-02-15T18:24:21.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:21.010+0000 7f1c18a02700 1 -- 172.21.15.42:0/1783648869 shutdown_connections 2024-02-15T18:24:21.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:21.010+0000 7f1c18a02700 1 --2- 172.21.15.42:0/1783648869 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f1bfc05c300 0x7f1bfc05e7c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:21.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:21.010+0000 7f1c18a02700 1 --2- 172.21.15.42:0/1783648869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c140f6530 0x7f1c14106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:21.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:21.010+0000 7f1c18a02700 1 --2- 172.21.15.42:0/1783648869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c140f77d0 0x7f1c14107390 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:21.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:21.010+0000 7f1c18a02700 1 -- 172.21.15.42:0/1783648869 >> 172.21.15.42:0/1783648869 conn(0x7f1c140f1c90 msgr2=0x7f1c140fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:21.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:21.010+0000 7f1c18a02700 1 -- 172.21.15.42:0/1783648869 shutdown_connections 2024-02-15T18:24:21.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:21.010+0000 7f1c18a02700 1 -- 172.21.15.42:0/1783648869 wait complete. 2024-02-15T18:24:21.753 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-02-15T18:24:22.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:20.828000+0000 mgr.smithi042.kfoihb (mgr.14162) 261 : cluster [DBG] pgmap v225: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-15T18:24:22.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:21.008446+0000 mon.smithi042 (mon.0) 542 : audit [INF] from='client.? 172.21.15.42:0/1783648869' entity='client.admin' 2024-02-15T18:24:22.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:21.012207+0000 mon.smithi042 (mon.0) 543 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:24:22.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:21.017379+0000 mon.smithi042 (mon.0) 544 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:22.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:20.828000+0000 mgr.smithi042.kfoihb (mgr.14162) 261 : cluster [DBG] pgmap v225: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-15T18:24:22.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:21.008446+0000 mon.smithi042 (mon.0) 542 : audit [INF] from='client.? 172.21.15.42:0/1783648869' entity='client.admin' 2024-02-15T18:24:22.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:21.012207+0000 mon.smithi042 (mon.0) 543 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:24:22.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:21.017379+0000 mon.smithi042 (mon.0) 544 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:23.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.979+0000 7fc9e7fff700 1 -- 172.21.15.42:0/1303389086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e80f5430 msgr2=0x7fc9e80f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.979+0000 7fc9e7fff700 1 --2- 172.21.15.42:0/1303389086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e80f5430 0x7fc9e80f5850 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fc9dc004660 tx=0x7fc9dc009b30).stop 2024-02-15T18:24:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.979+0000 7fc9e7fff700 1 -- 172.21.15.42:0/1303389086 shutdown_connections 2024-02-15T18:24:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.979+0000 7fc9e7fff700 1 --2- 172.21.15.42:0/1303389086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e80f66d0 0x7fc9e80f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.979+0000 7fc9e7fff700 1 --2- 172.21.15.42:0/1303389086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e80f5430 0x7fc9e80f5850 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.979+0000 7fc9e7fff700 1 -- 172.21.15.42:0/1303389086 >> 172.21.15.42:0/1303389086 conn(0x7fc9e80f0b90 msgr2=0x7fc9e80f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.979+0000 7fc9e7fff700 1 -- 172.21.15.42:0/1303389086 shutdown_connections 2024-02-15T18:24:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.979+0000 7fc9e7fff700 1 -- 172.21.15.42:0/1303389086 wait complete. 2024-02-15T18:24:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.979+0000 7fc9e7fff700 1 Processor -- start 2024-02-15T18:24:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.979+0000 7fc9e7fff700 1 -- start start 2024-02-15T18:24:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9e7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e80f5430 0x7fc9e8109b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9e7fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e80f66d0 0x7fc9e810c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9e7fff700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e80603e0 con 0x7fc9e80f5430 2024-02-15T18:24:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9e7fff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e8060520 con 0x7fc9e80f66d0 2024-02-15T18:24:23.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9e6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e80f5430 0x7fc9e8109b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:23.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9e6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e80f5430 0x7fc9e8109b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46572/0 (socket says 172.21.15.42:46572) 2024-02-15T18:24:23.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9e6ffd700 1 -- 172.21.15.42:0/2111583342 learned_addr learned my addr 172.21.15.42:0/2111583342 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:23.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9e67fc700 1 --2- 172.21.15.42:0/2111583342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e80f66d0 0x7fc9e810c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:23.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9e6ffd700 1 -- 172.21.15.42:0/2111583342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e80f66d0 msgr2=0x7fc9e810c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:23.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9e6ffd700 1 --2- 172.21.15.42:0/2111583342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e80f66d0 0x7fc9e810c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:23.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9e6ffd700 1 -- 172.21.15.42:0/2111583342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc005040 con 0x7fc9e80f5430 2024-02-15T18:24:23.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9e6ffd700 1 --2- 172.21.15.42:0/2111583342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e80f5430 0x7fc9e8109b20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fc9dc013fd0 tx=0x7fc9dc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:23.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9cf7fe700 1 -- 172.21.15.42:0/2111583342 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9dc018070 con 0x7fc9e80f5430 2024-02-15T18:24:23.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9e7fff700 1 -- 172.21.15.42:0/2111583342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e810c570 con 0x7fc9e80f5430 2024-02-15T18:24:23.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9cf7fe700 1 -- 172.21.15.42:0/2111583342 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fc9dc007c70 con 0x7fc9e80f5430 2024-02-15T18:24:23.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.980+0000 7fc9cf7fe700 1 -- 172.21.15.42:0/2111583342 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9dc01c620 con 0x7fc9e80f5430 2024-02-15T18:24:23.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.981+0000 7fc9e7fff700 1 -- 172.21.15.42:0/2111583342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e810cad0 con 0x7fc9e80f5430 2024-02-15T18:24:23.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.981+0000 7fc9e7fff700 1 -- 172.21.15.42:0/2111583342 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e8045ad0 con 0x7fc9e80f5430 2024-02-15T18:24:23.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.982+0000 7fc9cf7fe700 1 -- 172.21.15.42:0/2111583342 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fc9dc01d420 con 0x7fc9e80f5430 2024-02-15T18:24:23.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.982+0000 7fc9cf7fe700 1 --2- 172.21.15.42:0/2111583342 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc9d005c300 0x7fc9d005e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:23.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.982+0000 7fc9cf7fe700 1 -- 172.21.15.42:0/2111583342 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fc9dc077c60 con 0x7fc9e80f5430 2024-02-15T18:24:23.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.982+0000 7fc9e67fc700 1 --2- 172.21.15.42:0/2111583342 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc9d005c300 0x7fc9d005e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:23.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.983+0000 7fc9e67fc700 1 --2- 172.21.15.42:0/2111583342 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc9d005c300 0x7fc9d005e7c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fc9d80041d0 tx=0x7fc9d8009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:23.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:23.986+0000 7fc9cf7fe700 1 -- 172.21.15.42:0/2111583342 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc9dc04a9a0 con 0x7fc9e80f5430 2024-02-15T18:24:24.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:24.179+0000 7fc9e7fff700 1 -- 172.21.15.42:0/2111583342 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9e8057f10 con 0x7fc9d005c300 2024-02-15T18:24:24.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:24.184+0000 7fc9cf7fe700 1 -- 172.21.15.42:0/2111583342 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fc9e8057f10 con 0x7fc9d005c300 2024-02-15T18:24:24.183 INFO:teuthology.orchestra.run.smithi042.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:24:24.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:24.186+0000 7fc9e7fff700 1 -- 172.21.15.42:0/2111583342 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc9d005c300 msgr2=0x7fc9d005e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:24.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:24.186+0000 7fc9e7fff700 1 --2- 172.21.15.42:0/2111583342 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc9d005c300 0x7fc9d005e7c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fc9d80041d0 tx=0x7fc9d8009040).stop 2024-02-15T18:24:24.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:24.186+0000 7fc9e7fff700 1 -- 172.21.15.42:0/2111583342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e80f5430 msgr2=0x7fc9e8109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:24.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:24.186+0000 7fc9e7fff700 1 --2- 172.21.15.42:0/2111583342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e80f5430 0x7fc9e8109b20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fc9dc013fd0 tx=0x7fc9dc009b30).stop 2024-02-15T18:24:24.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:24.186+0000 7fc9e7fff700 1 -- 172.21.15.42:0/2111583342 shutdown_connections 2024-02-15T18:24:24.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:24.186+0000 7fc9e7fff700 1 --2- 172.21.15.42:0/2111583342 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc9d005c300 0x7fc9d005e7c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:24.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:24.186+0000 7fc9e7fff700 1 --2- 172.21.15.42:0/2111583342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9e80f5430 0x7fc9e8109b20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:24.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:24.186+0000 7fc9e7fff700 1 --2- 172.21.15.42:0/2111583342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e80f66d0 0x7fc9e810c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:24.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:24.186+0000 7fc9e7fff700 1 -- 172.21.15.42:0/2111583342 >> 172.21.15.42:0/2111583342 conn(0x7fc9e80f0b90 msgr2=0x7fc9e80fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:24.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:24.186+0000 7fc9e7fff700 1 -- 172.21.15.42:0/2111583342 shutdown_connections 2024-02-15T18:24:24.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:24.186+0000 7fc9e7fff700 1 -- 172.21.15.42:0/2111583342 wait complete. 2024-02-15T18:24:24.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:22.828341+0000 mgr.smithi042.kfoihb (mgr.14162) 262 : cluster [DBG] pgmap v226: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-15T18:24:24.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:22.828341+0000 mgr.smithi042.kfoihb (mgr.14162) 262 : cluster [DBG] pgmap v226: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-15T18:24:25.093 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-15T18:24:25.094 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-02-15T18:24:25.094 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; sleep 30 ; done' 2024-02-15T18:24:25.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:24.181331+0000 mgr.smithi042.kfoihb (mgr.14162) 263 : audit [DBG] from='client.14508 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:24:25.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:24:24.182022+0000 mgr.smithi042.kfoihb (mgr.14162) 264 : cephadm [INF] Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:24:25.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:24.185362+0000 mon.smithi042 (mon.0) 545 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:25.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:24.185910+0000 mon.smithi042 (mon.0) 546 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:24:25.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:24.190592+0000 mon.smithi042 (mon.0) 547 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:25.449 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:24:24.193526+0000 mgr.smithi042.kfoihb (mgr.14162) 265 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:24:25.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:24.181331+0000 mgr.smithi042.kfoihb (mgr.14162) 263 : audit [DBG] from='client.14508 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:24:25.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:24:24.182022+0000 mgr.smithi042.kfoihb (mgr.14162) 264 : cephadm [INF] Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:24:25.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:24.185362+0000 mon.smithi042 (mon.0) 545 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:25.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:24.185910+0000 mon.smithi042 (mon.0) 546 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:24:25.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:24.190592+0000 mon.smithi042 (mon.0) 547 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:25.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:24:24.193526+0000 mgr.smithi042.kfoihb (mgr.14162) 265 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-15T18:24:26.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:24.828766+0000 mgr.smithi042.kfoihb (mgr.14162) 266 : cluster [DBG] pgmap v227: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-15T18:24:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:24.828766+0000 mgr.smithi042.kfoihb (mgr.14162) 266 : cluster [DBG] pgmap v227: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-15T18:24:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.238+0000 7f82bf59e700 1 -- 172.21.15.42:0/2579974673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82c00f8540 msgr2=0x7f82c00f46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.238+0000 7f82bf59e700 1 --2- 172.21.15.42:0/2579974673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82c00f8540 0x7f82c00f46c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f82b4004660 tx=0x7f82b4009b30).stop 2024-02-15T18:24:27.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.238+0000 7f82bf59e700 1 -- 172.21.15.42:0/2579974673 shutdown_connections 2024-02-15T18:24:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.238+0000 7f82bf59e700 1 --2- 172.21.15.42:0/2579974673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82c00f8540 0x7f82c00f46c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.238+0000 7f82bf59e700 1 --2- 172.21.15.42:0/2579974673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82c00f7ba0 0x7f82c00f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.238+0000 7f82bf59e700 1 -- 172.21.15.42:0/2579974673 >> 172.21.15.42:0/2579974673 conn(0x7f82c00f0320 msgr2=0x7f82c00f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.238+0000 7f82bf59e700 1 -- 172.21.15.42:0/2579974673 shutdown_connections 2024-02-15T18:24:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.238+0000 7f82bf59e700 1 -- 172.21.15.42:0/2579974673 wait complete. 2024-02-15T18:24:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.239+0000 7f82bf59e700 1 Processor -- start 2024-02-15T18:24:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.239+0000 7f82bf59e700 1 -- start start 2024-02-15T18:24:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.239+0000 7f82bf59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82c00f7ba0 0x7f82c00f7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.239+0000 7f82bf59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82c00f8540 0x7f82c00f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.239+0000 7f82bf59e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c00f5e50 con 0x7f82c00f7ba0 2024-02-15T18:24:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.239+0000 7f82bf59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c00f5f90 con 0x7f82c00f8540 2024-02-15T18:24:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.239+0000 7f82be59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82c00f7ba0 0x7f82c00f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.239+0000 7f82be59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82c00f7ba0 0x7f82c00f7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46588/0 (socket says 172.21.15.42:46588) 2024-02-15T18:24:27.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.239+0000 7f82be59c700 1 -- 172.21.15.42:0/3775175832 learned_addr learned my addr 172.21.15.42:0/3775175832 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:27.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.239+0000 7f82bdd9b700 1 --2- 172.21.15.42:0/3775175832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82c00f8540 0x7f82c00f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.240+0000 7f82be59c700 1 -- 172.21.15.42:0/3775175832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82c00f8540 msgr2=0x7f82c00f58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.240+0000 7f82be59c700 1 --2- 172.21.15.42:0/3775175832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82c00f8540 0x7f82c00f58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.240+0000 7f82be59c700 1 -- 172.21.15.42:0/3775175832 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b4005040 con 0x7f82c00f7ba0 2024-02-15T18:24:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.240+0000 7f82be59c700 1 --2- 172.21.15.42:0/3775175832 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82c00f7ba0 0x7f82c00f7470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f82b000e7c0 tx=0x7f82b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.240+0000 7f82aeffd700 1 -- 172.21.15.42:0/3775175832 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82b00092e0 con 0x7f82c00f7ba0 2024-02-15T18:24:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.240+0000 7f82aeffd700 1 -- 172.21.15.42:0/3775175832 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f82b000f040 con 0x7f82c00f7ba0 2024-02-15T18:24:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.240+0000 7f82bf59e700 1 -- 172.21.15.42:0/3775175832 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c00f6270 con 0x7f82c00f7ba0 2024-02-15T18:24:27.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.240+0000 7f82aeffd700 1 -- 172.21.15.42:0/3775175832 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82b0014920 con 0x7f82c00f7ba0 2024-02-15T18:24:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.240+0000 7f82bf59e700 1 -- 172.21.15.42:0/3775175832 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c010ee20 con 0x7f82c00f7ba0 2024-02-15T18:24:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.241+0000 7f82bf59e700 1 -- 172.21.15.42:0/3775175832 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c0045ad0 con 0x7f82c00f7ba0 2024-02-15T18:24:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.241+0000 7f82aeffd700 1 -- 172.21.15.42:0/3775175832 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f82b001b030 con 0x7f82c00f7ba0 2024-02-15T18:24:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.242+0000 7f82aeffd700 1 --2- 172.21.15.42:0/3775175832 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f82a805c300 0x7f82a805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.242+0000 7f82aeffd700 1 -- 172.21.15.42:0/3775175832 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f82b0075d00 con 0x7f82c00f7ba0 2024-02-15T18:24:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.242+0000 7f82bdd9b700 1 --2- 172.21.15.42:0/3775175832 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f82a805c300 0x7f82a805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:27.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.242+0000 7f82bdd9b700 1 --2- 172.21.15.42:0/3775175832 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f82a805c300 0x7f82a805e7c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f82b4009720 tx=0x7f82b4016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:27.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.246+0000 7f82aeffd700 1 -- 172.21.15.42:0/3775175832 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f82b0079030 con 0x7f82c00f7ba0 2024-02-15T18:24:27.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.484+0000 7f82bf59e700 1 -- 172.21.15.42:0/3775175832 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f82c00fef30 con 0x7f82a805c300 2024-02-15T18:24:27.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.485+0000 7f82aeffd700 1 -- 172.21.15.42:0/3775175832 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+278 (secure 0 0 0) 0x7f82c00fef30 con 0x7f82a805c300 2024-02-15T18:24:27.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.486+0000 7f82bf59e700 1 -- 172.21.15.42:0/3775175832 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f82a805c300 msgr2=0x7f82a805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:27.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.486+0000 7f82bf59e700 1 --2- 172.21.15.42:0/3775175832 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f82a805c300 0x7f82a805e7c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f82b4009720 tx=0x7f82b4016040).stop 2024-02-15T18:24:27.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.486+0000 7f82bf59e700 1 -- 172.21.15.42:0/3775175832 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82c00f7ba0 msgr2=0x7f82c00f7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:27.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.486+0000 7f82bf59e700 1 --2- 172.21.15.42:0/3775175832 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82c00f7ba0 0x7f82c00f7470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f82b000e7c0 tx=0x7f82b000b040).stop 2024-02-15T18:24:27.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.487+0000 7f82bf59e700 1 -- 172.21.15.42:0/3775175832 shutdown_connections 2024-02-15T18:24:27.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.487+0000 7f82bf59e700 1 --2- 172.21.15.42:0/3775175832 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f82a805c300 0x7f82a805e7c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.487+0000 7f82bf59e700 1 --2- 172.21.15.42:0/3775175832 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82c00f7ba0 0x7f82c00f7470 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.487+0000 7f82bf59e700 1 --2- 172.21.15.42:0/3775175832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82c00f8540 0x7f82c00f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.487+0000 7f82bf59e700 1 -- 172.21.15.42:0/3775175832 >> 172.21.15.42:0/3775175832 conn(0x7f82c00f0320 msgr2=0x7f82c00fb430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:27.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.487+0000 7f82bf59e700 1 -- 172.21.15.42:0/3775175832 shutdown_connections 2024-02-15T18:24:27.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.487+0000 7f82bf59e700 1 -- 172.21.15.42:0/3775175832 wait complete. 2024-02-15T18:24:27.497 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:24:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.606+0000 7fe058c88700 1 -- 172.21.15.42:0/3015308806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0540f5710 msgr2=0x7fe0540f5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.606+0000 7fe058c88700 1 --2- 172.21.15.42:0/3015308806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0540f5710 0x7fe0540f5b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fe048004660 tx=0x7fe048009b30).stop 2024-02-15T18:24:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.607+0000 7fe058c88700 1 -- 172.21.15.42:0/3015308806 shutdown_connections 2024-02-15T18:24:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.607+0000 7fe058c88700 1 --2- 172.21.15.42:0/3015308806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0540f5710 0x7fe0540f5b90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.607+0000 7fe058c88700 1 --2- 172.21.15.42:0/3015308806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0540f6a10 0x7fe0540f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.607+0000 7fe058c88700 1 -- 172.21.15.42:0/3015308806 >> 172.21.15.42:0/3015308806 conn(0x7fe0540f0320 msgr2=0x7fe0540f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.607+0000 7fe058c88700 1 -- 172.21.15.42:0/3015308806 shutdown_connections 2024-02-15T18:24:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.607+0000 7fe058c88700 1 -- 172.21.15.42:0/3015308806 wait complete. 2024-02-15T18:24:27.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.607+0000 7fe058c88700 1 Processor -- start 2024-02-15T18:24:27.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.607+0000 7fe058c88700 1 -- start start 2024-02-15T18:24:27.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe058c88700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0540f5710 0x7fe05410bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:27.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe058c88700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0540f6a10 0x7fe05410e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:27.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe058c88700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540f5290 con 0x7fe0540f5710 2024-02-15T18:24:27.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe058c88700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540f53d0 con 0x7fe0540f6a10 2024-02-15T18:24:27.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe0537fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0540f5710 0x7fe05410bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:27.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe0537fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0540f5710 0x7fe05410bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46608/0 (socket says 172.21.15.42:46608) 2024-02-15T18:24:27.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe0537fe700 1 -- 172.21.15.42:0/733406483 learned_addr learned my addr 172.21.15.42:0/733406483 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:27.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe052ffd700 1 --2- 172.21.15.42:0/733406483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0540f6a10 0x7fe05410e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:27.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe0537fe700 1 -- 172.21.15.42:0/733406483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0540f6a10 msgr2=0x7fe05410e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:27.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe0537fe700 1 --2- 172.21.15.42:0/733406483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0540f6a10 0x7fe05410e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe0537fe700 1 -- 172.21.15.42:0/733406483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe048005040 con 0x7fe0540f5710 2024-02-15T18:24:27.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe0537fe700 1 --2- 172.21.15.42:0/733406483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0540f5710 0x7fe05410bf40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fe04400e7c0 tx=0x7fe04400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:27.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe03bfff700 1 -- 172.21.15.42:0/733406483 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0440092e0 con 0x7fe0540f5710 2024-02-15T18:24:27.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe058c88700 1 -- 172.21.15.42:0/733406483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05410e930 con 0x7fe0540f5710 2024-02-15T18:24:27.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe03bfff700 1 -- 172.21.15.42:0/733406483 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fe04400f040 con 0x7fe0540f5710 2024-02-15T18:24:27.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe03bfff700 1 -- 172.21.15.42:0/733406483 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe044014920 con 0x7fe0540f5710 2024-02-15T18:24:27.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.608+0000 7fe058c88700 1 -- 172.21.15.42:0/733406483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05410edb0 con 0x7fe0540f5710 2024-02-15T18:24:27.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.609+0000 7fe058c88700 1 -- 172.21.15.42:0/733406483 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe054045ad0 con 0x7fe0540f5710 2024-02-15T18:24:27.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.610+0000 7fe03bfff700 1 -- 172.21.15.42:0/733406483 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fe04401b030 con 0x7fe0540f5710 2024-02-15T18:24:27.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.611+0000 7fe03bfff700 1 --2- 172.21.15.42:0/733406483 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe03c05c300 0x7fe03c05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:27.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.611+0000 7fe03bfff700 1 -- 172.21.15.42:0/733406483 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fe044075b90 con 0x7fe0540f5710 2024-02-15T18:24:27.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.611+0000 7fe052ffd700 1 --2- 172.21.15.42:0/733406483 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe03c05c300 0x7fe03c05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:27.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.611+0000 7fe052ffd700 1 --2- 172.21.15.42:0/733406483 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe03c05c300 0x7fe03c05e7c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fe048009720 tx=0x7fe048016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:27.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.616+0000 7fe03bfff700 1 -- 172.21.15.42:0/733406483 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe0440488a0 con 0x7fe0540f5710 2024-02-15T18:24:27.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.845+0000 7fe058c88700 1 -- 172.21.15.42:0/733406483 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe054057f10 con 0x7fe03c05c300 2024-02-15T18:24:27.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.846+0000 7fe03bfff700 1 -- 172.21.15.42:0/733406483 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+278 (secure 0 0 0) 0x7fe054057f10 con 0x7fe03c05c300 2024-02-15T18:24:27.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.848+0000 7fe058c88700 1 -- 172.21.15.42:0/733406483 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe03c05c300 msgr2=0x7fe03c05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:27.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.848+0000 7fe058c88700 1 --2- 172.21.15.42:0/733406483 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe03c05c300 0x7fe03c05e7c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fe048009720 tx=0x7fe048016040).stop 2024-02-15T18:24:27.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.848+0000 7fe058c88700 1 -- 172.21.15.42:0/733406483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0540f5710 msgr2=0x7fe05410bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:27.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.848+0000 7fe058c88700 1 --2- 172.21.15.42:0/733406483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0540f5710 0x7fe05410bf40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fe04400e7c0 tx=0x7fe04400b040).stop 2024-02-15T18:24:27.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.848+0000 7fe058c88700 1 -- 172.21.15.42:0/733406483 shutdown_connections 2024-02-15T18:24:27.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.848+0000 7fe058c88700 1 --2- 172.21.15.42:0/733406483 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fe03c05c300 0x7fe03c05e7c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.848+0000 7fe058c88700 1 --2- 172.21.15.42:0/733406483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0540f5710 0x7fe05410bf40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.848+0000 7fe058c88700 1 --2- 172.21.15.42:0/733406483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0540f6a10 0x7fe05410e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.848+0000 7fe058c88700 1 -- 172.21.15.42:0/733406483 >> 172.21.15.42:0/733406483 conn(0x7fe0540f0320 msgr2=0x7fe0540fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:27.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.848+0000 7fe058c88700 1 -- 172.21.15.42:0/733406483 shutdown_connections 2024-02-15T18:24:27.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.848+0000 7fe058c88700 1 -- 172.21.15.42:0/733406483 wait complete. 2024-02-15T18:24:27.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.963+0000 7f75ef298700 1 -- 172.21.15.42:0/1956647507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e80f6550 msgr2=0x7f75e80f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:27.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.963+0000 7f75ef298700 1 --2- 172.21.15.42:0/1956647507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e80f6550 0x7f75e80f69b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f75d8004660 tx=0x7f75d8009b30).stop 2024-02-15T18:24:27.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.963+0000 7f75ef298700 1 -- 172.21.15.42:0/1956647507 shutdown_connections 2024-02-15T18:24:27.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.963+0000 7f75ef298700 1 --2- 172.21.15.42:0/1956647507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e80f6550 0x7f75e80f69b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.963+0000 7f75ef298700 1 --2- 172.21.15.42:0/1956647507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75e80f5bb0 0x7f75e80f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.963+0000 7f75ef298700 1 -- 172.21.15.42:0/1956647507 >> 172.21.15.42:0/1956647507 conn(0x7f75e80f04c0 msgr2=0x7f75e80f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:27.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.963+0000 7f75ef298700 1 -- 172.21.15.42:0/1956647507 shutdown_connections 2024-02-15T18:24:27.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.963+0000 7f75ef298700 1 -- 172.21.15.42:0/1956647507 wait complete. 2024-02-15T18:24:27.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.964+0000 7f75ef298700 1 Processor -- start 2024-02-15T18:24:27.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.964+0000 7f75ef298700 1 -- start start 2024-02-15T18:24:27.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.964+0000 7f75ef298700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75e80f5bb0 0x7f75e8109d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:27.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.964+0000 7f75ef298700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e80f6550 0x7f75e810c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:27.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.964+0000 7f75ef298700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e80f5680 con 0x7f75e80f6550 2024-02-15T18:24:27.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.964+0000 7f75ef298700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e80f57c0 con 0x7f75e80f5bb0 2024-02-15T18:24:27.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.964+0000 7f75ec833700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e80f6550 0x7f75e810c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:27.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.964+0000 7f75ec833700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e80f6550 0x7f75e810c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46636/0 (socket says 172.21.15.42:46636) 2024-02-15T18:24:27.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.964+0000 7f75ec833700 1 -- 172.21.15.42:0/3515828003 learned_addr learned my addr 172.21.15.42:0/3515828003 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:27.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.964+0000 7f75ed034700 1 --2- 172.21.15.42:0/3515828003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75e80f5bb0 0x7f75e8109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:27.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.964+0000 7f75ec833700 1 -- 172.21.15.42:0/3515828003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75e80f5bb0 msgr2=0x7f75e8109d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:27.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.964+0000 7f75ec833700 1 --2- 172.21.15.42:0/3515828003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75e80f5bb0 0x7f75e8109d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:27.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.964+0000 7f75ec833700 1 -- 172.21.15.42:0/3515828003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d8005040 con 0x7f75e80f6550 2024-02-15T18:24:27.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.965+0000 7f75ec833700 1 --2- 172.21.15.42:0/3515828003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e80f6550 0x7f75e810c270 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f75d80043d0 tx=0x7f75d800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:27.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.965+0000 7f75ddffb700 1 -- 172.21.15.42:0/3515828003 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75d8019070 con 0x7f75e80f6550 2024-02-15T18:24:27.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.965+0000 7f75ddffb700 1 -- 172.21.15.42:0/3515828003 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f75d8007e80 con 0x7f75e80f6550 2024-02-15T18:24:27.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.965+0000 7f75ef298700 1 -- 172.21.15.42:0/3515828003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e810c7c0 con 0x7f75e80f6550 2024-02-15T18:24:27.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.965+0000 7f75ddffb700 1 -- 172.21.15.42:0/3515828003 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75d8014d90 con 0x7f75e80f6550 2024-02-15T18:24:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.965+0000 7f75ef298700 1 -- 172.21.15.42:0/3515828003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e810ccf0 con 0x7f75e80f6550 2024-02-15T18:24:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.967+0000 7f75ddffb700 1 -- 172.21.15.42:0/3515828003 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f75d8025070 con 0x7f75e80f6550 2024-02-15T18:24:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.967+0000 7f75ddffb700 1 --2- 172.21.15.42:0/3515828003 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f75d405c2b0 0x7f75d405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.967+0000 7f75ddffb700 1 -- 172.21.15.42:0/3515828003 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f75d8076dc0 con 0x7f75e80f6550 2024-02-15T18:24:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.968+0000 7f75ed034700 1 --2- 172.21.15.42:0/3515828003 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f75d405c2b0 0x7f75d405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:27.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.968+0000 7f75ed034700 1 --2- 172.21.15.42:0/3515828003 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f75d405c2b0 0x7f75d405e770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f75e80f5300 tx=0x7f75e400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:27.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.969+0000 7f75ef298700 1 -- 172.21.15.42:0/3515828003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e80f4150 con 0x7f75e80f6550 2024-02-15T18:24:27.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:27.974+0000 7f75ddffb700 1 -- 172.21.15.42:0/3515828003 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f75d8049b00 con 0x7f75e80f6550 2024-02-15T18:24:28.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.180+0000 7f75ef298700 1 -- 172.21.15.42:0/3515828003 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f75e80fdeb0 con 0x7f75d405c2b0 2024-02-15T18:24:28.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.191+0000 7f75ddffb700 1 -- 172.21.15.42:0/3515828003 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2684 (secure 0 0 0) 0x7f75e80fdeb0 con 0x7f75d405c2b0 2024-02-15T18:24:28.190 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE VERSION IMAGE ID CONTAINER ID 2024-02-15T18:24:28.190 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (4m) 17s ago 6m 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:24:28.190 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (6m) 17s ago 6m 16.2.4 8d91d370c2b8 94d74c33c758 2024-02-15T18:24:28.190 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (4m) 23s ago 4m 16.2.4 8d91d370c2b8 ff0f56f9475a 2024-02-15T18:24:28.190 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (4m) 17s ago 5m 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:24:28.190 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (35s) 17s ago 35s 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:24:28.190 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (41s) 17s ago 40s 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:24:28.190 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (38s) 23s ago 38s 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:24:28.190 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (45s) 23s ago 45s 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:24:28.190 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:9283 running (7m) 17s ago 7m 16.2.4 8d91d370c2b8 a56a503c9572 2024-02-15T18:24:28.191 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (4m) 23s ago 4m 16.2.4 8d91d370c2b8 f2a90ea3fad1 2024-02-15T18:24:28.191 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (7m) 17s ago 7m 16.2.4 8d91d370c2b8 9dcbc5c49e05 2024-02-15T18:24:28.191 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (4m) 23s ago 4m 16.2.4 8d91d370c2b8 c3dd4902e2a2 2024-02-15T18:24:28.191 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (5m) 17s ago 5m 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:24:28.191 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (4m) 23s ago 4m 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:24:28.191 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (4m) 17s ago 4m 16.2.4 8d91d370c2b8 973745b31d5a 2024-02-15T18:24:28.191 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (3m) 17s ago 3m 16.2.4 8d91d370c2b8 2b3ae86111e8 2024-02-15T18:24:28.191 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (3m) 17s ago 3m 16.2.4 8d91d370c2b8 3706071e4698 2024-02-15T18:24:28.191 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (3m) 23s ago 3m 16.2.4 8d91d370c2b8 e6fe8e77c6fe 2024-02-15T18:24:28.191 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (2m) 23s ago 2m 16.2.4 8d91d370c2b8 24582012e234 2024-02-15T18:24:28.191 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (2m) 23s ago 2m 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:24:28.191 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (3m) 17s ago 5m 2.18.1 de242295e225 7fffb9a3be63 2024-02-15T18:24:28.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.193+0000 7f75ef298700 1 -- 172.21.15.42:0/3515828003 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f75d405c2b0 msgr2=0x7f75d405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:28.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.193+0000 7f75ef298700 1 --2- 172.21.15.42:0/3515828003 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f75d405c2b0 0x7f75d405e770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f75e80f5300 tx=0x7f75e400b040).stop 2024-02-15T18:24:28.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.193+0000 7f75ef298700 1 -- 172.21.15.42:0/3515828003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e80f6550 msgr2=0x7f75e810c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:28.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.193+0000 7f75ef298700 1 --2- 172.21.15.42:0/3515828003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e80f6550 0x7f75e810c270 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f75d80043d0 tx=0x7f75d800b040).stop 2024-02-15T18:24:28.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.193+0000 7f75ef298700 1 -- 172.21.15.42:0/3515828003 shutdown_connections 2024-02-15T18:24:28.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.193+0000 7f75ef298700 1 --2- 172.21.15.42:0/3515828003 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f75d405c2b0 0x7f75d405e770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.193+0000 7f75ef298700 1 --2- 172.21.15.42:0/3515828003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75e80f5bb0 0x7f75e8109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.193+0000 7f75ef298700 1 --2- 172.21.15.42:0/3515828003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75e80f6550 0x7f75e810c270 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.193+0000 7f75ef298700 1 -- 172.21.15.42:0/3515828003 >> 172.21.15.42:0/3515828003 conn(0x7f75e80f04c0 msgr2=0x7f75e80fa450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:28.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.193+0000 7f75ef298700 1 -- 172.21.15.42:0/3515828003 shutdown_connections 2024-02-15T18:24:28.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.193+0000 7f75ef298700 1 -- 172.21.15.42:0/3515828003 wait complete. 2024-02-15T18:24:28.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.310+0000 7fc0bc69f700 1 -- 172.21.15.42:0/1564961173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0b40f9880 msgr2=0x7fc0b40fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:28.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.310+0000 7fc0bc69f700 1 --2- 172.21.15.42:0/1564961173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0b40f9880 0x7fc0b40fbc70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fc0a4004660 tx=0x7fc0a4009b30).stop 2024-02-15T18:24:28.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.310+0000 7fc0bc69f700 1 -- 172.21.15.42:0/1564961173 shutdown_connections 2024-02-15T18:24:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.310+0000 7fc0bc69f700 1 --2- 172.21.15.42:0/1564961173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0b40f9880 0x7fc0b40fbc70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.310+0000 7fc0bc69f700 1 --2- 172.21.15.42:0/1564961173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0b40602b0 0x7fc0b40f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.310+0000 7fc0bc69f700 1 -- 172.21.15.42:0/1564961173 >> 172.21.15.42:0/1564961173 conn(0x7fc0b40f0d40 msgr2=0x7fc0b40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.310+0000 7fc0bc69f700 1 -- 172.21.15.42:0/1564961173 shutdown_connections 2024-02-15T18:24:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.310+0000 7fc0bc69f700 1 -- 172.21.15.42:0/1564961173 wait complete. 2024-02-15T18:24:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.310+0000 7fc0bc69f700 1 Processor -- start 2024-02-15T18:24:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.310+0000 7fc0bc69f700 1 -- start start 2024-02-15T18:24:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.311+0000 7fc0bc69f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0b40602b0 0x7fc0b40f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.311+0000 7fc0bc69f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0b40f9880 0x7fc0b40f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.311+0000 7fc0bc69f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b40f5760 con 0x7fc0b40602b0 2024-02-15T18:24:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.311+0000 7fc0bc69f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b40f58a0 con 0x7fc0b40f9880 2024-02-15T18:24:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.311+0000 7fc0ba43b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0b40602b0 0x7fc0b40f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.311+0000 7fc0ba43b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0b40602b0 0x7fc0b40f6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46652/0 (socket says 172.21.15.42:46652) 2024-02-15T18:24:28.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.311+0000 7fc0ba43b700 1 -- 172.21.15.42:0/173014529 learned_addr learned my addr 172.21.15.42:0/173014529 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.311+0000 7fc0b9c3a700 1 --2- 172.21.15.42:0/173014529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0b40f9880 0x7fc0b40f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.311+0000 7fc0ba43b700 1 -- 172.21.15.42:0/173014529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0b40f9880 msgr2=0x7fc0b40f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.311+0000 7fc0ba43b700 1 --2- 172.21.15.42:0/173014529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0b40f9880 0x7fc0b40f5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.311+0000 7fc0ba43b700 1 -- 172.21.15.42:0/173014529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a4005040 con 0x7fc0b40602b0 2024-02-15T18:24:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.311+0000 7fc0b9c3a700 1 --2- 172.21.15.42:0/173014529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0b40f9880 0x7fc0b40f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:24:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.311+0000 7fc0ba43b700 1 --2- 172.21.15.42:0/173014529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0b40602b0 0x7fc0b40f6d80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fc0b000e7f0 tx=0x7fc0b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.312+0000 7fc0aaffd700 1 -- 172.21.15.42:0/173014529 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0b00092e0 con 0x7fc0b40602b0 2024-02-15T18:24:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.312+0000 7fc0bc69f700 1 -- 172.21.15.42:0/173014529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b40f5b80 con 0x7fc0b40602b0 2024-02-15T18:24:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.312+0000 7fc0aaffd700 1 -- 172.21.15.42:0/173014529 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fc0b000f040 con 0x7fc0b40602b0 2024-02-15T18:24:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.312+0000 7fc0aaffd700 1 -- 172.21.15.42:0/173014529 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0b0014880 con 0x7fc0b40602b0 2024-02-15T18:24:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.312+0000 7fc0bc69f700 1 -- 172.21.15.42:0/173014529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b410efc0 con 0x7fc0b40602b0 2024-02-15T18:24:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.313+0000 7fc0bc69f700 1 -- 172.21.15.42:0/173014529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b4045ad0 con 0x7fc0b40602b0 2024-02-15T18:24:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.313+0000 7fc0aaffd700 1 -- 172.21.15.42:0/173014529 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fc0b001b030 con 0x7fc0b40602b0 2024-02-15T18:24:28.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.314+0000 7fc0aaffd700 1 --2- 172.21.15.42:0/173014529 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc0a005c2b0 0x7fc0a005e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:28.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.314+0000 7fc0aaffd700 1 -- 172.21.15.42:0/173014529 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fc0b0075cd0 con 0x7fc0b40602b0 2024-02-15T18:24:28.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.314+0000 7fc0b9c3a700 1 --2- 172.21.15.42:0/173014529 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc0a005c2b0 0x7fc0a005e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:28.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.314+0000 7fc0b9c3a700 1 --2- 172.21.15.42:0/173014529 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc0a005c2b0 0x7fc0a005e770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fc0a4009720 tx=0x7fc0a400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:28.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.318+0000 7fc0aaffd700 1 -- 172.21.15.42:0/173014529 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc0b0048a40 con 0x7fc0b40602b0 2024-02-15T18:24:28.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:26.829263+0000 mgr.smithi042.kfoihb (mgr.14162) 267 : cluster [DBG] pgmap v228: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 85 B/s wr, 1 op/s 2024-02-15T18:24:28.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:27.485783+0000 mgr.smithi042.kfoihb (mgr.14162) 268 : audit [DBG] from='client.14512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:24:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.562+0000 7fc0bc69f700 1 -- 172.21.15.42:0/173014529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc0b40fef90 con 0x7fc0b40602b0 2024-02-15T18:24:28.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.563+0000 7fc0aaffd700 1 -- 172.21.15.42:0/173014529 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+573 (secure 0 0 0) 0x7fc0b0019070 con 0x7fc0b40602b0 2024-02-15T18:24:28.562 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:24:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:24:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:24:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:24:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:24:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:24:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:24:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:24:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:24:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:24:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:24:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:24:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:24:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:24:28.563 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 14 2024-02-15T18:24:28.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:24:28.564 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:24:28.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.565+0000 7fc0bc69f700 1 -- 172.21.15.42:0/173014529 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc0a005c2b0 msgr2=0x7fc0a005e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:28.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.565+0000 7fc0bc69f700 1 --2- 172.21.15.42:0/173014529 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc0a005c2b0 0x7fc0a005e770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fc0a4009720 tx=0x7fc0a400b040).stop 2024-02-15T18:24:28.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.565+0000 7fc0bc69f700 1 -- 172.21.15.42:0/173014529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0b40602b0 msgr2=0x7fc0b40f6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:28.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.565+0000 7fc0bc69f700 1 --2- 172.21.15.42:0/173014529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0b40602b0 0x7fc0b40f6d80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fc0b000e7f0 tx=0x7fc0b000b040).stop 2024-02-15T18:24:28.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.565+0000 7fc0bc69f700 1 -- 172.21.15.42:0/173014529 shutdown_connections 2024-02-15T18:24:28.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.565+0000 7fc0bc69f700 1 --2- 172.21.15.42:0/173014529 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc0a005c2b0 0x7fc0a005e770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.565+0000 7fc0bc69f700 1 --2- 172.21.15.42:0/173014529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc0b40602b0 0x7fc0b40f6d80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.565+0000 7fc0bc69f700 1 --2- 172.21.15.42:0/173014529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0b40f9880 0x7fc0b40f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.565+0000 7fc0bc69f700 1 -- 172.21.15.42:0/173014529 >> 172.21.15.42:0/173014529 conn(0x7fc0b40f0d40 msgr2=0x7fc0b40f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:28.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.565+0000 7fc0bc69f700 1 -- 172.21.15.42:0/173014529 shutdown_connections 2024-02-15T18:24:28.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.565+0000 7fc0bc69f700 1 -- 172.21.15.42:0/173014529 wait complete. 2024-02-15T18:24:28.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.682+0000 7f368d505700 1 -- 172.21.15.42:0/2371242648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36880f66d0 msgr2=0x7f36880f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:28.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.682+0000 7f368d505700 1 --2- 172.21.15.42:0/2371242648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36880f66d0 0x7f36880f6b30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f367c004660 tx=0x7f367c009b30).stop 2024-02-15T18:24:28.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.682+0000 7f368d505700 1 -- 172.21.15.42:0/2371242648 shutdown_connections 2024-02-15T18:24:28.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.682+0000 7f368d505700 1 --2- 172.21.15.42:0/2371242648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36880f66d0 0x7f36880f6b30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.682+0000 7f368d505700 1 --2- 172.21.15.42:0/2371242648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36880f5430 0x7f36880f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.682+0000 7f368d505700 1 -- 172.21.15.42:0/2371242648 >> 172.21.15.42:0/2371242648 conn(0x7f36880f0b90 msgr2=0x7f36880f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:28.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.682+0000 7f368d505700 1 -- 172.21.15.42:0/2371242648 shutdown_connections 2024-02-15T18:24:28.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.682+0000 7f368d505700 1 -- 172.21.15.42:0/2371242648 wait complete. 2024-02-15T18:24:28.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.682+0000 7f368d505700 1 Processor -- start 2024-02-15T18:24:28.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.683+0000 7f368d505700 1 -- start start 2024-02-15T18:24:28.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.683+0000 7f368d505700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36880f5430 0x7f3688109ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:28.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.683+0000 7f368d505700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36880f66d0 0x7f368810bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:28.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.683+0000 7f368d505700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36880603c0 con 0x7f36880f66d0 2024-02-15T18:24:28.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.683+0000 7f368d505700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3688060500 con 0x7f36880f5430 2024-02-15T18:24:28.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.683+0000 7f36877fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36880f66d0 0x7f368810bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:28.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.683+0000 7f36877fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36880f66d0 0x7f368810bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46668/0 (socket says 172.21.15.42:46668) 2024-02-15T18:24:28.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.683+0000 7f36877fe700 1 -- 172.21.15.42:0/2050758003 learned_addr learned my addr 172.21.15.42:0/2050758003 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:28.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.683+0000 7f3687fff700 1 --2- 172.21.15.42:0/2050758003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36880f5430 0x7f3688109ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:28.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.683+0000 7f36877fe700 1 -- 172.21.15.42:0/2050758003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36880f5430 msgr2=0x7f3688109ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:28.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.683+0000 7f36877fe700 1 --2- 172.21.15.42:0/2050758003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36880f5430 0x7f3688109ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.683+0000 7f36877fe700 1 -- 172.21.15.42:0/2050758003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c005040 con 0x7f36880f66d0 2024-02-15T18:24:28.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.683+0000 7f36877fe700 1 --2- 172.21.15.42:0/2050758003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36880f66d0 0x7f368810bfd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f367c009f50 tx=0x7f367c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:28.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.684+0000 7f3684ff9700 1 -- 172.21.15.42:0/2050758003 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f367c018070 con 0x7f36880f66d0 2024-02-15T18:24:28.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.684+0000 7f368d505700 1 -- 172.21.15.42:0/2050758003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368810c520 con 0x7f36880f66d0 2024-02-15T18:24:28.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.684+0000 7f3684ff9700 1 -- 172.21.15.42:0/2050758003 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f367c007ba0 con 0x7f36880f66d0 2024-02-15T18:24:28.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.684+0000 7f3684ff9700 1 -- 172.21.15.42:0/2050758003 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f367c01c420 con 0x7f36880f66d0 2024-02-15T18:24:28.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.684+0000 7f368d505700 1 -- 172.21.15.42:0/2050758003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368810ca50 con 0x7f36880f66d0 2024-02-15T18:24:28.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.685+0000 7f3684ff9700 1 -- 172.21.15.42:0/2050758003 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f367c01d420 con 0x7f36880f66d0 2024-02-15T18:24:28.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.685+0000 7f368d505700 1 -- 172.21.15.42:0/2050758003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3688059100 con 0x7f36880f66d0 2024-02-15T18:24:28.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.685+0000 7f3684ff9700 1 --2- 172.21.15.42:0/2050758003 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f367005c300 0x7f367005e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:28.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.685+0000 7f3684ff9700 1 -- 172.21.15.42:0/2050758003 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f367c077ad0 con 0x7f36880f66d0 2024-02-15T18:24:28.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.686+0000 7f3687fff700 1 --2- 172.21.15.42:0/2050758003 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f367005c300 0x7f367005e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:28.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.686+0000 7f3687fff700 1 --2- 172.21.15.42:0/2050758003 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f367005c300 0x7f367005e7c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f3688060040 tx=0x7f367800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:28.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.690+0000 7f3684ff9700 1 -- 172.21.15.42:0/2050758003 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f367c04a7b0 con 0x7f36880f66d0 2024-02-15T18:24:28.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:26.829263+0000 mgr.smithi042.kfoihb (mgr.14162) 267 : cluster [DBG] pgmap v228: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 85 B/s wr, 1 op/s 2024-02-15T18:24:28.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:27.485783+0000 mgr.smithi042.kfoihb (mgr.14162) 268 : audit [DBG] from='client.14512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:24:28.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.923+0000 7f368d505700 1 -- 172.21.15.42:0/2050758003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3688045ad0 con 0x7f36880f66d0 2024-02-15T18:24:28.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.923+0000 7f3684ff9700 1 -- 172.21.15.42:0/2050758003 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1626 (secure 0 0 0) 0x7f367c04ddd0 con 0x7f36880f66d0 2024-02-15T18:24:28.922 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:24:28.922 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:24:28.922 INFO:teuthology.orchestra.run.smithi042.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:24:28.923 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:24:28.923 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:24:28.923 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:24:28.923 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:24:28.923 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:24:28.923 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:24:28.923 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:24:28.923 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:24:28.923 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:24:28.923 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:24:28.923 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:24:28.923 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:24:28.924 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:24:28.924 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:24:28.924 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:24:28.924 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:24:28.924 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:24:28.924 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:24:28.924 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:24:28.924 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:24:28.924 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:24:28.924 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:24:28.924 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:24:28.924 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:24:28.924 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:24:28.925 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:24:28.925 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:24:28.925 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:24:28.925 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000]] 2024-02-15T18:24:28.925 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509]] 2024-02-15T18:24:28.925 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:24:28.925 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:24:28.925 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:24:28.925 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:24:28.925 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366]] 2024-02-15T18:24:28.925 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796]] 2024-02-15T18:24:28.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.925+0000 7f368d505700 1 -- 172.21.15.42:0/2050758003 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f367005c300 msgr2=0x7f367005e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:28.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.925+0000 7f368d505700 1 --2- 172.21.15.42:0/2050758003 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f367005c300 0x7f367005e7c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f3688060040 tx=0x7f367800b040).stop 2024-02-15T18:24:28.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.925+0000 7f368d505700 1 -- 172.21.15.42:0/2050758003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36880f66d0 msgr2=0x7f368810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:28.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.925+0000 7f368d505700 1 --2- 172.21.15.42:0/2050758003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36880f66d0 0x7f368810bfd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f367c009f50 tx=0x7f367c009b30).stop 2024-02-15T18:24:28.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.925+0000 7f368d505700 1 -- 172.21.15.42:0/2050758003 shutdown_connections 2024-02-15T18:24:28.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.925+0000 7f368d505700 1 --2- 172.21.15.42:0/2050758003 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f367005c300 0x7f367005e7c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.925+0000 7f368d505700 1 --2- 172.21.15.42:0/2050758003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36880f5430 0x7f3688109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.925+0000 7f368d505700 1 --2- 172.21.15.42:0/2050758003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36880f66d0 0x7f368810bfd0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:28.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.925+0000 7f368d505700 1 -- 172.21.15.42:0/2050758003 >> 172.21.15.42:0/2050758003 conn(0x7f36880f0b90 msgr2=0x7f36880fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:28.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.925+0000 7f368d505700 1 -- 172.21.15.42:0/2050758003 shutdown_connections 2024-02-15T18:24:28.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:28.925+0000 7f368d505700 1 -- 172.21.15.42:0/2050758003 wait complete. 2024-02-15T18:24:28.927 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:24:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.050+0000 7f45e359e700 1 -- 172.21.15.42:0/433627895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e4060310 msgr2=0x7f45e40f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.050+0000 7f45e359e700 1 --2- 172.21.15.42:0/433627895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e4060310 0x7f45e40f9110 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f45d4004660 tx=0x7f45d4009b30).stop 2024-02-15T18:24:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.050+0000 7f45e359e700 1 -- 172.21.15.42:0/433627895 shutdown_connections 2024-02-15T18:24:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.050+0000 7f45e359e700 1 --2- 172.21.15.42:0/433627895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45e40f9690 0x7f45e40fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.050+0000 7f45e359e700 1 --2- 172.21.15.42:0/433627895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e4060310 0x7f45e40f9110 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.050+0000 7f45e359e700 1 -- 172.21.15.42:0/433627895 >> 172.21.15.42:0/433627895 conn(0x7f45e40f0b90 msgr2=0x7f45e40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.050+0000 7f45e359e700 1 -- 172.21.15.42:0/433627895 shutdown_connections 2024-02-15T18:24:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.050+0000 7f45e359e700 1 -- 172.21.15.42:0/433627895 wait complete. 2024-02-15T18:24:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.050+0000 7f45e359e700 1 Processor -- start 2024-02-15T18:24:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.050+0000 7f45e359e700 1 -- start start 2024-02-15T18:24:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.051+0000 7f45e359e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e4060310 0x7f45e4106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.051+0000 7f45e359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45e40f9690 0x7f45e4107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.051+0000 7f45e359e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e4107900 con 0x7f45e4060310 2024-02-15T18:24:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.051+0000 7f45e359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e4108110 con 0x7f45e40f9690 2024-02-15T18:24:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.051+0000 7f45e259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e4060310 0x7f45e4106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.051+0000 7f45e259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e4060310 0x7f45e4106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46684/0 (socket says 172.21.15.42:46684) 2024-02-15T18:24:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.051+0000 7f45e259c700 1 -- 172.21.15.42:0/3871952525 learned_addr learned my addr 172.21.15.42:0/3871952525 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.051+0000 7f45e1d9b700 1 --2- 172.21.15.42:0/3871952525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45e40f9690 0x7f45e4107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.051+0000 7f45e259c700 1 -- 172.21.15.42:0/3871952525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45e40f9690 msgr2=0x7f45e4107300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.051+0000 7f45e259c700 1 --2- 172.21.15.42:0/3871952525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45e40f9690 0x7f45e4107300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.051+0000 7f45e259c700 1 -- 172.21.15.42:0/3871952525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d4005040 con 0x7f45e4060310 2024-02-15T18:24:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.051+0000 7f45e259c700 1 --2- 172.21.15.42:0/3871952525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e4060310 0x7f45e4106e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f45d4000c00 tx=0x7f45d400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.051+0000 7f45d2ffd700 1 -- 172.21.15.42:0/3871952525 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45d4019070 con 0x7f45e4060310 2024-02-15T18:24:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.051+0000 7f45d2ffd700 1 -- 172.21.15.42:0/3871952525 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f45d4007e80 con 0x7f45e4060310 2024-02-15T18:24:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.052+0000 7f45e359e700 1 -- 172.21.15.42:0/3871952525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e4108330 con 0x7f45e4060310 2024-02-15T18:24:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.052+0000 7f45d2ffd700 1 -- 172.21.15.42:0/3871952525 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45d4014db0 con 0x7f45e4060310 2024-02-15T18:24:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.052+0000 7f45e359e700 1 -- 172.21.15.42:0/3871952525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e4108890 con 0x7f45e4060310 2024-02-15T18:24:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.053+0000 7f45e359e700 1 -- 172.21.15.42:0/3871952525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e4059100 con 0x7f45e4060310 2024-02-15T18:24:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.053+0000 7f45d2ffd700 1 -- 172.21.15.42:0/3871952525 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f45d4025070 con 0x7f45e4060310 2024-02-15T18:24:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.053+0000 7f45d2ffd700 1 --2- 172.21.15.42:0/3871952525 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f45cc05c300 0x7f45cc05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.053+0000 7f45d2ffd700 1 -- 172.21.15.42:0/3871952525 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f45d4076ca0 con 0x7f45e4060310 2024-02-15T18:24:29.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.054+0000 7f45e1d9b700 1 --2- 172.21.15.42:0/3871952525 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f45cc05c300 0x7f45cc05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:29.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.054+0000 7f45e1d9b700 1 --2- 172.21.15.42:0/3871952525 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f45cc05c300 0x7f45cc05e7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f45d8009fd0 tx=0x7f45d800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:29.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.058+0000 7f45d2ffd700 1 -- 172.21.15.42:0/3871952525 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f45d407b070 con 0x7f45e4060310 2024-02-15T18:24:29.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.287+0000 7f45e359e700 1 -- 172.21.15.42:0/3871952525 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f45e40fef00 con 0x7f45cc05c300 2024-02-15T18:24:29.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.288+0000 7f45d2ffd700 1 -- 172.21.15.42:0/3871952525 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+278 (secure 0 0 0) 0x7f45e40fef00 con 0x7f45cc05c300 2024-02-15T18:24:29.287 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:24:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1", 2024-02-15T18:24:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:24:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [], 2024-02-15T18:24:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "", 2024-02-15T18:24:29.288 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 image" 2024-02-15T18:24:29.288 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:24:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.290+0000 7f45e359e700 1 -- 172.21.15.42:0/3871952525 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f45cc05c300 msgr2=0x7f45cc05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.290+0000 7f45e359e700 1 --2- 172.21.15.42:0/3871952525 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f45cc05c300 0x7f45cc05e7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f45d8009fd0 tx=0x7f45d800b040).stop 2024-02-15T18:24:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.290+0000 7f45e359e700 1 -- 172.21.15.42:0/3871952525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e4060310 msgr2=0x7f45e4106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.290+0000 7f45e359e700 1 --2- 172.21.15.42:0/3871952525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e4060310 0x7f45e4106e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f45d4000c00 tx=0x7f45d400b040).stop 2024-02-15T18:24:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.290+0000 7f45e359e700 1 -- 172.21.15.42:0/3871952525 shutdown_connections 2024-02-15T18:24:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.290+0000 7f45e359e700 1 --2- 172.21.15.42:0/3871952525 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f45cc05c300 0x7f45cc05e7c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.290+0000 7f45e359e700 1 --2- 172.21.15.42:0/3871952525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f45e4060310 0x7f45e4106e10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.290+0000 7f45e359e700 1 --2- 172.21.15.42:0/3871952525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f45e40f9690 0x7f45e4107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:29.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.290+0000 7f45e359e700 1 -- 172.21.15.42:0/3871952525 >> 172.21.15.42:0/3871952525 conn(0x7f45e40f0b90 msgr2=0x7f45e40f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:29.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.290+0000 7f45e359e700 1 -- 172.21.15.42:0/3871952525 shutdown_connections 2024-02-15T18:24:29.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:29.290+0000 7f45e359e700 1 -- 172.21.15.42:0/3871952525 wait complete. 2024-02-15T18:24:29.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:27.847045+0000 mgr.smithi042.kfoihb (mgr.14162) 269 : audit [DBG] from='client.14516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:24:29.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:28.182020+0000 mgr.smithi042.kfoihb (mgr.14162) 270 : audit [DBG] from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:24:29.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:28.564189+0000 mon.smithi042 (mon.0) 548 : audit [DBG] from='client.? 172.21.15.42:0/173014529' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:24:29.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:28.924257+0000 mon.smithi042 (mon.0) 549 : audit [DBG] from='client.? 172.21.15.42:0/2050758003' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:24:29.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:27.847045+0000 mgr.smithi042.kfoihb (mgr.14162) 269 : audit [DBG] from='client.14516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:24:29.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:28.182020+0000 mgr.smithi042.kfoihb (mgr.14162) 270 : audit [DBG] from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:24:29.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:28.564189+0000 mon.smithi042 (mon.0) 548 : audit [DBG] from='client.? 172.21.15.42:0/173014529' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:24:29.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:28.924257+0000 mon.smithi042 (mon.0) 549 : audit [DBG] from='client.? 172.21.15.42:0/2050758003' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:24:30.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:28.829693+0000 mgr.smithi042.kfoihb (mgr.14162) 271 : cluster [DBG] pgmap v229: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 2 op/s 2024-02-15T18:24:30.562 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:29.289185+0000 mgr.smithi042.kfoihb (mgr.14162) 272 : audit [DBG] from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:24:30.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:28.829693+0000 mgr.smithi042.kfoihb (mgr.14162) 271 : cluster [DBG] pgmap v229: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 2 op/s 2024-02-15T18:24:30.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:29.289185+0000 mgr.smithi042.kfoihb (mgr.14162) 272 : audit [DBG] from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:24:32.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:30.830091+0000 mgr.smithi042.kfoihb (mgr.14162) 273 : cluster [DBG] pgmap v230: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-02-15T18:24:32.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:30.830091+0000 mgr.smithi042.kfoihb (mgr.14162) 273 : cluster [DBG] pgmap v230: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-02-15T18:24:34.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:32.830425+0000 mgr.smithi042.kfoihb (mgr.14162) 274 : cluster [DBG] pgmap v231: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:24:34.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:32.830425+0000 mgr.smithi042.kfoihb (mgr.14162) 274 : cluster [DBG] pgmap v231: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:24:36.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:34.830798+0000 mgr.smithi042.kfoihb (mgr.14162) 275 : cluster [DBG] pgmap v232: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:24:36.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:34.830798+0000 mgr.smithi042.kfoihb (mgr.14162) 275 : cluster [DBG] pgmap v232: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:24:38.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:36.831201+0000 mgr.smithi042.kfoihb (mgr.14162) 276 : cluster [DBG] pgmap v233: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:24:38.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:36.831201+0000 mgr.smithi042.kfoihb (mgr.14162) 276 : cluster [DBG] pgmap v233: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:24:40.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:38.831618+0000 mgr.smithi042.kfoihb (mgr.14162) 277 : cluster [DBG] pgmap v234: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:24:40.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:38.831618+0000 mgr.smithi042.kfoihb (mgr.14162) 277 : cluster [DBG] pgmap v234: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:24:42.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:40.832035+0000 mgr.smithi042.kfoihb (mgr.14162) 278 : cluster [DBG] pgmap v235: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:24:42.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:40.832035+0000 mgr.smithi042.kfoihb (mgr.14162) 278 : cluster [DBG] pgmap v235: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:24:44.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:42.832410+0000 mgr.smithi042.kfoihb (mgr.14162) 279 : cluster [DBG] pgmap v236: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:24:44.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:42.832410+0000 mgr.smithi042.kfoihb (mgr.14162) 279 : cluster [DBG] pgmap v236: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:24:46.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:46 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:44.832817+0000 mgr.smithi042.kfoihb (mgr.14162) 280 : cluster [DBG] pgmap v237: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:24:46.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:44.832817+0000 mgr.smithi042.kfoihb (mgr.14162) 280 : cluster [DBG] pgmap v237: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:24:48.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:46.833226+0000 mgr.smithi042.kfoihb (mgr.14162) 281 : cluster [DBG] pgmap v238: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:24:48.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:46.833226+0000 mgr.smithi042.kfoihb (mgr.14162) 281 : cluster [DBG] pgmap v238: 65 pgs: 65 active+clean; 7.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:24:50.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:48.833656+0000 mgr.smithi042.kfoihb (mgr.14162) 282 : cluster [DBG] pgmap v239: 65 pgs: 65 active+clean; 9.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-15T18:24:50.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:48.833656+0000 mgr.smithi042.kfoihb (mgr.14162) 282 : cluster [DBG] pgmap v239: 65 pgs: 65 active+clean; 9.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-15T18:24:52.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:50.834273+0000 mgr.smithi042.kfoihb (mgr.14162) 283 : cluster [DBG] pgmap v240: 65 pgs: 65 active+clean; 9.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-02-15T18:24:52.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:50.834273+0000 mgr.smithi042.kfoihb (mgr.14162) 283 : cluster [DBG] pgmap v240: 65 pgs: 65 active+clean; 9.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-02-15T18:24:54.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:52.834770+0000 mgr.smithi042.kfoihb (mgr.14162) 284 : cluster [DBG] pgmap v241: 65 pgs: 65 active+clean; 9.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-02-15T18:24:54.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:53.023422+0000 mon.smithi042 (mon.0) 550 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:54.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:24:53.023847+0000 mgr.smithi042.kfoihb (mgr.14162) 285 : cephadm [INF] Upgrade: Target is version 16.2.14-521-g0e714d9a (pacific) 2024-02-15T18:24:54.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:24:53.023914+0000 mgr.smithi042.kfoihb (mgr.14162) 286 : cephadm [INF] Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc'] 2024-02-15T18:24:54.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:53.024600+0000 mon.smithi042 (mon.0) 551 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:24:54.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:24:53.025419+0000 mgr.smithi042.kfoihb (mgr.14162) 287 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi042.kfoihb) 2024-02-15T18:24:54.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:24:53.252663+0000 mgr.smithi042.kfoihb (mgr.14162) 288 : cephadm [INF] Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc on smithi191 2024-02-15T18:24:54.452 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:52.834770+0000 mgr.smithi042.kfoihb (mgr.14162) 284 : cluster [DBG] pgmap v241: 65 pgs: 65 active+clean; 9.3 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-02-15T18:24:54.452 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:53.023422+0000 mon.smithi042 (mon.0) 550 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:24:54.453 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:24:53.023847+0000 mgr.smithi042.kfoihb (mgr.14162) 285 : cephadm [INF] Upgrade: Target is version 16.2.14-521-g0e714d9a (pacific) 2024-02-15T18:24:54.453 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:24:53.023914+0000 mgr.smithi042.kfoihb (mgr.14162) 286 : cephadm [INF] Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc'] 2024-02-15T18:24:54.453 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:53.024600+0000 mon.smithi042 (mon.0) 551 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:24:54.453 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:24:53.025419+0000 mgr.smithi042.kfoihb (mgr.14162) 287 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi042.kfoihb) 2024-02-15T18:24:54.453 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:24:53.252663+0000 mgr.smithi042.kfoihb (mgr.14162) 288 : cephadm [INF] Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc on smithi191 2024-02-15T18:24:56.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:54.835442+0000 mgr.smithi042.kfoihb (mgr.14162) 289 : cluster [DBG] pgmap v242: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-15T18:24:56.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:54.835442+0000 mgr.smithi042.kfoihb (mgr.14162) 289 : cluster [DBG] pgmap v242: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-15T18:24:57.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:56.933640+0000 mon.smithi042 (mon.0) 552 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:24:57.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:56.933899+0000 mon.smithi042 (mon.0) 553 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:24:57.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:56.933640+0000 mon.smithi042 (mon.0) 552 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:24:57.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:56.933899+0000 mon.smithi042 (mon.0) 553 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:24:58.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:24:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:56.835892+0000 mgr.smithi042.kfoihb (mgr.14162) 290 : cluster [DBG] pgmap v243: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-02-15T18:24:58.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:24:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:56.835892+0000 mgr.smithi042.kfoihb (mgr.14162) 290 : cluster [DBG] pgmap v243: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-02-15T18:24:59.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.410+0000 7f93e0bc1700 1 -- 172.21.15.42:0/2830199141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93dc0f5ed0 msgr2=0x7f93dc0f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:59.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.410+0000 7f93e0bc1700 1 --2- 172.21.15.42:0/2830199141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93dc0f5ed0 0x7f93dc0f6900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f93d0004660 tx=0x7f93d0009b30).stop 2024-02-15T18:24:59.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.410+0000 7f93e0bc1700 1 -- 172.21.15.42:0/2830199141 shutdown_connections 2024-02-15T18:24:59.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.410+0000 7f93e0bc1700 1 --2- 172.21.15.42:0/2830199141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93dc0f5ed0 0x7f93dc0f6900 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:59.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.410+0000 7f93e0bc1700 1 --2- 172.21.15.42:0/2830199141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93dc0f55c0 0x7f93dc0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:59.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.410+0000 7f93e0bc1700 1 -- 172.21.15.42:0/2830199141 >> 172.21.15.42:0/2830199141 conn(0x7f93dc0f0de0 msgr2=0x7f93dc0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:59.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.410+0000 7f93e0bc1700 1 -- 172.21.15.42:0/2830199141 shutdown_connections 2024-02-15T18:24:59.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.410+0000 7f93e0bc1700 1 -- 172.21.15.42:0/2830199141 wait complete. 2024-02-15T18:24:59.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.411+0000 7f93e0bc1700 1 Processor -- start 2024-02-15T18:24:59.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.411+0000 7f93e0bc1700 1 -- start start 2024-02-15T18:24:59.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.411+0000 7f93e0bc1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93dc0f55c0 0x7f93dc109ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:59.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.411+0000 7f93e0bc1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93dc0f5ed0 0x7f93dc10c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:59.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.411+0000 7f93e0bc1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc0604a0 con 0x7f93dc0f5ed0 2024-02-15T18:24:59.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.411+0000 7f93e0bc1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc0605e0 con 0x7f93dc0f55c0 2024-02-15T18:24:59.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.411+0000 7f93d9d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93dc0f5ed0 0x7f93dc10c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:59.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.411+0000 7f93da59c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93dc0f55c0 0x7f93dc109ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:59.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.411+0000 7f93d9d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93dc0f5ed0 0x7f93dc10c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57652/0 (socket says 172.21.15.42:57652) 2024-02-15T18:24:59.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.411+0000 7f93d9d9b700 1 -- 172.21.15.42:0/3268843821 learned_addr learned my addr 172.21.15.42:0/3268843821 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:59.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.411+0000 7f93d9d9b700 1 -- 172.21.15.42:0/3268843821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93dc0f55c0 msgr2=0x7f93dc109ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.411+0000 7f93d9d9b700 1 --2- 172.21.15.42:0/3268843821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93dc0f55c0 0x7f93dc109ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.411+0000 7f93d9d9b700 1 -- 172.21.15.42:0/3268843821 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d0005040 con 0x7f93dc0f5ed0 2024-02-15T18:24:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.411+0000 7f93da59c700 1 --2- 172.21.15.42:0/3268843821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93dc0f55c0 0x7f93dc109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:24:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.412+0000 7f93d9d9b700 1 --2- 172.21.15.42:0/3268843821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93dc0f5ed0 0x7f93dc10c1e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f93d0000c00 tx=0x7f93d000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.412+0000 7f93caffd700 1 -- 172.21.15.42:0/3268843821 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93d0019070 con 0x7f93dc0f5ed0 2024-02-15T18:24:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.412+0000 7f93caffd700 1 -- 172.21.15.42:0/3268843821 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f93d0007e80 con 0x7f93dc0f5ed0 2024-02-15T18:24:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.412+0000 7f93e0bc1700 1 -- 172.21.15.42:0/3268843821 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93dc10c730 con 0x7f93dc0f5ed0 2024-02-15T18:24:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.412+0000 7f93caffd700 1 -- 172.21.15.42:0/3268843821 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93d0014db0 con 0x7f93dc0f5ed0 2024-02-15T18:24:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.412+0000 7f93e0bc1700 1 -- 172.21.15.42:0/3268843821 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93dc10cc60 con 0x7f93dc0f5ed0 2024-02-15T18:24:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.413+0000 7f93e0bc1700 1 -- 172.21.15.42:0/3268843821 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93dc0f4a50 con 0x7f93dc0f5ed0 2024-02-15T18:24:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.413+0000 7f93caffd700 1 -- 172.21.15.42:0/3268843821 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f93d0025070 con 0x7f93dc0f5ed0 2024-02-15T18:24:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.413+0000 7f93caffd700 1 --2- 172.21.15.42:0/3268843821 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f93c405c2b0 0x7f93c405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.414+0000 7f93caffd700 1 -- 172.21.15.42:0/3268843821 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f93d0076cc0 con 0x7f93dc0f5ed0 2024-02-15T18:24:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.414+0000 7f93da59c700 1 --2- 172.21.15.42:0/3268843821 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f93c405c2b0 0x7f93c405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.414+0000 7f93da59c700 1 --2- 172.21.15.42:0/3268843821 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f93c405c2b0 0x7f93c405e770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f93dc060120 tx=0x7f93cc00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.418+0000 7f93caffd700 1 -- 172.21.15.42:0/3268843821 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f93d0049a00 con 0x7f93dc0f5ed0 2024-02-15T18:24:59.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.615+0000 7f93e0bc1700 1 -- 172.21.15.42:0/3268843821 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93dc0fe030 con 0x7f93c405c2b0 2024-02-15T18:24:59.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.616+0000 7f93caffd700 1 -- 172.21.15.42:0/3268843821 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+372 (secure 0 0 0) 0x7f93dc0fe030 con 0x7f93c405c2b0 2024-02-15T18:24:59.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.618+0000 7f93e0bc1700 1 -- 172.21.15.42:0/3268843821 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f93c405c2b0 msgr2=0x7f93c405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:59.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.618+0000 7f93e0bc1700 1 --2- 172.21.15.42:0/3268843821 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f93c405c2b0 0x7f93c405e770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f93dc060120 tx=0x7f93cc00b040).stop 2024-02-15T18:24:59.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.618+0000 7f93e0bc1700 1 -- 172.21.15.42:0/3268843821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93dc0f5ed0 msgr2=0x7f93dc10c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:59.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.618+0000 7f93e0bc1700 1 --2- 172.21.15.42:0/3268843821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93dc0f5ed0 0x7f93dc10c1e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7f93d0000c00 tx=0x7f93d000b040).stop 2024-02-15T18:24:59.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.618+0000 7f93e0bc1700 1 -- 172.21.15.42:0/3268843821 shutdown_connections 2024-02-15T18:24:59.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.618+0000 7f93e0bc1700 1 --2- 172.21.15.42:0/3268843821 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f93c405c2b0 0x7f93c405e770 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:59.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.618+0000 7f93e0bc1700 1 --2- 172.21.15.42:0/3268843821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93dc0f55c0 0x7f93dc109ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:59.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.618+0000 7f93e0bc1700 1 --2- 172.21.15.42:0/3268843821 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93dc0f5ed0 0x7f93dc10c1e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:59.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.618+0000 7f93e0bc1700 1 -- 172.21.15.42:0/3268843821 >> 172.21.15.42:0/3268843821 conn(0x7f93dc0f0de0 msgr2=0x7f93dc0fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:59.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.618+0000 7f93e0bc1700 1 -- 172.21.15.42:0/3268843821 shutdown_connections 2024-02-15T18:24:59.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.618+0000 7f93e0bc1700 1 -- 172.21.15.42:0/3268843821 wait complete. 2024-02-15T18:24:59.627 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:24:59.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.721+0000 7f0c90ea1700 1 -- 172.21.15.42:0/4063114827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c8c0f77d0 msgr2=0x7f0c8c0f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:59.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.721+0000 7f0c90ea1700 1 --2- 172.21.15.42:0/4063114827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c8c0f77d0 0x7f0c8c0f7c30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f0c74004660 tx=0x7f0c74009b30).stop 2024-02-15T18:24:59.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.721+0000 7f0c90ea1700 1 -- 172.21.15.42:0/4063114827 shutdown_connections 2024-02-15T18:24:59.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.721+0000 7f0c90ea1700 1 --2- 172.21.15.42:0/4063114827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c8c0f77d0 0x7f0c8c0f7c30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:59.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.721+0000 7f0c90ea1700 1 --2- 172.21.15.42:0/4063114827 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c8c0f6530 0x7f0c8c0f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:59.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.721+0000 7f0c90ea1700 1 -- 172.21.15.42:0/4063114827 >> 172.21.15.42:0/4063114827 conn(0x7f0c8c0f1c90 msgr2=0x7f0c8c0f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:59.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.721+0000 7f0c90ea1700 1 -- 172.21.15.42:0/4063114827 shutdown_connections 2024-02-15T18:24:59.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.722+0000 7f0c90ea1700 1 -- 172.21.15.42:0/4063114827 wait complete. 2024-02-15T18:24:59.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.722+0000 7f0c90ea1700 1 Processor -- start 2024-02-15T18:24:59.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.722+0000 7f0c90ea1700 1 -- start start 2024-02-15T18:24:59.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.722+0000 7f0c90ea1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c8c0f6530 0x7f0c8c106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:59.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.722+0000 7f0c90ea1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c8c0f77d0 0x7f0c8c1072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:59.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.722+0000 7f0c90ea1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c1078a0 con 0x7f0c8c0f77d0 2024-02-15T18:24:59.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.722+0000 7f0c90ea1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c1080e0 con 0x7f0c8c0f6530 2024-02-15T18:24:59.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.723+0000 7f0c8affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c8c0f77d0 0x7f0c8c1072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:59.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.723+0000 7f0c8b7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c8c0f6530 0x7f0c8c106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:59.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.723+0000 7f0c8affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c8c0f77d0 0x7f0c8c1072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57676/0 (socket says 172.21.15.42:57676) 2024-02-15T18:24:59.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.723+0000 7f0c8affd700 1 -- 172.21.15.42:0/1511690468 learned_addr learned my addr 172.21.15.42:0/1511690468 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:24:59.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.723+0000 7f0c8b7fe700 1 --2- 172.21.15.42:0/1511690468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c8c0f6530 0x7f0c8c106db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.42:48344/0 (socket says 172.21.15.42:48344) 2024-02-15T18:24:59.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.723+0000 7f0c8affd700 1 -- 172.21.15.42:0/1511690468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c8c0f6530 msgr2=0x7f0c8c106db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:59.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.723+0000 7f0c8affd700 1 --2- 172.21.15.42:0/1511690468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c8c0f6530 0x7f0c8c106db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:59.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.723+0000 7f0c8affd700 1 -- 172.21.15.42:0/1511690468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c74005040 con 0x7f0c8c0f77d0 2024-02-15T18:24:59.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.723+0000 7f0c8affd700 1 --2- 172.21.15.42:0/1511690468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c8c0f77d0 0x7f0c8c1072a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f0c74000c00 tx=0x7f0c7400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:59.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.723+0000 7f0c83fff700 1 -- 172.21.15.42:0/1511690468 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c74019070 con 0x7f0c8c0f77d0 2024-02-15T18:24:59.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.723+0000 7f0c90ea1700 1 -- 172.21.15.42:0/1511690468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8c108360 con 0x7f0c8c0f77d0 2024-02-15T18:24:59.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.723+0000 7f0c90ea1700 1 -- 172.21.15.42:0/1511690468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c108840 con 0x7f0c8c0f77d0 2024-02-15T18:24:59.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.723+0000 7f0c83fff700 1 -- 172.21.15.42:0/1511690468 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f0c74007e80 con 0x7f0c8c0f77d0 2024-02-15T18:24:59.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.723+0000 7f0c83fff700 1 -- 172.21.15.42:0/1511690468 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c74014db0 con 0x7f0c8c0f77d0 2024-02-15T18:24:59.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.724+0000 7f0c90ea1700 1 -- 172.21.15.42:0/1511690468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c70004fa0 con 0x7f0c8c0f77d0 2024-02-15T18:24:59.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.724+0000 7f0c83fff700 1 -- 172.21.15.42:0/1511690468 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f0c74025070 con 0x7f0c8c0f77d0 2024-02-15T18:24:59.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.725+0000 7f0c83fff700 1 --2- 172.21.15.42:0/1511690468 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0c7805c300 0x7f0c7805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:24:59.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.725+0000 7f0c83fff700 1 -- 172.21.15.42:0/1511690468 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f0c74076d20 con 0x7f0c8c0f77d0 2024-02-15T18:24:59.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.725+0000 7f0c8b7fe700 1 --2- 172.21.15.42:0/1511690468 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0c7805c300 0x7f0c7805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:24:59.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.725+0000 7f0c8b7fe700 1 --2- 172.21.15.42:0/1511690468 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0c7805c300 0x7f0c7805e7c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f0c8c107b70 tx=0x7f0c7c00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:24:59.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.729+0000 7f0c83fff700 1 -- 172.21.15.42:0/1511690468 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0c74049980 con 0x7f0c8c0f77d0 2024-02-15T18:24:59.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.921+0000 7f0c90ea1700 1 -- 172.21.15.42:0/1511690468 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c70000bc0 con 0x7f0c7805c300 2024-02-15T18:24:59.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.922+0000 7f0c83fff700 1 -- 172.21.15.42:0/1511690468 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+372 (secure 0 0 0) 0x7f0c70000bc0 con 0x7f0c7805c300 2024-02-15T18:24:59.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.924+0000 7f0c90ea1700 1 -- 172.21.15.42:0/1511690468 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0c7805c300 msgr2=0x7f0c7805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:59.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.924+0000 7f0c90ea1700 1 --2- 172.21.15.42:0/1511690468 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0c7805c300 0x7f0c7805e7c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f0c8c107b70 tx=0x7f0c7c00b040).stop 2024-02-15T18:24:59.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.924+0000 7f0c90ea1700 1 -- 172.21.15.42:0/1511690468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c8c0f77d0 msgr2=0x7f0c8c1072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:24:59.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.924+0000 7f0c90ea1700 1 --2- 172.21.15.42:0/1511690468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c8c0f77d0 0x7f0c8c1072a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f0c74000c00 tx=0x7f0c7400b040).stop 2024-02-15T18:24:59.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.924+0000 7f0c90ea1700 1 -- 172.21.15.42:0/1511690468 shutdown_connections 2024-02-15T18:24:59.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.924+0000 7f0c90ea1700 1 --2- 172.21.15.42:0/1511690468 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f0c7805c300 0x7f0c7805e7c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:59.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.924+0000 7f0c90ea1700 1 --2- 172.21.15.42:0/1511690468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c8c0f6530 0x7f0c8c106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:59.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.924+0000 7f0c90ea1700 1 --2- 172.21.15.42:0/1511690468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c8c0f77d0 0x7f0c8c1072a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:24:59.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.924+0000 7f0c90ea1700 1 -- 172.21.15.42:0/1511690468 >> 172.21.15.42:0/1511690468 conn(0x7f0c8c0f1c90 msgr2=0x7f0c8c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:24:59.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.924+0000 7f0c90ea1700 1 -- 172.21.15.42:0/1511690468 shutdown_connections 2024-02-15T18:24:59.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:24:59.924+0000 7f0c90ea1700 1 -- 172.21.15.42:0/1511690468 wait complete. 2024-02-15T18:25:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.026+0000 7f2f17c73700 1 -- 172.21.15.42:0/1163524354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f100f55c0 msgr2=0x7f2f100f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.026+0000 7f2f17c73700 1 --2- 172.21.15.42:0/1163524354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f100f55c0 0x7f2f100f59e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f2f0c004660 tx=0x7f2f0c009b30).stop 2024-02-15T18:25:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.026+0000 7f2f17c73700 1 -- 172.21.15.42:0/1163524354 shutdown_connections 2024-02-15T18:25:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.026+0000 7f2f17c73700 1 --2- 172.21.15.42:0/1163524354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f100f6860 0x7f2f100f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.026+0000 7f2f17c73700 1 --2- 172.21.15.42:0/1163524354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f100f55c0 0x7f2f100f59e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.026+0000 7f2f17c73700 1 -- 172.21.15.42:0/1163524354 >> 172.21.15.42:0/1163524354 conn(0x7f2f100f0d40 msgr2=0x7f2f100f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.026+0000 7f2f17c73700 1 -- 172.21.15.42:0/1163524354 shutdown_connections 2024-02-15T18:25:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.026+0000 7f2f17c73700 1 -- 172.21.15.42:0/1163524354 wait complete. 2024-02-15T18:25:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.027+0000 7f2f17c73700 1 Processor -- start 2024-02-15T18:25:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.027+0000 7f2f17c73700 1 -- start start 2024-02-15T18:25:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.027+0000 7f2f17c73700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f100f55c0 0x7f2f10109d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.027+0000 7f2f17c73700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f100f6860 0x7f2f1010c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.027+0000 7f2f17c73700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100603c0 con 0x7f2f100f55c0 2024-02-15T18:25:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.027+0000 7f2f17c73700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10060500 con 0x7f2f100f6860 2024-02-15T18:25:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.027+0000 7f2f15a0f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f100f55c0 0x7f2f10109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.027+0000 7f2f15a0f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f100f55c0 0x7f2f10109d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57702/0 (socket says 172.21.15.42:57702) 2024-02-15T18:25:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.027+0000 7f2f15a0f700 1 -- 172.21.15.42:0/3929647421 learned_addr learned my addr 172.21.15.42:0/3929647421 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:25:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.027+0000 7f2f1520e700 1 --2- 172.21.15.42:0/3929647421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f100f6860 0x7f2f1010c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.028+0000 7f2f15a0f700 1 -- 172.21.15.42:0/3929647421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f100f6860 msgr2=0x7f2f1010c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.028+0000 7f2f15a0f700 1 --2- 172.21.15.42:0/3929647421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f100f6860 0x7f2f1010c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.028+0000 7f2f15a0f700 1 -- 172.21.15.42:0/3929647421 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c005040 con 0x7f2f100f55c0 2024-02-15T18:25:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.028+0000 7f2f15a0f700 1 --2- 172.21.15.42:0/3929647421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f100f55c0 0x7f2f10109d20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f2f0c002040 tx=0x7f2f0c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.028+0000 7f2f067fc700 1 -- 172.21.15.42:0/3929647421 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f0c019070 con 0x7f2f100f55c0 2024-02-15T18:25:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.028+0000 7f2f067fc700 1 -- 172.21.15.42:0/3929647421 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f2f0c007e80 con 0x7f2f100f55c0 2024-02-15T18:25:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.028+0000 7f2f17c73700 1 -- 172.21.15.42:0/3929647421 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1010c770 con 0x7f2f100f55c0 2024-02-15T18:25:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.028+0000 7f2f067fc700 1 -- 172.21.15.42:0/3929647421 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f0c014db0 con 0x7f2f100f55c0 2024-02-15T18:25:00.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.028+0000 7f2f17c73700 1 -- 172.21.15.42:0/3929647421 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1010cca0 con 0x7f2f100f55c0 2024-02-15T18:25:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.029+0000 7f2f17c73700 1 -- 172.21.15.42:0/3929647421 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f100f49b0 con 0x7f2f100f55c0 2024-02-15T18:25:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.029+0000 7f2f067fc700 1 -- 172.21.15.42:0/3929647421 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f2f0c025070 con 0x7f2f100f55c0 2024-02-15T18:25:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.030+0000 7f2f067fc700 1 --2- 172.21.15.42:0/3929647421 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2efc05c250 0x7f2efc05e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.030+0000 7f2f067fc700 1 -- 172.21.15.42:0/3929647421 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f2f0c076da0 con 0x7f2f100f55c0 2024-02-15T18:25:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.030+0000 7f2f1520e700 1 --2- 172.21.15.42:0/3929647421 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2efc05c250 0x7f2efc05e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:00.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.030+0000 7f2f1520e700 1 --2- 172.21.15.42:0/3929647421 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2efc05c250 0x7f2efc05e710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f2f00009fd0 tx=0x7f2f0000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:00.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.034+0000 7f2f067fc700 1 -- 172.21.15.42:0/3929647421 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f2f0c049a00 con 0x7f2f100f55c0 2024-02-15T18:25:00.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.220+0000 7f2f17c73700 1 -- 172.21.15.42:0/3929647421 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f100fe010 con 0x7f2efc05c250 2024-02-15T18:25:00.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.228+0000 7f2f067fc700 1 -- 172.21.15.42:0/3929647421 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2684 (secure 0 0 0) 0x7f2f100fe010 con 0x7f2efc05c250 2024-02-15T18:25:00.227 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE VERSION IMAGE ID CONTAINER ID 2024-02-15T18:25:00.227 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (4m) 49s ago 6m 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:25:00.227 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (6m) 49s ago 6m 16.2.4 8d91d370c2b8 94d74c33c758 2024-02-15T18:25:00.227 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (5m) 55s ago 5m 16.2.4 8d91d370c2b8 ff0f56f9475a 2024-02-15T18:25:00.227 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (4m) 49s ago 6m 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:25:00.227 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (67s) 49s ago 67s 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:25:00.227 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (73s) 49s ago 72s 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:25:00.227 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (70s) 55s ago 70s 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:25:00.227 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (77s) 55s ago 77s 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:25:00.227 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:9283 running (7m) 49s ago 7m 16.2.4 8d91d370c2b8 a56a503c9572 2024-02-15T18:25:00.227 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (5m) 55s ago 5m 16.2.4 8d91d370c2b8 f2a90ea3fad1 2024-02-15T18:25:00.228 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (8m) 49s ago 8m 16.2.4 8d91d370c2b8 9dcbc5c49e05 2024-02-15T18:25:00.228 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (5m) 55s ago 5m 16.2.4 8d91d370c2b8 c3dd4902e2a2 2024-02-15T18:25:00.228 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (6m) 49s ago 6m 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:25:00.228 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (5m) 55s ago 5m 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:25:00.228 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (4m) 49s ago 4m 16.2.4 8d91d370c2b8 973745b31d5a 2024-02-15T18:25:00.228 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (4m) 49s ago 4m 16.2.4 8d91d370c2b8 2b3ae86111e8 2024-02-15T18:25:00.228 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (3m) 49s ago 3m 16.2.4 8d91d370c2b8 3706071e4698 2024-02-15T18:25:00.228 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (3m) 55s ago 3m 16.2.4 8d91d370c2b8 e6fe8e77c6fe 2024-02-15T18:25:00.228 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (3m) 55s ago 3m 16.2.4 8d91d370c2b8 24582012e234 2024-02-15T18:25:00.228 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (2m) 55s ago 2m 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:25:00.228 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (4m) 49s ago 5m 2.18.1 de242295e225 7fffb9a3be63 2024-02-15T18:25:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.229+0000 7f2f17c73700 1 -- 172.21.15.42:0/3929647421 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2efc05c250 msgr2=0x7f2efc05e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.229+0000 7f2f17c73700 1 --2- 172.21.15.42:0/3929647421 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2efc05c250 0x7f2efc05e710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f2f00009fd0 tx=0x7f2f0000b040).stop 2024-02-15T18:25:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.230+0000 7f2f17c73700 1 -- 172.21.15.42:0/3929647421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f100f55c0 msgr2=0x7f2f10109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.230+0000 7f2f17c73700 1 --2- 172.21.15.42:0/3929647421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f100f55c0 0x7f2f10109d20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f2f0c002040 tx=0x7f2f0c00b040).stop 2024-02-15T18:25:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.230+0000 7f2f17c73700 1 -- 172.21.15.42:0/3929647421 shutdown_connections 2024-02-15T18:25:00.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.230+0000 7f2f17c73700 1 --2- 172.21.15.42:0/3929647421 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f2efc05c250 0x7f2efc05e710 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.230+0000 7f2f17c73700 1 --2- 172.21.15.42:0/3929647421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f100f55c0 0x7f2f10109d20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.230+0000 7f2f17c73700 1 --2- 172.21.15.42:0/3929647421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f100f6860 0x7f2f1010c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.230+0000 7f2f17c73700 1 -- 172.21.15.42:0/3929647421 >> 172.21.15.42:0/3929647421 conn(0x7f2f100f0d40 msgr2=0x7f2f100fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:00.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.230+0000 7f2f17c73700 1 -- 172.21.15.42:0/3929647421 shutdown_connections 2024-02-15T18:25:00.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.230+0000 7f2f17c73700 1 -- 172.21.15.42:0/3929647421 wait complete. 2024-02-15T18:25:00.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.332+0000 7fd6babd3700 1 -- 172.21.15.42:0/420057438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b40f7980 msgr2=0x7fd6b40f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:00.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.332+0000 7fd6babd3700 1 --2- 172.21.15.42:0/420057438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b40f7980 0x7fd6b40f7de0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fd6a0004660 tx=0x7fd6a0009b30).stop 2024-02-15T18:25:00.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.332+0000 7fd6babd3700 1 -- 172.21.15.42:0/420057438 shutdown_connections 2024-02-15T18:25:00.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.332+0000 7fd6babd3700 1 --2- 172.21.15.42:0/420057438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b40f7980 0x7fd6b40f7de0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.332+0000 7fd6babd3700 1 --2- 172.21.15.42:0/420057438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b40f66e0 0x7fd6b40f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.332+0000 7fd6babd3700 1 -- 172.21.15.42:0/420057438 >> 172.21.15.42:0/420057438 conn(0x7fd6b40f1e40 msgr2=0x7fd6b40f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:00.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.332+0000 7fd6babd3700 1 -- 172.21.15.42:0/420057438 shutdown_connections 2024-02-15T18:25:00.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.332+0000 7fd6babd3700 1 -- 172.21.15.42:0/420057438 wait complete. 2024-02-15T18:25:00.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.333+0000 7fd6babd3700 1 Processor -- start 2024-02-15T18:25:00.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.333+0000 7fd6babd3700 1 -- start start 2024-02-15T18:25:00.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.333+0000 7fd6babd3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b40f66e0 0x7fd6b4107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:00.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.333+0000 7fd6babd3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b40f7980 0x7fd6b41074f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:00.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.333+0000 7fd6babd3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4107aa0 con 0x7fd6b40f66e0 2024-02-15T18:25:00.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.333+0000 7fd6babd3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b4107be0 con 0x7fd6b40f7980 2024-02-15T18:25:00.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.333+0000 7fd6b896f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b40f66e0 0x7fd6b4107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:00.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.333+0000 7fd6b896f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b40f66e0 0x7fd6b4107000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57728/0 (socket says 172.21.15.42:57728) 2024-02-15T18:25:00.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.333+0000 7fd6b896f700 1 -- 172.21.15.42:0/4160221170 learned_addr learned my addr 172.21.15.42:0/4160221170 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:25:00.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.333+0000 7fd6b3fff700 1 --2- 172.21.15.42:0/4160221170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b40f7980 0x7fd6b41074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:00.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.334+0000 7fd6b896f700 1 -- 172.21.15.42:0/4160221170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b40f7980 msgr2=0x7fd6b41074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:00.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.334+0000 7fd6b896f700 1 --2- 172.21.15.42:0/4160221170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b40f7980 0x7fd6b41074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.334+0000 7fd6b896f700 1 -- 172.21.15.42:0/4160221170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a0005040 con 0x7fd6b40f66e0 2024-02-15T18:25:00.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.334+0000 7fd6b896f700 1 --2- 172.21.15.42:0/4160221170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b40f66e0 0x7fd6b4107000 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fd6a8002700 tx=0x7fd6a8009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:00.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.334+0000 7fd6b17fa700 1 -- 172.21.15.42:0/4160221170 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6a80079c0 con 0x7fd6b40f66e0 2024-02-15T18:25:00.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.334+0000 7fd6babd3700 1 -- 172.21.15.42:0/4160221170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b4108580 con 0x7fd6b40f66e0 2024-02-15T18:25:00.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.334+0000 7fd6b17fa700 1 -- 172.21.15.42:0/4160221170 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fd6a800f040 con 0x7fd6b40f66e0 2024-02-15T18:25:00.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.334+0000 7fd6b17fa700 1 -- 172.21.15.42:0/4160221170 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6a8014940 con 0x7fd6b40f66e0 2024-02-15T18:25:00.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.334+0000 7fd6babd3700 1 -- 172.21.15.42:0/4160221170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b4108b40 con 0x7fd6b40f66e0 2024-02-15T18:25:00.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.335+0000 7fd6babd3700 1 -- 172.21.15.42:0/4160221170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b4045ad0 con 0x7fd6b40f66e0 2024-02-15T18:25:00.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.335+0000 7fd6b17fa700 1 -- 172.21.15.42:0/4160221170 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fd6a801b060 con 0x7fd6b40f66e0 2024-02-15T18:25:00.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.336+0000 7fd6b17fa700 1 --2- 172.21.15.42:0/4160221170 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd6a405c2b0 0x7fd6a405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:00.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.336+0000 7fd6b17fa700 1 -- 172.21.15.42:0/4160221170 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fd6a8075cf0 con 0x7fd6b40f66e0 2024-02-15T18:25:00.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.336+0000 7fd6b3fff700 1 --2- 172.21.15.42:0/4160221170 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd6a405c2b0 0x7fd6a405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:00.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.336+0000 7fd6b3fff700 1 --2- 172.21.15.42:0/4160221170 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd6a405c2b0 0x7fd6a405e770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd6a0004630 tx=0x7fd6a0009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:00.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.340+0000 7fd6b17fa700 1 -- 172.21.15.42:0/4160221170 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd6a80489b0 con 0x7fd6b40f66e0 2024-02-15T18:25:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.563+0000 7fd6babd3700 1 -- 172.21.15.42:0/4160221170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd6b40ff130 con 0x7fd6b40f66e0 2024-02-15T18:25:00.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.564+0000 7fd6b17fa700 1 -- 172.21.15.42:0/4160221170 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+573 (secure 0 0 0) 0x7fd6a804bfd0 con 0x7fd6b40f66e0 2024-02-15T18:25:00.563 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:25:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:25:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:25:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:25:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:25:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:25:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:25:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:25:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:25:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:25:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:25:00.563 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:25:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:25:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:25:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 14 2024-02-15T18:25:00.564 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:25:00.564 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:25:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.566+0000 7fd6babd3700 1 -- 172.21.15.42:0/4160221170 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd6a405c2b0 msgr2=0x7fd6a405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.566+0000 7fd6babd3700 1 --2- 172.21.15.42:0/4160221170 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd6a405c2b0 0x7fd6a405e770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fd6a0004630 tx=0x7fd6a0009b30).stop 2024-02-15T18:25:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.566+0000 7fd6babd3700 1 -- 172.21.15.42:0/4160221170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b40f66e0 msgr2=0x7fd6b4107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.566+0000 7fd6babd3700 1 --2- 172.21.15.42:0/4160221170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b40f66e0 0x7fd6b4107000 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7fd6a8002700 tx=0x7fd6a8009d80).stop 2024-02-15T18:25:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.566+0000 7fd6babd3700 1 -- 172.21.15.42:0/4160221170 shutdown_connections 2024-02-15T18:25:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.566+0000 7fd6babd3700 1 --2- 172.21.15.42:0/4160221170 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fd6a405c2b0 0x7fd6a405e770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.566+0000 7fd6babd3700 1 --2- 172.21.15.42:0/4160221170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b40f66e0 0x7fd6b4107000 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.566+0000 7fd6babd3700 1 --2- 172.21.15.42:0/4160221170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b40f7980 0x7fd6b41074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.566+0000 7fd6babd3700 1 -- 172.21.15.42:0/4160221170 >> 172.21.15.42:0/4160221170 conn(0x7fd6b40f1e40 msgr2=0x7fd6b40fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.566+0000 7fd6babd3700 1 -- 172.21.15.42:0/4160221170 shutdown_connections 2024-02-15T18:25:00.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.566+0000 7fd6babd3700 1 -- 172.21.15.42:0/4160221170 wait complete. 2024-02-15T18:25:00.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.667+0000 7fcaea8cc700 1 -- 172.21.15.42:0/1184892249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae40f6860 msgr2=0x7fcae40f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:00.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.667+0000 7fcaea8cc700 1 --2- 172.21.15.42:0/1184892249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae40f6860 0x7fcae40f6cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7fcacc004660 tx=0x7fcacc009b30).stop 2024-02-15T18:25:00.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.667+0000 7fcaea8cc700 1 -- 172.21.15.42:0/1184892249 shutdown_connections 2024-02-15T18:25:00.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.667+0000 7fcaea8cc700 1 --2- 172.21.15.42:0/1184892249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae40f6860 0x7fcae40f6cc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.668+0000 7fcaea8cc700 1 --2- 172.21.15.42:0/1184892249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcae40f55c0 0x7fcae40f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.668+0000 7fcaea8cc700 1 -- 172.21.15.42:0/1184892249 >> 172.21.15.42:0/1184892249 conn(0x7fcae40f0d40 msgr2=0x7fcae40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:00.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.668+0000 7fcaea8cc700 1 -- 172.21.15.42:0/1184892249 shutdown_connections 2024-02-15T18:25:00.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.668+0000 7fcaea8cc700 1 -- 172.21.15.42:0/1184892249 wait complete. 2024-02-15T18:25:00.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.668+0000 7fcaea8cc700 1 Processor -- start 2024-02-15T18:25:00.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.668+0000 7fcaea8cc700 1 -- start start 2024-02-15T18:25:00.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcaea8cc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcae40f55c0 0x7fcae4105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:00.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcaea8cc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae40f6860 0x7fcae41063a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:00.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcaea8cc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae4106950 con 0x7fcae40f6860 2024-02-15T18:25:00.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcaea8cc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae4106a90 con 0x7fcae40f55c0 2024-02-15T18:25:00.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcae37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae40f6860 0x7fcae41063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:00.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcae37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae40f6860 0x7fcae41063a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57756/0 (socket says 172.21.15.42:57756) 2024-02-15T18:25:00.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcae37fe700 1 -- 172.21.15.42:0/34336807 learned_addr learned my addr 172.21.15.42:0/34336807 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:25:00.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcae3fff700 1 --2- 172.21.15.42:0/34336807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcae40f55c0 0x7fcae4105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:00.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcae37fe700 1 -- 172.21.15.42:0/34336807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcae40f55c0 msgr2=0x7fcae4105eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:00.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcae37fe700 1 --2- 172.21.15.42:0/34336807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcae40f55c0 0x7fcae4105eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcae37fe700 1 -- 172.21.15.42:0/34336807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc005040 con 0x7fcae40f6860 2024-02-15T18:25:00.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcae37fe700 1 --2- 172.21.15.42:0/34336807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae40f6860 0x7fcae41063a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fcacc000c00 tx=0x7fcacc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:00.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcae0ff9700 1 -- 172.21.15.42:0/34336807 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcacc019070 con 0x7fcae40f6860 2024-02-15T18:25:00.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcae0ff9700 1 -- 172.21.15.42:0/34336807 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fcacc007e80 con 0x7fcae40f6860 2024-02-15T18:25:00.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcaea8cc700 1 -- 172.21.15.42:0/34336807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae41073d0 con 0x7fcae40f6860 2024-02-15T18:25:00.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.669+0000 7fcae0ff9700 1 -- 172.21.15.42:0/34336807 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcacc014db0 con 0x7fcae40f6860 2024-02-15T18:25:00.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.670+0000 7fcaea8cc700 1 -- 172.21.15.42:0/34336807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae4107930 con 0x7fcae40f6860 2024-02-15T18:25:00.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.670+0000 7fcaea8cc700 1 -- 172.21.15.42:0/34336807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae4045ad0 con 0x7fcae40f6860 2024-02-15T18:25:00.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.671+0000 7fcae0ff9700 1 -- 172.21.15.42:0/34336807 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fcacc025070 con 0x7fcae40f6860 2024-02-15T18:25:00.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.671+0000 7fcae0ff9700 1 --2- 172.21.15.42:0/34336807 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fcad005c250 0x7fcad005e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:00.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.671+0000 7fcae0ff9700 1 -- 172.21.15.42:0/34336807 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fcacc076da0 con 0x7fcae40f6860 2024-02-15T18:25:00.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.671+0000 7fcae3fff700 1 --2- 172.21.15.42:0/34336807 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fcad005c250 0x7fcad005e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:00.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.672+0000 7fcae3fff700 1 --2- 172.21.15.42:0/34336807 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fcad005c250 0x7fcad005e710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fcad4009fd0 tx=0x7fcad400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:00.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.675+0000 7fcae0ff9700 1 -- 172.21.15.42:0/34336807 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fcacc049a00 con 0x7fcae40f6860 2024-02-15T18:25:00.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:24:58.836470+0000 mgr.smithi042.kfoihb (mgr.14162) 291 : cluster [DBG] pgmap v244: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-02-15T18:25:00.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:24:58.836470+0000 mgr.smithi042.kfoihb (mgr.14162) 291 : cluster [DBG] pgmap v244: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 597 B/s wr, 3 op/s 2024-02-15T18:25:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.870+0000 7fcaea8cc700 1 -- 172.21.15.42:0/34336807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcae4027a00 con 0x7fcae40f6860 2024-02-15T18:25:00.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.871+0000 7fcae0ff9700 1 -- 172.21.15.42:0/34336807 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1626 (secure 0 0 0) 0x7fcacc023030 con 0x7fcae40f6860 2024-02-15T18:25:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:25:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:25:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:25:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:25:00.870 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:25:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:25:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:25:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:25:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:25:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:25:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:25:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:25:00.870 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:25:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:25:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:25:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:25:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:25:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:25:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:25:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:25:00.871 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:25:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:25:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:25:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:25:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:25:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:25:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:25:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:25:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:25:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:25:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:25:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000]] 2024-02-15T18:25:00.872 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509]] 2024-02-15T18:25:00.872 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:25:00.872 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:25:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:25:00.873 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:25:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366]] 2024-02-15T18:25:00.873 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796]] 2024-02-15T18:25:00.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.872+0000 7fcaea8cc700 1 -- 172.21.15.42:0/34336807 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fcad005c250 msgr2=0x7fcad005e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:00.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.872+0000 7fcaea8cc700 1 --2- 172.21.15.42:0/34336807 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fcad005c250 0x7fcad005e710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fcad4009fd0 tx=0x7fcad400b040).stop 2024-02-15T18:25:00.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.872+0000 7fcaea8cc700 1 -- 172.21.15.42:0/34336807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae40f6860 msgr2=0x7fcae41063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:00.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.872+0000 7fcaea8cc700 1 --2- 172.21.15.42:0/34336807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae40f6860 0x7fcae41063a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fcacc000c00 tx=0x7fcacc00b040).stop 2024-02-15T18:25:00.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.872+0000 7fcaea8cc700 1 -- 172.21.15.42:0/34336807 shutdown_connections 2024-02-15T18:25:00.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.872+0000 7fcaea8cc700 1 --2- 172.21.15.42:0/34336807 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fcad005c250 0x7fcad005e710 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.872+0000 7fcaea8cc700 1 --2- 172.21.15.42:0/34336807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcae40f55c0 0x7fcae4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.872+0000 7fcaea8cc700 1 --2- 172.21.15.42:0/34336807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcae40f6860 0x7fcae41063a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.872+0000 7fcaea8cc700 1 -- 172.21.15.42:0/34336807 >> 172.21.15.42:0/34336807 conn(0x7fcae40f0d40 msgr2=0x7fcae40fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:00.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.872+0000 7fcaea8cc700 1 -- 172.21.15.42:0/34336807 shutdown_connections 2024-02-15T18:25:00.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.873+0000 7fcaea8cc700 1 -- 172.21.15.42:0/34336807 wait complete. 2024-02-15T18:25:00.874 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:25:00.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.975+0000 7f9327d04700 1 -- 172.21.15.42:0/3437166563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93200f55c0 msgr2=0x7f93200f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:00.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.975+0000 7f9327d04700 1 --2- 172.21.15.42:0/3437166563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93200f55c0 0x7f93200f59e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f9314004660 tx=0x7f9314009b30).stop 2024-02-15T18:25:00.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.975+0000 7f9327d04700 1 -- 172.21.15.42:0/3437166563 shutdown_connections 2024-02-15T18:25:00.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.975+0000 7f9327d04700 1 --2- 172.21.15.42:0/3437166563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93200f6860 0x7f93200f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.975+0000 7f9327d04700 1 --2- 172.21.15.42:0/3437166563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93200f55c0 0x7f93200f59e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.975+0000 7f9327d04700 1 -- 172.21.15.42:0/3437166563 >> 172.21.15.42:0/3437166563 conn(0x7f93200f0d40 msgr2=0x7f93200f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:00.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.975+0000 7f9327d04700 1 -- 172.21.15.42:0/3437166563 shutdown_connections 2024-02-15T18:25:00.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.975+0000 7f9327d04700 1 -- 172.21.15.42:0/3437166563 wait complete. 2024-02-15T18:25:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.976+0000 7f9327d04700 1 Processor -- start 2024-02-15T18:25:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.976+0000 7f9327d04700 1 -- start start 2024-02-15T18:25:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.976+0000 7f9327d04700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93200f55c0 0x7f93200f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.976+0000 7f9327d04700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93200f6860 0x7f93200f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.976+0000 7f9327d04700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93200f8100 con 0x7f93200f55c0 2024-02-15T18:25:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.976+0000 7f9327d04700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93200f8240 con 0x7f93200f6860 2024-02-15T18:25:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.976+0000 7f9325aa0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93200f55c0 0x7f93200f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.976+0000 7f9325aa0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93200f55c0 0x7f93200f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57764/0 (socket says 172.21.15.42:57764) 2024-02-15T18:25:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.976+0000 7f9325aa0700 1 -- 172.21.15.42:0/4165109262 learned_addr learned my addr 172.21.15.42:0/4165109262 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:25:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.976+0000 7f932529f700 1 --2- 172.21.15.42:0/4165109262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93200f6860 0x7f93200f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.976+0000 7f9325aa0700 1 -- 172.21.15.42:0/4165109262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93200f6860 msgr2=0x7f93200f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:00.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.976+0000 7f9325aa0700 1 --2- 172.21.15.42:0/4165109262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93200f6860 0x7f93200f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:00.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.976+0000 7f9325aa0700 1 -- 172.21.15.42:0/4165109262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314005040 con 0x7f93200f55c0 2024-02-15T18:25:00.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.977+0000 7f9325aa0700 1 --2- 172.21.15.42:0/4165109262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93200f55c0 0x7f93200f9720 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f9314004e30 tx=0x7f9314009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:00.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.977+0000 7f93127fc700 1 -- 172.21.15.42:0/4165109262 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9314018070 con 0x7f93200f55c0 2024-02-15T18:25:00.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.977+0000 7f93127fc700 1 -- 172.21.15.42:0/4165109262 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f9314007ba0 con 0x7f93200f55c0 2024-02-15T18:25:00.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.977+0000 7f9327d04700 1 -- 172.21.15.42:0/4165109262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93200f84c0 con 0x7f93200f55c0 2024-02-15T18:25:00.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.977+0000 7f93127fc700 1 -- 172.21.15.42:0/4165109262 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f931401c620 con 0x7f93200f55c0 2024-02-15T18:25:00.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.977+0000 7f9327d04700 1 -- 172.21.15.42:0/4165109262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93200f89a0 con 0x7f93200f55c0 2024-02-15T18:25:00.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.978+0000 7f9327d04700 1 -- 172.21.15.42:0/4165109262 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9320045ad0 con 0x7f93200f55c0 2024-02-15T18:25:00.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.978+0000 7f93127fc700 1 -- 172.21.15.42:0/4165109262 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f931401d9a0 con 0x7f93200f55c0 2024-02-15T18:25:00.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.978+0000 7f93127fc700 1 --2- 172.21.15.42:0/4165109262 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f930c05c2b0 0x7f930c05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:00.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.978+0000 7f93127fc700 1 -- 172.21.15.42:0/4165109262 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f9314077c90 con 0x7f93200f55c0 2024-02-15T18:25:00.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.979+0000 7f932529f700 1 --2- 172.21.15.42:0/4165109262 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f930c05c2b0 0x7f930c05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:00.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.979+0000 7f932529f700 1 --2- 172.21.15.42:0/4165109262 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f930c05c2b0 0x7f930c05e770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f931c0077b0 tx=0x7f931c009ce0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:00.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:00.982+0000 7f93127fc700 1 -- 172.21.15.42:0/4165109262 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f931404a950 con 0x7f93200f55c0 2024-02-15T18:25:01.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:01.174+0000 7f9327d04700 1 -- 172.21.15.42:0/4165109262 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93200faca0 con 0x7f930c05c2b0 2024-02-15T18:25:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:01.175+0000 7f93127fc700 1 -- 172.21.15.42:0/4165109262 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+372 (secure 0 0 0) 0x7f93200faca0 con 0x7f930c05c2b0 2024-02-15T18:25:01.173 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:25:01.174 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:25:01.174 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:25:01.174 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [], 2024-02-15T18:25:01.174 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "0/21 ceph daemons upgraded", 2024-02-15T18:25:01.174 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc image on host smithi191" 2024-02-15T18:25:01.174 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:25:01.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:01.177+0000 7f9327d04700 1 -- 172.21.15.42:0/4165109262 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f930c05c2b0 msgr2=0x7f930c05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:01.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:01.177+0000 7f9327d04700 1 --2- 172.21.15.42:0/4165109262 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f930c05c2b0 0x7f930c05e770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f931c0077b0 tx=0x7f931c009ce0).stop 2024-02-15T18:25:01.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:01.177+0000 7f9327d04700 1 -- 172.21.15.42:0/4165109262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93200f55c0 msgr2=0x7f93200f9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:01.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:01.177+0000 7f9327d04700 1 --2- 172.21.15.42:0/4165109262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93200f55c0 0x7f93200f9720 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f9314004e30 tx=0x7f9314009b30).stop 2024-02-15T18:25:01.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:01.177+0000 7f9327d04700 1 -- 172.21.15.42:0/4165109262 shutdown_connections 2024-02-15T18:25:01.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:01.177+0000 7f9327d04700 1 --2- 172.21.15.42:0/4165109262 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7f930c05c2b0 0x7f930c05e770 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:01.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:01.177+0000 7f9327d04700 1 --2- 172.21.15.42:0/4165109262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93200f55c0 0x7f93200f9720 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:01.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:01.177+0000 7f9327d04700 1 --2- 172.21.15.42:0/4165109262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93200f6860 0x7f93200f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:01.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:01.177+0000 7f9327d04700 1 -- 172.21.15.42:0/4165109262 >> 172.21.15.42:0/4165109262 conn(0x7f93200f0d40 msgr2=0x7f932005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:01.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:01.177+0000 7f9327d04700 1 -- 172.21.15.42:0/4165109262 shutdown_connections 2024-02-15T18:25:01.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:01.177+0000 7f9327d04700 1 -- 172.21.15.42:0/4165109262 wait complete. 2024-02-15T18:25:01.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:59.616733+0000 mgr.smithi042.kfoihb (mgr.14162) 292 : audit [DBG] from='client.14536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:25:01.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:24:59.923342+0000 mgr.smithi042.kfoihb (mgr.14162) 293 : audit [DBG] from='client.14540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:25:01.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:00.222009+0000 mgr.smithi042.kfoihb (mgr.14162) 294 : audit [DBG] from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:25:01.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:00.565191+0000 mon.smithi042 (mon.0) 554 : audit [DBG] from='client.? 172.21.15.42:0/4160221170' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:25:01.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:00.872017+0000 mon.smithi042 (mon.0) 555 : audit [DBG] from='client.? 172.21.15.42:0/34336807' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:25:01.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:59.616733+0000 mgr.smithi042.kfoihb (mgr.14162) 292 : audit [DBG] from='client.14536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:25:01.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:24:59.923342+0000 mgr.smithi042.kfoihb (mgr.14162) 293 : audit [DBG] from='client.14540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:25:01.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:00.222009+0000 mgr.smithi042.kfoihb (mgr.14162) 294 : audit [DBG] from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:25:01.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:00.565191+0000 mon.smithi042 (mon.0) 554 : audit [DBG] from='client.? 172.21.15.42:0/4160221170' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:25:01.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:00.872017+0000 mon.smithi042 (mon.0) 555 : audit [DBG] from='client.? 172.21.15.42:0/34336807' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:25:02.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:00.836831+0000 mgr.smithi042.kfoihb (mgr.14162) 295 : cluster [DBG] pgmap v245: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-02-15T18:25:02.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:01.175948+0000 mgr.smithi042.kfoihb (mgr.14162) 296 : audit [DBG] from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:25:02.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:00.836831+0000 mgr.smithi042.kfoihb (mgr.14162) 295 : cluster [DBG] pgmap v245: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 426 B/s wr, 2 op/s 2024-02-15T18:25:02.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:01.175948+0000 mgr.smithi042.kfoihb (mgr.14162) 296 : audit [DBG] from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:25:04.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:02.837342+0000 mgr.smithi042.kfoihb (mgr.14162) 297 : cluster [DBG] pgmap v246: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-02-15T18:25:04.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:02.837342+0000 mgr.smithi042.kfoihb (mgr.14162) 297 : cluster [DBG] pgmap v246: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-02-15T18:25:06.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:04.837808+0000 mgr.smithi042.kfoihb (mgr.14162) 298 : cluster [DBG] pgmap v247: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 341 B/s wr, 3 op/s 2024-02-15T18:25:06.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:04.837808+0000 mgr.smithi042.kfoihb (mgr.14162) 298 : cluster [DBG] pgmap v247: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 341 B/s wr, 3 op/s 2024-02-15T18:25:08.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:06.838323+0000 mgr.smithi042.kfoihb (mgr.14162) 299 : cluster [DBG] pgmap v248: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 2 op/s 2024-02-15T18:25:08.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:06.838323+0000 mgr.smithi042.kfoihb (mgr.14162) 299 : cluster [DBG] pgmap v248: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 170 B/s wr, 2 op/s 2024-02-15T18:25:10.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:08.838831+0000 mgr.smithi042.kfoihb (mgr.14162) 300 : cluster [DBG] pgmap v249: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-02-15T18:25:10.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:08.838831+0000 mgr.smithi042.kfoihb (mgr.14162) 300 : cluster [DBG] pgmap v249: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-02-15T18:25:12.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:10.839405+0000 mgr.smithi042.kfoihb (mgr.14162) 301 : cluster [DBG] pgmap v250: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-02-15T18:25:12.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:10.839405+0000 mgr.smithi042.kfoihb (mgr.14162) 301 : cluster [DBG] pgmap v250: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-02-15T18:25:14.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:12.839973+0000 mgr.smithi042.kfoihb (mgr.14162) 302 : cluster [DBG] pgmap v251: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:14.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:12.839973+0000 mgr.smithi042.kfoihb (mgr.14162) 302 : cluster [DBG] pgmap v251: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:16.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:14.840501+0000 mgr.smithi042.kfoihb (mgr.14162) 303 : cluster [DBG] pgmap v252: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:25:16.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:14.840501+0000 mgr.smithi042.kfoihb (mgr.14162) 303 : cluster [DBG] pgmap v252: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:25:18.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:16.841143+0000 mgr.smithi042.kfoihb (mgr.14162) 304 : cluster [DBG] pgmap v253: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:18.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:16.841143+0000 mgr.smithi042.kfoihb (mgr.14162) 304 : cluster [DBG] pgmap v253: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:20.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:18.841546+0000 mgr.smithi042.kfoihb (mgr.14162) 305 : cluster [DBG] pgmap v254: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:25:20.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:18.841546+0000 mgr.smithi042.kfoihb (mgr.14162) 305 : cluster [DBG] pgmap v254: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:25:22.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:20.841941+0000 mgr.smithi042.kfoihb (mgr.14162) 306 : cluster [DBG] pgmap v255: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:22.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:20.841941+0000 mgr.smithi042.kfoihb (mgr.14162) 306 : cluster [DBG] pgmap v255: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:24.448 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:22.842283+0000 mgr.smithi042.kfoihb (mgr.14162) 307 : cluster [DBG] pgmap v256: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:24.448 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:23.733602+0000 mon.smithi042 (mon.0) 556 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:25:24.448 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:23.734279+0000 mon.smithi042 (mon.0) 557 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi191.gftdzo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:25:24.448 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:23.734973+0000 mon.smithi042 (mon.0) 558 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:25:24.448 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:23.735544+0000 mon.smithi042 (mon.0) 559 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:25:24.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:22.842283+0000 mgr.smithi042.kfoihb (mgr.14162) 307 : cluster [DBG] pgmap v256: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:24.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:23.733602+0000 mon.smithi042 (mon.0) 556 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:25:24.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:23.734279+0000 mon.smithi042 (mon.0) 557 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi191.gftdzo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:25:24.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:23.734973+0000 mon.smithi042 (mon.0) 558 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:25:24.811 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:23.735544+0000 mon.smithi042 (mon.0) 559 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:25:25.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:25:23.729256+0000 mgr.smithi042.kfoihb (mgr.14162) 308 : cephadm [INF] Upgrade: Updating mgr.smithi191.gftdzo 2024-02-15T18:25:25.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:25:23.736129+0000 mgr.smithi042.kfoihb (mgr.14162) 309 : cephadm [INF] Deploying daemon mgr.smithi191.gftdzo on smithi191 2024-02-15T18:25:25.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:25:23.729256+0000 mgr.smithi042.kfoihb (mgr.14162) 308 : cephadm [INF] Upgrade: Updating mgr.smithi191.gftdzo 2024-02-15T18:25:25.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:25:23.736129+0000 mgr.smithi042.kfoihb (mgr.14162) 309 : cephadm [INF] Deploying daemon mgr.smithi191.gftdzo on smithi191 2024-02-15T18:25:26.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:24.842882+0000 mgr.smithi042.kfoihb (mgr.14162) 310 : cluster [DBG] pgmap v257: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:25:26.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:24.842882+0000 mgr.smithi042.kfoihb (mgr.14162) 310 : cluster [DBG] pgmap v257: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:25:28.559 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:26.843318+0000 mgr.smithi042.kfoihb (mgr.14162) 311 : cluster [DBG] pgmap v258: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:28.560 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:27.278715+0000 mon.smithi042 (mon.0) 560 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:25:28.560 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:27.279526+0000 mon.smithi042 (mon.0) 561 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:25:28.560 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:26.843318+0000 mgr.smithi042.kfoihb (mgr.14162) 311 : cluster [DBG] pgmap v258: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:28.560 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:27.278715+0000 mon.smithi042 (mon.0) 560 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:25:28.560 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:27.279526+0000 mon.smithi042 (mon.0) 561 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:25:30.422 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:28.843819+0000 mgr.smithi042.kfoihb (mgr.14162) 312 : cluster [DBG] pgmap v259: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:25:30.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:28.843819+0000 mgr.smithi042.kfoihb (mgr.14162) 312 : cluster [DBG] pgmap v259: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:25:31.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.293+0000 7fc0199d6700 1 -- 172.21.15.42:0/150758003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc014060250 msgr2=0x7fc0140f91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:31.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.293+0000 7fc0199d6700 1 --2- 172.21.15.42:0/150758003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc014060250 0x7fc0140f91d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fc008004660 tx=0x7fc008009b30).stop 2024-02-15T18:25:31.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.293+0000 7fc0199d6700 1 -- 172.21.15.42:0/150758003 shutdown_connections 2024-02-15T18:25:31.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.293+0000 7fc0199d6700 1 --2- 172.21.15.42:0/150758003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0140f96c0 0x7fc0140fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.293+0000 7fc0199d6700 1 --2- 172.21.15.42:0/150758003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc014060250 0x7fc0140f91d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.293+0000 7fc0199d6700 1 -- 172.21.15.42:0/150758003 >> 172.21.15.42:0/150758003 conn(0x7fc0140f0b90 msgr2=0x7fc0140f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:31.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.293+0000 7fc0199d6700 1 -- 172.21.15.42:0/150758003 shutdown_connections 2024-02-15T18:25:31.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.293+0000 7fc0199d6700 1 -- 172.21.15.42:0/150758003 wait complete. 2024-02-15T18:25:31.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.293+0000 7fc0199d6700 1 Processor -- start 2024-02-15T18:25:31.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.293+0000 7fc0199d6700 1 -- start start 2024-02-15T18:25:31.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.294+0000 7fc0199d6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc014060250 0x7fc0140f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:31.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.294+0000 7fc0199d6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0140f96c0 0x7fc0140f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:31.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.294+0000 7fc0199d6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140f5630 con 0x7fc014060250 2024-02-15T18:25:31.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.294+0000 7fc0199d6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140f5770 con 0x7fc0140f96c0 2024-02-15T18:25:31.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.294+0000 7fc0189d4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc014060250 0x7fc0140f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:31.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.294+0000 7fc0189d4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc014060250 0x7fc0140f6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48234/0 (socket says 172.21.15.42:48234) 2024-02-15T18:25:31.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.294+0000 7fc0189d4700 1 -- 172.21.15.42:0/1352585902 learned_addr learned my addr 172.21.15.42:0/1352585902 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:25:31.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.294+0000 7fc013fff700 1 --2- 172.21.15.42:0/1352585902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0140f96c0 0x7fc0140f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:31.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.294+0000 7fc0189d4700 1 -- 172.21.15.42:0/1352585902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0140f96c0 msgr2=0x7fc0140f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:31.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.294+0000 7fc0189d4700 1 --2- 172.21.15.42:0/1352585902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0140f96c0 0x7fc0140f50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.294+0000 7fc0189d4700 1 -- 172.21.15.42:0/1352585902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc008005040 con 0x7fc014060250 2024-02-15T18:25:31.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.294+0000 7fc0189d4700 1 --2- 172.21.15.42:0/1352585902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc014060250 0x7fc0140f6c50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fc008000c00 tx=0x7fc00800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:31.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.295+0000 7fc0117fa700 1 -- 172.21.15.42:0/1352585902 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc008019070 con 0x7fc014060250 2024-02-15T18:25:31.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.295+0000 7fc0117fa700 1 -- 172.21.15.42:0/1352585902 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fc008007e80 con 0x7fc014060250 2024-02-15T18:25:31.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.295+0000 7fc0199d6700 1 -- 172.21.15.42:0/1352585902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0140f59f0 con 0x7fc014060250 2024-02-15T18:25:31.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.295+0000 7fc0117fa700 1 -- 172.21.15.42:0/1352585902 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc008014db0 con 0x7fc014060250 2024-02-15T18:25:31.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.295+0000 7fc0199d6700 1 -- 172.21.15.42:0/1352585902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0140f5f50 con 0x7fc014060250 2024-02-15T18:25:31.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.296+0000 7fc0199d6700 1 -- 172.21.15.42:0/1352585902 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc014045ad0 con 0x7fc014060250 2024-02-15T18:25:31.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.297+0000 7fc0117fa700 1 -- 172.21.15.42:0/1352585902 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fc008025070 con 0x7fc014060250 2024-02-15T18:25:31.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.297+0000 7fc0117fa700 1 --2- 172.21.15.42:0/1352585902 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc00405c2a0 0x7fc00405e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:31.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.297+0000 7fc0117fa700 1 -- 172.21.15.42:0/1352585902 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc008077260 con 0x7fc014060250 2024-02-15T18:25:31.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.297+0000 7fc013fff700 1 --2- 172.21.15.42:0/1352585902 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc00405c2a0 0x7fc00405e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:31.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.298+0000 7fc013fff700 1 --2- 172.21.15.42:0/1352585902 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc00405c2a0 0x7fc00405e760 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fc000009fd0 tx=0x7fc00000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:31.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.301+0000 7fc0117fa700 1 -- 172.21.15.42:0/1352585902 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc00804d990 con 0x7fc014060250 2024-02-15T18:25:31.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.501+0000 7fc0199d6700 1 -- 172.21.15.42:0/1352585902 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc014057f10 con 0x7fc00405c2a0 2024-02-15T18:25:31.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.502+0000 7fc0117fa700 1 -- 172.21.15.42:0/1352585902 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+274 (secure 0 0 0) 0x7fc014057f10 con 0x7fc00405c2a0 2024-02-15T18:25:31.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.504+0000 7fc0199d6700 1 -- 172.21.15.42:0/1352585902 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc00405c2a0 msgr2=0x7fc00405e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:31.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.504+0000 7fc0199d6700 1 --2- 172.21.15.42:0/1352585902 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc00405c2a0 0x7fc00405e760 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fc000009fd0 tx=0x7fc00000b040).stop 2024-02-15T18:25:31.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.504+0000 7fc0199d6700 1 -- 172.21.15.42:0/1352585902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc014060250 msgr2=0x7fc0140f6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:31.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.504+0000 7fc0199d6700 1 --2- 172.21.15.42:0/1352585902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc014060250 0x7fc0140f6c50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fc008000c00 tx=0x7fc00800b040).stop 2024-02-15T18:25:31.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.504+0000 7fc0199d6700 1 -- 172.21.15.42:0/1352585902 shutdown_connections 2024-02-15T18:25:31.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.504+0000 7fc0199d6700 1 --2- 172.21.15.42:0/1352585902 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fc00405c2a0 0x7fc00405e760 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.504+0000 7fc0199d6700 1 --2- 172.21.15.42:0/1352585902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc014060250 0x7fc0140f6c50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.504+0000 7fc0199d6700 1 --2- 172.21.15.42:0/1352585902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc0140f96c0 0x7fc0140f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.504+0000 7fc0199d6700 1 -- 172.21.15.42:0/1352585902 >> 172.21.15.42:0/1352585902 conn(0x7fc0140f0b90 msgr2=0x7fc0140f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:31.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.504+0000 7fc0199d6700 1 -- 172.21.15.42:0/1352585902 shutdown_connections 2024-02-15T18:25:31.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.505+0000 7fc0199d6700 1 -- 172.21.15.42:0/1352585902 wait complete. 2024-02-15T18:25:31.512 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:25:31.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.608+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/3780954607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd680f6ac0 msgr2=0x7fdd680f6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:31.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.608+0000 7fdd6d8fb700 1 --2- 172.21.15.42:0/3780954607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd680f6ac0 0x7fdd680f6ee0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fdd5c004660 tx=0x7fdd5c009b30).stop 2024-02-15T18:25:31.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.608+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/3780954607 shutdown_connections 2024-02-15T18:25:31.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.608+0000 7fdd6d8fb700 1 --2- 172.21.15.42:0/3780954607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd680f57d0 0x7fdd680f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.608+0000 7fdd6d8fb700 1 --2- 172.21.15.42:0/3780954607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd680f6ac0 0x7fdd680f6ee0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.608+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/3780954607 >> 172.21.15.42:0/3780954607 conn(0x7fdd680f0320 msgr2=0x7fdd680f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:31.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.609+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/3780954607 shutdown_connections 2024-02-15T18:25:31.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.609+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/3780954607 wait complete. 2024-02-15T18:25:31.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.609+0000 7fdd6d8fb700 1 Processor -- start 2024-02-15T18:25:31.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.609+0000 7fdd6d8fb700 1 -- start start 2024-02-15T18:25:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.609+0000 7fdd6d8fb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd680f57d0 0x7fdd68108fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.609+0000 7fdd6d8fb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd680f6ac0 0x7fdd681094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.609+0000 7fdd6d8fb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd68109a70 con 0x7fdd680f6ac0 2024-02-15T18:25:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.609+0000 7fdd6d8fb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd68109bb0 con 0x7fdd680f57d0 2024-02-15T18:25:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.610+0000 7fdd67fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd680f6ac0 0x7fdd681094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.610+0000 7fdd67fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd680f6ac0 0x7fdd681094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53980/0 (socket says 172.21.15.42:53980) 2024-02-15T18:25:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.610+0000 7fdd67fff700 1 -- 172.21.15.42:0/2012933237 learned_addr learned my addr 172.21.15.42:0/2012933237 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:25:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.610+0000 7fdd6c8f9700 1 --2- 172.21.15.42:0/2012933237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd680f57d0 0x7fdd68108fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.610+0000 7fdd67fff700 1 -- 172.21.15.42:0/2012933237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd680f57d0 msgr2=0x7fdd68108fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.610+0000 7fdd67fff700 1 --2- 172.21.15.42:0/2012933237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd680f57d0 0x7fdd68108fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.610+0000 7fdd67fff700 1 -- 172.21.15.42:0/2012933237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c005040 con 0x7fdd680f6ac0 2024-02-15T18:25:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.610+0000 7fdd67fff700 1 --2- 172.21.15.42:0/2012933237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd680f6ac0 0x7fdd681094c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fdd54002700 tx=0x7fdd54009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.610+0000 7fdd657fa700 1 -- 172.21.15.42:0/2012933237 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd54009320 con 0x7fdd680f6ac0 2024-02-15T18:25:31.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.610+0000 7fdd657fa700 1 -- 172.21.15.42:0/2012933237 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fdd5400f040 con 0x7fdd680f6ac0 2024-02-15T18:25:31.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.610+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/2012933237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6810a550 con 0x7fdd680f6ac0 2024-02-15T18:25:31.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.610+0000 7fdd657fa700 1 -- 172.21.15.42:0/2012933237 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd540149a0 con 0x7fdd680f6ac0 2024-02-15T18:25:31.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.610+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/2012933237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6810aa90 con 0x7fdd680f6ac0 2024-02-15T18:25:31.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.611+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/2012933237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd68045ad0 con 0x7fdd680f6ac0 2024-02-15T18:25:31.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.612+0000 7fdd657fa700 1 -- 172.21.15.42:0/2012933237 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fdd540077b0 con 0x7fdd680f6ac0 2024-02-15T18:25:31.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.612+0000 7fdd657fa700 1 --2- 172.21.15.42:0/2012933237 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fdd5805c300 0x7fdd5805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:31.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.612+0000 7fdd657fa700 1 -- 172.21.15.42:0/2012933237 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdd54076220 con 0x7fdd680f6ac0 2024-02-15T18:25:31.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.612+0000 7fdd6c8f9700 1 --2- 172.21.15.42:0/2012933237 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fdd5805c300 0x7fdd5805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:31.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.612+0000 7fdd6c8f9700 1 --2- 172.21.15.42:0/2012933237 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fdd5805c300 0x7fdd5805e7c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fdd5c004630 tx=0x7fdd5c009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:31.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.616+0000 7fdd657fa700 1 -- 172.21.15.42:0/2012933237 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fdd54048e30 con 0x7fdd680f6ac0 2024-02-15T18:25:31.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.809+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/2012933237 --> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd680faa20 con 0x7fdd5805c300 2024-02-15T18:25:31.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.810+0000 7fdd657fa700 1 -- 172.21.15.42:0/2012933237 <== mgr.14162 v2:172.21.15.42:6800/1691818213 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+274 (secure 0 0 0) 0x7fdd680faa20 con 0x7fdd5805c300 2024-02-15T18:25:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.812+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/2012933237 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fdd5805c300 msgr2=0x7fdd5805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.812+0000 7fdd6d8fb700 1 --2- 172.21.15.42:0/2012933237 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fdd5805c300 0x7fdd5805e7c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fdd5c004630 tx=0x7fdd5c009b30).stop 2024-02-15T18:25:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.812+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/2012933237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd680f6ac0 msgr2=0x7fdd681094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.812+0000 7fdd6d8fb700 1 --2- 172.21.15.42:0/2012933237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd680f6ac0 0x7fdd681094c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fdd54002700 tx=0x7fdd54009e00).stop 2024-02-15T18:25:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.812+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/2012933237 shutdown_connections 2024-02-15T18:25:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.812+0000 7fdd6d8fb700 1 --2- 172.21.15.42:0/2012933237 >> [v2:172.21.15.42:6800/1691818213,v1:172.21.15.42:6801/1691818213] conn(0x7fdd5805c300 0x7fdd5805e7c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.812+0000 7fdd6d8fb700 1 --2- 172.21.15.42:0/2012933237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd680f57d0 0x7fdd68108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.812+0000 7fdd6d8fb700 1 --2- 172.21.15.42:0/2012933237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd680f6ac0 0x7fdd681094c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.812+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/2012933237 >> 172.21.15.42:0/2012933237 conn(0x7fdd680f0320 msgr2=0x7fdd680f4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.812+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/2012933237 shutdown_connections 2024-02-15T18:25:31.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.812+0000 7fdd6d8fb700 1 -- 172.21.15.42:0/2012933237 wait complete. 2024-02-15T18:25:31.870 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:30.572277+0000 mon.smithi042 (mon.0) 562 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:25:31.870 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:30.870911+0000 mon.smithi042 (mon.0) 563 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:25:31.871 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:30.874236+0000 mon.smithi042 (mon.0) 564 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:25:31.871 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:30.877357+0000 mon.smithi042 (mon.0) 565 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr fail", "who": "smithi042.kfoihb"}]: dispatch 2024-02-15T18:25:31.871 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:30.881010+0000 mon.smithi042 (mon.0) 566 : cluster [DBG] osdmap e45: 6 total, 6 up, 6 in 2024-02-15T18:25:31.871 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:30.967586+0000 mon.smithi042 (mon.0) 567 : cluster [DBG] Standby manager daemon smithi042.kfoihb started 2024-02-15T18:25:31.897 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:30.572277+0000 mon.smithi042 (mon.0) 562 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:25:31.897 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:30.870911+0000 mon.smithi042 (mon.0) 563 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' 2024-02-15T18:25:31.897 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:30.874236+0000 mon.smithi042 (mon.0) 564 : audit [DBG] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:25:31.897 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:30.877357+0000 mon.smithi042 (mon.0) 565 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr fail", "who": "smithi042.kfoihb"}]: dispatch 2024-02-15T18:25:31.897 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:30.881010+0000 mon.smithi042 (mon.0) 566 : cluster [DBG] osdmap e45: 6 total, 6 up, 6 in 2024-02-15T18:25:31.897 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:30.967586+0000 mon.smithi042 (mon.0) 567 : cluster [DBG] Standby manager daemon smithi042.kfoihb started 2024-02-15T18:25:31.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.922+0000 7f3339644700 1 -- 172.21.15.42:0/3349505437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33340f55c0 msgr2=0x7f33340f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.922+0000 7f3339644700 1 --2- 172.21.15.42:0/3349505437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33340f55c0 0x7f33340f59e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f3324004660 tx=0x7f3324009b30).stop 2024-02-15T18:25:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.922+0000 7f3339644700 1 -- 172.21.15.42:0/3349505437 shutdown_connections 2024-02-15T18:25:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.922+0000 7f3339644700 1 --2- 172.21.15.42:0/3349505437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33340f6860 0x7f33340f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.922+0000 7f3339644700 1 --2- 172.21.15.42:0/3349505437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33340f55c0 0x7f33340f59e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.922+0000 7f3339644700 1 -- 172.21.15.42:0/3349505437 >> 172.21.15.42:0/3349505437 conn(0x7f33340f0d40 msgr2=0x7f33340f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.923+0000 7f3339644700 1 -- 172.21.15.42:0/3349505437 shutdown_connections 2024-02-15T18:25:31.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.923+0000 7f3339644700 1 -- 172.21.15.42:0/3349505437 wait complete. 2024-02-15T18:25:31.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.923+0000 7f3339644700 1 Processor -- start 2024-02-15T18:25:31.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.923+0000 7f3339644700 1 -- start start 2024-02-15T18:25:31.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.923+0000 7f3339644700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33340f55c0 0x7f3334107010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:31.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f3339644700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33340f6860 0x7f3334107500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:31.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f3339644700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3334107ab0 con 0x7f33340f55c0 2024-02-15T18:25:31.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f3339644700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3334107bf0 con 0x7f33340f6860 2024-02-15T18:25:31.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f3332ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33340f55c0 0x7f3334107010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:31.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f3332ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33340f55c0 0x7f3334107010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54014/0 (socket says 172.21.15.42:54014) 2024-02-15T18:25:31.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f3332ffd700 1 -- 172.21.15.42:0/729985654 learned_addr learned my addr 172.21.15.42:0/729985654 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:25:31.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f33327fc700 1 --2- 172.21.15.42:0/729985654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33340f6860 0x7f3334107500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:31.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f3332ffd700 1 -- 172.21.15.42:0/729985654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33340f6860 msgr2=0x7f3334107500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:31.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f3332ffd700 1 --2- 172.21.15.42:0/729985654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33340f6860 0x7f3334107500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:31.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f3332ffd700 1 -- 172.21.15.42:0/729985654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3324005040 con 0x7f33340f55c0 2024-02-15T18:25:31.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f33327fc700 1 --2- 172.21.15.42:0/729985654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33340f6860 0x7f3334107500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:25:31.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f3332ffd700 1 --2- 172.21.15.42:0/729985654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33340f55c0 0x7f3334107010 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f3324002040 tx=0x7f332400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:31.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f331b7fe700 1 -- 172.21.15.42:0/729985654 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3324019070 con 0x7f33340f55c0 2024-02-15T18:25:31.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f331b7fe700 1 -- 172.21.15.42:0/729985654 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f3324007e80 con 0x7f33340f55c0 2024-02-15T18:25:31.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f331b7fe700 1 -- 172.21.15.42:0/729985654 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3324014db0 con 0x7f33340f55c0 2024-02-15T18:25:31.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.924+0000 7f3339644700 1 -- 172.21.15.42:0/729985654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3334108530 con 0x7f33340f55c0 2024-02-15T18:25:31.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.925+0000 7f3339644700 1 -- 172.21.15.42:0/729985654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3334108a90 con 0x7f33340f55c0 2024-02-15T18:25:31.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.926+0000 7f3339644700 1 -- 172.21.15.42:0/729985654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3334045ad0 con 0x7f33340f55c0 2024-02-15T18:25:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.926+0000 7f331b7fe700 1 -- 172.21.15.42:0/729985654 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 71921+0+0 (secure 0 0 0) 0x7f3324025070 con 0x7f33340f55c0 2024-02-15T18:25:31.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.927+0000 7f331b7fe700 1 -- 172.21.15.42:0/729985654 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3324076af0 con 0x7f33340f55c0 2024-02-15T18:25:31.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:31.932+0000 7f331b7fe700 1 -- 172.21.15.42:0/729985654 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f33240497b0 con 0x7f33340f55c0 2024-02-15T18:25:32.780 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:30.844248+0000 mgr.smithi042.kfoihb (mgr.14162) 313 : cluster [DBG] pgmap v260: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:32.780 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:25:30.875104+0000 mgr.smithi042.kfoihb (mgr.14162) 314 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi042.kfoihb) 2024-02-15T18:25:32.780 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:25:30.877148+0000 mgr.smithi042.kfoihb (mgr.14162) 315 : cephadm [INF] Failing over to other MGR 2024-02-15T18:25:32.780 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:31.503376+0000 mgr.smithi042.kfoihb (mgr.14162) 316 : audit [DBG] from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:25:32.780 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:31.876683+0000 mon.smithi042 (mon.0) 568 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "mgr fail", "who": "smithi042.kfoihb"}]': finished 2024-02-15T18:25:32.780 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:31.876753+0000 mon.smithi042 (mon.0) 569 : cluster [DBG] mgrmap e19: smithi191.gftdzo(active, starting, since 0.998841s), standbys: smithi042.kfoihb 2024-02-15T18:25:33.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:30.844248+0000 mgr.smithi042.kfoihb (mgr.14162) 313 : cluster [DBG] pgmap v260: 65 pgs: 65 active+clean; 11 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:33.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:25:30.875104+0000 mgr.smithi042.kfoihb (mgr.14162) 314 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi042.kfoihb) 2024-02-15T18:25:33.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:25:30.877148+0000 mgr.smithi042.kfoihb (mgr.14162) 315 : cephadm [INF] Failing over to other MGR 2024-02-15T18:25:33.061 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:31.503376+0000 mgr.smithi042.kfoihb (mgr.14162) 316 : audit [DBG] from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:25:33.061 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:31.876683+0000 mon.smithi042 (mon.0) 568 : audit [INF] from='mgr.14162 172.21.15.42:0/2590505844' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "mgr fail", "who": "smithi042.kfoihb"}]': finished 2024-02-15T18:25:33.061 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:31.876753+0000 mon.smithi042 (mon.0) 569 : cluster [DBG] mgrmap e19: smithi191.gftdzo(active, starting, since 0.998841s), standbys: smithi042.kfoihb 2024-02-15T18:25:33.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:33.583+0000 7f331b7fe700 1 -- 172.21.15.42:0/729985654 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 20) v1 ==== 74396+0+0 (secure 0 0 0) 0x7f3324074c70 con 0x7f33340f55c0 2024-02-15T18:25:33.834 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:32.796231+0000 mon.smithi042 (mon.0) 570 : cluster [INF] Active manager daemon smithi191.gftdzo restarted 2024-02-15T18:25:33.835 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:32.796811+0000 mon.smithi042 (mon.0) 571 : cluster [INF] Activating manager daemon smithi191.gftdzo 2024-02-15T18:25:33.835 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:32.799481+0000 mon.smithi042 (mon.0) 572 : cluster [DBG] osdmap e46: 6 total, 6 up, 6 in 2024-02-15T18:25:33.835 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:32.800170+0000 mon.smithi191 (mon.1) 15 : audit [DBG] from='mgr.? 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/crt"}]: dispatch 2024-02-15T18:25:33.835 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:32.800545+0000 mon.smithi191 (mon.1) 16 : audit [DBG] from='mgr.? 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:25:33.835 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:32.802912+0000 mon.smithi191 (mon.1) 17 : audit [DBG] from='mgr.? 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/key"}]: dispatch 2024-02-15T18:25:33.835 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:32.803164+0000 mon.smithi191 (mon.1) 18 : audit [DBG] from='mgr.? 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:25:33.840 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:32.796231+0000 mon.smithi042 (mon.0) 570 : cluster [INF] Active manager daemon smithi191.gftdzo restarted 2024-02-15T18:25:33.840 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:32.796811+0000 mon.smithi042 (mon.0) 571 : cluster [INF] Activating manager daemon smithi191.gftdzo 2024-02-15T18:25:33.840 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:32.799481+0000 mon.smithi042 (mon.0) 572 : cluster [DBG] osdmap e46: 6 total, 6 up, 6 in 2024-02-15T18:25:33.840 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:32.800170+0000 mon.smithi191 (mon.1) 15 : audit [DBG] from='mgr.? 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/crt"}]: dispatch 2024-02-15T18:25:33.841 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:32.800545+0000 mon.smithi191 (mon.1) 16 : audit [DBG] from='mgr.? 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:25:33.841 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:32.802912+0000 mon.smithi191 (mon.1) 17 : audit [DBG] from='mgr.? 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/key"}]: dispatch 2024-02-15T18:25:33.841 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:32.803164+0000 mon.smithi191 (mon.1) 18 : audit [DBG] from='mgr.? 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:25:34.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.591+0000 7f331b7fe700 1 -- 172.21.15.42:0/729985654 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 21) v1 ==== 74466+0+0 (secure 0 0 0) 0x7f3324074ff0 con 0x7f33340f55c0 2024-02-15T18:25:34.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.591+0000 7f331b7fe700 1 --2- 172.21.15.42:0/729985654 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f331c0267b0 0x7f331c028bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:34.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.591+0000 7f331b7fe700 1 -- 172.21.15.42:0/729985654 --> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f331c0343d0 con 0x7f331c0267b0 2024-02-15T18:25:34.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.608+0000 7f33327fc700 1 --2- 172.21.15.42:0/729985654 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f331c0267b0 0x7f331c028bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:34.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.608+0000 7f33327fc700 1 --2- 172.21.15.42:0/729985654 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f331c0267b0 0x7f331c028bc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f3328009fd0 tx=0x7f332800b040).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:34.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.620+0000 7f331b7fe700 1 -- 172.21.15.42:0/729985654 <== mgr.24395 v2:172.21.15.191:6828/2058110468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3432 (secure 0 0 0) 0x7f332800f040 con 0x7f331c0267b0 2024-02-15T18:25:34.618 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:25:34.618 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (5m) 84s ago 7m 17.6M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:25:34.618 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (7m) 84s ago 7m 6400k - 16.2.4 8d91d370c2b8 94d74c33c758 2024-02-15T18:25:34.618 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (6m) 4s ago 6m 6405k - 16.2.4 8d91d370c2b8 ff0f56f9475a 2024-02-15T18:25:34.618 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (5m) 84s ago 6m 28.8M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:25:34.618 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (102s) 84s ago 101s 11.9M - 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:25:34.618 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (107s) 84s ago 107s 14.6M - 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:25:34.618 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (105s) 4s ago 104s 16.6M - 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:25:34.618 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (112s) 4s ago 111s 18.2M - 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:25:34.618 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:9283 running (8m) 84s ago 8m 408M - 16.2.4 8d91d370c2b8 a56a503c9572 2024-02-15T18:25:34.619 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (8s) 4s ago 6m 43.1M - 16.2.14-521-g0e714d9a 1fbc92c859dc b772ba3fed7c 2024-02-15T18:25:34.619 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (8m) 84s ago 8m 61.1M 2048M 16.2.4 8d91d370c2b8 9dcbc5c49e05 2024-02-15T18:25:34.619 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (5m) 4s ago 5m 38.4M 2048M 16.2.4 8d91d370c2b8 c3dd4902e2a2 2024-02-15T18:25:34.619 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (6m) 84s ago 6m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:25:34.619 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (5m) 4s ago 5m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:25:34.619 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (5m) 84s ago 5m 39.9M 4096M 16.2.4 8d91d370c2b8 973745b31d5a 2024-02-15T18:25:34.619 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (4m) 84s ago 4m 42.2M 4096M 16.2.4 8d91d370c2b8 2b3ae86111e8 2024-02-15T18:25:34.619 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (4m) 84s ago 4m 39.5M 4096M 16.2.4 8d91d370c2b8 3706071e4698 2024-02-15T18:25:34.619 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (4m) 4s ago 4m 39.8M 4096M 16.2.4 8d91d370c2b8 e6fe8e77c6fe 2024-02-15T18:25:34.619 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (3m) 4s ago 3m 42.2M 4096M 16.2.4 8d91d370c2b8 24582012e234 2024-02-15T18:25:34.619 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (3m) 4s ago 3m 39.3M 4096M 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:25:34.619 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (4m) 84s ago 6m 32.9M - 2.18.1 de242295e225 7fffb9a3be63 2024-02-15T18:25:34.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.622+0000 7f3339644700 1 -- 172.21.15.42:0/729985654 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f331c0267b0 msgr2=0x7f331c028bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:34.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.622+0000 7f3339644700 1 --2- 172.21.15.42:0/729985654 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f331c0267b0 0x7f331c028bc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f3328009fd0 tx=0x7f332800b040).stop 2024-02-15T18:25:34.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.622+0000 7f3339644700 1 -- 172.21.15.42:0/729985654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33340f55c0 msgr2=0x7f3334107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:34.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.622+0000 7f3339644700 1 --2- 172.21.15.42:0/729985654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33340f55c0 0x7f3334107010 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f3324002040 tx=0x7f332400b040).stop 2024-02-15T18:25:34.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.622+0000 7f3339644700 1 -- 172.21.15.42:0/729985654 shutdown_connections 2024-02-15T18:25:34.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.622+0000 7f3339644700 1 --2- 172.21.15.42:0/729985654 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f331c0267b0 0x7f331c028bc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:34.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.622+0000 7f3339644700 1 --2- 172.21.15.42:0/729985654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33340f55c0 0x7f3334107010 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:34.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.622+0000 7f3339644700 1 --2- 172.21.15.42:0/729985654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33340f6860 0x7f3334107500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:34.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.622+0000 7f3339644700 1 -- 172.21.15.42:0/729985654 >> 172.21.15.42:0/729985654 conn(0x7f33340f0d40 msgr2=0x7f33340fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:34.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.622+0000 7f3339644700 1 -- 172.21.15.42:0/729985654 shutdown_connections 2024-02-15T18:25:34.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.622+0000 7f3339644700 1 -- 172.21.15.42:0/729985654 wait complete. 2024-02-15T18:25:34.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.727+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/2165889391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b440f66d0 msgr2=0x7f6b440f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:34.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.727+0000 7f6b4a7bc700 1 --2- 172.21.15.42:0/2165889391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b440f66d0 0x7f6b440f6b30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f6b34004660 tx=0x7f6b34009b30).stop 2024-02-15T18:25:34.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.727+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/2165889391 shutdown_connections 2024-02-15T18:25:34.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.727+0000 7f6b4a7bc700 1 --2- 172.21.15.42:0/2165889391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b440f66d0 0x7f6b440f6b30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:34.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.727+0000 7f6b4a7bc700 1 --2- 172.21.15.42:0/2165889391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b440f5430 0x7f6b440f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:34.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.727+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/2165889391 >> 172.21.15.42:0/2165889391 conn(0x7f6b440f0b90 msgr2=0x7f6b440f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:34.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.727+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/2165889391 shutdown_connections 2024-02-15T18:25:34.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.727+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/2165889391 wait complete. 2024-02-15T18:25:34.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.727+0000 7f6b4a7bc700 1 Processor -- start 2024-02-15T18:25:34.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.727+0000 7f6b4a7bc700 1 -- start start 2024-02-15T18:25:34.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.727+0000 7f6b4a7bc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b440f5430 0x7f6b44106e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:34.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b4a7bc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b440f66d0 0x7f6b44107320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:34.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b4a7bc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b44107920 con 0x7f6b440f66d0 2024-02-15T18:25:34.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b4a7bc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b44108130 con 0x7f6b440f5430 2024-02-15T18:25:34.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b48fb9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b440f66d0 0x7f6b44107320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:34.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b48fb9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b440f66d0 0x7f6b44107320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54032/0 (socket says 172.21.15.42:54032) 2024-02-15T18:25:34.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b48fb9700 1 -- 172.21.15.42:0/1795558424 learned_addr learned my addr 172.21.15.42:0/1795558424 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:25:34.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b497ba700 1 --2- 172.21.15.42:0/1795558424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b440f5430 0x7f6b44106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:34.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b48fb9700 1 -- 172.21.15.42:0/1795558424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b440f5430 msgr2=0x7f6b44106e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:34.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b48fb9700 1 --2- 172.21.15.42:0/1795558424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b440f5430 0x7f6b44106e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:34.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b48fb9700 1 -- 172.21.15.42:0/1795558424 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b34005040 con 0x7f6b440f66d0 2024-02-15T18:25:34.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b497ba700 1 --2- 172.21.15.42:0/1795558424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b440f5430 0x7f6b44106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:25:34.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b48fb9700 1 --2- 172.21.15.42:0/1795558424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b440f66d0 0x7f6b44107320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f6b34009f50 tx=0x7f6b34009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:34.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b39ffb700 1 -- 172.21.15.42:0/1795558424 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b34018070 con 0x7f6b440f66d0 2024-02-15T18:25:34.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b39ffb700 1 -- 172.21.15.42:0/1795558424 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f6b34007d70 con 0x7f6b440f66d0 2024-02-15T18:25:34.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/1795558424 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b44108350 con 0x7f6b440f66d0 2024-02-15T18:25:34.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.728+0000 7f6b39ffb700 1 -- 172.21.15.42:0/1795558424 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b3401c620 con 0x7f6b440f66d0 2024-02-15T18:25:34.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.729+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/1795558424 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b441088b0 con 0x7f6b440f66d0 2024-02-15T18:25:34.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.730+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/1795558424 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b44059100 con 0x7f6b440f66d0 2024-02-15T18:25:34.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.730+0000 7f6b39ffb700 1 -- 172.21.15.42:0/1795558424 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) v1 ==== 74466+0+0 (secure 0 0 0) 0x7f6b3401c8e0 con 0x7f6b440f66d0 2024-02-15T18:25:34.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.731+0000 7f6b39ffb700 1 --2- 172.21.15.42:0/1795558424 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f6b3005e740 0x7f6b30060c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:34.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.731+0000 7f6b39ffb700 1 -- 172.21.15.42:0/1795558424 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6b3407acc0 con 0x7f6b440f66d0 2024-02-15T18:25:34.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.731+0000 7f6b497ba700 1 --2- 172.21.15.42:0/1795558424 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f6b3005e740 0x7f6b30060c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:34.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.732+0000 7f6b497ba700 1 --2- 172.21.15.42:0/1795558424 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f6b3005e740 0x7f6b30060c00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f6b44106ba0 tx=0x7f6b40009040).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:34.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:34.735+0000 7f6b39ffb700 1 -- 172.21.15.42:0/1795558424 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7f6b3404b3b0 con 0x7f6b440f66d0 2024-02-15T18:25:34.948 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:33.585650+0000 mon.smithi042 (mon.0) 573 : cluster [DBG] mgrmap e20: smithi191.gftdzo(active, starting, since 0.788996s), standbys: smithi042.kfoihb 2024-02-15T18:25:34.948 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.586786+0000 mon.smithi191 (mon.1) 19 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-02-15T18:25:34.948 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.586986+0000 mon.smithi191 (mon.1) 20 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:25:34.948 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.587574+0000 mon.smithi191 (mon.1) 21 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.vtwvlo"}]: dispatch 2024-02-15T18:25:34.948 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.587856+0000 mon.smithi191 (mon.1) 22 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.idnxkc"}]: dispatch 2024-02-15T18:25:34.948 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.588105+0000 mon.smithi191 (mon.1) 23 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.icwcnb"}]: dispatch 2024-02-15T18:25:34.948 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.588356+0000 mon.smithi191 (mon.1) 24 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.erlpxp"}]: dispatch 2024-02-15T18:25:34.948 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.588621+0000 mon.smithi191 (mon.1) 25 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mgr metadata", "who": "smithi191.gftdzo", "id": "smithi191.gftdzo"}]: dispatch 2024-02-15T18:25:34.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.588840+0000 mon.smithi191 (mon.1) 26 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mgr metadata", "who": "smithi042.kfoihb", "id": "smithi042.kfoihb"}]: dispatch 2024-02-15T18:25:34.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.589055+0000 mon.smithi191 (mon.1) 27 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:25:34.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.589302+0000 mon.smithi191 (mon.1) 28 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:25:34.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.589550+0000 mon.smithi191 (mon.1) 29 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:25:34.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.589781+0000 mon.smithi191 (mon.1) 30 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:25:34.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.590055+0000 mon.smithi191 (mon.1) 31 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:25:34.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.590324+0000 mon.smithi191 (mon.1) 32 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:25:34.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.590644+0000 mon.smithi191 (mon.1) 33 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:25:34.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.590921+0000 mon.smithi191 (mon.1) 34 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:25:34.949 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.591260+0000 mon.smithi191 (mon.1) 35 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:25:34.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:33.607066+0000 mon.smithi042 (mon.0) 574 : cluster [INF] Manager daemon smithi191.gftdzo is now available 2024-02-15T18:25:34.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.716721+0000 mon.smithi191 (mon.1) 36 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:25:34.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.718539+0000 mon.smithi191 (mon.1) 37 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:25:34.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.746433+0000 mon.smithi042 (mon.0) 575 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:34.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.769946+0000 mon.smithi191 (mon.1) 38 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi191.gftdzo/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:25:34.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.770436+0000 mon.smithi042 (mon.0) 576 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi191.gftdzo/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:25:34.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.781760+0000 mon.smithi191 (mon.1) 39 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi191.gftdzo/trash_purge_schedule"}]: dispatch 2024-02-15T18:25:34.950 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:33.782165+0000 mon.smithi042 (mon.0) 577 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi191.gftdzo/trash_purge_schedule"}]: dispatch 2024-02-15T18:25:34.996 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:33.585650+0000 mon.smithi042 (mon.0) 573 : cluster [DBG] mgrmap e20: smithi191.gftdzo(active, starting, since 0.788996s), standbys: smithi042.kfoihb 2024-02-15T18:25:34.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.586786+0000 mon.smithi191 (mon.1) 19 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-02-15T18:25:34.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.586986+0000 mon.smithi191 (mon.1) 20 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:25:34.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.587574+0000 mon.smithi191 (mon.1) 21 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.vtwvlo"}]: dispatch 2024-02-15T18:25:34.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.587856+0000 mon.smithi191 (mon.1) 22 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.idnxkc"}]: dispatch 2024-02-15T18:25:34.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.588105+0000 mon.smithi191 (mon.1) 23 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.icwcnb"}]: dispatch 2024-02-15T18:25:34.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.588356+0000 mon.smithi191 (mon.1) 24 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.erlpxp"}]: dispatch 2024-02-15T18:25:34.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.588621+0000 mon.smithi191 (mon.1) 25 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mgr metadata", "who": "smithi191.gftdzo", "id": "smithi191.gftdzo"}]: dispatch 2024-02-15T18:25:34.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.588840+0000 mon.smithi191 (mon.1) 26 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mgr metadata", "who": "smithi042.kfoihb", "id": "smithi042.kfoihb"}]: dispatch 2024-02-15T18:25:34.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.589055+0000 mon.smithi191 (mon.1) 27 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:25:34.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.589302+0000 mon.smithi191 (mon.1) 28 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:25:34.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.589550+0000 mon.smithi191 (mon.1) 29 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:25:34.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.589781+0000 mon.smithi191 (mon.1) 30 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:25:34.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.590055+0000 mon.smithi191 (mon.1) 31 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:25:34.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.590324+0000 mon.smithi191 (mon.1) 32 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:25:34.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.590644+0000 mon.smithi191 (mon.1) 33 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:25:34.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.590921+0000 mon.smithi191 (mon.1) 34 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:25:34.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.591260+0000 mon.smithi191 (mon.1) 35 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:25:34.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:33.607066+0000 mon.smithi042 (mon.0) 574 : cluster [INF] Manager daemon smithi191.gftdzo is now available 2024-02-15T18:25:34.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.716721+0000 mon.smithi191 (mon.1) 36 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:25:34.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.718539+0000 mon.smithi191 (mon.1) 37 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:25:34.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.746433+0000 mon.smithi042 (mon.0) 575 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:34.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.769946+0000 mon.smithi191 (mon.1) 38 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi191.gftdzo/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:25:34.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.770436+0000 mon.smithi042 (mon.0) 576 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi191.gftdzo/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:25:34.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.781760+0000 mon.smithi191 (mon.1) 39 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi191.gftdzo/trash_purge_schedule"}]: dispatch 2024-02-15T18:25:34.999 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:33.782165+0000 mon.smithi042 (mon.0) 577 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi191.gftdzo/trash_purge_schedule"}]: dispatch 2024-02-15T18:25:35.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.016+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/1795558424 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6b44045b20 con 0x7f6b440f66d0 2024-02-15T18:25:35.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.017+0000 7f6b39ffb700 1 -- 172.21.15.42:0/1795558424 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f6b3404e9d0 con 0x7f6b440f66d0 2024-02-15T18:25:35.015 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:25:35.015 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:25:35.015 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:25:35.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:25:35.015 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:25:35.015 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 1, 2024-02-15T18:25:35.015 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 1 2024-02-15T18:25:35.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:25:35.015 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:25:35.015 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:25:35.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:25:35.015 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:25:35.015 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:25:35.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:25:35.016 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:25:35.016 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 1, 2024-02-15T18:25:35.016 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 13 2024-02-15T18:25:35.016 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:25:35.016 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:25:35.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.019+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/1795558424 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f6b3005e740 msgr2=0x7f6b30060c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:35.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.019+0000 7f6b4a7bc700 1 --2- 172.21.15.42:0/1795558424 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f6b3005e740 0x7f6b30060c00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f6b44106ba0 tx=0x7f6b40009040).stop 2024-02-15T18:25:35.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.019+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/1795558424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b440f66d0 msgr2=0x7f6b44107320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:35.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.019+0000 7f6b4a7bc700 1 --2- 172.21.15.42:0/1795558424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b440f66d0 0x7f6b44107320 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f6b34009f50 tx=0x7f6b34009b30).stop 2024-02-15T18:25:35.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.021+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/1795558424 shutdown_connections 2024-02-15T18:25:35.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.021+0000 7f6b4a7bc700 1 --2- 172.21.15.42:0/1795558424 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f6b3005e740 0x7f6b30060c00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.021+0000 7f6b4a7bc700 1 --2- 172.21.15.42:0/1795558424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b440f5430 0x7f6b44106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.021+0000 7f6b4a7bc700 1 --2- 172.21.15.42:0/1795558424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b440f66d0 0x7f6b44107320 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.021+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/1795558424 >> 172.21.15.42:0/1795558424 conn(0x7f6b440f0b90 msgr2=0x7f6b4405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:35.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.021+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/1795558424 shutdown_connections 2024-02-15T18:25:35.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.021+0000 7f6b4a7bc700 1 -- 172.21.15.42:0/1795558424 wait complete. 2024-02-15T18:25:35.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.132+0000 7f920a991700 1 -- 172.21.15.42:0/592499281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92040f55c0 msgr2=0x7f92040f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:35.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.132+0000 7f920a991700 1 --2- 172.21.15.42:0/592499281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92040f55c0 0x7f92040f59e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f91f4004660 tx=0x7f91f4009b30).stop 2024-02-15T18:25:35.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.132+0000 7f920a991700 1 -- 172.21.15.42:0/592499281 shutdown_connections 2024-02-15T18:25:35.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.132+0000 7f920a991700 1 --2- 172.21.15.42:0/592499281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92040f6860 0x7f92040f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.132+0000 7f920a991700 1 --2- 172.21.15.42:0/592499281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92040f55c0 0x7f92040f59e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.132+0000 7f920a991700 1 -- 172.21.15.42:0/592499281 >> 172.21.15.42:0/592499281 conn(0x7f92040f0d40 msgr2=0x7f92040f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:35.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.134+0000 7f920a991700 1 -- 172.21.15.42:0/592499281 shutdown_connections 2024-02-15T18:25:35.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.134+0000 7f920a991700 1 -- 172.21.15.42:0/592499281 wait complete. 2024-02-15T18:25:35.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.135+0000 7f920a991700 1 Processor -- start 2024-02-15T18:25:35.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.135+0000 7f920a991700 1 -- start start 2024-02-15T18:25:35.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.135+0000 7f920a991700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92040f55c0 0x7f9204106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:35.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.135+0000 7f920a991700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92040f6860 0x7f92041074e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:35.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.135+0000 7f920a991700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9204107a90 con 0x7f92040f55c0 2024-02-15T18:25:35.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.135+0000 7f920a991700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9204107bd0 con 0x7f92040f6860 2024-02-15T18:25:35.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.135+0000 7f9203fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92040f55c0 0x7f9204106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:35.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.136+0000 7f9203fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92040f55c0 0x7f9204106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54060/0 (socket says 172.21.15.42:54060) 2024-02-15T18:25:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.136+0000 7f9203fff700 1 -- 172.21.15.42:0/3455558641 learned_addr learned my addr 172.21.15.42:0/3455558641 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:25:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.136+0000 7f92037fe700 1 --2- 172.21.15.42:0/3455558641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92040f6860 0x7f92041074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.136+0000 7f9203fff700 1 -- 172.21.15.42:0/3455558641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92040f6860 msgr2=0x7f92041074e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.136+0000 7f9203fff700 1 --2- 172.21.15.42:0/3455558641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92040f6860 0x7f92041074e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.136+0000 7f9203fff700 1 -- 172.21.15.42:0/3455558641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f4005040 con 0x7f92040f55c0 2024-02-15T18:25:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.136+0000 7f9203fff700 1 --2- 172.21.15.42:0/3455558641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92040f55c0 0x7f9204106ff0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f91f4004e30 tx=0x7f91f4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.136+0000 7f9200ff9700 1 -- 172.21.15.42:0/3455558641 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91f4018070 con 0x7f92040f55c0 2024-02-15T18:25:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.136+0000 7f9200ff9700 1 -- 172.21.15.42:0/3455558641 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f91f4007ba0 con 0x7f92040f55c0 2024-02-15T18:25:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.136+0000 7f9200ff9700 1 -- 172.21.15.42:0/3455558641 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91f401c3c0 con 0x7f92040f55c0 2024-02-15T18:25:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.136+0000 7f920a991700 1 -- 172.21.15.42:0/3455558641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9204108510 con 0x7f92040f55c0 2024-02-15T18:25:35.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.136+0000 7f920a991700 1 -- 172.21.15.42:0/3455558641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9204108a20 con 0x7f92040f55c0 2024-02-15T18:25:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.137+0000 7f920a991700 1 -- 172.21.15.42:0/3455558641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9204045ad0 con 0x7f92040f55c0 2024-02-15T18:25:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.138+0000 7f9200ff9700 1 -- 172.21.15.42:0/3455558641 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) v1 ==== 74466+0+0 (secure 0 0 0) 0x7f91f401d420 con 0x7f92040f55c0 2024-02-15T18:25:35.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.138+0000 7f9200ff9700 1 --2- 172.21.15.42:0/3455558641 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f91ec05e6f0 0x7f91ec060bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.138+0000 7f9200ff9700 1 -- 172.21.15.42:0/3455558641 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f91f407a980 con 0x7f92040f55c0 2024-02-15T18:25:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.138+0000 7f92037fe700 1 --2- 172.21.15.42:0/3455558641 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f91ec05e6f0 0x7f91ec060bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:35.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.139+0000 7f92037fe700 1 --2- 172.21.15.42:0/3455558641 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f91ec05e6f0 0x7f91ec060bb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f91f8004c40 tx=0x7f91f800b040).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:35.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.143+0000 7f9200ff9700 1 -- 172.21.15.42:0/3455558641 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7f91f404aff0 con 0x7f92040f55c0 2024-02-15T18:25:35.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.375+0000 7f920a991700 1 -- 172.21.15.42:0/3455558641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9204109230 con 0x7f92040f55c0 2024-02-15T18:25:35.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.375+0000 7f9200ff9700 1 -- 172.21.15.42:0/3455558641 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1626 (secure 0 0 0) 0x7f91f404e610 con 0x7f92040f55c0 2024-02-15T18:25:35.373 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:25:35.373 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:25:35.373 INFO:teuthology.orchestra.run.smithi042.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:25:35.373 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:25:35.374 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:25:35.374 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:25:35.374 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:25:35.374 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:25:35.374 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:25:35.374 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:25:35.374 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:25:35.374 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:25:35.374 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:25:35.374 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:25:35.374 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:25:35.374 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:25:35.374 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:25:35.374 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000]] 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509]] 2024-02-15T18:25:35.375 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:25:35.376 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:25:35.376 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:25:35.376 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:25:35.376 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366]] 2024-02-15T18:25:35.376 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796]] 2024-02-15T18:25:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.377+0000 7f920a991700 1 -- 172.21.15.42:0/3455558641 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f91ec05e6f0 msgr2=0x7f91ec060bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.377+0000 7f920a991700 1 --2- 172.21.15.42:0/3455558641 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f91ec05e6f0 0x7f91ec060bb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f91f8004c40 tx=0x7f91f800b040).stop 2024-02-15T18:25:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.377+0000 7f920a991700 1 -- 172.21.15.42:0/3455558641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92040f55c0 msgr2=0x7f9204106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.377+0000 7f920a991700 1 --2- 172.21.15.42:0/3455558641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92040f55c0 0x7f9204106ff0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f91f4004e30 tx=0x7f91f4009b30).stop 2024-02-15T18:25:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.377+0000 7f920a991700 1 -- 172.21.15.42:0/3455558641 shutdown_connections 2024-02-15T18:25:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.377+0000 7f920a991700 1 --2- 172.21.15.42:0/3455558641 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f91ec05e6f0 0x7f91ec060bb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.377+0000 7f920a991700 1 --2- 172.21.15.42:0/3455558641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92040f55c0 0x7f9204106ff0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.377+0000 7f920a991700 1 --2- 172.21.15.42:0/3455558641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92040f6860 0x7f92041074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.377+0000 7f920a991700 1 -- 172.21.15.42:0/3455558641 >> 172.21.15.42:0/3455558641 conn(0x7f92040f0d40 msgr2=0x7f920405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.377+0000 7f920a991700 1 -- 172.21.15.42:0/3455558641 shutdown_connections 2024-02-15T18:25:35.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.377+0000 7f920a991700 1 -- 172.21.15.42:0/3455558641 wait complete. 2024-02-15T18:25:35.378 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:25:35.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.500+0000 7f49308e7700 1 -- 172.21.15.42:0/3993479129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c0f7980 msgr2=0x7f492c0f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:35.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.500+0000 7f49308e7700 1 --2- 172.21.15.42:0/3993479129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c0f7980 0x7f492c0f7de0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f4920004660 tx=0x7f4920009b30).stop 2024-02-15T18:25:35.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.501+0000 7f49308e7700 1 -- 172.21.15.42:0/3993479129 shutdown_connections 2024-02-15T18:25:35.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.501+0000 7f49308e7700 1 --2- 172.21.15.42:0/3993479129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c0f7980 0x7f492c0f7de0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.501+0000 7f49308e7700 1 --2- 172.21.15.42:0/3993479129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f492c0f66e0 0x7f492c0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.501+0000 7f49308e7700 1 -- 172.21.15.42:0/3993479129 >> 172.21.15.42:0/3993479129 conn(0x7f492c0f1e40 msgr2=0x7f492c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:35.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.501+0000 7f49308e7700 1 -- 172.21.15.42:0/3993479129 shutdown_connections 2024-02-15T18:25:35.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.501+0000 7f49308e7700 1 -- 172.21.15.42:0/3993479129 wait complete. 2024-02-15T18:25:35.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.501+0000 7f49308e7700 1 Processor -- start 2024-02-15T18:25:35.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.501+0000 7f49308e7700 1 -- start start 2024-02-15T18:25:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.501+0000 7f49308e7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f492c0f66e0 0x7f492c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.501+0000 7f49308e7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c0f7980 0x7f492c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.501+0000 7f49308e7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c06a380 con 0x7f492c0f7980 2024-02-15T18:25:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.501+0000 7f49308e7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c06a4c0 con 0x7f492c0f66e0 2024-02-15T18:25:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.502+0000 7f492a59c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f492c0f66e0 0x7f492c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.502+0000 7f4929d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c0f7980 0x7f492c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.502+0000 7f492a59c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f492c0f66e0 0x7f492c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.42:54160/0 (socket says 172.21.15.42:54160) 2024-02-15T18:25:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.502+0000 7f492a59c700 1 -- 172.21.15.42:0/773193216 learned_addr learned my addr 172.21.15.42:0/773193216 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:25:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.502+0000 7f492a59c700 1 -- 172.21.15.42:0/773193216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c0f7980 msgr2=0x7f492c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.502+0000 7f492a59c700 1 --2- 172.21.15.42:0/773193216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c0f7980 0x7f492c069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.502+0000 7f492a59c700 1 -- 172.21.15.42:0/773193216 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4920005040 con 0x7f492c0f66e0 2024-02-15T18:25:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.502+0000 7f4929d9b700 1 --2- 172.21.15.42:0/773193216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c0f7980 0x7f492c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:25:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.502+0000 7f492a59c700 1 --2- 172.21.15.42:0/773193216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f492c0f66e0 0x7f492c06b9a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f491c00e7c0 tx=0x7f491c00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:35.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.502+0000 7f491affd700 1 -- 172.21.15.42:0/773193216 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f491c0092e0 con 0x7f492c0f66e0 2024-02-15T18:25:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.502+0000 7f491affd700 1 -- 172.21.15.42:0/773193216 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f491c00f040 con 0x7f492c0f66e0 2024-02-15T18:25:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.502+0000 7f491affd700 1 -- 172.21.15.42:0/773193216 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f491c014920 con 0x7f492c0f66e0 2024-02-15T18:25:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.503+0000 7f49308e7700 1 -- 172.21.15.42:0/773193216 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c06a7a0 con 0x7f492c0f66e0 2024-02-15T18:25:35.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.503+0000 7f49308e7700 1 -- 172.21.15.42:0/773193216 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c10f030 con 0x7f492c0f66e0 2024-02-15T18:25:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.504+0000 7f49308e7700 1 -- 172.21.15.42:0/773193216 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c045ad0 con 0x7f492c0f66e0 2024-02-15T18:25:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.504+0000 7f491affd700 1 -- 172.21.15.42:0/773193216 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 21) v1 ==== 74466+0+0 (secure 0 0 0) 0x7f491c01b030 con 0x7f492c0f66e0 2024-02-15T18:25:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.504+0000 7f491affd700 1 --2- 172.21.15.42:0/773193216 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f491405e6f0 0x7f4914060bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:25:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.504+0000 7f491affd700 1 -- 172.21.15.42:0/773193216 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f491c078a10 con 0x7f492c0f66e0 2024-02-15T18:25:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.505+0000 7f4929d9b700 1 --2- 172.21.15.42:0/773193216 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f491405e6f0 0x7f4914060bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:25:35.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.505+0000 7f4929d9b700 1 --2- 172.21.15.42:0/773193216 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f491405e6f0 0x7f4914060bb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f4920005010 tx=0x7f4920016040).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:25:35.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.509+0000 7f491affd700 1 -- 172.21.15.42:0/773193216 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7f491c0491b0 con 0x7f492c0f66e0 2024-02-15T18:25:35.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.601+0000 7f491affd700 1 -- 172.21.15.42:0/773193216 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mgrmap(e 22) v1 ==== 74714+0+0 (secure 0 0 0) 0x7f491c076b90 con 0x7f492c0f66e0 2024-02-15T18:25:35.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.730+0000 7f49308e7700 1 -- 172.21.15.42:0/773193216 --> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f492c0ff160 con 0x7f491405e6f0 2024-02-15T18:25:35.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.732+0000 7f491affd700 1 -- 172.21.15.42:0/773193216 <== mgr.24395 v2:172.21.15.191:6828/2058110468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f492000f070 con 0x7f491405e6f0 2024-02-15T18:25:35.730 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:25:35.730 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:25:35.730 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:25:35.730 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:25:35.730 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [], 2024-02-15T18:25:35.730 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "1/21 daemons upgraded", 2024-02-15T18:25:35.730 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "", 2024-02-15T18:25:35.730 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:25:35.730 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:25:35.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.734+0000 7f49308e7700 1 -- 172.21.15.42:0/773193216 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f491405e6f0 msgr2=0x7f4914060bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:35.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.734+0000 7f49308e7700 1 --2- 172.21.15.42:0/773193216 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f491405e6f0 0x7f4914060bb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f4920005010 tx=0x7f4920016040).stop 2024-02-15T18:25:35.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.734+0000 7f49308e7700 1 -- 172.21.15.42:0/773193216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f492c0f66e0 msgr2=0x7f492c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:25:35.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.734+0000 7f49308e7700 1 --2- 172.21.15.42:0/773193216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f492c0f66e0 0x7f492c06b9a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f491c00e7c0 tx=0x7f491c00b040).stop 2024-02-15T18:25:35.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.734+0000 7f49308e7700 1 -- 172.21.15.42:0/773193216 shutdown_connections 2024-02-15T18:25:35.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.734+0000 7f49308e7700 1 --2- 172.21.15.42:0/773193216 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f491405e6f0 0x7f4914060bb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.734+0000 7f49308e7700 1 --2- 172.21.15.42:0/773193216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f492c0f66e0 0x7f492c06b9a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.734+0000 7f49308e7700 1 --2- 172.21.15.42:0/773193216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c0f7980 0x7f492c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:25:35.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.734+0000 7f49308e7700 1 -- 172.21.15.42:0/773193216 >> 172.21.15.42:0/773193216 conn(0x7f492c0f1e40 msgr2=0x7f492c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:25:35.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.734+0000 7f49308e7700 1 -- 172.21.15.42:0/773193216 shutdown_connections 2024-02-15T18:25:35.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:25:35.734+0000 7f49308e7700 1 -- 172.21.15.42:0/773193216 wait complete. 2024-02-15T18:25:35.903 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:34.590154+0000 mon.smithi042 (mon.0) 578 : cluster [DBG] mgrmap e21: smithi191.gftdzo(active, since 1.79349s), standbys: smithi042.kfoihb 2024-02-15T18:25:35.903 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:34.610946+0000 mgr.smithi191.gftdzo (mgr.24395) 1 : audit [DBG] from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:25:35.903 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:34.611338+0000 mgr.smithi191.gftdzo (mgr.24395) 2 : cluster [DBG] pgmap v3: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:25:35.903 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:35.018035+0000 mon.smithi042 (mon.0) 579 : audit [DBG] from='client.? 172.21.15.42:0/1795558424' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:25:35.904 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:35.376402+0000 mon.smithi042 (mon.0) 580 : audit [DBG] from='client.? 172.21.15.42:0/3455558641' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:25:35.996 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:34.590154+0000 mon.smithi042 (mon.0) 578 : cluster [DBG] mgrmap e21: smithi191.gftdzo(active, since 1.79349s), standbys: smithi042.kfoihb 2024-02-15T18:25:35.996 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:34.610946+0000 mgr.smithi191.gftdzo (mgr.24395) 1 : audit [DBG] from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:25:35.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:34.611338+0000 mgr.smithi191.gftdzo (mgr.24395) 2 : cluster [DBG] pgmap v3: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:25:35.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:35.018035+0000 mon.smithi042 (mon.0) 579 : audit [DBG] from='client.? 172.21.15.42:0/1795558424' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:25:35.997 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:35.376402+0000 mon.smithi042 (mon.0) 580 : audit [DBG] from='client.? 172.21.15.42:0/3455558641' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:25:36.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:35.590343+0000 mgr.smithi191.gftdzo (mgr.24395) 3 : cluster [DBG] pgmap v4: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:25:36.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:35.603314+0000 mon.smithi042 (mon.0) 581 : cluster [DBG] mgrmap e22: smithi191.gftdzo(active, since 2s), standbys: smithi042.kfoihb 2024-02-15T18:25:36.948 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:35.732712+0000 mgr.smithi191.gftdzo (mgr.24395) 4 : audit [DBG] from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:25:37.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:35.590343+0000 mgr.smithi191.gftdzo (mgr.24395) 3 : cluster [DBG] pgmap v4: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:25:37.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:35.603314+0000 mon.smithi042 (mon.0) 581 : cluster [DBG] mgrmap e22: smithi191.gftdzo(active, since 2s), standbys: smithi042.kfoihb 2024-02-15T18:25:37.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:35.732712+0000 mgr.smithi191.gftdzo (mgr.24395) 4 : audit [DBG] from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:25:37.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:37.566914+0000 mon.smithi042 (mon.0) 582 : cluster [DBG] Standby manager daemon smithi042.kfoihb restarted 2024-02-15T18:25:37.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:37.567075+0000 mon.smithi042 (mon.0) 583 : cluster [DBG] Standby manager daemon smithi042.kfoihb started 2024-02-15T18:25:37.948 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:37.568866+0000 mon.smithi191 (mon.1) 40 : audit [DBG] from='mgr.? 172.21.15.42:0/3409235437' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.kfoihb/crt"}]: dispatch 2024-02-15T18:25:37.948 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:37.570462+0000 mon.smithi191 (mon.1) 41 : audit [DBG] from='mgr.? 172.21.15.42:0/3409235437' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:25:37.948 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:37.573064+0000 mon.smithi191 (mon.1) 42 : audit [DBG] from='mgr.? 172.21.15.42:0/3409235437' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.kfoihb/key"}]: dispatch 2024-02-15T18:25:37.948 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:37.573445+0000 mon.smithi191 (mon.1) 43 : audit [DBG] from='mgr.? 172.21.15.42:0/3409235437' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:25:38.018 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:37.566914+0000 mon.smithi042 (mon.0) 582 : cluster [DBG] Standby manager daemon smithi042.kfoihb restarted 2024-02-15T18:25:38.018 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:37.567075+0000 mon.smithi042 (mon.0) 583 : cluster [DBG] Standby manager daemon smithi042.kfoihb started 2024-02-15T18:25:38.018 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:37.568866+0000 mon.smithi191 (mon.1) 40 : audit [DBG] from='mgr.? 172.21.15.42:0/3409235437' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.kfoihb/crt"}]: dispatch 2024-02-15T18:25:38.018 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:37.570462+0000 mon.smithi191 (mon.1) 41 : audit [DBG] from='mgr.? 172.21.15.42:0/3409235437' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:25:38.018 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:37.573064+0000 mon.smithi191 (mon.1) 42 : audit [DBG] from='mgr.? 172.21.15.42:0/3409235437' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.kfoihb/key"}]: dispatch 2024-02-15T18:25:38.018 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:37.573445+0000 mon.smithi191 (mon.1) 43 : audit [DBG] from='mgr.? 172.21.15.42:0/3409235437' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:25:38.873 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:37.590648+0000 mgr.smithi191.gftdzo (mgr.24395) 5 : cluster [DBG] pgmap v5: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:25:38.873 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:37.610206+0000 mon.smithi042 (mon.0) 584 : cluster [DBG] mgrmap e23: smithi191.gftdzo(active, since 4s), standbys: smithi042.kfoihb 2024-02-15T18:25:38.873 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:38.074378+0000 mon.smithi042 (mon.0) 585 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:38.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:37.590648+0000 mgr.smithi191.gftdzo (mgr.24395) 5 : cluster [DBG] pgmap v5: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:25:38.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:37.610206+0000 mon.smithi042 (mon.0) 584 : cluster [DBG] mgrmap e23: smithi191.gftdzo(active, since 4s), standbys: smithi042.kfoihb 2024-02-15T18:25:38.948 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:38.074378+0000 mon.smithi042 (mon.0) 585 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:41.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:39.591027+0000 mgr.smithi191.gftdzo (mgr.24395) 6 : cluster [DBG] pgmap v6: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:25:41.448 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:40.108697+0000 mon.smithi042 (mon.0) 586 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:41.448 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:40.449957+0000 mon.smithi042 (mon.0) 587 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:41.448 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:40.456699+0000 mon.smithi042 (mon.0) 588 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:41.448 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:40.463321+0000 mon.smithi042 (mon.0) 589 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:41.448 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:40.469997+0000 mon.smithi042 (mon.0) 590 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:41.448 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:40.478643+0000 mon.smithi042 (mon.0) 591 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:41.560 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:39.591027+0000 mgr.smithi191.gftdzo (mgr.24395) 6 : cluster [DBG] pgmap v6: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:25:41.560 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:40.108697+0000 mon.smithi042 (mon.0) 586 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:41.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:40.449957+0000 mon.smithi042 (mon.0) 587 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:41.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:40.456699+0000 mon.smithi042 (mon.0) 588 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:41.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:40.463321+0000 mon.smithi042 (mon.0) 589 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:41.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:40.469997+0000 mon.smithi042 (mon.0) 590 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:41.561 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:40.478643+0000 mon.smithi042 (mon.0) 591 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:42.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:25:40.490744+0000 mgr.smithi191.gftdzo (mgr.24395) 7 : cephadm [INF] Reconfiguring prometheus.smithi042 (dependencies changed)... 2024-02-15T18:25:42.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:25:40.503289+0000 mgr.smithi191.gftdzo (mgr.24395) 8 : cephadm [INF] Reconfiguring daemon prometheus.smithi042 on smithi042 2024-02-15T18:25:42.560 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:25:40.490744+0000 mgr.smithi191.gftdzo (mgr.24395) 7 : cephadm [INF] Reconfiguring prometheus.smithi042 (dependencies changed)... 2024-02-15T18:25:42.560 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:25:40.503289+0000 mgr.smithi191.gftdzo (mgr.24395) 8 : cephadm [INF] Reconfiguring daemon prometheus.smithi042 on smithi042 2024-02-15T18:25:43.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:41.591510+0000 mgr.smithi191.gftdzo (mgr.24395) 9 : cluster [DBG] pgmap v7: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 880 B/s rd, 1 op/s 2024-02-15T18:25:43.560 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:41.591510+0000 mgr.smithi191.gftdzo (mgr.24395) 9 : cluster [DBG] pgmap v7: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 880 B/s rd, 1 op/s 2024-02-15T18:25:45.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:43.591891+0000 mgr.smithi191.gftdzo (mgr.24395) 10 : cluster [DBG] pgmap v8: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 684 B/s rd, 1 op/s 2024-02-15T18:25:45.560 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:43.591891+0000 mgr.smithi191.gftdzo (mgr.24395) 10 : cluster [DBG] pgmap v8: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 684 B/s rd, 1 op/s 2024-02-15T18:25:47.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:45.592437+0000 mgr.smithi191.gftdzo (mgr.24395) 11 : cluster [DBG] pgmap v9: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-15T18:25:47.560 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:45.592437+0000 mgr.smithi191.gftdzo (mgr.24395) 11 : cluster [DBG] pgmap v9: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-15T18:25:49.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:47.592790+0000 mgr.smithi191.gftdzo (mgr.24395) 12 : cluster [DBG] pgmap v10: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-02-15T18:25:49.560 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:47.592790+0000 mgr.smithi191.gftdzo (mgr.24395) 12 : cluster [DBG] pgmap v10: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-02-15T18:25:50.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:49.593126+0000 mgr.smithi191.gftdzo (mgr.24395) 13 : cluster [DBG] pgmap v11: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-02-15T18:25:50.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:49.593126+0000 mgr.smithi191.gftdzo (mgr.24395) 13 : cluster [DBG] pgmap v11: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-02-15T18:25:52.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:51.593596+0000 mgr.smithi191.gftdzo (mgr.24395) 14 : cluster [DBG] pgmap v12: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-02-15T18:25:52.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:51.593596+0000 mgr.smithi191.gftdzo (mgr.24395) 14 : cluster [DBG] pgmap v12: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-02-15T18:25:54.939 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:53.593980+0000 mgr.smithi191.gftdzo (mgr.24395) 15 : cluster [DBG] pgmap v13: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:54.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:53.593980+0000 mgr.smithi191.gftdzo (mgr.24395) 15 : cluster [DBG] pgmap v13: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:56.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:55.223716+0000 mon.smithi042 (mon.0) 592 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:56.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:55.229788+0000 mon.smithi191 (mon.1) 44 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:25:56.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:55.230412+0000 mgr.smithi191.gftdzo (mgr.24395) 16 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:25:56.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:55.231289+0000 mon.smithi191 (mon.1) 45 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi042.front.sepia.ceph.com:9095"}]: dispatch 2024-02-15T18:25:56.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:55.231726+0000 mgr.smithi191.gftdzo (mgr.24395) 17 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi042.front.sepia.ceph.com:9095"}]: dispatch 2024-02-15T18:25:56.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:55.236794+0000 mon.smithi042 (mon.0) 593 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:56.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:55.242530+0000 mon.smithi191 (mon.1) 46 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:25:56.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:55.248529+0000 mon.smithi042 (mon.0) 594 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:56.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:25:55.252429+0000 mgr.smithi191.gftdzo (mgr.24395) 18 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi191.gftdzo) 2024-02-15T18:25:56.554 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:25:55.253123+0000 mgr.smithi191.gftdzo (mgr.24395) 19 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi191.gftdzo) 2024-02-15T18:25:56.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:55.594536+0000 mgr.smithi191.gftdzo (mgr.24395) 20 : cluster [DBG] pgmap v14: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:25:56.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:55.223716+0000 mon.smithi042 (mon.0) 592 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:56.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:55.229788+0000 mon.smithi191 (mon.1) 44 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:25:56.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:55.230412+0000 mgr.smithi191.gftdzo (mgr.24395) 16 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:25:56.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:55.231289+0000 mon.smithi191 (mon.1) 45 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi042.front.sepia.ceph.com:9095"}]: dispatch 2024-02-15T18:25:56.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:55.231726+0000 mgr.smithi191.gftdzo (mgr.24395) 17 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi042.front.sepia.ceph.com:9095"}]: dispatch 2024-02-15T18:25:56.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:55.236794+0000 mon.smithi042 (mon.0) 593 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:56.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:55.242530+0000 mon.smithi191 (mon.1) 46 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:25:56.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:55.248529+0000 mon.smithi042 (mon.0) 594 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:56.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:25:55.252429+0000 mgr.smithi191.gftdzo (mgr.24395) 18 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi191.gftdzo) 2024-02-15T18:25:56.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:25:55.253123+0000 mgr.smithi191.gftdzo (mgr.24395) 19 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi191.gftdzo) 2024-02-15T18:25:56.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:55.594536+0000 mgr.smithi191.gftdzo (mgr.24395) 20 : cluster [DBG] pgmap v14: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:25:58.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:25:56.889228+0000 mgr.smithi191.gftdzo (mgr.24395) 21 : cephadm [INF] Upgrade: Updating mgr.smithi042.kfoihb 2024-02-15T18:25:58.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:56.894681+0000 mon.smithi042 (mon.0) 595 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:58.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:56.895872+0000 mon.smithi191 (mon.1) 47 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi042.kfoihb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:25:58.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:56.896280+0000 mon.smithi042 (mon.0) 596 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi042.kfoihb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:25:58.061 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:56.897225+0000 mon.smithi191 (mon.1) 48 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:25:58.061 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:25:56.898071+0000 mon.smithi191 (mon.1) 49 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:25:58.061 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:25:56.898751+0000 mgr.smithi191.gftdzo (mgr.24395) 22 : cephadm [INF] Deploying daemon mgr.smithi042.kfoihb on smithi042 2024-02-15T18:25:58.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:25:56.889228+0000 mgr.smithi191.gftdzo (mgr.24395) 21 : cephadm [INF] Upgrade: Updating mgr.smithi042.kfoihb 2024-02-15T18:25:58.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:56.894681+0000 mon.smithi042 (mon.0) 595 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:25:58.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:56.895872+0000 mon.smithi191 (mon.1) 47 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi042.kfoihb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:25:58.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:56.896280+0000 mon.smithi042 (mon.0) 596 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi042.kfoihb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:25:58.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:56.897225+0000 mon.smithi191 (mon.1) 48 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:25:58.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:25:56.898071+0000 mon.smithi191 (mon.1) 49 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:25:58.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:25:56.898751+0000 mgr.smithi191.gftdzo (mgr.24395) 22 : cephadm [INF] Deploying daemon mgr.smithi042.kfoihb on smithi042 2024-02-15T18:25:59.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:25:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:57.594915+0000 mgr.smithi191.gftdzo (mgr.24395) 23 : cluster [DBG] pgmap v15: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:25:59.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:25:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:57.594915+0000 mgr.smithi191.gftdzo (mgr.24395) 23 : cluster [DBG] pgmap v15: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:26:00.528 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:25:59.595351+0000 mgr.smithi191.gftdzo (mgr.24395) 24 : cluster [DBG] pgmap v16: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:26:00.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:25:59.595351+0000 mgr.smithi191.gftdzo (mgr.24395) 24 : cluster [DBG] pgmap v16: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:26:03.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:01.595751+0000 mgr.smithi191.gftdzo (mgr.24395) 25 : cluster [DBG] pgmap v17: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:26:03.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:02.025558+0000 mon.smithi042 (mon.0) 597 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:26:03.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:02.027168+0000 mon.smithi191 (mon.1) 50 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:03.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:01.595751+0000 mgr.smithi191.gftdzo (mgr.24395) 25 : cluster [DBG] pgmap v17: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:26:03.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:02.025558+0000 mon.smithi042 (mon.0) 597 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:26:03.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:02.027168+0000 mon.smithi191 (mon.1) 50 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:05.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:03.596112+0000 mgr.smithi191.gftdzo (mgr.24395) 26 : cluster [DBG] pgmap v18: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:26:05.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:03.596112+0000 mgr.smithi191.gftdzo (mgr.24395) 26 : cluster [DBG] pgmap v18: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:26:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.852+0000 7f6552897700 1 -- 172.21.15.42:0/927303254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f654c0f6860 msgr2=0x7f654c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.852+0000 7f6552897700 1 --2- 172.21.15.42:0/927303254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f654c0f6860 0x7f654c0f6cc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f6540004660 tx=0x7f6540009b30).stop 2024-02-15T18:26:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.852+0000 7f6552897700 1 -- 172.21.15.42:0/927303254 shutdown_connections 2024-02-15T18:26:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.852+0000 7f6552897700 1 --2- 172.21.15.42:0/927303254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f654c0f6860 0x7f654c0f6cc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.852+0000 7f6552897700 1 --2- 172.21.15.42:0/927303254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f654c0f55c0 0x7f654c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.852+0000 7f6552897700 1 -- 172.21.15.42:0/927303254 >> 172.21.15.42:0/927303254 conn(0x7f654c0f0d40 msgr2=0x7f654c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.852+0000 7f6552897700 1 -- 172.21.15.42:0/927303254 shutdown_connections 2024-02-15T18:26:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.852+0000 7f6552897700 1 -- 172.21.15.42:0/927303254 wait complete. 2024-02-15T18:26:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.852+0000 7f6552897700 1 Processor -- start 2024-02-15T18:26:05.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.853+0000 7f6552897700 1 -- start start 2024-02-15T18:26:05.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.853+0000 7f6552897700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f654c0f55c0 0x7f654c106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:05.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.853+0000 7f6552897700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f654c0f6860 0x7f654c1074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.853+0000 7f6552897700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c107a60 con 0x7f654c0f6860 2024-02-15T18:26:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.853+0000 7f6552897700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c107ba0 con 0x7f654c0f55c0 2024-02-15T18:26:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.853+0000 7f654b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f654c0f6860 0x7f654c1074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.853+0000 7f654bfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f654c0f55c0 0x7f654c106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.853+0000 7f654b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f654c0f6860 0x7f654c1074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42444/0 (socket says 172.21.15.42:42444) 2024-02-15T18:26:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.853+0000 7f654b7fe700 1 -- 172.21.15.42:0/3221144073 learned_addr learned my addr 172.21.15.42:0/3221144073 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:26:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.854+0000 7f654b7fe700 1 -- 172.21.15.42:0/3221144073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f654c0f55c0 msgr2=0x7f654c106fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.854+0000 7f654b7fe700 1 --2- 172.21.15.42:0/3221144073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f654c0f55c0 0x7f654c106fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.854+0000 7f654b7fe700 1 -- 172.21.15.42:0/3221144073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6540005040 con 0x7f654c0f6860 2024-02-15T18:26:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.854+0000 7f654b7fe700 1 --2- 172.21.15.42:0/3221144073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f654c0f6860 0x7f654c1074b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f6540009f50 tx=0x7f6540009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.854+0000 7f6548ff9700 1 -- 172.21.15.42:0/3221144073 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6540018070 con 0x7f654c0f6860 2024-02-15T18:26:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.854+0000 7f6548ff9700 1 -- 172.21.15.42:0/3221144073 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f6540007ba0 con 0x7f654c0f6860 2024-02-15T18:26:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.854+0000 7f6552897700 1 -- 172.21.15.42:0/3221144073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654c1084e0 con 0x7f654c0f6860 2024-02-15T18:26:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.854+0000 7f6548ff9700 1 -- 172.21.15.42:0/3221144073 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f654001c620 con 0x7f654c0f6860 2024-02-15T18:26:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.854+0000 7f6552897700 1 -- 172.21.15.42:0/3221144073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654c1089c0 con 0x7f654c0f6860 2024-02-15T18:26:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.856+0000 7f6552897700 1 -- 172.21.15.42:0/3221144073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654c045ad0 con 0x7f654c0f6860 2024-02-15T18:26:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.856+0000 7f6548ff9700 1 -- 172.21.15.42:0/3221144073 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) v1 ==== 74714+0+0 (secure 0 0 0) 0x7f654001d420 con 0x7f654c0f6860 2024-02-15T18:26:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.856+0000 7f6548ff9700 1 --2- 172.21.15.42:0/3221144073 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f653405e960 0x7f6534060e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.857+0000 7f6548ff9700 1 -- 172.21.15.42:0/3221144073 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f654007a340 con 0x7f654c0f6860 2024-02-15T18:26:05.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.857+0000 7f654bfff700 1 --2- 172.21.15.42:0/3221144073 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f653405e960 0x7f6534060e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:05.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.857+0000 7f654bfff700 1 --2- 172.21.15.42:0/3221144073 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f653405e960 0x7f6534060e20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f654c0f66c0 tx=0x7f653c00b040).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:05.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:05.862+0000 7f6548ff9700 1 -- 172.21.15.42:0/3221144073 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7f654004e250 con 0x7f654c0f6860 2024-02-15T18:26:06.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.077+0000 7f6552897700 1 -- 172.21.15.42:0/3221144073 --> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f654c0faca0 con 0x7f653405e960 2024-02-15T18:26:06.117 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:26:06.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.079+0000 7f6548ff9700 1 -- 172.21.15.42:0/3221144073 <== mgr.24395 v2:172.21.15.191:6828/2058110468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f654c0faca0 con 0x7f653405e960 2024-02-15T18:26:06.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.080+0000 7f6552897700 1 -- 172.21.15.42:0/3221144073 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f653405e960 msgr2=0x7f6534060e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.080+0000 7f6552897700 1 --2- 172.21.15.42:0/3221144073 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f653405e960 0x7f6534060e20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f654c0f66c0 tx=0x7f653c00b040).stop 2024-02-15T18:26:06.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.080+0000 7f6552897700 1 -- 172.21.15.42:0/3221144073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f654c0f6860 msgr2=0x7f654c1074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.080+0000 7f6552897700 1 --2- 172.21.15.42:0/3221144073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f654c0f6860 0x7f654c1074b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f6540009f50 tx=0x7f6540009b30).stop 2024-02-15T18:26:06.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.080+0000 7f6552897700 1 -- 172.21.15.42:0/3221144073 shutdown_connections 2024-02-15T18:26:06.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.080+0000 7f6552897700 1 --2- 172.21.15.42:0/3221144073 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f653405e960 0x7f6534060e20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.080+0000 7f6552897700 1 --2- 172.21.15.42:0/3221144073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f654c0f55c0 0x7f654c106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.080+0000 7f6552897700 1 --2- 172.21.15.42:0/3221144073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f654c0f6860 0x7f654c1074b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.080+0000 7f6552897700 1 -- 172.21.15.42:0/3221144073 >> 172.21.15.42:0/3221144073 conn(0x7f654c0f0d40 msgr2=0x7f654c05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:06.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.080+0000 7f6552897700 1 -- 172.21.15.42:0/3221144073 shutdown_connections 2024-02-15T18:26:06.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.081+0000 7f6552897700 1 -- 172.21.15.42:0/3221144073 wait complete. 2024-02-15T18:26:06.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.184+0000 7f715aba3700 1 -- 172.21.15.42:0/3706440081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71540f6860 msgr2=0x7f71540f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.184+0000 7f715aba3700 1 --2- 172.21.15.42:0/3706440081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71540f6860 0x7f71540f6cc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f7140004660 tx=0x7f7140009b30).stop 2024-02-15T18:26:06.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.184+0000 7f715aba3700 1 -- 172.21.15.42:0/3706440081 shutdown_connections 2024-02-15T18:26:06.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.184+0000 7f715aba3700 1 --2- 172.21.15.42:0/3706440081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71540f6860 0x7f71540f6cc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.184+0000 7f715aba3700 1 --2- 172.21.15.42:0/3706440081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71540f55c0 0x7f71540f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.184+0000 7f715aba3700 1 -- 172.21.15.42:0/3706440081 >> 172.21.15.42:0/3706440081 conn(0x7f71540f0d40 msgr2=0x7f71540f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:06.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.184+0000 7f715aba3700 1 -- 172.21.15.42:0/3706440081 shutdown_connections 2024-02-15T18:26:06.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.184+0000 7f715aba3700 1 -- 172.21.15.42:0/3706440081 wait complete. 2024-02-15T18:26:06.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.184+0000 7f715aba3700 1 Processor -- start 2024-02-15T18:26:06.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.184+0000 7f715aba3700 1 -- start start 2024-02-15T18:26:06.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.185+0000 7f715aba3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71540f55c0 0x7f7154109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:06.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.185+0000 7f715aba3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71540f6860 0x7f715410c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:06.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.185+0000 7f715aba3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71540603c0 con 0x7f71540f55c0 2024-02-15T18:26:06.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.185+0000 7f715aba3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7154060500 con 0x7f71540f6860 2024-02-15T18:26:06.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.185+0000 7f715893f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71540f55c0 0x7f7154109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:06.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.185+0000 7f715893f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71540f55c0 0x7f7154109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42474/0 (socket says 172.21.15.42:42474) 2024-02-15T18:26:06.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.185+0000 7f715893f700 1 -- 172.21.15.42:0/2339424478 learned_addr learned my addr 172.21.15.42:0/2339424478 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:26:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.185+0000 7f7153fff700 1 --2- 172.21.15.42:0/2339424478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71540f6860 0x7f715410c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.185+0000 7f715893f700 1 -- 172.21.15.42:0/2339424478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71540f6860 msgr2=0x7f715410c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.185+0000 7f715893f700 1 --2- 172.21.15.42:0/2339424478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71540f6860 0x7f715410c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.185+0000 7f715893f700 1 -- 172.21.15.42:0/2339424478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7140005040 con 0x7f71540f55c0 2024-02-15T18:26:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.185+0000 7f715893f700 1 --2- 172.21.15.42:0/2339424478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71540f55c0 0x7f7154109cd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f7148002700 tx=0x7f7148009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:06.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.185+0000 7f71517fa700 1 -- 172.21.15.42:0/2339424478 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71480079c0 con 0x7f71540f55c0 2024-02-15T18:26:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.186+0000 7f715aba3700 1 -- 172.21.15.42:0/2339424478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715410c780 con 0x7f71540f55c0 2024-02-15T18:26:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.186+0000 7f71517fa700 1 -- 172.21.15.42:0/2339424478 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f714800f040 con 0x7f71540f55c0 2024-02-15T18:26:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.186+0000 7f71517fa700 1 -- 172.21.15.42:0/2339424478 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7148014940 con 0x7f71540f55c0 2024-02-15T18:26:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.186+0000 7f715aba3700 1 -- 172.21.15.42:0/2339424478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715410cd10 con 0x7f71540f55c0 2024-02-15T18:26:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.187+0000 7f715aba3700 1 -- 172.21.15.42:0/2339424478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71540f49b0 con 0x7f71540f55c0 2024-02-15T18:26:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.187+0000 7f71517fa700 1 -- 172.21.15.42:0/2339424478 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) v1 ==== 74714+0+0 (secure 0 0 0) 0x7f714801b060 con 0x7f71540f55c0 2024-02-15T18:26:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.187+0000 7f71517fa700 1 --2- 172.21.15.42:0/2339424478 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f714405e960 0x7f7144060e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.187+0000 7f71517fa700 1 -- 172.21.15.42:0/2339424478 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7148078f30 con 0x7f71540f55c0 2024-02-15T18:26:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.187+0000 7f7153fff700 1 --2- 172.21.15.42:0/2339424478 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f714405e960 0x7f7144060e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:06.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.188+0000 7f7153fff700 1 --2- 172.21.15.42:0/2339424478 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f714405e960 0x7f7144060e20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f7140004630 tx=0x7f7140009b30).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:06.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.192+0000 7f71517fa700 1 -- 172.21.15.42:0/2339424478 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7f7148049400 con 0x7f71540f55c0 2024-02-15T18:26:06.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.379+0000 7f715aba3700 1 -- 172.21.15.42:0/2339424478 --> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7154057f10 con 0x7f714405e960 2024-02-15T18:26:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.380+0000 7f71517fa700 1 -- 172.21.15.42:0/2339424478 <== mgr.24395 v2:172.21.15.191:6828/2058110468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f7154057f10 con 0x7f714405e960 2024-02-15T18:26:06.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.383+0000 7f715aba3700 1 -- 172.21.15.42:0/2339424478 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f714405e960 msgr2=0x7f7144060e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.383+0000 7f715aba3700 1 --2- 172.21.15.42:0/2339424478 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f714405e960 0x7f7144060e20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f7140004630 tx=0x7f7140009b30).stop 2024-02-15T18:26:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.383+0000 7f715aba3700 1 -- 172.21.15.42:0/2339424478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71540f55c0 msgr2=0x7f7154109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.383+0000 7f715aba3700 1 --2- 172.21.15.42:0/2339424478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71540f55c0 0x7f7154109cd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f7148002700 tx=0x7f7148009d80).stop 2024-02-15T18:26:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.383+0000 7f715aba3700 1 -- 172.21.15.42:0/2339424478 shutdown_connections 2024-02-15T18:26:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.383+0000 7f715aba3700 1 --2- 172.21.15.42:0/2339424478 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f714405e960 0x7f7144060e20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.383+0000 7f715aba3700 1 --2- 172.21.15.42:0/2339424478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71540f55c0 0x7f7154109cd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.383+0000 7f715aba3700 1 --2- 172.21.15.42:0/2339424478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71540f6860 0x7f715410c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.383+0000 7f715aba3700 1 -- 172.21.15.42:0/2339424478 >> 172.21.15.42:0/2339424478 conn(0x7f71540f0d40 msgr2=0x7f71540fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.383+0000 7f715aba3700 1 -- 172.21.15.42:0/2339424478 shutdown_connections 2024-02-15T18:26:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.383+0000 7f715aba3700 1 -- 172.21.15.42:0/2339424478 wait complete. 2024-02-15T18:26:06.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.497+0000 7f9c5344f700 1 -- 172.21.15.42:0/3149665642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0f6860 msgr2=0x7f9c4c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.497+0000 7f9c5344f700 1 --2- 172.21.15.42:0/3149665642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0f6860 0x7f9c4c0f6cc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f9c3c004660 tx=0x7f9c3c009b30).stop 2024-02-15T18:26:06.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.497+0000 7f9c5344f700 1 -- 172.21.15.42:0/3149665642 shutdown_connections 2024-02-15T18:26:06.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.497+0000 7f9c5344f700 1 --2- 172.21.15.42:0/3149665642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0f6860 0x7f9c4c0f6cc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.497+0000 7f9c5344f700 1 --2- 172.21.15.42:0/3149665642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c4c0f55c0 0x7f9c4c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.497+0000 7f9c5344f700 1 -- 172.21.15.42:0/3149665642 >> 172.21.15.42:0/3149665642 conn(0x7f9c4c0f0d40 msgr2=0x7f9c4c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:06.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.498+0000 7f9c5344f700 1 -- 172.21.15.42:0/3149665642 shutdown_connections 2024-02-15T18:26:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.498+0000 7f9c5344f700 1 -- 172.21.15.42:0/3149665642 wait complete. 2024-02-15T18:26:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.498+0000 7f9c5344f700 1 Processor -- start 2024-02-15T18:26:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.498+0000 7f9c5344f700 1 -- start start 2024-02-15T18:26:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.498+0000 7f9c5344f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c4c0f55c0 0x7f9c4c109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.498+0000 7f9c5344f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0f6860 0x7f9c4c10c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.498+0000 7f9c5344f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c0603c0 con 0x7f9c4c0f6860 2024-02-15T18:26:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.498+0000 7f9c5344f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c060500 con 0x7f9c4c0f55c0 2024-02-15T18:26:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.499+0000 7f9c509ea700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0f6860 0x7f9c4c10c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.499+0000 7f9c511eb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c4c0f55c0 0x7f9c4c109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.499+0000 7f9c509ea700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0f6860 0x7f9c4c10c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42490/0 (socket says 172.21.15.42:42490) 2024-02-15T18:26:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.499+0000 7f9c509ea700 1 -- 172.21.15.42:0/397700235 learned_addr learned my addr 172.21.15.42:0/397700235 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:26:06.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.499+0000 7f9c509ea700 1 -- 172.21.15.42:0/397700235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c4c0f55c0 msgr2=0x7f9c4c109cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.499+0000 7f9c509ea700 1 --2- 172.21.15.42:0/397700235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c4c0f55c0 0x7f9c4c109cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.499+0000 7f9c509ea700 1 -- 172.21.15.42:0/397700235 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c005040 con 0x7f9c4c0f6860 2024-02-15T18:26:06.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.499+0000 7f9c511eb700 1 --2- 172.21.15.42:0/397700235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c4c0f55c0 0x7f9c4c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:26:06.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.499+0000 7f9c509ea700 1 --2- 172.21.15.42:0/397700235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0f6860 0x7f9c4c10c1d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f9c3c009f50 tx=0x7f9c3c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.499+0000 7f9c41ffb700 1 -- 172.21.15.42:0/397700235 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c3c018070 con 0x7f9c4c0f6860 2024-02-15T18:26:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.499+0000 7f9c41ffb700 1 -- 172.21.15.42:0/397700235 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f9c3c007ba0 con 0x7f9c4c0f6860 2024-02-15T18:26:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.499+0000 7f9c5344f700 1 -- 172.21.15.42:0/397700235 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4c10c720 con 0x7f9c4c0f6860 2024-02-15T18:26:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.499+0000 7f9c41ffb700 1 -- 172.21.15.42:0/397700235 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c3c01c420 con 0x7f9c4c0f6860 2024-02-15T18:26:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.500+0000 7f9c5344f700 1 -- 172.21.15.42:0/397700235 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4c10cbd0 con 0x7f9c4c0f6860 2024-02-15T18:26:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.501+0000 7f9c5344f700 1 -- 172.21.15.42:0/397700235 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4c045ad0 con 0x7f9c4c0f6860 2024-02-15T18:26:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.501+0000 7f9c41ffb700 1 -- 172.21.15.42:0/397700235 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) v1 ==== 74714+0+0 (secure 0 0 0) 0x7f9c3c01d420 con 0x7f9c4c0f6860 2024-02-15T18:26:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.502+0000 7f9c41ffb700 1 --2- 172.21.15.42:0/397700235 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f9c3805e960 0x7f9c38060e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.502+0000 7f9c41ffb700 1 -- 172.21.15.42:0/397700235 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9c3c07ac90 con 0x7f9c4c0f6860 2024-02-15T18:26:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.502+0000 7f9c511eb700 1 --2- 172.21.15.42:0/397700235 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f9c3805e960 0x7f9c38060e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:06.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.502+0000 7f9c511eb700 1 --2- 172.21.15.42:0/397700235 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f9c3805e960 0x7f9c38060e20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f9c4c060040 tx=0x7f9c4800b040).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:06.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.506+0000 7f9c41ffb700 1 -- 172.21.15.42:0/397700235 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7f9c3c04b200 con 0x7f9c4c0f6860 2024-02-15T18:26:06.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.700+0000 7f9c5344f700 1 -- 172.21.15.42:0/397700235 --> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c4c057f10 con 0x7f9c3805e960 2024-02-15T18:26:06.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.717+0000 7f9c41ffb700 1 -- 172.21.15.42:0/397700235 <== mgr.24395 v2:172.21.15.191:6828/2058110468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f9c4c057f10 con 0x7f9c3805e960 2024-02-15T18:26:06.714 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:26:06.714 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (5m) 26s ago 7m 17.6M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:26:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (7m) 26s ago 7m 6400k - 16.2.4 8d91d370c2b8 94d74c33c758 2024-02-15T18:26:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (6m) 28s ago 6m 6405k - 16.2.4 8d91d370c2b8 ff0f56f9475a 2024-02-15T18:26:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (5m) 26s ago 7m 34.7M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:26:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (2m) 26s ago 2m 12.7M - 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:26:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (2m) 26s ago 2m 15.5M - 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:26:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (2m) 28s ago 2m 17.2M - 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:26:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (2m) 28s ago 2m 19.1M - 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:26:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 starting - - - - 2024-02-15T18:26:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (40s) 28s ago 6m 413M - 16.2.14-521-g0e714d9a 1fbc92c859dc b772ba3fed7c 2024-02-15T18:26:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (9m) 26s ago 9m 46.9M 2048M 16.2.4 8d91d370c2b8 9dcbc5c49e05 2024-02-15T18:26:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (6m) 28s ago 6m 41.3M 2048M 16.2.4 8d91d370c2b8 c3dd4902e2a2 2024-02-15T18:26:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (7m) 26s ago 7m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:26:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (6m) 28s ago 6m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:26:06.715 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (5m) 26s ago 5m 40.6M 4096M 16.2.4 8d91d370c2b8 973745b31d5a 2024-02-15T18:26:06.716 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (5m) 26s ago 5m 43.0M 4096M 16.2.4 8d91d370c2b8 2b3ae86111e8 2024-02-15T18:26:06.716 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (5m) 26s ago 4m 40.5M 4096M 16.2.4 8d91d370c2b8 3706071e4698 2024-02-15T18:26:06.716 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (4m) 28s ago 4m 40.4M 4096M 16.2.4 8d91d370c2b8 e6fe8e77c6fe 2024-02-15T18:26:06.716 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (4m) 28s ago 4m 42.8M 4096M 16.2.4 8d91d370c2b8 24582012e234 2024-02-15T18:26:06.716 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (3m) 28s ago 3m 39.7M 4096M 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:26:06.716 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 starting - - - - 2024-02-15T18:26:06.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.720+0000 7f9c5344f700 1 -- 172.21.15.42:0/397700235 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f9c3805e960 msgr2=0x7f9c38060e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.720+0000 7f9c5344f700 1 --2- 172.21.15.42:0/397700235 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f9c3805e960 0x7f9c38060e20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f9c4c060040 tx=0x7f9c4800b040).stop 2024-02-15T18:26:06.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.720+0000 7f9c5344f700 1 -- 172.21.15.42:0/397700235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0f6860 msgr2=0x7f9c4c10c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.720+0000 7f9c5344f700 1 --2- 172.21.15.42:0/397700235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0f6860 0x7f9c4c10c1d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f9c3c009f50 tx=0x7f9c3c009b30).stop 2024-02-15T18:26:06.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.720+0000 7f9c5344f700 1 -- 172.21.15.42:0/397700235 shutdown_connections 2024-02-15T18:26:06.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.720+0000 7f9c5344f700 1 --2- 172.21.15.42:0/397700235 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f9c3805e960 0x7f9c38060e20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.720+0000 7f9c5344f700 1 --2- 172.21.15.42:0/397700235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c4c0f55c0 0x7f9c4c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.720+0000 7f9c5344f700 1 --2- 172.21.15.42:0/397700235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c4c0f6860 0x7f9c4c10c1d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.720+0000 7f9c5344f700 1 -- 172.21.15.42:0/397700235 >> 172.21.15.42:0/397700235 conn(0x7f9c4c0f0d40 msgr2=0x7f9c4c0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:06.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.720+0000 7f9c5344f700 1 -- 172.21.15.42:0/397700235 shutdown_connections 2024-02-15T18:26:06.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.720+0000 7f9c5344f700 1 -- 172.21.15.42:0/397700235 wait complete. 2024-02-15T18:26:06.811 INFO:tasks.workunit.client.1.smithi191.stderr:Note: switching to 'bedfc49c6a823fb0ff75911e476e1f3cb464ad3e'. 2024-02-15T18:26:06.811 INFO:tasks.workunit.client.1.smithi191.stderr: 2024-02-15T18:26:06.812 INFO:tasks.workunit.client.1.smithi191.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-15T18:26:06.812 INFO:tasks.workunit.client.1.smithi191.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-15T18:26:06.812 INFO:tasks.workunit.client.1.smithi191.stderr:state without impacting any branches by switching back to a branch. 2024-02-15T18:26:06.812 INFO:tasks.workunit.client.1.smithi191.stderr: 2024-02-15T18:26:06.812 INFO:tasks.workunit.client.1.smithi191.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-15T18:26:06.812 INFO:tasks.workunit.client.1.smithi191.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-15T18:26:06.812 INFO:tasks.workunit.client.1.smithi191.stderr: 2024-02-15T18:26:06.812 INFO:tasks.workunit.client.1.smithi191.stderr: git switch -c 2024-02-15T18:26:06.812 INFO:tasks.workunit.client.1.smithi191.stderr: 2024-02-15T18:26:06.812 INFO:tasks.workunit.client.1.smithi191.stderr:Or undo this operation with: 2024-02-15T18:26:06.813 INFO:tasks.workunit.client.1.smithi191.stderr: 2024-02-15T18:26:06.813 INFO:tasks.workunit.client.1.smithi191.stderr: git switch - 2024-02-15T18:26:06.813 INFO:tasks.workunit.client.1.smithi191.stderr: 2024-02-15T18:26:06.813 INFO:tasks.workunit.client.1.smithi191.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-15T18:26:06.813 INFO:tasks.workunit.client.1.smithi191.stderr: 2024-02-15T18:26:06.813 INFO:tasks.workunit.client.1.smithi191.stderr:HEAD is now at bedfc49c6a8 qa/suites/orch: whitelist warnings that are expected in test environments 2024-02-15T18:26:06.817 DEBUG:teuthology.orchestra.run.smithi191:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-02-15T18:26:06.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.835+0000 7febbd4cd700 1 -- 172.21.15.42:0/1301613394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febb80602b0 msgr2=0x7febb80f9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.835+0000 7febbd4cd700 1 --2- 172.21.15.42:0/1301613394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febb80602b0 0x7febb80f9300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7feba8004660 tx=0x7feba8009b30).stop 2024-02-15T18:26:06.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.835+0000 7febbd4cd700 1 -- 172.21.15.42:0/1301613394 shutdown_connections 2024-02-15T18:26:06.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.835+0000 7febbd4cd700 1 --2- 172.21.15.42:0/1301613394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febb80f9880 0x7febb80fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.835+0000 7febbd4cd700 1 --2- 172.21.15.42:0/1301613394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febb80602b0 0x7febb80f9300 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.835+0000 7febbd4cd700 1 -- 172.21.15.42:0/1301613394 >> 172.21.15.42:0/1301613394 conn(0x7febb80f0d40 msgr2=0x7febb80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:06.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.835+0000 7febbd4cd700 1 -- 172.21.15.42:0/1301613394 shutdown_connections 2024-02-15T18:26:06.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.835+0000 7febbd4cd700 1 -- 172.21.15.42:0/1301613394 wait complete. 2024-02-15T18:26:06.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.836+0000 7febbd4cd700 1 Processor -- start 2024-02-15T18:26:06.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.836+0000 7febbd4cd700 1 -- start start 2024-02-15T18:26:06.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.836+0000 7febbd4cd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febb80602b0 0x7febb80f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:06.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.836+0000 7febbd4cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febb80f9880 0x7febb80f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:06.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.836+0000 7febbd4cd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb80f5760 con 0x7febb80f9880 2024-02-15T18:26:06.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.836+0000 7febbd4cd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb80f58a0 con 0x7febb80602b0 2024-02-15T18:26:06.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.836+0000 7febb67fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febb80f9880 0x7febb80f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:06.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.836+0000 7febb67fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febb80f9880 0x7febb80f5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42508/0 (socket says 172.21.15.42:42508) 2024-02-15T18:26:06.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.836+0000 7febb67fc700 1 -- 172.21.15.42:0/3463162904 learned_addr learned my addr 172.21.15.42:0/3463162904 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:26:06.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.836+0000 7febb6ffd700 1 --2- 172.21.15.42:0/3463162904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febb80602b0 0x7febb80f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:06.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.836+0000 7febb67fc700 1 -- 172.21.15.42:0/3463162904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febb80602b0 msgr2=0x7febb80f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:06.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.836+0000 7febb67fc700 1 --2- 172.21.15.42:0/3463162904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febb80602b0 0x7febb80f6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:06.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.836+0000 7febb67fc700 1 -- 172.21.15.42:0/3463162904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba8005040 con 0x7febb80f9880 2024-02-15T18:26:06.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.837+0000 7febb67fc700 1 --2- 172.21.15.42:0/3463162904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febb80f9880 0x7febb80f5200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7febac00e7c0 tx=0x7febac00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:06.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.837+0000 7feb9f7fe700 1 -- 172.21.15.42:0/3463162904 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febac0092e0 con 0x7febb80f9880 2024-02-15T18:26:06.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.837+0000 7febbd4cd700 1 -- 172.21.15.42:0/3463162904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb80f5b80 con 0x7febb80f9880 2024-02-15T18:26:06.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.837+0000 7feb9f7fe700 1 -- 172.21.15.42:0/3463162904 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7febac00f040 con 0x7febb80f9880 2024-02-15T18:26:06.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.837+0000 7feb9f7fe700 1 -- 172.21.15.42:0/3463162904 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febac014920 con 0x7febb80f9880 2024-02-15T18:26:06.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.837+0000 7febbd4cd700 1 -- 172.21.15.42:0/3463162904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb810efe0 con 0x7febb80f9880 2024-02-15T18:26:06.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.838+0000 7febbd4cd700 1 -- 172.21.15.42:0/3463162904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb80f49b0 con 0x7febb80f9880 2024-02-15T18:26:06.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.838+0000 7feb9f7fe700 1 -- 172.21.15.42:0/3463162904 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) v1 ==== 74714+0+0 (secure 0 0 0) 0x7febac014b90 con 0x7febb80f9880 2024-02-15T18:26:06.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.839+0000 7feb9f7fe700 1 --2- 172.21.15.42:0/3463162904 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7feba005e960 0x7feba0060e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:06.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.839+0000 7feb9f7fe700 1 -- 172.21.15.42:0/3463162904 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7febac079220 con 0x7febb80f9880 2024-02-15T18:26:06.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.839+0000 7febb6ffd700 1 --2- 172.21.15.42:0/3463162904 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7feba005e960 0x7feba0060e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:06.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.840+0000 7febb6ffd700 1 --2- 172.21.15.42:0/3463162904 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7feba005e960 0x7feba0060e20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7feba8009720 tx=0x7feba8016040).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:06.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:06.843+0000 7feb9f7fe700 1 -- 172.21.15.42:0/3463162904 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7febac049770 con 0x7febb80f9880 2024-02-15T18:26:06.841 INFO:tasks.workunit.client.1.smithi191.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-15T18:26:06.843 INFO:tasks.workunit.client.1.smithi191.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-15T18:26:06.843 INFO:tasks.workunit.client.1.smithi191.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-15T18:26:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.092+0000 7febbd4cd700 1 -- 172.21.15.42:0/3463162904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7febb8027a00 con 0x7febb80f9880 2024-02-15T18:26:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.093+0000 7feb9f7fe700 1 -- 172.21.15.42:0/3463162904 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7febac04cd90 con 0x7febb80f9880 2024-02-15T18:26:07.091 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:26:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:26:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:26:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:26:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:26:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 1, 2024-02-15T18:26:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 1 2024-02-15T18:26:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:26:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:26:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:26:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:26:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:26:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:26:07.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:26:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:26:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 1, 2024-02-15T18:26:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 13 2024-02-15T18:26:07.092 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:26:07.092 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:26:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.095+0000 7febbd4cd700 1 -- 172.21.15.42:0/3463162904 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7feba005e960 msgr2=0x7feba0060e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.095+0000 7febbd4cd700 1 --2- 172.21.15.42:0/3463162904 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7feba005e960 0x7feba0060e20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7feba8009720 tx=0x7feba8016040).stop 2024-02-15T18:26:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.095+0000 7febbd4cd700 1 -- 172.21.15.42:0/3463162904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febb80f9880 msgr2=0x7febb80f5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:07.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.095+0000 7febbd4cd700 1 --2- 172.21.15.42:0/3463162904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febb80f9880 0x7febb80f5200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7febac00e7c0 tx=0x7febac00b040).stop 2024-02-15T18:26:07.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.095+0000 7febbd4cd700 1 -- 172.21.15.42:0/3463162904 shutdown_connections 2024-02-15T18:26:07.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.095+0000 7febbd4cd700 1 --2- 172.21.15.42:0/3463162904 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7feba005e960 0x7feba0060e20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.095+0000 7febbd4cd700 1 --2- 172.21.15.42:0/3463162904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febb80602b0 0x7febb80f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.095+0000 7febbd4cd700 1 --2- 172.21.15.42:0/3463162904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febb80f9880 0x7febb80f5200 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.095+0000 7febbd4cd700 1 -- 172.21.15.42:0/3463162904 >> 172.21.15.42:0/3463162904 conn(0x7febb80f0d40 msgr2=0x7febb80f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:07.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.095+0000 7febbd4cd700 1 -- 172.21.15.42:0/3463162904 shutdown_connections 2024-02-15T18:26:07.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.095+0000 7febbd4cd700 1 -- 172.21.15.42:0/3463162904 wait complete. 2024-02-15T18:26:07.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.210+0000 7f52eb59e700 1 -- 172.21.15.42:0/400717537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52ec0fba80 msgr2=0x7f52ec0fde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:07.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.210+0000 7f52eb59e700 1 --2- 172.21.15.42:0/400717537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52ec0fba80 0x7f52ec0fde70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f52e0004690 tx=0x7f52e0009b30).stop 2024-02-15T18:26:07.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.210+0000 7f52eb59e700 1 -- 172.21.15.42:0/400717537 shutdown_connections 2024-02-15T18:26:07.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.210+0000 7f52eb59e700 1 --2- 172.21.15.42:0/400717537 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52ec0fba80 0x7f52ec0fde70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.210+0000 7f52eb59e700 1 --2- 172.21.15.42:0/400717537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ec060290 0x7f52ec0fb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.210+0000 7f52eb59e700 1 -- 172.21.15.42:0/400717537 >> 172.21.15.42:0/400717537 conn(0x7f52ec0f0b90 msgr2=0x7f52ec0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:07.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.210+0000 7f52eb59e700 1 -- 172.21.15.42:0/400717537 shutdown_connections 2024-02-15T18:26:07.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.210+0000 7f52eb59e700 1 -- 172.21.15.42:0/400717537 wait complete. 2024-02-15T18:26:07.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.210+0000 7f52eb59e700 1 Processor -- start 2024-02-15T18:26:07.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.210+0000 7f52eb59e700 1 -- start start 2024-02-15T18:26:07.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.211+0000 7f52eb59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52ec060290 0x7f52ec108fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:07.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.211+0000 7f52eb59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ec0fba80 0x7f52ec1094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:07.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.211+0000 7f52eb59e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec109a70 con 0x7f52ec060290 2024-02-15T18:26:07.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.211+0000 7f52eb59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec109bb0 con 0x7f52ec0fba80 2024-02-15T18:26:07.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.211+0000 7f52ea59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52ec060290 0x7f52ec108fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:07.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.211+0000 7f52ea59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52ec060290 0x7f52ec108fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42528/0 (socket says 172.21.15.42:42528) 2024-02-15T18:26:07.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.211+0000 7f52ea59c700 1 -- 172.21.15.42:0/4028805032 learned_addr learned my addr 172.21.15.42:0/4028805032 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:26:07.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.211+0000 7f52ea59c700 1 -- 172.21.15.42:0/4028805032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ec0fba80 msgr2=0x7f52ec1094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:07.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.211+0000 7f52e9d9b700 1 --2- 172.21.15.42:0/4028805032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ec0fba80 0x7f52ec1094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:07.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.211+0000 7f52ea59c700 1 --2- 172.21.15.42:0/4028805032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ec0fba80 0x7f52ec1094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.211+0000 7f52ea59c700 1 -- 172.21.15.42:0/4028805032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e0005040 con 0x7f52ec060290 2024-02-15T18:26:07.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.211+0000 7f52e9d9b700 1 --2- 172.21.15.42:0/4028805032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ec0fba80 0x7f52ec1094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:26:07.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.211+0000 7f52ea59c700 1 --2- 172.21.15.42:0/4028805032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52ec060290 0x7f52ec108fd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f52dc00e7f0 tx=0x7f52dc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:07.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.212+0000 7f52daffd700 1 -- 172.21.15.42:0/4028805032 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52dc0092e0 con 0x7f52ec060290 2024-02-15T18:26:07.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.212+0000 7f52eb59e700 1 -- 172.21.15.42:0/4028805032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec10a550 con 0x7f52ec060290 2024-02-15T18:26:07.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.212+0000 7f52daffd700 1 -- 172.21.15.42:0/4028805032 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f52dc00f040 con 0x7f52ec060290 2024-02-15T18:26:07.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.212+0000 7f52daffd700 1 -- 172.21.15.42:0/4028805032 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52dc014880 con 0x7f52ec060290 2024-02-15T18:26:07.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.212+0000 7f52eb59e700 1 -- 172.21.15.42:0/4028805032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec10aa90 con 0x7f52ec060290 2024-02-15T18:26:07.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.213+0000 7f52eb59e700 1 -- 172.21.15.42:0/4028805032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ec045ad0 con 0x7f52ec060290 2024-02-15T18:26:07.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.213+0000 7f52daffd700 1 -- 172.21.15.42:0/4028805032 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) v1 ==== 74714+0+0 (secure 0 0 0) 0x7f52dc01b030 con 0x7f52ec060290 2024-02-15T18:26:07.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.214+0000 7f52daffd700 1 --2- 172.21.15.42:0/4028805032 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f52d405e9b0 0x7f52d4060e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:07.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.214+0000 7f52daffd700 1 -- 172.21.15.42:0/4028805032 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f52dc078ef0 con 0x7f52ec060290 2024-02-15T18:26:07.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.214+0000 7f52e9d9b700 1 --2- 172.21.15.42:0/4028805032 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f52d405e9b0 0x7f52d4060e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:07.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.214+0000 7f52e9d9b700 1 --2- 172.21.15.42:0/4028805032 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f52d405e9b0 0x7f52d4060e70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f52e00043d0 tx=0x7f52e000b040).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:07.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.222+0000 7f52daffd700 1 -- 172.21.15.42:0/4028805032 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7f52dc04cea0 con 0x7f52ec060290 2024-02-15T18:26:07.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:05.596589+0000 mgr.smithi191.gftdzo (mgr.24395) 27 : cluster [DBG] pgmap v19: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:26:07.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:06.079462+0000 mgr.smithi191.gftdzo (mgr.24395) 28 : audit [DBG] from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:07.400 INFO:tasks.workunit.client.1.smithi191.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-15T18:26:07.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:05.596589+0000 mgr.smithi191.gftdzo (mgr.24395) 27 : cluster [DBG] pgmap v19: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:26:07.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:06.079462+0000 mgr.smithi191.gftdzo (mgr.24395) 28 : audit [DBG] from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:07.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.455+0000 7f52eb59e700 1 -- 172.21.15.42:0/4028805032 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f52ec10acb0 con 0x7f52ec060290 2024-02-15T18:26:07.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.455+0000 7f52daffd700 1 -- 172.21.15.42:0/4028805032 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1626 (secure 0 0 0) 0x7f52dc019020 con 0x7f52ec060290 2024-02-15T18:26:07.452 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:26:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:26:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:26:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:26:07.453 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:26:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:26:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:26:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:26:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:26:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:26:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:26:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:26:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:26:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:26:07.453 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:26:07.454 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:26:07.455 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:26:07.455 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000]] 2024-02-15T18:26:07.455 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509]] 2024-02-15T18:26:07.455 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:26:07.455 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:26:07.455 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:26:07.455 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:26:07.455 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366]] 2024-02-15T18:26:07.455 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796]] 2024-02-15T18:26:07.456 INFO:tasks.workunit.client.1.smithi191.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-15T18:26:07.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.457+0000 7f52eb59e700 1 -- 172.21.15.42:0/4028805032 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f52d405e9b0 msgr2=0x7f52d4060e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:07.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.457+0000 7f52eb59e700 1 --2- 172.21.15.42:0/4028805032 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f52d405e9b0 0x7f52d4060e70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f52e00043d0 tx=0x7f52e000b040).stop 2024-02-15T18:26:07.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.457+0000 7f52eb59e700 1 -- 172.21.15.42:0/4028805032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52ec060290 msgr2=0x7f52ec108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:07.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.457+0000 7f52eb59e700 1 --2- 172.21.15.42:0/4028805032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52ec060290 0x7f52ec108fd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f52dc00e7f0 tx=0x7f52dc00b040).stop 2024-02-15T18:26:07.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.457+0000 7f52eb59e700 1 -- 172.21.15.42:0/4028805032 shutdown_connections 2024-02-15T18:26:07.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.457+0000 7f52eb59e700 1 --2- 172.21.15.42:0/4028805032 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7f52d405e9b0 0x7f52d4060e70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.457+0000 7f52eb59e700 1 --2- 172.21.15.42:0/4028805032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52ec060290 0x7f52ec108fd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.457+0000 7f52eb59e700 1 --2- 172.21.15.42:0/4028805032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ec0fba80 0x7f52ec1094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.457+0000 7f52eb59e700 1 -- 172.21.15.42:0/4028805032 >> 172.21.15.42:0/4028805032 conn(0x7f52ec0f0b90 msgr2=0x7f52ec0f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:07.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.457+0000 7f52eb59e700 1 -- 172.21.15.42:0/4028805032 shutdown_connections 2024-02-15T18:26:07.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.457+0000 7f52eb59e700 1 -- 172.21.15.42:0/4028805032 wait complete. 2024-02-15T18:26:07.457 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:26:07.491 INFO:tasks.workunit.client.1.smithi191.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-15T18:26:07.493 INFO:tasks.workunit.client.1.smithi191.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-15T18:26:07.493 INFO:tasks.workunit.client.1.smithi191.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-15T18:26:07.529 INFO:tasks.workunit.client.1.smithi191.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-15T18:26:07.532 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-15T18:26:07.533 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-02-15T18:26:07.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.564+0000 7fa435418700 1 -- 172.21.15.42:0/1086066341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa430060290 msgr2=0x7fa4300fb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:07.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.564+0000 7fa435418700 1 --2- 172.21.15.42:0/1086066341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa430060290 0x7fa4300fb5a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fa420004690 tx=0x7fa420009b30).stop 2024-02-15T18:26:07.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.564+0000 7fa435418700 1 -- 172.21.15.42:0/1086066341 shutdown_connections 2024-02-15T18:26:07.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.564+0000 7fa435418700 1 --2- 172.21.15.42:0/1086066341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4300fba90 0x7fa4300fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.564+0000 7fa435418700 1 --2- 172.21.15.42:0/1086066341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa430060290 0x7fa4300fb5a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.564+0000 7fa435418700 1 -- 172.21.15.42:0/1086066341 >> 172.21.15.42:0/1086066341 conn(0x7fa4300f0ba0 msgr2=0x7fa4300f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:07.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.564+0000 7fa435418700 1 -- 172.21.15.42:0/1086066341 shutdown_connections 2024-02-15T18:26:07.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.564+0000 7fa435418700 1 -- 172.21.15.42:0/1086066341 wait complete. 2024-02-15T18:26:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.564+0000 7fa435418700 1 Processor -- start 2024-02-15T18:26:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.565+0000 7fa435418700 1 -- start start 2024-02-15T18:26:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.565+0000 7fa435418700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa430060290 0x7fa430109000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:07.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.565+0000 7fa435418700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4300fba90 0x7fa4301094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.565+0000 7fa435418700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa430109aa0 con 0x7fa430060290 2024-02-15T18:26:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.565+0000 7fa435418700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa430109be0 con 0x7fa4300fba90 2024-02-15T18:26:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.565+0000 7fa42ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa430060290 0x7fa430109000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.565+0000 7fa42ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa430060290 0x7fa430109000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42550/0 (socket says 172.21.15.42:42550) 2024-02-15T18:26:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.565+0000 7fa42ffff700 1 -- 172.21.15.42:0/1633978444 learned_addr learned my addr 172.21.15.42:0/1633978444 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:26:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.565+0000 7fa42f7fe700 1 --2- 172.21.15.42:0/1633978444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4300fba90 0x7fa4301094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.565+0000 7fa42ffff700 1 -- 172.21.15.42:0/1633978444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4300fba90 msgr2=0x7fa4301094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.565+0000 7fa42ffff700 1 --2- 172.21.15.42:0/1633978444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4300fba90 0x7fa4301094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.565+0000 7fa42ffff700 1 -- 172.21.15.42:0/1633978444 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420005040 con 0x7fa430060290 2024-02-15T18:26:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.565+0000 7fa42f7fe700 1 --2- 172.21.15.42:0/1633978444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4300fba90 0x7fa4301094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:26:07.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.565+0000 7fa42ffff700 1 --2- 172.21.15.42:0/1633978444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa430060290 0x7fa430109000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fa420004d70 tx=0x7fa42000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:07.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.566+0000 7fa42cff9700 1 -- 172.21.15.42:0/1633978444 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa420019070 con 0x7fa430060290 2024-02-15T18:26:07.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.566+0000 7fa42cff9700 1 -- 172.21.15.42:0/1633978444 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fa42001e420 con 0x7fa430060290 2024-02-15T18:26:07.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.566+0000 7fa435418700 1 -- 172.21.15.42:0/1633978444 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43010a520 con 0x7fa430060290 2024-02-15T18:26:07.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.566+0000 7fa42cff9700 1 -- 172.21.15.42:0/1633978444 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa420014660 con 0x7fa430060290 2024-02-15T18:26:07.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.566+0000 7fa435418700 1 -- 172.21.15.42:0/1633978444 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43010aa00 con 0x7fa430060290 2024-02-15T18:26:07.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.567+0000 7fa4167fc700 1 -- 172.21.15.42:0/1633978444 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa408004fa0 con 0x7fa430060290 2024-02-15T18:26:07.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.567+0000 7fa42cff9700 1 -- 172.21.15.42:0/1633978444 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) v1 ==== 74714+0+0 (secure 0 0 0) 0x7fa4200149d0 con 0x7fa430060290 2024-02-15T18:26:07.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.568+0000 7fa42cff9700 1 --2- 172.21.15.42:0/1633978444 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7fa41805e9b0 0x7fa418060e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:07.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.568+0000 7fa42cff9700 1 -- 172.21.15.42:0/1633978444 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa42007a680 con 0x7fa430060290 2024-02-15T18:26:07.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.568+0000 7fa42f7fe700 1 --2- 172.21.15.42:0/1633978444 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7fa41805e9b0 0x7fa418060e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:07.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.568+0000 7fa42f7fe700 1 --2- 172.21.15.42:0/1633978444 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7fa41805e9b0 0x7fa418060e70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa424009da0 tx=0x7fa42400b040).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:07.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.572+0000 7fa42cff9700 1 -- 172.21.15.42:0/1633978444 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7fa42004e680 con 0x7fa430060290 2024-02-15T18:26:07.591 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-02-15T18:26:07.593 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-15T18:26:07.593 DEBUG:teuthology.orchestra.run.smithi191:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bedfc49c6a823fb0ff75911e476e1f3cb464ad3e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-02-15T18:26:07.654 INFO:tasks.workunit.client.1.smithi191.stderr:+ mkdir -p fsstress 2024-02-15T18:26:07.656 INFO:tasks.workunit.client.1.smithi191.stderr:+ pushd fsstress 2024-02-15T18:26:07.657 INFO:tasks.workunit.client.1.smithi191.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-15T18:26:07.657 INFO:tasks.workunit.client.1.smithi191.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-15T18:26:07.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.780+0000 7fa4167fc700 1 -- 172.21.15.42:0/1633978444 --> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa408000bc0 con 0x7fa41805e9b0 2024-02-15T18:26:07.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.782+0000 7fa42cff9700 1 -- 172.21.15.42:0/1633978444 <== mgr.24395 v2:172.21.15.191:6828/2058110468 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7fa408000bc0 con 0x7fa41805e9b0 2024-02-15T18:26:07.779 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:26:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:26:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:26:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:26:07.779 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [], 2024-02-15T18:26:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "", 2024-02-15T18:26:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mgr daemons", 2024-02-15T18:26:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:26:07.780 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:26:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.784+0000 7fa4167fc700 1 -- 172.21.15.42:0/1633978444 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7fa41805e9b0 msgr2=0x7fa418060e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.784+0000 7fa4167fc700 1 --2- 172.21.15.42:0/1633978444 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7fa41805e9b0 0x7fa418060e70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa424009da0 tx=0x7fa42400b040).stop 2024-02-15T18:26:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.784+0000 7fa4167fc700 1 -- 172.21.15.42:0/1633978444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa430060290 msgr2=0x7fa430109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:07.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.784+0000 7fa4167fc700 1 --2- 172.21.15.42:0/1633978444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa430060290 0x7fa430109000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fa420004d70 tx=0x7fa42000b040).stop 2024-02-15T18:26:07.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.785+0000 7fa4167fc700 1 -- 172.21.15.42:0/1633978444 shutdown_connections 2024-02-15T18:26:07.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.785+0000 7fa4167fc700 1 --2- 172.21.15.42:0/1633978444 >> [v2:172.21.15.191:6828/2058110468,v1:172.21.15.191:6829/2058110468] conn(0x7fa41805e9b0 0x7fa418060e70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.785+0000 7fa4167fc700 1 --2- 172.21.15.42:0/1633978444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa430060290 0x7fa430109000 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.785+0000 7fa4167fc700 1 --2- 172.21.15.42:0/1633978444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4300fba90 0x7fa4301094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:07.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.785+0000 7fa4167fc700 1 -- 172.21.15.42:0/1633978444 >> 172.21.15.42:0/1633978444 conn(0x7fa4300f0ba0 msgr2=0x7fa4300f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:07.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.785+0000 7fa4167fc700 1 -- 172.21.15.42:0/1633978444 shutdown_connections 2024-02-15T18:26:07.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:07.785+0000 7fa4167fc700 1 -- 172.21.15.42:0/1633978444 wait complete. 2024-02-15T18:26:08.181 INFO:tasks.workunit.client.1.smithi191.stderr:+ tar xzf ltp-full.tgz 2024-02-15T18:26:08.208 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:06.381349+0000 mgr.smithi191.gftdzo (mgr.24395) 29 : audit [DBG] from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:08.208 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:06.702212+0000 mgr.smithi191.gftdzo (mgr.24395) 30 : audit [DBG] from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:08.208 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:07.094070+0000 mon.smithi042 (mon.0) 598 : audit [DBG] from='client.? 172.21.15.42:0/3463162904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:26:08.208 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:07.456352+0000 mon.smithi042 (mon.0) 599 : audit [DBG] from='client.? 172.21.15.42:0/4028805032' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:26:08.255 INFO:tasks.workunit.client.0.smithi042.stderr:Note: switching to 'bedfc49c6a823fb0ff75911e476e1f3cb464ad3e'. 2024-02-15T18:26:08.255 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-02-15T18:26:08.255 INFO:tasks.workunit.client.0.smithi042.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-15T18:26:08.255 INFO:tasks.workunit.client.0.smithi042.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-15T18:26:08.256 INFO:tasks.workunit.client.0.smithi042.stderr:state without impacting any branches by switching back to a branch. 2024-02-15T18:26:08.256 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-02-15T18:26:08.256 INFO:tasks.workunit.client.0.smithi042.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-15T18:26:08.256 INFO:tasks.workunit.client.0.smithi042.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-15T18:26:08.256 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-02-15T18:26:08.256 INFO:tasks.workunit.client.0.smithi042.stderr: git switch -c 2024-02-15T18:26:08.256 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-02-15T18:26:08.256 INFO:tasks.workunit.client.0.smithi042.stderr:Or undo this operation with: 2024-02-15T18:26:08.256 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-02-15T18:26:08.256 INFO:tasks.workunit.client.0.smithi042.stderr: git switch - 2024-02-15T18:26:08.256 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-02-15T18:26:08.256 INFO:tasks.workunit.client.0.smithi042.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-15T18:26:08.256 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-02-15T18:26:08.256 INFO:tasks.workunit.client.0.smithi042.stderr:HEAD is now at bedfc49c6a8 qa/suites/orch: whitelist warnings that are expected in test environments 2024-02-15T18:26:08.261 DEBUG:teuthology.orchestra.run.smithi042:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-15T18:26:08.357 INFO:tasks.workunit.client.0.smithi042.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-15T18:26:08.359 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-15T18:26:08.359 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-15T18:26:08.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:06.381349+0000 mgr.smithi191.gftdzo (mgr.24395) 29 : audit [DBG] from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:08.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:06.702212+0000 mgr.smithi191.gftdzo (mgr.24395) 30 : audit [DBG] from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:08.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:07.094070+0000 mon.smithi042 (mon.0) 598 : audit [DBG] from='client.? 172.21.15.42:0/3463162904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:26:08.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:07.456352+0000 mon.smithi042 (mon.0) 599 : audit [DBG] from='client.? 172.21.15.42:0/4028805032' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:26:09.194 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-15T18:26:09.273 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-15T18:26:09.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:07.596893+0000 mgr.smithi191.gftdzo (mgr.24395) 31 : cluster [DBG] pgmap v20: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:26:09.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:07.782643+0000 mgr.smithi191.gftdzo (mgr.24395) 32 : audit [DBG] from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:09.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:08.517881+0000 mon.smithi042 (mon.0) 600 : cluster [DBG] Standby manager daemon smithi042.kfoihb restarted 2024-02-15T18:26:09.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:08.517984+0000 mon.smithi042 (mon.0) 601 : cluster [DBG] Standby manager daemon smithi042.kfoihb started 2024-02-15T18:26:09.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:08.519296+0000 mon.smithi191 (mon.1) 51 : audit [DBG] from='mgr.? 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.kfoihb/crt"}]: dispatch 2024-02-15T18:26:09.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:08.519848+0000 mon.smithi191 (mon.1) 52 : audit [DBG] from='mgr.? 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:26:09.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:08.521951+0000 mon.smithi191 (mon.1) 53 : audit [DBG] from='mgr.? 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.kfoihb/key"}]: dispatch 2024-02-15T18:26:09.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:08.522847+0000 mon.smithi191 (mon.1) 54 : audit [DBG] from='mgr.? 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:26:09.320 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-15T18:26:09.321 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-15T18:26:09.321 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-15T18:26:09.361 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-15T18:26:09.365 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-02-15T18:26:09.365 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-15T18:26:09.384 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-02-15T18:26:09.385 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-15T18:26:09.385 DEBUG:teuthology.orchestra.run.smithi042:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bedfc49c6a823fb0ff75911e476e1f3cb464ad3e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-02-15T18:26:09.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:07.596893+0000 mgr.smithi191.gftdzo (mgr.24395) 31 : cluster [DBG] pgmap v20: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:26:09.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:07.782643+0000 mgr.smithi191.gftdzo (mgr.24395) 32 : audit [DBG] from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:09.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:08.517881+0000 mon.smithi042 (mon.0) 600 : cluster [DBG] Standby manager daemon smithi042.kfoihb restarted 2024-02-15T18:26:09.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:08.517984+0000 mon.smithi042 (mon.0) 601 : cluster [DBG] Standby manager daemon smithi042.kfoihb started 2024-02-15T18:26:09.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:08.519296+0000 mon.smithi191 (mon.1) 51 : audit [DBG] from='mgr.? 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.kfoihb/crt"}]: dispatch 2024-02-15T18:26:09.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:08.519848+0000 mon.smithi191 (mon.1) 52 : audit [DBG] from='mgr.? 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:26:09.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:08.521951+0000 mon.smithi191 (mon.1) 53 : audit [DBG] from='mgr.? 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.kfoihb/key"}]: dispatch 2024-02-15T18:26:09.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:08.522847+0000 mon.smithi191 (mon.1) 54 : audit [DBG] from='mgr.? 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:26:09.451 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir -p fsstress 2024-02-15T18:26:09.452 INFO:tasks.workunit.client.0.smithi042.stderr:+ pushd fsstress 2024-02-15T18:26:09.453 INFO:tasks.workunit.client.0.smithi042.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-15T18:26:09.453 INFO:tasks.workunit.client.0.smithi042.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-15T18:26:10.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:09.042558+0000 mon.smithi042 (mon.0) 602 : cluster [DBG] mgrmap e24: smithi191.gftdzo(active, since 36s), standbys: smithi042.kfoihb 2024-02-15T18:26:10.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:09.654637+0000 mon.smithi042 (mon.0) 603 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:26:10.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:09.655945+0000 mon.smithi191 (mon.1) 55 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:26:10.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:09.656620+0000 mon.smithi191 (mon.1) 56 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:26:10.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:09.660411+0000 mon.smithi042 (mon.0) 604 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:26:10.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:09.666045+0000 mon.smithi191 (mon.1) 57 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:10.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:09.671026+0000 mon.smithi191 (mon.1) 58 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mgr fail", "who": "smithi191.gftdzo"}]: dispatch 2024-02-15T18:26:10.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:09.671673+0000 mon.smithi042 (mon.0) 605 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mgr fail", "who": "smithi191.gftdzo"}]: dispatch 2024-02-15T18:26:10.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:09.674361+0000 mon.smithi042 (mon.0) 606 : cluster [DBG] osdmap e47: 6 total, 6 up, 6 in 2024-02-15T18:26:10.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:09.675656+0000 mon.smithi191 (mon.1) 59 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-15T18:26:10.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:09.675943+0000 mon.smithi042 (mon.0) 607 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-15T18:26:10.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:09.676718+0000 mon.smithi191 (mon.1) 60 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:26:10.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:09.677002+0000 mon.smithi042 (mon.0) 608 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:26:10.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:09.042558+0000 mon.smithi042 (mon.0) 602 : cluster [DBG] mgrmap e24: smithi191.gftdzo(active, since 36s), standbys: smithi042.kfoihb 2024-02-15T18:26:10.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:09.654637+0000 mon.smithi042 (mon.0) 603 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:26:10.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:09.655945+0000 mon.smithi191 (mon.1) 55 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:26:10.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:09.656620+0000 mon.smithi191 (mon.1) 56 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:26:10.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:09.660411+0000 mon.smithi042 (mon.0) 604 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' 2024-02-15T18:26:10.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:09.666045+0000 mon.smithi191 (mon.1) 57 : audit [DBG] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:10.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:09.671026+0000 mon.smithi191 (mon.1) 58 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mgr fail", "who": "smithi191.gftdzo"}]: dispatch 2024-02-15T18:26:10.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:09.671673+0000 mon.smithi042 (mon.0) 605 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "mgr fail", "who": "smithi191.gftdzo"}]: dispatch 2024-02-15T18:26:10.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:09.674361+0000 mon.smithi042 (mon.0) 606 : cluster [DBG] osdmap e47: 6 total, 6 up, 6 in 2024-02-15T18:26:10.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:09.675656+0000 mon.smithi191 (mon.1) 59 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-15T18:26:10.448 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:09.675943+0000 mon.smithi042 (mon.0) 607 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-15T18:26:10.448 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:09.676718+0000 mon.smithi191 (mon.1) 60 : audit [INF] from='mgr.24395 172.21.15.191:0/2232364028' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:26:10.448 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:09.677002+0000 mon.smithi042 (mon.0) 608 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:26:11.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:09.597248+0000 mgr.smithi191.gftdzo (mgr.24395) 33 : cluster [DBG] pgmap v21: 65 pgs: 65 active+clean; 12 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-02-15T18:26:11.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:26:09.657226+0000 mgr.smithi191.gftdzo (mgr.24395) 34 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:26:11.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:26:09.668070+0000 mgr.smithi191.gftdzo (mgr.24395) 35 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi191.gftdzo) 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:26:09.668606+0000 mgr.smithi191.gftdzo (mgr.24395) 36 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi191.gftdzo) 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:26:09.670836+0000 mgr.smithi191.gftdzo (mgr.24395) 37 : cephadm [INF] Failing over to other MGR 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.665260+0000 mon.smithi042 (mon.0) 609 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd='[{"prefix": "mgr fail", "who": "smithi191.gftdzo"}]': finished 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:10.665362+0000 mon.smithi042 (mon.0) 610 : cluster [DBG] mgrmap e25: smithi042.kfoihb(active, starting, since 0.993199s) 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.666540+0000 mon.smithi042 (mon.0) 611 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.666685+0000 mon.smithi042 (mon.0) 612 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.667097+0000 mon.smithi042 (mon.0) 613 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.vtwvlo"}]: dispatch 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.667331+0000 mon.smithi042 (mon.0) 614 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.idnxkc"}]: dispatch 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.667539+0000 mon.smithi042 (mon.0) 615 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.icwcnb"}]: dispatch 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.667740+0000 mon.smithi042 (mon.0) 616 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.erlpxp"}]: dispatch 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.668052+0000 mon.smithi042 (mon.0) 617 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr metadata", "who": "smithi042.kfoihb", "id": "smithi042.kfoihb"}]: dispatch 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.668245+0000 mon.smithi042 (mon.0) 618 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.668505+0000 mon.smithi042 (mon.0) 619 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.668714+0000 mon.smithi042 (mon.0) 620 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:26:11.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.668936+0000 mon.smithi042 (mon.0) 621 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:26:11.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.669297+0000 mon.smithi042 (mon.0) 622 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:26:11.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.669673+0000 mon.smithi042 (mon.0) 623 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:26:11.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.670045+0000 mon.smithi042 (mon.0) 624 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:26:11.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.670336+0000 mon.smithi042 (mon.0) 625 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:26:11.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.670688+0000 mon.smithi042 (mon.0) 626 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:26:11.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.676772+0000 mon.smithi042 (mon.0) 627 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-15T18:26:11.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:10.676865+0000 mon.smithi042 (mon.0) 628 : cluster [DBG] osdmap e48: 6 total, 6 up, 6 in 2024-02-15T18:26:11.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:10.686132+0000 mon.smithi042 (mon.0) 629 : cluster [INF] Manager daemon smithi042.kfoihb is now available 2024-02-15T18:26:11.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:26:10.698051+0000 mgr.smithi042.kfoihb (mgr.24435) 1 : cephadm [INF] inventory: adjusted host smithi191 addr 'smithi191' -> '172.21.15.191' 2024-02-15T18:26:11.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.701520+0000 mon.smithi042 (mon.0) 630 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:11.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.801983+0000 mon.smithi042 (mon.0) 631 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:11.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.803550+0000 mon.smithi042 (mon.0) 632 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:11.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.842365+0000 mon.smithi042 (mon.0) 633 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:26:11.199 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:10.852103+0000 mon.smithi042 (mon.0) 634 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:26:11.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:09.597248+0000 mgr.smithi191.gftdzo (mgr.24395) 33 : cluster [DBG] pgmap v21: 65 pgs: 65 active+clean; 12 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-02-15T18:26:11.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:26:09.657226+0000 mgr.smithi191.gftdzo (mgr.24395) 34 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:26:11.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:26:09.668070+0000 mgr.smithi191.gftdzo (mgr.24395) 35 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi191.gftdzo) 2024-02-15T18:26:11.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:26:09.668606+0000 mgr.smithi191.gftdzo (mgr.24395) 36 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi191.gftdzo) 2024-02-15T18:26:11.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:26:09.670836+0000 mgr.smithi191.gftdzo (mgr.24395) 37 : cephadm [INF] Failing over to other MGR 2024-02-15T18:26:11.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.665260+0000 mon.smithi042 (mon.0) 609 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd='[{"prefix": "mgr fail", "who": "smithi191.gftdzo"}]': finished 2024-02-15T18:26:11.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:10.665362+0000 mon.smithi042 (mon.0) 610 : cluster [DBG] mgrmap e25: smithi042.kfoihb(active, starting, since 0.993199s) 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.666540+0000 mon.smithi042 (mon.0) 611 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.666685+0000 mon.smithi042 (mon.0) 612 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.667097+0000 mon.smithi042 (mon.0) 613 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.vtwvlo"}]: dispatch 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.667331+0000 mon.smithi042 (mon.0) 614 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.idnxkc"}]: dispatch 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.667539+0000 mon.smithi042 (mon.0) 615 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.icwcnb"}]: dispatch 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.667740+0000 mon.smithi042 (mon.0) 616 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.erlpxp"}]: dispatch 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.668052+0000 mon.smithi042 (mon.0) 617 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr metadata", "who": "smithi042.kfoihb", "id": "smithi042.kfoihb"}]: dispatch 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.668245+0000 mon.smithi042 (mon.0) 618 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.668505+0000 mon.smithi042 (mon.0) 619 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.668714+0000 mon.smithi042 (mon.0) 620 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.668936+0000 mon.smithi042 (mon.0) 621 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.669297+0000 mon.smithi042 (mon.0) 622 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.669673+0000 mon.smithi042 (mon.0) 623 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.670045+0000 mon.smithi042 (mon.0) 624 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-15T18:26:11.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.670336+0000 mon.smithi042 (mon.0) 625 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-15T18:26:11.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.670688+0000 mon.smithi042 (mon.0) 626 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-15T18:26:11.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.676772+0000 mon.smithi042 (mon.0) 627 : audit [INF] from='mgr.24395 ' entity='mgr.smithi191.gftdzo' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-15T18:26:11.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:10.676865+0000 mon.smithi042 (mon.0) 628 : cluster [DBG] osdmap e48: 6 total, 6 up, 6 in 2024-02-15T18:26:11.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:10.686132+0000 mon.smithi042 (mon.0) 629 : cluster [INF] Manager daemon smithi042.kfoihb is now available 2024-02-15T18:26:11.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:26:10.698051+0000 mgr.smithi042.kfoihb (mgr.24435) 1 : cephadm [INF] inventory: adjusted host smithi191 addr 'smithi191' -> '172.21.15.191' 2024-02-15T18:26:11.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.701520+0000 mon.smithi042 (mon.0) 630 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:11.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.801983+0000 mon.smithi042 (mon.0) 631 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:11.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.803550+0000 mon.smithi042 (mon.0) 632 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:11.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.842365+0000 mon.smithi042 (mon.0) 633 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/mirror_snapshot_schedule"}]: dispatch 2024-02-15T18:26:11.312 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:10.852103+0000 mon.smithi042 (mon.0) 634 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.kfoihb/trash_purge_schedule"}]: dispatch 2024-02-15T18:26:12.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:11.680563+0000 mon.smithi042 (mon.0) 635 : cluster [DBG] mgrmap e26: smithi042.kfoihb(active, since 2s) 2024-02-15T18:26:12.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:11.693533+0000 mgr.smithi042.kfoihb (mgr.24435) 2 : cluster [DBG] pgmap v4: 65 pgs: 65 active+clean; 7.1 MiB data, 51 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:26:12.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:11.680563+0000 mon.smithi042 (mon.0) 635 : cluster [DBG] mgrmap e26: smithi042.kfoihb(active, since 2s) 2024-02-15T18:26:12.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:11.693533+0000 mgr.smithi042.kfoihb (mgr.24435) 2 : cluster [DBG] pgmap v4: 65 pgs: 65 active+clean; 7.1 MiB data, 51 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:26:14.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:12.669590+0000 mgr.smithi042.kfoihb (mgr.24435) 3 : cluster [DBG] pgmap v5: 65 pgs: 65 active+clean; 7.1 MiB data, 51 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:26:14.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:12.683218+0000 mon.smithi042 (mon.0) 636 : cluster [DBG] mgrmap e27: smithi042.kfoihb(active, since 3s) 2024-02-15T18:26:14.157 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:12.669590+0000 mgr.smithi042.kfoihb (mgr.24435) 3 : cluster [DBG] pgmap v5: 65 pgs: 65 active+clean; 7.1 MiB data, 51 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:26:14.157 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:12.683218+0000 mon.smithi042 (mon.0) 636 : cluster [DBG] mgrmap e27: smithi042.kfoihb(active, since 3s) 2024-02-15T18:26:15.447 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:14.669902+0000 mgr.smithi042.kfoihb (mgr.24435) 4 : cluster [DBG] pgmap v6: 65 pgs: 65 active+clean; 7.1 MiB data, 51 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:26:15.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:14.669902+0000 mgr.smithi042.kfoihb (mgr.24435) 4 : cluster [DBG] pgmap v6: 65 pgs: 65 active+clean; 7.1 MiB data, 51 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:26:16.571 INFO:tasks.workunit.client.0.smithi042.stderr:+ tar xzf ltp-full.tgz 2024-02-15T18:26:16.871 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:15.731382+0000 mon.smithi042 (mon.0) 637 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:17.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:15.731382+0000 mon.smithi042 (mon.0) 637 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:17.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:16.670223+0000 mgr.smithi042.kfoihb (mgr.24435) 5 : cluster [DBG] pgmap v7: 65 pgs: 65 active+clean; 7.1 MiB data, 51 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:26:17.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:17.531577+0000 mon.smithi042 (mon.0) 638 : cluster [DBG] Standby manager daemon smithi191.gftdzo started 2024-02-15T18:26:17.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:17.533197+0000 mon.smithi191 (mon.1) 61 : audit [DBG] from='mgr.? 172.21.15.191:0/3190529249' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/crt"}]: dispatch 2024-02-15T18:26:17.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:17.533761+0000 mon.smithi191 (mon.1) 62 : audit [DBG] from='mgr.? 172.21.15.191:0/3190529249' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:26:17.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:17.535892+0000 mon.smithi191 (mon.1) 63 : audit [DBG] from='mgr.? 172.21.15.191:0/3190529249' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/key"}]: dispatch 2024-02-15T18:26:17.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:17.536759+0000 mon.smithi191 (mon.1) 64 : audit [DBG] from='mgr.? 172.21.15.191:0/3190529249' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:26:17.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:17.650261+0000 mon.smithi042 (mon.0) 639 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:18.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:16.670223+0000 mgr.smithi042.kfoihb (mgr.24435) 5 : cluster [DBG] pgmap v7: 65 pgs: 65 active+clean; 7.1 MiB data, 51 MiB used, 536 GiB / 536 GiB avail 2024-02-15T18:26:18.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:17.531577+0000 mon.smithi042 (mon.0) 638 : cluster [DBG] Standby manager daemon smithi191.gftdzo started 2024-02-15T18:26:18.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:17.533197+0000 mon.smithi191 (mon.1) 61 : audit [DBG] from='mgr.? 172.21.15.191:0/3190529249' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/crt"}]: dispatch 2024-02-15T18:26:18.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:17.533761+0000 mon.smithi191 (mon.1) 62 : audit [DBG] from='mgr.? 172.21.15.191:0/3190529249' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:26:18.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:17.535892+0000 mon.smithi191 (mon.1) 63 : audit [DBG] from='mgr.? 172.21.15.191:0/3190529249' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/key"}]: dispatch 2024-02-15T18:26:18.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:17.536759+0000 mon.smithi191 (mon.1) 64 : audit [DBG] from='mgr.? 172.21.15.191:0/3190529249' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:26:18.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:17.650261+0000 mon.smithi042 (mon.0) 639 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:18.887 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:17.743744+0000 mon.smithi042 (mon.0) 640 : cluster [DBG] mgrmap e28: smithi042.kfoihb(active, since 8s), standbys: smithi191.gftdzo 2024-02-15T18:26:18.887 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:17.743923+0000 mon.smithi042 (mon.0) 641 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr metadata", "who": "smithi191.gftdzo", "id": "smithi191.gftdzo"}]: dispatch 2024-02-15T18:26:18.887 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:17.958481+0000 mon.smithi042 (mon.0) 642 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:26:18.887 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:17.959313+0000 mon.smithi042 (mon.0) 643 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:26:18.887 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:17.963458+0000 mon.smithi042 (mon.0) 644 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:18.887 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:17.968695+0000 mon.smithi042 (mon.0) 645 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:19.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:17.743744+0000 mon.smithi042 (mon.0) 640 : cluster [DBG] mgrmap e28: smithi042.kfoihb(active, since 8s), standbys: smithi191.gftdzo 2024-02-15T18:26:19.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:17.743923+0000 mon.smithi042 (mon.0) 641 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr metadata", "who": "smithi191.gftdzo", "id": "smithi191.gftdzo"}]: dispatch 2024-02-15T18:26:19.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:17.958481+0000 mon.smithi042 (mon.0) 642 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:26:19.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:17.959313+0000 mon.smithi042 (mon.0) 643 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:26:19.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:17.963458+0000 mon.smithi042 (mon.0) 644 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:19.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:17.968695+0000 mon.smithi042 (mon.0) 645 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:20.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:26:17.959980+0000 mgr.smithi042.kfoihb (mgr.24435) 6 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:26:20.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:18.670788+0000 mgr.smithi042.kfoihb (mgr.24435) 7 : cluster [DBG] pgmap v8: 65 pgs: 65 active+clean; 60 MiB data, 251 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 8.0 MiB/s wr, 679 op/s 2024-02-15T18:26:20.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:26:17.959980+0000 mgr.smithi042.kfoihb (mgr.24435) 6 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:26:20.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:18.670788+0000 mgr.smithi042.kfoihb (mgr.24435) 7 : cluster [DBG] pgmap v8: 65 pgs: 65 active+clean; 60 MiB data, 251 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 8.0 MiB/s wr, 679 op/s 2024-02-15T18:26:20.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:26:19.765183+0000 mgr.smithi042.kfoihb (mgr.24435) 8 : cephadm [INF] Upgrade: Updating mgr.smithi191.gftdzo 2024-02-15T18:26:20.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:19.765522+0000 mon.smithi042 (mon.0) 646 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi191.gftdzo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:26:20.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:19.766234+0000 mon.smithi042 (mon.0) 647 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:26:20.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:19.766933+0000 mon.smithi042 (mon.0) 648 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:26:20.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:26:19.767512+0000 mgr.smithi042.kfoihb (mgr.24435) 9 : cephadm [INF] Deploying daemon mgr.smithi191.gftdzo on smithi191 2024-02-15T18:26:21.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:26:19.765183+0000 mgr.smithi042.kfoihb (mgr.24435) 8 : cephadm [INF] Upgrade: Updating mgr.smithi191.gftdzo 2024-02-15T18:26:21.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:19.765522+0000 mon.smithi042 (mon.0) 646 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi191.gftdzo", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-15T18:26:21.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:19.766234+0000 mon.smithi042 (mon.0) 647 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-15T18:26:21.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:19.766933+0000 mon.smithi042 (mon.0) 648 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:26:21.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:26:19.767512+0000 mgr.smithi042.kfoihb (mgr.24435) 9 : cephadm [INF] Deploying daemon mgr.smithi191.gftdzo on smithi191 2024-02-15T18:26:22.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:20.671079+0000 mgr.smithi042.kfoihb (mgr.24435) 10 : cluster [DBG] pgmap v9: 65 pgs: 65 active+clean; 60 MiB data, 251 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 6.2 MiB/s wr, 527 op/s 2024-02-15T18:26:22.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:20.671079+0000 mgr.smithi042.kfoihb (mgr.24435) 10 : cluster [DBG] pgmap v9: 65 pgs: 65 active+clean; 60 MiB data, 251 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 6.2 MiB/s wr, 527 op/s 2024-02-15T18:26:23.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:22.671728+0000 mgr.smithi042.kfoihb (mgr.24435) 11 : cluster [DBG] pgmap v10: 65 pgs: 65 active+clean; 72 MiB data, 300 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 448 op/s 2024-02-15T18:26:24.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:22.671728+0000 mgr.smithi042.kfoihb (mgr.24435) 11 : cluster [DBG] pgmap v10: 65 pgs: 65 active+clean; 72 MiB data, 300 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 6.2 MiB/s wr, 448 op/s 2024-02-15T18:26:25.957 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:24.672057+0000 mgr.smithi042.kfoihb (mgr.24435) 12 : cluster [DBG] pgmap v11: 65 pgs: 65 active+clean; 72 MiB data, 300 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 5.6 MiB/s wr, 410 op/s 2024-02-15T18:26:25.957 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:24.736349+0000 mon.smithi042 (mon.0) 649 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:25.957 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:24.737187+0000 mon.smithi042 (mon.0) 650 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:26.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:24.672057+0000 mgr.smithi042.kfoihb (mgr.24435) 12 : cluster [DBG] pgmap v11: 65 pgs: 65 active+clean; 72 MiB data, 300 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 5.6 MiB/s wr, 410 op/s 2024-02-15T18:26:26.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:24.736349+0000 mon.smithi042 (mon.0) 649 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:26.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:24.737187+0000 mon.smithi042 (mon.0) 650 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:28.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:26.672471+0000 mgr.smithi042.kfoihb (mgr.24435) 13 : cluster [DBG] pgmap v12: 65 pgs: 65 active+clean; 72 MiB data, 300 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 5.6 MiB/s wr, 410 op/s 2024-02-15T18:26:28.156 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:26.672471+0000 mgr.smithi042.kfoihb (mgr.24435) 13 : cluster [DBG] pgmap v12: 65 pgs: 65 active+clean; 72 MiB data, 300 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 5.6 MiB/s wr, 410 op/s 2024-02-15T18:26:29.779 INFO:tasks.workunit.client.1.smithi191.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-15T18:26:29.780 INFO:tasks.workunit.client.1.smithi191.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-15T18:26:29.780 INFO:tasks.workunit.client.1.smithi191.stderr:+ make 2024-02-15T18:26:30.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:28.673179+0000 mgr.smithi042.kfoihb (mgr.24435) 14 : cluster [DBG] pgmap v13: 65 pgs: 65 active+clean; 106 MiB data, 448 MiB used, 536 GiB / 536 GiB avail; 2.9 MiB/s rd, 8.7 MiB/s wr, 776 op/s 2024-02-15T18:26:30.176 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:28.673179+0000 mgr.smithi042.kfoihb (mgr.24435) 14 : cluster [DBG] pgmap v13: 65 pgs: 65 active+clean; 106 MiB data, 448 MiB used, 536 GiB / 536 GiB avail; 2.9 MiB/s rd, 8.7 MiB/s wr, 776 op/s 2024-02-15T18:26:30.361 INFO:tasks.workunit.client.1.smithi191.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-15T18:26:31.881 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:30.673556+0000 mgr.smithi042.kfoihb (mgr.24435) 15 : cluster [DBG] pgmap v14: 65 pgs: 65 active+clean; 106 MiB data, 448 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 381 op/s 2024-02-15T18:26:31.881 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:31.551287+0000 mon.smithi042 (mon.0) 651 : cluster [DBG] Standby manager daemon smithi191.gftdzo restarted 2024-02-15T18:26:31.881 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:31.551462+0000 mon.smithi042 (mon.0) 652 : cluster [DBG] Standby manager daemon smithi191.gftdzo started 2024-02-15T18:26:31.881 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:31.554477+0000 mon.smithi191 (mon.1) 65 : audit [DBG] from='mgr.? 172.21.15.191:0/2255132882' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/crt"}]: dispatch 2024-02-15T18:26:31.881 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:31.555342+0000 mon.smithi191 (mon.1) 66 : audit [DBG] from='mgr.? 172.21.15.191:0/2255132882' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:26:31.881 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:31.557991+0000 mon.smithi191 (mon.1) 67 : audit [DBG] from='mgr.? 172.21.15.191:0/2255132882' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/key"}]: dispatch 2024-02-15T18:26:31.881 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:31.558356+0000 mon.smithi191 (mon.1) 68 : audit [DBG] from='mgr.? 172.21.15.191:0/2255132882' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:26:31.892 INFO:tasks.workunit.client.1.smithi191.stderr:++ readlink -f fsstress 2024-02-15T18:26:31.893 INFO:tasks.workunit.client.1.smithi191.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-15T18:26:31.893 INFO:tasks.workunit.client.1.smithi191.stderr:+ popd 2024-02-15T18:26:31.893 INFO:tasks.workunit.client.1.smithi191.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-15T18:26:31.893 INFO:tasks.workunit.client.1.smithi191.stderr:+ popd 2024-02-15T18:26:31.894 INFO:tasks.workunit.client.1.smithi191.stdout:~/cephtest/mnt.1/client.1/tmp 2024-02-15T18:26:31.894 INFO:tasks.workunit.client.1.smithi191.stderr:++ mktemp -d -p . 2024-02-15T18:26:31.895 INFO:tasks.workunit.client.1.smithi191.stderr:+ T=./tmp.ca64ueUWIM 2024-02-15T18:26:31.895 INFO:tasks.workunit.client.1.smithi191.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.ca64ueUWIM -l 1 -n 1000 -p 10 -v 2024-02-15T18:26:31.896 INFO:tasks.workunit.client.1.smithi191.stdout:seed = 1707317114 2024-02-15T18:26:31.901 INFO:tasks.workunit.client.1.smithi191.stdout:1/0: truncate - no filename 2024-02-15T18:26:31.903 INFO:tasks.workunit.client.1.smithi191.stdout:2/0: write - no filename 2024-02-15T18:26:31.903 INFO:tasks.workunit.client.1.smithi191.stdout:2/1: dwrite - no filename 2024-02-15T18:26:31.903 INFO:tasks.workunit.client.1.smithi191.stdout:2/2: rename - no filename 2024-02-15T18:26:31.903 INFO:tasks.workunit.client.1.smithi191.stdout:2/3: rename - no filename 2024-02-15T18:26:31.903 INFO:tasks.workunit.client.1.smithi191.stdout:2/4: dread - no filename 2024-02-15T18:26:31.903 INFO:tasks.workunit.client.1.smithi191.stdout:2/5: write - no filename 2024-02-15T18:26:31.903 INFO:tasks.workunit.client.1.smithi191.stdout:1/1: mkdir d0 0 2024-02-15T18:26:31.903 INFO:tasks.workunit.client.1.smithi191.stdout:1/2: dwrite - no filename 2024-02-15T18:26:31.903 INFO:tasks.workunit.client.1.smithi191.stdout:1/3: write - no filename 2024-02-15T18:26:31.904 INFO:tasks.workunit.client.1.smithi191.stdout:3/0: chown . 3062 1 2024-02-15T18:26:31.904 INFO:tasks.workunit.client.1.smithi191.stdout:3/1: link - no file 2024-02-15T18:26:31.904 INFO:tasks.workunit.client.1.smithi191.stdout:3/2: write - no filename 2024-02-15T18:26:31.904 INFO:tasks.workunit.client.1.smithi191.stdout:3/3: truncate - no filename 2024-02-15T18:26:31.904 INFO:tasks.workunit.client.1.smithi191.stdout:3/4: write - no filename 2024-02-15T18:26:31.905 INFO:tasks.workunit.client.1.smithi191.stdout:2/6: symlink l0 0 2024-02-15T18:26:31.905 INFO:tasks.workunit.client.1.smithi191.stdout:2/7: dwrite - no filename 2024-02-15T18:26:31.905 INFO:tasks.workunit.client.1.smithi191.stdout:1/4: creat d0/f1 x:0 0 0 2024-02-15T18:26:31.906 INFO:tasks.workunit.client.1.smithi191.stdout:1/5: truncate d0/f1 600606 0 2024-02-15T18:26:31.906 INFO:tasks.workunit.client.1.smithi191.stdout:4/0: write - no filename 2024-02-15T18:26:31.907 INFO:tasks.workunit.client.1.smithi191.stdout:2/8: creat f1 x:0 0 0 2024-02-15T18:26:31.907 INFO:tasks.workunit.client.1.smithi191.stdout:2/9: stat l0 0 2024-02-15T18:26:31.908 INFO:tasks.workunit.client.1.smithi191.stdout:5/0: dread - no filename 2024-02-15T18:26:31.912 INFO:tasks.workunit.client.1.smithi191.stdout:4/1: creat f0 x:0 0 0 2024-02-15T18:26:31.912 INFO:tasks.workunit.client.1.smithi191.stdout:4/2: rmdir - no directory 2024-02-15T18:26:31.912 INFO:tasks.workunit.client.1.smithi191.stdout:6/0: readlink - no filename 2024-02-15T18:26:31.912 INFO:tasks.workunit.client.1.smithi191.stdout:6/1: chown . 466481204 1 2024-02-15T18:26:31.912 INFO:tasks.workunit.client.1.smithi191.stdout:6/2: link - no file 2024-02-15T18:26:31.914 INFO:tasks.workunit.client.1.smithi191.stdout:2/10: symlink l2 0 2024-02-15T18:26:31.914 INFO:tasks.workunit.client.1.smithi191.stdout:4/3: creat f1 x:0 0 0 2024-02-15T18:26:31.914 INFO:tasks.workunit.client.1.smithi191.stdout:6/3: creat f0 x:0 0 0 2024-02-15T18:26:31.916 INFO:tasks.workunit.client.1.smithi191.stdout:6/4: creat f1 x:0 0 0 2024-02-15T18:26:31.926 INFO:tasks.workunit.client.1.smithi191.stdout:7/0: getdents . 0 2024-02-15T18:26:31.926 INFO:tasks.workunit.client.1.smithi191.stdout:7/1: fdatasync - no filename 2024-02-15T18:26:31.926 INFO:tasks.workunit.client.1.smithi191.stdout:7/2: rename - no filename 2024-02-15T18:26:31.926 INFO:tasks.workunit.client.1.smithi191.stdout:7/3: write - no filename 2024-02-15T18:26:31.940 INFO:tasks.workunit.client.1.smithi191.stdout:7/4: mkdir d0 0 2024-02-15T18:26:31.940 INFO:tasks.workunit.client.1.smithi191.stdout:7/5: dread - no filename 2024-02-15T18:26:31.940 INFO:tasks.workunit.client.1.smithi191.stdout:7/6: dread - no filename 2024-02-15T18:26:31.940 INFO:tasks.workunit.client.1.smithi191.stdout:7/7: truncate - no filename 2024-02-15T18:26:31.940 INFO:tasks.workunit.client.1.smithi191.stdout:7/8: getdents d0 0 2024-02-15T18:26:31.940 INFO:tasks.workunit.client.1.smithi191.stdout:8/0: rename - no filename 2024-02-15T18:26:31.940 INFO:tasks.workunit.client.1.smithi191.stdout:8/1: write - no filename 2024-02-15T18:26:31.940 INFO:tasks.workunit.client.1.smithi191.stdout:8/2: getdents . 0 2024-02-15T18:26:31.940 INFO:tasks.workunit.client.1.smithi191.stdout:8/3: read - no filename 2024-02-15T18:26:31.940 INFO:tasks.workunit.client.1.smithi191.stdout:8/4: dread - no filename 2024-02-15T18:26:31.941 INFO:tasks.workunit.client.1.smithi191.stdout:8/5: read - no filename 2024-02-15T18:26:31.941 INFO:tasks.workunit.client.1.smithi191.stdout:9/0: dread - no filename 2024-02-15T18:26:31.941 INFO:tasks.workunit.client.1.smithi191.stdout:9/1: dread - no filename 2024-02-15T18:26:31.942 INFO:tasks.workunit.client.1.smithi191.stdout:7/9: mkdir d0/d1 0 2024-02-15T18:26:31.942 INFO:tasks.workunit.client.1.smithi191.stdout:8/6: mknod c0 0 2024-02-15T18:26:31.942 INFO:tasks.workunit.client.1.smithi191.stdout:8/7: stat c0 0 2024-02-15T18:26:31.943 INFO:tasks.workunit.client.1.smithi191.stdout:9/2: mkdir d0 0 2024-02-15T18:26:31.945 INFO:tasks.workunit.client.1.smithi191.stdout:7/10: mknod d0/d1/c2 0 2024-02-15T18:26:31.955 INFO:tasks.workunit.client.1.smithi191.stdout:9/3: creat d0/f1 x:0 0 0 2024-02-15T18:26:31.955 INFO:tasks.workunit.client.1.smithi191.stdout:9/4: truncate d0/f1 330032 0 2024-02-15T18:26:31.957 INFO:tasks.workunit.client.1.smithi191.stdout:9/5: mknod d0/c2 0 2024-02-15T18:26:31.961 INFO:tasks.workunit.client.1.smithi191.stdout:8/8: link c0 c1 0 2024-02-15T18:26:31.962 INFO:tasks.workunit.client.1.smithi191.stdout:7/11: mkdir d0/d1/d3 0 2024-02-15T18:26:31.962 INFO:tasks.workunit.client.1.smithi191.stdout:7/12: chown d0/d1/c2 1889 1 2024-02-15T18:26:31.962 INFO:tasks.workunit.client.1.smithi191.stdout:7/13: dwrite - no filename 2024-02-15T18:26:31.964 INFO:tasks.workunit.client.1.smithi191.stdout:8/9: creat f2 x:0 0 0 2024-02-15T18:26:31.969 INFO:tasks.workunit.client.1.smithi191.stdout:8/10: mkdir d3 0 2024-02-15T18:26:31.969 INFO:tasks.workunit.client.1.smithi191.stdout:8/11: dread - f2 zero size 2024-02-15T18:26:31.973 INFO:tasks.workunit.client.1.smithi191.stdout:7/14: creat d0/d1/d3/f4 x:0 0 0 2024-02-15T18:26:31.977 INFO:tasks.workunit.client.1.smithi191.stdout:8/12: link c1 d3/c4 0 2024-02-15T18:26:31.982 INFO:tasks.workunit.client.1.smithi191.stdout:8/13: mknod d3/c5 0 2024-02-15T18:26:31.986 INFO:tasks.workunit.client.1.smithi191.stdout:8/14: rename f2 to d3/f6 0 2024-02-15T18:26:31.989 INFO:tasks.workunit.client.1.smithi191.stdout:8/15: rename c0 to d3/c7 0 2024-02-15T18:26:32.041 INFO:tasks.workunit.client.1.smithi191.stdout:1/6: read d0/f1 [343123,33335] 0 2024-02-15T18:26:32.062 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:30.673556+0000 mgr.smithi042.kfoihb (mgr.24435) 15 : cluster [DBG] pgmap v14: 65 pgs: 65 active+clean; 106 MiB data, 448 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 381 op/s 2024-02-15T18:26:32.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:31.551287+0000 mon.smithi042 (mon.0) 651 : cluster [DBG] Standby manager daemon smithi191.gftdzo restarted 2024-02-15T18:26:32.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:31.551462+0000 mon.smithi042 (mon.0) 652 : cluster [DBG] Standby manager daemon smithi191.gftdzo started 2024-02-15T18:26:32.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:31.554477+0000 mon.smithi191 (mon.1) 65 : audit [DBG] from='mgr.? 172.21.15.191:0/2255132882' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/crt"}]: dispatch 2024-02-15T18:26:32.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:31.555342+0000 mon.smithi191 (mon.1) 66 : audit [DBG] from='mgr.? 172.21.15.191:0/2255132882' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-15T18:26:32.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:31.557991+0000 mon.smithi191 (mon.1) 67 : audit [DBG] from='mgr.? 172.21.15.191:0/2255132882' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi191.gftdzo/key"}]: dispatch 2024-02-15T18:26:32.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:31.558356+0000 mon.smithi191 (mon.1) 68 : audit [DBG] from='mgr.? 172.21.15.191:0/2255132882' entity='mgr.smithi191.gftdzo' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-15T18:26:32.067 INFO:tasks.workunit.client.1.smithi191.stdout:1/7: symlink d0/l2 0 2024-02-15T18:26:32.084 INFO:tasks.workunit.client.1.smithi191.stdout:1/8: creat d0/f3 x:0 0 0 2024-02-15T18:26:32.105 INFO:tasks.workunit.client.1.smithi191.stdout:1/9: creat d0/f4 x:0 0 0 2024-02-15T18:26:32.219 INFO:tasks.workunit.client.1.smithi191.stdout:4/4: dwrite f0 [0,4194304] 0 2024-02-15T18:26:32.328 INFO:tasks.workunit.client.1.smithi191.stdout:9/6: dread d0/f1 [0,4194304] 0 2024-02-15T18:26:32.445 INFO:tasks.workunit.client.1.smithi191.stdout:9/7: dread d0/f1 [0,4194304] 0 2024-02-15T18:26:32.499 INFO:tasks.workunit.client.1.smithi191.stdout:2/11: dwrite f1 [0,4194304] 0 2024-02-15T18:26:32.929 INFO:tasks.workunit.client.1.smithi191.stdout:1/10: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:26:32.974 INFO:tasks.workunit.client.1.smithi191.stdout:4/5: mkdir d2 0 2024-02-15T18:26:32.974 INFO:tasks.workunit.client.1.smithi191.stdout:4/6: chown f1 1167072 1 2024-02-15T18:26:33.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:31.759544+0000 mon.smithi042 (mon.0) 653 : cluster [DBG] mgrmap e29: smithi042.kfoihb(active, since 22s), standbys: smithi191.gftdzo 2024-02-15T18:26:33.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:31.759544+0000 mon.smithi042 (mon.0) 653 : cluster [DBG] mgrmap e29: smithi042.kfoihb(active, since 22s), standbys: smithi191.gftdzo 2024-02-15T18:26:33.317 INFO:tasks.workunit.client.1.smithi191.stdout:6/5: fsync f0 0 2024-02-15T18:26:33.323 INFO:tasks.workunit.client.1.smithi191.stdout:9/8: fsync d0/f1 0 2024-02-15T18:26:33.327 INFO:tasks.workunit.client.1.smithi191.stdout:7/15: getdents d0/d1/d3 0 2024-02-15T18:26:33.330 INFO:tasks.workunit.client.1.smithi191.stdout:8/16: unlink d3/f6 0 2024-02-15T18:26:33.330 INFO:tasks.workunit.client.1.smithi191.stdout:8/17: write - no filename 2024-02-15T18:26:33.369 INFO:tasks.workunit.client.1.smithi191.stdout:2/12: dwrite f1 [0,4194304] 0 2024-02-15T18:26:33.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:32.675792+0000 mgr.smithi042.kfoihb (mgr.24435) 16 : cluster [DBG] pgmap v15: 65 pgs: 65 active+clean; 120 MiB data, 506 MiB used, 536 GiB / 536 GiB avail; 3.0 MiB/s rd, 5.3 MiB/s wr, 399 op/s 2024-02-15T18:26:34.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:32.675792+0000 mgr.smithi042.kfoihb (mgr.24435) 16 : cluster [DBG] pgmap v15: 65 pgs: 65 active+clean; 120 MiB data, 506 MiB used, 536 GiB / 536 GiB avail; 3.0 MiB/s rd, 5.3 MiB/s wr, 399 op/s 2024-02-15T18:26:35.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:34.676174+0000 mgr.smithi042.kfoihb (mgr.24435) 17 : cluster [DBG] pgmap v16: 65 pgs: 65 active+clean; 120 MiB data, 506 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 383 op/s 2024-02-15T18:26:35.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:34.676174+0000 mgr.smithi042.kfoihb (mgr.24435) 17 : cluster [DBG] pgmap v16: 65 pgs: 65 active+clean; 120 MiB data, 506 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 383 op/s 2024-02-15T18:26:37.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.900+0000 7fbc7e064700 1 -- 172.21.15.42:0/3594481766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc780f6530 msgr2=0x7fbc780f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:37.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.900+0000 7fbc7e064700 1 --2- 172.21.15.42:0/3594481766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc780f6530 0x7fbc780f6950 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7fbc74004660 tx=0x7fbc74009b30).stop 2024-02-15T18:26:37.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.900+0000 7fbc7e064700 1 -- 172.21.15.42:0/3594481766 shutdown_connections 2024-02-15T18:26:37.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.900+0000 7fbc7e064700 1 --2- 172.21.15.42:0/3594481766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc780f77d0 0x7fbc780f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:37.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.900+0000 7fbc7e064700 1 --2- 172.21.15.42:0/3594481766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc780f6530 0x7fbc780f6950 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:37.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.900+0000 7fbc7e064700 1 -- 172.21.15.42:0/3594481766 >> 172.21.15.42:0/3594481766 conn(0x7fbc780f1c90 msgr2=0x7fbc780f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:37.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.901+0000 7fbc7e064700 1 -- 172.21.15.42:0/3594481766 shutdown_connections 2024-02-15T18:26:37.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.901+0000 7fbc7e064700 1 -- 172.21.15.42:0/3594481766 wait complete. 2024-02-15T18:26:37.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.901+0000 7fbc7e064700 1 Processor -- start 2024-02-15T18:26:37.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.901+0000 7fbc7e064700 1 -- start start 2024-02-15T18:26:37.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.901+0000 7fbc7e064700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc780f6530 0x7fbc7806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:37.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.901+0000 7fbc7e064700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc780f77d0 0x7fbc78069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:37.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.901+0000 7fbc7e064700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7806a380 con 0x7fbc780f6530 2024-02-15T18:26:37.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.901+0000 7fbc7e064700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7806a4c0 con 0x7fbc780f77d0 2024-02-15T18:26:37.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.902+0000 7fbc7d062700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc780f6530 0x7fbc7806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:37.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.902+0000 7fbc7d062700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc780f6530 0x7fbc7806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46976/0 (socket says 172.21.15.42:46976) 2024-02-15T18:26:37.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.902+0000 7fbc7d062700 1 -- 172.21.15.42:0/3555773511 learned_addr learned my addr 172.21.15.42:0/3555773511 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:26:37.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.902+0000 7fbc7c861700 1 --2- 172.21.15.42:0/3555773511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc780f77d0 0x7fbc78069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:37.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.902+0000 7fbc7d062700 1 -- 172.21.15.42:0/3555773511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc780f77d0 msgr2=0x7fbc78069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:37.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.902+0000 7fbc7d062700 1 --2- 172.21.15.42:0/3555773511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc780f77d0 0x7fbc78069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:37.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.902+0000 7fbc7d062700 1 -- 172.21.15.42:0/3555773511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc74005040 con 0x7fbc780f6530 2024-02-15T18:26:37.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.902+0000 7fbc7d062700 1 --2- 172.21.15.42:0/3555773511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc780f6530 0x7fbc7806b9a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fbc74002040 tx=0x7fbc7400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:37.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.902+0000 7fbc6dffb700 1 -- 172.21.15.42:0/3555773511 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc74019070 con 0x7fbc780f6530 2024-02-15T18:26:37.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.902+0000 7fbc6dffb700 1 -- 172.21.15.42:0/3555773511 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fbc74007e80 con 0x7fbc780f6530 2024-02-15T18:26:37.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.902+0000 7fbc7e064700 1 -- 172.21.15.42:0/3555773511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7806a740 con 0x7fbc780f6530 2024-02-15T18:26:37.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.902+0000 7fbc6dffb700 1 -- 172.21.15.42:0/3555773511 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc74014db0 con 0x7fbc780f6530 2024-02-15T18:26:37.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.902+0000 7fbc7e064700 1 -- 172.21.15.42:0/3555773511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7806aca0 con 0x7fbc780f6530 2024-02-15T18:26:37.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.903+0000 7fbc7e064700 1 -- 172.21.15.42:0/3555773511 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc78045ad0 con 0x7fbc780f6530 2024-02-15T18:26:37.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.904+0000 7fbc6dffb700 1 -- 172.21.15.42:0/3555773511 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) v1 ==== 77194+0+0 (secure 0 0 0) 0x7fbc74025070 con 0x7fbc780f6530 2024-02-15T18:26:37.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.904+0000 7fbc6dffb700 1 --2- 172.21.15.42:0/3555773511 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc64060fa0 0x7fbc64063460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:37.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.904+0000 7fbc6dffb700 1 -- 172.21.15.42:0/3555773511 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fbc7407d0e0 con 0x7fbc780f6530 2024-02-15T18:26:37.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.905+0000 7fbc7c861700 1 --2- 172.21.15.42:0/3555773511 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc64060fa0 0x7fbc64063460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:37.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.905+0000 7fbc7c861700 1 --2- 172.21.15.42:0/3555773511 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc64060fa0 0x7fbc64063460 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fbc68009fd0 tx=0x7fbc6800b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:37.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:37.909+0000 7fbc6dffb700 1 -- 172.21.15.42:0/3555773511 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7fbc7404d470 con 0x7fbc780f6530 2024-02-15T18:26:38.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:36.676562+0000 mgr.smithi042.kfoihb (mgr.24435) 18 : cluster [DBG] pgmap v17: 65 pgs: 65 active+clean; 120 MiB data, 506 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 383 op/s 2024-02-15T18:26:38.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:37.376263+0000 mon.smithi042 (mon.0) 654 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:38.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:37.377191+0000 mon.smithi042 (mon.0) 655 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:26:38.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:37.378127+0000 mon.smithi042 (mon.0) 656 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:26:38.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:37.382588+0000 mon.smithi042 (mon.0) 657 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:38.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:37.389331+0000 mon.smithi042 (mon.0) 658 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:38.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:37.392316+0000 mon.smithi042 (mon.0) 659 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:26:38.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:37.397030+0000 mon.smithi042 (mon.0) 660 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:38.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:37.397909+0000 mon.smithi042 (mon.0) 661 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi042.kfoihb"}]: dispatch 2024-02-15T18:26:38.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:37.401013+0000 mon.smithi042 (mon.0) 662 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi042.kfoihb"}]': finished 2024-02-15T18:26:38.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:37.401704+0000 mon.smithi042 (mon.0) 663 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi191.gftdzo"}]: dispatch 2024-02-15T18:26:38.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:37.404626+0000 mon.smithi042 (mon.0) 664 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi191.gftdzo"}]': finished 2024-02-15T18:26:38.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:37.405766+0000 mon.smithi042 (mon.0) 665 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-15T18:26:38.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.096+0000 7fbc7e064700 1 -- 172.21.15.42:0/3555773511 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc780fef80 con 0x7fbc64060fa0 2024-02-15T18:26:38.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.098+0000 7fbc6dffb700 1 -- 172.21.15.42:0/3555773511 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fbc780fef80 con 0x7fbc64060fa0 2024-02-15T18:26:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.101+0000 7fbc7e064700 1 -- 172.21.15.42:0/3555773511 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc64060fa0 msgr2=0x7fbc64063460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.101+0000 7fbc7e064700 1 --2- 172.21.15.42:0/3555773511 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc64060fa0 0x7fbc64063460 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fbc68009fd0 tx=0x7fbc6800b040).stop 2024-02-15T18:26:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.101+0000 7fbc7e064700 1 -- 172.21.15.42:0/3555773511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc780f6530 msgr2=0x7fbc7806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.101+0000 7fbc7e064700 1 --2- 172.21.15.42:0/3555773511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc780f6530 0x7fbc7806b9a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fbc74002040 tx=0x7fbc7400b040).stop 2024-02-15T18:26:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.101+0000 7fbc7e064700 1 -- 172.21.15.42:0/3555773511 shutdown_connections 2024-02-15T18:26:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.101+0000 7fbc7e064700 1 --2- 172.21.15.42:0/3555773511 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc64060fa0 0x7fbc64063460 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.101+0000 7fbc7e064700 1 --2- 172.21.15.42:0/3555773511 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc780f6530 0x7fbc7806b9a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.101+0000 7fbc7e064700 1 --2- 172.21.15.42:0/3555773511 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc780f77d0 0x7fbc78069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.101+0000 7fbc7e064700 1 -- 172.21.15.42:0/3555773511 >> 172.21.15.42:0/3555773511 conn(0x7fbc780f1c90 msgr2=0x7fbc780fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:38.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.101+0000 7fbc7e064700 1 -- 172.21.15.42:0/3555773511 shutdown_connections 2024-02-15T18:26:38.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.101+0000 7fbc7e064700 1 -- 172.21.15.42:0/3555773511 wait complete. 2024-02-15T18:26:38.108 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:26:38.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:36.676562+0000 mgr.smithi042.kfoihb (mgr.24435) 18 : cluster [DBG] pgmap v17: 65 pgs: 65 active+clean; 120 MiB data, 506 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 383 op/s 2024-02-15T18:26:38.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:37.376263+0000 mon.smithi042 (mon.0) 654 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:38.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:37.377191+0000 mon.smithi042 (mon.0) 655 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:26:38.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:37.378127+0000 mon.smithi042 (mon.0) 656 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:26:38.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:37.382588+0000 mon.smithi042 (mon.0) 657 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:38.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:37.389331+0000 mon.smithi042 (mon.0) 658 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:38.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:37.392316+0000 mon.smithi042 (mon.0) 659 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:26:38.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:37.397030+0000 mon.smithi042 (mon.0) 660 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:38.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:37.397909+0000 mon.smithi042 (mon.0) 661 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi042.kfoihb"}]: dispatch 2024-02-15T18:26:38.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:37.401013+0000 mon.smithi042 (mon.0) 662 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi042.kfoihb"}]': finished 2024-02-15T18:26:38.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:37.401704+0000 mon.smithi042 (mon.0) 663 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi191.gftdzo"}]: dispatch 2024-02-15T18:26:38.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:37.404626+0000 mon.smithi042 (mon.0) 664 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi191.gftdzo"}]': finished 2024-02-15T18:26:38.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:37.405766+0000 mon.smithi042 (mon.0) 665 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-15T18:26:38.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.204+0000 7fe1b29e1700 1 -- 172.21.15.42:0/884637871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1ac0f66e0 msgr2=0x7fe1ac0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:38.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.204+0000 7fe1b29e1700 1 --2- 172.21.15.42:0/884637871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1ac0f66e0 0x7fe1ac0f6b00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fe19c004660 tx=0x7fe19c009b30).stop 2024-02-15T18:26:38.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.204+0000 7fe1b29e1700 1 -- 172.21.15.42:0/884637871 shutdown_connections 2024-02-15T18:26:38.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.204+0000 7fe1b29e1700 1 --2- 172.21.15.42:0/884637871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1ac0f7980 0x7fe1ac0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.204+0000 7fe1b29e1700 1 --2- 172.21.15.42:0/884637871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1ac0f66e0 0x7fe1ac0f6b00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.204+0000 7fe1b29e1700 1 -- 172.21.15.42:0/884637871 >> 172.21.15.42:0/884637871 conn(0x7fe1ac0f1e40 msgr2=0x7fe1ac0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:38.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.204+0000 7fe1b29e1700 1 -- 172.21.15.42:0/884637871 shutdown_connections 2024-02-15T18:26:38.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.204+0000 7fe1b29e1700 1 -- 172.21.15.42:0/884637871 wait complete. 2024-02-15T18:26:38.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.204+0000 7fe1b29e1700 1 Processor -- start 2024-02-15T18:26:38.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.204+0000 7fe1b29e1700 1 -- start start 2024-02-15T18:26:38.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1b29e1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1ac0f66e0 0x7fe1ac107030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:38.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1b29e1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1ac0f7980 0x7fe1ac107520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:38.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1b29e1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac107ad0 con 0x7fe1ac0f66e0 2024-02-15T18:26:38.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1b29e1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac107c10 con 0x7fe1ac0f7980 2024-02-15T18:26:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1abfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1ac0f66e0 0x7fe1ac107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1abfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1ac0f66e0 0x7fe1ac107030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46984/0 (socket says 172.21.15.42:46984) 2024-02-15T18:26:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1abfff700 1 -- 172.21.15.42:0/2215500036 learned_addr learned my addr 172.21.15.42:0/2215500036 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:26:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1ab7fe700 1 --2- 172.21.15.42:0/2215500036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1ac0f7980 0x7fe1ac107520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1abfff700 1 -- 172.21.15.42:0/2215500036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1ac0f7980 msgr2=0x7fe1ac107520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1abfff700 1 --2- 172.21.15.42:0/2215500036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1ac0f7980 0x7fe1ac107520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1abfff700 1 -- 172.21.15.42:0/2215500036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19c005040 con 0x7fe1ac0f66e0 2024-02-15T18:26:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1ab7fe700 1 --2- 172.21.15.42:0/2215500036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1ac0f7980 0x7fe1ac107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:26:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1abfff700 1 --2- 172.21.15.42:0/2215500036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1ac0f66e0 0x7fe1ac107030 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fe19c002040 tx=0x7fe19c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1a8ff9700 1 -- 172.21.15.42:0/2215500036 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe19c019070 con 0x7fe1ac0f66e0 2024-02-15T18:26:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1a8ff9700 1 -- 172.21.15.42:0/2215500036 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fe19c007e80 con 0x7fe1ac0f66e0 2024-02-15T18:26:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1b29e1700 1 -- 172.21.15.42:0/2215500036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ac108550 con 0x7fe1ac0f66e0 2024-02-15T18:26:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1a8ff9700 1 -- 172.21.15.42:0/2215500036 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe19c014db0 con 0x7fe1ac0f66e0 2024-02-15T18:26:38.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.205+0000 7fe1b29e1700 1 -- 172.21.15.42:0/2215500036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ac108ab0 con 0x7fe1ac0f66e0 2024-02-15T18:26:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.206+0000 7fe1b29e1700 1 -- 172.21.15.42:0/2215500036 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ac045ad0 con 0x7fe1ac0f66e0 2024-02-15T18:26:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.207+0000 7fe1a8ff9700 1 -- 172.21.15.42:0/2215500036 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) v1 ==== 77194+0+0 (secure 0 0 0) 0x7fe19c025070 con 0x7fe1ac0f66e0 2024-02-15T18:26:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.207+0000 7fe1a8ff9700 1 --2- 172.21.15.42:0/2215500036 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe194060fb0 0x7fe194063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.207+0000 7fe1a8ff9700 1 -- 172.21.15.42:0/2215500036 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe19c07cfb0 con 0x7fe1ac0f66e0 2024-02-15T18:26:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.207+0000 7fe1ab7fe700 1 --2- 172.21.15.42:0/2215500036 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe194060fb0 0x7fe194063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:38.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.208+0000 7fe1ab7fe700 1 --2- 172.21.15.42:0/2215500036 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe194060fb0 0x7fe194063470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fe1a0009fd0 tx=0x7fe1a000b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:38.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.212+0000 7fe1a8ff9700 1 -- 172.21.15.42:0/2215500036 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7fe19c050df0 con 0x7fe1ac0f66e0 2024-02-15T18:26:38.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.399+0000 7fe1b29e1700 1 -- 172.21.15.42:0/2215500036 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1ac0ff0c0 con 0x7fe194060fb0 2024-02-15T18:26:38.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.400+0000 7fe1a8ff9700 1 -- 172.21.15.42:0/2215500036 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fe1ac0ff0c0 con 0x7fe194060fb0 2024-02-15T18:26:38.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.402+0000 7fe1b29e1700 1 -- 172.21.15.42:0/2215500036 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe194060fb0 msgr2=0x7fe194063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:38.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.402+0000 7fe1b29e1700 1 --2- 172.21.15.42:0/2215500036 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe194060fb0 0x7fe194063470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fe1a0009fd0 tx=0x7fe1a000b040).stop 2024-02-15T18:26:38.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.402+0000 7fe1b29e1700 1 -- 172.21.15.42:0/2215500036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1ac0f66e0 msgr2=0x7fe1ac107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:38.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.402+0000 7fe1b29e1700 1 --2- 172.21.15.42:0/2215500036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1ac0f66e0 0x7fe1ac107030 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7fe19c002040 tx=0x7fe19c00b040).stop 2024-02-15T18:26:38.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.402+0000 7fe1b29e1700 1 -- 172.21.15.42:0/2215500036 shutdown_connections 2024-02-15T18:26:38.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.402+0000 7fe1b29e1700 1 --2- 172.21.15.42:0/2215500036 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe194060fb0 0x7fe194063470 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.402+0000 7fe1b29e1700 1 --2- 172.21.15.42:0/2215500036 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1ac0f66e0 0x7fe1ac107030 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.402+0000 7fe1b29e1700 1 --2- 172.21.15.42:0/2215500036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe1ac0f7980 0x7fe1ac107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.402+0000 7fe1b29e1700 1 -- 172.21.15.42:0/2215500036 >> 172.21.15.42:0/2215500036 conn(0x7fe1ac0f1e40 msgr2=0x7fe1ac0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:38.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.402+0000 7fe1b29e1700 1 -- 172.21.15.42:0/2215500036 shutdown_connections 2024-02-15T18:26:38.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.402+0000 7fe1b29e1700 1 -- 172.21.15.42:0/2215500036 wait complete. 2024-02-15T18:26:38.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.504+0000 7f6b9d075700 1 -- 172.21.15.42:0/915394141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b980f6860 msgr2=0x7f6b980f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:38.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.504+0000 7f6b9d075700 1 --2- 172.21.15.42:0/915394141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b980f6860 0x7f6b980f6cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f6b80004660 tx=0x7f6b80009b30).stop 2024-02-15T18:26:38.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.504+0000 7f6b9d075700 1 -- 172.21.15.42:0/915394141 shutdown_connections 2024-02-15T18:26:38.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.504+0000 7f6b9d075700 1 --2- 172.21.15.42:0/915394141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b980f6860 0x7f6b980f6cc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.504+0000 7f6b9d075700 1 --2- 172.21.15.42:0/915394141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b980f55c0 0x7f6b980f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.504+0000 7f6b9d075700 1 -- 172.21.15.42:0/915394141 >> 172.21.15.42:0/915394141 conn(0x7f6b980f0d40 msgr2=0x7f6b980f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:38.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.505+0000 7f6b9d075700 1 -- 172.21.15.42:0/915394141 shutdown_connections 2024-02-15T18:26:38.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.505+0000 7f6b9d075700 1 -- 172.21.15.42:0/915394141 wait complete. 2024-02-15T18:26:38.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.505+0000 7f6b9d075700 1 Processor -- start 2024-02-15T18:26:38.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.505+0000 7f6b9d075700 1 -- start start 2024-02-15T18:26:38.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.505+0000 7f6b9d075700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b980f55c0 0x7f6b98105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:38.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b9d075700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b980f6860 0x7f6b981063a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:38.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b9d075700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b98106950 con 0x7f6b980f6860 2024-02-15T18:26:38.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b9d075700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b98106a90 con 0x7f6b980f55c0 2024-02-15T18:26:38.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b9659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b980f6860 0x7f6b981063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:38.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b9659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b980f6860 0x7f6b981063a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47000/0 (socket says 172.21.15.42:47000) 2024-02-15T18:26:38.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b9659c700 1 -- 172.21.15.42:0/193585674 learned_addr learned my addr 172.21.15.42:0/193585674 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:26:38.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b96d9d700 1 --2- 172.21.15.42:0/193585674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b980f55c0 0x7f6b98105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:38.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b9659c700 1 -- 172.21.15.42:0/193585674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b980f55c0 msgr2=0x7f6b98105eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:38.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b9659c700 1 --2- 172.21.15.42:0/193585674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b980f55c0 0x7f6b98105eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b9659c700 1 -- 172.21.15.42:0/193585674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b80005040 con 0x7f6b980f6860 2024-02-15T18:26:38.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b9659c700 1 --2- 172.21.15.42:0/193585674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b980f6860 0x7f6b981063a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f6b80000c00 tx=0x7f6b8000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:38.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b8f7fe700 1 -- 172.21.15.42:0/193585674 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b80019070 con 0x7f6b980f6860 2024-02-15T18:26:38.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b9d075700 1 -- 172.21.15.42:0/193585674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b981073d0 con 0x7f6b980f6860 2024-02-15T18:26:38.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b8f7fe700 1 -- 172.21.15.42:0/193585674 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f6b80007e80 con 0x7f6b980f6860 2024-02-15T18:26:38.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b8f7fe700 1 -- 172.21.15.42:0/193585674 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b80014db0 con 0x7f6b980f6860 2024-02-15T18:26:38.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.506+0000 7f6b9d075700 1 -- 172.21.15.42:0/193585674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b98107930 con 0x7f6b980f6860 2024-02-15T18:26:38.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.507+0000 7f6b9d075700 1 -- 172.21.15.42:0/193585674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b98045ad0 con 0x7f6b980f6860 2024-02-15T18:26:38.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.508+0000 7f6b8f7fe700 1 -- 172.21.15.42:0/193585674 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) v1 ==== 77194+0+0 (secure 0 0 0) 0x7f6b80025070 con 0x7f6b980f6860 2024-02-15T18:26:38.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.508+0000 7f6b8f7fe700 1 --2- 172.21.15.42:0/193585674 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6b84060f50 0x7f6b84063410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:38.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.508+0000 7f6b8f7fe700 1 -- 172.21.15.42:0/193585674 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f6b8007d100 con 0x7f6b980f6860 2024-02-15T18:26:38.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.508+0000 7f6b96d9d700 1 --2- 172.21.15.42:0/193585674 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6b84060f50 0x7f6b84063410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:38.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.508+0000 7f6b96d9d700 1 --2- 172.21.15.42:0/193585674 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6b84060f50 0x7f6b84063410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f6b88004fd0 tx=0x7f6b8800b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:38.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.512+0000 7f6b8f7fe700 1 -- 172.21.15.42:0/193585674 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7f6b80050f30 con 0x7f6b980f6860 2024-02-15T18:26:38.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.696+0000 7f6b9d075700 1 -- 172.21.15.42:0/193585674 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b980fdfa0 con 0x7f6b84060f50 2024-02-15T18:26:38.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.705+0000 7f6b8f7fe700 1 -- 172.21.15.42:0/193585674 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f6b980fdfa0 con 0x7f6b84060f50 2024-02-15T18:26:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:26:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (6m) 21s ago 8m 17.6M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:26:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (8m) 21s ago 8m 6400k - 16.2.4 8d91d370c2b8 94d74c33c758 2024-02-15T18:26:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (7m) 1s ago 7m 6405k - 16.2.4 8d91d370c2b8 ff0f56f9475a 2024-02-15T18:26:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (6m) 21s ago 7m 34.1M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:26:38.702 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (2m) 21s ago 2m 13.2M - 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (2m) 21s ago 2m 15.9M - 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (2m) 1s ago 2m 75.1M - 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (2m) 1s ago 2m 143M - 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (37s) 21s ago 9m 410M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (14s) 1s ago 7m 40.5M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (9m) 21s ago 9m 51.6M 2048M 16.2.4 8d91d370c2b8 9dcbc5c49e05 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (7m) 1s ago 7m 49.5M 2048M 16.2.4 8d91d370c2b8 c3dd4902e2a2 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (7m) 21s ago 7m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (6m) 1s ago 6m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (6m) 21s ago 6m 42.5M 4096M 16.2.4 8d91d370c2b8 973745b31d5a 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (5m) 21s ago 5m 47.0M 4096M 16.2.4 8d91d370c2b8 2b3ae86111e8 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (5m) 21s ago 5m 41.0M 4096M 16.2.4 8d91d370c2b8 3706071e4698 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (5m) 1s ago 5m 79.8M 4096M 16.2.4 8d91d370c2b8 e6fe8e77c6fe 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (4m) 1s ago 4m 80.6M 4096M 16.2.4 8d91d370c2b8 24582012e234 2024-02-15T18:26:38.703 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (4m) 1s ago 4m 74.7M 4096M 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:26:38.704 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (44s) 21s ago 7m 42.8M - 2.18.1 de242295e225 63b6dfb825ed 2024-02-15T18:26:38.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.707+0000 7f6b9d075700 1 -- 172.21.15.42:0/193585674 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6b84060f50 msgr2=0x7f6b84063410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:38.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.707+0000 7f6b9d075700 1 --2- 172.21.15.42:0/193585674 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6b84060f50 0x7f6b84063410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f6b88004fd0 tx=0x7f6b8800b040).stop 2024-02-15T18:26:38.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.707+0000 7f6b9d075700 1 -- 172.21.15.42:0/193585674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b980f6860 msgr2=0x7f6b981063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:38.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.707+0000 7f6b9d075700 1 --2- 172.21.15.42:0/193585674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b980f6860 0x7f6b981063a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f6b80000c00 tx=0x7f6b8000b040).stop 2024-02-15T18:26:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.707+0000 7f6b9d075700 1 -- 172.21.15.42:0/193585674 shutdown_connections 2024-02-15T18:26:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.707+0000 7f6b9d075700 1 --2- 172.21.15.42:0/193585674 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6b84060f50 0x7f6b84063410 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.707+0000 7f6b9d075700 1 --2- 172.21.15.42:0/193585674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b980f55c0 0x7f6b98105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.707+0000 7f6b9d075700 1 --2- 172.21.15.42:0/193585674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b980f6860 0x7f6b981063a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.707+0000 7f6b9d075700 1 -- 172.21.15.42:0/193585674 >> 172.21.15.42:0/193585674 conn(0x7f6b980f0d40 msgr2=0x7f6b980fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.707+0000 7f6b9d075700 1 -- 172.21.15.42:0/193585674 shutdown_connections 2024-02-15T18:26:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.707+0000 7f6b9d075700 1 -- 172.21.15.42:0/193585674 wait complete. 2024-02-15T18:26:38.791 INFO:tasks.workunit.client.1.smithi191.stdout:1/11: creat d0/f5 x:0 0 0 2024-02-15T18:26:38.791 INFO:tasks.workunit.client.1.smithi191.stdout:1/12: write d0/f4 [585148,125908] 0 2024-02-15T18:26:38.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.810+0000 7ff3670fb700 1 -- 172.21.15.42:0/785511061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3600f66e0 msgr2=0x7ff3600f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:38.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.810+0000 7ff3670fb700 1 --2- 172.21.15.42:0/785511061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3600f66e0 0x7ff3600f6b00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7ff354004660 tx=0x7ff354009b30).stop 2024-02-15T18:26:38.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.810+0000 7ff3670fb700 1 -- 172.21.15.42:0/785511061 shutdown_connections 2024-02-15T18:26:38.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.810+0000 7ff3670fb700 1 --2- 172.21.15.42:0/785511061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3600f7980 0x7ff3600f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.810+0000 7ff3670fb700 1 --2- 172.21.15.42:0/785511061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3600f66e0 0x7ff3600f6b00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.810+0000 7ff3670fb700 1 -- 172.21.15.42:0/785511061 >> 172.21.15.42:0/785511061 conn(0x7ff3600f1e40 msgr2=0x7ff3600f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:38.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.810+0000 7ff3670fb700 1 -- 172.21.15.42:0/785511061 shutdown_connections 2024-02-15T18:26:38.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.810+0000 7ff3670fb700 1 -- 172.21.15.42:0/785511061 wait complete. 2024-02-15T18:26:38.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.810+0000 7ff3670fb700 1 Processor -- start 2024-02-15T18:26:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.810+0000 7ff3670fb700 1 -- start start 2024-02-15T18:26:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff3670fb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3600f66e0 0x7ff360107030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff3670fb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3600f7980 0x7ff360107520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff3670fb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff360107ad0 con 0x7ff3600f7980 2024-02-15T18:26:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff3670fb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff360107c10 con 0x7ff3600f66e0 2024-02-15T18:26:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff35ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3600f7980 0x7ff360107520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff35ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3600f7980 0x7ff360107520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47016/0 (socket says 172.21.15.42:47016) 2024-02-15T18:26:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff35ffff700 1 -- 172.21.15.42:0/3915843573 learned_addr learned my addr 172.21.15.42:0/3915843573 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:26:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff364e97700 1 --2- 172.21.15.42:0/3915843573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3600f66e0 0x7ff360107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff35ffff700 1 -- 172.21.15.42:0/3915843573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3600f66e0 msgr2=0x7ff360107030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff35ffff700 1 --2- 172.21.15.42:0/3915843573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3600f66e0 0x7ff360107030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff35ffff700 1 -- 172.21.15.42:0/3915843573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff354005040 con 0x7ff3600f7980 2024-02-15T18:26:38.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff364e97700 1 --2- 172.21.15.42:0/3915843573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3600f66e0 0x7ff360107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:26:38.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff35ffff700 1 --2- 172.21.15.42:0/3915843573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3600f7980 0x7ff360107520 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7ff34c00e7f0 tx=0x7ff34c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:38.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff35d7fa700 1 -- 172.21.15.42:0/3915843573 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff34c0092e0 con 0x7ff3600f7980 2024-02-15T18:26:38.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff35d7fa700 1 -- 172.21.15.42:0/3915843573 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7ff34c00f040 con 0x7ff3600f7980 2024-02-15T18:26:38.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff3670fb700 1 -- 172.21.15.42:0/3915843573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3601085b0 con 0x7ff3600f7980 2024-02-15T18:26:38.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff35d7fa700 1 -- 172.21.15.42:0/3915843573 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff34c014850 con 0x7ff3600f7980 2024-02-15T18:26:38.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.811+0000 7ff3670fb700 1 -- 172.21.15.42:0/3915843573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff360108b70 con 0x7ff3600f7980 2024-02-15T18:26:38.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.812+0000 7ff3670fb700 1 -- 172.21.15.42:0/3915843573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff360045ad0 con 0x7ff3600f7980 2024-02-15T18:26:38.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.813+0000 7ff35d7fa700 1 -- 172.21.15.42:0/3915843573 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) v1 ==== 77194+0+0 (secure 0 0 0) 0x7ff34c01b030 con 0x7ff3600f7980 2024-02-15T18:26:38.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.813+0000 7ff35d7fa700 1 --2- 172.21.15.42:0/3915843573 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff350060fb0 0x7ff350063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:38.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.813+0000 7ff35d7fa700 1 -- 172.21.15.42:0/3915843573 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff34c07be90 con 0x7ff3600f7980 2024-02-15T18:26:38.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.813+0000 7ff364e97700 1 --2- 172.21.15.42:0/3915843573 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff350060fb0 0x7ff350063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:38.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.814+0000 7ff364e97700 1 --2- 172.21.15.42:0/3915843573 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff350060fb0 0x7ff350063470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff354005010 tx=0x7ff354016040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:38.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:38.817+0000 7ff35d7fa700 1 -- 172.21.15.42:0/3915843573 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7ff34c04fd60 con 0x7ff3600f7980 2024-02-15T18:26:38.997 INFO:tasks.workunit.client.1.smithi191.stdout:4/7: creat d2/f3 x:0 0 0 2024-02-15T18:26:39.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.061+0000 7ff3670fb700 1 -- 172.21.15.42:0/3915843573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff360027a00 con 0x7ff3600f7980 2024-02-15T18:26:39.059 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.061+0000 7ff35d7fa700 1 -- 172.21.15.42:0/3915843573 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+697 (secure 0 0 0) 0x7ff34c019070 con 0x7ff3600f7980 2024-02-15T18:26:39.059 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:26:39.059 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:26:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:26:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:26:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:26:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:26:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:26:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:26:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:26:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:26:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:26:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:26:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:26:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:26:39.060 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2, 2024-02-15T18:26:39.063 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 12 2024-02-15T18:26:39.063 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:26:39.063 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:26:39.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:26:37.378917+0000 mgr.smithi042.kfoihb (mgr.24435) 19 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:26:39.063 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:26:37.393173+0000 mgr.smithi042.kfoihb (mgr.24435) 20 : cephadm [INF] Upgrade: Setting container_image for all mgr 2024-02-15T18:26:39.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.063+0000 7ff3670fb700 1 -- 172.21.15.42:0/3915843573 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff350060fb0 msgr2=0x7ff350063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:39.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.063+0000 7ff3670fb700 1 --2- 172.21.15.42:0/3915843573 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff350060fb0 0x7ff350063470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff354005010 tx=0x7ff354016040).stop 2024-02-15T18:26:39.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.063+0000 7ff3670fb700 1 -- 172.21.15.42:0/3915843573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3600f7980 msgr2=0x7ff360107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:39.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.063+0000 7ff3670fb700 1 --2- 172.21.15.42:0/3915843573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3600f7980 0x7ff360107520 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7ff34c00e7f0 tx=0x7ff34c00b040).stop 2024-02-15T18:26:39.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.063+0000 7ff3670fb700 1 -- 172.21.15.42:0/3915843573 shutdown_connections 2024-02-15T18:26:39.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.063+0000 7ff3670fb700 1 --2- 172.21.15.42:0/3915843573 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff350060fb0 0x7ff350063470 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.063+0000 7ff3670fb700 1 --2- 172.21.15.42:0/3915843573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3600f66e0 0x7ff360107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.063+0000 7ff3670fb700 1 --2- 172.21.15.42:0/3915843573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff3600f7980 0x7ff360107520 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.063+0000 7ff3670fb700 1 -- 172.21.15.42:0/3915843573 >> 172.21.15.42:0/3915843573 conn(0x7ff3600f1e40 msgr2=0x7ff3600fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:39.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.063+0000 7ff3670fb700 1 -- 172.21.15.42:0/3915843573 shutdown_connections 2024-02-15T18:26:39.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.063+0000 7ff3670fb700 1 -- 172.21.15.42:0/3915843573 wait complete. 2024-02-15T18:26:39.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.168+0000 7fd134900700 1 -- 172.21.15.42:0/783198707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1300f6bc0 msgr2=0x7fd1300f6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:39.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.168+0000 7fd134900700 1 --2- 172.21.15.42:0/783198707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1300f6bc0 0x7fd1300f6fe0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fd120004660 tx=0x7fd120009b30).stop 2024-02-15T18:26:39.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.168+0000 7fd134900700 1 -- 172.21.15.42:0/783198707 shutdown_connections 2024-02-15T18:26:39.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.168+0000 7fd134900700 1 --2- 172.21.15.42:0/783198707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1300f7560 0x7fd1300f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.168+0000 7fd134900700 1 --2- 172.21.15.42:0/783198707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1300f6bc0 0x7fd1300f6fe0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.168+0000 7fd134900700 1 -- 172.21.15.42:0/783198707 >> 172.21.15.42:0/783198707 conn(0x7fd1300f04c0 msgr2=0x7fd1300f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.169+0000 7fd134900700 1 -- 172.21.15.42:0/783198707 shutdown_connections 2024-02-15T18:26:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.169+0000 7fd134900700 1 -- 172.21.15.42:0/783198707 wait complete. 2024-02-15T18:26:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.169+0000 7fd134900700 1 Processor -- start 2024-02-15T18:26:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.169+0000 7fd134900700 1 -- start start 2024-02-15T18:26:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.169+0000 7fd134900700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1300f6bc0 0x7fd130109dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:39.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.169+0000 7fd134900700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1300f7560 0x7fd13010c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.169+0000 7fd134900700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1300f5640 con 0x7fd1300f6bc0 2024-02-15T18:26:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.169+0000 7fd134900700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1300f5780 con 0x7fd1300f7560 2024-02-15T18:26:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.170+0000 7fd12e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1300f6bc0 0x7fd130109dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:39.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.170+0000 7fd12e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1300f6bc0 0x7fd130109dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47028/0 (socket says 172.21.15.42:47028) 2024-02-15T18:26:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.170+0000 7fd12dd9b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1300f7560 0x7fd13010c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.170+0000 7fd12e59c700 1 -- 172.21.15.42:0/1122273321 learned_addr learned my addr 172.21.15.42:0/1122273321 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:26:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.170+0000 7fd12e59c700 1 -- 172.21.15.42:0/1122273321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1300f7560 msgr2=0x7fd13010c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.170+0000 7fd12e59c700 1 --2- 172.21.15.42:0/1122273321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1300f7560 0x7fd13010c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.170+0000 7fd12e59c700 1 -- 172.21.15.42:0/1122273321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd120005040 con 0x7fd1300f6bc0 2024-02-15T18:26:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.170+0000 7fd12dd9b700 1 --2- 172.21.15.42:0/1122273321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1300f7560 0x7fd13010c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:26:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.170+0000 7fd12e59c700 1 --2- 172.21.15.42:0/1122273321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1300f6bc0 0x7fd130109dd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fd1200043d0 tx=0x7fd12000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.170+0000 7fd11effd700 1 -- 172.21.15.42:0/1122273321 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd120019070 con 0x7fd1300f6bc0 2024-02-15T18:26:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.170+0000 7fd11effd700 1 -- 172.21.15.42:0/1122273321 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fd120007e80 con 0x7fd1300f6bc0 2024-02-15T18:26:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.170+0000 7fd11effd700 1 -- 172.21.15.42:0/1122273321 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd120014cf0 con 0x7fd1300f6bc0 2024-02-15T18:26:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.170+0000 7fd134900700 1 -- 172.21.15.42:0/1122273321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13010c820 con 0x7fd1300f6bc0 2024-02-15T18:26:39.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.170+0000 7fd134900700 1 -- 172.21.15.42:0/1122273321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13010cd50 con 0x7fd1300f6bc0 2024-02-15T18:26:39.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.171+0000 7fd134900700 1 -- 172.21.15.42:0/1122273321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1300f4150 con 0x7fd1300f6bc0 2024-02-15T18:26:39.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.172+0000 7fd11effd700 1 -- 172.21.15.42:0/1122273321 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) v1 ==== 77194+0+0 (secure 0 0 0) 0x7fd120025070 con 0x7fd1300f6bc0 2024-02-15T18:26:39.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.172+0000 7fd11effd700 1 --2- 172.21.15.42:0/1122273321 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd118060fb0 0x7fd118063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:39.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.172+0000 7fd11effd700 1 -- 172.21.15.42:0/1122273321 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd12007d040 con 0x7fd1300f6bc0 2024-02-15T18:26:39.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.172+0000 7fd12dd9b700 1 --2- 172.21.15.42:0/1122273321 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd118060fb0 0x7fd118063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:39.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.172+0000 7fd12dd9b700 1 --2- 172.21.15.42:0/1122273321 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd118060fb0 0x7fd118063470 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fd124009fd0 tx=0x7fd12400b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:39.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.176+0000 7fd11effd700 1 -- 172.21.15.42:0/1122273321 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7fd12004d4b0 con 0x7fd1300f6bc0 2024-02-15T18:26:39.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:26:37.378917+0000 mgr.smithi042.kfoihb (mgr.24435) 19 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:26:39.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:26:37.393173+0000 mgr.smithi042.kfoihb (mgr.24435) 20 : cephadm [INF] Upgrade: Setting container_image for all mgr 2024-02-15T18:26:39.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.403+0000 7fd134900700 1 -- 172.21.15.42:0/1122273321 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd1300fe000 con 0x7fd1300f6bc0 2024-02-15T18:26:39.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.404+0000 7fd11effd700 1 -- 172.21.15.42:0/1122273321 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1626 (secure 0 0 0) 0x7fd120023070 con 0x7fd1300f6bc0 2024-02-15T18:26:39.401 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:26:39.401 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:26:39.401 INFO:teuthology.orchestra.run.smithi042.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:26:39.401 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:26:39.401 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:26:39.401 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:26:39.402 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:26:39.403 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000]] 2024-02-15T18:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509]] 2024-02-15T18:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366]] 2024-02-15T18:26:39.404 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796]] 2024-02-15T18:26:39.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.406+0000 7fd134900700 1 -- 172.21.15.42:0/1122273321 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd118060fb0 msgr2=0x7fd118063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:39.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.406+0000 7fd134900700 1 --2- 172.21.15.42:0/1122273321 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd118060fb0 0x7fd118063470 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fd124009fd0 tx=0x7fd12400b040).stop 2024-02-15T18:26:39.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.406+0000 7fd134900700 1 -- 172.21.15.42:0/1122273321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1300f6bc0 msgr2=0x7fd130109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:39.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.406+0000 7fd134900700 1 --2- 172.21.15.42:0/1122273321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1300f6bc0 0x7fd130109dd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fd1200043d0 tx=0x7fd12000b040).stop 2024-02-15T18:26:39.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.406+0000 7fd134900700 1 -- 172.21.15.42:0/1122273321 shutdown_connections 2024-02-15T18:26:39.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.406+0000 7fd134900700 1 --2- 172.21.15.42:0/1122273321 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd118060fb0 0x7fd118063470 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.406+0000 7fd134900700 1 --2- 172.21.15.42:0/1122273321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1300f6bc0 0x7fd130109dd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.406+0000 7fd134900700 1 --2- 172.21.15.42:0/1122273321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1300f7560 0x7fd13010c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.406+0000 7fd134900700 1 -- 172.21.15.42:0/1122273321 >> 172.21.15.42:0/1122273321 conn(0x7fd1300f04c0 msgr2=0x7fd1300fa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:39.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.406+0000 7fd134900700 1 -- 172.21.15.42:0/1122273321 shutdown_connections 2024-02-15T18:26:39.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.406+0000 7fd134900700 1 -- 172.21.15.42:0/1122273321 wait complete. 2024-02-15T18:26:39.406 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:26:39.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.514+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/3881843694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc180f55c0 msgr2=0x7fbc180f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:39.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.514+0000 7fbc1f6ed700 1 --2- 172.21.15.42:0/3881843694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc180f55c0 0x7fbc180f59e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7fbc100099e0 tx=0x7fbc10004d10).stop 2024-02-15T18:26:39.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.514+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/3881843694 shutdown_connections 2024-02-15T18:26:39.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.514+0000 7fbc1f6ed700 1 --2- 172.21.15.42:0/3881843694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc180f6860 0x7fbc180f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.514+0000 7fbc1f6ed700 1 --2- 172.21.15.42:0/3881843694 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc180f55c0 0x7fbc180f59e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.514+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/3881843694 >> 172.21.15.42:0/3881843694 conn(0x7fbc180f0d40 msgr2=0x7fbc180f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:39.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.514+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/3881843694 shutdown_connections 2024-02-15T18:26:39.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.514+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/3881843694 wait complete. 2024-02-15T18:26:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.515+0000 7fbc1f6ed700 1 Processor -- start 2024-02-15T18:26:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.515+0000 7fbc1f6ed700 1 -- start start 2024-02-15T18:26:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.515+0000 7fbc1f6ed700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc180f55c0 0x7fbc180f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.515+0000 7fbc1f6ed700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc180f6860 0x7fbc180f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.515+0000 7fbc1f6ed700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc180f8100 con 0x7fbc180f55c0 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.515+0000 7fbc1f6ed700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc180f8240 con 0x7fbc180f6860 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.515+0000 7fbc1d489700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc180f55c0 0x7fbc180f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.515+0000 7fbc1d489700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc180f55c0 0x7fbc180f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47048/0 (socket says 172.21.15.42:47048) 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.515+0000 7fbc1d489700 1 -- 172.21.15.42:0/103861137 learned_addr learned my addr 172.21.15.42:0/103861137 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.515+0000 7fbc1cc88700 1 --2- 172.21.15.42:0/103861137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc180f6860 0x7fbc180f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.515+0000 7fbc1d489700 1 -- 172.21.15.42:0/103861137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc180f6860 msgr2=0x7fbc180f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.515+0000 7fbc1d489700 1 --2- 172.21.15.42:0/103861137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc180f6860 0x7fbc180f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.515+0000 7fbc1d489700 1 -- 172.21.15.42:0/103861137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc10009640 con 0x7fbc180f55c0 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.516+0000 7fbc1d489700 1 --2- 172.21.15.42:0/103861137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc180f55c0 0x7fbc180f9720 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fbc10007f70 tx=0x7fbc10004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.516+0000 7fbc15ffb700 1 -- 172.21.15.42:0/103861137 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc1000e070 con 0x7fbc180f55c0 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.516+0000 7fbc15ffb700 1 -- 172.21.15.42:0/103861137 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fbc1000be70 con 0x7fbc180f55c0 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.516+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/103861137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc180f84c0 con 0x7fbc180f55c0 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.516+0000 7fbc15ffb700 1 -- 172.21.15.42:0/103861137 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc10018070 con 0x7fbc180f55c0 2024-02-15T18:26:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.516+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/103861137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc180f89a0 con 0x7fbc180f55c0 2024-02-15T18:26:39.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.517+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/103861137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc18045ad0 con 0x7fbc180f55c0 2024-02-15T18:26:39.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.517+0000 7fbc15ffb700 1 -- 172.21.15.42:0/103861137 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 29) v1 ==== 77194+0+0 (secure 0 0 0) 0x7fbc10024060 con 0x7fbc180f55c0 2024-02-15T18:26:39.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.518+0000 7fbc15ffb700 1 --2- 172.21.15.42:0/103861137 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc00060fb0 0x7fbc00063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:26:39.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.518+0000 7fbc15ffb700 1 -- 172.21.15.42:0/103861137 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fbc1007d850 con 0x7fbc180f55c0 2024-02-15T18:26:39.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.518+0000 7fbc1cc88700 1 --2- 172.21.15.42:0/103861137 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc00060fb0 0x7fbc00063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:26:39.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.518+0000 7fbc1cc88700 1 --2- 172.21.15.42:0/103861137 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc00060fb0 0x7fbc00063470 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fbc0c004e20 tx=0x7fbc0c00b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:26:39.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.522+0000 7fbc15ffb700 1 -- 172.21.15.42:0/103861137 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162484 (secure 0 0 0) 0x7fbc1004dcc0 con 0x7fbc180f55c0 2024-02-15T18:26:39.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.709+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/103861137 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc180faca0 con 0x7fbc00060fb0 2024-02-15T18:26:39.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.710+0000 7fbc15ffb700 1 -- 172.21.15.42:0/103861137 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fbc180faca0 con 0x7fbc00060fb0 2024-02-15T18:26:39.707 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:26:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:26:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:26:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:26:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:26:39.707 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-02-15T18:26:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:26:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "2/21 daemons upgraded", 2024-02-15T18:26:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mon daemons", 2024-02-15T18:26:39.708 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:26:39.708 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:26:39.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.712+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/103861137 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc00060fb0 msgr2=0x7fbc00063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:39.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.712+0000 7fbc1f6ed700 1 --2- 172.21.15.42:0/103861137 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc00060fb0 0x7fbc00063470 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fbc0c004e20 tx=0x7fbc0c00b040).stop 2024-02-15T18:26:39.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.712+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/103861137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc180f55c0 msgr2=0x7fbc180f9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:26:39.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.712+0000 7fbc1f6ed700 1 --2- 172.21.15.42:0/103861137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc180f55c0 0x7fbc180f9720 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fbc10007f70 tx=0x7fbc10004d10).stop 2024-02-15T18:26:39.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.712+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/103861137 shutdown_connections 2024-02-15T18:26:39.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.712+0000 7fbc1f6ed700 1 --2- 172.21.15.42:0/103861137 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc00060fb0 0x7fbc00063470 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.712+0000 7fbc1f6ed700 1 --2- 172.21.15.42:0/103861137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc180f55c0 0x7fbc180f9720 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.712+0000 7fbc1f6ed700 1 --2- 172.21.15.42:0/103861137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc180f6860 0x7fbc180f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:26:39.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.712+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/103861137 >> 172.21.15.42:0/103861137 conn(0x7fbc180f0d40 msgr2=0x7fbc1805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:26:39.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.712+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/103861137 shutdown_connections 2024-02-15T18:26:39.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:26:39.712+0000 7fbc1f6ed700 1 -- 172.21.15.42:0/103861137 wait complete. 2024-02-15T18:26:39.872 INFO:tasks.workunit.client.1.smithi191.stdout:6/6: symlink l2 0 2024-02-15T18:26:39.877 INFO:tasks.workunit.client.1.smithi191.stdout:9/9: link d0/f1 d0/f3 0 2024-02-15T18:26:39.881 INFO:tasks.workunit.client.1.smithi191.stdout:7/16: creat d0/f5 x:0 0 0 2024-02-15T18:26:39.883 INFO:tasks.workunit.client.1.smithi191.stdout:8/18: symlink d3/l8 0 2024-02-15T18:26:39.883 INFO:tasks.workunit.client.1.smithi191.stdout:8/19: dwrite - no filename 2024-02-15T18:26:39.883 INFO:tasks.workunit.client.1.smithi191.stdout:8/20: dread - no filename 2024-02-15T18:26:39.911 INFO:tasks.workunit.client.1.smithi191.stdout:1/13: mknod d0/c6 0 2024-02-15T18:26:39.912 INFO:tasks.workunit.client.1.smithi191.stdout:1/14: write d0/f3 [3688199,94724] 0 2024-02-15T18:26:39.913 INFO:tasks.workunit.client.1.smithi191.stdout:4/8: link f0 d2/f4 0 2024-02-15T18:26:39.914 INFO:tasks.workunit.client.1.smithi191.stdout:6/7: creat f3 x:0 0 0 2024-02-15T18:26:39.915 INFO:tasks.workunit.client.1.smithi191.stdout:9/10: creat d0/f4 x:0 0 0 2024-02-15T18:26:39.916 INFO:tasks.workunit.client.1.smithi191.stdout:8/21: symlink d3/l9 0 2024-02-15T18:26:39.921 INFO:tasks.workunit.client.1.smithi191.stdout:1/15: mknod d0/c7 0 2024-02-15T18:26:39.923 INFO:tasks.workunit.client.1.smithi191.stdout:4/9: unlink f0 0 2024-02-15T18:26:39.924 INFO:tasks.workunit.client.1.smithi191.stdout:6/8: link f3 f4 0 2024-02-15T18:26:39.926 INFO:tasks.workunit.client.1.smithi191.stdout:8/22: unlink d3/c5 0 2024-02-15T18:26:39.927 INFO:tasks.workunit.client.1.smithi191.stdout:1/16: mkdir d0/d8 0 2024-02-15T18:26:39.928 INFO:tasks.workunit.client.1.smithi191.stdout:6/9: creat f5 x:0 0 0 2024-02-15T18:26:39.935 INFO:tasks.workunit.client.1.smithi191.stdout:9/11: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:26:39.964 INFO:tasks.workunit.client.1.smithi191.stdout:1/17: creat d0/f9 x:0 0 0 2024-02-15T18:26:39.965 INFO:tasks.workunit.client.1.smithi191.stdout:1/18: write d0/f3 [1329274,2379] 0 2024-02-15T18:26:39.966 INFO:tasks.workunit.client.1.smithi191.stdout:6/10: creat f6 x:0 0 0 2024-02-15T18:26:39.966 INFO:tasks.workunit.client.1.smithi191.stdout:6/11: chown f1 6797662 1 2024-02-15T18:26:39.966 INFO:tasks.workunit.client.1.smithi191.stdout:6/12: fdatasync f1 0 2024-02-15T18:26:39.966 INFO:tasks.workunit.client.1.smithi191.stdout:6/13: truncate f0 255161 0 2024-02-15T18:26:39.966 INFO:tasks.workunit.client.1.smithi191.stdout:6/14: stat f1 0 2024-02-15T18:26:40.000 INFO:tasks.workunit.client.1.smithi191.stdout:1/19: symlink d0/la 0 2024-02-15T18:26:40.000 INFO:tasks.workunit.client.1.smithi191.stdout:1/20: write d0/f5 [997459,130782] 0 2024-02-15T18:26:40.000 INFO:tasks.workunit.client.1.smithi191.stdout:6/15: unlink f4 0 2024-02-15T18:26:40.008 INFO:tasks.workunit.client.1.smithi191.stdout:2/13: truncate f1 12792 0 2024-02-15T18:26:40.013 INFO:tasks.workunit.client.1.smithi191.stdout:6/16: fsync f5 0 2024-02-15T18:26:40.014 INFO:tasks.workunit.client.1.smithi191.stdout:8/23: rmdir d3 39 2024-02-15T18:26:40.014 INFO:tasks.workunit.client.1.smithi191.stdout:8/24: write - no filename 2024-02-15T18:26:40.014 INFO:tasks.workunit.client.1.smithi191.stdout:8/25: write - no filename 2024-02-15T18:26:40.017 INFO:tasks.workunit.client.1.smithi191.stdout:1/21: dread d0/f3 [0,4194304] 0 2024-02-15T18:26:40.017 INFO:tasks.workunit.client.1.smithi191.stdout:1/22: getdents d0/d8 0 2024-02-15T18:26:40.023 INFO:tasks.workunit.client.1.smithi191.stdout:2/14: write f1 [1003976,15662] 0 2024-02-15T18:26:40.025 INFO:tasks.workunit.client.1.smithi191.stdout:6/17: symlink l7 0 2024-02-15T18:26:40.025 INFO:tasks.workunit.client.1.smithi191.stdout:6/18: dread f0 [0,4194304] 0 2024-02-15T18:26:40.026 INFO:tasks.workunit.client.1.smithi191.stdout:6/19: dread f0 [0,4194304] 0 2024-02-15T18:26:40.027 INFO:tasks.workunit.client.1.smithi191.stdout:8/26: unlink d3/c7 0 2024-02-15T18:26:40.030 INFO:tasks.workunit.client.1.smithi191.stdout:2/15: rename l2 to l3 0 2024-02-15T18:26:40.034 INFO:tasks.workunit.client.1.smithi191.stdout:2/16: symlink l4 0 2024-02-15T18:26:40.038 INFO:tasks.workunit.client.1.smithi191.stdout:8/27: rmdir d3 39 2024-02-15T18:26:40.038 INFO:tasks.workunit.client.1.smithi191.stdout:8/28: fsync - no filename 2024-02-15T18:26:40.038 INFO:tasks.workunit.client.1.smithi191.stdout:8/29: write - no filename 2024-02-15T18:26:40.041 INFO:tasks.workunit.client.1.smithi191.stdout:8/30: rmdir d3 39 2024-02-15T18:26:40.043 INFO:tasks.workunit.client.1.smithi191.stdout:8/31: creat d3/fa x:0 0 0 2024-02-15T18:26:40.050 INFO:tasks.workunit.client.1.smithi191.stdout:6/20: dwrite f3 [0,4194304] 0 2024-02-15T18:26:40.051 INFO:tasks.workunit.client.1.smithi191.stdout:6/21: readlink l7 0 2024-02-15T18:26:40.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:38.098726+0000 mgr.smithi042.kfoihb (mgr.24435) 21 : audit [DBG] from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:40.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:38.401263+0000 mgr.smithi042.kfoihb (mgr.24435) 22 : audit [DBG] from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:40.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:38.677049+0000 mgr.smithi042.kfoihb (mgr.24435) 23 : cluster [DBG] pgmap v18: 65 pgs: 65 active+clean; 175 MiB data, 749 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 9.4 MiB/s wr, 888 op/s 2024-02-15T18:26:40.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:38.697618+0000 mgr.smithi042.kfoihb (mgr.24435) 24 : audit [DBG] from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:40.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:39.062315+0000 mon.smithi042 (mon.0) 666 : audit [DBG] from='client.? 172.21.15.42:0/3915843573' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:26:40.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:39.405010+0000 mon.smithi042 (mon.0) 667 : audit [DBG] from='client.? 172.21.15.42:0/1122273321' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:26:40.073 INFO:tasks.workunit.client.1.smithi191.stdout:6/22: dwrite f6 [0,4194304] 0 2024-02-15T18:26:40.073 INFO:tasks.workunit.client.1.smithi191.stdout:6/23: dread - f1 zero size 2024-02-15T18:26:40.074 INFO:tasks.workunit.client.1.smithi191.stdout:8/32: dwrite d3/fa [0,4194304] 0 2024-02-15T18:26:40.075 INFO:tasks.workunit.client.1.smithi191.stdout:8/33: write d3/fa [2133833,52801] 0 2024-02-15T18:26:40.083 INFO:tasks.workunit.client.1.smithi191.stdout:6/24: dread f3 [0,4194304] 0 2024-02-15T18:26:40.090 INFO:tasks.workunit.client.1.smithi191.stdout:8/34: truncate d3/fa 2941894 0 2024-02-15T18:26:40.092 INFO:tasks.workunit.client.1.smithi191.stdout:8/35: mknod d3/cb 0 2024-02-15T18:26:40.092 INFO:tasks.workunit.client.1.smithi191.stdout:6/25: dread f3 [0,4194304] 0 2024-02-15T18:26:40.093 INFO:tasks.workunit.client.1.smithi191.stdout:8/36: creat d3/fc x:0 0 0 2024-02-15T18:26:40.094 INFO:tasks.workunit.client.1.smithi191.stdout:6/26: mkdir d8 0 2024-02-15T18:26:40.096 INFO:tasks.workunit.client.1.smithi191.stdout:8/37: creat d3/fd x:0 0 0 2024-02-15T18:26:40.096 INFO:tasks.workunit.client.1.smithi191.stdout:6/27: mkdir d8/d9 0 2024-02-15T18:26:40.100 INFO:tasks.workunit.client.1.smithi191.stdout:8/38: symlink d3/le 0 2024-02-15T18:26:40.102 INFO:tasks.workunit.client.1.smithi191.stdout:6/28: unlink f3 0 2024-02-15T18:26:40.103 INFO:tasks.workunit.client.1.smithi191.stdout:6/29: creat d8/d9/fa x:0 0 0 2024-02-15T18:26:40.112 INFO:tasks.workunit.client.1.smithi191.stdout:8/39: dread d3/fa [0,4194304] 0 2024-02-15T18:26:40.113 INFO:tasks.workunit.client.1.smithi191.stdout:8/40: creat d3/ff x:0 0 0 2024-02-15T18:26:40.113 INFO:tasks.workunit.client.1.smithi191.stdout:8/41: read - d3/fd zero size 2024-02-15T18:26:40.114 INFO:tasks.workunit.client.1.smithi191.stdout:8/42: mknod d3/c10 0 2024-02-15T18:26:40.136 INFO:tasks.workunit.client.1.smithi191.stdout:8/43: dwrite d3/fa [0,4194304] 0 2024-02-15T18:26:40.136 INFO:tasks.workunit.client.1.smithi191.stdout:8/44: fdatasync d3/fc 0 2024-02-15T18:26:40.138 INFO:tasks.workunit.client.1.smithi191.stdout:8/45: creat d3/f11 x:0 0 0 2024-02-15T18:26:40.139 INFO:tasks.workunit.client.1.smithi191.stdout:8/46: creat d3/f12 x:0 0 0 2024-02-15T18:26:40.140 INFO:tasks.workunit.client.1.smithi191.stdout:8/47: chown d3/c10 614829 1 2024-02-15T18:26:40.141 INFO:tasks.workunit.client.1.smithi191.stdout:8/48: mknod d3/c13 0 2024-02-15T18:26:40.141 INFO:tasks.workunit.client.1.smithi191.stdout:8/49: chown d3/l8 66397 1 2024-02-15T18:26:40.160 INFO:tasks.workunit.client.1.smithi191.stdout:8/50: dwrite d3/fd [0,4194304] 0 2024-02-15T18:26:40.165 INFO:tasks.workunit.client.1.smithi191.stdout:8/51: mkdir d3/d14 0 2024-02-15T18:26:40.166 INFO:tasks.workunit.client.1.smithi191.stdout:8/52: truncate d3/ff 17898 0 2024-02-15T18:26:40.166 INFO:tasks.workunit.client.1.smithi191.stdout:8/53: chown d3/d14 935111 1 2024-02-15T18:26:40.173 INFO:tasks.workunit.client.1.smithi191.stdout:8/54: dread d3/fd [0,4194304] 0 2024-02-15T18:26:40.174 INFO:tasks.workunit.client.1.smithi191.stdout:8/55: truncate d3/fa 4954109 0 2024-02-15T18:26:40.195 INFO:tasks.workunit.client.1.smithi191.stdout:8/56: rename d3/ff to d3/f15 0 2024-02-15T18:26:40.195 INFO:tasks.workunit.client.1.smithi191.stdout:8/57: stat d3 0 2024-02-15T18:26:40.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:38.098726+0000 mgr.smithi042.kfoihb (mgr.24435) 21 : audit [DBG] from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:40.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:38.401263+0000 mgr.smithi042.kfoihb (mgr.24435) 22 : audit [DBG] from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:40.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:38.677049+0000 mgr.smithi042.kfoihb (mgr.24435) 23 : cluster [DBG] pgmap v18: 65 pgs: 65 active+clean; 175 MiB data, 749 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 9.4 MiB/s wr, 888 op/s 2024-02-15T18:26:40.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:38.697618+0000 mgr.smithi042.kfoihb (mgr.24435) 24 : audit [DBG] from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:40.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:39.062315+0000 mon.smithi042 (mon.0) 666 : audit [DBG] from='client.? 172.21.15.42:0/3915843573' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:26:40.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:39.405010+0000 mon.smithi042 (mon.0) 667 : audit [DBG] from='client.? 172.21.15.42:0/1122273321' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:26:40.197 INFO:tasks.workunit.client.1.smithi191.stdout:8/58: mkdir d3/d16 0 2024-02-15T18:26:40.199 INFO:tasks.workunit.client.1.smithi191.stdout:8/59: creat d3/d16/f17 x:0 0 0 2024-02-15T18:26:40.201 INFO:tasks.workunit.client.1.smithi191.stdout:8/60: creat d3/f18 x:0 0 0 2024-02-15T18:26:40.202 INFO:tasks.workunit.client.1.smithi191.stdout:8/61: read d3/fa [443891,3927] 0 2024-02-15T18:26:40.202 INFO:tasks.workunit.client.1.smithi191.stdout:8/62: chown d3 1 1 2024-02-15T18:26:40.203 INFO:tasks.workunit.client.1.smithi191.stdout:8/63: mknod d3/c19 0 2024-02-15T18:26:40.203 INFO:tasks.workunit.client.1.smithi191.stdout:8/64: chown d3/d16/f17 535171807 1 2024-02-15T18:26:40.206 INFO:tasks.workunit.client.1.smithi191.stdout:8/65: rmdir d3/d14 0 2024-02-15T18:26:40.211 INFO:tasks.workunit.client.1.smithi191.stdout:8/66: unlink c1 0 2024-02-15T18:26:40.213 INFO:tasks.workunit.client.1.smithi191.stdout:8/67: mkdir d3/d16/d1a 0 2024-02-15T18:26:40.214 INFO:tasks.workunit.client.1.smithi191.stdout:8/68: chown d3/fa 94 1 2024-02-15T18:26:40.214 INFO:tasks.workunit.client.1.smithi191.stdout:8/69: write d3/fc [280894,44866] 0 2024-02-15T18:26:40.215 INFO:tasks.workunit.client.1.smithi191.stdout:8/70: creat d3/d16/d1a/f1b x:0 0 0 2024-02-15T18:26:40.217 INFO:tasks.workunit.client.1.smithi191.stdout:8/71: write d3/fa [4250549,9066] 0 2024-02-15T18:26:40.219 INFO:tasks.workunit.client.1.smithi191.stdout:8/72: rename d3/f11 to d3/f1c 0 2024-02-15T18:26:40.221 INFO:tasks.workunit.client.1.smithi191.stdout:8/73: write d3/fd [3961002,36735] 0 2024-02-15T18:26:40.221 INFO:tasks.workunit.client.1.smithi191.stdout:8/74: stat d3/d16 0 2024-02-15T18:26:40.228 INFO:tasks.workunit.client.1.smithi191.stdout:8/75: dread d3/fd [0,4194304] 0 2024-02-15T18:26:40.228 INFO:tasks.workunit.client.1.smithi191.stdout:8/76: readlink d3/le 0 2024-02-15T18:26:40.229 INFO:tasks.workunit.client.1.smithi191.stdout:8/77: creat d3/d16/f1d x:0 0 0 2024-02-15T18:26:40.231 INFO:tasks.workunit.client.1.smithi191.stdout:8/78: symlink d3/d16/d1a/l1e 0 2024-02-15T18:26:40.231 INFO:tasks.workunit.client.1.smithi191.stdout:8/79: readlink d3/l8 0 2024-02-15T18:26:40.248 INFO:tasks.workunit.client.1.smithi191.stdout:8/80: truncate d3/fd 1593402 0 2024-02-15T18:26:40.260 INFO:tasks.workunit.client.1.smithi191.stdout:8/81: dread d3/fd [0,4194304] 0 2024-02-15T18:26:40.261 INFO:tasks.workunit.client.1.smithi191.stdout:8/82: write d3/d16/d1a/f1b [231588,50785] 0 2024-02-15T18:26:40.262 INFO:tasks.workunit.client.1.smithi191.stdout:8/83: creat d3/d16/f1f x:0 0 0 2024-02-15T18:26:40.265 INFO:tasks.workunit.client.1.smithi191.stdout:8/84: dread d3/fd [0,4194304] 0 2024-02-15T18:26:40.281 INFO:tasks.workunit.client.1.smithi191.stdout:8/85: dwrite d3/fd [0,4194304] 0 2024-02-15T18:26:40.282 INFO:tasks.workunit.client.1.smithi191.stdout:8/86: creat d3/d16/d1a/f20 x:0 0 0 2024-02-15T18:26:40.284 INFO:tasks.workunit.client.1.smithi191.stdout:8/87: stat d3/cb 0 2024-02-15T18:26:40.286 INFO:tasks.workunit.client.1.smithi191.stdout:8/88: creat d3/f21 x:0 0 0 2024-02-15T18:26:40.306 INFO:tasks.workunit.client.1.smithi191.stdout:8/89: dwrite d3/d16/d1a/f1b [0,4194304] 0 2024-02-15T18:26:40.308 INFO:tasks.workunit.client.1.smithi191.stdout:8/90: dread - d3/f18 zero size 2024-02-15T18:26:40.317 INFO:tasks.workunit.client.1.smithi191.stdout:8/91: symlink d3/l22 0 2024-02-15T18:26:40.317 INFO:tasks.workunit.client.1.smithi191.stdout:8/92: readlink d3/l22 0 2024-02-15T18:26:40.327 INFO:tasks.workunit.client.1.smithi191.stdout:8/93: dread d3/fd [0,4194304] 0 2024-02-15T18:26:40.327 INFO:tasks.workunit.client.1.smithi191.stdout:8/94: chown d3/d16/f1d 6 1 2024-02-15T18:26:40.327 INFO:tasks.workunit.client.1.smithi191.stdout:8/95: mkdir d3/d16/d1a/d23 0 2024-02-15T18:26:40.329 INFO:tasks.workunit.client.1.smithi191.stdout:8/96: dread d3/f15 [0,4194304] 0 2024-02-15T18:26:40.330 INFO:tasks.workunit.client.1.smithi191.stdout:8/97: write d3/d16/f17 [342483,120847] 0 2024-02-15T18:26:40.330 INFO:tasks.workunit.client.1.smithi191.stdout:8/98: chown d3/f21 250681 1 2024-02-15T18:26:40.337 INFO:tasks.workunit.client.1.smithi191.stdout:8/99: rmdir d3/d16/d1a 39 2024-02-15T18:26:40.340 INFO:tasks.workunit.client.1.smithi191.stdout:8/100: truncate d3/f1c 37350 0 2024-02-15T18:26:40.340 INFO:tasks.workunit.client.1.smithi191.stdout:8/101: fdatasync d3/d16/f1f 0 2024-02-15T18:26:40.340 INFO:tasks.workunit.client.1.smithi191.stdout:8/102: fsync d3/f18 0 2024-02-15T18:26:40.341 INFO:tasks.workunit.client.1.smithi191.stdout:8/103: symlink d3/d16/d1a/d23/l24 0 2024-02-15T18:26:40.342 INFO:tasks.workunit.client.1.smithi191.stdout:8/104: symlink d3/d16/d1a/d23/l25 0 2024-02-15T18:26:40.343 INFO:tasks.workunit.client.1.smithi191.stdout:8/105: getdents d3/d16 0 2024-02-15T18:26:40.344 INFO:tasks.workunit.client.1.smithi191.stdout:8/106: creat d3/d16/f26 x:0 0 0 2024-02-15T18:26:40.351 INFO:tasks.workunit.client.1.smithi191.stdout:8/107: read - d3/d16/f1f zero size 2024-02-15T18:26:40.351 INFO:tasks.workunit.client.1.smithi191.stdout:8/108: write d3/f1c [670270,78989] 0 2024-02-15T18:26:40.366 INFO:tasks.workunit.client.1.smithi191.stdout:8/109: dwrite d3/f21 [0,4194304] 0 2024-02-15T18:26:40.371 INFO:tasks.workunit.client.1.smithi191.stdout:8/110: dread d3/fc [0,4194304] 0 2024-02-15T18:26:40.379 INFO:tasks.workunit.client.1.smithi191.stdout:8/111: symlink d3/d16/d1a/l27 0 2024-02-15T18:26:40.380 INFO:tasks.workunit.client.1.smithi191.stdout:8/112: chown d3/l8 11946532 1 2024-02-15T18:26:40.381 INFO:tasks.workunit.client.1.smithi191.stdout:8/113: dread - d3/d16/f1d zero size 2024-02-15T18:26:40.384 INFO:tasks.workunit.client.1.smithi191.stdout:8/114: mknod d3/c28 0 2024-02-15T18:26:40.384 INFO:tasks.workunit.client.1.smithi191.stdout:8/115: stat d3/d16/d1a/l1e 0 2024-02-15T18:26:40.384 INFO:tasks.workunit.client.1.smithi191.stdout:8/116: creat d3/f29 x:0 0 0 2024-02-15T18:26:40.386 INFO:tasks.workunit.client.1.smithi191.stdout:8/117: getdents d3/d16 0 2024-02-15T18:26:40.388 INFO:tasks.workunit.client.1.smithi191.stdout:8/118: mkdir d3/d16/d1a/d23/d2a 0 2024-02-15T18:26:40.419 INFO:tasks.workunit.client.1.smithi191.stdout:8/119: dwrite d3/fa [0,4194304] 0 2024-02-15T18:26:40.420 INFO:tasks.workunit.client.1.smithi191.stdout:8/120: symlink d3/l2b 0 2024-02-15T18:26:40.426 INFO:tasks.workunit.client.1.smithi191.stdout:8/121: rename d3/d16 to d3/d2c 0 2024-02-15T18:26:40.441 INFO:tasks.workunit.client.1.smithi191.stdout:8/122: dwrite d3/f15 [0,4194304] 0 2024-02-15T18:26:40.444 INFO:tasks.workunit.client.1.smithi191.stdout:8/123: creat d3/d2c/d1a/f2d x:0 0 0 2024-02-15T18:26:40.444 INFO:tasks.workunit.client.1.smithi191.stdout:8/124: stat d3/d2c/d1a/l27 0 2024-02-15T18:26:40.460 INFO:tasks.workunit.client.1.smithi191.stdout:8/125: dwrite d3/d2c/d1a/f20 [0,4194304] 0 2024-02-15T18:26:40.461 INFO:tasks.workunit.client.1.smithi191.stdout:8/126: read d3/f21 [2875827,35271] 0 2024-02-15T18:26:40.465 INFO:tasks.workunit.client.1.smithi191.stdout:8/127: dread d3/d2c/f17 [0,4194304] 0 2024-02-15T18:26:40.473 INFO:tasks.workunit.client.1.smithi191.stdout:8/128: readlink d3/le 0 2024-02-15T18:26:40.488 INFO:tasks.workunit.client.1.smithi191.stdout:8/129: dwrite d3/d2c/d1a/f2d [0,4194304] 0 2024-02-15T18:26:40.491 INFO:tasks.workunit.client.1.smithi191.stdout:8/130: write d3/f15 [2883676,15085] 0 2024-02-15T18:26:40.501 INFO:tasks.workunit.client.1.smithi191.stdout:8/131: mkdir d3/d2c/d1a/d23/d2e 0 2024-02-15T18:26:40.502 INFO:tasks.workunit.client.1.smithi191.stdout:8/132: link d3/cb d3/d2c/d1a/c2f 0 2024-02-15T18:26:40.504 INFO:tasks.workunit.client.1.smithi191.stdout:8/133: symlink d3/l30 0 2024-02-15T18:26:40.505 INFO:tasks.workunit.client.1.smithi191.stdout:8/134: creat d3/f31 x:0 0 0 2024-02-15T18:26:40.509 INFO:tasks.workunit.client.1.smithi191.stdout:8/135: getdents d3/d2c/d1a/d23 0 2024-02-15T18:26:40.514 INFO:tasks.workunit.client.1.smithi191.stdout:8/136: dread d3/f1c [0,4194304] 0 2024-02-15T18:26:40.531 INFO:tasks.workunit.client.1.smithi191.stdout:8/137: dwrite d3/d2c/f1d [0,4194304] 0 2024-02-15T18:26:40.531 INFO:tasks.workunit.client.1.smithi191.stdout:8/138: readlink d3/l30 0 2024-02-15T18:26:40.531 INFO:tasks.workunit.client.1.smithi191.stdout:8/139: readlink d3/d2c/d1a/d23/l25 0 2024-02-15T18:26:40.532 INFO:tasks.workunit.client.1.smithi191.stdout:8/140: symlink d3/d2c/d1a/d23/d2e/l32 0 2024-02-15T18:26:40.538 INFO:tasks.workunit.client.1.smithi191.stdout:8/141: dread d3/fd [0,4194304] 0 2024-02-15T18:26:40.538 INFO:tasks.workunit.client.1.smithi191.stdout:8/142: read - d3/f29 zero size 2024-02-15T18:26:40.538 INFO:tasks.workunit.client.1.smithi191.stdout:8/143: write d3/fc [729861,19184] 0 2024-02-15T18:26:40.546 INFO:tasks.workunit.client.1.smithi191.stdout:8/144: rename d3/c19 to d3/d2c/d1a/d23/d2a/c33 0 2024-02-15T18:26:40.548 INFO:tasks.workunit.client.1.smithi191.stdout:8/145: mkdir d3/d2c/d1a/d23/d2a/d34 0 2024-02-15T18:26:40.549 INFO:tasks.workunit.client.1.smithi191.stdout:8/146: creat d3/d2c/d1a/d23/d2a/f35 x:0 0 0 2024-02-15T18:26:40.565 INFO:tasks.workunit.client.1.smithi191.stdout:8/147: dwrite d3/d2c/d1a/f2d [0,4194304] 0 2024-02-15T18:26:40.566 INFO:tasks.workunit.client.1.smithi191.stdout:8/148: truncate d3/f12 568550 0 2024-02-15T18:26:40.567 INFO:tasks.workunit.client.1.smithi191.stdout:8/149: truncate d3/d2c/d1a/f20 1250339 0 2024-02-15T18:26:40.568 INFO:tasks.workunit.client.1.smithi191.stdout:8/150: mkdir d3/d2c/d1a/d36 0 2024-02-15T18:26:40.578 INFO:tasks.workunit.client.1.smithi191.stdout:8/151: dread d3/d2c/d1a/f1b [0,4194304] 0 2024-02-15T18:26:40.593 INFO:tasks.workunit.client.1.smithi191.stdout:8/152: dwrite d3/d2c/d1a/f20 [0,4194304] 0 2024-02-15T18:26:40.594 INFO:tasks.workunit.client.1.smithi191.stdout:8/153: rmdir d3/d2c/d1a/d23/d2e 39 2024-02-15T18:26:40.600 INFO:tasks.workunit.client.1.smithi191.stdout:8/154: dread d3/d2c/d1a/f1b [0,4194304] 0 2024-02-15T18:26:40.602 INFO:tasks.workunit.client.1.smithi191.stdout:8/155: unlink d3/c4 0 2024-02-15T18:26:40.604 INFO:tasks.workunit.client.1.smithi191.stdout:8/156: mknod d3/d2c/d1a/d23/d2a/d34/c37 0 2024-02-15T18:26:40.604 INFO:tasks.workunit.client.1.smithi191.stdout:8/157: chown d3/l8 3343948 1 2024-02-15T18:26:40.605 INFO:tasks.workunit.client.1.smithi191.stdout:8/158: mkdir d3/d2c/d1a/d23/d2e/d38 0 2024-02-15T18:26:40.606 INFO:tasks.workunit.client.1.smithi191.stdout:8/159: unlink d3/f1c 0 2024-02-15T18:26:40.608 INFO:tasks.workunit.client.1.smithi191.stdout:8/160: write d3/fa [1636035,84180] 0 2024-02-15T18:26:40.608 INFO:tasks.workunit.client.1.smithi191.stdout:8/161: stat d3/le 0 2024-02-15T18:26:40.608 INFO:tasks.workunit.client.1.smithi191.stdout:8/162: dread - d3/d2c/f1f zero size 2024-02-15T18:26:40.608 INFO:tasks.workunit.client.1.smithi191.stdout:8/163: dread - d3/f18 zero size 2024-02-15T18:26:40.610 INFO:tasks.workunit.client.1.smithi191.stdout:8/164: unlink d3/l9 0 2024-02-15T18:26:40.610 INFO:tasks.workunit.client.1.smithi191.stdout:8/165: chown d3/l22 223822 1 2024-02-15T18:26:40.611 INFO:tasks.workunit.client.1.smithi191.stdout:8/166: creat d3/f39 x:0 0 0 2024-02-15T18:26:40.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:39.711301+0000 mgr.smithi042.kfoihb (mgr.24435) 25 : audit [DBG] from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:41.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:39.711301+0000 mgr.smithi042.kfoihb (mgr.24435) 25 : audit [DBG] from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:26:42.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:40.677555+0000 mgr.smithi042.kfoihb (mgr.24435) 26 : cluster [DBG] pgmap v19: 65 pgs: 65 active+clean; 175 MiB data, 749 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 6.4 MiB/s wr, 522 op/s 2024-02-15T18:26:42.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:40.677555+0000 mgr.smithi042.kfoihb (mgr.24435) 26 : cluster [DBG] pgmap v19: 65 pgs: 65 active+clean; 175 MiB data, 749 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 6.4 MiB/s wr, 522 op/s 2024-02-15T18:26:43.696 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:26:42.389222+0000 mgr.smithi042.kfoihb (mgr.24435) 27 : cephadm [INF] Upgrade: Updating mon.smithi042 2024-02-15T18:26:43.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:42.393118+0000 mon.smithi042 (mon.0) 668 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:43.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:42.393811+0000 mon.smithi042 (mon.0) 669 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:26:43.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:42.394391+0000 mon.smithi042 (mon.0) 670 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:26:43.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:42.394947+0000 mon.smithi042 (mon.0) 671 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:26:43.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:26:42.395479+0000 mgr.smithi042.kfoihb (mgr.24435) 28 : cephadm [INF] Deploying daemon mon.smithi042 on smithi042 2024-02-15T18:26:43.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:42.678067+0000 mgr.smithi042.kfoihb (mgr.24435) 29 : cluster [DBG] pgmap v20: 65 pgs: 65 active+clean; 242 MiB data, 927 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s rd, 14 MiB/s wr, 851 op/s 2024-02-15T18:26:43.799 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:26:42.389222+0000 mgr.smithi042.kfoihb (mgr.24435) 27 : cephadm [INF] Upgrade: Updating mon.smithi042 2024-02-15T18:26:43.799 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:42.393118+0000 mon.smithi042 (mon.0) 668 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:43.799 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:42.393811+0000 mon.smithi042 (mon.0) 669 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:26:43.799 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:42.394391+0000 mon.smithi042 (mon.0) 670 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:26:43.799 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: audit 2024-02-15T18:26:42.394947+0000 mon.smithi042 (mon.0) 671 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:26:43.799 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cephadm 2024-02-15T18:26:42.395479+0000 mgr.smithi042.kfoihb (mgr.24435) 28 : cephadm [INF] Deploying daemon mon.smithi042 on smithi042 2024-02-15T18:26:43.799 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: cluster 2024-02-15T18:26:42.678067+0000 mgr.smithi042.kfoihb (mgr.24435) 29 : cluster [DBG] pgmap v20: 65 pgs: 65 active+clean; 242 MiB data, 927 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s rd, 14 MiB/s wr, 851 op/s 2024-02-15T18:26:44.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:43 smithi042 systemd[1]: Stopping Ceph mon.smithi042 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:26:44.559 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:26:44.073+0000 7f0b64509700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi042 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-15T18:26:44.559 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042[38244]: debug 2024-02-15T18:26:44.073+0000 7f0b64509700 -1 mon.smithi042@0(leader) e2 *** Got Signal Terminated *** 2024-02-15T18:26:45.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:44 smithi042 bash[77437]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042 2024-02-15T18:26:45.559 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 systemd[1]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi042.service: Succeeded. 2024-02-15T18:26:45.559 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 systemd[1]: Stopped Ceph mon.smithi042 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:26:45.560 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 systemd[1]: Starting Ceph mon.smithi042 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:26:46.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.909+0000 7fd948789880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-15T18:26:46.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.909+0000 7fd948789880 0 ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable), process ceph-mon, pid 7 2024-02-15T18:26:46.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.909+0000 7fd948789880 0 pidfile_write: ignore empty --pid-file 2024-02-15T18:26:46.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.913+0000 7fd948789880 0 load: jerasure load: lrc load: isa 2024-02-15T18:26:46.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: RocksDB version: 6.8.1 2024-02-15T18:26:46.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-15T18:26:46.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Compile date Feb 8 2024 2024-02-15T18:26:46.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: DB SUMMARY 2024-02-15T18:26:46.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: CURRENT file: CURRENT 2024-02-15T18:26:46.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-15T18:26:46.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: MANIFEST file: MANIFEST-000008 size: 579 Bytes 2024-02-15T18:26:46.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi042/store.db dir, Total Num: 1, files: 000017.sst 2024-02-15T18:26:46.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi042/store.db: 000015.log size: 0 ; 2024-02-15T18:26:46.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.error_if_exists: 0 2024-02-15T18:26:46.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.create_if_missing: 0 2024-02-15T18:26:46.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.paranoid_checks: 1 2024-02-15T18:26:46.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.env: 0x5640ad4bb080 2024-02-15T18:26:46.315 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.fs: Posix File System 2024-02-15T18:26:46.315 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.info_log: 0x5640aea2a560 2024-02-15T18:26:46.315 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-15T18:26:46.315 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.statistics: (nil) 2024-02-15T18:26:46.315 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.use_fsync: 0 2024-02-15T18:26:46.315 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_log_file_size: 0 2024-02-15T18:26:46.315 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-15T18:26:46.315 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-15T18:26:46.315 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-15T18:26:46.315 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-15T18:26:46.315 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.allow_fallocate: 1 2024-02-15T18:26:46.315 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-15T18:26:46.315 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-15T18:26:46.315 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.use_direct_reads: 0 2024-02-15T18:26:46.316 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-15T18:26:46.316 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-15T18:26:46.316 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.db_log_dir: 2024-02-15T18:26:46.316 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi042/store.db 2024-02-15T18:26:46.316 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-15T18:26:46.316 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_subcompactions: 1 2024-02-15T18:26:46.316 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_background_flushes: -1 2024-02-15T18:26:46.316 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-15T18:26:46.316 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-15T18:26:46.316 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-15T18:26:46.316 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-15T18:26:46.316 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-15T18:26:46.316 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-15T18:26:46.316 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-15T18:26:46.317 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.write_buffer_manager: 0x5640af71c420 2024-02-15T18:26:46.317 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-15T18:26:46.317 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-15T18:26:46.317 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-15T18:26:46.317 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-15T18:26:46.317 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-15T18:26:46.317 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-15T18:26:46.317 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-15T18:26:46.317 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-15T18:26:46.317 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-15T18:26:46.317 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.unordered_write: 0 2024-02-15T18:26:46.317 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-15T18:26:46.317 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-15T18:26:46.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-15T18:26:46.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-15T18:26:46.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.row_cache: None 2024-02-15T18:26:46.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.wal_filter: None 2024-02-15T18:26:46.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-15T18:26:46.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-15T18:26:46.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.preserve_deletes: 0 2024-02-15T18:26:46.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.two_write_queues: 0 2024-02-15T18:26:46.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-15T18:26:46.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.atomic_flush: 0 2024-02-15T18:26:46.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-15T18:26:46.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-15T18:26:46.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.log_readahead_size: 0 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_background_jobs: 2 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_background_compactions: -1 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_open_files: -1 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-15T18:26:46.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-15T18:26:46.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-15T18:26:46.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-15T18:26:46.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Compression algorithms supported: 2024-02-15T18:26:46.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-15T18:26:46.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: kZSTD supported: 0 2024-02-15T18:26:46.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: kXpressCompression supported: 0 2024-02-15T18:26:46.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-15T18:26:46.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: kLZ4Compression supported: 1 2024-02-15T18:26:46.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: kBZip2Compression supported: 0 2024-02-15T18:26:46.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: kZlibCompression supported: 1 2024-02-15T18:26:46.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: kSnappyCompression supported: 1 2024-02-15T18:26:46.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-15T18:26:46.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi042/store.db/MANIFEST-000008 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.merge_operator: 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compaction_filter: None 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5640ae9cfd28) 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cache_index_and_filter_blocks: 1 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: pin_top_level_index_and_filter: 1 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: index_type: 0 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: data_block_index_type: 0 2024-02-15T18:26:46.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: index_shortening: 1 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: data_block_hash_table_util_ratio: 0.750000 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: hash_index_allow_collision: 1 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: checksum: 1 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: no_block_cache: 0 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: block_cache: 0x5640aea04f10 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: block_cache_name: BinnedLRUCache 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: block_cache_options: 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: capacity : 536870912 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: num_shard_bits : 4 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: strict_capacity_limit : 0 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: high_pri_pool_ratio: 0.000 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: block_cache_compressed: (nil) 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: persistent_cache: (nil) 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: block_size: 4096 2024-02-15T18:26:46.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: block_size_deviation: 10 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: block_restart_interval: 16 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: index_block_restart_interval: 1 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: metadata_block_size: 4096 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: partition_filters: 0 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: use_delta_encoding: 1 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: whole_key_filtering: 1 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: verify_compression: 0 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: read_amp_bytes_per_bit: 0 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: format_version: 2 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: enable_index_compression: 1 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: block_align: 0 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-15T18:26:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-15T18:26:46.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compression: NoCompression 2024-02-15T18:26:46.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-15T18:26:46.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-15T18:26:46.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-15T18:26:46.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.num_levels: 7 2024-02-15T18:26:46.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-15T18:26:46.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-15T18:26:46.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-15T18:26:46.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-15T18:26:46.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-15T18:26:46.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-15T18:26:46.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-15T18:26:46.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:26:46.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-15T18:26:46.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-15T18:26:46.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-15T18:26:46.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-15T18:26:46.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-15T18:26:46.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:26:46.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-15T18:26:46.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-15T18:26:46.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-15T18:26:46.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-15T18:26:46.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-15T18:26:46.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-15T18:26:46.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-15T18:26:46.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-15T18:26:46.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-15T18:26:46.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-15T18:26:46.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-15T18:26:46.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-15T18:26:46.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-15T18:26:46.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-15T18:26:46.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-15T18:26:46.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-15T18:26:46.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-15T18:26:46.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-15T18:26:46.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-15T18:26:46.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-15T18:26:46.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-15T18:26:46.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-15T18:26:46.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-15T18:26:46.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-15T18:26:46.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-15T18:26:46.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-15T18:26:46.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-15T18:26:46.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-15T18:26:46.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-15T18:26:46.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-15T18:26:46.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-15T18:26:46.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-15T18:26:46.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-15T18:26:46.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.table_properties_collectors: 2024-02-15T18:26:46.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.inplace_update_support: 0 2024-02-15T18:26:46.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-15T18:26:46.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-15T18:26:46.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-15T18:26:46.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-15T18:26:46.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.bloom_locality: 0 2024-02-15T18:26:46.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.max_successive_merges: 0 2024-02-15T18:26:46.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-15T18:26:46.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-15T18:26:46.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-15T18:26:46.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-15T18:26:46.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.ttl: 2592000 2024-02-15T18:26:46.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.914+0000 7fd948789880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-15T18:26:46.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.915+0000 7fd948789880 4 rocksdb: [version_set.cc:3291] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-15T18:26:46.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.915+0000 7fd948789880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi042/store.db/MANIFEST-000008 succeeded,manifest_file_number is 8, next_file_number is 19, last_sequence is 8837, log_number is 15,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-15T18:26:46.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.915+0000 7fd948789880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 15 2024-02-15T18:26:46.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.915+0000 7fd948789880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708021605916469, "job": 1, "event": "recovery_started", "log_files": [15]} 2024-02-15T18:26:46.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.915+0000 7fd948789880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #15 mode 2 2024-02-15T18:26:46.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.915+0000 7fd948789880 4 rocksdb: [version_set.cc:3826] Creating manifest 19 2024-02-15T18:26:46.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.915+0000 7fd948789880 4 rocksdb: [version_set.cc:3291] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-15T18:26:46.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.916+0000 7fd948789880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708021605917029, "job": 1, "event": "recovery_finished"} 2024-02-15T18:26:46.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.916+0000 7fd948789880 4 rocksdb: DB pointer 0x5640aeacd800 2024-02-15T18:26:46.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.916+0000 7fd92f57a700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-15T18:26:46.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.916+0000 7fd92f57a700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-15T18:26:46.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: ** DB Stats ** 2024-02-15T18:26:46.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:26:46.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-15T18:26:46.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-15T18:26:46.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:26:46.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-15T18:26:46.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-15T18:26:46.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:26:46.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: ** Compaction Stats [default] ** 2024-02-15T18:26:46.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:26:46.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:26:46.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: L6 1/0 22.65 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:26:46.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Sum 1/0 22.65 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:26:46.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:26:46.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: ** Compaction Stats [default] ** 2024-02-15T18:26:46.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:26:46.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:26:46.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:26:46.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-15T18:26:46.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:26:46.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:26:46.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:26:46.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:26:46.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:26:46.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:26:46.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:26:46.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:26:46.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: ** Compaction Stats [default] ** 2024-02-15T18:26:46.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:26:46.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:26:46.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: L6 1/0 22.65 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:26:46.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Sum 1/0 22.65 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:26:46.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:26:46.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: ** Compaction Stats [default] ** 2024-02-15T18:26:46.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:26:46.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:26:46.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:26:46.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-15T18:26:46.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:26:46.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:26:46.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:26:46.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:26:46.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:26:46.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:26:46.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:26:46.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:26:46.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.917+0000 7fd948789880 0 starting mon.smithi042 rank 0 at public addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] at bind addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi042 fsid 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:26:46.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.918+0000 7fd948789880 1 mon.smithi042@-1(???) e2 preinit fsid 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:26:46.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.918+0000 7fd948789880 0 mon.smithi042@-1(???).mds e13 new map 2024-02-15T18:26:46.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.918+0000 7fd948789880 0 mon.smithi042@-1(???).mds e13 print_map 2024-02-15T18:26:46.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: e13 2024-02-15T18:26:46.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:26:46.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:26:46.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: legacy client fscid: 1 2024-02-15T18:26:46.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Filesystem 'cephfs' (1) 2024-02-15T18:26:46.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: fs_name cephfs 2024-02-15T18:26:46.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: epoch 11 2024-02-15T18:26:46.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: flags 32 2024-02-15T18:26:46.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:26:46.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:26:46.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: tableserver 0 2024-02-15T18:26:46.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: root 0 2024-02-15T18:26:46.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: session_timeout 60 2024-02-15T18:26:46.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: session_autoclose 300 2024-02-15T18:26:46.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: max_file_size 1099511627776 2024-02-15T18:26:46.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: required_client_features {} 2024-02-15T18:26:46.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: last_failure 0 2024-02-15T18:26:46.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: last_failure_osd_epoch 0 2024-02-15T18:26:46.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:26:46.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: max_mds 1 2024-02-15T18:26:46.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: in 0 2024-02-15T18:26:46.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: up {0=24263} 2024-02-15T18:26:46.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: failed 2024-02-15T18:26:46.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: damaged 2024-02-15T18:26:46.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: stopped 2024-02-15T18:26:46.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: data_pools [3] 2024-02-15T18:26:46.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: metadata_pool 2 2024-02-15T18:26:46.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: inline_data disabled 2024-02-15T18:26:46.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: balancer 2024-02-15T18:26:46.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: standby_count_wanted 1 2024-02-15T18:26:46.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: [mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:26:46.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: [mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:26:46.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: Standby daemons: 2024-02-15T18:26:46.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: [mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:26:46.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: [mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:26:46.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: 2024-02-15T18:26:46.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.919+0000 7fd948789880 0 mon.smithi042@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-15T18:26:46.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.919+0000 7fd948789880 0 mon.smithi042@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-15T18:26:46.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.919+0000 7fd948789880 0 mon.smithi042@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-15T18:26:46.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.919+0000 7fd948789880 0 mon.smithi042@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-15T18:26:46.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:26:42.389222+0000 mgr.smithi042.kfoihb (mgr.24435) 27 : cephadm [INF] Upgrade: Updating mon.smithi042 2024-02-15T18:26:46.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:42.393118+0000 mon.smithi042 (mon.0) 668 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:26:46.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:42.393811+0000 mon.smithi042 (mon.0) 669 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:26:46.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:42.394391+0000 mon.smithi042 (mon.0) 670 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:26:46.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:42.394947+0000 mon.smithi042 (mon.0) 671 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:26:46.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:26:42.395479+0000 mgr.smithi042.kfoihb (mgr.24435) 28 : cephadm [INF] Deploying daemon mon.smithi042 on smithi042 2024-02-15T18:26:46.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:42.678067+0000 mgr.smithi042.kfoihb (mgr.24435) 29 : cluster [DBG] pgmap v20: 65 pgs: 65 active+clean; 242 MiB data, 927 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s rd, 14 MiB/s wr, 851 op/s 2024-02-15T18:26:46.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:26:45.919+0000 7fd948789880 1 mon.smithi042@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-02-15T18:26:46.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 bash[77631]: ff538d2bc6d9de4a32e253d379240c6ea4c6fbeb56e5c5c8c15e07dca6e8803c 2024-02-15T18:26:46.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:45 smithi042 systemd[1]: Started Ceph mon.smithi042 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:26:47.361 INFO:tasks.workunit.client.0.smithi042.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-15T18:26:47.361 INFO:tasks.workunit.client.0.smithi042.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-15T18:26:47.361 INFO:tasks.workunit.client.0.smithi042.stderr:+ make 2024-02-15T18:26:47.457 INFO:tasks.workunit.client.0.smithi042.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-15T18:26:47.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:44.078877+0000 mon.smithi191 (mon.1) 69 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-02-15T18:26:47.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:44.079975+0000 mon.smithi191 (mon.1) 70 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:26:47.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:44.678404+0000 mgr.smithi042.kfoihb (mgr.24435) 30 : cluster [DBG] pgmap v21: 65 pgs: 65 active+clean; 242 MiB data, 927 MiB used, 536 GiB / 536 GiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 833 op/s 2024-02-15T18:26:47.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:46.678870+0000 mgr.smithi042.kfoihb (mgr.24435) 31 : cluster [DBG] pgmap v22: 65 pgs: 65 active+clean; 242 MiB data, 927 MiB used, 536 GiB / 536 GiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 833 op/s 2024-02-15T18:26:47.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:47.330902+0000 mon.smithi042 (mon.0) 1 : cluster [INF] mon.smithi042 calling monitor election 2024-02-15T18:26:47.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:47.343903+0000 mon.smithi042 (mon.0) 2 : cluster [INF] mon.smithi042 is new leader, mons smithi042,smithi191 in quorum (ranks 0,1) 2024-02-15T18:26:47.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:47.349078+0000 mon.smithi042 (mon.0) 3 : cluster [DBG] monmap e2: 2 mons at {smithi042=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],smithi191=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} 2024-02-15T18:26:47.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:47.349557+0000 mon.smithi042 (mon.0) 4 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 2 up:standby 2024-02-15T18:26:47.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:47.349587+0000 mon.smithi042 (mon.0) 5 : cluster [DBG] osdmap e48: 6 total, 6 up, 6 in 2024-02-15T18:26:47.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:47.350547+0000 mon.smithi042 (mon.0) 6 : cluster [DBG] mgrmap e29: smithi042.kfoihb(active, since 37s), standbys: smithi191.gftdzo 2024-02-15T18:26:47.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:47.358008+0000 mon.smithi042 (mon.0) 7 : cluster [INF] overall HEALTH_OK 2024-02-15T18:26:47.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:47.358176+0000 mon.smithi042 (mon.0) 8 : audit [INF] from='mgr.24435 ' entity='' 2024-02-15T18:26:47.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:47.362136+0000 mon.smithi191 (mon.1) 71 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:47.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:44.078877+0000 mon.smithi191 (mon.1) 69 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-02-15T18:26:47.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:44.079975+0000 mon.smithi191 (mon.1) 70 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:26:47.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:44.678404+0000 mgr.smithi042.kfoihb (mgr.24435) 30 : cluster [DBG] pgmap v21: 65 pgs: 65 active+clean; 242 MiB data, 927 MiB used, 536 GiB / 536 GiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 833 op/s 2024-02-15T18:26:47.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:46.678870+0000 mgr.smithi042.kfoihb (mgr.24435) 31 : cluster [DBG] pgmap v22: 65 pgs: 65 active+clean; 242 MiB data, 927 MiB used, 536 GiB / 536 GiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 833 op/s 2024-02-15T18:26:47.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:47.330902+0000 mon.smithi042 (mon.0) 1 : cluster [INF] mon.smithi042 calling monitor election 2024-02-15T18:26:47.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:47.343903+0000 mon.smithi042 (mon.0) 2 : cluster [INF] mon.smithi042 is new leader, mons smithi042,smithi191 in quorum (ranks 0,1) 2024-02-15T18:26:47.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:47.349078+0000 mon.smithi042 (mon.0) 3 : cluster [DBG] monmap e2: 2 mons at {smithi042=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],smithi191=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} 2024-02-15T18:26:47.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:47.349557+0000 mon.smithi042 (mon.0) 4 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 2 up:standby 2024-02-15T18:26:47.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:47.349587+0000 mon.smithi042 (mon.0) 5 : cluster [DBG] osdmap e48: 6 total, 6 up, 6 in 2024-02-15T18:26:47.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:47.350547+0000 mon.smithi042 (mon.0) 6 : cluster [DBG] mgrmap e29: smithi042.kfoihb(active, since 37s), standbys: smithi191.gftdzo 2024-02-15T18:26:47.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:47.358008+0000 mon.smithi042 (mon.0) 7 : cluster [INF] overall HEALTH_OK 2024-02-15T18:26:47.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:47.358176+0000 mon.smithi042 (mon.0) 8 : audit [INF] from='mgr.24435 ' entity='' 2024-02-15T18:26:47.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:47.362136+0000 mon.smithi191 (mon.1) 71 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:26:48.507 INFO:tasks.workunit.client.0.smithi042.stderr:++ readlink -f fsstress 2024-02-15T18:26:48.508 INFO:tasks.workunit.client.0.smithi042.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-15T18:26:48.508 INFO:tasks.workunit.client.0.smithi042.stderr:+ popd 2024-02-15T18:26:48.508 INFO:tasks.workunit.client.0.smithi042.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-15T18:26:48.508 INFO:tasks.workunit.client.0.smithi042.stderr:+ popd 2024-02-15T18:26:48.509 INFO:tasks.workunit.client.0.smithi042.stdout:~/cephtest/mnt.0/client.0/tmp 2024-02-15T18:26:48.509 INFO:tasks.workunit.client.0.smithi042.stderr:++ mktemp -d -p . 2024-02-15T18:26:48.510 INFO:tasks.workunit.client.0.smithi042.stderr:+ T=./tmp.JrUCqlHuGi 2024-02-15T18:26:48.510 INFO:tasks.workunit.client.0.smithi042.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.JrUCqlHuGi -l 1 -n 1000 -p 10 -v 2024-02-15T18:26:48.511 INFO:tasks.workunit.client.0.smithi042.stdout:seed = 1707718556 2024-02-15T18:26:48.514 INFO:tasks.workunit.client.0.smithi042.stdout:0/0: dwrite - no filename 2024-02-15T18:26:48.514 INFO:tasks.workunit.client.0.smithi042.stdout:0/1: truncate - no filename 2024-02-15T18:26:48.514 INFO:tasks.workunit.client.0.smithi042.stdout:1/0: truncate - no filename 2024-02-15T18:26:48.514 INFO:tasks.workunit.client.0.smithi042.stdout:1/1: dwrite - no filename 2024-02-15T18:26:48.514 INFO:tasks.workunit.client.0.smithi042.stdout:1/2: unlink - no file 2024-02-15T18:26:48.515 INFO:tasks.workunit.client.0.smithi042.stdout:0/2: creat f0 x:0 0 0 2024-02-15T18:26:48.515 INFO:tasks.workunit.client.0.smithi042.stdout:0/3: dread - f0 zero size 2024-02-15T18:26:48.516 INFO:tasks.workunit.client.0.smithi042.stdout:2/0: readlink - no filename 2024-02-15T18:26:48.516 INFO:tasks.workunit.client.0.smithi042.stdout:2/1: readlink - no filename 2024-02-15T18:26:48.516 INFO:tasks.workunit.client.0.smithi042.stdout:2/2: unlink - no file 2024-02-15T18:26:48.516 INFO:tasks.workunit.client.0.smithi042.stdout:2/3: dread - no filename 2024-02-15T18:26:48.516 INFO:tasks.workunit.client.0.smithi042.stdout:0/4: creat f1 x:0 0 0 2024-02-15T18:26:48.517 INFO:tasks.workunit.client.0.smithi042.stdout:3/0: readlink - no filename 2024-02-15T18:26:48.517 INFO:tasks.workunit.client.0.smithi042.stdout:2/4: mkdir d0 0 2024-02-15T18:26:48.518 INFO:tasks.workunit.client.0.smithi042.stdout:4/0: link - no file 2024-02-15T18:26:48.518 INFO:tasks.workunit.client.0.smithi042.stdout:4/1: readlink - no filename 2024-02-15T18:26:48.518 INFO:tasks.workunit.client.0.smithi042.stdout:4/2: dwrite - no filename 2024-02-15T18:26:48.519 INFO:tasks.workunit.client.0.smithi042.stdout:3/1: symlink l0 0 2024-02-15T18:26:48.519 INFO:tasks.workunit.client.0.smithi042.stdout:3/2: dread - no filename 2024-02-15T18:26:48.519 INFO:tasks.workunit.client.0.smithi042.stdout:2/5: mknod d0/c1 0 2024-02-15T18:26:48.520 INFO:tasks.workunit.client.0.smithi042.stdout:5/0: write - no filename 2024-02-15T18:26:48.520 INFO:tasks.workunit.client.0.smithi042.stdout:5/1: rename - no filename 2024-02-15T18:26:48.520 INFO:tasks.workunit.client.0.smithi042.stdout:5/2: readlink - no filename 2024-02-15T18:26:48.520 INFO:tasks.workunit.client.0.smithi042.stdout:5/3: unlink - no file 2024-02-15T18:26:48.520 INFO:tasks.workunit.client.0.smithi042.stdout:5/4: truncate - no filename 2024-02-15T18:26:48.521 INFO:tasks.workunit.client.0.smithi042.stdout:5/5: rename - no filename 2024-02-15T18:26:48.521 INFO:tasks.workunit.client.0.smithi042.stdout:5/6: stat - no entries 2024-02-15T18:26:48.521 INFO:tasks.workunit.client.0.smithi042.stdout:5/7: write - no filename 2024-02-15T18:26:48.521 INFO:tasks.workunit.client.0.smithi042.stdout:5/8: dread - no filename 2024-02-15T18:26:48.521 INFO:tasks.workunit.client.0.smithi042.stdout:4/3: mknod c0 0 2024-02-15T18:26:48.521 INFO:tasks.workunit.client.0.smithi042.stdout:4/4: truncate - no filename 2024-02-15T18:26:48.521 INFO:tasks.workunit.client.0.smithi042.stdout:4/5: dwrite - no filename 2024-02-15T18:26:48.522 INFO:tasks.workunit.client.0.smithi042.stdout:3/3: mkdir d1 0 2024-02-15T18:26:48.522 INFO:tasks.workunit.client.0.smithi042.stdout:3/4: write - no filename 2024-02-15T18:26:48.522 INFO:tasks.workunit.client.0.smithi042.stdout:3/5: chown d1 1794164 1 2024-02-15T18:26:48.522 INFO:tasks.workunit.client.0.smithi042.stdout:2/6: rename d0/c1 to d0/c2 0 2024-02-15T18:26:48.523 INFO:tasks.workunit.client.0.smithi042.stdout:6/0: write - no filename 2024-02-15T18:26:48.526 INFO:tasks.workunit.client.0.smithi042.stdout:5/9: mkdir d0 0 2024-02-15T18:26:48.526 INFO:tasks.workunit.client.0.smithi042.stdout:4/6: link c0 c1 0 2024-02-15T18:26:48.526 INFO:tasks.workunit.client.0.smithi042.stdout:4/7: write - no filename 2024-02-15T18:26:48.526 INFO:tasks.workunit.client.0.smithi042.stdout:4/8: dwrite - no filename 2024-02-15T18:26:48.526 INFO:tasks.workunit.client.0.smithi042.stdout:4/9: dwrite - no filename 2024-02-15T18:26:48.526 INFO:tasks.workunit.client.0.smithi042.stdout:5/10: dwrite - no filename 2024-02-15T18:26:48.526 INFO:tasks.workunit.client.0.smithi042.stdout:4/10: readlink - no filename 2024-02-15T18:26:48.526 INFO:tasks.workunit.client.0.smithi042.stdout:5/11: write - no filename 2024-02-15T18:26:48.526 INFO:tasks.workunit.client.0.smithi042.stdout:3/6: mknod d1/c2 0 2024-02-15T18:26:48.526 INFO:tasks.workunit.client.0.smithi042.stdout:3/7: dwrite - no filename 2024-02-15T18:26:48.530 INFO:tasks.workunit.client.0.smithi042.stdout:7/0: dwrite - no filename 2024-02-15T18:26:48.530 INFO:tasks.workunit.client.0.smithi042.stdout:7/1: rmdir - no directory 2024-02-15T18:26:48.531 INFO:tasks.workunit.client.0.smithi042.stdout:7/2: dwrite - no filename 2024-02-15T18:26:48.531 INFO:tasks.workunit.client.0.smithi042.stdout:7/3: fsync - no filename 2024-02-15T18:26:48.531 INFO:tasks.workunit.client.0.smithi042.stdout:2/7: rmdir d0 39 2024-02-15T18:26:48.532 INFO:tasks.workunit.client.0.smithi042.stdout:6/1: mkdir d0 0 2024-02-15T18:26:48.534 INFO:tasks.workunit.client.0.smithi042.stdout:4/11: unlink c1 0 2024-02-15T18:26:48.534 INFO:tasks.workunit.client.0.smithi042.stdout:4/12: readlink - no filename 2024-02-15T18:26:48.534 INFO:tasks.workunit.client.0.smithi042.stdout:4/13: dwrite - no filename 2024-02-15T18:26:48.534 INFO:tasks.workunit.client.0.smithi042.stdout:4/14: dwrite - no filename 2024-02-15T18:26:48.535 INFO:tasks.workunit.client.0.smithi042.stdout:4/15: readlink - no filename 2024-02-15T18:26:48.535 INFO:tasks.workunit.client.0.smithi042.stdout:5/12: symlink d0/l1 0 2024-02-15T18:26:48.535 INFO:tasks.workunit.client.0.smithi042.stdout:5/13: dread - no filename 2024-02-15T18:26:48.542 INFO:tasks.workunit.client.0.smithi042.stdout:7/4: symlink l0 0 2024-02-15T18:26:48.542 INFO:tasks.workunit.client.0.smithi042.stdout:2/8: creat d0/f3 x:0 0 0 2024-02-15T18:26:48.542 INFO:tasks.workunit.client.0.smithi042.stdout:2/9: dread - d0/f3 zero size 2024-02-15T18:26:48.542 INFO:tasks.workunit.client.0.smithi042.stdout:2/10: fdatasync d0/f3 0 2024-02-15T18:26:48.544 INFO:tasks.workunit.client.0.smithi042.stdout:4/16: link c0 c2 0 2024-02-15T18:26:48.544 INFO:tasks.workunit.client.0.smithi042.stdout:4/17: chown c0 0 1 2024-02-15T18:26:48.544 INFO:tasks.workunit.client.0.smithi042.stdout:4/18: truncate - no filename 2024-02-15T18:26:48.544 INFO:tasks.workunit.client.0.smithi042.stdout:4/19: fsync - no filename 2024-02-15T18:26:48.544 INFO:tasks.workunit.client.0.smithi042.stdout:4/20: write - no filename 2024-02-15T18:26:48.544 INFO:tasks.workunit.client.0.smithi042.stdout:4/21: write - no filename 2024-02-15T18:26:48.546 INFO:tasks.workunit.client.0.smithi042.stdout:6/2: mknod d0/c1 0 2024-02-15T18:26:48.546 INFO:tasks.workunit.client.0.smithi042.stdout:3/8: rmdir d1 39 2024-02-15T18:26:48.546 INFO:tasks.workunit.client.0.smithi042.stdout:7/5: unlink l0 0 2024-02-15T18:26:48.547 INFO:tasks.workunit.client.0.smithi042.stdout:4/22: mknod c3 0 2024-02-15T18:26:48.552 INFO:tasks.workunit.client.0.smithi042.stdout:7/6: creat f1 x:0 0 0 2024-02-15T18:26:48.552 INFO:tasks.workunit.client.0.smithi042.stdout:7/7: write f1 [527796,11536] 0 2024-02-15T18:26:48.552 INFO:tasks.workunit.client.0.smithi042.stdout:6/3: unlink d0/c1 0 2024-02-15T18:26:48.552 INFO:tasks.workunit.client.0.smithi042.stdout:6/4: link - no file 2024-02-15T18:26:48.552 INFO:tasks.workunit.client.0.smithi042.stdout:6/5: truncate - no filename 2024-02-15T18:26:48.554 INFO:tasks.workunit.client.0.smithi042.stdout:3/9: rename d1/c2 to d1/c3 0 2024-02-15T18:26:48.554 INFO:tasks.workunit.client.0.smithi042.stdout:7/8: mknod c2 0 2024-02-15T18:26:48.558 INFO:tasks.workunit.client.0.smithi042.stdout:6/6: mknod d0/c2 0 2024-02-15T18:26:48.558 INFO:tasks.workunit.client.0.smithi042.stdout:6/7: truncate - no filename 2024-02-15T18:26:48.558 INFO:tasks.workunit.client.0.smithi042.stdout:6/8: dread - no filename 2024-02-15T18:26:48.558 INFO:tasks.workunit.client.0.smithi042.stdout:6/9: rename d0 to d0/d3 22 2024-02-15T18:26:48.659 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:47.409245+0000 mon.smithi042 (mon.0) 9 : cluster [DBG] mgrmap e30: smithi042.kfoihb(active, since 37s), standbys: smithi191.gftdzo 2024-02-15T18:26:48.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:47.409245+0000 mon.smithi042 (mon.0) 9 : cluster [DBG] mgrmap e30: smithi042.kfoihb(active, since 37s), standbys: smithi191.gftdzo 2024-02-15T18:26:48.821 INFO:tasks.workunit.client.0.smithi042.stdout:0/5: dwrite f0 [0,4194304] 0 2024-02-15T18:26:48.821 INFO:tasks.workunit.client.0.smithi042.stdout:0/6: readlink - no filename 2024-02-15T18:26:48.821 INFO:tasks.workunit.client.0.smithi042.stdout:0/7: write f1 [581943,44983] 0 2024-02-15T18:26:48.851 INFO:tasks.workunit.client.0.smithi042.stdout:7/9: dread f1 [0,4194304] 0 2024-02-15T18:26:48.944 INFO:tasks.workunit.client.0.smithi042.stdout:9/0: dread - no filename 2024-02-15T18:26:48.991 INFO:tasks.workunit.client.0.smithi042.stdout:3/10: rmdir d1 39 2024-02-15T18:26:48.991 INFO:tasks.workunit.client.0.smithi042.stdout:3/11: chown d1 6 1 2024-02-15T18:26:48.991 INFO:tasks.workunit.client.0.smithi042.stdout:3/12: write - no filename 2024-02-15T18:26:48.997 INFO:tasks.workunit.client.0.smithi042.stdout:2/11: link d0/c2 d0/c4 0 2024-02-15T18:26:49.007 INFO:tasks.workunit.client.0.smithi042.stdout:6/10: getdents d0 0 2024-02-15T18:26:49.007 INFO:tasks.workunit.client.0.smithi042.stdout:6/11: dwrite - no filename 2024-02-15T18:26:49.007 INFO:tasks.workunit.client.0.smithi042.stdout:6/12: getdents d0 0 2024-02-15T18:26:49.007 INFO:tasks.workunit.client.0.smithi042.stdout:6/13: write - no filename 2024-02-15T18:26:49.007 INFO:tasks.workunit.client.0.smithi042.stdout:6/14: dread - no filename 2024-02-15T18:26:49.175 INFO:tasks.workunit.client.0.smithi042.stdout:0/8: creat f2 x:0 0 0 2024-02-15T18:26:49.505 INFO:tasks.workunit.client.0.smithi042.stdout:7/10: getdents . 0 2024-02-15T18:26:49.696 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:48.679364+0000 mgr.smithi042.kfoihb (mgr.24435) 32 : cluster [DBG] pgmap v23: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 881 op/s 2024-02-15T18:26:49.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:48.679364+0000 mgr.smithi042.kfoihb (mgr.24435) 32 : cluster [DBG] pgmap v23: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 6.5 MiB/s rd, 14 MiB/s wr, 881 op/s 2024-02-15T18:26:49.924 INFO:tasks.workunit.client.0.smithi042.stdout:8/0: mkdir d0 0 2024-02-15T18:26:49.925 INFO:tasks.workunit.client.0.smithi042.stdout:9/1: creat f0 x:0 0 0 2024-02-15T18:26:49.925 INFO:tasks.workunit.client.0.smithi042.stdout:9/2: dread - f0 zero size 2024-02-15T18:26:50.043 INFO:tasks.workunit.client.0.smithi042.stdout:3/13: mkdir d1/d4 0 2024-02-15T18:26:50.043 INFO:tasks.workunit.client.0.smithi042.stdout:3/14: chown d1 1503749897 1 2024-02-15T18:26:50.044 INFO:tasks.workunit.client.0.smithi042.stdout:2/12: creat d0/f5 x:0 0 0 2024-02-15T18:26:50.044 INFO:tasks.workunit.client.0.smithi042.stdout:2/13: dread - d0/f5 zero size 2024-02-15T18:26:50.059 INFO:tasks.workunit.client.0.smithi042.stdout:6/15: unlink d0/c2 0 2024-02-15T18:26:50.127 INFO:tasks.workunit.client.0.smithi042.stdout:0/9: symlink l3 0 2024-02-15T18:26:50.707 INFO:tasks.workunit.client.0.smithi042.stdout:9/3: link f0 f1 0 2024-02-15T18:26:50.707 INFO:tasks.workunit.client.0.smithi042.stdout:9/4: truncate f1 190935 0 2024-02-15T18:26:50.708 INFO:tasks.workunit.client.0.smithi042.stdout:9/5: write f1 [1036170,119485] 0 2024-02-15T18:26:50.708 INFO:tasks.workunit.client.0.smithi042.stdout:3/15: rmdir d1/d4 0 2024-02-15T18:26:50.711 INFO:tasks.workunit.client.0.smithi042.stdout:0/10: unlink f0 0 2024-02-15T18:26:50.712 INFO:tasks.workunit.client.0.smithi042.stdout:9/6: unlink f0 0 2024-02-15T18:26:50.713 INFO:tasks.workunit.client.0.smithi042.stdout:3/16: mknod d1/c5 0 2024-02-15T18:26:50.713 INFO:tasks.workunit.client.0.smithi042.stdout:3/17: chown d1 4666 1 2024-02-15T18:26:50.714 INFO:tasks.workunit.client.0.smithi042.stdout:9/7: creat f2 x:0 0 0 2024-02-15T18:26:50.714 INFO:tasks.workunit.client.0.smithi042.stdout:3/18: mknod d1/c6 0 2024-02-15T18:26:50.714 INFO:tasks.workunit.client.0.smithi042.stdout:3/19: dread - no filename 2024-02-15T18:26:50.714 INFO:tasks.workunit.client.0.smithi042.stdout:3/20: write - no filename 2024-02-15T18:26:50.714 INFO:tasks.workunit.client.0.smithi042.stdout:3/21: dread - no filename 2024-02-15T18:26:50.718 INFO:tasks.workunit.client.0.smithi042.stdout:9/8: creat f3 x:0 0 0 2024-02-15T18:26:50.726 INFO:tasks.workunit.client.0.smithi042.stdout:3/22: creat d1/f7 x:0 0 0 2024-02-15T18:26:50.726 INFO:tasks.workunit.client.0.smithi042.stdout:3/23: chown d1/c3 593515 1 2024-02-15T18:26:50.730 INFO:tasks.workunit.client.0.smithi042.stdout:9/9: creat f4 x:0 0 0 2024-02-15T18:26:50.730 INFO:tasks.workunit.client.0.smithi042.stdout:0/11: dwrite f2 [0,4194304] 0 2024-02-15T18:26:50.736 INFO:tasks.workunit.client.0.smithi042.stdout:9/10: link f3 f5 0 2024-02-15T18:26:50.736 INFO:tasks.workunit.client.0.smithi042.stdout:9/11: truncate f2 868406 0 2024-02-15T18:26:50.736 INFO:tasks.workunit.client.0.smithi042.stdout:9/12: read - f3 zero size 2024-02-15T18:26:50.736 INFO:tasks.workunit.client.0.smithi042.stdout:9/13: read - f3 zero size 2024-02-15T18:26:50.744 INFO:tasks.workunit.client.0.smithi042.stdout:3/24: dwrite d1/f7 [0,4194304] 0 2024-02-15T18:26:50.988 INFO:tasks.workunit.client.0.smithi042.stdout:0/12: mkdir d4 0 2024-02-15T18:26:51.029 INFO:tasks.workunit.client.0.smithi042.stdout:0/13: mknod d4/c5 0 2024-02-15T18:26:51.029 INFO:tasks.workunit.client.0.smithi042.stdout:0/14: chown l3 30 1 2024-02-15T18:26:51.046 INFO:tasks.workunit.client.0.smithi042.stdout:6/16: rmdir d0 0 2024-02-15T18:26:51.057 INFO:tasks.workunit.client.0.smithi042.stdout:0/15: rename f2 to d4/f6 0 2024-02-15T18:26:51.072 INFO:tasks.workunit.client.0.smithi042.stdout:6/17: creat f4 x:0 0 0 2024-02-15T18:26:51.079 INFO:tasks.workunit.client.0.smithi042.stdout:0/16: unlink f1 0 2024-02-15T18:26:51.116 INFO:tasks.workunit.client.0.smithi042.stdout:0/17: creat d4/f7 x:0 0 0 2024-02-15T18:26:51.126 INFO:tasks.workunit.client.0.smithi042.stdout:5/14: sync 2024-02-15T18:26:51.126 INFO:tasks.workunit.client.0.smithi042.stdout:4/23: sync 2024-02-15T18:26:51.126 INFO:tasks.workunit.client.0.smithi042.stdout:1/3: sync 2024-02-15T18:26:51.126 INFO:tasks.workunit.client.0.smithi042.stdout:1/4: link - no file 2024-02-15T18:26:51.127 INFO:tasks.workunit.client.0.smithi042.stdout:1/5: write - no filename 2024-02-15T18:26:51.127 INFO:tasks.workunit.client.0.smithi042.stdout:7/11: sync 2024-02-15T18:26:51.130 INFO:tasks.workunit.client.0.smithi042.stdout:3/25: truncate d1/f7 1293158 0 2024-02-15T18:26:51.132 INFO:tasks.workunit.client.0.smithi042.stdout:8/1: sync 2024-02-15T18:26:51.132 INFO:tasks.workunit.client.0.smithi042.stdout:8/2: truncate - no filename 2024-02-15T18:26:51.132 INFO:tasks.workunit.client.0.smithi042.stdout:8/3: dread - no filename 2024-02-15T18:26:51.132 INFO:tasks.workunit.client.0.smithi042.stdout:8/4: write - no filename 2024-02-15T18:26:51.132 INFO:tasks.workunit.client.0.smithi042.stdout:2/14: sync 2024-02-15T18:26:51.134 INFO:tasks.workunit.client.0.smithi042.stdout:6/18: fsync f4 0 2024-02-15T18:26:51.135 INFO:tasks.workunit.client.0.smithi042.stdout:9/14: getdents . 0 2024-02-15T18:26:51.135 INFO:tasks.workunit.client.0.smithi042.stdout:6/19: dread - f4 zero size 2024-02-15T18:26:51.135 INFO:tasks.workunit.client.0.smithi042.stdout:6/20: dread - f4 zero size 2024-02-15T18:26:51.135 INFO:tasks.workunit.client.0.smithi042.stdout:6/21: write f4 [400362,73106] 0 2024-02-15T18:26:51.135 INFO:tasks.workunit.client.0.smithi042.stdout:0/18: creat d4/f8 x:0 0 0 2024-02-15T18:26:51.135 INFO:tasks.workunit.client.0.smithi042.stdout:3/26: fsync d1/f7 0 2024-02-15T18:26:51.137 INFO:tasks.workunit.client.0.smithi042.stdout:5/15: symlink d0/l2 0 2024-02-15T18:26:51.137 INFO:tasks.workunit.client.0.smithi042.stdout:5/16: chown d0/l1 85 1 2024-02-15T18:26:51.137 INFO:tasks.workunit.client.0.smithi042.stdout:5/17: write - no filename 2024-02-15T18:26:51.137 INFO:tasks.workunit.client.0.smithi042.stdout:5/18: write - no filename 2024-02-15T18:26:51.138 INFO:tasks.workunit.client.0.smithi042.stdout:5/19: truncate - no filename 2024-02-15T18:26:51.138 INFO:tasks.workunit.client.0.smithi042.stdout:1/6: mknod c0 0 2024-02-15T18:26:51.138 INFO:tasks.workunit.client.0.smithi042.stdout:7/12: mknod c3 0 2024-02-15T18:26:51.142 INFO:tasks.workunit.client.0.smithi042.stdout:8/5: rmdir d0 0 2024-02-15T18:26:51.142 INFO:tasks.workunit.client.0.smithi042.stdout:8/6: write - no filename 2024-02-15T18:26:51.142 INFO:tasks.workunit.client.0.smithi042.stdout:2/15: rmdir d0 39 2024-02-15T18:26:51.147 INFO:tasks.workunit.client.0.smithi042.stdout:6/22: rename f4 to f5 0 2024-02-15T18:26:51.147 INFO:tasks.workunit.client.0.smithi042.stdout:6/23: write f5 [940063,26653] 0 2024-02-15T18:26:51.148 INFO:tasks.workunit.client.0.smithi042.stdout:0/19: mkdir d4/d9 0 2024-02-15T18:26:51.149 INFO:tasks.workunit.client.0.smithi042.stdout:3/27: rename d1/c5 to d1/c8 0 2024-02-15T18:26:51.149 INFO:tasks.workunit.client.0.smithi042.stdout:3/28: fdatasync d1/f7 0 2024-02-15T18:26:51.151 INFO:tasks.workunit.client.0.smithi042.stdout:6/24: dread f5 [0,4194304] 0 2024-02-15T18:26:51.154 INFO:tasks.workunit.client.0.smithi042.stdout:1/7: mknod c1 0 2024-02-15T18:26:51.154 INFO:tasks.workunit.client.0.smithi042.stdout:1/8: fsync - no filename 2024-02-15T18:26:51.155 INFO:tasks.workunit.client.0.smithi042.stdout:1/9: write - no filename 2024-02-15T18:26:51.155 INFO:tasks.workunit.client.0.smithi042.stdout:1/10: dwrite - no filename 2024-02-15T18:26:51.155 INFO:tasks.workunit.client.0.smithi042.stdout:1/11: chown c1 266 1 2024-02-15T18:26:51.155 INFO:tasks.workunit.client.0.smithi042.stdout:1/12: dread - no filename 2024-02-15T18:26:51.155 INFO:tasks.workunit.client.0.smithi042.stdout:1/13: chown c1 5 1 2024-02-15T18:26:51.155 INFO:tasks.workunit.client.0.smithi042.stdout:1/14: write - no filename 2024-02-15T18:26:51.155 INFO:tasks.workunit.client.0.smithi042.stdout:1/15: write - no filename 2024-02-15T18:26:51.156 INFO:tasks.workunit.client.0.smithi042.stdout:7/13: getdents . 0 2024-02-15T18:26:51.158 INFO:tasks.workunit.client.0.smithi042.stdout:1/16: unlink c0 0 2024-02-15T18:26:51.158 INFO:tasks.workunit.client.0.smithi042.stdout:7/14: write f1 [71372,49355] 0 2024-02-15T18:26:51.158 INFO:tasks.workunit.client.0.smithi042.stdout:7/15: write f1 [1233475,28961] 0 2024-02-15T18:26:51.158 INFO:tasks.workunit.client.0.smithi042.stdout:9/15: mkdir d6 0 2024-02-15T18:26:51.159 INFO:tasks.workunit.client.0.smithi042.stdout:1/17: creat f2 x:0 0 0 2024-02-15T18:26:51.161 INFO:tasks.workunit.client.0.smithi042.stdout:8/7: symlink l1 0 2024-02-15T18:26:51.161 INFO:tasks.workunit.client.0.smithi042.stdout:7/16: rename f1 to f4 0 2024-02-15T18:26:51.161 INFO:tasks.workunit.client.0.smithi042.stdout:3/29: dread d1/f7 [0,4194304] 0 2024-02-15T18:26:51.162 INFO:tasks.workunit.client.0.smithi042.stdout:8/8: mknod c2 0 2024-02-15T18:26:51.163 INFO:tasks.workunit.client.0.smithi042.stdout:7/17: creat f5 x:0 0 0 2024-02-15T18:26:51.164 INFO:tasks.workunit.client.0.smithi042.stdout:8/9: mknod c3 0 2024-02-15T18:26:51.166 INFO:tasks.workunit.client.0.smithi042.stdout:7/18: mknod c6 0 2024-02-15T18:26:51.167 INFO:tasks.workunit.client.0.smithi042.stdout:2/16: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:26:51.168 INFO:tasks.workunit.client.0.smithi042.stdout:9/16: write f1 [2123111,30408] 0 2024-02-15T18:26:51.171 INFO:tasks.workunit.client.0.smithi042.stdout:8/10: symlink l4 0 2024-02-15T18:26:51.171 INFO:tasks.workunit.client.0.smithi042.stdout:8/11: read - no filename 2024-02-15T18:26:51.171 INFO:tasks.workunit.client.0.smithi042.stdout:8/12: dread - no filename 2024-02-15T18:26:51.172 INFO:tasks.workunit.client.0.smithi042.stdout:7/19: creat f7 x:0 0 0 2024-02-15T18:26:51.175 INFO:tasks.workunit.client.0.smithi042.stdout:2/17: symlink d0/l6 0 2024-02-15T18:26:51.175 INFO:tasks.workunit.client.0.smithi042.stdout:2/18: fsync d0/f3 0 2024-02-15T18:26:51.176 INFO:tasks.workunit.client.0.smithi042.stdout:9/17: mkdir d6/d7 0 2024-02-15T18:26:51.181 INFO:tasks.workunit.client.0.smithi042.stdout:9/18: rename f1 to d6/f8 0 2024-02-15T18:26:51.182 INFO:tasks.workunit.client.0.smithi042.stdout:9/19: mkdir d6/d7/d9 0 2024-02-15T18:26:51.183 INFO:tasks.workunit.client.0.smithi042.stdout:0/20: dwrite d4/f8 [0,4194304] 0 2024-02-15T18:26:51.184 INFO:tasks.workunit.client.0.smithi042.stdout:6/25: dwrite f5 [0,4194304] 0 2024-02-15T18:26:51.185 INFO:tasks.workunit.client.0.smithi042.stdout:9/20: dread - f5 zero size 2024-02-15T18:26:51.185 INFO:tasks.workunit.client.0.smithi042.stdout:9/21: chown f5 2845 1 2024-02-15T18:26:51.185 INFO:tasks.workunit.client.0.smithi042.stdout:9/22: write d6/f8 [2378322,130849] 0 2024-02-15T18:26:51.185 INFO:tasks.workunit.client.0.smithi042.stdout:9/23: chown d6 913135 1 2024-02-15T18:26:51.186 INFO:tasks.workunit.client.0.smithi042.stdout:1/18: getdents . 0 2024-02-15T18:26:51.187 INFO:tasks.workunit.client.0.smithi042.stdout:3/30: dwrite d1/f7 [0,4194304] 0 2024-02-15T18:26:51.187 INFO:tasks.workunit.client.0.smithi042.stdout:3/31: fdatasync d1/f7 0 2024-02-15T18:26:51.187 INFO:tasks.workunit.client.0.smithi042.stdout:3/32: readlink l0 0 2024-02-15T18:26:51.187 INFO:tasks.workunit.client.0.smithi042.stdout:3/33: fsync d1/f7 0 2024-02-15T18:26:51.188 INFO:tasks.workunit.client.0.smithi042.stdout:0/21: read d4/f6 [532269,100286] 0 2024-02-15T18:26:51.191 INFO:tasks.workunit.client.0.smithi042.stdout:2/19: write d0/f3 [4448473,112603] 0 2024-02-15T18:26:51.191 INFO:tasks.workunit.client.0.smithi042.stdout:2/20: truncate d0/f5 268894 0 2024-02-15T18:26:51.191 INFO:tasks.workunit.client.0.smithi042.stdout:2/21: readlink d0/l6 0 2024-02-15T18:26:51.193 INFO:tasks.workunit.client.0.smithi042.stdout:6/26: rename f5 to f6 0 2024-02-15T18:26:51.199 INFO:tasks.workunit.client.0.smithi042.stdout:9/24: mkdir d6/da 0 2024-02-15T18:26:51.200 INFO:tasks.workunit.client.0.smithi042.stdout:1/19: mknod c3 0 2024-02-15T18:26:51.211 INFO:tasks.workunit.client.0.smithi042.stdout:3/34: getdents d1 0 2024-02-15T18:26:51.211 INFO:tasks.workunit.client.0.smithi042.stdout:0/22: creat d4/d9/fa x:0 0 0 2024-02-15T18:26:51.211 INFO:tasks.workunit.client.0.smithi042.stdout:2/22: creat d0/f7 x:0 0 0 2024-02-15T18:26:51.211 INFO:tasks.workunit.client.0.smithi042.stdout:9/25: creat d6/fb x:0 0 0 2024-02-15T18:26:51.212 INFO:tasks.workunit.client.0.smithi042.stdout:1/20: chown c1 1961 1 2024-02-15T18:26:51.212 INFO:tasks.workunit.client.0.smithi042.stdout:2/23: dread d0/f5 [0,4194304] 0 2024-02-15T18:26:51.213 INFO:tasks.workunit.client.0.smithi042.stdout:6/27: dwrite f6 [0,4194304] 0 2024-02-15T18:26:51.213 INFO:tasks.workunit.client.0.smithi042.stdout:6/28: chown f6 14 1 2024-02-15T18:26:51.213 INFO:tasks.workunit.client.0.smithi042.stdout:0/23: rename d4/c5 to d4/cb 0 2024-02-15T18:26:51.214 INFO:tasks.workunit.client.0.smithi042.stdout:2/24: mkdir d0/d8 0 2024-02-15T18:26:51.216 INFO:tasks.workunit.client.0.smithi042.stdout:6/29: rename f6 to f7 0 2024-02-15T18:26:51.216 INFO:tasks.workunit.client.0.smithi042.stdout:6/30: chown f7 88 1 2024-02-15T18:26:51.216 INFO:tasks.workunit.client.0.smithi042.stdout:0/24: creat d4/fc x:0 0 0 2024-02-15T18:26:51.216 INFO:tasks.workunit.client.0.smithi042.stdout:6/31: write f7 [2400172,7059] 0 2024-02-15T18:26:51.217 INFO:tasks.workunit.client.0.smithi042.stdout:1/21: symlink l4 0 2024-02-15T18:26:51.217 INFO:tasks.workunit.client.0.smithi042.stdout:0/25: mknod d4/d9/cd 0 2024-02-15T18:26:51.217 INFO:tasks.workunit.client.0.smithi042.stdout:2/25: write d0/f3 [4118922,97573] 0 2024-02-15T18:26:51.218 INFO:tasks.workunit.client.0.smithi042.stdout:1/22: creat f5 x:0 0 0 2024-02-15T18:26:51.221 INFO:tasks.workunit.client.0.smithi042.stdout:2/26: rename d0/c2 to d0/c9 0 2024-02-15T18:26:51.221 INFO:tasks.workunit.client.0.smithi042.stdout:2/27: dread - d0/f7 zero size 2024-02-15T18:26:51.222 INFO:tasks.workunit.client.0.smithi042.stdout:1/23: creat f6 x:0 0 0 2024-02-15T18:26:51.223 INFO:tasks.workunit.client.0.smithi042.stdout:2/28: mkdir d0/d8/da 0 2024-02-15T18:26:51.232 INFO:tasks.workunit.client.0.smithi042.stdout:3/35: dread d1/f7 [0,4194304] 0 2024-02-15T18:26:51.233 INFO:tasks.workunit.client.0.smithi042.stdout:0/26: rmdir d4 39 2024-02-15T18:26:51.233 INFO:tasks.workunit.client.0.smithi042.stdout:2/29: read d0/f3 [3875938,127120] 0 2024-02-15T18:26:51.235 INFO:tasks.workunit.client.0.smithi042.stdout:9/26: dwrite f5 [0,4194304] 0 2024-02-15T18:26:51.235 INFO:tasks.workunit.client.0.smithi042.stdout:3/36: unlink l0 0 2024-02-15T18:26:51.236 INFO:tasks.workunit.client.0.smithi042.stdout:0/27: truncate d4/f6 3248782 0 2024-02-15T18:26:51.240 INFO:tasks.workunit.client.0.smithi042.stdout:9/27: symlink d6/lc 0 2024-02-15T18:26:51.241 INFO:tasks.workunit.client.0.smithi042.stdout:3/37: creat d1/f9 x:0 0 0 2024-02-15T18:26:51.243 INFO:tasks.workunit.client.0.smithi042.stdout:0/28: rename d4/d9/fa to d4/fe 0 2024-02-15T18:26:51.245 INFO:tasks.workunit.client.0.smithi042.stdout:9/28: unlink d6/lc 0 2024-02-15T18:26:51.245 INFO:tasks.workunit.client.0.smithi042.stdout:9/29: readlink - no filename 2024-02-15T18:26:51.245 INFO:tasks.workunit.client.0.smithi042.stdout:9/30: readlink - no filename 2024-02-15T18:26:51.245 INFO:tasks.workunit.client.0.smithi042.stdout:9/31: truncate d6/fb 907645 0 2024-02-15T18:26:51.245 INFO:tasks.workunit.client.0.smithi042.stdout:9/32: write d6/fb [964227,128273] 0 2024-02-15T18:26:51.252 INFO:tasks.workunit.client.0.smithi042.stdout:1/24: dwrite f5 [0,4194304] 0 2024-02-15T18:26:51.254 INFO:tasks.workunit.client.0.smithi042.stdout:6/32: dwrite f7 [0,4194304] 0 2024-02-15T18:26:51.255 INFO:tasks.workunit.client.0.smithi042.stdout:2/30: rmdir d0 39 2024-02-15T18:26:51.257 INFO:tasks.workunit.client.0.smithi042.stdout:0/29: write d4/f6 [3838707,86454] 0 2024-02-15T18:26:51.259 INFO:tasks.workunit.client.0.smithi042.stdout:6/33: symlink l8 0 2024-02-15T18:26:51.263 INFO:tasks.workunit.client.0.smithi042.stdout:1/25: rename c1 to c7 0 2024-02-15T18:26:51.263 INFO:tasks.workunit.client.0.smithi042.stdout:6/34: mkdir d9 0 2024-02-15T18:26:51.263 INFO:tasks.workunit.client.0.smithi042.stdout:1/26: read f5 [3674093,105250] 0 2024-02-15T18:26:51.263 INFO:tasks.workunit.client.0.smithi042.stdout:3/38: dwrite d1/f7 [0,4194304] 0 2024-02-15T18:26:51.264 INFO:tasks.workunit.client.0.smithi042.stdout:3/39: chown d1/c8 67423 1 2024-02-15T18:26:51.264 INFO:tasks.workunit.client.0.smithi042.stdout:6/35: creat d9/fa x:0 0 0 2024-02-15T18:26:51.264 INFO:tasks.workunit.client.0.smithi042.stdout:3/40: write d1/f9 [324303,127088] 0 2024-02-15T18:26:51.264 INFO:tasks.workunit.client.0.smithi042.stdout:3/41: fdatasync d1/f7 0 2024-02-15T18:26:51.264 INFO:tasks.workunit.client.0.smithi042.stdout:3/42: readlink - no filename 2024-02-15T18:26:51.264 INFO:tasks.workunit.client.0.smithi042.stdout:6/36: symlink d9/lb 0 2024-02-15T18:26:51.265 INFO:tasks.workunit.client.0.smithi042.stdout:1/27: mknod c8 0 2024-02-15T18:26:51.266 INFO:tasks.workunit.client.0.smithi042.stdout:9/33: dread f3 [0,4194304] 0 2024-02-15T18:26:51.266 INFO:tasks.workunit.client.0.smithi042.stdout:1/28: fsync f2 0 2024-02-15T18:26:51.267 INFO:tasks.workunit.client.0.smithi042.stdout:1/29: fsync f5 0 2024-02-15T18:26:51.267 INFO:tasks.workunit.client.0.smithi042.stdout:1/30: fsync f2 0 2024-02-15T18:26:51.267 INFO:tasks.workunit.client.0.smithi042.stdout:6/37: symlink d9/lc 0 2024-02-15T18:26:51.270 INFO:tasks.workunit.client.0.smithi042.stdout:6/38: symlink d9/ld 0 2024-02-15T18:26:51.271 INFO:tasks.workunit.client.0.smithi042.stdout:1/31: symlink l9 0 2024-02-15T18:26:51.274 INFO:tasks.workunit.client.0.smithi042.stdout:1/32: mknod ca 0 2024-02-15T18:26:51.274 INFO:tasks.workunit.client.0.smithi042.stdout:1/33: chown l9 0 1 2024-02-15T18:26:51.275 INFO:tasks.workunit.client.0.smithi042.stdout:0/30: dwrite d4/f8 [0,4194304] 0 2024-02-15T18:26:51.280 INFO:tasks.workunit.client.0.smithi042.stdout:3/43: dread d1/f7 [0,4194304] 0 2024-02-15T18:26:51.280 INFO:tasks.workunit.client.0.smithi042.stdout:3/44: chown d1 1732 1 2024-02-15T18:26:51.282 INFO:tasks.workunit.client.0.smithi042.stdout:6/39: truncate f7 557476 0 2024-02-15T18:26:51.282 INFO:tasks.workunit.client.0.smithi042.stdout:0/31: dread d4/f8 [0,4194304] 0 2024-02-15T18:26:51.284 INFO:tasks.workunit.client.0.smithi042.stdout:3/45: getdents d1 0 2024-02-15T18:26:51.284 INFO:tasks.workunit.client.0.smithi042.stdout:3/46: truncate d1/f9 656931 0 2024-02-15T18:26:51.291 INFO:tasks.workunit.client.0.smithi042.stdout:9/34: dwrite d6/fb [0,4194304] 0 2024-02-15T18:26:51.292 INFO:tasks.workunit.client.0.smithi042.stdout:2/31: dread d0/f3 [0,4194304] 0 2024-02-15T18:26:51.294 INFO:tasks.workunit.client.0.smithi042.stdout:2/32: write d0/f3 [3122037,81010] 0 2024-02-15T18:26:51.294 INFO:tasks.workunit.client.0.smithi042.stdout:2/33: stat d0/d8 0 2024-02-15T18:26:51.294 INFO:tasks.workunit.client.0.smithi042.stdout:9/35: rename d6/d7 to d6/dd 0 2024-02-15T18:26:51.294 INFO:tasks.workunit.client.0.smithi042.stdout:2/34: dread d0/f5 [0,4194304] 0 2024-02-15T18:26:51.295 INFO:tasks.workunit.client.0.smithi042.stdout:2/35: write d0/f7 [976489,104454] 0 2024-02-15T18:26:51.295 INFO:tasks.workunit.client.0.smithi042.stdout:2/36: write d0/f5 [18886,96238] 0 2024-02-15T18:26:51.295 INFO:tasks.workunit.client.0.smithi042.stdout:2/37: chown d0/f7 317 1 2024-02-15T18:26:51.296 INFO:tasks.workunit.client.0.smithi042.stdout:9/36: creat d6/dd/fe x:0 0 0 2024-02-15T18:26:51.297 INFO:tasks.workunit.client.0.smithi042.stdout:9/37: stat f2 0 2024-02-15T18:26:51.298 INFO:tasks.workunit.client.0.smithi042.stdout:9/38: mknod d6/da/cf 0 2024-02-15T18:26:51.303 INFO:tasks.workunit.client.0.smithi042.stdout:1/34: dwrite f5 [4194304,4194304] 0 2024-02-15T18:26:51.303 INFO:tasks.workunit.client.0.smithi042.stdout:1/35: dread - f2 zero size 2024-02-15T18:26:51.303 INFO:tasks.workunit.client.0.smithi042.stdout:1/36: write f6 [949042,114351] 0 2024-02-15T18:26:51.303 INFO:tasks.workunit.client.0.smithi042.stdout:0/32: dwrite d4/f7 [0,4194304] 0 2024-02-15T18:26:51.327 INFO:tasks.workunit.client.0.smithi042.stdout:6/40: truncate f7 336748 0 2024-02-15T18:26:51.327 INFO:tasks.workunit.client.0.smithi042.stdout:6/41: readlink d9/ld 0 2024-02-15T18:26:51.328 INFO:tasks.workunit.client.0.smithi042.stdout:6/42: rename l8 to d9/le 0 2024-02-15T18:26:51.329 INFO:tasks.workunit.client.0.smithi042.stdout:6/43: symlink d9/lf 0 2024-02-15T18:26:51.340 INFO:tasks.workunit.client.0.smithi042.stdout:1/37: dwrite f2 [0,4194304] 0 2024-02-15T18:26:51.341 INFO:tasks.workunit.client.0.smithi042.stdout:2/38: dwrite d0/f7 [0,4194304] 0 2024-02-15T18:26:51.341 INFO:tasks.workunit.client.0.smithi042.stdout:0/33: dwrite d4/fc [0,4194304] 0 2024-02-15T18:26:51.342 INFO:tasks.workunit.client.0.smithi042.stdout:1/38: getdents . 0 2024-02-15T18:26:51.344 INFO:tasks.workunit.client.0.smithi042.stdout:4/24: sync 2024-02-15T18:26:51.349 INFO:tasks.workunit.client.0.smithi042.stdout:2/39: rename d0/d8/da to d0/d8/db 0 2024-02-15T18:26:51.349 INFO:tasks.workunit.client.0.smithi042.stdout:2/40: write d0/f5 [318801,93355] 0 2024-02-15T18:26:51.352 INFO:tasks.workunit.client.0.smithi042.stdout:1/39: creat fb x:0 0 0 2024-02-15T18:26:51.353 INFO:tasks.workunit.client.0.smithi042.stdout:6/44: link f7 d9/f10 0 2024-02-15T18:26:51.353 INFO:tasks.workunit.client.0.smithi042.stdout:6/45: truncate d9/f10 410713 0 2024-02-15T18:26:51.353 INFO:tasks.workunit.client.0.smithi042.stdout:6/46: readlink d9/lb 0 2024-02-15T18:26:51.355 INFO:tasks.workunit.client.0.smithi042.stdout:4/25: rename c0 to c4 0 2024-02-15T18:26:51.362 INFO:tasks.workunit.client.0.smithi042.stdout:0/34: dwrite d4/f6 [0,4194304] 0 2024-02-15T18:26:51.364 INFO:tasks.workunit.client.0.smithi042.stdout:0/35: rename d4/f8 to d4/d9/ff 0 2024-02-15T18:26:51.365 INFO:tasks.workunit.client.0.smithi042.stdout:6/47: fsync d9/f10 0 2024-02-15T18:26:51.366 INFO:tasks.workunit.client.0.smithi042.stdout:4/26: rename c4 to c5 0 2024-02-15T18:26:51.368 INFO:tasks.workunit.client.0.smithi042.stdout:6/48: getdents d9 0 2024-02-15T18:26:51.368 INFO:tasks.workunit.client.0.smithi042.stdout:6/49: fdatasync d9/f10 0 2024-02-15T18:26:51.369 INFO:tasks.workunit.client.0.smithi042.stdout:6/50: stat d9/le 0 2024-02-15T18:26:51.369 INFO:tasks.workunit.client.0.smithi042.stdout:4/27: stat c5 0 2024-02-15T18:26:51.369 INFO:tasks.workunit.client.0.smithi042.stdout:4/28: rmdir - no directory 2024-02-15T18:26:51.369 INFO:tasks.workunit.client.0.smithi042.stdout:4/29: write - no filename 2024-02-15T18:26:51.369 INFO:tasks.workunit.client.0.smithi042.stdout:4/30: dread - no filename 2024-02-15T18:26:51.369 INFO:tasks.workunit.client.0.smithi042.stdout:4/31: stat c3 0 2024-02-15T18:26:51.370 INFO:tasks.workunit.client.0.smithi042.stdout:1/40: dwrite f2 [0,4194304] 0 2024-02-15T18:26:51.370 INFO:tasks.workunit.client.0.smithi042.stdout:6/51: mkdir d9/d11 0 2024-02-15T18:26:51.370 INFO:tasks.workunit.client.0.smithi042.stdout:1/41: write f2 [2988333,51014] 0 2024-02-15T18:26:51.370 INFO:tasks.workunit.client.0.smithi042.stdout:6/52: read f7 [347199,99264] 0 2024-02-15T18:26:51.372 INFO:tasks.workunit.client.0.smithi042.stdout:4/32: link c3 c6 0 2024-02-15T18:26:51.372 INFO:tasks.workunit.client.0.smithi042.stdout:4/33: write - no filename 2024-02-15T18:26:51.372 INFO:tasks.workunit.client.0.smithi042.stdout:4/34: fsync - no filename 2024-02-15T18:26:51.372 INFO:tasks.workunit.client.0.smithi042.stdout:4/35: dwrite - no filename 2024-02-15T18:26:51.372 INFO:tasks.workunit.client.0.smithi042.stdout:4/36: dwrite - no filename 2024-02-15T18:26:51.372 INFO:tasks.workunit.client.0.smithi042.stdout:4/37: fdatasync - no filename 2024-02-15T18:26:51.372 INFO:tasks.workunit.client.0.smithi042.stdout:4/38: chown c5 206990726 1 2024-02-15T18:26:51.373 INFO:tasks.workunit.client.0.smithi042.stdout:1/42: creat fc x:0 0 0 2024-02-15T18:26:51.373 INFO:tasks.workunit.client.0.smithi042.stdout:0/36: dread d4/f6 [0,4194304] 0 2024-02-15T18:26:51.374 INFO:tasks.workunit.client.0.smithi042.stdout:6/53: creat d9/f12 x:0 0 0 2024-02-15T18:26:51.374 INFO:tasks.workunit.client.0.smithi042.stdout:1/43: write f2 [2637432,129328] 0 2024-02-15T18:26:51.375 INFO:tasks.workunit.client.0.smithi042.stdout:4/39: rename c6 to c7 0 2024-02-15T18:26:51.375 INFO:tasks.workunit.client.0.smithi042.stdout:4/40: chown c2 42818 1 2024-02-15T18:26:51.375 INFO:tasks.workunit.client.0.smithi042.stdout:6/54: dread d9/f10 [0,4194304] 0 2024-02-15T18:26:51.375 INFO:tasks.workunit.client.0.smithi042.stdout:6/55: write d9/fa [310518,48058] 0 2024-02-15T18:26:51.375 INFO:tasks.workunit.client.0.smithi042.stdout:6/56: chown d9/lc 1734751 1 2024-02-15T18:26:51.375 INFO:tasks.workunit.client.0.smithi042.stdout:0/37: symlink d4/l10 0 2024-02-15T18:26:51.377 INFO:tasks.workunit.client.0.smithi042.stdout:1/44: read f5 [6690739,100536] 0 2024-02-15T18:26:51.377 INFO:tasks.workunit.client.0.smithi042.stdout:1/45: stat f5 0 2024-02-15T18:26:51.377 INFO:tasks.workunit.client.0.smithi042.stdout:4/41: symlink l8 0 2024-02-15T18:26:51.377 INFO:tasks.workunit.client.0.smithi042.stdout:4/42: chown c7 281390 1 2024-02-15T18:26:51.377 INFO:tasks.workunit.client.0.smithi042.stdout:1/46: write f2 [4130882,51152] 0 2024-02-15T18:26:51.378 INFO:tasks.workunit.client.0.smithi042.stdout:1/47: dread - fb zero size 2024-02-15T18:26:51.378 INFO:tasks.workunit.client.0.smithi042.stdout:6/57: mknod d9/c13 0 2024-02-15T18:26:51.378 INFO:tasks.workunit.client.0.smithi042.stdout:0/38: symlink d4/l11 0 2024-02-15T18:26:51.378 INFO:tasks.workunit.client.0.smithi042.stdout:0/39: write d4/f7 [4918368,62184] 0 2024-02-15T18:26:51.379 INFO:tasks.workunit.client.0.smithi042.stdout:4/43: creat f9 x:0 0 0 2024-02-15T18:26:51.380 INFO:tasks.workunit.client.0.smithi042.stdout:6/58: creat d9/f14 x:0 0 0 2024-02-15T18:26:51.381 INFO:tasks.workunit.client.0.smithi042.stdout:6/59: readlink d9/ld 0 2024-02-15T18:26:51.381 INFO:tasks.workunit.client.0.smithi042.stdout:0/40: symlink d4/l12 0 2024-02-15T18:26:51.381 INFO:tasks.workunit.client.0.smithi042.stdout:0/41: read d4/d9/ff [2783880,103770] 0 2024-02-15T18:26:51.382 INFO:tasks.workunit.client.0.smithi042.stdout:4/44: mkdir da 0 2024-02-15T18:26:51.382 INFO:tasks.workunit.client.0.smithi042.stdout:0/42: write d4/f7 [1482156,36811] 0 2024-02-15T18:26:51.382 INFO:tasks.workunit.client.0.smithi042.stdout:1/48: rename f5 to fd 0 2024-02-15T18:26:51.383 INFO:tasks.workunit.client.0.smithi042.stdout:1/49: rmdir - no directory 2024-02-15T18:26:51.384 INFO:tasks.workunit.client.0.smithi042.stdout:6/60: creat d9/d11/f15 x:0 0 0 2024-02-15T18:26:51.384 INFO:tasks.workunit.client.0.smithi042.stdout:6/61: dread - d9/f14 zero size 2024-02-15T18:26:51.385 INFO:tasks.workunit.client.0.smithi042.stdout:4/45: mkdir da/db 0 2024-02-15T18:26:51.387 INFO:tasks.workunit.client.0.smithi042.stdout:6/62: rename d9/c13 to d9/d11/c16 0 2024-02-15T18:26:51.387 INFO:tasks.workunit.client.0.smithi042.stdout:0/43: dread d4/f6 [0,4194304] 0 2024-02-15T18:26:51.389 INFO:tasks.workunit.client.0.smithi042.stdout:4/46: mknod da/cc 0 2024-02-15T18:26:51.389 INFO:tasks.workunit.client.0.smithi042.stdout:4/47: dread - f9 zero size 2024-02-15T18:26:51.389 INFO:tasks.workunit.client.0.smithi042.stdout:4/48: stat da/cc 0 2024-02-15T18:26:51.389 INFO:tasks.workunit.client.0.smithi042.stdout:1/50: mkdir de 0 2024-02-15T18:26:51.389 INFO:tasks.workunit.client.0.smithi042.stdout:1/51: chown fc 360938 1 2024-02-15T18:26:51.389 INFO:tasks.workunit.client.0.smithi042.stdout:1/52: stat fd 0 2024-02-15T18:26:51.390 INFO:tasks.workunit.client.0.smithi042.stdout:4/49: rmdir da 39 2024-02-15T18:26:51.390 INFO:tasks.workunit.client.0.smithi042.stdout:4/50: chown f9 4224 1 2024-02-15T18:26:51.401 INFO:tasks.workunit.client.0.smithi042.stdout:4/51: fsync f9 0 2024-02-15T18:26:51.401 INFO:tasks.workunit.client.0.smithi042.stdout:4/52: getdents da/db 0 2024-02-15T18:26:51.406 INFO:tasks.workunit.client.0.smithi042.stdout:4/53: creat da/fd x:0 0 0 2024-02-15T18:26:51.406 INFO:tasks.workunit.client.0.smithi042.stdout:4/54: truncate da/fd 779559 0 2024-02-15T18:26:51.408 INFO:tasks.workunit.client.0.smithi042.stdout:4/55: creat da/db/fe x:0 0 0 2024-02-15T18:26:51.409 INFO:tasks.workunit.client.0.smithi042.stdout:6/63: dwrite d9/fa [0,4194304] 0 2024-02-15T18:26:51.409 INFO:tasks.workunit.client.0.smithi042.stdout:6/64: fdatasync d9/f10 0 2024-02-15T18:26:51.410 INFO:tasks.workunit.client.0.smithi042.stdout:6/65: creat d9/f17 x:0 0 0 2024-02-15T18:26:51.411 INFO:tasks.workunit.client.0.smithi042.stdout:0/44: dwrite d4/fe [0,4194304] 0 2024-02-15T18:26:51.412 INFO:tasks.workunit.client.0.smithi042.stdout:6/66: rename d9/f12 to d9/d11/f18 0 2024-02-15T18:26:51.412 INFO:tasks.workunit.client.0.smithi042.stdout:6/67: dread - d9/f17 zero size 2024-02-15T18:26:51.412 INFO:tasks.workunit.client.0.smithi042.stdout:6/68: chown d9/d11 121 1 2024-02-15T18:26:51.413 INFO:tasks.workunit.client.0.smithi042.stdout:1/53: dwrite f2 [0,4194304] 0 2024-02-15T18:26:51.413 INFO:tasks.workunit.client.0.smithi042.stdout:6/69: read f7 [212254,96000] 0 2024-02-15T18:26:51.416 INFO:tasks.workunit.client.0.smithi042.stdout:0/45: mknod d4/c13 0 2024-02-15T18:26:51.418 INFO:tasks.workunit.client.0.smithi042.stdout:1/54: mknod de/cf 0 2024-02-15T18:26:51.419 INFO:tasks.workunit.client.0.smithi042.stdout:6/70: mkdir d9/d11/d19 0 2024-02-15T18:26:51.422 INFO:tasks.workunit.client.0.smithi042.stdout:0/46: link d4/f7 d4/d9/f14 0 2024-02-15T18:26:51.423 INFO:tasks.workunit.client.0.smithi042.stdout:1/55: rename de/cf to de/c10 0 2024-02-15T18:26:51.423 INFO:tasks.workunit.client.0.smithi042.stdout:6/71: mkdir d9/d11/d19/d1a 0 2024-02-15T18:26:51.424 INFO:tasks.workunit.client.0.smithi042.stdout:6/72: write d9/fa [2919019,104604] 0 2024-02-15T18:26:51.424 INFO:tasks.workunit.client.0.smithi042.stdout:6/73: readlink d9/lc 0 2024-02-15T18:26:51.424 INFO:tasks.workunit.client.0.smithi042.stdout:6/74: dread - d9/d11/f18 zero size 2024-02-15T18:26:51.450 INFO:tasks.workunit.client.0.smithi042.stdout:6/75: dwrite d9/f14 [0,4194304] 0 2024-02-15T18:26:51.453 INFO:tasks.workunit.client.0.smithi042.stdout:6/76: link d9/fa d9/f1b 0 2024-02-15T18:26:51.454 INFO:tasks.workunit.client.0.smithi042.stdout:6/77: rename d9/d11/d19/d1a to d9/d11/d19/d1c 0 2024-02-15T18:26:51.456 INFO:tasks.workunit.client.0.smithi042.stdout:6/78: mkdir d9/d11/d1d 0 2024-02-15T18:26:51.456 INFO:tasks.workunit.client.0.smithi042.stdout:6/79: read - d9/d11/f18 zero size 2024-02-15T18:26:51.456 INFO:tasks.workunit.client.0.smithi042.stdout:6/80: write d9/f17 [146637,115788] 0 2024-02-15T18:26:51.456 INFO:tasks.workunit.client.0.smithi042.stdout:6/81: chown d9/f17 3 1 2024-02-15T18:26:51.456 INFO:tasks.workunit.client.0.smithi042.stdout:6/82: truncate d9/d11/f18 149551 0 2024-02-15T18:26:51.485 INFO:tasks.workunit.client.0.smithi042.stdout:6/83: read d9/fa [1433492,22331] 0 2024-02-15T18:26:51.490 INFO:tasks.workunit.client.0.smithi042.stdout:4/56: dwrite f9 [0,4194304] 0 2024-02-15T18:26:51.490 INFO:tasks.workunit.client.0.smithi042.stdout:4/57: truncate da/db/fe 949927 0 2024-02-15T18:26:51.491 INFO:tasks.workunit.client.0.smithi042.stdout:1/56: dwrite fc [0,4194304] 0 2024-02-15T18:26:51.492 INFO:tasks.workunit.client.0.smithi042.stdout:6/84: dread d9/fa [0,4194304] 0 2024-02-15T18:26:51.528 INFO:tasks.workunit.client.0.smithi042.stdout:1/57: mknod de/c11 0 2024-02-15T18:26:51.528 INFO:tasks.workunit.client.0.smithi042.stdout:1/58: chown fc 6422 1 2024-02-15T18:26:51.529 INFO:tasks.workunit.client.0.smithi042.stdout:6/85: rename d9/ld to d9/d11/l1e 0 2024-02-15T18:26:51.529 INFO:tasks.workunit.client.0.smithi042.stdout:6/86: write f7 [572724,12263] 0 2024-02-15T18:26:51.530 INFO:tasks.workunit.client.0.smithi042.stdout:1/59: rename de/c10 to de/c12 0 2024-02-15T18:26:51.534 INFO:tasks.workunit.client.0.smithi042.stdout:4/58: dwrite f9 [0,4194304] 0 2024-02-15T18:26:51.534 INFO:tasks.workunit.client.0.smithi042.stdout:1/60: truncate fc 643025 0 2024-02-15T18:26:51.538 INFO:tasks.workunit.client.0.smithi042.stdout:1/61: creat de/f13 x:0 0 0 2024-02-15T18:26:51.538 INFO:tasks.workunit.client.0.smithi042.stdout:1/62: chown de/f13 6 1 2024-02-15T18:26:51.538 INFO:tasks.workunit.client.0.smithi042.stdout:1/63: write de/f13 [451127,58345] 0 2024-02-15T18:26:51.541 INFO:tasks.workunit.client.0.smithi042.stdout:6/87: dwrite d9/f1b [0,4194304] 0 2024-02-15T18:26:51.543 INFO:tasks.workunit.client.0.smithi042.stdout:7/20: sync 2024-02-15T18:26:51.543 INFO:tasks.workunit.client.0.smithi042.stdout:5/20: sync 2024-02-15T18:26:51.543 INFO:tasks.workunit.client.0.smithi042.stdout:8/13: sync 2024-02-15T18:26:51.543 INFO:tasks.workunit.client.0.smithi042.stdout:8/14: dwrite - no filename 2024-02-15T18:26:51.543 INFO:tasks.workunit.client.0.smithi042.stdout:8/15: dwrite - no filename 2024-02-15T18:26:51.543 INFO:tasks.workunit.client.0.smithi042.stdout:3/47: sync 2024-02-15T18:26:51.545 INFO:tasks.workunit.client.0.smithi042.stdout:3/48: write d1/f7 [3929548,25623] 0 2024-02-15T18:26:51.547 INFO:tasks.workunit.client.0.smithi042.stdout:6/88: unlink f7 0 2024-02-15T18:26:51.547 INFO:tasks.workunit.client.0.smithi042.stdout:6/89: chown d9/f17 465592 1 2024-02-15T18:26:51.547 INFO:tasks.workunit.client.0.smithi042.stdout:6/90: fdatasync d9/d11/f15 0 2024-02-15T18:26:51.548 INFO:tasks.workunit.client.0.smithi042.stdout:7/21: truncate f4 628638 0 2024-02-15T18:26:51.550 INFO:tasks.workunit.client.0.smithi042.stdout:5/21: creat d0/f3 x:0 0 0 2024-02-15T18:26:51.554 INFO:tasks.workunit.client.0.smithi042.stdout:3/49: rmdir d1 39 2024-02-15T18:26:51.554 INFO:tasks.workunit.client.0.smithi042.stdout:8/16: mknod c5 0 2024-02-15T18:26:51.554 INFO:tasks.workunit.client.0.smithi042.stdout:7/22: rename c6 to c8 0 2024-02-15T18:26:51.554 INFO:tasks.workunit.client.0.smithi042.stdout:7/23: write f7 [824745,86398] 0 2024-02-15T18:26:51.556 INFO:tasks.workunit.client.0.smithi042.stdout:3/50: link d1/f9 d1/fa 0 2024-02-15T18:26:51.558 INFO:tasks.workunit.client.0.smithi042.stdout:8/17: rename l4 to l6 0 2024-02-15T18:26:51.558 INFO:tasks.workunit.client.0.smithi042.stdout:8/18: dwrite - no filename 2024-02-15T18:26:51.558 INFO:tasks.workunit.client.0.smithi042.stdout:8/19: write - no filename 2024-02-15T18:26:51.558 INFO:tasks.workunit.client.0.smithi042.stdout:8/20: stat l1 0 2024-02-15T18:26:51.558 INFO:tasks.workunit.client.0.smithi042.stdout:8/21: dread - no filename 2024-02-15T18:26:51.558 INFO:tasks.workunit.client.0.smithi042.stdout:8/22: read - no filename 2024-02-15T18:26:51.559 INFO:tasks.workunit.client.0.smithi042.stdout:3/51: read d1/f7 [3251281,99718] 0 2024-02-15T18:26:51.561 INFO:tasks.workunit.client.0.smithi042.stdout:8/23: link l1 l7 0 2024-02-15T18:26:51.561 INFO:tasks.workunit.client.0.smithi042.stdout:8/24: write - no filename 2024-02-15T18:26:51.561 INFO:tasks.workunit.client.0.smithi042.stdout:3/52: mknod d1/cb 0 2024-02-15T18:26:51.561 INFO:tasks.workunit.client.0.smithi042.stdout:8/25: chown l7 16 1 2024-02-15T18:26:51.561 INFO:tasks.workunit.client.0.smithi042.stdout:8/26: write - no filename 2024-02-15T18:26:51.563 INFO:tasks.workunit.client.0.smithi042.stdout:3/53: rename d1/c6 to d1/cc 0 2024-02-15T18:26:51.565 INFO:tasks.workunit.client.0.smithi042.stdout:3/54: mkdir d1/dd 0 2024-02-15T18:26:51.565 INFO:tasks.workunit.client.0.smithi042.stdout:1/64: dwrite fb [0,4194304] 0 2024-02-15T18:26:51.565 INFO:tasks.workunit.client.0.smithi042.stdout:3/55: rmdir d1 39 2024-02-15T18:26:51.566 INFO:tasks.workunit.client.0.smithi042.stdout:3/56: read d1/f9 [53202,106196] 0 2024-02-15T18:26:51.567 INFO:tasks.workunit.client.0.smithi042.stdout:1/65: unlink de/f13 0 2024-02-15T18:26:51.567 INFO:tasks.workunit.client.0.smithi042.stdout:1/66: chown fd 2023417096 1 2024-02-15T18:26:51.568 INFO:tasks.workunit.client.0.smithi042.stdout:4/59: dwrite f9 [0,4194304] 0 2024-02-15T18:26:51.569 INFO:tasks.workunit.client.0.smithi042.stdout:3/57: rename d1/dd to d1/de 0 2024-02-15T18:26:51.569 INFO:tasks.workunit.client.0.smithi042.stdout:1/67: readlink l9 0 2024-02-15T18:26:51.574 INFO:tasks.workunit.client.0.smithi042.stdout:3/58: dread d1/fa [0,4194304] 0 2024-02-15T18:26:51.574 INFO:tasks.workunit.client.0.smithi042.stdout:1/68: rename l9 to de/l14 0 2024-02-15T18:26:51.576 INFO:tasks.workunit.client.0.smithi042.stdout:1/69: mkdir de/d15 0 2024-02-15T18:26:51.578 INFO:tasks.workunit.client.0.smithi042.stdout:6/91: dwrite d9/d11/f15 [0,4194304] 0 2024-02-15T18:26:51.579 INFO:tasks.workunit.client.0.smithi042.stdout:7/24: getdents . 0 2024-02-15T18:26:51.579 INFO:tasks.workunit.client.0.smithi042.stdout:5/22: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:26:51.581 INFO:tasks.workunit.client.0.smithi042.stdout:8/27: rename l6 to l8 0 2024-02-15T18:26:51.581 INFO:tasks.workunit.client.0.smithi042.stdout:8/28: write - no filename 2024-02-15T18:26:51.581 INFO:tasks.workunit.client.0.smithi042.stdout:8/29: dread - no filename 2024-02-15T18:26:51.581 INFO:tasks.workunit.client.0.smithi042.stdout:8/30: readlink l8 0 2024-02-15T18:26:51.582 INFO:tasks.workunit.client.0.smithi042.stdout:4/60: dwrite da/db/fe [0,4194304] 0 2024-02-15T18:26:51.588 INFO:tasks.workunit.client.0.smithi042.stdout:3/59: dread d1/f7 [0,4194304] 0 2024-02-15T18:26:51.589 INFO:tasks.workunit.client.0.smithi042.stdout:3/60: dread d1/f9 [0,4194304] 0 2024-02-15T18:26:51.589 INFO:tasks.workunit.client.0.smithi042.stdout:3/61: fsync d1/fa 0 2024-02-15T18:26:51.592 INFO:tasks.workunit.client.0.smithi042.stdout:9/39: sync 2024-02-15T18:26:51.592 INFO:tasks.workunit.client.0.smithi042.stdout:9/40: readlink - no filename 2024-02-15T18:26:51.592 INFO:tasks.workunit.client.0.smithi042.stdout:0/47: sync 2024-02-15T18:26:51.592 INFO:tasks.workunit.client.0.smithi042.stdout:2/41: sync 2024-02-15T18:26:51.592 INFO:tasks.workunit.client.0.smithi042.stdout:0/48: readlink d4/l10 0 2024-02-15T18:26:51.595 INFO:tasks.workunit.client.0.smithi042.stdout:7/25: rename c3 to c9 0 2024-02-15T18:26:51.596 INFO:tasks.workunit.client.0.smithi042.stdout:5/23: symlink d0/l4 0 2024-02-15T18:26:51.597 INFO:tasks.workunit.client.0.smithi042.stdout:8/31: creat f9 x:0 0 0 2024-02-15T18:26:51.597 INFO:tasks.workunit.client.0.smithi042.stdout:5/24: write d0/f3 [2561269,96183] 0 2024-02-15T18:26:51.598 INFO:tasks.workunit.client.0.smithi042.stdout:5/25: chown d0 0 1 2024-02-15T18:26:51.600 INFO:tasks.workunit.client.0.smithi042.stdout:4/61: unlink c7 0 2024-02-15T18:26:51.602 INFO:tasks.workunit.client.0.smithi042.stdout:3/62: link d1/c8 d1/de/cf 0 2024-02-15T18:26:51.603 INFO:tasks.workunit.client.0.smithi042.stdout:9/41: creat d6/da/f10 x:0 0 0 2024-02-15T18:26:51.603 INFO:tasks.workunit.client.0.smithi042.stdout:2/42: symlink d0/lc 0 2024-02-15T18:26:51.604 INFO:tasks.workunit.client.0.smithi042.stdout:0/49: mkdir d4/d9/d15 0 2024-02-15T18:26:51.605 INFO:tasks.workunit.client.0.smithi042.stdout:0/50: write d4/fc [1930953,75325] 0 2024-02-15T18:26:51.605 INFO:tasks.workunit.client.0.smithi042.stdout:8/32: link c3 ca 0 2024-02-15T18:26:51.605 INFO:tasks.workunit.client.0.smithi042.stdout:8/33: write f9 [48493,54158] 0 2024-02-15T18:26:51.606 INFO:tasks.workunit.client.0.smithi042.stdout:5/26: mknod d0/c5 0 2024-02-15T18:26:51.606 INFO:tasks.workunit.client.0.smithi042.stdout:5/27: stat d0 0 2024-02-15T18:26:51.606 INFO:tasks.workunit.client.0.smithi042.stdout:5/28: stat d0 0 2024-02-15T18:26:51.606 INFO:tasks.workunit.client.0.smithi042.stdout:5/29: chown d0/c5 53 1 2024-02-15T18:26:51.607 INFO:tasks.workunit.client.0.smithi042.stdout:3/63: truncate d1/f7 987207 0 2024-02-15T18:26:51.607 INFO:tasks.workunit.client.0.smithi042.stdout:3/64: chown d1/de/cf 388003820 1 2024-02-15T18:26:51.608 INFO:tasks.workunit.client.0.smithi042.stdout:9/42: getdents d6/dd 0 2024-02-15T18:26:51.609 INFO:tasks.workunit.client.0.smithi042.stdout:2/43: creat d0/d8/db/fd x:0 0 0 2024-02-15T18:26:51.609 INFO:tasks.workunit.client.0.smithi042.stdout:2/44: chown d0/f3 14392 1 2024-02-15T18:26:51.609 INFO:tasks.workunit.client.0.smithi042.stdout:2/45: chown d0/lc 489923261 1 2024-02-15T18:26:51.610 INFO:tasks.workunit.client.0.smithi042.stdout:6/92: rename d9/lc to d9/d11/d19/l1f 0 2024-02-15T18:26:51.612 INFO:tasks.workunit.client.0.smithi042.stdout:7/26: unlink f5 0 2024-02-15T18:26:51.617 INFO:tasks.workunit.client.0.smithi042.stdout:0/51: rmdir d4/d9 39 2024-02-15T18:26:51.618 INFO:tasks.workunit.client.0.smithi042.stdout:8/34: creat fb x:0 0 0 2024-02-15T18:26:51.618 INFO:tasks.workunit.client.0.smithi042.stdout:0/52: write d4/fc [1340828,107034] 0 2024-02-15T18:26:51.619 INFO:tasks.workunit.client.0.smithi042.stdout:5/30: mknod d0/c6 0 2024-02-15T18:26:51.621 INFO:tasks.workunit.client.0.smithi042.stdout:9/43: link d6/f8 d6/da/f11 0 2024-02-15T18:26:51.621 INFO:tasks.workunit.client.0.smithi042.stdout:9/44: write d6/da/f11 [3344625,96705] 0 2024-02-15T18:26:51.621 INFO:tasks.workunit.client.0.smithi042.stdout:9/45: chown d6/dd 88 1 2024-02-15T18:26:51.622 INFO:tasks.workunit.client.0.smithi042.stdout:6/93: creat d9/d11/f20 x:0 0 0 2024-02-15T18:26:51.623 INFO:tasks.workunit.client.0.smithi042.stdout:9/46: dread f2 [0,4194304] 0 2024-02-15T18:26:51.623 INFO:tasks.workunit.client.0.smithi042.stdout:8/35: rename f9 to fc 0 2024-02-15T18:26:51.623 INFO:tasks.workunit.client.0.smithi042.stdout:8/36: truncate fb 499356 0 2024-02-15T18:26:51.623 INFO:tasks.workunit.client.0.smithi042.stdout:8/37: chown ca 62010637 1 2024-02-15T18:26:51.624 INFO:tasks.workunit.client.0.smithi042.stdout:9/47: write d6/da/f11 [2009362,82135] 0 2024-02-15T18:26:51.625 INFO:tasks.workunit.client.0.smithi042.stdout:3/65: truncate d1/f7 1977937 0 2024-02-15T18:26:51.628 INFO:tasks.workunit.client.0.smithi042.stdout:7/27: truncate f7 167422 0 2024-02-15T18:26:51.628 INFO:tasks.workunit.client.0.smithi042.stdout:7/28: fdatasync f7 0 2024-02-15T18:26:51.629 INFO:tasks.workunit.client.0.smithi042.stdout:9/48: creat d6/da/f12 x:0 0 0 2024-02-15T18:26:51.629 INFO:tasks.workunit.client.0.smithi042.stdout:9/49: chown d6/fb 13802 1 2024-02-15T18:26:51.630 INFO:tasks.workunit.client.0.smithi042.stdout:3/66: creat d1/de/f10 x:0 0 0 2024-02-15T18:26:51.630 INFO:tasks.workunit.client.0.smithi042.stdout:3/67: fsync d1/fa 0 2024-02-15T18:26:51.630 INFO:tasks.workunit.client.0.smithi042.stdout:3/68: readlink - no filename 2024-02-15T18:26:51.630 INFO:tasks.workunit.client.0.smithi042.stdout:3/69: fdatasync d1/f7 0 2024-02-15T18:26:51.631 INFO:tasks.workunit.client.0.smithi042.stdout:6/94: unlink d9/f10 0 2024-02-15T18:26:51.632 INFO:tasks.workunit.client.0.smithi042.stdout:2/46: dwrite d0/f7 [0,4194304] 0 2024-02-15T18:26:51.633 INFO:tasks.workunit.client.0.smithi042.stdout:7/29: creat fa x:0 0 0 2024-02-15T18:26:51.635 INFO:tasks.workunit.client.0.smithi042.stdout:5/31: dread d0/f3 [0,4194304] 0 2024-02-15T18:26:51.636 INFO:tasks.workunit.client.0.smithi042.stdout:0/53: dwrite d4/f7 [0,4194304] 0 2024-02-15T18:26:51.639 INFO:tasks.workunit.client.0.smithi042.stdout:9/50: creat d6/da/f13 x:0 0 0 2024-02-15T18:26:51.640 INFO:tasks.workunit.client.0.smithi042.stdout:3/70: getdents d1 0 2024-02-15T18:26:51.643 INFO:tasks.workunit.client.0.smithi042.stdout:2/47: mkdir d0/d8/db/de 0 2024-02-15T18:26:51.644 INFO:tasks.workunit.client.0.smithi042.stdout:5/32: dread d0/f3 [0,4194304] 0 2024-02-15T18:26:51.645 INFO:tasks.workunit.client.0.smithi042.stdout:0/54: mkdir d4/d16 0 2024-02-15T18:26:51.646 INFO:tasks.workunit.client.0.smithi042.stdout:9/51: rename d6/da/f11 to d6/dd/d9/f14 0 2024-02-15T18:26:51.646 INFO:tasks.workunit.client.0.smithi042.stdout:2/48: rename d0/d8/db to d0/d8/db/df 22 2024-02-15T18:26:51.646 INFO:tasks.workunit.client.0.smithi042.stdout:0/55: read d4/f6 [485058,130456] 0 2024-02-15T18:26:51.647 INFO:tasks.workunit.client.0.smithi042.stdout:3/71: mkdir d1/d11 0 2024-02-15T18:26:51.648 INFO:tasks.workunit.client.0.smithi042.stdout:5/33: mknod d0/c7 0 2024-02-15T18:26:51.650 INFO:tasks.workunit.client.0.smithi042.stdout:2/49: mkdir d0/d8/d10 0 2024-02-15T18:26:51.652 INFO:tasks.workunit.client.0.smithi042.stdout:7/30: dwrite f7 [0,4194304] 0 2024-02-15T18:26:51.652 INFO:tasks.workunit.client.0.smithi042.stdout:0/56: symlink d4/d9/l17 0 2024-02-15T18:26:51.654 INFO:tasks.workunit.client.0.smithi042.stdout:2/50: write d0/f7 [494552,27387] 0 2024-02-15T18:26:51.654 INFO:tasks.workunit.client.0.smithi042.stdout:6/95: rename d9/f1b to d9/f21 0 2024-02-15T18:26:51.655 INFO:tasks.workunit.client.0.smithi042.stdout:5/34: link d0/c6 d0/c8 0 2024-02-15T18:26:51.657 INFO:tasks.workunit.client.0.smithi042.stdout:9/52: rmdir d6/da 39 2024-02-15T18:26:51.658 INFO:tasks.workunit.client.0.smithi042.stdout:3/72: getdents d1 0 2024-02-15T18:26:51.660 INFO:tasks.workunit.client.0.smithi042.stdout:7/31: unlink f7 0 2024-02-15T18:26:51.660 INFO:tasks.workunit.client.0.smithi042.stdout:7/32: write fa [365051,130443] 0 2024-02-15T18:26:51.661 INFO:tasks.workunit.client.0.smithi042.stdout:0/57: creat d4/d9/d15/f18 x:0 0 0 2024-02-15T18:26:51.663 INFO:tasks.workunit.client.0.smithi042.stdout:6/96: creat d9/d11/d1d/f22 x:0 0 0 2024-02-15T18:26:51.664 INFO:tasks.workunit.client.0.smithi042.stdout:5/35: link d0/l4 d0/l9 0 2024-02-15T18:26:51.664 INFO:tasks.workunit.client.0.smithi042.stdout:9/53: dread f5 [0,4194304] 0 2024-02-15T18:26:51.664 INFO:tasks.workunit.client.0.smithi042.stdout:9/54: stat d6/da/f13 0 2024-02-15T18:26:51.665 INFO:tasks.workunit.client.0.smithi042.stdout:7/33: chown f4 19 1 2024-02-15T18:26:51.666 INFO:tasks.workunit.client.0.smithi042.stdout:6/97: truncate d9/d11/f15 3737212 0 2024-02-15T18:26:51.668 INFO:tasks.workunit.client.0.smithi042.stdout:9/55: symlink d6/l15 0 2024-02-15T18:26:51.668 INFO:tasks.workunit.client.0.smithi042.stdout:9/56: read - d6/dd/fe zero size 2024-02-15T18:26:51.668 INFO:tasks.workunit.client.0.smithi042.stdout:9/57: write d6/da/f13 [341532,16438] 0 2024-02-15T18:26:51.668 INFO:tasks.workunit.client.0.smithi042.stdout:9/58: readlink d6/l15 0 2024-02-15T18:26:51.668 INFO:tasks.workunit.client.0.smithi042.stdout:9/59: read d6/da/f13 [147412,19179] 0 2024-02-15T18:26:51.670 INFO:tasks.workunit.client.0.smithi042.stdout:9/60: creat d6/dd/d9/f16 x:0 0 0 2024-02-15T18:26:51.670 INFO:tasks.workunit.client.0.smithi042.stdout:9/61: truncate d6/da/f12 84656 0 2024-02-15T18:26:51.670 INFO:tasks.workunit.client.0.smithi042.stdout:9/62: read d6/f8 [371719,30161] 0 2024-02-15T18:26:51.671 INFO:tasks.workunit.client.0.smithi042.stdout:6/98: rename d9/le to d9/d11/l23 0 2024-02-15T18:26:51.673 INFO:tasks.workunit.client.0.smithi042.stdout:9/63: link d6/da/f13 d6/f17 0 2024-02-15T18:26:51.673 INFO:tasks.workunit.client.0.smithi042.stdout:9/64: dread - d6/da/f10 zero size 2024-02-15T18:26:51.673 INFO:tasks.workunit.client.0.smithi042.stdout:9/65: truncate d6/da/f10 173310 0 2024-02-15T18:26:51.674 INFO:tasks.workunit.client.0.smithi042.stdout:6/99: mkdir d9/d11/d19/d24 0 2024-02-15T18:26:51.675 INFO:tasks.workunit.client.0.smithi042.stdout:9/66: creat d6/dd/d9/f18 x:0 0 0 2024-02-15T18:26:51.677 INFO:tasks.workunit.client.0.smithi042.stdout:6/100: creat d9/d11/d19/f25 x:0 0 0 2024-02-15T18:26:51.677 INFO:tasks.workunit.client.0.smithi042.stdout:6/101: dread - d9/d11/d1d/f22 zero size 2024-02-15T18:26:51.678 INFO:tasks.workunit.client.0.smithi042.stdout:9/67: creat d6/dd/f19 x:0 0 0 2024-02-15T18:26:51.681 INFO:tasks.workunit.client.0.smithi042.stdout:6/102: link d9/f14 d9/f26 0 2024-02-15T18:26:51.681 INFO:tasks.workunit.client.0.smithi042.stdout:6/103: truncate d9/f14 747610 0 2024-02-15T18:26:51.684 INFO:tasks.workunit.client.0.smithi042.stdout:6/104: unlink d9/d11/f18 0 2024-02-15T18:26:51.686 INFO:tasks.workunit.client.0.smithi042.stdout:3/73: dwrite d1/fa [0,4194304] 0 2024-02-15T18:26:51.689 INFO:tasks.workunit.client.0.smithi042.stdout:3/74: mknod d1/d11/c12 0 2024-02-15T18:26:51.692 INFO:tasks.workunit.client.0.smithi042.stdout:0/58: dwrite d4/f6 [0,4194304] 0 2024-02-15T18:26:51.692 INFO:tasks.workunit.client.0.smithi042.stdout:0/59: fdatasync d4/f7 0 2024-02-15T18:26:51.692 INFO:tasks.workunit.client.0.smithi042.stdout:0/60: stat d4/l10 0 2024-02-15T18:26:51.692 INFO:tasks.workunit.client.0.smithi042.stdout:0/61: chown d4/d16 32326284 1 2024-02-15T18:26:51.695 INFO:tasks.workunit.client.0.smithi042.stdout:0/62: getdents d4 0 2024-02-15T18:26:51.698 INFO:tasks.workunit.client.0.smithi042.stdout:9/68: getdents d6/dd 0 2024-02-15T18:26:51.698 INFO:tasks.workunit.client.0.smithi042.stdout:3/75: write d1/fa [210159,18600] 0 2024-02-15T18:26:51.699 INFO:tasks.workunit.client.0.smithi042.stdout:0/63: mknod d4/d16/c19 0 2024-02-15T18:26:51.699 INFO:tasks.workunit.client.0.smithi042.stdout:5/36: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:26:51.699 INFO:tasks.workunit.client.0.smithi042.stdout:5/37: fdatasync d0/f3 0 2024-02-15T18:26:51.706 INFO:tasks.workunit.client.0.smithi042.stdout:9/69: mknod d6/dd/d9/c1a 0 2024-02-15T18:26:51.708 INFO:tasks.workunit.client.0.smithi042.stdout:3/76: creat d1/de/f13 x:0 0 0 2024-02-15T18:26:51.709 INFO:tasks.workunit.client.0.smithi042.stdout:5/38: creat d0/fa x:0 0 0 2024-02-15T18:26:51.710 INFO:tasks.workunit.client.0.smithi042.stdout:9/70: rmdir d6/da 39 2024-02-15T18:26:51.710 INFO:tasks.workunit.client.0.smithi042.stdout:9/71: chown d6/fb 1638795 1 2024-02-15T18:26:51.710 INFO:tasks.workunit.client.0.smithi042.stdout:9/72: write d6/da/f10 [463771,112049] 0 2024-02-15T18:26:51.710 INFO:tasks.workunit.client.0.smithi042.stdout:9/73: chown d6 1584000 1 2024-02-15T18:26:51.710 INFO:tasks.workunit.client.0.smithi042.stdout:9/74: dread - d6/dd/f19 zero size 2024-02-15T18:26:51.711 INFO:tasks.workunit.client.0.smithi042.stdout:7/34: dwrite fa [0,4194304] 0 2024-02-15T18:26:51.711 INFO:tasks.workunit.client.0.smithi042.stdout:6/105: dwrite d9/fa [0,4194304] 0 2024-02-15T18:26:51.712 INFO:tasks.workunit.client.0.smithi042.stdout:9/75: symlink d6/dd/l1b 0 2024-02-15T18:26:51.715 INFO:tasks.workunit.client.0.smithi042.stdout:7/35: creat fb x:0 0 0 2024-02-15T18:26:51.717 INFO:tasks.workunit.client.0.smithi042.stdout:0/64: truncate d4/f7 1569773 0 2024-02-15T18:26:51.718 INFO:tasks.workunit.client.0.smithi042.stdout:0/65: read d4/d9/ff [1124811,127583] 0 2024-02-15T18:26:51.719 INFO:tasks.workunit.client.0.smithi042.stdout:5/39: rmdir d0 39 2024-02-15T18:26:51.719 INFO:tasks.workunit.client.0.smithi042.stdout:5/40: stat d0/l2 0 2024-02-15T18:26:51.720 INFO:tasks.workunit.client.0.smithi042.stdout:5/41: fdatasync d0/fa 0 2024-02-15T18:26:51.724 INFO:tasks.workunit.client.0.smithi042.stdout:0/66: getdents d4/d9 0 2024-02-15T18:26:51.724 INFO:tasks.workunit.client.0.smithi042.stdout:0/67: write d4/d9/ff [4310930,65210] 0 2024-02-15T18:26:51.725 INFO:tasks.workunit.client.0.smithi042.stdout:6/106: dread d9/f17 [0,4194304] 0 2024-02-15T18:26:51.725 INFO:tasks.workunit.client.0.smithi042.stdout:5/42: mkdir d0/db 0 2024-02-15T18:26:51.726 INFO:tasks.workunit.client.0.smithi042.stdout:9/76: dwrite d6/f17 [0,4194304] 0 2024-02-15T18:26:51.726 INFO:tasks.workunit.client.0.smithi042.stdout:0/68: unlink d4/l12 0 2024-02-15T18:26:51.727 INFO:tasks.workunit.client.0.smithi042.stdout:6/107: rename d9/d11/d19/f25 to d9/d11/d19/d1c/f27 0 2024-02-15T18:26:51.728 INFO:tasks.workunit.client.0.smithi042.stdout:6/108: chown d9/f26 3828 1 2024-02-15T18:26:51.729 INFO:tasks.workunit.client.0.smithi042.stdout:9/77: creat d6/da/f1c x:0 0 0 2024-02-15T18:26:51.730 INFO:tasks.workunit.client.0.smithi042.stdout:6/109: mknod d9/d11/d19/c28 0 2024-02-15T18:26:51.731 INFO:tasks.workunit.client.0.smithi042.stdout:9/78: mkdir d6/da/d1d 0 2024-02-15T18:26:51.735 INFO:tasks.workunit.client.0.smithi042.stdout:7/36: dwrite fa [0,4194304] 0 2024-02-15T18:26:51.735 INFO:tasks.workunit.client.0.smithi042.stdout:7/37: chown fb 8710695 1 2024-02-15T18:26:51.737 INFO:tasks.workunit.client.0.smithi042.stdout:7/38: chown c8 5292 1 2024-02-15T18:26:51.739 INFO:tasks.workunit.client.0.smithi042.stdout:7/39: symlink lc 0 2024-02-15T18:26:51.740 INFO:tasks.workunit.client.0.smithi042.stdout:7/40: chown c2 1282 1 2024-02-15T18:26:51.743 INFO:tasks.workunit.client.0.smithi042.stdout:0/69: fdatasync d4/d9/ff 0 2024-02-15T18:26:51.745 INFO:tasks.workunit.client.0.smithi042.stdout:5/43: truncate d0/f3 3720838 0 2024-02-15T18:26:51.748 INFO:tasks.workunit.client.0.smithi042.stdout:7/41: link f4 fd 0 2024-02-15T18:26:51.750 INFO:tasks.workunit.client.0.smithi042.stdout:9/79: dwrite d6/dd/d9/f18 [0,4194304] 0 2024-02-15T18:26:51.753 INFO:tasks.workunit.client.0.smithi042.stdout:9/80: symlink d6/da/d1d/l1e 0 2024-02-15T18:26:51.764 INFO:tasks.workunit.client.0.smithi042.stdout:7/42: dwrite f4 [0,4194304] 0 2024-02-15T18:26:51.764 INFO:tasks.workunit.client.0.smithi042.stdout:0/70: dwrite d4/d9/d15/f18 [0,4194304] 0 2024-02-15T18:26:51.766 INFO:tasks.workunit.client.0.smithi042.stdout:5/44: read d0/f3 [1701911,94286] 0 2024-02-15T18:26:51.766 INFO:tasks.workunit.client.0.smithi042.stdout:9/81: rmdir d6/da 39 2024-02-15T18:26:51.768 INFO:tasks.workunit.client.0.smithi042.stdout:7/43: creat fe x:0 0 0 2024-02-15T18:26:51.771 INFO:tasks.workunit.client.0.smithi042.stdout:0/71: rename d4/cb to d4/d16/c1a 0 2024-02-15T18:26:51.772 INFO:tasks.workunit.client.0.smithi042.stdout:5/45: mkdir d0/dc 0 2024-02-15T18:26:51.773 INFO:tasks.workunit.client.0.smithi042.stdout:0/72: symlink d4/d9/d15/l1b 0 2024-02-15T18:26:51.774 INFO:tasks.workunit.client.0.smithi042.stdout:5/46: creat d0/db/fd x:0 0 0 2024-02-15T18:26:51.775 INFO:tasks.workunit.client.0.smithi042.stdout:7/44: link fe ff 0 2024-02-15T18:26:51.776 INFO:tasks.workunit.client.0.smithi042.stdout:7/45: mkdir d10 0 2024-02-15T18:26:51.778 INFO:tasks.workunit.client.0.smithi042.stdout:7/46: rename lc to d10/l11 0 2024-02-15T18:26:51.778 INFO:tasks.workunit.client.0.smithi042.stdout:7/47: write ff [779638,80608] 0 2024-02-15T18:26:51.779 INFO:tasks.workunit.client.0.smithi042.stdout:7/48: mkdir d10/d12 0 2024-02-15T18:26:51.779 INFO:tasks.workunit.client.0.smithi042.stdout:7/49: fsync fb 0 2024-02-15T18:26:51.780 INFO:tasks.workunit.client.0.smithi042.stdout:7/50: symlink d10/d12/l13 0 2024-02-15T18:26:51.780 INFO:tasks.workunit.client.0.smithi042.stdout:7/51: rmdir d10/d12 39 2024-02-15T18:26:51.796 INFO:tasks.workunit.client.0.smithi042.stdout:0/73: dwrite d4/fe [4194304,4194304] 0 2024-02-15T18:26:51.797 INFO:tasks.workunit.client.0.smithi042.stdout:0/74: read d4/fc [530755,120179] 0 2024-02-15T18:26:51.798 INFO:tasks.workunit.client.0.smithi042.stdout:0/75: mknod d4/d9/c1c 0 2024-02-15T18:26:51.798 INFO:tasks.workunit.client.0.smithi042.stdout:0/76: fsync d4/fe 0 2024-02-15T18:26:51.800 INFO:tasks.workunit.client.0.smithi042.stdout:0/77: mknod d4/c1d 0 2024-02-15T18:26:51.801 INFO:tasks.workunit.client.0.smithi042.stdout:0/78: symlink d4/d9/d15/l1e 0 2024-02-15T18:26:51.801 INFO:tasks.workunit.client.0.smithi042.stdout:0/79: chown d4/d9/f14 1266 1 2024-02-15T18:26:51.802 INFO:tasks.workunit.client.0.smithi042.stdout:0/80: symlink d4/d9/l1f 0 2024-02-15T18:26:51.802 INFO:tasks.workunit.client.0.smithi042.stdout:0/81: readlink d4/d9/l17 0 2024-02-15T18:26:51.802 INFO:tasks.workunit.client.0.smithi042.stdout:0/82: creat d4/f20 x:0 0 0 2024-02-15T18:26:51.803 INFO:tasks.workunit.client.0.smithi042.stdout:0/83: creat d4/d9/d15/f21 x:0 0 0 2024-02-15T18:26:51.804 INFO:tasks.workunit.client.0.smithi042.stdout:0/84: creat d4/d9/f22 x:0 0 0 2024-02-15T18:26:51.807 INFO:tasks.workunit.client.0.smithi042.stdout:7/52: dwrite fa [0,4194304] 0 2024-02-15T18:26:51.807 INFO:tasks.workunit.client.0.smithi042.stdout:7/53: write fb [224476,91178] 0 2024-02-15T18:26:51.807 INFO:tasks.workunit.client.0.smithi042.stdout:5/47: dwrite d0/db/fd [0,4194304] 0 2024-02-15T18:26:51.808 INFO:tasks.workunit.client.0.smithi042.stdout:7/54: creat d10/f14 x:0 0 0 2024-02-15T18:26:51.810 INFO:tasks.workunit.client.0.smithi042.stdout:0/85: unlink d4/d16/c1a 0 2024-02-15T18:26:51.831 INFO:tasks.workunit.client.0.smithi042.stdout:0/86: dread d4/d9/d15/f18 [0,4194304] 0 2024-02-15T18:26:51.832 INFO:tasks.workunit.client.0.smithi042.stdout:5/48: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:26:51.834 INFO:tasks.workunit.client.0.smithi042.stdout:7/55: dwrite fd [4194304,4194304] 0 2024-02-15T18:26:51.834 INFO:tasks.workunit.client.0.smithi042.stdout:7/56: readlink d10/d12/l13 0 2024-02-15T18:26:51.834 INFO:tasks.workunit.client.0.smithi042.stdout:1/70: sync 2024-02-15T18:26:51.835 INFO:tasks.workunit.client.0.smithi042.stdout:1/71: mkdir de/d16 0 2024-02-15T18:26:51.837 INFO:tasks.workunit.client.0.smithi042.stdout:7/57: truncate fa 1113878 0 2024-02-15T18:26:51.837 INFO:tasks.workunit.client.0.smithi042.stdout:7/58: dread - d10/f14 zero size 2024-02-15T18:26:51.837 INFO:tasks.workunit.client.0.smithi042.stdout:7/59: fdatasync fe 0 2024-02-15T18:26:51.839 INFO:tasks.workunit.client.0.smithi042.stdout:1/72: rename de/l14 to de/d15/l17 0 2024-02-15T18:26:51.839 INFO:tasks.workunit.client.0.smithi042.stdout:1/73: stat de/d16 0 2024-02-15T18:26:51.840 INFO:tasks.workunit.client.0.smithi042.stdout:7/60: mkdir d10/d15 0 2024-02-15T18:26:51.840 INFO:tasks.workunit.client.0.smithi042.stdout:7/61: fdatasync f4 0 2024-02-15T18:26:51.841 INFO:tasks.workunit.client.0.smithi042.stdout:7/62: write fb [110172,108847] 0 2024-02-15T18:26:51.842 INFO:tasks.workunit.client.0.smithi042.stdout:1/74: creat de/d16/f18 x:0 0 0 2024-02-15T18:26:51.843 INFO:tasks.workunit.client.0.smithi042.stdout:1/75: write f2 [2805058,37837] 0 2024-02-15T18:26:51.843 INFO:tasks.workunit.client.0.smithi042.stdout:7/63: creat d10/d12/f16 x:0 0 0 2024-02-15T18:26:51.846 INFO:tasks.workunit.client.0.smithi042.stdout:1/76: rename fd to de/d16/f19 0 2024-02-15T18:26:51.846 INFO:tasks.workunit.client.0.smithi042.stdout:1/77: chown de/d15/l17 227 1 2024-02-15T18:26:51.855 INFO:tasks.workunit.client.0.smithi042.stdout:0/87: dwrite d4/f20 [0,4194304] 0 2024-02-15T18:26:51.857 INFO:tasks.workunit.client.0.smithi042.stdout:5/49: dwrite d0/db/fd [0,4194304] 0 2024-02-15T18:26:51.857 INFO:tasks.workunit.client.0.smithi042.stdout:5/50: fsync d0/f3 0 2024-02-15T18:26:51.857 INFO:tasks.workunit.client.0.smithi042.stdout:5/51: dread - d0/fa zero size 2024-02-15T18:26:51.857 INFO:tasks.workunit.client.0.smithi042.stdout:0/88: mknod d4/c23 0 2024-02-15T18:26:51.861 INFO:tasks.workunit.client.0.smithi042.stdout:7/64: rmdir d10 39 2024-02-15T18:26:51.863 INFO:tasks.workunit.client.0.smithi042.stdout:5/52: rmdir d0/dc 0 2024-02-15T18:26:51.864 INFO:tasks.workunit.client.0.smithi042.stdout:0/89: link d4/f7 d4/d16/f24 0 2024-02-15T18:26:51.866 INFO:tasks.workunit.client.0.smithi042.stdout:7/65: rename f4 to d10/f17 0 2024-02-15T18:26:51.866 INFO:tasks.workunit.client.0.smithi042.stdout:7/66: chown d10/d12/l13 29625 1 2024-02-15T18:26:51.866 INFO:tasks.workunit.client.0.smithi042.stdout:7/67: chown d10/f14 7709210 1 2024-02-15T18:26:51.866 INFO:tasks.workunit.client.0.smithi042.stdout:7/68: readlink d10/l11 0 2024-02-15T18:26:51.867 INFO:tasks.workunit.client.0.smithi042.stdout:5/53: creat d0/fe x:0 0 0 2024-02-15T18:26:51.867 INFO:tasks.workunit.client.0.smithi042.stdout:7/69: write fd [7994827,129358] 0 2024-02-15T18:26:51.867 INFO:tasks.workunit.client.0.smithi042.stdout:7/70: getdents d10/d15 0 2024-02-15T18:26:51.867 INFO:tasks.workunit.client.0.smithi042.stdout:0/90: truncate d4/d9/ff 339895 0 2024-02-15T18:26:51.868 INFO:tasks.workunit.client.0.smithi042.stdout:5/54: creat d0/db/ff x:0 0 0 2024-02-15T18:26:51.869 INFO:tasks.workunit.client.0.smithi042.stdout:7/71: rename c8 to d10/d12/c18 0 2024-02-15T18:26:51.870 INFO:tasks.workunit.client.0.smithi042.stdout:7/72: write ff [1025824,75912] 0 2024-02-15T18:26:51.870 INFO:tasks.workunit.client.0.smithi042.stdout:5/55: creat d0/db/f10 x:0 0 0 2024-02-15T18:26:51.870 INFO:tasks.workunit.client.0.smithi042.stdout:0/91: dread d4/d9/f14 [0,4194304] 0 2024-02-15T18:26:51.871 INFO:tasks.workunit.client.0.smithi042.stdout:5/56: symlink d0/l11 0 2024-02-15T18:26:51.871 INFO:tasks.workunit.client.0.smithi042.stdout:5/57: truncate d0/db/f10 938134 0 2024-02-15T18:26:51.872 INFO:tasks.workunit.client.0.smithi042.stdout:1/78: dwrite f2 [0,4194304] 0 2024-02-15T18:26:51.874 INFO:tasks.workunit.client.0.smithi042.stdout:0/92: rename d4/fc to d4/d9/d15/f25 0 2024-02-15T18:26:51.880 INFO:tasks.workunit.client.0.smithi042.stdout:0/93: mknod d4/d9/c26 0 2024-02-15T18:26:51.880 INFO:tasks.workunit.client.0.smithi042.stdout:5/58: getdents d0/db 0 2024-02-15T18:26:51.882 INFO:tasks.workunit.client.0.smithi042.stdout:5/59: truncate d0/db/fd 3501037 0 2024-02-15T18:26:51.889 INFO:tasks.workunit.client.0.smithi042.stdout:0/94: dread d4/d9/d15/f25 [0,4194304] 0 2024-02-15T18:26:51.902 INFO:tasks.workunit.client.0.smithi042.stdout:7/73: dwrite d10/f17 [0,4194304] 0 2024-02-15T18:26:51.904 INFO:tasks.workunit.client.0.smithi042.stdout:1/79: dwrite f2 [4194304,4194304] 0 2024-02-15T18:26:51.904 INFO:tasks.workunit.client.0.smithi042.stdout:7/74: mknod d10/d15/c19 0 2024-02-15T18:26:51.904 INFO:tasks.workunit.client.0.smithi042.stdout:0/95: dread d4/fe [0,4194304] 0 2024-02-15T18:26:51.904 INFO:tasks.workunit.client.0.smithi042.stdout:0/96: truncate d4/d16/f24 2114458 0 2024-02-15T18:26:51.904 INFO:tasks.workunit.client.0.smithi042.stdout:5/60: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:26:51.907 INFO:tasks.workunit.client.0.smithi042.stdout:5/61: write d0/db/fd [2414596,127634] 0 2024-02-15T18:26:51.907 INFO:tasks.workunit.client.0.smithi042.stdout:1/80: mknod de/c1a 0 2024-02-15T18:26:51.907 INFO:tasks.workunit.client.0.smithi042.stdout:1/81: fdatasync fc 0 2024-02-15T18:26:51.907 INFO:tasks.workunit.client.0.smithi042.stdout:1/82: chown de/c12 26519 1 2024-02-15T18:26:51.909 INFO:tasks.workunit.client.0.smithi042.stdout:7/75: unlink fe 0 2024-02-15T18:26:51.909 INFO:tasks.workunit.client.0.smithi042.stdout:0/97: creat d4/d9/d15/f27 x:0 0 0 2024-02-15T18:26:51.909 INFO:tasks.workunit.client.0.smithi042.stdout:0/98: chown d4/d9/c26 64257 1 2024-02-15T18:26:51.910 INFO:tasks.workunit.client.0.smithi042.stdout:0/99: chown d4/d9/l17 21263241 1 2024-02-15T18:26:51.912 INFO:tasks.workunit.client.0.smithi042.stdout:1/83: mkdir de/d15/d1b 0 2024-02-15T18:26:51.912 INFO:tasks.workunit.client.0.smithi042.stdout:8/38: sync 2024-02-15T18:26:51.912 INFO:tasks.workunit.client.0.smithi042.stdout:4/62: sync 2024-02-15T18:26:51.915 INFO:tasks.workunit.client.0.smithi042.stdout:7/76: rename ff to d10/d15/f1a 0 2024-02-15T18:26:51.916 INFO:tasks.workunit.client.0.smithi042.stdout:3/77: sync 2024-02-15T18:26:51.917 INFO:tasks.workunit.client.0.smithi042.stdout:6/110: sync 2024-02-15T18:26:51.917 INFO:tasks.workunit.client.0.smithi042.stdout:2/51: sync 2024-02-15T18:26:51.917 INFO:tasks.workunit.client.0.smithi042.stdout:9/82: sync 2024-02-15T18:26:51.918 INFO:tasks.workunit.client.0.smithi042.stdout:0/100: mkdir d4/d9/d28 0 2024-02-15T18:26:51.918 INFO:tasks.workunit.client.0.smithi042.stdout:0/101: chown d4/d9/l1f 235 1 2024-02-15T18:26:51.919 INFO:tasks.workunit.client.0.smithi042.stdout:1/84: mknod de/d15/c1c 0 2024-02-15T18:26:51.920 INFO:tasks.workunit.client.0.smithi042.stdout:4/63: truncate f9 3289957 0 2024-02-15T18:26:51.920 INFO:tasks.workunit.client.0.smithi042.stdout:4/64: chown da/db 1342 1 2024-02-15T18:26:51.920 INFO:tasks.workunit.client.0.smithi042.stdout:4/65: chown c3 1761672 1 2024-02-15T18:26:51.921 INFO:tasks.workunit.client.0.smithi042.stdout:3/78: rmdir d1/d11 39 2024-02-15T18:26:51.923 INFO:tasks.workunit.client.0.smithi042.stdout:6/111: mknod d9/d11/c29 0 2024-02-15T18:26:51.924 INFO:tasks.workunit.client.0.smithi042.stdout:2/52: dread d0/f3 [0,4194304] 0 2024-02-15T18:26:51.924 INFO:tasks.workunit.client.0.smithi042.stdout:9/83: link d6/da/f10 d6/dd/d9/f1f 0 2024-02-15T18:26:51.926 INFO:tasks.workunit.client.0.smithi042.stdout:0/102: link d4/d9/c26 d4/d9/d28/c29 0 2024-02-15T18:26:51.926 INFO:tasks.workunit.client.0.smithi042.stdout:0/103: chown d4/c13 39 1 2024-02-15T18:26:51.927 INFO:tasks.workunit.client.0.smithi042.stdout:1/85: creat de/d16/f1d x:0 0 0 2024-02-15T18:26:51.927 INFO:tasks.workunit.client.0.smithi042.stdout:1/86: read - de/d16/f1d zero size 2024-02-15T18:26:51.927 INFO:tasks.workunit.client.0.smithi042.stdout:1/87: fsync de/d16/f19 0 2024-02-15T18:26:51.928 INFO:tasks.workunit.client.0.smithi042.stdout:4/66: mknod da/db/cf 0 2024-02-15T18:26:51.928 INFO:tasks.workunit.client.0.smithi042.stdout:5/62: dwrite d0/fa [0,4194304] 0 2024-02-15T18:26:51.929 INFO:tasks.workunit.client.0.smithi042.stdout:8/39: dwrite fc [0,4194304] 0 2024-02-15T18:26:51.929 INFO:tasks.workunit.client.0.smithi042.stdout:8/40: rmdir - no directory 2024-02-15T18:26:51.929 INFO:tasks.workunit.client.0.smithi042.stdout:8/41: chown c5 74 1 2024-02-15T18:26:51.929 INFO:tasks.workunit.client.0.smithi042.stdout:3/79: mkdir d1/d11/d14 0 2024-02-15T18:26:51.930 INFO:tasks.workunit.client.0.smithi042.stdout:7/77: unlink d10/d15/f1a 0 2024-02-15T18:26:51.932 INFO:tasks.workunit.client.0.smithi042.stdout:4/67: write f9 [3884545,90324] 0 2024-02-15T18:26:51.932 INFO:tasks.workunit.client.0.smithi042.stdout:6/112: creat d9/d11/d19/f2a x:0 0 0 2024-02-15T18:26:51.933 INFO:tasks.workunit.client.0.smithi042.stdout:1/88: symlink de/l1e 0 2024-02-15T18:26:51.937 INFO:tasks.workunit.client.0.smithi042.stdout:5/63: unlink d0/l11 0 2024-02-15T18:26:51.941 INFO:tasks.workunit.client.0.smithi042.stdout:8/42: mkdir dd 0 2024-02-15T18:26:51.941 INFO:tasks.workunit.client.0.smithi042.stdout:3/80: creat d1/f15 x:0 0 0 2024-02-15T18:26:51.942 INFO:tasks.workunit.client.0.smithi042.stdout:3/81: truncate d1/de/f13 204655 0 2024-02-15T18:26:51.944 INFO:tasks.workunit.client.0.smithi042.stdout:7/78: link d10/f14 d10/f1b 0 2024-02-15T18:26:51.947 INFO:tasks.workunit.client.0.smithi042.stdout:6/113: creat d9/d11/f2b x:0 0 0 2024-02-15T18:26:51.948 INFO:tasks.workunit.client.0.smithi042.stdout:1/89: symlink de/d15/l1f 0 2024-02-15T18:26:51.949 INFO:tasks.workunit.client.0.smithi042.stdout:5/64: mkdir d0/d12 0 2024-02-15T18:26:51.949 INFO:tasks.workunit.client.0.smithi042.stdout:5/65: chown d0/f3 487 1 2024-02-15T18:26:51.950 INFO:tasks.workunit.client.0.smithi042.stdout:3/82: creat d1/de/f16 x:0 0 0 2024-02-15T18:26:51.951 INFO:tasks.workunit.client.0.smithi042.stdout:7/79: symlink d10/d12/l1c 0 2024-02-15T18:26:51.951 INFO:tasks.workunit.client.0.smithi042.stdout:7/80: chown d10/l11 688299 1 2024-02-15T18:26:51.951 INFO:tasks.workunit.client.0.smithi042.stdout:6/114: symlink d9/d11/d1d/l2c 0 2024-02-15T18:26:51.951 INFO:tasks.workunit.client.0.smithi042.stdout:6/115: truncate d9/d11/f20 697019 0 2024-02-15T18:26:51.953 INFO:tasks.workunit.client.0.smithi042.stdout:1/90: creat de/d15/f20 x:0 0 0 2024-02-15T18:26:51.953 INFO:tasks.workunit.client.0.smithi042.stdout:1/91: chown de/l1e 2986525 1 2024-02-15T18:26:51.953 INFO:tasks.workunit.client.0.smithi042.stdout:3/83: write d1/f9 [2069438,120794] 0 2024-02-15T18:26:51.954 INFO:tasks.workunit.client.0.smithi042.stdout:5/66: rename d0/c6 to d0/db/c13 0 2024-02-15T18:26:51.955 INFO:tasks.workunit.client.0.smithi042.stdout:5/67: write d0/db/f10 [1889021,69918] 0 2024-02-15T18:26:51.956 INFO:tasks.workunit.client.0.smithi042.stdout:7/81: creat d10/d15/f1d x:0 0 0 2024-02-15T18:26:51.956 INFO:tasks.workunit.client.0.smithi042.stdout:7/82: chown c9 13847 1 2024-02-15T18:26:51.957 INFO:tasks.workunit.client.0.smithi042.stdout:6/116: unlink d9/d11/d1d/l2c 0 2024-02-15T18:26:51.957 INFO:tasks.workunit.client.0.smithi042.stdout:1/92: chown c7 18 1 2024-02-15T18:26:51.958 INFO:tasks.workunit.client.0.smithi042.stdout:3/84: creat d1/d11/f17 x:0 0 0 2024-02-15T18:26:51.959 INFO:tasks.workunit.client.0.smithi042.stdout:5/68: mkdir d0/d14 0 2024-02-15T18:26:51.959 INFO:tasks.workunit.client.0.smithi042.stdout:5/69: truncate d0/fe 61345 0 2024-02-15T18:26:51.959 INFO:tasks.workunit.client.0.smithi042.stdout:7/83: mknod d10/c1e 0 2024-02-15T18:26:51.959 INFO:tasks.workunit.client.0.smithi042.stdout:7/84: dread - d10/d15/f1d zero size 2024-02-15T18:26:51.961 INFO:tasks.workunit.client.0.smithi042.stdout:3/85: symlink d1/de/l18 0 2024-02-15T18:26:51.961 INFO:tasks.workunit.client.0.smithi042.stdout:3/86: stat d1 0 2024-02-15T18:26:51.962 INFO:tasks.workunit.client.0.smithi042.stdout:7/85: mknod d10/d12/c1f 0 2024-02-15T18:26:51.963 INFO:tasks.workunit.client.0.smithi042.stdout:3/87: unlink d1/d11/f17 0 2024-02-15T18:26:51.963 INFO:tasks.workunit.client.0.smithi042.stdout:2/53: dwrite d0/d8/db/fd [0,4194304] 0 2024-02-15T18:26:51.964 INFO:tasks.workunit.client.0.smithi042.stdout:1/93: dread fb [0,4194304] 0 2024-02-15T18:26:51.964 INFO:tasks.workunit.client.0.smithi042.stdout:0/104: dwrite d4/f20 [0,4194304] 0 2024-02-15T18:26:51.965 INFO:tasks.workunit.client.0.smithi042.stdout:9/84: dwrite d6/da/f1c [0,4194304] 0 2024-02-15T18:26:51.965 INFO:tasks.workunit.client.0.smithi042.stdout:9/85: truncate d6/f8 3466802 0 2024-02-15T18:26:51.965 INFO:tasks.workunit.client.0.smithi042.stdout:9/86: stat d6/da/f12 0 2024-02-15T18:26:51.966 INFO:tasks.workunit.client.0.smithi042.stdout:7/86: write d10/f17 [6982874,36144] 0 2024-02-15T18:26:51.966 INFO:tasks.workunit.client.0.smithi042.stdout:4/68: rename da/db to da/d10 0 2024-02-15T18:26:51.970 INFO:tasks.workunit.client.0.smithi042.stdout:3/88: mkdir d1/de/d19 0 2024-02-15T18:26:51.973 INFO:tasks.workunit.client.0.smithi042.stdout:5/70: dwrite d0/fe [0,4194304] 0 2024-02-15T18:26:51.978 INFO:tasks.workunit.client.0.smithi042.stdout:6/117: rmdir d9/d11/d1d 39 2024-02-15T18:26:51.978 INFO:tasks.workunit.client.0.smithi042.stdout:6/118: read d9/d11/f20 [369856,56953] 0 2024-02-15T18:26:51.978 INFO:tasks.workunit.client.0.smithi042.stdout:6/119: truncate d9/f21 4440847 0 2024-02-15T18:26:51.978 INFO:tasks.workunit.client.0.smithi042.stdout:6/120: chown d9/d11/c16 121149 1 2024-02-15T18:26:51.979 INFO:tasks.workunit.client.0.smithi042.stdout:2/54: creat d0/d8/f11 x:0 0 0 2024-02-15T18:26:51.980 INFO:tasks.workunit.client.0.smithi042.stdout:2/55: truncate d0/d8/f11 443888 0 2024-02-15T18:26:51.981 INFO:tasks.workunit.client.0.smithi042.stdout:0/105: rmdir d4/d9 39 2024-02-15T18:26:51.981 INFO:tasks.workunit.client.0.smithi042.stdout:0/106: readlink d4/d9/d15/l1b 0 2024-02-15T18:26:51.981 INFO:tasks.workunit.client.0.smithi042.stdout:0/107: fsync d4/f20 0 2024-02-15T18:26:51.983 INFO:tasks.workunit.client.0.smithi042.stdout:1/94: getdents de/d15 0 2024-02-15T18:26:51.983 INFO:tasks.workunit.client.0.smithi042.stdout:1/95: chown de/d16/f18 145256 1 2024-02-15T18:26:51.988 INFO:tasks.workunit.client.0.smithi042.stdout:9/87: link f2 d6/dd/f20 0 2024-02-15T18:26:51.989 INFO:tasks.workunit.client.0.smithi042.stdout:7/87: creat d10/d12/f20 x:0 0 0 2024-02-15T18:26:51.990 INFO:tasks.workunit.client.0.smithi042.stdout:7/88: dread - d10/f14 zero size 2024-02-15T18:26:51.990 INFO:tasks.workunit.client.0.smithi042.stdout:4/69: rename da/d10 to da/d11 0 2024-02-15T18:26:51.991 INFO:tasks.workunit.client.0.smithi042.stdout:3/89: mkdir d1/de/d1a 0 2024-02-15T18:26:51.991 INFO:tasks.workunit.client.0.smithi042.stdout:3/90: write d1/de/f10 [423343,131028] 0 2024-02-15T18:26:51.991 INFO:tasks.workunit.client.0.smithi042.stdout:3/91: stat d1/f15 0 2024-02-15T18:26:51.993 INFO:tasks.workunit.client.0.smithi042.stdout:5/71: link d0/fa d0/db/f15 0 2024-02-15T18:26:51.995 INFO:tasks.workunit.client.0.smithi042.stdout:6/121: creat d9/f2d x:0 0 0 2024-02-15T18:26:51.996 INFO:tasks.workunit.client.0.smithi042.stdout:5/72: write d0/f3 [1915736,59956] 0 2024-02-15T18:26:51.999 INFO:tasks.workunit.client.0.smithi042.stdout:2/56: creat d0/d8/d10/f12 x:0 0 0 2024-02-15T18:26:52.003 INFO:tasks.workunit.client.0.smithi042.stdout:0/108: mknod d4/d9/d28/c2a 0 2024-02-15T18:26:52.009 INFO:tasks.workunit.client.0.smithi042.stdout:1/96: rename de/d16/f19 to de/f21 0 2024-02-15T18:26:52.009 INFO:tasks.workunit.client.0.smithi042.stdout:1/97: write de/d15/f20 [300288,106860] 0 2024-02-15T18:26:52.010 INFO:tasks.workunit.client.0.smithi042.stdout:9/88: rmdir d6/da 39 2024-02-15T18:26:52.011 INFO:tasks.workunit.client.0.smithi042.stdout:4/70: rmdir da 39 2024-02-15T18:26:52.011 INFO:tasks.workunit.client.0.smithi042.stdout:4/71: chown da/d11 37 1 2024-02-15T18:26:52.013 INFO:tasks.workunit.client.0.smithi042.stdout:5/73: creat d0/d14/f16 x:0 0 0 2024-02-15T18:26:52.015 INFO:tasks.workunit.client.0.smithi042.stdout:0/109: getdents d4/d9/d15 0 2024-02-15T18:26:52.016 INFO:tasks.workunit.client.0.smithi042.stdout:9/89: rename d6/fb to d6/f21 0 2024-02-15T18:26:52.016 INFO:tasks.workunit.client.0.smithi042.stdout:5/74: dread d0/db/f10 [0,4194304] 0 2024-02-15T18:26:52.016 INFO:tasks.workunit.client.0.smithi042.stdout:9/90: rename d6 to d6/dd/d9/d22 22 2024-02-15T18:26:52.016 INFO:tasks.workunit.client.0.smithi042.stdout:4/72: mknod da/d11/c12 0 2024-02-15T18:26:52.016 INFO:tasks.workunit.client.0.smithi042.stdout:4/73: chown da/cc 49 1 2024-02-15T18:26:52.018 INFO:tasks.workunit.client.0.smithi042.stdout:0/110: link d4/d9/c1c d4/d9/d28/c2b 0 2024-02-15T18:26:52.018 INFO:tasks.workunit.client.0.smithi042.stdout:0/111: chown d4/f7 2436 1 2024-02-15T18:26:52.018 INFO:tasks.workunit.client.0.smithi042.stdout:5/75: symlink d0/l17 0 2024-02-15T18:26:52.019 INFO:tasks.workunit.client.0.smithi042.stdout:4/74: link c5 da/d11/c13 0 2024-02-15T18:26:52.020 INFO:tasks.workunit.client.0.smithi042.stdout:0/112: symlink d4/l2c 0 2024-02-15T18:26:52.021 INFO:tasks.workunit.client.0.smithi042.stdout:5/76: mknod d0/db/c18 0 2024-02-15T18:26:52.022 INFO:tasks.workunit.client.0.smithi042.stdout:9/91: link f4 d6/da/f23 0 2024-02-15T18:26:52.022 INFO:tasks.workunit.client.0.smithi042.stdout:5/77: dread d0/db/f10 [0,4194304] 0 2024-02-15T18:26:52.023 INFO:tasks.workunit.client.0.smithi042.stdout:3/92: dwrite d1/de/f13 [0,4194304] 0 2024-02-15T18:26:52.023 INFO:tasks.workunit.client.0.smithi042.stdout:4/75: rename l8 to da/l14 0 2024-02-15T18:26:52.024 INFO:tasks.workunit.client.0.smithi042.stdout:4/76: write da/d11/fe [960523,25251] 0 2024-02-15T18:26:52.024 INFO:tasks.workunit.client.0.smithi042.stdout:7/89: dwrite fa [0,4194304] 0 2024-02-15T18:26:52.024 INFO:tasks.workunit.client.0.smithi042.stdout:4/77: stat da/d11/c12 0 2024-02-15T18:26:52.024 INFO:tasks.workunit.client.0.smithi042.stdout:2/57: dwrite d0/d8/db/fd [0,4194304] 0 2024-02-15T18:26:52.025 INFO:tasks.workunit.client.0.smithi042.stdout:0/113: link d4/d9/ff d4/d9/d28/f2d 0 2024-02-15T18:26:52.027 INFO:tasks.workunit.client.0.smithi042.stdout:4/78: fsync da/d11/fe 0 2024-02-15T18:26:52.028 INFO:tasks.workunit.client.0.smithi042.stdout:9/92: creat d6/da/d1d/f24 x:0 0 0 2024-02-15T18:26:52.033 INFO:tasks.workunit.client.0.smithi042.stdout:5/78: dread d0/fa [0,4194304] 0 2024-02-15T18:26:52.033 INFO:tasks.workunit.client.0.smithi042.stdout:1/98: truncate de/d15/f20 249822 0 2024-02-15T18:26:52.035 INFO:tasks.workunit.client.0.smithi042.stdout:3/93: symlink d1/de/d19/l1b 0 2024-02-15T18:26:52.036 INFO:tasks.workunit.client.0.smithi042.stdout:3/94: dread d1/de/f10 [0,4194304] 0 2024-02-15T18:26:52.040 INFO:tasks.workunit.client.0.smithi042.stdout:7/90: creat d10/f21 x:0 0 0 2024-02-15T18:26:52.041 INFO:tasks.workunit.client.0.smithi042.stdout:2/58: mknod d0/d8/c13 0 2024-02-15T18:26:52.042 INFO:tasks.workunit.client.0.smithi042.stdout:2/59: dread d0/d8/f11 [0,4194304] 0 2024-02-15T18:26:52.044 INFO:tasks.workunit.client.0.smithi042.stdout:0/114: rename d4/d9/d28 to d4/d16/d2e 0 2024-02-15T18:26:52.045 INFO:tasks.workunit.client.0.smithi042.stdout:9/93: creat d6/da/f25 x:0 0 0 2024-02-15T18:26:52.046 INFO:tasks.workunit.client.0.smithi042.stdout:5/79: symlink d0/db/l19 0 2024-02-15T18:26:52.048 INFO:tasks.workunit.client.0.smithi042.stdout:1/99: truncate fb 118724 0 2024-02-15T18:26:52.049 INFO:tasks.workunit.client.0.smithi042.stdout:1/100: dread - de/d16/f18 zero size 2024-02-15T18:26:52.049 INFO:tasks.workunit.client.0.smithi042.stdout:3/95: creat d1/d11/f1c x:0 0 0 2024-02-15T18:26:52.049 INFO:tasks.workunit.client.0.smithi042.stdout:3/96: getdents d1/de/d1a 0 2024-02-15T18:26:52.051 INFO:tasks.workunit.client.0.smithi042.stdout:7/91: rename d10/f17 to d10/f22 0 2024-02-15T18:26:52.051 INFO:tasks.workunit.client.0.smithi042.stdout:2/60: dread d0/f7 [0,4194304] 0 2024-02-15T18:26:52.054 INFO:tasks.workunit.client.0.smithi042.stdout:0/115: rmdir d4/d16 39 2024-02-15T18:26:52.054 INFO:tasks.workunit.client.0.smithi042.stdout:0/116: fsync d4/f20 0 2024-02-15T18:26:52.055 INFO:tasks.workunit.client.0.smithi042.stdout:9/94: symlink d6/dd/l26 0 2024-02-15T18:26:52.057 INFO:tasks.workunit.client.0.smithi042.stdout:5/80: rename d0/c7 to d0/db/c1a 0 2024-02-15T18:26:52.058 INFO:tasks.workunit.client.0.smithi042.stdout:1/101: symlink de/d15/l22 0 2024-02-15T18:26:52.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:50.679661+0000 mgr.smithi042.kfoihb (mgr.24435) 33 : cluster [DBG] pgmap v24: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.7 MiB/s rd, 8.8 MiB/s wr, 376 op/s 2024-02-15T18:26:52.059 INFO:tasks.workunit.client.0.smithi042.stdout:3/97: truncate d1/fa 175835 0 2024-02-15T18:26:52.060 INFO:tasks.workunit.client.0.smithi042.stdout:3/98: truncate d1/de/f10 795588 0 2024-02-15T18:26:52.067 INFO:tasks.workunit.client.0.smithi042.stdout:3/99: dread d1/f7 [0,4194304] 0 2024-02-15T18:26:52.067 INFO:tasks.workunit.client.0.smithi042.stdout:3/100: dread - d1/d11/f1c zero size 2024-02-15T18:26:52.067 INFO:tasks.workunit.client.0.smithi042.stdout:7/92: mkdir d10/d12/d23 0 2024-02-15T18:26:52.067 INFO:tasks.workunit.client.0.smithi042.stdout:0/117: mknod d4/d9/d15/c2f 0 2024-02-15T18:26:52.067 INFO:tasks.workunit.client.0.smithi042.stdout:9/95: symlink d6/dd/l27 0 2024-02-15T18:26:52.067 INFO:tasks.workunit.client.0.smithi042.stdout:5/81: symlink d0/d12/l1b 0 2024-02-15T18:26:52.067 INFO:tasks.workunit.client.0.smithi042.stdout:9/96: write d6/da/f13 [4859265,74898] 0 2024-02-15T18:26:52.067 INFO:tasks.workunit.client.0.smithi042.stdout:9/97: fsync d6/dd/f20 0 2024-02-15T18:26:52.069 INFO:tasks.workunit.client.0.smithi042.stdout:3/101: symlink d1/de/d19/l1d 0 2024-02-15T18:26:52.069 INFO:tasks.workunit.client.0.smithi042.stdout:3/102: truncate d1/fa 1015603 0 2024-02-15T18:26:52.069 INFO:tasks.workunit.client.0.smithi042.stdout:3/103: chown d1/d11 5 1 2024-02-15T18:26:52.069 INFO:tasks.workunit.client.0.smithi042.stdout:3/104: stat d1/f15 0 2024-02-15T18:26:52.070 INFO:tasks.workunit.client.0.smithi042.stdout:3/105: truncate d1/de/f16 525823 0 2024-02-15T18:26:52.070 INFO:tasks.workunit.client.0.smithi042.stdout:3/106: write d1/d11/f1c [863135,106235] 0 2024-02-15T18:26:52.070 INFO:tasks.workunit.client.0.smithi042.stdout:7/93: mknod d10/d12/c24 0 2024-02-15T18:26:52.070 INFO:tasks.workunit.client.0.smithi042.stdout:7/94: dread - d10/d15/f1d zero size 2024-02-15T18:26:52.071 INFO:tasks.workunit.client.0.smithi042.stdout:5/82: truncate d0/db/f10 1295807 0 2024-02-15T18:26:52.072 INFO:tasks.workunit.client.0.smithi042.stdout:2/61: dwrite d0/f7 [4194304,4194304] 0 2024-02-15T18:26:52.072 INFO:tasks.workunit.client.0.smithi042.stdout:2/62: getdents d0/d8/db/de 0 2024-02-15T18:26:52.073 INFO:tasks.workunit.client.0.smithi042.stdout:2/63: stat d0/d8 0 2024-02-15T18:26:52.075 INFO:tasks.workunit.client.0.smithi042.stdout:7/95: rename d10/d12/l1c to d10/d15/l25 0 2024-02-15T18:26:52.075 INFO:tasks.workunit.client.0.smithi042.stdout:5/83: creat d0/d14/f1c x:0 0 0 2024-02-15T18:26:52.075 INFO:tasks.workunit.client.0.smithi042.stdout:3/107: dread d1/de/f13 [0,4194304] 0 2024-02-15T18:26:52.075 INFO:tasks.workunit.client.0.smithi042.stdout:0/118: rmdir d4 39 2024-02-15T18:26:52.079 INFO:tasks.workunit.client.0.smithi042.stdout:2/64: creat d0/d8/db/de/f14 x:0 0 0 2024-02-15T18:26:52.081 INFO:tasks.workunit.client.0.smithi042.stdout:3/108: dread d1/f7 [0,4194304] 0 2024-02-15T18:26:52.081 INFO:tasks.workunit.client.0.smithi042.stdout:5/84: unlink d0/l9 0 2024-02-15T18:26:52.082 INFO:tasks.workunit.client.0.smithi042.stdout:0/119: rmdir d4/d9 39 2024-02-15T18:26:52.083 INFO:tasks.workunit.client.0.smithi042.stdout:3/109: unlink d1/de/cf 0 2024-02-15T18:26:52.083 INFO:tasks.workunit.client.0.smithi042.stdout:0/120: dread d4/d16/d2e/f2d [0,4194304] 0 2024-02-15T18:26:52.083 INFO:tasks.workunit.client.0.smithi042.stdout:5/85: symlink d0/l1d 0 2024-02-15T18:26:52.084 INFO:tasks.workunit.client.0.smithi042.stdout:3/110: getdents d1/de/d19 0 2024-02-15T18:26:52.086 INFO:tasks.workunit.client.0.smithi042.stdout:0/121: creat d4/d16/f30 x:0 0 0 2024-02-15T18:26:52.088 INFO:tasks.workunit.client.0.smithi042.stdout:1/102: dwrite de/d16/f1d [0,4194304] 0 2024-02-15T18:26:52.088 INFO:tasks.workunit.client.0.smithi042.stdout:1/103: write de/d16/f18 [341557,112887] 0 2024-02-15T18:26:52.088 INFO:tasks.workunit.client.0.smithi042.stdout:5/86: rename d0/db/c18 to d0/db/c1e 0 2024-02-15T18:26:52.090 INFO:tasks.workunit.client.0.smithi042.stdout:3/111: unlink d1/fa 0 2024-02-15T18:26:52.091 INFO:tasks.workunit.client.0.smithi042.stdout:2/65: dread d0/f3 [0,4194304] 0 2024-02-15T18:26:52.091 INFO:tasks.workunit.client.0.smithi042.stdout:9/98: dwrite d6/da/f10 [0,4194304] 0 2024-02-15T18:26:52.093 INFO:tasks.workunit.client.0.smithi042.stdout:1/104: mknod de/d15/d1b/c23 0 2024-02-15T18:26:52.095 INFO:tasks.workunit.client.0.smithi042.stdout:1/105: dread fb [0,4194304] 0 2024-02-15T18:26:52.155 INFO:tasks.workunit.client.0.smithi042.stdout:5/87: creat d0/db/f1f x:0 0 0 2024-02-15T18:26:52.155 INFO:tasks.workunit.client.0.smithi042.stdout:5/88: readlink d0/d12/l1b 0 2024-02-15T18:26:52.161 INFO:tasks.workunit.client.0.smithi042.stdout:7/96: dwrite d10/d12/f16 [0,4194304] 0 2024-02-15T18:26:52.161 INFO:tasks.workunit.client.0.smithi042.stdout:0/122: dread d4/d9/d15/f18 [0,4194304] 0 2024-02-15T18:26:52.161 INFO:tasks.workunit.client.0.smithi042.stdout:3/112: rename d1/de/f13 to d1/de/d1a/f1e 0 2024-02-15T18:26:52.161 INFO:tasks.workunit.client.0.smithi042.stdout:3/113: stat d1/de/d19/l1b 0 2024-02-15T18:26:52.161 INFO:tasks.workunit.client.0.smithi042.stdout:2/66: unlink d0/d8/d10/f12 0 2024-02-15T18:26:52.161 INFO:tasks.workunit.client.0.smithi042.stdout:9/99: truncate d6/dd/d9/f18 1799677 0 2024-02-15T18:26:52.161 INFO:tasks.workunit.client.0.smithi042.stdout:9/100: read d6/dd/f20 [354344,96243] 0 2024-02-15T18:26:52.161 INFO:tasks.workunit.client.0.smithi042.stdout:1/106: mknod de/c24 0 2024-02-15T18:26:52.162 INFO:tasks.workunit.client.0.smithi042.stdout:1/107: fdatasync de/d15/f20 0 2024-02-15T18:26:52.162 INFO:tasks.workunit.client.0.smithi042.stdout:0/123: truncate d4/f20 2396307 0 2024-02-15T18:26:52.162 INFO:tasks.workunit.client.0.smithi042.stdout:2/67: dread d0/f7 [0,4194304] 0 2024-02-15T18:26:52.162 INFO:tasks.workunit.client.0.smithi042.stdout:5/89: rename d0/db/l19 to d0/d14/l20 0 2024-02-15T18:26:52.162 INFO:tasks.workunit.client.0.smithi042.stdout:5/90: chown d0/c8 4 1 2024-02-15T18:26:52.162 INFO:tasks.workunit.client.0.smithi042.stdout:3/114: creat d1/de/d19/f1f x:0 0 0 2024-02-15T18:26:52.162 INFO:tasks.workunit.client.0.smithi042.stdout:7/97: dwrite d10/f1b [0,4194304] 0 2024-02-15T18:26:52.162 INFO:tasks.workunit.client.0.smithi042.stdout:9/101: symlink d6/da/d1d/l28 0 2024-02-15T18:26:52.162 INFO:tasks.workunit.client.0.smithi042.stdout:7/98: read fb [98038,76713] 0 2024-02-15T18:26:52.162 INFO:tasks.workunit.client.0.smithi042.stdout:1/108: creat de/d15/d1b/f25 x:0 0 0 2024-02-15T18:26:52.162 INFO:tasks.workunit.client.0.smithi042.stdout:1/109: dread - de/d15/d1b/f25 zero size 2024-02-15T18:26:52.162 INFO:tasks.workunit.client.0.smithi042.stdout:0/124: creat d4/d9/f31 x:0 0 0 2024-02-15T18:26:52.162 INFO:tasks.workunit.client.0.smithi042.stdout:0/125: stat d4/d9/l17 0 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:2/68: creat d0/f15 x:0 0 0 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:2/69: readlink d0/lc 0 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:2/70: chown d0/d8/db 378665 1 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:2/71: dread - d0/d8/db/de/f14 zero size 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:5/91: rename d0/db/ff to d0/d12/f21 0 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:5/92: fdatasync d0/db/fd 0 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:5/93: write d0/fe [4880200,74102] 0 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:5/94: truncate d0/d14/f16 504859 0 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:8/43: sync 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:8/44: write fc [4654833,77058] 0 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:1/110: dread de/d16/f1d [0,4194304] 0 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:8/45: read fc [2900834,71322] 0 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:8/46: fsync fb 0 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:2/72: mknod d0/d8/c16 0 2024-02-15T18:26:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:3/115: rename d1/d11/f1c to d1/de/f20 0 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:5/95: mkdir d0/d22 0 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:9/102: dwrite d6/dd/fe [0,4194304] 0 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:4/79: sync 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:6/122: sync 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:4/80: chown da/d11/c13 519927 1 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:6/123: chown d9/f17 616 1 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:6/124: write d9/f21 [50763,42144] 0 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:6/125: truncate d9/d11/f2b 103262 0 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:6/126: dread - d9/d11/d19/f2a zero size 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:1/111: creat de/d16/f26 x:0 0 0 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:5/96: creat d0/f23 x:0 0 0 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:4/81: rename c2 to da/d11/c15 0 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:0/126: dwrite d4/fe [8388608,4194304] 0 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:5/97: mknod d0/d22/c24 0 2024-02-15T18:26:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:1/112: mknod de/d15/c27 0 2024-02-15T18:26:52.165 INFO:tasks.workunit.client.0.smithi042.stdout:1/113: stat fc 0 2024-02-15T18:26:52.165 INFO:tasks.workunit.client.0.smithi042.stdout:2/73: dread d0/d8/db/fd [0,4194304] 0 2024-02-15T18:26:52.165 INFO:tasks.workunit.client.0.smithi042.stdout:2/74: write d0/d8/db/de/f14 [188411,91929] 0 2024-02-15T18:26:52.165 INFO:tasks.workunit.client.0.smithi042.stdout:2/75: write d0/f15 [69818,125642] 0 2024-02-15T18:26:52.165 INFO:tasks.workunit.client.0.smithi042.stdout:0/127: rename d4/d9/f14 to d4/d16/d2e/f32 0 2024-02-15T18:26:52.165 INFO:tasks.workunit.client.0.smithi042.stdout:3/116: dwrite d1/de/f20 [0,4194304] 0 2024-02-15T18:26:52.165 INFO:tasks.workunit.client.0.smithi042.stdout:3/117: chown d1/de/d19/l1d 94 1 2024-02-15T18:26:52.165 INFO:tasks.workunit.client.0.smithi042.stdout:0/128: mknod d4/d16/d2e/c33 0 2024-02-15T18:26:52.165 INFO:tasks.workunit.client.0.smithi042.stdout:1/114: mknod de/d16/c28 0 2024-02-15T18:26:52.165 INFO:tasks.workunit.client.0.smithi042.stdout:8/47: dwrite fc [0,4194304] 0 2024-02-15T18:26:52.165 INFO:tasks.workunit.client.0.smithi042.stdout:5/98: rename d0/l1 to d0/d12/l25 0 2024-02-15T18:26:52.165 INFO:tasks.workunit.client.0.smithi042.stdout:2/76: dread d0/f15 [0,4194304] 0 2024-02-15T18:26:52.165 INFO:tasks.workunit.client.0.smithi042.stdout:2/77: dread d0/d8/f11 [0,4194304] 0 2024-02-15T18:26:52.165 INFO:tasks.workunit.client.0.smithi042.stdout:3/118: link d1/d11/c12 d1/d11/d14/c21 0 2024-02-15T18:26:52.166 INFO:tasks.workunit.client.0.smithi042.stdout:0/129: mkdir d4/d9/d34 0 2024-02-15T18:26:52.166 INFO:tasks.workunit.client.0.smithi042.stdout:8/48: mkdir dd/de 0 2024-02-15T18:26:52.166 INFO:tasks.workunit.client.0.smithi042.stdout:6/127: dwrite d9/f21 [0,4194304] 0 2024-02-15T18:26:52.167 INFO:tasks.workunit.client.0.smithi042.stdout:8/49: read fb [285753,64609] 0 2024-02-15T18:26:52.167 INFO:tasks.workunit.client.0.smithi042.stdout:4/82: dwrite da/d11/fe [0,4194304] 0 2024-02-15T18:26:52.167 INFO:tasks.workunit.client.0.smithi042.stdout:1/115: rename fc to de/f29 0 2024-02-15T18:26:52.167 INFO:tasks.workunit.client.0.smithi042.stdout:2/78: dread d0/d8/db/fd [0,4194304] 0 2024-02-15T18:26:52.167 INFO:tasks.workunit.client.0.smithi042.stdout:5/99: link d0/l17 d0/d22/l26 0 2024-02-15T18:26:52.167 INFO:tasks.workunit.client.0.smithi042.stdout:5/100: chown d0/c5 148 1 2024-02-15T18:26:52.167 INFO:tasks.workunit.client.0.smithi042.stdout:3/119: symlink d1/de/l22 0 2024-02-15T18:26:52.168 INFO:tasks.workunit.client.0.smithi042.stdout:6/128: symlink d9/d11/l2e 0 2024-02-15T18:26:52.169 INFO:tasks.workunit.client.0.smithi042.stdout:4/83: symlink da/d11/l16 0 2024-02-15T18:26:52.170 INFO:tasks.workunit.client.0.smithi042.stdout:0/130: rename d4/d16/c19 to d4/c35 0 2024-02-15T18:26:52.171 INFO:tasks.workunit.client.0.smithi042.stdout:0/131: write d4/d9/d15/f25 [3674435,114998] 0 2024-02-15T18:26:52.171 INFO:tasks.workunit.client.0.smithi042.stdout:0/132: write d4/d9/d15/f18 [4629570,42065] 0 2024-02-15T18:26:52.171 INFO:tasks.workunit.client.0.smithi042.stdout:2/79: creat d0/d8/d10/f17 x:0 0 0 2024-02-15T18:26:52.172 INFO:tasks.workunit.client.0.smithi042.stdout:5/101: truncate d0/fe 4658947 0 2024-02-15T18:26:52.173 INFO:tasks.workunit.client.0.smithi042.stdout:1/116: dread f6 [0,4194304] 0 2024-02-15T18:26:52.173 INFO:tasks.workunit.client.0.smithi042.stdout:4/84: stat c3 0 2024-02-15T18:26:52.176 INFO:tasks.workunit.client.0.smithi042.stdout:2/80: creat d0/d8/db/de/f18 x:0 0 0 2024-02-15T18:26:52.177 INFO:tasks.workunit.client.0.smithi042.stdout:5/102: symlink d0/d14/l27 0 2024-02-15T18:26:52.177 INFO:tasks.workunit.client.0.smithi042.stdout:5/103: write d0/d14/f1c [701455,104815] 0 2024-02-15T18:26:52.177 INFO:tasks.workunit.client.0.smithi042.stdout:5/104: dread d0/d14/f16 [0,4194304] 0 2024-02-15T18:26:52.178 INFO:tasks.workunit.client.0.smithi042.stdout:8/50: dwrite fc [0,4194304] 0 2024-02-15T18:26:52.178 INFO:tasks.workunit.client.0.smithi042.stdout:1/117: creat de/d16/f2a x:0 0 0 2024-02-15T18:26:52.178 INFO:tasks.workunit.client.0.smithi042.stdout:8/51: read fc [2590507,13658] 0 2024-02-15T18:26:52.179 INFO:tasks.workunit.client.0.smithi042.stdout:4/85: rmdir da 39 2024-02-15T18:26:52.181 INFO:tasks.workunit.client.0.smithi042.stdout:4/86: dread da/fd [0,4194304] 0 2024-02-15T18:26:52.182 INFO:tasks.workunit.client.0.smithi042.stdout:4/87: chown c5 115 1 2024-02-15T18:26:52.183 INFO:tasks.workunit.client.0.smithi042.stdout:5/105: write d0/f3 [3082176,113876] 0 2024-02-15T18:26:52.185 INFO:tasks.workunit.client.0.smithi042.stdout:0/133: rmdir d4/d9 39 2024-02-15T18:26:52.186 INFO:tasks.workunit.client.0.smithi042.stdout:0/134: dread - d4/d9/f31 zero size 2024-02-15T18:26:52.186 INFO:tasks.workunit.client.0.smithi042.stdout:2/81: creat d0/f19 x:0 0 0 2024-02-15T18:26:52.186 INFO:tasks.workunit.client.0.smithi042.stdout:2/82: write d0/d8/f11 [1204631,13699] 0 2024-02-15T18:26:52.188 INFO:tasks.workunit.client.0.smithi042.stdout:1/118: symlink de/d16/l2b 0 2024-02-15T18:26:52.189 INFO:tasks.workunit.client.0.smithi042.stdout:4/88: mknod da/c17 0 2024-02-15T18:26:52.189 INFO:tasks.workunit.client.0.smithi042.stdout:5/106: creat d0/d14/f28 x:0 0 0 2024-02-15T18:26:52.190 INFO:tasks.workunit.client.0.smithi042.stdout:4/89: write f9 [3722006,39838] 0 2024-02-15T18:26:52.191 INFO:tasks.workunit.client.0.smithi042.stdout:2/83: mkdir d0/d8/db/de/d1a 0 2024-02-15T18:26:52.192 INFO:tasks.workunit.client.0.smithi042.stdout:2/84: dread - d0/f19 zero size 2024-02-15T18:26:52.192 INFO:tasks.workunit.client.0.smithi042.stdout:1/119: mkdir de/d15/d2c 0 2024-02-15T18:26:52.193 INFO:tasks.workunit.client.0.smithi042.stdout:5/107: link d0/fa d0/d14/f29 0 2024-02-15T18:26:52.195 INFO:tasks.workunit.client.0.smithi042.stdout:4/90: link da/d11/l16 da/l18 0 2024-02-15T18:26:52.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:50.679661+0000 mgr.smithi042.kfoihb (mgr.24435) 33 : cluster [DBG] pgmap v24: 65 pgs: 65 active+clean; 254 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 5.7 MiB/s rd, 8.8 MiB/s wr, 376 op/s 2024-02-15T18:26:52.196 INFO:tasks.workunit.client.0.smithi042.stdout:1/120: mknod de/d15/d2c/c2d 0 2024-02-15T18:26:52.197 INFO:tasks.workunit.client.0.smithi042.stdout:4/91: symlink da/d11/l19 0 2024-02-15T18:26:52.197 INFO:tasks.workunit.client.0.smithi042.stdout:4/92: chown da/d11/fe 1480 1 2024-02-15T18:26:52.197 INFO:tasks.workunit.client.0.smithi042.stdout:2/85: dread d0/d8/db/fd [0,4194304] 0 2024-02-15T18:26:52.199 INFO:tasks.workunit.client.0.smithi042.stdout:4/93: creat da/f1a x:0 0 0 2024-02-15T18:26:52.199 INFO:tasks.workunit.client.0.smithi042.stdout:4/94: fsync da/fd 0 2024-02-15T18:26:52.200 INFO:tasks.workunit.client.0.smithi042.stdout:0/135: write d4/d9/d15/f25 [10078,72543] 0 2024-02-15T18:26:52.200 INFO:tasks.workunit.client.0.smithi042.stdout:0/136: chown d4/d16/d2e/c33 49 1 2024-02-15T18:26:52.200 INFO:tasks.workunit.client.0.smithi042.stdout:0/137: write d4/f6 [5145929,81521] 0 2024-02-15T18:26:52.201 INFO:tasks.workunit.client.0.smithi042.stdout:4/95: mkdir da/d11/d1b 0 2024-02-15T18:26:52.201 INFO:tasks.workunit.client.0.smithi042.stdout:8/52: dwrite fc [4194304,4194304] 0 2024-02-15T18:26:52.202 INFO:tasks.workunit.client.0.smithi042.stdout:3/120: dwrite d1/f9 [0,4194304] 0 2024-02-15T18:26:52.207 INFO:tasks.workunit.client.0.smithi042.stdout:5/108: dread d0/db/fd [0,4194304] 0 2024-02-15T18:26:52.207 INFO:tasks.workunit.client.0.smithi042.stdout:5/109: chown d0/c8 43711 1 2024-02-15T18:26:52.213 INFO:tasks.workunit.client.0.smithi042.stdout:1/121: rename de/d16 to de/d15/d2e 0 2024-02-15T18:26:52.213 INFO:tasks.workunit.client.0.smithi042.stdout:1/122: write fb [1066752,21204] 0 2024-02-15T18:26:52.213 INFO:tasks.workunit.client.0.smithi042.stdout:1/123: stat de/d15/d2e/f18 0 2024-02-15T18:26:52.218 INFO:tasks.workunit.client.0.smithi042.stdout:4/96: link f9 da/d11/d1b/f1c 0 2024-02-15T18:26:52.218 INFO:tasks.workunit.client.0.smithi042.stdout:4/97: stat da 0 2024-02-15T18:26:52.218 INFO:tasks.workunit.client.0.smithi042.stdout:4/98: chown da/d11/l16 5 1 2024-02-15T18:26:52.219 INFO:tasks.workunit.client.0.smithi042.stdout:8/53: getdents dd 0 2024-02-15T18:26:52.219 INFO:tasks.workunit.client.0.smithi042.stdout:8/54: chown fc 261890435 1 2024-02-15T18:26:52.222 INFO:tasks.workunit.client.0.smithi042.stdout:0/138: dwrite d4/fe [4194304,4194304] 0 2024-02-15T18:26:52.222 INFO:tasks.workunit.client.0.smithi042.stdout:4/99: creat da/d11/f1d x:0 0 0 2024-02-15T18:26:52.223 INFO:tasks.workunit.client.0.smithi042.stdout:8/55: creat dd/de/ff x:0 0 0 2024-02-15T18:26:52.224 INFO:tasks.workunit.client.0.smithi042.stdout:0/139: rename d4/d9/d15/f25 to d4/d9/d34/f36 0 2024-02-15T18:26:52.224 INFO:tasks.workunit.client.0.smithi042.stdout:8/56: rmdir dd 39 2024-02-15T18:26:52.225 INFO:tasks.workunit.client.0.smithi042.stdout:8/57: fdatasync dd/de/ff 0 2024-02-15T18:26:52.225 INFO:tasks.workunit.client.0.smithi042.stdout:8/58: creat dd/de/f10 x:0 0 0 2024-02-15T18:26:52.225 INFO:tasks.workunit.client.0.smithi042.stdout:8/59: getdents dd 0 2024-02-15T18:26:52.226 INFO:tasks.workunit.client.0.smithi042.stdout:8/60: unlink c5 0 2024-02-15T18:26:52.226 INFO:tasks.workunit.client.0.smithi042.stdout:8/61: read - dd/de/f10 zero size 2024-02-15T18:26:52.228 INFO:tasks.workunit.client.0.smithi042.stdout:8/62: mkdir dd/d11 0 2024-02-15T18:26:52.228 INFO:tasks.workunit.client.0.smithi042.stdout:8/63: read fc [6344350,38755] 0 2024-02-15T18:26:52.228 INFO:tasks.workunit.client.0.smithi042.stdout:3/121: dwrite d1/f9 [0,4194304] 0 2024-02-15T18:26:52.234 INFO:tasks.workunit.client.0.smithi042.stdout:3/122: rename d1/de/d19/l1d to d1/de/l23 0 2024-02-15T18:26:52.234 INFO:tasks.workunit.client.0.smithi042.stdout:3/123: chown d1/de/l22 70824 1 2024-02-15T18:26:52.248 INFO:tasks.workunit.client.0.smithi042.stdout:0/140: dwrite d4/d9/f22 [0,4194304] 0 2024-02-15T18:26:52.248 INFO:tasks.workunit.client.0.smithi042.stdout:0/141: readlink d4/d9/l1f 0 2024-02-15T18:26:52.250 INFO:tasks.workunit.client.0.smithi042.stdout:0/142: mknod d4/d9/d15/c37 0 2024-02-15T18:26:52.264 INFO:tasks.workunit.client.0.smithi042.stdout:8/64: dwrite dd/de/ff [0,4194304] 0 2024-02-15T18:26:52.264 INFO:tasks.workunit.client.0.smithi042.stdout:3/124: dwrite d1/de/f20 [0,4194304] 0 2024-02-15T18:26:52.264 INFO:tasks.workunit.client.0.smithi042.stdout:3/125: rename d1 to d1/de/d1a/d24 22 2024-02-15T18:26:52.265 INFO:tasks.workunit.client.0.smithi042.stdout:0/143: dwrite d4/d9/ff [0,4194304] 0 2024-02-15T18:26:52.268 INFO:tasks.workunit.client.0.smithi042.stdout:0/144: dread d4/d9/d34/f36 [0,4194304] 0 2024-02-15T18:26:52.269 INFO:tasks.workunit.client.0.smithi042.stdout:0/145: dread d4/d16/d2e/f32 [0,4194304] 0 2024-02-15T18:26:52.270 INFO:tasks.workunit.client.0.smithi042.stdout:0/146: write d4/d9/ff [1893741,106021] 0 2024-02-15T18:26:52.271 INFO:tasks.workunit.client.0.smithi042.stdout:7/99: sync 2024-02-15T18:26:52.271 INFO:tasks.workunit.client.0.smithi042.stdout:7/100: chown d10/f1b 5 1 2024-02-15T18:26:52.272 INFO:tasks.workunit.client.0.smithi042.stdout:3/126: creat d1/de/d1a/f25 x:0 0 0 2024-02-15T18:26:52.273 INFO:tasks.workunit.client.0.smithi042.stdout:7/101: truncate fd 8035367 0 2024-02-15T18:26:52.274 INFO:tasks.workunit.client.0.smithi042.stdout:7/102: creat d10/f26 x:0 0 0 2024-02-15T18:26:52.276 INFO:tasks.workunit.client.0.smithi042.stdout:7/103: creat d10/d12/d23/f27 x:0 0 0 2024-02-15T18:26:52.276 INFO:tasks.workunit.client.0.smithi042.stdout:7/104: stat d10/d12/d23/f27 0 2024-02-15T18:26:52.278 INFO:tasks.workunit.client.0.smithi042.stdout:7/105: symlink d10/d12/l28 0 2024-02-15T18:26:52.279 INFO:tasks.workunit.client.0.smithi042.stdout:0/147: dread d4/fe [8388608,4194304] 0 2024-02-15T18:26:52.282 INFO:tasks.workunit.client.0.smithi042.stdout:0/148: unlink d4/d9/c26 0 2024-02-15T18:26:52.283 INFO:tasks.workunit.client.0.smithi042.stdout:0/149: mkdir d4/d38 0 2024-02-15T18:26:52.284 INFO:tasks.workunit.client.0.smithi042.stdout:7/106: rmdir d10/d12 39 2024-02-15T18:26:52.285 INFO:tasks.workunit.client.0.smithi042.stdout:0/150: mkdir d4/d38/d39 0 2024-02-15T18:26:52.286 INFO:tasks.workunit.client.0.smithi042.stdout:0/151: unlink d4/d16/f30 0 2024-02-15T18:26:52.287 INFO:tasks.workunit.client.0.smithi042.stdout:0/152: readlink d4/d9/l1f 0 2024-02-15T18:26:52.287 INFO:tasks.workunit.client.0.smithi042.stdout:0/153: creat d4/d38/f3a x:0 0 0 2024-02-15T18:26:52.300 INFO:tasks.workunit.client.0.smithi042.stdout:7/107: dwrite d10/f21 [0,4194304] 0 2024-02-15T18:26:52.305 INFO:tasks.workunit.client.0.smithi042.stdout:6/129: sync 2024-02-15T18:26:52.310 INFO:tasks.workunit.client.0.smithi042.stdout:6/130: rename d9/d11/d1d/f22 to d9/f2f 0 2024-02-15T18:26:52.311 INFO:tasks.workunit.client.0.smithi042.stdout:6/131: symlink d9/d11/d19/d24/l30 0 2024-02-15T18:26:52.312 INFO:tasks.workunit.client.0.smithi042.stdout:6/132: mkdir d9/d11/d19/d1c/d31 0 2024-02-15T18:26:52.313 INFO:tasks.workunit.client.0.smithi042.stdout:6/133: write d9/f2f [259304,109482] 0 2024-02-15T18:26:52.313 INFO:tasks.workunit.client.0.smithi042.stdout:6/134: truncate d9/f14 564499 0 2024-02-15T18:26:52.314 INFO:tasks.workunit.client.0.smithi042.stdout:6/135: creat d9/f32 x:0 0 0 2024-02-15T18:26:52.315 INFO:tasks.workunit.client.0.smithi042.stdout:6/136: rmdir d9/d11/d19/d1c 39 2024-02-15T18:26:52.315 INFO:tasks.workunit.client.0.smithi042.stdout:6/137: rmdir d9/d11/d19/d24 39 2024-02-15T18:26:52.320 INFO:tasks.workunit.client.0.smithi042.stdout:7/108: dwrite d10/d15/f1d [0,4194304] 0 2024-02-15T18:26:52.320 INFO:tasks.workunit.client.0.smithi042.stdout:9/103: sync 2024-02-15T18:26:52.320 INFO:tasks.workunit.client.0.smithi042.stdout:9/104: dread - d6/dd/f19 zero size 2024-02-15T18:26:52.320 INFO:tasks.workunit.client.0.smithi042.stdout:9/105: chown d6/f17 495 1 2024-02-15T18:26:52.321 INFO:tasks.workunit.client.0.smithi042.stdout:9/106: fsync d6/f21 0 2024-02-15T18:26:52.324 INFO:tasks.workunit.client.0.smithi042.stdout:6/138: fsync d9/f26 0 2024-02-15T18:26:52.324 INFO:tasks.workunit.client.0.smithi042.stdout:6/139: read d9/d11/f2b [46197,43056] 0 2024-02-15T18:26:52.324 INFO:tasks.workunit.client.0.smithi042.stdout:6/140: write d9/f21 [4754797,3706] 0 2024-02-15T18:26:52.326 INFO:tasks.workunit.client.0.smithi042.stdout:7/109: link d10/f22 d10/d12/d23/f29 0 2024-02-15T18:26:52.329 INFO:tasks.workunit.client.0.smithi042.stdout:6/141: rename d9/f26 to d9/d11/d1d/f33 0 2024-02-15T18:26:52.329 INFO:tasks.workunit.client.0.smithi042.stdout:6/142: fsync d9/d11/d19/f2a 0 2024-02-15T18:26:52.329 INFO:tasks.workunit.client.0.smithi042.stdout:6/143: dread - d9/d11/d19/d1c/f27 zero size 2024-02-15T18:26:52.330 INFO:tasks.workunit.client.0.smithi042.stdout:6/144: creat d9/d11/d19/d1c/d31/f34 x:0 0 0 2024-02-15T18:26:52.331 INFO:tasks.workunit.client.0.smithi042.stdout:7/110: getdents d10/d12/d23 0 2024-02-15T18:26:52.334 INFO:tasks.workunit.client.0.smithi042.stdout:7/111: link d10/d12/l28 d10/d15/l2a 0 2024-02-15T18:26:52.336 INFO:tasks.workunit.client.0.smithi042.stdout:6/145: rmdir d9/d11/d19/d1c/d31 39 2024-02-15T18:26:52.336 INFO:tasks.workunit.client.0.smithi042.stdout:6/146: readlink d9/d11/d19/d24/l30 0 2024-02-15T18:26:52.336 INFO:tasks.workunit.client.0.smithi042.stdout:6/147: chown d9/d11/d1d 114819 1 2024-02-15T18:26:52.337 INFO:tasks.workunit.client.0.smithi042.stdout:6/148: fdatasync d9/f32 0 2024-02-15T18:26:52.337 INFO:tasks.workunit.client.0.smithi042.stdout:6/149: dread - d9/d11/d19/d1c/d31/f34 zero size 2024-02-15T18:26:52.337 INFO:tasks.workunit.client.0.smithi042.stdout:6/150: chown d9/f14 0 1 2024-02-15T18:26:52.337 INFO:tasks.workunit.client.0.smithi042.stdout:7/112: symlink d10/l2b 0 2024-02-15T18:26:52.337 INFO:tasks.workunit.client.0.smithi042.stdout:7/113: chown d10/f14 37 1 2024-02-15T18:26:52.339 INFO:tasks.workunit.client.0.smithi042.stdout:7/114: rename d10/d12/l28 to d10/d12/d23/l2c 0 2024-02-15T18:26:52.340 INFO:tasks.workunit.client.0.smithi042.stdout:7/115: mknod d10/d15/c2d 0 2024-02-15T18:26:52.341 INFO:tasks.workunit.client.0.smithi042.stdout:7/116: rmdir d10/d12 39 2024-02-15T18:26:52.342 INFO:tasks.workunit.client.0.smithi042.stdout:7/117: unlink d10/d15/l2a 0 2024-02-15T18:26:52.344 INFO:tasks.workunit.client.0.smithi042.stdout:7/118: symlink d10/d12/l2e 0 2024-02-15T18:26:52.344 INFO:tasks.workunit.client.0.smithi042.stdout:7/119: fdatasync d10/f1b 0 2024-02-15T18:26:52.345 INFO:tasks.workunit.client.0.smithi042.stdout:7/120: fsync d10/f26 0 2024-02-15T18:26:52.345 INFO:tasks.workunit.client.0.smithi042.stdout:7/121: readlink d10/l2b 0 2024-02-15T18:26:52.345 INFO:tasks.workunit.client.0.smithi042.stdout:7/122: link d10/f1b d10/f2f 0 2024-02-15T18:26:52.351 INFO:tasks.workunit.client.0.smithi042.stdout:7/123: write fa [3566913,92442] 0 2024-02-15T18:26:52.352 INFO:tasks.workunit.client.0.smithi042.stdout:6/151: dwrite d9/f2f [0,4194304] 0 2024-02-15T18:26:52.354 INFO:tasks.workunit.client.0.smithi042.stdout:1/124: sync 2024-02-15T18:26:52.354 INFO:tasks.workunit.client.0.smithi042.stdout:8/65: sync 2024-02-15T18:26:52.354 INFO:tasks.workunit.client.0.smithi042.stdout:4/100: sync 2024-02-15T18:26:52.354 INFO:tasks.workunit.client.0.smithi042.stdout:5/110: sync 2024-02-15T18:26:52.354 INFO:tasks.workunit.client.0.smithi042.stdout:2/86: sync 2024-02-15T18:26:52.356 INFO:tasks.workunit.client.0.smithi042.stdout:4/101: write f9 [3119217,98529] 0 2024-02-15T18:26:52.359 INFO:tasks.workunit.client.0.smithi042.stdout:8/66: mkdir dd/de/d12 0 2024-02-15T18:26:52.359 INFO:tasks.workunit.client.0.smithi042.stdout:8/67: fsync fc 0 2024-02-15T18:26:52.359 INFO:tasks.workunit.client.0.smithi042.stdout:5/111: mkdir d0/d12/d2a 0 2024-02-15T18:26:52.360 INFO:tasks.workunit.client.0.smithi042.stdout:1/125: symlink de/d15/d1b/l2f 0 2024-02-15T18:26:52.360 INFO:tasks.workunit.client.0.smithi042.stdout:1/126: stat de/c11 0 2024-02-15T18:26:52.360 INFO:tasks.workunit.client.0.smithi042.stdout:1/127: chown de/d15/d2e/f18 1417918910 1 2024-02-15T18:26:52.361 INFO:tasks.workunit.client.0.smithi042.stdout:2/87: truncate d0/f5 222356 0 2024-02-15T18:26:52.361 INFO:tasks.workunit.client.0.smithi042.stdout:2/88: stat d0/d8/db/de 0 2024-02-15T18:26:52.361 INFO:tasks.workunit.client.0.smithi042.stdout:2/89: fdatasync d0/f15 0 2024-02-15T18:26:52.361 INFO:tasks.workunit.client.0.smithi042.stdout:2/90: chown d0/c9 597214 1 2024-02-15T18:26:52.361 INFO:tasks.workunit.client.0.smithi042.stdout:8/68: write fc [4485122,75917] 0 2024-02-15T18:26:52.361 INFO:tasks.workunit.client.0.smithi042.stdout:8/69: chown fc 19917365 1 2024-02-15T18:26:52.365 INFO:tasks.workunit.client.0.smithi042.stdout:4/102: creat da/d11/d1b/f1e x:0 0 0 2024-02-15T18:26:52.365 INFO:tasks.workunit.client.0.smithi042.stdout:5/112: rename d0/d14/l20 to d0/db/l2b 0 2024-02-15T18:26:52.365 INFO:tasks.workunit.client.0.smithi042.stdout:4/103: truncate da/d11/f1d 944047 0 2024-02-15T18:26:52.366 INFO:tasks.workunit.client.0.smithi042.stdout:4/104: dread da/d11/f1d [0,4194304] 0 2024-02-15T18:26:52.367 INFO:tasks.workunit.client.0.smithi042.stdout:1/128: rename de/c24 to de/d15/c30 0 2024-02-15T18:26:52.367 INFO:tasks.workunit.client.0.smithi042.stdout:5/113: mknod d0/c2c 0 2024-02-15T18:26:52.369 INFO:tasks.workunit.client.0.smithi042.stdout:4/105: link da/c17 da/d11/c1f 0 2024-02-15T18:26:52.369 INFO:tasks.workunit.client.0.smithi042.stdout:4/106: dread - da/d11/d1b/f1e zero size 2024-02-15T18:26:52.369 INFO:tasks.workunit.client.0.smithi042.stdout:1/129: creat de/d15/f31 x:0 0 0 2024-02-15T18:26:52.370 INFO:tasks.workunit.client.0.smithi042.stdout:5/114: mkdir d0/d22/d2d 0 2024-02-15T18:26:52.371 INFO:tasks.workunit.client.0.smithi042.stdout:1/130: dread fb [0,4194304] 0 2024-02-15T18:26:52.372 INFO:tasks.workunit.client.0.smithi042.stdout:1/131: creat de/d15/d2e/f32 x:0 0 0 2024-02-15T18:26:52.372 INFO:tasks.workunit.client.0.smithi042.stdout:1/132: write de/d15/f20 [574812,95581] 0 2024-02-15T18:26:52.374 INFO:tasks.workunit.client.0.smithi042.stdout:1/133: dread de/f29 [0,4194304] 0 2024-02-15T18:26:52.374 INFO:tasks.workunit.client.0.smithi042.stdout:1/134: fdatasync de/d15/d2e/f32 0 2024-02-15T18:26:52.375 INFO:tasks.workunit.client.0.smithi042.stdout:1/135: dread de/d15/d2e/f18 [0,4194304] 0 2024-02-15T18:26:52.379 INFO:tasks.workunit.client.0.smithi042.stdout:6/152: dwrite d9/d11/f20 [0,4194304] 0 2024-02-15T18:26:52.379 INFO:tasks.workunit.client.0.smithi042.stdout:7/124: dwrite d10/f14 [0,4194304] 0 2024-02-15T18:26:52.382 INFO:tasks.workunit.client.0.smithi042.stdout:6/153: rmdir d9/d11/d19/d1c/d31 39 2024-02-15T18:26:52.387 INFO:tasks.workunit.client.0.smithi042.stdout:7/125: link d10/c1e d10/d12/c30 0 2024-02-15T18:26:52.387 INFO:tasks.workunit.client.0.smithi042.stdout:6/154: truncate d9/fa 2024901 0 2024-02-15T18:26:52.390 INFO:tasks.workunit.client.0.smithi042.stdout:7/126: symlink d10/d15/l31 0 2024-02-15T18:26:52.396 INFO:tasks.workunit.client.0.smithi042.stdout:4/107: dread f9 [0,4194304] 0 2024-02-15T18:26:52.397 INFO:tasks.workunit.client.0.smithi042.stdout:5/115: dwrite d0/db/fd [0,4194304] 0 2024-02-15T18:26:52.397 INFO:tasks.workunit.client.0.smithi042.stdout:4/108: dread da/fd [0,4194304] 0 2024-02-15T18:26:52.397 INFO:tasks.workunit.client.0.smithi042.stdout:4/109: dread - da/d11/d1b/f1e zero size 2024-02-15T18:26:52.398 INFO:tasks.workunit.client.0.smithi042.stdout:5/116: creat d0/d14/f2e x:0 0 0 2024-02-15T18:26:52.399 INFO:tasks.workunit.client.0.smithi042.stdout:4/110: creat da/d11/f20 x:0 0 0 2024-02-15T18:26:52.400 INFO:tasks.workunit.client.0.smithi042.stdout:4/111: write da/d11/d1b/f1c [1576949,54462] 0 2024-02-15T18:26:52.400 INFO:tasks.workunit.client.0.smithi042.stdout:5/117: truncate d0/d14/f29 572202 0 2024-02-15T18:26:52.402 INFO:tasks.workunit.client.0.smithi042.stdout:5/118: symlink d0/d22/l2f 0 2024-02-15T18:26:52.403 INFO:tasks.workunit.client.0.smithi042.stdout:5/119: rmdir d0/db 39 2024-02-15T18:26:52.403 INFO:tasks.workunit.client.0.smithi042.stdout:5/120: chown d0/db/f15 29 1 2024-02-15T18:26:52.403 INFO:tasks.workunit.client.0.smithi042.stdout:5/121: chown d0/db 335 1 2024-02-15T18:26:52.404 INFO:tasks.workunit.client.0.smithi042.stdout:5/122: symlink d0/d22/l30 0 2024-02-15T18:26:52.408 INFO:tasks.workunit.client.0.smithi042.stdout:1/136: dwrite de/d15/f20 [0,4194304] 0 2024-02-15T18:26:52.408 INFO:tasks.workunit.client.0.smithi042.stdout:1/137: truncate de/d15/f20 4960945 0 2024-02-15T18:26:52.408 INFO:tasks.workunit.client.0.smithi042.stdout:1/138: chown de/c1a 0 1 2024-02-15T18:26:52.408 INFO:tasks.workunit.client.0.smithi042.stdout:1/139: chown fb 2 1 2024-02-15T18:26:52.408 INFO:tasks.workunit.client.0.smithi042.stdout:1/140: chown de/c12 33331728 1 2024-02-15T18:26:52.408 INFO:tasks.workunit.client.0.smithi042.stdout:0/154: sync 2024-02-15T18:26:52.408 INFO:tasks.workunit.client.0.smithi042.stdout:3/127: sync 2024-02-15T18:26:52.410 INFO:tasks.workunit.client.0.smithi042.stdout:7/127: dwrite d10/f22 [0,4194304] 0 2024-02-15T18:26:52.411 INFO:tasks.workunit.client.0.smithi042.stdout:7/128: chown d10/c1e 717 1 2024-02-15T18:26:52.411 INFO:tasks.workunit.client.0.smithi042.stdout:2/91: dwrite d0/f5 [0,4194304] 0 2024-02-15T18:26:52.411 INFO:tasks.workunit.client.0.smithi042.stdout:2/92: chown d0/f15 1 1 2024-02-15T18:26:52.411 INFO:tasks.workunit.client.0.smithi042.stdout:0/155: getdents d4/d9/d15 0 2024-02-15T18:26:52.412 INFO:tasks.workunit.client.0.smithi042.stdout:0/156: chown d4/f6 28097 1 2024-02-15T18:26:52.412 INFO:tasks.workunit.client.0.smithi042.stdout:0/157: write d4/d9/d15/f21 [166402,40499] 0 2024-02-15T18:26:52.413 INFO:tasks.workunit.client.0.smithi042.stdout:3/128: mknod d1/de/d1a/c26 0 2024-02-15T18:26:52.413 INFO:tasks.workunit.client.0.smithi042.stdout:3/129: chown d1/f9 0 1 2024-02-15T18:26:52.413 INFO:tasks.workunit.client.0.smithi042.stdout:3/130: chown d1/f9 21 1 2024-02-15T18:26:52.413 INFO:tasks.workunit.client.0.smithi042.stdout:3/131: chown d1 260 1 2024-02-15T18:26:52.414 INFO:tasks.workunit.client.0.smithi042.stdout:6/155: rmdir d9 39 2024-02-15T18:26:52.421 INFO:tasks.workunit.client.0.smithi042.stdout:7/129: truncate d10/f21 3727504 0 2024-02-15T18:26:52.424 INFO:tasks.workunit.client.0.smithi042.stdout:2/93: rmdir d0/d8/db/de 39 2024-02-15T18:26:52.424 INFO:tasks.workunit.client.0.smithi042.stdout:2/94: dread - d0/d8/d10/f17 zero size 2024-02-15T18:26:52.424 INFO:tasks.workunit.client.0.smithi042.stdout:2/95: dread - d0/d8/d10/f17 zero size 2024-02-15T18:26:52.425 INFO:tasks.workunit.client.0.smithi042.stdout:0/158: creat d4/f3b x:0 0 0 2024-02-15T18:26:52.425 INFO:tasks.workunit.client.0.smithi042.stdout:0/159: stat d4/d16/f24 0 2024-02-15T18:26:52.425 INFO:tasks.workunit.client.0.smithi042.stdout:0/160: chown d4/f7 3 1 2024-02-15T18:26:52.425 INFO:tasks.workunit.client.0.smithi042.stdout:3/132: creat d1/de/f27 x:0 0 0 2024-02-15T18:26:52.428 INFO:tasks.workunit.client.0.smithi042.stdout:6/156: getdents d9/d11/d1d 0 2024-02-15T18:26:52.428 INFO:tasks.workunit.client.0.smithi042.stdout:7/130: mknod d10/c32 0 2024-02-15T18:26:52.428 INFO:tasks.workunit.client.0.smithi042.stdout:2/96: getdents d0/d8/d10 0 2024-02-15T18:26:52.429 INFO:tasks.workunit.client.0.smithi042.stdout:0/161: creat d4/d38/f3c x:0 0 0 2024-02-15T18:26:52.430 INFO:tasks.workunit.client.0.smithi042.stdout:7/131: rename d10/d12/c24 to d10/d15/c33 0 2024-02-15T18:26:52.431 INFO:tasks.workunit.client.0.smithi042.stdout:2/97: mkdir d0/d8/d1b 0 2024-02-15T18:26:52.431 INFO:tasks.workunit.client.0.smithi042.stdout:2/98: truncate d0/d8/db/de/f18 884911 0 2024-02-15T18:26:52.432 INFO:tasks.workunit.client.0.smithi042.stdout:0/162: link d4/fe d4/d38/d39/f3d 0 2024-02-15T18:26:52.432 INFO:tasks.workunit.client.0.smithi042.stdout:7/132: creat d10/d15/f34 x:0 0 0 2024-02-15T18:26:52.433 INFO:tasks.workunit.client.0.smithi042.stdout:2/99: link d0/d8/f11 d0/d8/d10/f1c 0 2024-02-15T18:26:52.434 INFO:tasks.workunit.client.0.smithi042.stdout:7/133: creat d10/d12/d23/f35 x:0 0 0 2024-02-15T18:26:52.434 INFO:tasks.workunit.client.0.smithi042.stdout:7/134: chown fa 484121 1 2024-02-15T18:26:52.435 INFO:tasks.workunit.client.0.smithi042.stdout:7/135: creat d10/d15/f36 x:0 0 0 2024-02-15T18:26:52.435 INFO:tasks.workunit.client.0.smithi042.stdout:7/136: dread - d10/d12/d23/f27 zero size 2024-02-15T18:26:52.436 INFO:tasks.workunit.client.0.smithi042.stdout:7/137: truncate d10/f2f 1722000 0 2024-02-15T18:26:52.436 INFO:tasks.workunit.client.0.smithi042.stdout:2/100: dread d0/d8/d10/f1c [0,4194304] 0 2024-02-15T18:26:52.437 INFO:tasks.workunit.client.0.smithi042.stdout:7/138: creat d10/d12/d23/f37 x:0 0 0 2024-02-15T18:26:52.437 INFO:tasks.workunit.client.0.smithi042.stdout:7/139: readlink d10/d12/l2e 0 2024-02-15T18:26:52.437 INFO:tasks.workunit.client.0.smithi042.stdout:2/101: dread d0/d8/d10/f1c [0,4194304] 0 2024-02-15T18:26:52.438 INFO:tasks.workunit.client.0.smithi042.stdout:7/140: symlink d10/l38 0 2024-02-15T18:26:52.438 INFO:tasks.workunit.client.0.smithi042.stdout:7/141: dread - d10/d12/f20 zero size 2024-02-15T18:26:52.439 INFO:tasks.workunit.client.0.smithi042.stdout:7/142: mkdir d10/d15/d39 0 2024-02-15T18:26:52.443 INFO:tasks.workunit.client.0.smithi042.stdout:0/163: dread d4/f20 [0,4194304] 0 2024-02-15T18:26:52.444 INFO:tasks.workunit.client.0.smithi042.stdout:6/157: dread d9/d11/f15 [0,4194304] 0 2024-02-15T18:26:52.445 INFO:tasks.workunit.client.0.smithi042.stdout:6/158: mkdir d9/d11/d19/d35 0 2024-02-15T18:26:52.446 INFO:tasks.workunit.client.0.smithi042.stdout:6/159: unlink d9/d11/c16 0 2024-02-15T18:26:52.446 INFO:tasks.workunit.client.0.smithi042.stdout:6/160: write d9/d11/d19/d1c/f27 [256614,71539] 0 2024-02-15T18:26:52.446 INFO:tasks.workunit.client.0.smithi042.stdout:6/161: write d9/d11/d19/f2a [293642,103229] 0 2024-02-15T18:26:52.447 INFO:tasks.workunit.client.0.smithi042.stdout:5/123: dwrite d0/d14/f16 [0,4194304] 0 2024-02-15T18:26:52.447 INFO:tasks.workunit.client.0.smithi042.stdout:4/112: dwrite f9 [0,4194304] 0 2024-02-15T18:26:52.448 INFO:tasks.workunit.client.0.smithi042.stdout:6/162: mkdir d9/d11/d19/d36 0 2024-02-15T18:26:52.448 INFO:tasks.workunit.client.0.smithi042.stdout:6/163: chown d9/d11/d19 4338 1 2024-02-15T18:26:52.448 INFO:tasks.workunit.client.0.smithi042.stdout:1/141: dwrite de/d15/d2e/f26 [0,4194304] 0 2024-02-15T18:26:52.449 INFO:tasks.workunit.client.0.smithi042.stdout:1/142: readlink de/d15/l17 0 2024-02-15T18:26:52.449 INFO:tasks.workunit.client.0.smithi042.stdout:3/133: dwrite d1/de/f10 [0,4194304] 0 2024-02-15T18:26:52.449 INFO:tasks.workunit.client.0.smithi042.stdout:6/164: read d9/f17 [38496,26136] 0 2024-02-15T18:26:52.449 INFO:tasks.workunit.client.0.smithi042.stdout:3/134: write d1/de/d1a/f1e [3212557,79357] 0 2024-02-15T18:26:52.449 INFO:tasks.workunit.client.0.smithi042.stdout:3/135: readlink d1/de/l22 0 2024-02-15T18:26:52.449 INFO:tasks.workunit.client.0.smithi042.stdout:1/143: dread de/f29 [0,4194304] 0 2024-02-15T18:26:52.452 INFO:tasks.workunit.client.0.smithi042.stdout:5/124: mknod d0/c31 0 2024-02-15T18:26:52.452 INFO:tasks.workunit.client.0.smithi042.stdout:5/125: write d0/fe [4655297,129456] 0 2024-02-15T18:26:52.452 INFO:tasks.workunit.client.0.smithi042.stdout:5/126: truncate d0/db/f15 1267991 0 2024-02-15T18:26:52.453 INFO:tasks.workunit.client.0.smithi042.stdout:4/113: creat da/f21 x:0 0 0 2024-02-15T18:26:52.453 INFO:tasks.workunit.client.0.smithi042.stdout:4/114: dread da/fd [0,4194304] 0 2024-02-15T18:26:52.454 INFO:tasks.workunit.client.0.smithi042.stdout:9/107: sync 2024-02-15T18:26:52.454 INFO:tasks.workunit.client.0.smithi042.stdout:9/108: read - d6/dd/d9/f16 zero size 2024-02-15T18:26:52.454 INFO:tasks.workunit.client.0.smithi042.stdout:9/109: fsync d6/f17 0 2024-02-15T18:26:52.457 INFO:tasks.workunit.client.0.smithi042.stdout:9/110: dread d6/dd/d9/f14 [0,4194304] 0 2024-02-15T18:26:52.457 INFO:tasks.workunit.client.0.smithi042.stdout:9/111: stat d6/da/f13 0 2024-02-15T18:26:52.458 INFO:tasks.workunit.client.0.smithi042.stdout:6/165: creat d9/d11/d19/f37 x:0 0 0 2024-02-15T18:26:52.458 INFO:tasks.workunit.client.0.smithi042.stdout:6/166: write d9/f14 [855613,56317] 0 2024-02-15T18:26:52.461 INFO:tasks.workunit.client.0.smithi042.stdout:3/136: rename d1/de/d1a/c26 to d1/de/d1a/c28 0 2024-02-15T18:26:52.461 INFO:tasks.workunit.client.0.smithi042.stdout:1/144: truncate f6 542510 0 2024-02-15T18:26:52.461 INFO:tasks.workunit.client.0.smithi042.stdout:3/137: dread - d1/de/f27 zero size 2024-02-15T18:26:52.462 INFO:tasks.workunit.client.0.smithi042.stdout:5/127: creat d0/d22/d2d/f32 x:0 0 0 2024-02-15T18:26:52.462 INFO:tasks.workunit.client.0.smithi042.stdout:5/128: chown d0/c2c 559338 1 2024-02-15T18:26:52.463 INFO:tasks.workunit.client.0.smithi042.stdout:4/115: creat da/d11/f22 x:0 0 0 2024-02-15T18:26:52.466 INFO:tasks.workunit.client.0.smithi042.stdout:3/138: mknod d1/d11/d14/c29 0 2024-02-15T18:26:52.468 INFO:tasks.workunit.client.0.smithi042.stdout:4/116: creat da/d11/f23 x:0 0 0 2024-02-15T18:26:52.470 INFO:tasks.workunit.client.0.smithi042.stdout:3/139: rename d1/cb to d1/de/d19/c2a 0 2024-02-15T18:26:52.473 INFO:tasks.workunit.client.0.smithi042.stdout:1/145: dread de/f21 [4194304,4194304] 0 2024-02-15T18:26:52.475 INFO:tasks.workunit.client.0.smithi042.stdout:5/129: dread d0/fa [0,4194304] 0 2024-02-15T18:26:52.481 INFO:tasks.workunit.client.0.smithi042.stdout:9/112: dwrite d6/da/f23 [0,4194304] 0 2024-02-15T18:26:52.488 INFO:tasks.workunit.client.0.smithi042.stdout:4/117: unlink da/d11/c1f 0 2024-02-15T18:26:52.488 INFO:tasks.workunit.client.0.smithi042.stdout:6/167: dwrite d9/d11/f2b [0,4194304] 0 2024-02-15T18:26:52.490 INFO:tasks.workunit.client.0.smithi042.stdout:7/143: dread d10/f1b [0,4194304] 0 2024-02-15T18:26:52.490 INFO:tasks.workunit.client.0.smithi042.stdout:7/144: stat d10/f14 0 2024-02-15T18:26:52.490 INFO:tasks.workunit.client.0.smithi042.stdout:7/145: write d10/d12/d23/f37 [160593,62981] 0 2024-02-15T18:26:52.491 INFO:tasks.workunit.client.0.smithi042.stdout:7/146: readlink d10/l38 0 2024-02-15T18:26:52.491 INFO:tasks.workunit.client.0.smithi042.stdout:3/140: unlink d1/de/f16 0 2024-02-15T18:26:52.491 INFO:tasks.workunit.client.0.smithi042.stdout:3/141: stat d1/f9 0 2024-02-15T18:26:52.492 INFO:tasks.workunit.client.0.smithi042.stdout:3/142: write d1/de/f20 [2010050,45465] 0 2024-02-15T18:26:52.492 INFO:tasks.workunit.client.0.smithi042.stdout:3/143: write d1/f15 [905251,114886] 0 2024-02-15T18:26:52.492 INFO:tasks.workunit.client.0.smithi042.stdout:5/130: mknod d0/c33 0 2024-02-15T18:26:52.498 INFO:tasks.workunit.client.0.smithi042.stdout:9/113: creat d6/da/f29 x:0 0 0 2024-02-15T18:26:52.498 INFO:tasks.workunit.client.0.smithi042.stdout:9/114: write d6/dd/f20 [1063013,75678] 0 2024-02-15T18:26:52.503 INFO:tasks.workunit.client.0.smithi042.stdout:4/118: creat da/f24 x:0 0 0 2024-02-15T18:26:52.503 INFO:tasks.workunit.client.0.smithi042.stdout:7/147: unlink d10/d12/d23/f37 0 2024-02-15T18:26:52.504 INFO:tasks.workunit.client.0.smithi042.stdout:3/144: creat d1/de/d1a/f2b x:0 0 0 2024-02-15T18:26:52.504 INFO:tasks.workunit.client.0.smithi042.stdout:3/145: dread - d1/de/d19/f1f zero size 2024-02-15T18:26:52.504 INFO:tasks.workunit.client.0.smithi042.stdout:3/146: truncate d1/de/d1a/f2b 582806 0 2024-02-15T18:26:52.505 INFO:tasks.workunit.client.0.smithi042.stdout:5/131: creat d0/f34 x:0 0 0 2024-02-15T18:26:52.505 INFO:tasks.workunit.client.0.smithi042.stdout:5/132: write d0/d14/f29 [1438215,102687] 0 2024-02-15T18:26:52.506 INFO:tasks.workunit.client.0.smithi042.stdout:4/119: rename da/fd to da/f25 0 2024-02-15T18:26:52.509 INFO:tasks.workunit.client.0.smithi042.stdout:3/147: stat d1/d11/c12 0 2024-02-15T18:26:52.510 INFO:tasks.workunit.client.0.smithi042.stdout:4/120: creat da/f26 x:0 0 0 2024-02-15T18:26:52.510 INFO:tasks.workunit.client.0.smithi042.stdout:4/121: chown da/c17 1 1 2024-02-15T18:26:52.510 INFO:tasks.workunit.client.0.smithi042.stdout:4/122: dread - da/f21 zero size 2024-02-15T18:26:52.510 INFO:tasks.workunit.client.0.smithi042.stdout:6/168: dwrite d9/d11/f2b [0,4194304] 0 2024-02-15T18:26:52.511 INFO:tasks.workunit.client.0.smithi042.stdout:3/148: getdents d1/d11/d14 0 2024-02-15T18:26:52.512 INFO:tasks.workunit.client.0.smithi042.stdout:6/169: creat d9/d11/d19/f38 x:0 0 0 2024-02-15T18:26:52.512 INFO:tasks.workunit.client.0.smithi042.stdout:6/170: write d9/d11/d19/f38 [508522,93618] 0 2024-02-15T18:26:52.512 INFO:tasks.workunit.client.0.smithi042.stdout:6/171: chown d9/d11/d19/d1c/f27 54305 1 2024-02-15T18:26:52.512 INFO:tasks.workunit.client.0.smithi042.stdout:6/172: write d9/d11/d19/f2a [516539,70779] 0 2024-02-15T18:26:52.512 INFO:tasks.workunit.client.0.smithi042.stdout:6/173: chown d9/d11/d19/f37 38654 1 2024-02-15T18:26:52.513 INFO:tasks.workunit.client.0.smithi042.stdout:3/149: rename d1/de/f20 to d1/d11/f2c 0 2024-02-15T18:26:52.514 INFO:tasks.workunit.client.0.smithi042.stdout:3/150: write d1/f15 [1296589,93086] 0 2024-02-15T18:26:52.520 INFO:tasks.workunit.client.0.smithi042.stdout:7/148: dread d10/f22 [4194304,4194304] 0 2024-02-15T18:26:52.524 INFO:tasks.workunit.client.0.smithi042.stdout:3/151: creat d1/de/f2d x:0 0 0 2024-02-15T18:26:52.530 INFO:tasks.workunit.client.0.smithi042.stdout:7/149: rename d10/d12/f16 to d10/d15/d39/f3a 0 2024-02-15T18:26:52.530 INFO:tasks.workunit.client.0.smithi042.stdout:3/152: mkdir d1/d11/d2e 0 2024-02-15T18:26:52.532 INFO:tasks.workunit.client.0.smithi042.stdout:7/150: getdents d10/d15 0 2024-02-15T18:26:52.532 INFO:tasks.workunit.client.0.smithi042.stdout:3/153: creat d1/de/d1a/f2f x:0 0 0 2024-02-15T18:26:52.533 INFO:tasks.workunit.client.0.smithi042.stdout:9/115: dwrite f5 [0,4194304] 0 2024-02-15T18:26:52.536 INFO:tasks.workunit.client.0.smithi042.stdout:5/133: dwrite d0/db/f1f [0,4194304] 0 2024-02-15T18:26:52.539 INFO:tasks.workunit.client.0.smithi042.stdout:7/151: unlink d10/d12/d23/f29 0 2024-02-15T18:26:52.539 INFO:tasks.workunit.client.0.smithi042.stdout:3/154: symlink d1/d11/d14/l30 0 2024-02-15T18:26:52.539 INFO:tasks.workunit.client.0.smithi042.stdout:7/152: dread - d10/d15/f36 zero size 2024-02-15T18:26:52.542 INFO:tasks.workunit.client.0.smithi042.stdout:5/134: rmdir d0/d22/d2d 39 2024-02-15T18:26:52.542 INFO:tasks.workunit.client.0.smithi042.stdout:5/135: chown d0/c33 100659466 1 2024-02-15T18:26:52.542 INFO:tasks.workunit.client.0.smithi042.stdout:5/136: dread - d0/f34 zero size 2024-02-15T18:26:52.544 INFO:tasks.workunit.client.0.smithi042.stdout:7/153: creat d10/d12/d23/f3b x:0 0 0 2024-02-15T18:26:52.544 INFO:tasks.workunit.client.0.smithi042.stdout:7/154: chown d10/d15 0 1 2024-02-15T18:26:52.544 INFO:tasks.workunit.client.0.smithi042.stdout:7/155: fsync d10/d15/f34 0 2024-02-15T18:26:52.544 INFO:tasks.workunit.client.0.smithi042.stdout:7/156: getdents d10/d15 0 2024-02-15T18:26:52.544 INFO:tasks.workunit.client.0.smithi042.stdout:7/157: dread - d10/d12/d23/f3b zero size 2024-02-15T18:26:52.544 INFO:tasks.workunit.client.0.smithi042.stdout:7/158: truncate d10/d12/d23/f3b 402152 0 2024-02-15T18:26:52.544 INFO:tasks.workunit.client.0.smithi042.stdout:3/155: creat d1/de/d1a/f31 x:0 0 0 2024-02-15T18:26:52.545 INFO:tasks.workunit.client.0.smithi042.stdout:7/159: dread d10/f1b [0,4194304] 0 2024-02-15T18:26:52.545 INFO:tasks.workunit.client.0.smithi042.stdout:3/156: unlink d1/cc 0 2024-02-15T18:26:52.547 INFO:tasks.workunit.client.0.smithi042.stdout:7/160: mknod d10/c3c 0 2024-02-15T18:26:52.549 INFO:tasks.workunit.client.0.smithi042.stdout:3/157: rename d1/de/d1a/f31 to d1/f32 0 2024-02-15T18:26:52.550 INFO:tasks.workunit.client.0.smithi042.stdout:4/123: dwrite da/d11/d1b/f1c [0,4194304] 0 2024-02-15T18:26:52.551 INFO:tasks.workunit.client.0.smithi042.stdout:6/174: dwrite d9/f14 [0,4194304] 0 2024-02-15T18:26:52.551 INFO:tasks.workunit.client.0.smithi042.stdout:6/175: truncate d9/d11/f2b 4999764 0 2024-02-15T18:26:52.551 INFO:tasks.workunit.client.0.smithi042.stdout:7/161: truncate d10/f21 56998 0 2024-02-15T18:26:52.552 INFO:tasks.workunit.client.0.smithi042.stdout:9/116: dwrite f2 [0,4194304] 0 2024-02-15T18:26:52.567 INFO:tasks.workunit.client.0.smithi042.stdout:4/124: mkdir da/d11/d27 0 2024-02-15T18:26:52.569 INFO:tasks.workunit.client.0.smithi042.stdout:6/176: mknod d9/d11/d1d/c39 0 2024-02-15T18:26:52.569 INFO:tasks.workunit.client.0.smithi042.stdout:6/177: write d9/f21 [1294179,41733] 0 2024-02-15T18:26:52.571 INFO:tasks.workunit.client.0.smithi042.stdout:9/117: unlink d6/da/d1d/l28 0 2024-02-15T18:26:52.572 INFO:tasks.workunit.client.0.smithi042.stdout:4/125: mknod da/d11/d27/c28 0 2024-02-15T18:26:52.573 INFO:tasks.workunit.client.0.smithi042.stdout:3/158: dwrite d1/de/d1a/f1e [0,4194304] 0 2024-02-15T18:26:52.573 INFO:tasks.workunit.client.0.smithi042.stdout:3/159: stat d1/de/d19/l1b 0 2024-02-15T18:26:52.573 INFO:tasks.workunit.client.0.smithi042.stdout:3/160: dread - d1/de/d1a/f2f zero size 2024-02-15T18:26:52.573 INFO:tasks.workunit.client.0.smithi042.stdout:3/161: fsync d1/d11/f2c 0 2024-02-15T18:26:52.573 INFO:tasks.workunit.client.0.smithi042.stdout:3/162: write d1/de/d1a/f1e [4265530,53441] 0 2024-02-15T18:26:52.573 INFO:tasks.workunit.client.0.smithi042.stdout:3/163: truncate d1/f7 2812633 0 2024-02-15T18:26:52.574 INFO:tasks.workunit.client.0.smithi042.stdout:6/178: rmdir d9 39 2024-02-15T18:26:52.575 INFO:tasks.workunit.client.0.smithi042.stdout:9/118: symlink d6/l2a 0 2024-02-15T18:26:52.584 INFO:tasks.workunit.client.0.smithi042.stdout:4/126: rmdir da/d11 39 2024-02-15T18:26:52.584 INFO:tasks.workunit.client.0.smithi042.stdout:6/179: creat d9/d11/d19/d1c/d31/f3a x:0 0 0 2024-02-15T18:26:52.584 INFO:tasks.workunit.client.0.smithi042.stdout:9/119: unlink d6/l2a 0 2024-02-15T18:26:52.585 INFO:tasks.workunit.client.0.smithi042.stdout:4/127: getdents da/d11/d27 0 2024-02-15T18:26:52.586 INFO:tasks.workunit.client.0.smithi042.stdout:9/120: creat d6/dd/f2b x:0 0 0 2024-02-15T18:26:52.586 INFO:tasks.workunit.client.0.smithi042.stdout:9/121: chown d6/da/f10 372275 1 2024-02-15T18:26:52.588 INFO:tasks.workunit.client.0.smithi042.stdout:9/122: write d6/dd/d9/f1f [3648047,61397] 0 2024-02-15T18:26:52.588 INFO:tasks.workunit.client.0.smithi042.stdout:9/123: fsync d6/da/f25 0 2024-02-15T18:26:52.588 INFO:tasks.workunit.client.0.smithi042.stdout:6/180: unlink d9/lf 0 2024-02-15T18:26:52.588 INFO:tasks.workunit.client.0.smithi042.stdout:6/181: dread - d9/d11/d19/d1c/d31/f3a zero size 2024-02-15T18:26:52.589 INFO:tasks.workunit.client.0.smithi042.stdout:9/124: symlink d6/dd/l2c 0 2024-02-15T18:26:52.591 INFO:tasks.workunit.client.0.smithi042.stdout:6/182: creat d9/d11/d19/d24/f3b x:0 0 0 2024-02-15T18:26:52.591 INFO:tasks.workunit.client.0.smithi042.stdout:6/183: chown d9/d11/d19/d1c/d31/f3a 96443 1 2024-02-15T18:26:52.592 INFO:tasks.workunit.client.0.smithi042.stdout:9/125: mkdir d6/da/d1d/d2d 0 2024-02-15T18:26:52.597 INFO:tasks.workunit.client.0.smithi042.stdout:8/70: sync 2024-02-15T18:26:52.598 INFO:tasks.workunit.client.0.smithi042.stdout:3/164: dwrite d1/de/d1a/f1e [0,4194304] 0 2024-02-15T18:26:52.599 INFO:tasks.workunit.client.0.smithi042.stdout:8/71: creat dd/f13 x:0 0 0 2024-02-15T18:26:52.599 INFO:tasks.workunit.client.0.smithi042.stdout:8/72: getdents dd/d11 0 2024-02-15T18:26:52.600 INFO:tasks.workunit.client.0.smithi042.stdout:3/165: write d1/f7 [1804846,29359] 0 2024-02-15T18:26:52.601 INFO:tasks.workunit.client.0.smithi042.stdout:8/73: mknod dd/c14 0 2024-02-15T18:26:52.603 INFO:tasks.workunit.client.0.smithi042.stdout:3/166: rename d1/c3 to d1/de/c33 0 2024-02-15T18:26:52.604 INFO:tasks.workunit.client.0.smithi042.stdout:8/74: link dd/de/f10 dd/d11/f15 0 2024-02-15T18:26:52.607 INFO:tasks.workunit.client.0.smithi042.stdout:9/126: dread d6/da/f1c [0,4194304] 0 2024-02-15T18:26:52.607 INFO:tasks.workunit.client.0.smithi042.stdout:3/167: write d1/f9 [1406475,58146] 0 2024-02-15T18:26:52.609 INFO:tasks.workunit.client.0.smithi042.stdout:6/184: dwrite d9/d11/d19/d24/f3b [0,4194304] 0 2024-02-15T18:26:52.609 INFO:tasks.workunit.client.0.smithi042.stdout:6/185: readlink d9/d11/l1e 0 2024-02-15T18:26:52.609 INFO:tasks.workunit.client.0.smithi042.stdout:3/168: rmdir d1/de/d19 39 2024-02-15T18:26:52.612 INFO:tasks.workunit.client.0.smithi042.stdout:6/186: creat d9/f3c x:0 0 0 2024-02-15T18:26:52.618 INFO:tasks.workunit.client.0.smithi042.stdout:6/187: symlink d9/d11/d19/d36/l3d 0 2024-02-15T18:26:52.622 INFO:tasks.workunit.client.0.smithi042.stdout:6/188: rename d9/d11/d19/d24 to d9/d3e 0 2024-02-15T18:26:52.623 INFO:tasks.workunit.client.0.smithi042.stdout:6/189: chown d9/lb 42589041 1 2024-02-15T18:26:52.624 INFO:tasks.workunit.client.0.smithi042.stdout:6/190: chown d9/d11/d19/d1c/d31/f34 556 1 2024-02-15T18:26:52.624 INFO:tasks.workunit.client.0.smithi042.stdout:6/191: chown d9/d3e 7 1 2024-02-15T18:26:52.624 INFO:tasks.workunit.client.0.smithi042.stdout:6/192: write d9/f2d [44209,101924] 0 2024-02-15T18:26:52.624 INFO:tasks.workunit.client.0.smithi042.stdout:3/169: dwrite d1/de/f2d [0,4194304] 0 2024-02-15T18:26:52.625 INFO:tasks.workunit.client.0.smithi042.stdout:6/193: creat d9/d3e/f3f x:0 0 0 2024-02-15T18:26:52.627 INFO:tasks.workunit.client.0.smithi042.stdout:6/194: dread d9/d11/d19/f38 [0,4194304] 0 2024-02-15T18:26:52.629 INFO:tasks.workunit.client.0.smithi042.stdout:6/195: getdents d9/d11/d19/d1c/d31 0 2024-02-15T18:26:52.630 INFO:tasks.workunit.client.0.smithi042.stdout:6/196: write d9/d11/d19/d1c/d31/f3a [242127,47520] 0 2024-02-15T18:26:52.630 INFO:tasks.workunit.client.0.smithi042.stdout:3/170: dread d1/de/d1a/f1e [0,4194304] 0 2024-02-15T18:26:52.635 INFO:tasks.workunit.client.0.smithi042.stdout:2/102: sync 2024-02-15T18:26:52.635 INFO:tasks.workunit.client.0.smithi042.stdout:1/146: sync 2024-02-15T18:26:52.635 INFO:tasks.workunit.client.0.smithi042.stdout:0/164: sync 2024-02-15T18:26:52.636 INFO:tasks.workunit.client.0.smithi042.stdout:0/165: mknod d4/d38/c3e 0 2024-02-15T18:26:52.636 INFO:tasks.workunit.client.0.smithi042.stdout:0/166: dread - d4/d9/f31 zero size 2024-02-15T18:26:52.637 INFO:tasks.workunit.client.0.smithi042.stdout:0/167: symlink d4/d9/d34/l3f 0 2024-02-15T18:26:52.642 INFO:tasks.workunit.client.0.smithi042.stdout:0/168: creat d4/d9/d34/f40 x:0 0 0 2024-02-15T18:26:52.654 INFO:tasks.workunit.client.0.smithi042.stdout:1/147: dwrite f6 [0,4194304] 0 2024-02-15T18:26:52.657 INFO:tasks.workunit.client.0.smithi042.stdout:1/148: link de/l1e de/d15/l33 0 2024-02-15T18:26:52.658 INFO:tasks.workunit.client.0.smithi042.stdout:1/149: truncate de/d15/d2e/f26 4173819 0 2024-02-15T18:26:52.659 INFO:tasks.workunit.client.0.smithi042.stdout:1/150: creat de/d15/d2e/f34 x:0 0 0 2024-02-15T18:26:52.659 INFO:tasks.workunit.client.0.smithi042.stdout:1/151: truncate de/d15/d2e/f34 299026 0 2024-02-15T18:26:52.659 INFO:tasks.workunit.client.0.smithi042.stdout:2/103: dwrite d0/d8/f11 [0,4194304] 0 2024-02-15T18:26:52.660 INFO:tasks.workunit.client.0.smithi042.stdout:1/152: link f6 de/d15/d2c/f35 0 2024-02-15T18:26:52.662 INFO:tasks.workunit.client.0.smithi042.stdout:1/153: dread de/d15/f20 [4194304,4194304] 0 2024-02-15T18:26:52.662 INFO:tasks.workunit.client.0.smithi042.stdout:3/171: dwrite d1/de/f2d [4194304,4194304] 0 2024-02-15T18:26:52.662 INFO:tasks.workunit.client.0.smithi042.stdout:0/169: dwrite d4/d9/ff [0,4194304] 0 2024-02-15T18:26:52.664 INFO:tasks.workunit.client.0.smithi042.stdout:2/104: unlink d0/f5 0 2024-02-15T18:26:52.664 INFO:tasks.workunit.client.0.smithi042.stdout:2/105: chown d0/d8/db/fd 5 1 2024-02-15T18:26:52.664 INFO:tasks.workunit.client.0.smithi042.stdout:2/106: write d0/f7 [8659141,123244] 0 2024-02-15T18:26:52.665 INFO:tasks.workunit.client.0.smithi042.stdout:1/154: creat de/d15/d1b/f36 x:0 0 0 2024-02-15T18:26:52.667 INFO:tasks.workunit.client.0.smithi042.stdout:1/155: dread de/f29 [0,4194304] 0 2024-02-15T18:26:52.667 INFO:tasks.workunit.client.0.smithi042.stdout:1/156: write de/d15/d2e/f32 [831726,32551] 0 2024-02-15T18:26:52.671 INFO:tasks.workunit.client.0.smithi042.stdout:3/172: creat d1/f34 x:0 0 0 2024-02-15T18:26:52.672 INFO:tasks.workunit.client.0.smithi042.stdout:2/107: link d0/l6 d0/l1d 0 2024-02-15T18:26:52.674 INFO:tasks.workunit.client.0.smithi042.stdout:3/173: creat d1/d11/d14/f35 x:0 0 0 2024-02-15T18:26:52.676 INFO:tasks.workunit.client.0.smithi042.stdout:2/108: rename d0/c4 to d0/d8/db/c1e 0 2024-02-15T18:26:52.676 INFO:tasks.workunit.client.0.smithi042.stdout:3/174: symlink d1/de/d1a/l36 0 2024-02-15T18:26:52.679 INFO:tasks.workunit.client.0.smithi042.stdout:2/109: symlink d0/d8/db/de/d1a/l1f 0 2024-02-15T18:26:52.679 INFO:tasks.workunit.client.0.smithi042.stdout:2/110: dread - d0/f19 zero size 2024-02-15T18:26:52.679 INFO:tasks.workunit.client.0.smithi042.stdout:2/111: write d0/f3 [4723938,113664] 0 2024-02-15T18:26:52.679 INFO:tasks.workunit.client.0.smithi042.stdout:2/112: readlink d0/l1d 0 2024-02-15T18:26:52.679 INFO:tasks.workunit.client.0.smithi042.stdout:3/175: creat d1/d11/d2e/f37 x:0 0 0 2024-02-15T18:26:52.679 INFO:tasks.workunit.client.0.smithi042.stdout:3/176: dread - d1/de/f27 zero size 2024-02-15T18:26:52.680 INFO:tasks.workunit.client.0.smithi042.stdout:5/137: sync 2024-02-15T18:26:52.680 INFO:tasks.workunit.client.0.smithi042.stdout:7/162: sync 2024-02-15T18:26:52.680 INFO:tasks.workunit.client.0.smithi042.stdout:5/138: dread - d0/d22/d2d/f32 zero size 2024-02-15T18:26:52.680 INFO:tasks.workunit.client.0.smithi042.stdout:2/113: mkdir d0/d8/d20 0 2024-02-15T18:26:52.681 INFO:tasks.workunit.client.0.smithi042.stdout:3/177: getdents d1/de 0 2024-02-15T18:26:52.682 INFO:tasks.workunit.client.0.smithi042.stdout:5/139: creat d0/db/f35 x:0 0 0 2024-02-15T18:26:52.683 INFO:tasks.workunit.client.0.smithi042.stdout:2/114: mknod d0/d8/db/de/d1a/c21 0 2024-02-15T18:26:52.684 INFO:tasks.workunit.client.0.smithi042.stdout:3/178: symlink d1/d11/l38 0 2024-02-15T18:26:52.685 INFO:tasks.workunit.client.0.smithi042.stdout:5/140: symlink d0/d22/l36 0 2024-02-15T18:26:52.685 INFO:tasks.workunit.client.0.smithi042.stdout:5/141: chown d0/d14/f1c 0 1 2024-02-15T18:26:52.686 INFO:tasks.workunit.client.0.smithi042.stdout:2/115: symlink d0/d8/d1b/l22 0 2024-02-15T18:26:52.686 INFO:tasks.workunit.client.0.smithi042.stdout:2/116: chown d0/l1d 658719 1 2024-02-15T18:26:52.687 INFO:tasks.workunit.client.0.smithi042.stdout:3/179: creat d1/d11/d14/f39 x:0 0 0 2024-02-15T18:26:52.687 INFO:tasks.workunit.client.0.smithi042.stdout:1/157: dwrite de/d15/d2e/f34 [0,4194304] 0 2024-02-15T18:26:52.688 INFO:tasks.workunit.client.0.smithi042.stdout:5/142: mknod d0/d12/d2a/c37 0 2024-02-15T18:26:52.692 INFO:tasks.workunit.client.0.smithi042.stdout:2/117: truncate d0/d8/f11 2796748 0 2024-02-15T18:26:52.693 INFO:tasks.workunit.client.0.smithi042.stdout:3/180: creat d1/de/f3a x:0 0 0 2024-02-15T18:26:52.693 INFO:tasks.workunit.client.0.smithi042.stdout:3/181: chown d1/de/d1a/f1e 1168098 1 2024-02-15T18:26:52.696 INFO:tasks.workunit.client.0.smithi042.stdout:1/158: symlink de/d15/l37 0 2024-02-15T18:26:52.696 INFO:tasks.workunit.client.0.smithi042.stdout:1/159: chown de/d15/d2e/l2b 1857024575 1 2024-02-15T18:26:52.696 INFO:tasks.workunit.client.0.smithi042.stdout:1/160: write de/f29 [1228855,10757] 0 2024-02-15T18:26:52.697 INFO:tasks.workunit.client.0.smithi042.stdout:2/118: unlink d0/f15 0 2024-02-15T18:26:52.698 INFO:tasks.workunit.client.0.smithi042.stdout:1/161: read de/d15/d2e/f26 [1285718,112417] 0 2024-02-15T18:26:52.698 INFO:tasks.workunit.client.0.smithi042.stdout:2/119: stat d0/d8/db 0 2024-02-15T18:26:52.702 INFO:tasks.workunit.client.0.smithi042.stdout:7/163: dwrite d10/d15/f34 [0,4194304] 0 2024-02-15T18:26:52.702 INFO:tasks.workunit.client.0.smithi042.stdout:7/164: chown d10/d15/c33 3439 1 2024-02-15T18:26:52.707 INFO:tasks.workunit.client.0.smithi042.stdout:7/165: creat d10/d15/f3d x:0 0 0 2024-02-15T18:26:52.711 INFO:tasks.workunit.client.0.smithi042.stdout:5/143: dwrite d0/db/f1f [0,4194304] 0 2024-02-15T18:26:52.711 INFO:tasks.workunit.client.0.smithi042.stdout:5/144: truncate d0/d22/d2d/f32 885238 0 2024-02-15T18:26:52.717 INFO:tasks.workunit.client.0.smithi042.stdout:5/145: mkdir d0/d38 0 2024-02-15T18:26:52.718 INFO:tasks.workunit.client.0.smithi042.stdout:5/146: dread d0/d22/d2d/f32 [0,4194304] 0 2024-02-15T18:26:52.734 INFO:tasks.workunit.client.0.smithi042.stdout:5/147: dread d0/db/f1f [0,4194304] 0 2024-02-15T18:26:52.740 INFO:tasks.workunit.client.0.smithi042.stdout:3/182: dwrite d1/de/d1a/f1e [0,4194304] 0 2024-02-15T18:26:52.740 INFO:tasks.workunit.client.0.smithi042.stdout:1/162: dwrite f6 [0,4194304] 0 2024-02-15T18:26:52.740 INFO:tasks.workunit.client.0.smithi042.stdout:2/120: dwrite d0/d8/db/fd [0,4194304] 0 2024-02-15T18:26:52.740 INFO:tasks.workunit.client.0.smithi042.stdout:7/166: dwrite fb [0,4194304] 0 2024-02-15T18:26:52.740 INFO:tasks.workunit.client.0.smithi042.stdout:3/183: creat d1/d11/d2e/f3b x:0 0 0 2024-02-15T18:26:52.743 INFO:tasks.workunit.client.0.smithi042.stdout:5/148: getdents d0 0 2024-02-15T18:26:52.747 INFO:tasks.workunit.client.0.smithi042.stdout:2/121: symlink d0/l23 0 2024-02-15T18:26:52.748 INFO:tasks.workunit.client.0.smithi042.stdout:2/122: dread d0/d8/db/de/f18 [0,4194304] 0 2024-02-15T18:26:52.749 INFO:tasks.workunit.client.0.smithi042.stdout:3/184: rename d1/de/d19/c2a to d1/d11/c3c 0 2024-02-15T18:26:52.749 INFO:tasks.workunit.client.0.smithi042.stdout:3/185: readlink d1/de/l23 0 2024-02-15T18:26:52.750 INFO:tasks.workunit.client.0.smithi042.stdout:2/123: mknod d0/d8/c24 0 2024-02-15T18:26:52.750 INFO:tasks.workunit.client.0.smithi042.stdout:2/124: fsync d0/d8/db/fd 0 2024-02-15T18:26:52.751 INFO:tasks.workunit.client.0.smithi042.stdout:3/186: mknod d1/d11/d2e/c3d 0 2024-02-15T18:26:52.753 INFO:tasks.workunit.client.0.smithi042.stdout:3/187: creat d1/de/d19/f3e x:0 0 0 2024-02-15T18:26:52.753 INFO:tasks.workunit.client.0.smithi042.stdout:3/188: dread - d1/f32 zero size 2024-02-15T18:26:52.753 INFO:tasks.workunit.client.0.smithi042.stdout:7/167: dread d10/d15/f1d [0,4194304] 0 2024-02-15T18:26:52.754 INFO:tasks.workunit.client.0.smithi042.stdout:3/189: getdents d1 0 2024-02-15T18:26:52.755 INFO:tasks.workunit.client.0.smithi042.stdout:7/168: symlink d10/d15/l3e 0 2024-02-15T18:26:52.756 INFO:tasks.workunit.client.0.smithi042.stdout:3/190: symlink d1/de/l3f 0 2024-02-15T18:26:52.757 INFO:tasks.workunit.client.0.smithi042.stdout:5/149: dread d0/db/fd [0,4194304] 0 2024-02-15T18:26:52.757 INFO:tasks.workunit.client.0.smithi042.stdout:5/150: truncate d0/db/f35 244795 0 2024-02-15T18:26:52.757 INFO:tasks.workunit.client.0.smithi042.stdout:5/151: stat d0/d22/c24 0 2024-02-15T18:26:52.757 INFO:tasks.workunit.client.0.smithi042.stdout:1/163: dwrite fb [0,4194304] 0 2024-02-15T18:26:52.757 INFO:tasks.workunit.client.0.smithi042.stdout:1/164: fsync de/d15/f31 0 2024-02-15T18:26:52.757 INFO:tasks.workunit.client.0.smithi042.stdout:3/191: mknod d1/de/d19/c40 0 2024-02-15T18:26:52.757 INFO:tasks.workunit.client.0.smithi042.stdout:3/192: chown d1/de/f2d 46 1 2024-02-15T18:26:52.758 INFO:tasks.workunit.client.0.smithi042.stdout:5/152: mkdir d0/d39 0 2024-02-15T18:26:52.762 INFO:tasks.workunit.client.0.smithi042.stdout:8/75: sync 2024-02-15T18:26:52.762 INFO:tasks.workunit.client.0.smithi042.stdout:9/127: sync 2024-02-15T18:26:52.763 INFO:tasks.workunit.client.0.smithi042.stdout:4/128: sync 2024-02-15T18:26:52.763 INFO:tasks.workunit.client.0.smithi042.stdout:6/197: sync 2024-02-15T18:26:52.763 INFO:tasks.workunit.client.0.smithi042.stdout:0/170: sync 2024-02-15T18:26:52.763 INFO:tasks.workunit.client.0.smithi042.stdout:9/128: truncate d6/da/f25 492470 0 2024-02-15T18:26:52.763 INFO:tasks.workunit.client.0.smithi042.stdout:6/198: truncate d9/f2d 873731 0 2024-02-15T18:26:52.763 INFO:tasks.workunit.client.0.smithi042.stdout:0/171: readlink d4/d9/d15/l1e 0 2024-02-15T18:26:52.763 INFO:tasks.workunit.client.0.smithi042.stdout:6/199: write d9/d11/d19/d1c/d31/f3a [847013,23945] 0 2024-02-15T18:26:52.763 INFO:tasks.workunit.client.0.smithi042.stdout:4/129: write da/d11/fe [766752,48155] 0 2024-02-15T18:26:52.764 INFO:tasks.workunit.client.0.smithi042.stdout:2/125: dread d0/d8/db/fd [0,4194304] 0 2024-02-15T18:26:52.764 INFO:tasks.workunit.client.0.smithi042.stdout:2/126: chown d0/d8/c13 176057 1 2024-02-15T18:26:52.764 INFO:tasks.workunit.client.0.smithi042.stdout:5/153: write d0/db/f1f [3229274,99601] 0 2024-02-15T18:26:52.766 INFO:tasks.workunit.client.0.smithi042.stdout:1/165: creat de/d15/d2c/f38 x:0 0 0 2024-02-15T18:26:52.767 INFO:tasks.workunit.client.0.smithi042.stdout:3/193: unlink d1/de/l18 0 2024-02-15T18:26:52.767 INFO:tasks.workunit.client.0.smithi042.stdout:3/194: fsync d1/d11/d14/f35 0 2024-02-15T18:26:52.767 INFO:tasks.workunit.client.0.smithi042.stdout:3/195: readlink d1/de/l3f 0 2024-02-15T18:26:52.768 INFO:tasks.workunit.client.0.smithi042.stdout:8/76: getdents dd/de 0 2024-02-15T18:26:52.768 INFO:tasks.workunit.client.0.smithi042.stdout:8/77: dread - dd/de/f10 zero size 2024-02-15T18:26:52.768 INFO:tasks.workunit.client.0.smithi042.stdout:8/78: chown dd/de 1 1 2024-02-15T18:26:52.770 INFO:tasks.workunit.client.0.smithi042.stdout:0/172: unlink d4/d9/l17 0 2024-02-15T18:26:52.771 INFO:tasks.workunit.client.0.smithi042.stdout:4/130: mkdir da/d11/d29 0 2024-02-15T18:26:52.772 INFO:tasks.workunit.client.0.smithi042.stdout:5/154: mknod d0/db/c3a 0 2024-02-15T18:26:52.772 INFO:tasks.workunit.client.0.smithi042.stdout:5/155: chown d0/f23 662256061 1 2024-02-15T18:26:52.773 INFO:tasks.workunit.client.0.smithi042.stdout:1/166: mknod de/d15/d1b/c39 0 2024-02-15T18:26:52.773 INFO:tasks.workunit.client.0.smithi042.stdout:9/129: dread d6/dd/fe [0,4194304] 0 2024-02-15T18:26:52.773 INFO:tasks.workunit.client.0.smithi042.stdout:8/79: rmdir dd 39 2024-02-15T18:26:52.776 INFO:tasks.workunit.client.0.smithi042.stdout:0/173: unlink d4/d38/d39/f3d 0 2024-02-15T18:26:52.777 INFO:tasks.workunit.client.0.smithi042.stdout:7/169: dwrite d10/d15/f1d [0,4194304] 0 2024-02-15T18:26:52.777 INFO:tasks.workunit.client.0.smithi042.stdout:6/200: rename d9/d11/d19/d1c/d31/f34 to d9/f40 0 2024-02-15T18:26:52.782 INFO:tasks.workunit.client.0.smithi042.stdout:5/156: creat d0/d12/d2a/f3b x:0 0 0 2024-02-15T18:26:52.782 INFO:tasks.workunit.client.0.smithi042.stdout:5/157: write d0/d12/d2a/f3b [158133,64832] 0 2024-02-15T18:26:52.782 INFO:tasks.workunit.client.0.smithi042.stdout:1/167: link fb de/d15/d2e/f3a 0 2024-02-15T18:26:52.782 INFO:tasks.workunit.client.0.smithi042.stdout:1/168: write de/d15/d1b/f36 [238735,83349] 0 2024-02-15T18:26:52.788 INFO:tasks.workunit.client.0.smithi042.stdout:7/170: creat d10/d12/f3f x:0 0 0 2024-02-15T18:26:52.790 INFO:tasks.workunit.client.0.smithi042.stdout:2/127: rename d0/l6 to d0/d8/d10/l25 0 2024-02-15T18:26:52.791 INFO:tasks.workunit.client.0.smithi042.stdout:6/201: creat d9/d11/d1d/f41 x:0 0 0 2024-02-15T18:26:52.792 INFO:tasks.workunit.client.0.smithi042.stdout:5/158: creat d0/d38/f3c x:0 0 0 2024-02-15T18:26:52.793 INFO:tasks.workunit.client.0.smithi042.stdout:1/169: creat de/d15/d2e/f3b x:0 0 0 2024-02-15T18:26:52.795 INFO:tasks.workunit.client.0.smithi042.stdout:3/196: rename d1/de/l22 to d1/d11/d2e/l41 0 2024-02-15T18:26:52.796 INFO:tasks.workunit.client.0.smithi042.stdout:5/159: mknod d0/d22/d2d/c3d 0 2024-02-15T18:26:52.796 INFO:tasks.workunit.client.0.smithi042.stdout:5/160: truncate d0/d12/d2a/f3b 908787 0 2024-02-15T18:26:52.796 INFO:tasks.workunit.client.0.smithi042.stdout:7/171: write fd [3671333,12501] 0 2024-02-15T18:26:52.797 INFO:tasks.workunit.client.0.smithi042.stdout:0/174: rename d4/d9/ff to d4/f41 0 2024-02-15T18:26:52.798 INFO:tasks.workunit.client.0.smithi042.stdout:3/197: creat d1/de/f42 x:0 0 0 2024-02-15T18:26:52.801 INFO:tasks.workunit.client.0.smithi042.stdout:7/172: mkdir d10/d12/d40 0 2024-02-15T18:26:52.801 INFO:tasks.workunit.client.0.smithi042.stdout:7/173: chown d10/d15/l31 30422 1 2024-02-15T18:26:52.802 INFO:tasks.workunit.client.0.smithi042.stdout:3/198: creat d1/d11/d14/f43 x:0 0 0 2024-02-15T18:26:52.802 INFO:tasks.workunit.client.0.smithi042.stdout:3/199: chown d1/d11/d2e/f3b 202 1 2024-02-15T18:26:52.804 INFO:tasks.workunit.client.0.smithi042.stdout:7/174: rmdir d10/d12/d40 0 2024-02-15T18:26:52.806 INFO:tasks.workunit.client.0.smithi042.stdout:7/175: dread d10/f1b [0,4194304] 0 2024-02-15T18:26:52.806 INFO:tasks.workunit.client.0.smithi042.stdout:7/176: readlink d10/l38 0 2024-02-15T18:26:52.806 INFO:tasks.workunit.client.0.smithi042.stdout:7/177: chown d10/d15/l25 1328330 1 2024-02-15T18:26:52.808 INFO:tasks.workunit.client.0.smithi042.stdout:9/130: dwrite d6/da/f10 [0,4194304] 0 2024-02-15T18:26:52.809 INFO:tasks.workunit.client.0.smithi042.stdout:4/131: dwrite da/d11/fe [0,4194304] 0 2024-02-15T18:26:52.810 INFO:tasks.workunit.client.0.smithi042.stdout:9/131: creat d6/f2e x:0 0 0 2024-02-15T18:26:52.812 INFO:tasks.workunit.client.0.smithi042.stdout:9/132: mkdir d6/da/d2f 0 2024-02-15T18:26:52.812 INFO:tasks.workunit.client.0.smithi042.stdout:9/133: write d6/dd/f2b [614498,78845] 0 2024-02-15T18:26:52.813 INFO:tasks.workunit.client.0.smithi042.stdout:9/134: dread - d6/dd/d9/f16 zero size 2024-02-15T18:26:52.813 INFO:tasks.workunit.client.0.smithi042.stdout:9/135: chown f3 0 1 2024-02-15T18:26:52.813 INFO:tasks.workunit.client.0.smithi042.stdout:6/202: write d9/f2f [3662576,25369] 0 2024-02-15T18:26:52.813 INFO:tasks.workunit.client.0.smithi042.stdout:9/136: symlink d6/da/d1d/l30 0 2024-02-15T18:26:52.814 INFO:tasks.workunit.client.0.smithi042.stdout:9/137: chown d6/dd/d9/f14 1151536 1 2024-02-15T18:26:52.818 INFO:tasks.workunit.client.0.smithi042.stdout:6/203: symlink d9/l42 0 2024-02-15T18:26:52.820 INFO:tasks.workunit.client.0.smithi042.stdout:6/204: dread d9/d11/d19/d1c/f27 [0,4194304] 0 2024-02-15T18:26:52.820 INFO:tasks.workunit.client.0.smithi042.stdout:6/205: write d9/d11/f2b [5733519,88454] 0 2024-02-15T18:26:52.820 INFO:tasks.workunit.client.0.smithi042.stdout:6/206: stat d9/f32 0 2024-02-15T18:26:52.820 INFO:tasks.workunit.client.0.smithi042.stdout:6/207: chown d9/f2d 0 1 2024-02-15T18:26:52.820 INFO:tasks.workunit.client.0.smithi042.stdout:6/208: chown d9/lb 252 1 2024-02-15T18:26:52.821 INFO:tasks.workunit.client.0.smithi042.stdout:0/175: dread d4/fe [4194304,4194304] 0 2024-02-15T18:26:52.821 INFO:tasks.workunit.client.0.smithi042.stdout:6/209: rename d9/fa to d9/d11/d19/d36/f43 0 2024-02-15T18:26:52.823 INFO:tasks.workunit.client.0.smithi042.stdout:0/176: link d4/d16/d2e/f2d d4/d9/d15/f42 0 2024-02-15T18:26:52.825 INFO:tasks.workunit.client.0.smithi042.stdout:0/177: rename d4/d9/cd to d4/d9/c43 0 2024-02-15T18:26:52.826 INFO:tasks.workunit.client.0.smithi042.stdout:8/80: dwrite fb [0,4194304] 0 2024-02-15T18:26:52.826 INFO:tasks.workunit.client.0.smithi042.stdout:8/81: write dd/f13 [832824,41827] 0 2024-02-15T18:26:52.826 INFO:tasks.workunit.client.0.smithi042.stdout:8/82: write dd/d11/f15 [454789,108351] 0 2024-02-15T18:26:52.826 INFO:tasks.workunit.client.0.smithi042.stdout:0/178: truncate d4/d16/d2e/f2d 204868 0 2024-02-15T18:26:52.834 INFO:tasks.workunit.client.0.smithi042.stdout:0/179: dread d4/fe [4194304,4194304] 0 2024-02-15T18:26:52.836 INFO:tasks.workunit.client.0.smithi042.stdout:0/180: mknod d4/d38/d39/c44 0 2024-02-15T18:26:52.837 INFO:tasks.workunit.client.0.smithi042.stdout:0/181: symlink d4/d16/d2e/l45 0 2024-02-15T18:26:52.839 INFO:tasks.workunit.client.0.smithi042.stdout:8/83: dread fb [0,4194304] 0 2024-02-15T18:26:52.840 INFO:tasks.workunit.client.0.smithi042.stdout:2/128: dwrite d0/f19 [0,4194304] 0 2024-02-15T18:26:52.840 INFO:tasks.workunit.client.0.smithi042.stdout:1/170: dwrite de/d15/d2e/f1d [4194304,4194304] 0 2024-02-15T18:26:52.841 INFO:tasks.workunit.client.0.smithi042.stdout:8/84: symlink dd/l16 0 2024-02-15T18:26:52.841 INFO:tasks.workunit.client.0.smithi042.stdout:8/85: chown dd/c14 0 1 2024-02-15T18:26:52.841 INFO:tasks.workunit.client.0.smithi042.stdout:8/86: write fc [4262618,81553] 0 2024-02-15T18:26:52.842 INFO:tasks.workunit.client.0.smithi042.stdout:5/161: dwrite d0/fa [0,4194304] 0 2024-02-15T18:26:52.843 INFO:tasks.workunit.client.0.smithi042.stdout:2/129: symlink d0/d8/d1b/l26 0 2024-02-15T18:26:52.843 INFO:tasks.workunit.client.0.smithi042.stdout:2/130: fsync d0/d8/d10/f17 0 2024-02-15T18:26:52.843 INFO:tasks.workunit.client.0.smithi042.stdout:8/87: fdatasync dd/f13 0 2024-02-15T18:26:52.844 INFO:tasks.workunit.client.0.smithi042.stdout:6/210: dwrite d9/d11/d19/f37 [0,4194304] 0 2024-02-15T18:26:52.844 INFO:tasks.workunit.client.0.smithi042.stdout:6/211: stat d9/f3c 0 2024-02-15T18:26:52.844 INFO:tasks.workunit.client.0.smithi042.stdout:0/182: dread d4/d9/d15/f18 [0,4194304] 0 2024-02-15T18:26:52.844 INFO:tasks.workunit.client.0.smithi042.stdout:1/171: symlink de/d15/d2c/l3c 0 2024-02-15T18:26:52.845 INFO:tasks.workunit.client.0.smithi042.stdout:6/212: read d9/d3e/f3b [1058926,120709] 0 2024-02-15T18:26:52.852 INFO:tasks.workunit.client.0.smithi042.stdout:7/178: rmdir d10/d12 39 2024-02-15T18:26:52.852 INFO:tasks.workunit.client.0.smithi042.stdout:7/179: truncate d10/d12/d23/f35 676276 0 2024-02-15T18:26:52.853 INFO:tasks.workunit.client.0.smithi042.stdout:3/200: rmdir d1/de 39 2024-02-15T18:26:52.859 INFO:tasks.workunit.client.0.smithi042.stdout:6/213: dwrite d9/f40 [0,4194304] 0 2024-02-15T18:26:52.860 INFO:tasks.workunit.client.0.smithi042.stdout:9/138: rmdir d6 39 2024-02-15T18:26:52.861 INFO:tasks.workunit.client.0.smithi042.stdout:5/162: creat d0/d14/f3e x:0 0 0 2024-02-15T18:26:52.861 INFO:tasks.workunit.client.0.smithi042.stdout:5/163: fdatasync d0/f3 0 2024-02-15T18:26:52.863 INFO:tasks.workunit.client.0.smithi042.stdout:2/131: truncate d0/d8/f11 1865336 0 2024-02-15T18:26:52.863 INFO:tasks.workunit.client.0.smithi042.stdout:5/164: dread d0/db/f10 [0,4194304] 0 2024-02-15T18:26:52.865 INFO:tasks.workunit.client.0.smithi042.stdout:8/88: rmdir dd/d11 39 2024-02-15T18:26:52.872 INFO:tasks.workunit.client.0.smithi042.stdout:6/214: dread d9/d11/f2b [0,4194304] 0 2024-02-15T18:26:52.872 INFO:tasks.workunit.client.0.smithi042.stdout:0/183: creat d4/d9/f46 x:0 0 0 2024-02-15T18:26:52.877 INFO:tasks.workunit.client.0.smithi042.stdout:1/172: getdents de 0 2024-02-15T18:26:52.878 INFO:tasks.workunit.client.0.smithi042.stdout:7/180: rename fd to d10/d15/d39/f41 0 2024-02-15T18:26:52.878 INFO:tasks.workunit.client.0.smithi042.stdout:3/201: unlink d1/d11/d2e/l41 0 2024-02-15T18:26:52.883 INFO:tasks.workunit.client.0.smithi042.stdout:5/165: dread d0/d14/f29 [0,4194304] 0 2024-02-15T18:26:52.887 INFO:tasks.workunit.client.0.smithi042.stdout:0/184: dwrite d4/d16/d2e/f2d [0,4194304] 0 2024-02-15T18:26:52.887 INFO:tasks.workunit.client.0.smithi042.stdout:0/185: dread - d4/d38/f3c zero size 2024-02-15T18:26:52.913 INFO:tasks.workunit.client.0.smithi042.stdout:2/132: getdents d0/d8/d10 0 2024-02-15T18:26:52.914 INFO:tasks.workunit.client.0.smithi042.stdout:8/89: dwrite fc [0,4194304] 0 2024-02-15T18:26:52.914 INFO:tasks.workunit.client.0.smithi042.stdout:8/90: chown fc 1717756 1 2024-02-15T18:26:52.914 INFO:tasks.workunit.client.0.smithi042.stdout:5/166: dread d0/f3 [0,4194304] 0 2024-02-15T18:26:52.919 INFO:tasks.workunit.client.0.smithi042.stdout:1/173: mkdir de/d15/d3d 0 2024-02-15T18:26:52.920 INFO:tasks.workunit.client.0.smithi042.stdout:7/181: creat d10/d12/d23/f42 x:0 0 0 2024-02-15T18:26:52.922 INFO:tasks.workunit.client.0.smithi042.stdout:3/202: creat d1/de/d19/f44 x:0 0 0 2024-02-15T18:26:52.929 INFO:tasks.workunit.client.0.smithi042.stdout:9/139: creat d6/f31 x:0 0 0 2024-02-15T18:26:52.930 INFO:tasks.workunit.client.0.smithi042.stdout:0/186: mknod d4/d38/c47 0 2024-02-15T18:26:52.931 INFO:tasks.workunit.client.0.smithi042.stdout:5/167: rename d0/db/l2b to d0/d12/l3f 0 2024-02-15T18:26:52.931 INFO:tasks.workunit.client.0.smithi042.stdout:2/133: dwrite d0/f3 [4194304,4194304] 0 2024-02-15T18:26:52.932 INFO:tasks.workunit.client.0.smithi042.stdout:3/203: truncate d1/de/f10 102781 0 2024-02-15T18:26:52.934 INFO:tasks.workunit.client.0.smithi042.stdout:9/140: getdents d6/dd 0 2024-02-15T18:26:52.934 INFO:tasks.workunit.client.0.smithi042.stdout:2/134: read d0/d8/f11 [801520,35766] 0 2024-02-15T18:26:52.934 INFO:tasks.workunit.client.0.smithi042.stdout:0/187: symlink d4/l48 0 2024-02-15T18:26:52.935 INFO:tasks.workunit.client.0.smithi042.stdout:0/188: chown d4/d16 0 1 2024-02-15T18:26:52.935 INFO:tasks.workunit.client.0.smithi042.stdout:3/204: truncate d1/f15 303092 0 2024-02-15T18:26:52.937 INFO:tasks.workunit.client.0.smithi042.stdout:0/189: write d4/d9/d15/f42 [1872689,122687] 0 2024-02-15T18:26:52.937 INFO:tasks.workunit.client.0.smithi042.stdout:9/141: unlink d6/dd/d9/c1a 0 2024-02-15T18:26:52.938 INFO:tasks.workunit.client.0.smithi042.stdout:2/135: symlink d0/d8/db/l27 0 2024-02-15T18:26:52.938 INFO:tasks.workunit.client.0.smithi042.stdout:2/136: dread - d0/d8/d10/f17 zero size 2024-02-15T18:26:52.939 INFO:tasks.workunit.client.0.smithi042.stdout:9/142: read d6/da/f12 [72588,21603] 0 2024-02-15T18:26:52.939 INFO:tasks.workunit.client.0.smithi042.stdout:9/143: write d6/da/f12 [929495,24647] 0 2024-02-15T18:26:52.939 INFO:tasks.workunit.client.0.smithi042.stdout:5/168: dread d0/d14/f16 [0,4194304] 0 2024-02-15T18:26:52.941 INFO:tasks.workunit.client.0.smithi042.stdout:0/190: unlink d4/c35 0 2024-02-15T18:26:52.941 INFO:tasks.workunit.client.0.smithi042.stdout:0/191: chown d4/d9/d34/l3f 25362069 1 2024-02-15T18:26:52.941 INFO:tasks.workunit.client.0.smithi042.stdout:0/192: fsync d4/d9/f46 0 2024-02-15T18:26:52.941 INFO:tasks.workunit.client.0.smithi042.stdout:0/193: chown d4/d9/d15/c2f 28210 1 2024-02-15T18:26:52.941 INFO:tasks.workunit.client.0.smithi042.stdout:0/194: truncate d4/d38/f3c 379463 0 2024-02-15T18:26:52.941 INFO:tasks.workunit.client.0.smithi042.stdout:0/195: chown d4/d9/d15/c2f 1 1 2024-02-15T18:26:52.941 INFO:tasks.workunit.client.0.smithi042.stdout:0/196: chown d4/l11 456 1 2024-02-15T18:26:52.941 INFO:tasks.workunit.client.0.smithi042.stdout:2/137: creat d0/d8/f28 x:0 0 0 2024-02-15T18:26:52.943 INFO:tasks.workunit.client.0.smithi042.stdout:0/197: dread d4/d9/d15/f21 [0,4194304] 0 2024-02-15T18:26:52.943 INFO:tasks.workunit.client.0.smithi042.stdout:9/144: creat d6/dd/d9/f32 x:0 0 0 2024-02-15T18:26:52.944 INFO:tasks.workunit.client.0.smithi042.stdout:8/91: dwrite fb [0,4194304] 0 2024-02-15T18:26:52.944 INFO:tasks.workunit.client.0.smithi042.stdout:8/92: readlink l7 0 2024-02-15T18:26:52.945 INFO:tasks.workunit.client.0.smithi042.stdout:7/182: dwrite d10/f26 [0,4194304] 0 2024-02-15T18:26:52.946 INFO:tasks.workunit.client.0.smithi042.stdout:1/174: dwrite de/d15/d1b/f25 [0,4194304] 0 2024-02-15T18:26:52.946 INFO:tasks.workunit.client.0.smithi042.stdout:5/169: creat d0/d22/d2d/f40 x:0 0 0 2024-02-15T18:26:52.950 INFO:tasks.workunit.client.0.smithi042.stdout:2/138: mknod d0/d8/db/de/d1a/c29 0 2024-02-15T18:26:52.957 INFO:tasks.workunit.client.0.smithi042.stdout:0/198: creat d4/d38/f49 x:0 0 0 2024-02-15T18:26:52.960 INFO:tasks.workunit.client.0.smithi042.stdout:9/145: dwrite f5 [0,4194304] 0 2024-02-15T18:26:52.960 INFO:tasks.workunit.client.0.smithi042.stdout:9/146: dread - d6/f2e zero size 2024-02-15T18:26:52.964 INFO:tasks.workunit.client.0.smithi042.stdout:8/93: unlink c2 0 2024-02-15T18:26:52.966 INFO:tasks.workunit.client.0.smithi042.stdout:8/94: dread dd/d11/f15 [0,4194304] 0 2024-02-15T18:26:52.971 INFO:tasks.workunit.client.0.smithi042.stdout:4/132: sync 2024-02-15T18:26:52.971 INFO:tasks.workunit.client.0.smithi042.stdout:4/133: fsync da/d11/fe 0 2024-02-15T18:26:52.977 INFO:tasks.workunit.client.0.smithi042.stdout:1/175: chown c3 282022 1 2024-02-15T18:26:52.977 INFO:tasks.workunit.client.0.smithi042.stdout:6/215: sync 2024-02-15T18:26:52.979 INFO:tasks.workunit.client.0.smithi042.stdout:6/216: dread d9/d11/d19/d1c/d31/f3a [0,4194304] 0 2024-02-15T18:26:52.979 INFO:tasks.workunit.client.0.smithi042.stdout:5/170: mkdir d0/db/d41 0 2024-02-15T18:26:52.982 INFO:tasks.workunit.client.0.smithi042.stdout:2/139: rename d0/d8/c24 to d0/d8/db/de/c2a 0 2024-02-15T18:26:52.985 INFO:tasks.workunit.client.0.smithi042.stdout:0/199: unlink d4/d16/d2e/f2d 0 2024-02-15T18:26:52.985 INFO:tasks.workunit.client.0.smithi042.stdout:0/200: read - d4/d9/f31 zero size 2024-02-15T18:26:52.988 INFO:tasks.workunit.client.0.smithi042.stdout:9/147: unlink d6/dd/d9/f1f 0 2024-02-15T18:26:52.988 INFO:tasks.workunit.client.0.smithi042.stdout:8/95: creat dd/f17 x:0 0 0 2024-02-15T18:26:52.990 INFO:tasks.workunit.client.0.smithi042.stdout:4/134: rmdir da/d11/d27 39 2024-02-15T18:26:52.992 INFO:tasks.workunit.client.0.smithi042.stdout:1/176: mknod de/d15/d1b/c3e 0 2024-02-15T18:26:52.994 INFO:tasks.workunit.client.0.smithi042.stdout:5/171: mkdir d0/d22/d2d/d42 0 2024-02-15T18:26:52.997 INFO:tasks.workunit.client.0.smithi042.stdout:4/135: creat da/f2a x:0 0 0 2024-02-15T18:26:52.998 INFO:tasks.workunit.client.0.smithi042.stdout:6/217: dwrite d9/f21 [0,4194304] 0 2024-02-15T18:26:52.998 INFO:tasks.workunit.client.0.smithi042.stdout:8/96: dread fc [0,4194304] 0 2024-02-15T18:26:52.998 INFO:tasks.workunit.client.0.smithi042.stdout:6/218: truncate d9/d11/d19/d1c/f27 1260138 0 2024-02-15T18:26:52.999 INFO:tasks.workunit.client.0.smithi042.stdout:0/201: write d4/d9/d15/f42 [3982563,64594] 0 2024-02-15T18:26:52.999 INFO:tasks.workunit.client.0.smithi042.stdout:1/177: getdents de/d15/d2e 0 2024-02-15T18:26:52.999 INFO:tasks.workunit.client.0.smithi042.stdout:0/202: chown d4/d9/d15/l1e 0 1 2024-02-15T18:26:52.999 INFO:tasks.workunit.client.0.smithi042.stdout:0/203: truncate d4/f3b 730574 0 2024-02-15T18:26:53.001 INFO:tasks.workunit.client.0.smithi042.stdout:6/219: write d9/d11/d19/f37 [766320,45672] 0 2024-02-15T18:26:53.002 INFO:tasks.workunit.client.0.smithi042.stdout:5/172: rename d0/db/c13 to d0/d22/d2d/c43 0 2024-02-15T18:26:53.002 INFO:tasks.workunit.client.0.smithi042.stdout:5/173: readlink d0/d22/l2f 0 2024-02-15T18:26:53.003 INFO:tasks.workunit.client.0.smithi042.stdout:0/204: read d4/d9/d15/f18 [2562712,104174] 0 2024-02-15T18:26:53.003 INFO:tasks.workunit.client.0.smithi042.stdout:4/136: mkdir da/d2b 0 2024-02-15T18:26:53.003 INFO:tasks.workunit.client.0.smithi042.stdout:4/137: write da/f26 [389029,13520] 0 2024-02-15T18:26:53.009 INFO:tasks.workunit.client.0.smithi042.stdout:2/140: dwrite d0/d8/f28 [0,4194304] 0 2024-02-15T18:26:53.010 INFO:tasks.workunit.client.0.smithi042.stdout:9/148: dwrite d6/dd/d9/f32 [0,4194304] 0 2024-02-15T18:26:53.012 INFO:tasks.workunit.client.0.smithi042.stdout:8/97: rmdir dd 39 2024-02-15T18:26:53.012 INFO:tasks.workunit.client.0.smithi042.stdout:8/98: write dd/d11/f15 [945476,19879] 0 2024-02-15T18:26:53.016 INFO:tasks.workunit.client.0.smithi042.stdout:1/178: creat de/d15/d2e/f3f x:0 0 0 2024-02-15T18:26:53.019 INFO:tasks.workunit.client.0.smithi042.stdout:5/174: symlink d0/d22/d2d/d42/l44 0 2024-02-15T18:26:53.019 INFO:tasks.workunit.client.0.smithi042.stdout:8/99: dread dd/d11/f15 [0,4194304] 0 2024-02-15T18:26:53.020 INFO:tasks.workunit.client.0.smithi042.stdout:0/205: rename d4/l48 to d4/d38/d39/l4a 0 2024-02-15T18:26:53.021 INFO:tasks.workunit.client.0.smithi042.stdout:2/141: dread d0/f3 [0,4194304] 0 2024-02-15T18:26:53.022 INFO:tasks.workunit.client.0.smithi042.stdout:6/220: dread d9/d11/d19/f37 [0,4194304] 0 2024-02-15T18:26:53.025 INFO:tasks.workunit.client.0.smithi042.stdout:5/175: write d0/fa [913976,38161] 0 2024-02-15T18:26:53.025 INFO:tasks.workunit.client.0.smithi042.stdout:5/176: write d0/d14/f28 [419629,93348] 0 2024-02-15T18:26:53.037 INFO:tasks.workunit.client.0.smithi042.stdout:9/149: symlink d6/da/d2f/l33 0 2024-02-15T18:26:53.042 INFO:tasks.workunit.client.0.smithi042.stdout:1/179: symlink de/d15/d1b/l40 0 2024-02-15T18:26:53.043 INFO:tasks.workunit.client.0.smithi042.stdout:7/183: sync 2024-02-15T18:26:53.043 INFO:tasks.workunit.client.0.smithi042.stdout:3/205: sync 2024-02-15T18:26:53.043 INFO:tasks.workunit.client.0.smithi042.stdout:7/184: fsync d10/d15/f1d 0 2024-02-15T18:26:53.043 INFO:tasks.workunit.client.0.smithi042.stdout:8/100: dwrite dd/d11/f15 [0,4194304] 0 2024-02-15T18:26:53.043 INFO:tasks.workunit.client.0.smithi042.stdout:7/185: write d10/d12/f3f [754810,54283] 0 2024-02-15T18:26:53.043 INFO:tasks.workunit.client.0.smithi042.stdout:8/101: write fc [8355200,89524] 0 2024-02-15T18:26:53.043 INFO:tasks.workunit.client.0.smithi042.stdout:0/206: truncate d4/d16/f24 1244975 0 2024-02-15T18:26:53.043 INFO:tasks.workunit.client.0.smithi042.stdout:0/207: chown d4/d38 6847 1 2024-02-15T18:26:53.047 INFO:tasks.workunit.client.0.smithi042.stdout:2/142: creat d0/d8/db/de/d1a/f2b x:0 0 0 2024-02-15T18:26:53.053 INFO:tasks.workunit.client.0.smithi042.stdout:5/177: symlink d0/d22/l45 0 2024-02-15T18:26:53.054 INFO:tasks.workunit.client.0.smithi042.stdout:1/180: unlink de/d15/c1c 0 2024-02-15T18:26:53.057 INFO:tasks.workunit.client.0.smithi042.stdout:3/206: creat d1/d11/f45 x:0 0 0 2024-02-15T18:26:53.058 INFO:tasks.workunit.client.0.smithi042.stdout:3/207: dread d1/de/f10 [0,4194304] 0 2024-02-15T18:26:53.060 INFO:tasks.workunit.client.0.smithi042.stdout:8/102: symlink dd/d11/l18 0 2024-02-15T18:26:53.060 INFO:tasks.workunit.client.0.smithi042.stdout:8/103: write dd/f17 [364502,25083] 0 2024-02-15T18:26:53.061 INFO:tasks.workunit.client.0.smithi042.stdout:8/104: chown dd/de 7 1 2024-02-15T18:26:53.062 INFO:tasks.workunit.client.0.smithi042.stdout:0/208: link d4/d38/f3c d4/d9/d34/f4b 0 2024-02-15T18:26:53.062 INFO:tasks.workunit.client.0.smithi042.stdout:0/209: write d4/d38/f3a [442387,60887] 0 2024-02-15T18:26:53.064 INFO:tasks.workunit.client.0.smithi042.stdout:5/178: getdents d0/d12/d2a 0 2024-02-15T18:26:53.066 INFO:tasks.workunit.client.0.smithi042.stdout:9/150: dwrite d6/f31 [0,4194304] 0 2024-02-15T18:26:53.066 INFO:tasks.workunit.client.0.smithi042.stdout:9/151: fdatasync d6/da/f29 0 2024-02-15T18:26:53.066 INFO:tasks.workunit.client.0.smithi042.stdout:1/181: creat de/d15/d2c/f41 x:0 0 0 2024-02-15T18:26:53.067 INFO:tasks.workunit.client.0.smithi042.stdout:8/105: mknod dd/c19 0 2024-02-15T18:26:53.067 INFO:tasks.workunit.client.0.smithi042.stdout:8/106: chown ca 20538 1 2024-02-15T18:26:53.070 INFO:tasks.workunit.client.0.smithi042.stdout:5/179: symlink d0/d22/l46 0 2024-02-15T18:26:53.073 INFO:tasks.workunit.client.0.smithi042.stdout:7/186: dwrite d10/f22 [0,4194304] 0 2024-02-15T18:26:53.073 INFO:tasks.workunit.client.0.smithi042.stdout:9/152: symlink d6/l34 0 2024-02-15T18:26:53.075 INFO:tasks.workunit.client.0.smithi042.stdout:2/143: dwrite d0/f19 [0,4194304] 0 2024-02-15T18:26:53.075 INFO:tasks.workunit.client.0.smithi042.stdout:9/153: write d6/dd/d9/f32 [2141598,98952] 0 2024-02-15T18:26:53.075 INFO:tasks.workunit.client.0.smithi042.stdout:9/154: dread - d6/da/f29 zero size 2024-02-15T18:26:53.075 INFO:tasks.workunit.client.0.smithi042.stdout:8/107: creat dd/de/f1a x:0 0 0 2024-02-15T18:26:53.080 INFO:tasks.workunit.client.0.smithi042.stdout:3/208: dwrite d1/f34 [0,4194304] 0 2024-02-15T18:26:53.086 INFO:tasks.workunit.client.0.smithi042.stdout:8/108: dread dd/de/ff [0,4194304] 0 2024-02-15T18:26:53.087 INFO:tasks.workunit.client.0.smithi042.stdout:0/210: mkdir d4/d16/d2e/d4c 0 2024-02-15T18:26:53.088 INFO:tasks.workunit.client.0.smithi042.stdout:0/211: read d4/f6 [1259,35370] 0 2024-02-15T18:26:53.088 INFO:tasks.workunit.client.0.smithi042.stdout:0/212: read - d4/d38/f49 zero size 2024-02-15T18:26:53.088 INFO:tasks.workunit.client.0.smithi042.stdout:5/180: creat d0/f47 x:0 0 0 2024-02-15T18:26:53.094 INFO:tasks.workunit.client.0.smithi042.stdout:7/187: mkdir d10/d43 0 2024-02-15T18:26:53.094 INFO:tasks.workunit.client.0.smithi042.stdout:7/188: readlink d10/l11 0 2024-02-15T18:26:53.094 INFO:tasks.workunit.client.0.smithi042.stdout:7/189: write d10/d12/d23/f42 [680899,127853] 0 2024-02-15T18:26:53.095 INFO:tasks.workunit.client.0.smithi042.stdout:2/144: rename d0/l1d to d0/d8/db/de/l2c 0 2024-02-15T18:26:53.096 INFO:tasks.workunit.client.0.smithi042.stdout:5/181: dread d0/fe [0,4194304] 0 2024-02-15T18:26:53.096 INFO:tasks.workunit.client.0.smithi042.stdout:5/182: dread - d0/f34 zero size 2024-02-15T18:26:53.107 INFO:tasks.workunit.client.0.smithi042.stdout:9/155: mknod d6/da/c35 0 2024-02-15T18:26:53.111 INFO:tasks.workunit.client.0.smithi042.stdout:3/209: rename d1/de/f27 to d1/de/d1a/f46 0 2024-02-15T18:26:53.113 INFO:tasks.workunit.client.0.smithi042.stdout:9/156: symlink d6/da/d1d/d2d/l36 0 2024-02-15T18:26:53.117 INFO:tasks.workunit.client.0.smithi042.stdout:5/183: rename d0/c31 to d0/d22/c48 0 2024-02-15T18:26:53.117 INFO:tasks.workunit.client.0.smithi042.stdout:9/157: dread d6/dd/d9/f14 [0,4194304] 0 2024-02-15T18:26:53.117 INFO:tasks.workunit.client.0.smithi042.stdout:3/210: creat d1/d11/d14/f47 x:0 0 0 2024-02-15T18:26:53.117 INFO:tasks.workunit.client.0.smithi042.stdout:5/184: write d0/db/f1f [5197169,67008] 0 2024-02-15T18:26:53.117 INFO:tasks.workunit.client.0.smithi042.stdout:9/158: read d6/da/f12 [203232,18740] 0 2024-02-15T18:26:53.117 INFO:tasks.workunit.client.0.smithi042.stdout:9/159: truncate d6/dd/d9/f16 699459 0 2024-02-15T18:26:53.117 INFO:tasks.workunit.client.0.smithi042.stdout:7/190: dwrite d10/d12/f3f [0,4194304] 0 2024-02-15T18:26:53.118 INFO:tasks.workunit.client.0.smithi042.stdout:2/145: dwrite d0/d8/f11 [0,4194304] 0 2024-02-15T18:26:53.118 INFO:tasks.workunit.client.0.smithi042.stdout:7/191: write d10/d15/f34 [807814,72456] 0 2024-02-15T18:26:53.120 INFO:tasks.workunit.client.0.smithi042.stdout:2/146: write d0/d8/f28 [229121,91184] 0 2024-02-15T18:26:53.121 INFO:tasks.workunit.client.0.smithi042.stdout:8/109: rmdir dd/de 39 2024-02-15T18:26:53.121 INFO:tasks.workunit.client.0.smithi042.stdout:5/185: creat d0/d22/d2d/d42/f49 x:0 0 0 2024-02-15T18:26:53.131 INFO:tasks.workunit.client.0.smithi042.stdout:9/160: mknod d6/da/d1d/d2d/c37 0 2024-02-15T18:26:53.133 INFO:tasks.workunit.client.0.smithi042.stdout:4/138: sync 2024-02-15T18:26:53.134 INFO:tasks.workunit.client.0.smithi042.stdout:3/211: dwrite d1/de/f42 [0,4194304] 0 2024-02-15T18:26:53.158 INFO:tasks.workunit.client.0.smithi042.stdout:5/186: dwrite d0/d12/f21 [0,4194304] 0 2024-02-15T18:26:53.159 INFO:tasks.workunit.client.0.smithi042.stdout:4/139: dwrite da/f25 [0,4194304] 0 2024-02-15T18:26:53.160 INFO:tasks.workunit.client.0.smithi042.stdout:4/140: dread da/d11/f1d [0,4194304] 0 2024-02-15T18:26:53.160 INFO:tasks.workunit.client.0.smithi042.stdout:4/141: read - da/d11/f20 zero size 2024-02-15T18:26:53.160 INFO:tasks.workunit.client.0.smithi042.stdout:7/192: dwrite d10/d15/f34 [0,4194304] 0 2024-02-15T18:26:53.167 INFO:tasks.workunit.client.0.smithi042.stdout:3/212: dwrite d1/d11/d2e/f37 [0,4194304] 0 2024-02-15T18:26:53.168 INFO:tasks.workunit.client.0.smithi042.stdout:3/213: getdents d1 0 2024-02-15T18:26:53.168 INFO:tasks.workunit.client.0.smithi042.stdout:3/214: chown d1/de/d19/f44 8233631 1 2024-02-15T18:26:53.225 INFO:tasks.workunit.client.0.smithi042.stdout:8/110: mknod dd/de/c1b 0 2024-02-15T18:26:53.227 INFO:tasks.workunit.client.0.smithi042.stdout:8/111: dread dd/f17 [0,4194304] 0 2024-02-15T18:26:53.227 INFO:tasks.workunit.client.0.smithi042.stdout:9/161: creat d6/da/d1d/f38 x:0 0 0 2024-02-15T18:26:53.227 INFO:tasks.workunit.client.0.smithi042.stdout:9/162: read d6/dd/f20 [3114696,47634] 0 2024-02-15T18:26:53.235 INFO:tasks.workunit.client.0.smithi042.stdout:5/187: creat d0/d22/f4a x:0 0 0 2024-02-15T18:26:53.236 INFO:tasks.workunit.client.0.smithi042.stdout:5/188: dread - d0/d22/d2d/d42/f49 zero size 2024-02-15T18:26:53.239 INFO:tasks.workunit.client.0.smithi042.stdout:4/142: creat da/d11/d29/f2c x:0 0 0 2024-02-15T18:26:53.239 INFO:tasks.workunit.client.0.smithi042.stdout:4/143: write da/d11/f23 [31871,20519] 0 2024-02-15T18:26:53.241 INFO:tasks.workunit.client.0.smithi042.stdout:7/193: truncate d10/f1b 1311568 0 2024-02-15T18:26:53.241 INFO:tasks.workunit.client.0.smithi042.stdout:7/194: fsync d10/d15/d39/f41 0 2024-02-15T18:26:53.252 INFO:tasks.workunit.client.0.smithi042.stdout:3/215: creat d1/de/d19/f48 x:0 0 0 2024-02-15T18:26:53.252 INFO:tasks.workunit.client.0.smithi042.stdout:3/216: readlink d1/d11/l38 0 2024-02-15T18:26:53.252 INFO:tasks.workunit.client.0.smithi042.stdout:3/217: readlink d1/de/l3f 0 2024-02-15T18:26:53.252 INFO:tasks.workunit.client.0.smithi042.stdout:3/218: readlink d1/de/d19/l1b 0 2024-02-15T18:26:53.252 INFO:tasks.workunit.client.0.smithi042.stdout:3/219: dread - d1/de/d1a/f2f zero size 2024-02-15T18:26:53.252 INFO:tasks.workunit.client.0.smithi042.stdout:3/220: dread - d1/f32 zero size 2024-02-15T18:26:53.253 INFO:tasks.workunit.client.0.smithi042.stdout:2/147: truncate d0/f19 3745705 0 2024-02-15T18:26:53.254 INFO:tasks.workunit.client.0.smithi042.stdout:2/148: read d0/d8/db/de/f14 [223041,40936] 0 2024-02-15T18:26:53.255 INFO:tasks.workunit.client.0.smithi042.stdout:8/112: mknod dd/de/c1c 0 2024-02-15T18:26:53.257 INFO:tasks.workunit.client.0.smithi042.stdout:9/163: creat d6/da/d1d/d2d/f39 x:0 0 0 2024-02-15T18:26:53.257 INFO:tasks.workunit.client.0.smithi042.stdout:9/164: chown d6/da/d2f/l33 2780446 1 2024-02-15T18:26:53.258 INFO:tasks.workunit.client.0.smithi042.stdout:5/189: mknod d0/d38/c4b 0 2024-02-15T18:26:53.258 INFO:tasks.workunit.client.0.smithi042.stdout:5/190: chown d0/db/d41 33210004 1 2024-02-15T18:26:53.260 INFO:tasks.workunit.client.0.smithi042.stdout:4/144: mknod da/c2d 0 2024-02-15T18:26:53.264 INFO:tasks.workunit.client.0.smithi042.stdout:7/195: unlink d10/d15/d39/f41 0 2024-02-15T18:26:53.264 INFO:tasks.workunit.client.0.smithi042.stdout:1/182: sync 2024-02-15T18:26:53.264 INFO:tasks.workunit.client.0.smithi042.stdout:6/221: sync 2024-02-15T18:26:53.264 INFO:tasks.workunit.client.0.smithi042.stdout:0/213: sync 2024-02-15T18:26:53.265 INFO:tasks.workunit.client.0.smithi042.stdout:0/214: read d4/f6 [529927,102582] 0 2024-02-15T18:26:53.265 INFO:tasks.workunit.client.0.smithi042.stdout:3/221: getdents d1/d11/d14 0 2024-02-15T18:26:53.268 INFO:tasks.workunit.client.0.smithi042.stdout:2/149: symlink d0/d8/d20/l2d 0 2024-02-15T18:26:53.270 INFO:tasks.workunit.client.0.smithi042.stdout:8/113: rename ca to dd/c1d 0 2024-02-15T18:26:53.270 INFO:tasks.workunit.client.0.smithi042.stdout:8/114: stat dd/l16 0 2024-02-15T18:26:53.272 INFO:tasks.workunit.client.0.smithi042.stdout:8/115: dread dd/f13 [0,4194304] 0 2024-02-15T18:26:53.272 INFO:tasks.workunit.client.0.smithi042.stdout:8/116: chown dd/d11 2848294 1 2024-02-15T18:26:53.272 INFO:tasks.workunit.client.0.smithi042.stdout:9/165: mknod d6/da/d1d/d2d/c3a 0 2024-02-15T18:26:53.274 INFO:tasks.workunit.client.0.smithi042.stdout:5/191: creat d0/db/d41/f4c x:0 0 0 2024-02-15T18:26:53.275 INFO:tasks.workunit.client.0.smithi042.stdout:5/192: write d0/d14/f29 [874880,126573] 0 2024-02-15T18:26:53.276 INFO:tasks.workunit.client.0.smithi042.stdout:5/193: fdatasync d0/d14/f2e 0 2024-02-15T18:26:53.279 INFO:tasks.workunit.client.0.smithi042.stdout:7/196: dread d10/d15/f1d [0,4194304] 0 2024-02-15T18:26:53.279 INFO:tasks.workunit.client.0.smithi042.stdout:7/197: dread - d10/d12/d23/f27 zero size 2024-02-15T18:26:53.279 INFO:tasks.workunit.client.0.smithi042.stdout:1/183: unlink de/d15/d1b/l40 0 2024-02-15T18:26:53.279 INFO:tasks.workunit.client.0.smithi042.stdout:1/184: write de/f21 [2893009,116006] 0 2024-02-15T18:26:53.279 INFO:tasks.workunit.client.0.smithi042.stdout:6/222: symlink d9/d11/d19/d35/l44 0 2024-02-15T18:26:53.280 INFO:tasks.workunit.client.0.smithi042.stdout:3/222: symlink d1/d11/d14/l49 0 2024-02-15T18:26:53.280 INFO:tasks.workunit.client.0.smithi042.stdout:3/223: chown d1/de/d19/c40 155808958 1 2024-02-15T18:26:53.282 INFO:tasks.workunit.client.0.smithi042.stdout:2/150: mkdir d0/d8/db/de/d1a/d2e 0 2024-02-15T18:26:53.283 INFO:tasks.workunit.client.0.smithi042.stdout:4/145: dwrite da/f2a [0,4194304] 0 2024-02-15T18:26:53.283 INFO:tasks.workunit.client.0.smithi042.stdout:9/166: symlink d6/da/l3b 0 2024-02-15T18:26:53.284 INFO:tasks.workunit.client.0.smithi042.stdout:9/167: read d6/dd/d9/f32 [3882472,88043] 0 2024-02-15T18:26:53.285 INFO:tasks.workunit.client.0.smithi042.stdout:8/117: dread dd/de/f10 [0,4194304] 0 2024-02-15T18:26:53.287 INFO:tasks.workunit.client.0.smithi042.stdout:5/194: rename d0/d14/f3e to d0/f4d 0 2024-02-15T18:26:53.291 INFO:tasks.workunit.client.0.smithi042.stdout:2/151: dread d0/f7 [4194304,4194304] 0 2024-02-15T18:26:53.298 INFO:tasks.workunit.client.0.smithi042.stdout:1/185: write de/d15/d2e/f1d [5567640,128650] 0 2024-02-15T18:26:53.299 INFO:tasks.workunit.client.0.smithi042.stdout:6/223: mknod d9/d11/d1d/c45 0 2024-02-15T18:26:53.299 INFO:tasks.workunit.client.0.smithi042.stdout:6/224: readlink d9/lb 0 2024-02-15T18:26:53.305 INFO:tasks.workunit.client.0.smithi042.stdout:4/146: symlink da/d11/l2e 0 2024-02-15T18:26:53.307 INFO:tasks.workunit.client.0.smithi042.stdout:9/168: mkdir d6/dd/d3c 0 2024-02-15T18:26:53.309 INFO:tasks.workunit.client.0.smithi042.stdout:9/169: dread d6/dd/d9/f16 [0,4194304] 0 2024-02-15T18:26:53.309 INFO:tasks.workunit.client.0.smithi042.stdout:9/170: chown d6/da/d1d/d2d/f39 1 1 2024-02-15T18:26:53.309 INFO:tasks.workunit.client.0.smithi042.stdout:8/118: creat dd/de/d12/f1e x:0 0 0 2024-02-15T18:26:53.309 INFO:tasks.workunit.client.0.smithi042.stdout:8/119: dread - dd/de/f1a zero size 2024-02-15T18:26:53.313 INFO:tasks.workunit.client.0.smithi042.stdout:5/195: mknod d0/db/d41/c4e 0 2024-02-15T18:26:53.315 INFO:tasks.workunit.client.0.smithi042.stdout:2/152: getdents d0/d8/d1b 0 2024-02-15T18:26:53.315 INFO:tasks.workunit.client.0.smithi042.stdout:7/198: dwrite d10/f22 [4194304,4194304] 0 2024-02-15T18:26:53.318 INFO:tasks.workunit.client.0.smithi042.stdout:1/186: mknod de/d15/d1b/c42 0 2024-02-15T18:26:53.319 INFO:tasks.workunit.client.0.smithi042.stdout:6/225: mkdir d9/d11/d46 0 2024-02-15T18:26:53.323 INFO:tasks.workunit.client.0.smithi042.stdout:3/224: mknod d1/d11/d14/c4a 0 2024-02-15T18:26:53.323 INFO:tasks.workunit.client.0.smithi042.stdout:9/171: creat d6/da/d2f/f3d x:0 0 0 2024-02-15T18:26:53.323 INFO:tasks.workunit.client.0.smithi042.stdout:8/120: link dd/c19 dd/d11/c1f 0 2024-02-15T18:26:53.323 INFO:tasks.workunit.client.0.smithi042.stdout:9/172: truncate d6/f2e 833304 0 2024-02-15T18:26:53.325 INFO:tasks.workunit.client.0.smithi042.stdout:4/147: dwrite da/d11/d1b/f1c [0,4194304] 0 2024-02-15T18:26:53.326 INFO:tasks.workunit.client.0.smithi042.stdout:5/196: rename d0/d22/c24 to d0/d39/c4f 0 2024-02-15T18:26:53.328 INFO:tasks.workunit.client.0.smithi042.stdout:2/153: mkdir d0/d8/d20/d2f 0 2024-02-15T18:26:53.329 INFO:tasks.workunit.client.0.smithi042.stdout:1/187: dwrite de/d15/d2c/f38 [0,4194304] 0 2024-02-15T18:26:53.332 INFO:tasks.workunit.client.0.smithi042.stdout:5/197: write d0/d14/f29 [3544386,13953] 0 2024-02-15T18:26:53.333 INFO:tasks.workunit.client.0.smithi042.stdout:5/198: dread d0/d22/d2d/f32 [0,4194304] 0 2024-02-15T18:26:53.333 INFO:tasks.workunit.client.0.smithi042.stdout:5/199: fsync d0/d22/f4a 0 2024-02-15T18:26:53.336 INFO:tasks.workunit.client.0.smithi042.stdout:7/199: mknod d10/d12/d23/c44 0 2024-02-15T18:26:53.336 INFO:tasks.workunit.client.0.smithi042.stdout:7/200: write d10/d15/f36 [13822,31630] 0 2024-02-15T18:26:53.336 INFO:tasks.workunit.client.0.smithi042.stdout:1/188: dread de/d15/d2c/f38 [0,4194304] 0 2024-02-15T18:26:53.338 INFO:tasks.workunit.client.0.smithi042.stdout:6/226: getdents d9/d11/d19/d1c/d31 0 2024-02-15T18:26:53.340 INFO:tasks.workunit.client.0.smithi042.stdout:6/227: read d9/d11/d19/d1c/d31/f3a [818556,87286] 0 2024-02-15T18:26:53.340 INFO:tasks.workunit.client.0.smithi042.stdout:6/228: truncate d9/d11/d19/d1c/d31/f3a 1740113 0 2024-02-15T18:26:53.340 INFO:tasks.workunit.client.0.smithi042.stdout:6/229: fdatasync d9/f40 0 2024-02-15T18:26:53.340 INFO:tasks.workunit.client.0.smithi042.stdout:6/230: readlink d9/d11/l23 0 2024-02-15T18:26:53.340 INFO:tasks.workunit.client.0.smithi042.stdout:6/231: fsync d9/f40 0 2024-02-15T18:26:53.343 INFO:tasks.workunit.client.0.smithi042.stdout:9/173: creat d6/da/f3e x:0 0 0 2024-02-15T18:26:53.343 INFO:tasks.workunit.client.0.smithi042.stdout:9/174: chown d6/dd/l26 51289 1 2024-02-15T18:26:53.346 INFO:tasks.workunit.client.0.smithi042.stdout:4/148: mkdir da/d11/d1b/d2f 0 2024-02-15T18:26:53.348 INFO:tasks.workunit.client.0.smithi042.stdout:2/154: unlink d0/d8/db/l27 0 2024-02-15T18:26:53.348 INFO:tasks.workunit.client.0.smithi042.stdout:2/155: write d0/d8/db/de/f18 [70900,91415] 0 2024-02-15T18:26:53.350 INFO:tasks.workunit.client.0.smithi042.stdout:9/175: dread f2 [0,4194304] 0 2024-02-15T18:26:53.351 INFO:tasks.workunit.client.0.smithi042.stdout:8/121: dwrite dd/de/f1a [0,4194304] 0 2024-02-15T18:26:53.352 INFO:tasks.workunit.client.0.smithi042.stdout:5/200: mkdir d0/d50 0 2024-02-15T18:26:53.352 INFO:tasks.workunit.client.0.smithi042.stdout:5/201: write d0/d38/f3c [203249,21746] 0 2024-02-15T18:26:53.352 INFO:tasks.workunit.client.0.smithi042.stdout:3/225: dwrite d1/de/d1a/f1e [4194304,4194304] 0 2024-02-15T18:26:53.353 INFO:tasks.workunit.client.0.smithi042.stdout:7/201: creat d10/d43/f45 x:0 0 0 2024-02-15T18:26:53.360 INFO:tasks.workunit.client.0.smithi042.stdout:1/189: creat de/d15/f43 x:0 0 0 2024-02-15T18:26:53.361 INFO:tasks.workunit.client.0.smithi042.stdout:1/190: write fb [1326810,30648] 0 2024-02-15T18:26:53.362 INFO:tasks.workunit.client.0.smithi042.stdout:4/149: creat da/d11/d29/f30 x:0 0 0 2024-02-15T18:26:53.364 INFO:tasks.workunit.client.0.smithi042.stdout:2/156: creat d0/d8/d20/d2f/f30 x:0 0 0 2024-02-15T18:26:53.364 INFO:tasks.workunit.client.0.smithi042.stdout:2/157: chown d0/d8/d1b 80 1 2024-02-15T18:26:53.365 INFO:tasks.workunit.client.0.smithi042.stdout:2/158: truncate d0/d8/db/de/d1a/f2b 470593 0 2024-02-15T18:26:53.365 INFO:tasks.workunit.client.0.smithi042.stdout:2/159: stat d0/d8/d10/f1c 0 2024-02-15T18:26:53.368 INFO:tasks.workunit.client.0.smithi042.stdout:8/122: unlink dd/c19 0 2024-02-15T18:26:53.368 INFO:tasks.workunit.client.0.smithi042.stdout:8/123: fdatasync dd/de/f10 0 2024-02-15T18:26:53.368 INFO:tasks.workunit.client.0.smithi042.stdout:5/202: dwrite d0/d14/f29 [4194304,4194304] 0 2024-02-15T18:26:53.373 INFO:tasks.workunit.client.0.smithi042.stdout:3/226: creat d1/de/d19/f4b x:0 0 0 2024-02-15T18:26:53.375 INFO:tasks.workunit.client.0.smithi042.stdout:3/227: write d1/de/f2d [2985410,111095] 0 2024-02-15T18:26:53.382 INFO:tasks.workunit.client.0.smithi042.stdout:8/124: dwrite dd/f13 [0,4194304] 0 2024-02-15T18:26:53.389 INFO:tasks.workunit.client.0.smithi042.stdout:7/202: rename d10/l11 to d10/d15/d39/l46 0 2024-02-15T18:26:53.391 INFO:tasks.workunit.client.0.smithi042.stdout:1/191: mkdir de/d15/d1b/d44 0 2024-02-15T18:26:53.392 INFO:tasks.workunit.client.0.smithi042.stdout:4/150: symlink da/d11/d1b/d2f/l31 0 2024-02-15T18:26:53.395 INFO:tasks.workunit.client.0.smithi042.stdout:2/160: mkdir d0/d8/d10/d31 0 2024-02-15T18:26:53.395 INFO:tasks.workunit.client.0.smithi042.stdout:2/161: readlink d0/d8/d1b/l26 0 2024-02-15T18:26:53.395 INFO:tasks.workunit.client.0.smithi042.stdout:3/228: rmdir d1/de/d1a 39 2024-02-15T18:26:53.395 INFO:tasks.workunit.client.0.smithi042.stdout:3/229: fdatasync d1/de/d1a/f2f 0 2024-02-15T18:26:53.410 INFO:tasks.workunit.client.0.smithi042.stdout:7/203: symlink d10/d12/l47 0 2024-02-15T18:26:53.410 INFO:tasks.workunit.client.0.smithi042.stdout:7/204: write d10/d15/f3d [306096,58348] 0 2024-02-15T18:26:53.410 INFO:tasks.workunit.client.0.smithi042.stdout:1/192: dwrite f2 [4194304,4194304] 0 2024-02-15T18:26:53.412 INFO:tasks.workunit.client.0.smithi042.stdout:5/203: rename d0/d22/l26 to d0/d12/d2a/l51 0 2024-02-15T18:26:53.412 INFO:tasks.workunit.client.0.smithi042.stdout:5/204: getdents d0/d50 0 2024-02-15T18:26:53.415 INFO:tasks.workunit.client.0.smithi042.stdout:8/125: unlink dd/d11/l18 0 2024-02-15T18:26:53.417 INFO:tasks.workunit.client.0.smithi042.stdout:2/162: link d0/d8/db/de/d1a/c29 d0/d8/d20/c32 0 2024-02-15T18:26:53.417 INFO:tasks.workunit.client.0.smithi042.stdout:4/151: dwrite da/f24 [0,4194304] 0 2024-02-15T18:26:53.418 INFO:tasks.workunit.client.0.smithi042.stdout:4/152: read da/f26 [60304,104727] 0 2024-02-15T18:26:53.422 INFO:tasks.workunit.client.0.smithi042.stdout:7/205: link d10/c32 d10/d15/c48 0 2024-02-15T18:26:53.422 INFO:tasks.workunit.client.0.smithi042.stdout:0/215: sync 2024-02-15T18:26:53.422 INFO:tasks.workunit.client.0.smithi042.stdout:1/193: truncate de/d15/d2e/f3b 512968 0 2024-02-15T18:26:53.424 INFO:tasks.workunit.client.0.smithi042.stdout:5/205: mkdir d0/db/d41/d52 0 2024-02-15T18:26:53.424 INFO:tasks.workunit.client.0.smithi042.stdout:5/206: write d0/f34 [584821,8218] 0 2024-02-15T18:26:53.427 INFO:tasks.workunit.client.0.smithi042.stdout:2/163: rename d0/lc to d0/d8/d1b/l33 0 2024-02-15T18:26:53.436 INFO:tasks.workunit.client.0.smithi042.stdout:3/230: unlink d1/d11/d14/c21 0 2024-02-15T18:26:53.437 INFO:tasks.workunit.client.0.smithi042.stdout:6/232: sync 2024-02-15T18:26:53.439 INFO:tasks.workunit.client.0.smithi042.stdout:7/206: getdents d10/d15/d39 0 2024-02-15T18:26:53.439 INFO:tasks.workunit.client.0.smithi042.stdout:7/207: truncate d10/d12/d23/f42 1486920 0 2024-02-15T18:26:53.441 INFO:tasks.workunit.client.0.smithi042.stdout:0/216: truncate d4/f6 3926993 0 2024-02-15T18:26:53.441 INFO:tasks.workunit.client.0.smithi042.stdout:0/217: chown d4/d9/d15/f27 10 1 2024-02-15T18:26:53.441 INFO:tasks.workunit.client.0.smithi042.stdout:1/194: creat de/d15/d2e/f45 x:0 0 0 2024-02-15T18:26:53.444 INFO:tasks.workunit.client.0.smithi042.stdout:5/207: mkdir d0/d38/d53 0 2024-02-15T18:26:53.455 INFO:tasks.workunit.client.0.smithi042.stdout:2/164: unlink d0/d8/d10/f1c 0 2024-02-15T18:26:53.455 INFO:tasks.workunit.client.0.smithi042.stdout:4/153: rename da/f2a to da/d11/d29/f32 0 2024-02-15T18:26:53.455 INFO:tasks.workunit.client.0.smithi042.stdout:2/165: fdatasync d0/d8/f28 0 2024-02-15T18:26:53.458 INFO:tasks.workunit.client.0.smithi042.stdout:1/195: rmdir de/d15/d3d 0 2024-02-15T18:26:53.460 INFO:tasks.workunit.client.0.smithi042.stdout:8/126: getdents dd/d11 0 2024-02-15T18:26:53.465 INFO:tasks.workunit.client.0.smithi042.stdout:1/196: write de/d15/d2e/f34 [2230955,18244] 0 2024-02-15T18:26:53.466 INFO:tasks.workunit.client.0.smithi042.stdout:1/197: readlink de/d15/l22 0 2024-02-15T18:26:53.466 INFO:tasks.workunit.client.0.smithi042.stdout:1/198: chown de/d15/d2e/f45 11 1 2024-02-15T18:26:53.466 INFO:tasks.workunit.client.0.smithi042.stdout:1/199: fdatasync de/f21 0 2024-02-15T18:26:53.467 INFO:tasks.workunit.client.0.smithi042.stdout:3/231: dwrite d1/d11/f2c [4194304,4194304] 0 2024-02-15T18:26:53.467 INFO:tasks.workunit.client.0.smithi042.stdout:8/127: write fb [3888606,30593] 0 2024-02-15T18:26:53.469 INFO:tasks.workunit.client.0.smithi042.stdout:3/232: chown d1/d11/d14/f43 723951 1 2024-02-15T18:26:53.471 INFO:tasks.workunit.client.0.smithi042.stdout:1/200: write de/d15/d2e/f1d [7390059,58228] 0 2024-02-15T18:26:53.471 INFO:tasks.workunit.client.0.smithi042.stdout:1/201: readlink de/d15/l37 0 2024-02-15T18:26:53.471 INFO:tasks.workunit.client.0.smithi042.stdout:8/128: getdents dd/de/d12 0 2024-02-15T18:26:53.473 INFO:tasks.workunit.client.0.smithi042.stdout:0/218: dwrite d4/f7 [0,4194304] 0 2024-02-15T18:26:53.473 INFO:tasks.workunit.client.0.smithi042.stdout:0/219: chown d4/d16/d2e/d4c 3723329 1 2024-02-15T18:26:53.473 INFO:tasks.workunit.client.0.smithi042.stdout:3/233: mkdir d1/d11/d2e/d4c 0 2024-02-15T18:26:53.476 INFO:tasks.workunit.client.0.smithi042.stdout:6/233: dwrite d9/f21 [4194304,4194304] 0 2024-02-15T18:26:53.476 INFO:tasks.workunit.client.0.smithi042.stdout:7/208: dwrite fa [0,4194304] 0 2024-02-15T18:26:53.476 INFO:tasks.workunit.client.0.smithi042.stdout:7/209: readlink d10/d15/l31 0 2024-02-15T18:26:53.476 INFO:tasks.workunit.client.0.smithi042.stdout:6/234: read - d9/d3e/f3f zero size 2024-02-15T18:26:53.476 INFO:tasks.workunit.client.0.smithi042.stdout:7/210: chown d10/c32 697291 1 2024-02-15T18:26:53.476 INFO:tasks.workunit.client.0.smithi042.stdout:7/211: readlink d10/d12/d23/l2c 0 2024-02-15T18:26:53.478 INFO:tasks.workunit.client.0.smithi042.stdout:0/220: mkdir d4/d9/d34/d4d 0 2024-02-15T18:26:53.481 INFO:tasks.workunit.client.0.smithi042.stdout:2/166: dwrite d0/d8/d20/d2f/f30 [0,4194304] 0 2024-02-15T18:26:53.484 INFO:tasks.workunit.client.0.smithi042.stdout:1/202: mkdir de/d15/d2e/d46 0 2024-02-15T18:26:53.488 INFO:tasks.workunit.client.0.smithi042.stdout:8/129: dwrite dd/f13 [0,4194304] 0 2024-02-15T18:26:53.488 INFO:tasks.workunit.client.0.smithi042.stdout:8/130: getdents dd/de/d12 0 2024-02-15T18:26:53.493 INFO:tasks.workunit.client.0.smithi042.stdout:8/131: dread dd/f13 [0,4194304] 0 2024-02-15T18:26:53.494 INFO:tasks.workunit.client.0.smithi042.stdout:8/132: dread - dd/de/d12/f1e zero size 2024-02-15T18:26:53.496 INFO:tasks.workunit.client.0.smithi042.stdout:7/212: dwrite d10/f14 [0,4194304] 0 2024-02-15T18:26:53.496 INFO:tasks.workunit.client.0.smithi042.stdout:0/221: mkdir d4/d9/d34/d4e 0 2024-02-15T18:26:53.497 INFO:tasks.workunit.client.0.smithi042.stdout:3/234: mkdir d1/d11/d14/d4d 0 2024-02-15T18:26:53.498 INFO:tasks.workunit.client.0.smithi042.stdout:3/235: truncate d1/de/d19/f3e 345833 0 2024-02-15T18:26:53.512 INFO:tasks.workunit.client.0.smithi042.stdout:3/236: dwrite d1/d11/d2e/f3b [0,4194304] 0 2024-02-15T18:26:53.512 INFO:tasks.workunit.client.0.smithi042.stdout:3/237: truncate d1/f32 53809 0 2024-02-15T18:26:53.512 INFO:tasks.workunit.client.0.smithi042.stdout:3/238: stat d1/d11/f2c 0 2024-02-15T18:26:53.512 INFO:tasks.workunit.client.0.smithi042.stdout:2/167: creat d0/f34 x:0 0 0 2024-02-15T18:26:53.518 INFO:tasks.workunit.client.0.smithi042.stdout:6/235: symlink d9/l47 0 2024-02-15T18:26:53.518 INFO:tasks.workunit.client.0.smithi042.stdout:6/236: write d9/d11/d19/d1c/f27 [606433,7816] 0 2024-02-15T18:26:53.523 INFO:tasks.workunit.client.0.smithi042.stdout:1/203: creat de/d15/d2e/d46/f47 x:0 0 0 2024-02-15T18:26:53.526 INFO:tasks.workunit.client.0.smithi042.stdout:9/176: sync 2024-02-15T18:26:53.526 INFO:tasks.workunit.client.0.smithi042.stdout:9/177: chown d6/dd/l1b 167 1 2024-02-15T18:26:53.526 INFO:tasks.workunit.client.0.smithi042.stdout:9/178: write d6/da/d1d/d2d/f39 [460217,57108] 0 2024-02-15T18:26:53.527 INFO:tasks.workunit.client.0.smithi042.stdout:7/213: mkdir d10/d49 0 2024-02-15T18:26:53.530 INFO:tasks.workunit.client.0.smithi042.stdout:2/168: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:26:53.530 INFO:tasks.workunit.client.0.smithi042.stdout:3/239: mknod d1/d11/d2e/d4c/c4e 0 2024-02-15T18:26:53.530 INFO:tasks.workunit.client.0.smithi042.stdout:3/240: truncate d1/de/d19/f44 213715 0 2024-02-15T18:26:53.530 INFO:tasks.workunit.client.0.smithi042.stdout:3/241: write d1/de/f3a [49798,94410] 0 2024-02-15T18:26:53.531 INFO:tasks.workunit.client.0.smithi042.stdout:3/242: dread d1/de/d1a/f2b [0,4194304] 0 2024-02-15T18:26:53.531 INFO:tasks.workunit.client.0.smithi042.stdout:3/243: readlink d1/d11/d14/l49 0 2024-02-15T18:26:53.532 INFO:tasks.workunit.client.0.smithi042.stdout:6/237: creat d9/d11/d19/f48 x:0 0 0 2024-02-15T18:26:53.532 INFO:tasks.workunit.client.0.smithi042.stdout:6/238: fdatasync d9/d11/f2b 0 2024-02-15T18:26:53.535 INFO:tasks.workunit.client.0.smithi042.stdout:1/204: creat de/f48 x:0 0 0 2024-02-15T18:26:53.538 INFO:tasks.workunit.client.0.smithi042.stdout:9/179: unlink d6/da/c35 0 2024-02-15T18:26:53.540 INFO:tasks.workunit.client.0.smithi042.stdout:8/133: truncate dd/f13 2361256 0 2024-02-15T18:26:53.542 INFO:tasks.workunit.client.0.smithi042.stdout:7/214: unlink fb 0 2024-02-15T18:26:53.543 INFO:tasks.workunit.client.0.smithi042.stdout:7/215: write d10/d15/f34 [1758602,127368] 0 2024-02-15T18:26:53.545 INFO:tasks.workunit.client.0.smithi042.stdout:0/222: truncate d4/f7 449118 0 2024-02-15T18:26:53.556 INFO:tasks.workunit.client.0.smithi042.stdout:2/169: mkdir d0/d35 0 2024-02-15T18:26:53.561 INFO:tasks.workunit.client.0.smithi042.stdout:3/244: creat d1/d11/d14/d4d/f4f x:0 0 0 2024-02-15T18:26:53.561 INFO:tasks.workunit.client.0.smithi042.stdout:6/239: mkdir d9/d11/d19/d35/d49 0 2024-02-15T18:26:53.561 INFO:tasks.workunit.client.0.smithi042.stdout:1/205: dread - de/d15/d2e/f2a zero size 2024-02-15T18:26:53.561 INFO:tasks.workunit.client.0.smithi042.stdout:9/180: mknod d6/dd/c3f 0 2024-02-15T18:26:53.562 INFO:tasks.workunit.client.0.smithi042.stdout:6/240: write d9/d3e/f3b [233110,97348] 0 2024-02-15T18:26:53.564 INFO:tasks.workunit.client.0.smithi042.stdout:4/154: sync 2024-02-15T18:26:53.564 INFO:tasks.workunit.client.0.smithi042.stdout:5/208: sync 2024-02-15T18:26:53.565 INFO:tasks.workunit.client.0.smithi042.stdout:8/134: mknod dd/d11/c20 0 2024-02-15T18:26:53.569 INFO:tasks.workunit.client.0.smithi042.stdout:7/216: link d10/d12/d23/f35 d10/d12/d23/f4a 0 2024-02-15T18:26:53.570 INFO:tasks.workunit.client.0.smithi042.stdout:3/245: unlink d1/de/d1a/f25 0 2024-02-15T18:26:53.570 INFO:tasks.workunit.client.0.smithi042.stdout:3/246: truncate d1/de/d19/f48 983823 0 2024-02-15T18:26:53.570 INFO:tasks.workunit.client.0.smithi042.stdout:1/206: creat de/d15/d1b/f49 x:0 0 0 2024-02-15T18:26:53.570 INFO:tasks.workunit.client.0.smithi042.stdout:3/247: write d1/de/d1a/f1e [8369784,73551] 0 2024-02-15T18:26:53.572 INFO:tasks.workunit.client.0.smithi042.stdout:0/223: write d4/d16/d2e/f32 [240734,93983] 0 2024-02-15T18:26:53.572 INFO:tasks.workunit.client.0.smithi042.stdout:9/181: mkdir d6/da/d1d/d2d/d40 0 2024-02-15T18:26:53.574 INFO:tasks.workunit.client.0.smithi042.stdout:0/224: write d4/d38/f3a [399287,54031] 0 2024-02-15T18:26:53.575 INFO:tasks.workunit.client.0.smithi042.stdout:6/241: symlink d9/d11/d19/d35/d49/l4a 0 2024-02-15T18:26:53.579 INFO:tasks.workunit.client.0.smithi042.stdout:5/209: symlink d0/db/d41/l54 0 2024-02-15T18:26:53.579 INFO:tasks.workunit.client.0.smithi042.stdout:5/210: write d0/d14/f28 [953835,121819] 0 2024-02-15T18:26:53.580 INFO:tasks.workunit.client.0.smithi042.stdout:8/135: creat dd/d11/f21 x:0 0 0 2024-02-15T18:26:53.580 INFO:tasks.workunit.client.0.smithi042.stdout:3/248: dread d1/de/f3a [0,4194304] 0 2024-02-15T18:26:53.586 INFO:tasks.workunit.client.0.smithi042.stdout:9/182: creat d6/dd/d9/f41 x:0 0 0 2024-02-15T18:26:53.586 INFO:tasks.workunit.client.0.smithi042.stdout:0/225: dread d4/fe [4194304,4194304] 0 2024-02-15T18:26:53.587 INFO:tasks.workunit.client.0.smithi042.stdout:0/226: fdatasync d4/d9/d34/f4b 0 2024-02-15T18:26:53.587 INFO:tasks.workunit.client.0.smithi042.stdout:0/227: fdatasync d4/d9/f22 0 2024-02-15T18:26:53.587 INFO:tasks.workunit.client.0.smithi042.stdout:1/207: creat de/d15/d2c/f4a x:0 0 0 2024-02-15T18:26:53.588 INFO:tasks.workunit.client.0.smithi042.stdout:5/211: unlink d0/d12/l25 0 2024-02-15T18:26:53.591 INFO:tasks.workunit.client.0.smithi042.stdout:3/249: rmdir d1/de/d19 39 2024-02-15T18:26:53.591 INFO:tasks.workunit.client.0.smithi042.stdout:3/250: chown d1/de/d1a/f46 0 1 2024-02-15T18:26:53.591 INFO:tasks.workunit.client.0.smithi042.stdout:3/251: chown d1/d11/d14/d4d/f4f 85 1 2024-02-15T18:26:53.593 INFO:tasks.workunit.client.0.smithi042.stdout:4/155: dwrite da/d11/f1d [0,4194304] 0 2024-02-15T18:26:53.593 INFO:tasks.workunit.client.0.smithi042.stdout:4/156: dread - da/d11/d29/f2c zero size 2024-02-15T18:26:53.593 INFO:tasks.workunit.client.0.smithi042.stdout:4/157: stat da/d11/d1b/d2f 0 2024-02-15T18:26:53.598 INFO:tasks.workunit.client.0.smithi042.stdout:1/208: creat de/d15/d2c/f4b x:0 0 0 2024-02-15T18:26:53.598 INFO:tasks.workunit.client.0.smithi042.stdout:0/228: truncate d4/f6 3466314 0 2024-02-15T18:26:53.599 INFO:tasks.workunit.client.0.smithi042.stdout:1/209: write de/d15/d2e/f3f [901302,78203] 0 2024-02-15T18:26:53.599 INFO:tasks.workunit.client.0.smithi042.stdout:2/170: dwrite d0/d8/f11 [0,4194304] 0 2024-02-15T18:26:53.602 INFO:tasks.workunit.client.0.smithi042.stdout:3/252: symlink d1/l50 0 2024-02-15T18:26:53.605 INFO:tasks.workunit.client.0.smithi042.stdout:4/158: creat da/d11/d29/f33 x:0 0 0 2024-02-15T18:26:53.605 INFO:tasks.workunit.client.0.smithi042.stdout:4/159: chown da/c17 136815 1 2024-02-15T18:26:53.605 INFO:tasks.workunit.client.0.smithi042.stdout:4/160: chown da/f1a 64739 1 2024-02-15T18:26:53.605 INFO:tasks.workunit.client.0.smithi042.stdout:4/161: write da/d11/f20 [994090,105681] 0 2024-02-15T18:26:53.605 INFO:tasks.workunit.client.0.smithi042.stdout:4/162: write da/d11/f22 [468057,81979] 0 2024-02-15T18:26:53.607 INFO:tasks.workunit.client.0.smithi042.stdout:0/229: unlink d4/d38/c3e 0 2024-02-15T18:26:53.608 INFO:tasks.workunit.client.0.smithi042.stdout:8/136: dwrite dd/f13 [0,4194304] 0 2024-02-15T18:26:53.608 INFO:tasks.workunit.client.0.smithi042.stdout:8/137: dread - dd/d11/f21 zero size 2024-02-15T18:26:53.608 INFO:tasks.workunit.client.0.smithi042.stdout:8/138: chown dd/d11/f21 145 1 2024-02-15T18:26:53.608 INFO:tasks.workunit.client.0.smithi042.stdout:1/210: rename de/d15/d2e/d46/f47 to de/d15/f4c 0 2024-02-15T18:26:53.608 INFO:tasks.workunit.client.0.smithi042.stdout:1/211: chown de/d15/f43 18717239 1 2024-02-15T18:26:53.612 INFO:tasks.workunit.client.0.smithi042.stdout:3/253: mknod d1/d11/d2e/d4c/c51 0 2024-02-15T18:26:53.612 INFO:tasks.workunit.client.0.smithi042.stdout:1/212: write f6 [3626617,111595] 0 2024-02-15T18:26:53.612 INFO:tasks.workunit.client.0.smithi042.stdout:4/163: creat da/d11/d1b/d2f/f34 x:0 0 0 2024-02-15T18:26:53.613 INFO:tasks.workunit.client.0.smithi042.stdout:2/171: dread d0/d8/db/fd [0,4194304] 0 2024-02-15T18:26:53.615 INFO:tasks.workunit.client.0.smithi042.stdout:0/230: creat d4/f4f x:0 0 0 2024-02-15T18:26:53.615 INFO:tasks.workunit.client.0.smithi042.stdout:7/217: dwrite d10/f1b [0,4194304] 0 2024-02-15T18:26:53.616 INFO:tasks.workunit.client.0.smithi042.stdout:8/139: mkdir dd/de/d22 0 2024-02-15T18:26:53.616 INFO:tasks.workunit.client.0.smithi042.stdout:6/242: dwrite d9/d11/d19/f2a [0,4194304] 0 2024-02-15T18:26:53.617 INFO:tasks.workunit.client.0.smithi042.stdout:0/231: dread d4/d9/d34/f4b [0,4194304] 0 2024-02-15T18:26:53.619 INFO:tasks.workunit.client.0.smithi042.stdout:9/183: dwrite d6/da/d1d/f24 [0,4194304] 0 2024-02-15T18:26:53.619 INFO:tasks.workunit.client.0.smithi042.stdout:0/232: write d4/d38/f3c [277210,114509] 0 2024-02-15T18:26:53.619 INFO:tasks.workunit.client.0.smithi042.stdout:0/233: fdatasync d4/f7 0 2024-02-15T18:26:53.619 INFO:tasks.workunit.client.0.smithi042.stdout:8/140: write fb [3798613,7855] 0 2024-02-15T18:26:53.629 INFO:tasks.workunit.client.0.smithi042.stdout:4/164: link da/f1a da/d2b/f35 0 2024-02-15T18:26:53.633 INFO:tasks.workunit.client.0.smithi042.stdout:2/172: getdents d0/d8/db/de 0 2024-02-15T18:26:53.636 INFO:tasks.workunit.client.0.smithi042.stdout:3/254: dwrite d1/d11/f45 [0,4194304] 0 2024-02-15T18:26:53.637 INFO:tasks.workunit.client.0.smithi042.stdout:6/243: dwrite d9/d11/d1d/f33 [0,4194304] 0 2024-02-15T18:26:53.638 INFO:tasks.workunit.client.0.smithi042.stdout:6/244: write d9/d11/d19/d1c/d31/f3a [1870129,79352] 0 2024-02-15T18:26:53.641 INFO:tasks.workunit.client.0.smithi042.stdout:7/218: unlink d10/d12/d23/f35 0 2024-02-15T18:26:53.642 INFO:tasks.workunit.client.0.smithi042.stdout:7/219: read d10/d15/f34 [1462410,24329] 0 2024-02-15T18:26:53.666 INFO:tasks.workunit.client.0.smithi042.stdout:9/184: mknod d6/da/c42 0 2024-02-15T18:26:53.666 INFO:tasks.workunit.client.0.smithi042.stdout:9/185: write d6/f17 [4813971,38897] 0 2024-02-15T18:26:53.670 INFO:tasks.workunit.client.0.smithi042.stdout:0/234: symlink d4/d16/l50 0 2024-02-15T18:26:53.673 INFO:tasks.workunit.client.0.smithi042.stdout:8/141: unlink dd/c1d 0 2024-02-15T18:26:53.681 INFO:tasks.workunit.client.0.smithi042.stdout:4/165: symlink da/d11/l36 0 2024-02-15T18:26:53.682 INFO:tasks.workunit.client.0.smithi042.stdout:4/166: fsync f9 0 2024-02-15T18:26:53.683 INFO:tasks.workunit.client.0.smithi042.stdout:0/235: dwrite d4/f3b [0,4194304] 0 2024-02-15T18:26:53.683 INFO:tasks.workunit.client.0.smithi042.stdout:0/236: fsync d4/f4f 0 2024-02-15T18:26:53.683 INFO:tasks.workunit.client.0.smithi042.stdout:2/173: mkdir d0/d8/db/de/d36 0 2024-02-15T18:26:53.685 INFO:tasks.workunit.client.0.smithi042.stdout:4/167: read f9 [1916306,100377] 0 2024-02-15T18:26:53.685 INFO:tasks.workunit.client.0.smithi042.stdout:4/168: chown c5 245829 1 2024-02-15T18:26:53.688 INFO:tasks.workunit.client.0.smithi042.stdout:2/174: dread d0/f7 [4194304,4194304] 0 2024-02-15T18:26:53.688 INFO:tasks.workunit.client.0.smithi042.stdout:2/175: chown d0/d8/c16 3 1 2024-02-15T18:26:53.692 INFO:tasks.workunit.client.0.smithi042.stdout:3/255: mknod d1/c52 0 2024-02-15T18:26:53.696 INFO:tasks.workunit.client.0.smithi042.stdout:6/245: rename d9/d11/d19/f37 to d9/d11/d46/f4b 0 2024-02-15T18:26:53.700 INFO:tasks.workunit.client.0.smithi042.stdout:7/220: mknod d10/c4b 0 2024-02-15T18:26:53.703 INFO:tasks.workunit.client.0.smithi042.stdout:8/142: truncate fc 985467 0 2024-02-15T18:26:53.713 INFO:tasks.workunit.client.0.smithi042.stdout:0/237: symlink d4/d9/d15/l51 0 2024-02-15T18:26:53.716 INFO:tasks.workunit.client.0.smithi042.stdout:4/169: creat da/d2b/f37 x:0 0 0 2024-02-15T18:26:53.716 INFO:tasks.workunit.client.0.smithi042.stdout:4/170: truncate da/d11/d29/f2c 293676 0 2024-02-15T18:26:53.718 INFO:tasks.workunit.client.0.smithi042.stdout:3/256: creat d1/d11/d14/f53 x:0 0 0 2024-02-15T18:26:53.718 INFO:tasks.workunit.client.0.smithi042.stdout:3/257: dread - d1/de/d1a/f2f zero size 2024-02-15T18:26:53.719 INFO:tasks.workunit.client.0.smithi042.stdout:3/258: dread d1/de/f3a [0,4194304] 0 2024-02-15T18:26:53.720 INFO:tasks.workunit.client.0.smithi042.stdout:9/186: rename d6/dd/d9/f16 to d6/da/d1d/d2d/d40/f43 0 2024-02-15T18:26:53.721 INFO:tasks.workunit.client.0.smithi042.stdout:6/246: creat d9/d11/d19/d35/d49/f4c x:0 0 0 2024-02-15T18:26:53.721 INFO:tasks.workunit.client.0.smithi042.stdout:6/247: dread - d9/d3e/f3f zero size 2024-02-15T18:26:53.722 INFO:tasks.workunit.client.0.smithi042.stdout:6/248: stat d9/d11/f20 0 2024-02-15T18:26:53.722 INFO:tasks.workunit.client.0.smithi042.stdout:6/249: readlink d9/l42 0 2024-02-15T18:26:53.722 INFO:tasks.workunit.client.0.smithi042.stdout:6/250: write d9/d11/d46/f4b [4178358,120377] 0 2024-02-15T18:26:53.722 INFO:tasks.workunit.client.0.smithi042.stdout:7/221: symlink d10/d15/l4c 0 2024-02-15T18:26:53.723 INFO:tasks.workunit.client.0.smithi042.stdout:8/143: mkdir dd/d23 0 2024-02-15T18:26:53.723 INFO:tasks.workunit.client.0.smithi042.stdout:8/144: readlink l8 0 2024-02-15T18:26:53.724 INFO:tasks.workunit.client.0.smithi042.stdout:7/222: dread d10/f21 [0,4194304] 0 2024-02-15T18:26:53.724 INFO:tasks.workunit.client.0.smithi042.stdout:7/223: fdatasync d10/d15/f34 0 2024-02-15T18:26:53.725 INFO:tasks.workunit.client.0.smithi042.stdout:4/171: mkdir da/d2b/d38 0 2024-02-15T18:26:53.727 INFO:tasks.workunit.client.0.smithi042.stdout:2/176: mknod d0/d8/db/de/c37 0 2024-02-15T18:26:53.730 INFO:tasks.workunit.client.0.smithi042.stdout:3/259: creat d1/d11/d2e/f54 x:0 0 0 2024-02-15T18:26:53.732 INFO:tasks.workunit.client.0.smithi042.stdout:9/187: rmdir d6/da/d2f 39 2024-02-15T18:26:53.732 INFO:tasks.workunit.client.0.smithi042.stdout:9/188: readlink d6/da/d1d/l1e 0 2024-02-15T18:26:53.734 INFO:tasks.workunit.client.0.smithi042.stdout:9/189: write f4 [737234,71266] 0 2024-02-15T18:26:53.734 INFO:tasks.workunit.client.0.smithi042.stdout:9/190: fsync d6/dd/f2b 0 2024-02-15T18:26:53.735 INFO:tasks.workunit.client.0.smithi042.stdout:8/145: creat dd/de/d12/f24 x:0 0 0 2024-02-15T18:26:53.737 INFO:tasks.workunit.client.0.smithi042.stdout:0/238: symlink d4/d16/d2e/l52 0 2024-02-15T18:26:53.737 INFO:tasks.workunit.client.0.smithi042.stdout:0/239: write d4/d16/f24 [1481807,119814] 0 2024-02-15T18:26:53.737 INFO:tasks.workunit.client.0.smithi042.stdout:7/224: dread fa [0,4194304] 0 2024-02-15T18:26:53.737 INFO:tasks.workunit.client.0.smithi042.stdout:7/225: stat d10/d12/f20 0 2024-02-15T18:26:53.738 INFO:tasks.workunit.client.0.smithi042.stdout:4/172: getdents da/d11/d1b/d2f 0 2024-02-15T18:26:53.738 INFO:tasks.workunit.client.0.smithi042.stdout:2/177: dread d0/f19 [0,4194304] 0 2024-02-15T18:26:53.740 INFO:tasks.workunit.client.0.smithi042.stdout:3/260: unlink d1/de/d1a/f46 0 2024-02-15T18:26:53.740 INFO:tasks.workunit.client.0.smithi042.stdout:2/178: write d0/d8/f28 [3640074,94571] 0 2024-02-15T18:26:53.740 INFO:tasks.workunit.client.0.smithi042.stdout:2/179: dread - d0/d8/d10/f17 zero size 2024-02-15T18:26:53.741 INFO:tasks.workunit.client.0.smithi042.stdout:2/180: truncate d0/d8/d10/f17 313550 0 2024-02-15T18:26:53.744 INFO:tasks.workunit.client.0.smithi042.stdout:0/240: unlink d4/d9/d15/c2f 0 2024-02-15T18:26:53.746 INFO:tasks.workunit.client.0.smithi042.stdout:7/226: mknod d10/d15/c4d 0 2024-02-15T18:26:53.748 INFO:tasks.workunit.client.0.smithi042.stdout:4/173: rmdir da/d2b/d38 0 2024-02-15T18:26:53.748 INFO:tasks.workunit.client.0.smithi042.stdout:4/174: readlink da/d11/l16 0 2024-02-15T18:26:53.748 INFO:tasks.workunit.client.0.smithi042.stdout:6/251: dwrite d9/f17 [0,4194304] 0 2024-02-15T18:26:53.749 INFO:tasks.workunit.client.0.smithi042.stdout:9/191: dread d6/da/d1d/f24 [0,4194304] 0 2024-02-15T18:26:53.749 INFO:tasks.workunit.client.0.smithi042.stdout:9/192: readlink d6/da/d1d/l1e 0 2024-02-15T18:26:53.753 INFO:tasks.workunit.client.0.smithi042.stdout:2/181: mkdir d0/d8/d20/d38 0 2024-02-15T18:26:53.753 INFO:tasks.workunit.client.0.smithi042.stdout:2/182: chown d0/d8/d10/l25 12926 1 2024-02-15T18:26:53.753 INFO:tasks.workunit.client.0.smithi042.stdout:7/227: creat d10/d43/f4e x:0 0 0 2024-02-15T18:26:53.759 INFO:tasks.workunit.client.0.smithi042.stdout:8/146: dwrite fb [0,4194304] 0 2024-02-15T18:26:53.759 INFO:tasks.workunit.client.0.smithi042.stdout:8/147: chown l7 22156 1 2024-02-15T18:26:53.759 INFO:tasks.workunit.client.0.smithi042.stdout:5/212: sync 2024-02-15T18:26:53.760 INFO:tasks.workunit.client.0.smithi042.stdout:5/213: chown d0/d22/l36 0 1 2024-02-15T18:26:53.760 INFO:tasks.workunit.client.0.smithi042.stdout:3/261: rmdir d1/d11/d14 39 2024-02-15T18:26:53.760 INFO:tasks.workunit.client.0.smithi042.stdout:8/148: write dd/f13 [1866678,123203] 0 2024-02-15T18:26:53.760 INFO:tasks.workunit.client.0.smithi042.stdout:4/175: creat da/d11/f39 x:0 0 0 2024-02-15T18:26:53.762 INFO:tasks.workunit.client.0.smithi042.stdout:4/176: dread da/d11/f20 [0,4194304] 0 2024-02-15T18:26:53.763 INFO:tasks.workunit.client.0.smithi042.stdout:4/177: write da/d11/d1b/f1c [1534260,43129] 0 2024-02-15T18:26:53.763 INFO:tasks.workunit.client.0.smithi042.stdout:4/178: chown da/d11/d1b/f1e 28 1 2024-02-15T18:26:53.775 INFO:tasks.workunit.client.0.smithi042.stdout:6/252: getdents d9/d11/d19/d35/d49 0 2024-02-15T18:26:53.780 INFO:tasks.workunit.client.0.smithi042.stdout:9/193: creat d6/dd/d3c/f44 x:0 0 0 2024-02-15T18:26:53.780 INFO:tasks.workunit.client.0.smithi042.stdout:9/194: truncate d6/dd/f20 4411597 0 2024-02-15T18:26:53.783 INFO:tasks.workunit.client.0.smithi042.stdout:2/183: write d0/d8/db/de/f14 [426865,97042] 0 2024-02-15T18:26:53.783 INFO:tasks.workunit.client.0.smithi042.stdout:2/184: fdatasync d0/d8/d10/f17 0 2024-02-15T18:26:53.783 INFO:tasks.workunit.client.0.smithi042.stdout:7/228: write d10/d12/d23/f3b [513393,34048] 0 2024-02-15T18:26:53.789 INFO:tasks.workunit.client.0.smithi042.stdout:5/214: unlink d0/d22/l46 0 2024-02-15T18:26:53.790 INFO:tasks.workunit.client.0.smithi042.stdout:5/215: truncate d0/d22/f4a 828698 0 2024-02-15T18:26:53.790 INFO:tasks.workunit.client.0.smithi042.stdout:3/262: truncate d1/d11/d14/f35 405493 0 2024-02-15T18:26:53.793 INFO:tasks.workunit.client.0.smithi042.stdout:1/213: sync 2024-02-15T18:26:53.793 INFO:tasks.workunit.client.0.smithi042.stdout:1/214: chown de/d15/d2c/f35 4 1 2024-02-15T18:26:53.793 INFO:tasks.workunit.client.0.smithi042.stdout:1/215: truncate de/d15/d2e/f2a 222480 0 2024-02-15T18:26:53.794 INFO:tasks.workunit.client.0.smithi042.stdout:1/216: stat de/c12 0 2024-02-15T18:26:53.795 INFO:tasks.workunit.client.0.smithi042.stdout:8/149: getdents dd/de 0 2024-02-15T18:26:53.795 INFO:tasks.workunit.client.0.smithi042.stdout:8/150: dread - dd/d11/f21 zero size 2024-02-15T18:26:53.796 INFO:tasks.workunit.client.0.smithi042.stdout:8/151: write dd/de/f10 [3831102,83151] 0 2024-02-15T18:26:53.796 INFO:tasks.workunit.client.0.smithi042.stdout:4/179: mkdir da/d3a 0 2024-02-15T18:26:53.796 INFO:tasks.workunit.client.0.smithi042.stdout:2/185: dwrite d0/d8/d20/d2f/f30 [0,4194304] 0 2024-02-15T18:26:53.797 INFO:tasks.workunit.client.0.smithi042.stdout:6/253: symlink d9/d11/d19/d1c/l4d 0 2024-02-15T18:26:53.800 INFO:tasks.workunit.client.0.smithi042.stdout:9/195: rename f5 to d6/dd/d9/f45 0 2024-02-15T18:26:53.800 INFO:tasks.workunit.client.0.smithi042.stdout:9/196: chown d6/da/f3e 1768 1 2024-02-15T18:26:53.801 INFO:tasks.workunit.client.0.smithi042.stdout:9/197: write f4 [1868850,57250] 0 2024-02-15T18:26:53.802 INFO:tasks.workunit.client.0.smithi042.stdout:7/229: creat d10/d43/f4f x:0 0 0 2024-02-15T18:26:53.803 INFO:tasks.workunit.client.0.smithi042.stdout:5/216: mknod d0/d14/c55 0 2024-02-15T18:26:53.803 INFO:tasks.workunit.client.0.smithi042.stdout:5/217: fsync d0/db/f15 0 2024-02-15T18:26:53.803 INFO:tasks.workunit.client.0.smithi042.stdout:4/180: dread da/d11/fe [0,4194304] 0 2024-02-15T18:26:53.804 INFO:tasks.workunit.client.0.smithi042.stdout:3/263: symlink d1/d11/d14/d4d/l55 0 2024-02-15T18:26:53.805 INFO:tasks.workunit.client.0.smithi042.stdout:2/186: truncate d0/f19 3092078 0 2024-02-15T18:26:53.807 INFO:tasks.workunit.client.0.smithi042.stdout:6/254: rename d9/d3e/f3f to d9/d11/d19/d35/f4e 0 2024-02-15T18:26:53.807 INFO:tasks.workunit.client.0.smithi042.stdout:6/255: fdatasync d9/d11/d46/f4b 0 2024-02-15T18:26:53.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:52.680200+0000 mgr.smithi042.kfoihb (mgr.24435) 34 : cluster [DBG] pgmap v25: 65 pgs: 65 active+clean; 380 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 26 MiB/s wr, 603 op/s 2024-02-15T18:26:53.811 INFO:tasks.workunit.client.0.smithi042.stdout:7/230: creat d10/d43/f50 x:0 0 0 2024-02-15T18:26:53.813 INFO:tasks.workunit.client.0.smithi042.stdout:4/181: mkdir da/d3b 0 2024-02-15T18:26:53.821 INFO:tasks.workunit.client.0.smithi042.stdout:2/187: creat d0/d8/f39 x:0 0 0 2024-02-15T18:26:53.823 INFO:tasks.workunit.client.0.smithi042.stdout:6/256: rmdir d9/d11/d19/d35 39 2024-02-15T18:26:53.823 INFO:tasks.workunit.client.0.smithi042.stdout:1/217: dwrite f2 [8388608,4194304] 0 2024-02-15T18:26:53.823 INFO:tasks.workunit.client.0.smithi042.stdout:1/218: chown de/d15/d1b/l2f 1206465 1 2024-02-15T18:26:53.824 INFO:tasks.workunit.client.0.smithi042.stdout:2/188: write d0/d8/f11 [2836997,36103] 0 2024-02-15T18:26:53.824 INFO:tasks.workunit.client.0.smithi042.stdout:9/198: rename d6/da/f12 to d6/da/d1d/f46 0 2024-02-15T18:26:53.825 INFO:tasks.workunit.client.0.smithi042.stdout:3/264: dwrite d1/de/f10 [0,4194304] 0 2024-02-15T18:26:53.825 INFO:tasks.workunit.client.0.smithi042.stdout:8/152: dwrite dd/d11/f15 [4194304,4194304] 0 2024-02-15T18:26:53.825 INFO:tasks.workunit.client.0.smithi042.stdout:2/189: read d0/f3 [5080933,44056] 0 2024-02-15T18:26:53.827 INFO:tasks.workunit.client.0.smithi042.stdout:7/231: write d10/d12/d23/f4a [263525,94018] 0 2024-02-15T18:26:53.831 INFO:tasks.workunit.client.0.smithi042.stdout:8/153: dread dd/f13 [0,4194304] 0 2024-02-15T18:26:53.832 INFO:tasks.workunit.client.0.smithi042.stdout:6/257: mkdir d9/d11/d19/d1c/d31/d4f 0 2024-02-15T18:26:53.833 INFO:tasks.workunit.client.0.smithi042.stdout:1/219: mknod de/d15/c4d 0 2024-02-15T18:26:53.836 INFO:tasks.workunit.client.0.smithi042.stdout:5/218: dwrite d0/d14/f1c [0,4194304] 0 2024-02-15T18:26:53.836 INFO:tasks.workunit.client.0.smithi042.stdout:5/219: readlink d0/d22/l45 0 2024-02-15T18:26:53.837 INFO:tasks.workunit.client.0.smithi042.stdout:5/220: dread d0/db/f10 [0,4194304] 0 2024-02-15T18:26:53.839 INFO:tasks.workunit.client.0.smithi042.stdout:4/182: dwrite da/d11/d29/f33 [0,4194304] 0 2024-02-15T18:26:53.840 INFO:tasks.workunit.client.0.smithi042.stdout:6/258: dread d9/d11/d46/f4b [0,4194304] 0 2024-02-15T18:26:53.840 INFO:tasks.workunit.client.0.smithi042.stdout:6/259: stat d9/d11/d1d/f33 0 2024-02-15T18:26:53.848 INFO:tasks.workunit.client.0.smithi042.stdout:9/199: creat d6/dd/d3c/f47 x:0 0 0 2024-02-15T18:26:53.851 INFO:tasks.workunit.client.0.smithi042.stdout:2/190: mkdir d0/d8/d10/d31/d3a 0 2024-02-15T18:26:53.851 INFO:tasks.workunit.client.0.smithi042.stdout:2/191: stat d0/d8/db/de/c37 0 2024-02-15T18:26:53.852 INFO:tasks.workunit.client.0.smithi042.stdout:6/260: dwrite d9/d11/d19/d35/d49/f4c [0,4194304] 0 2024-02-15T18:26:53.852 INFO:tasks.workunit.client.0.smithi042.stdout:8/154: symlink dd/l25 0 2024-02-15T18:26:53.854 INFO:tasks.workunit.client.0.smithi042.stdout:1/220: creat de/f4e x:0 0 0 2024-02-15T18:26:53.854 INFO:tasks.workunit.client.0.smithi042.stdout:1/221: chown de/c1a 118 1 2024-02-15T18:26:53.861 INFO:tasks.workunit.client.0.smithi042.stdout:5/221: creat d0/d22/d2d/f56 x:0 0 0 2024-02-15T18:26:53.861 INFO:tasks.workunit.client.0.smithi042.stdout:5/222: write d0/d22/d2d/f56 [868599,28838] 0 2024-02-15T18:26:53.864 INFO:tasks.workunit.client.0.smithi042.stdout:4/183: creat da/d11/f3c x:0 0 0 2024-02-15T18:26:53.864 INFO:tasks.workunit.client.0.smithi042.stdout:4/184: truncate da/d11/f23 179887 0 2024-02-15T18:26:53.867 INFO:tasks.workunit.client.0.smithi042.stdout:9/200: mkdir d6/da/d1d/d2d/d40/d48 0 2024-02-15T18:26:53.868 INFO:tasks.workunit.client.0.smithi042.stdout:2/192: rename d0/d8/d10/l25 to d0/d8/d20/d2f/l3b 0 2024-02-15T18:26:53.868 INFO:tasks.workunit.client.0.smithi042.stdout:2/193: fdatasync d0/f19 0 2024-02-15T18:26:53.869 INFO:tasks.workunit.client.0.smithi042.stdout:7/232: mkdir d10/d12/d23/d51 0 2024-02-15T18:26:53.872 INFO:tasks.workunit.client.0.smithi042.stdout:5/223: dwrite d0/db/f10 [0,4194304] 0 2024-02-15T18:26:53.874 INFO:tasks.workunit.client.0.smithi042.stdout:6/261: symlink d9/d11/d19/d35/d49/l50 0 2024-02-15T18:26:53.877 INFO:tasks.workunit.client.0.smithi042.stdout:8/155: symlink dd/de/d12/l26 0 2024-02-15T18:26:53.878 INFO:tasks.workunit.client.0.smithi042.stdout:8/156: truncate dd/f17 1295840 0 2024-02-15T18:26:53.878 INFO:tasks.workunit.client.0.smithi042.stdout:8/157: write dd/f13 [2498171,103340] 0 2024-02-15T18:26:53.879 INFO:tasks.workunit.client.0.smithi042.stdout:1/222: rmdir de/d15/d2e 39 2024-02-15T18:26:53.880 INFO:tasks.workunit.client.0.smithi042.stdout:8/158: dread fc [0,4194304] 0 2024-02-15T18:26:53.881 INFO:tasks.workunit.client.0.smithi042.stdout:4/185: mknod da/d11/d29/c3d 0 2024-02-15T18:26:53.881 INFO:tasks.workunit.client.0.smithi042.stdout:4/186: dread - da/f1a zero size 2024-02-15T18:26:53.881 INFO:tasks.workunit.client.0.smithi042.stdout:4/187: chown da/cc 9 1 2024-02-15T18:26:53.884 INFO:tasks.workunit.client.0.smithi042.stdout:9/201: creat d6/da/d1d/d2d/d40/f49 x:0 0 0 2024-02-15T18:26:53.884 INFO:tasks.workunit.client.0.smithi042.stdout:9/202: dread - d6/da/d1d/f38 zero size 2024-02-15T18:26:53.886 INFO:tasks.workunit.client.0.smithi042.stdout:7/233: symlink d10/d15/d39/l52 0 2024-02-15T18:26:53.891 INFO:tasks.workunit.client.0.smithi042.stdout:4/188: dread da/d11/d29/f32 [0,4194304] 0 2024-02-15T18:26:53.891 INFO:tasks.workunit.client.0.smithi042.stdout:4/189: stat da/f1a 0 2024-02-15T18:26:53.894 INFO:tasks.workunit.client.0.smithi042.stdout:5/224: creat d0/d38/d53/f57 x:0 0 0 2024-02-15T18:26:53.897 INFO:tasks.workunit.client.0.smithi042.stdout:6/262: getdents d9/d11 0 2024-02-15T18:26:53.897 INFO:tasks.workunit.client.0.smithi042.stdout:6/263: fdatasync d9/d3e/f3b 0 2024-02-15T18:26:53.898 INFO:tasks.workunit.client.0.smithi042.stdout:1/223: creat de/d15/d2c/f4f x:0 0 0 2024-02-15T18:26:53.899 INFO:tasks.workunit.client.0.smithi042.stdout:8/159: symlink dd/de/d12/l27 0 2024-02-15T18:26:53.899 INFO:tasks.workunit.client.0.smithi042.stdout:8/160: write dd/d11/f21 [113658,46970] 0 2024-02-15T18:26:53.901 INFO:tasks.workunit.client.0.smithi042.stdout:9/203: creat d6/dd/f4a x:0 0 0 2024-02-15T18:26:53.901 INFO:tasks.workunit.client.0.smithi042.stdout:8/161: dread fc [0,4194304] 0 2024-02-15T18:26:53.902 INFO:tasks.workunit.client.0.smithi042.stdout:4/190: rename da/d11/f20 to da/d11/d1b/f3e 0 2024-02-15T18:26:53.903 INFO:tasks.workunit.client.0.smithi042.stdout:7/234: dwrite d10/d15/f36 [0,4194304] 0 2024-02-15T18:26:53.904 INFO:tasks.workunit.client.0.smithi042.stdout:6/264: link d9/d11/d19/d1c/f27 d9/d11/d19/d35/f51 0 2024-02-15T18:26:53.905 INFO:tasks.workunit.client.0.smithi042.stdout:1/224: creat de/d15/d2e/d46/f50 x:0 0 0 2024-02-15T18:26:53.909 INFO:tasks.workunit.client.0.smithi042.stdout:8/162: link dd/f17 dd/d23/f28 0 2024-02-15T18:26:53.911 INFO:tasks.workunit.client.0.smithi042.stdout:8/163: write dd/de/ff [4005567,115401] 0 2024-02-15T18:26:53.911 INFO:tasks.workunit.client.0.smithi042.stdout:4/191: unlink da/d11/d29/f33 0 2024-02-15T18:26:53.917 INFO:tasks.workunit.client.0.smithi042.stdout:8/164: read dd/de/f10 [5156136,42875] 0 2024-02-15T18:26:53.917 INFO:tasks.workunit.client.0.smithi042.stdout:8/165: fdatasync dd/f13 0 2024-02-15T18:26:53.917 INFO:tasks.workunit.client.0.smithi042.stdout:8/166: chown dd/d11 414263 1 2024-02-15T18:26:53.919 INFO:tasks.workunit.client.0.smithi042.stdout:7/235: creat d10/f53 x:0 0 0 2024-02-15T18:26:53.921 INFO:tasks.workunit.client.0.smithi042.stdout:9/204: dwrite d6/dd/fe [0,4194304] 0 2024-02-15T18:26:53.921 INFO:tasks.workunit.client.0.smithi042.stdout:9/205: readlink d6/da/d2f/l33 0 2024-02-15T18:26:53.922 INFO:tasks.workunit.client.0.smithi042.stdout:6/265: creat d9/d11/d19/d35/d49/f52 x:0 0 0 2024-02-15T18:26:53.922 INFO:tasks.workunit.client.0.smithi042.stdout:6/266: write d9/d3e/f3b [4859922,107533] 0 2024-02-15T18:26:53.924 INFO:tasks.workunit.client.0.smithi042.stdout:6/267: dread d9/d11/d19/f38 [0,4194304] 0 2024-02-15T18:26:53.924 INFO:tasks.workunit.client.0.smithi042.stdout:6/268: write d9/d11/d1d/f41 [885840,97369] 0 2024-02-15T18:26:53.924 INFO:tasks.workunit.client.0.smithi042.stdout:6/269: chown d9/d11/f2b 135287975 1 2024-02-15T18:26:53.924 INFO:tasks.workunit.client.0.smithi042.stdout:6/270: read - d9/d11/d19/d35/f4e zero size 2024-02-15T18:26:53.924 INFO:tasks.workunit.client.0.smithi042.stdout:6/271: truncate d9/f3c 482507 0 2024-02-15T18:26:53.927 INFO:tasks.workunit.client.0.smithi042.stdout:8/167: dread dd/de/f10 [4194304,4194304] 0 2024-02-15T18:26:53.927 INFO:tasks.workunit.client.0.smithi042.stdout:8/168: fsync dd/d23/f28 0 2024-02-15T18:26:53.931 INFO:tasks.workunit.client.0.smithi042.stdout:6/272: dread d9/d11/d19/d35/d49/f4c [0,4194304] 0 2024-02-15T18:26:53.934 INFO:tasks.workunit.client.0.smithi042.stdout:1/225: rmdir de/d15 39 2024-02-15T18:26:53.939 INFO:tasks.workunit.client.0.smithi042.stdout:4/192: rename da/d2b/f35 to da/d11/d27/f3f 0 2024-02-15T18:26:53.942 INFO:tasks.workunit.client.0.smithi042.stdout:7/236: mknod d10/d12/c54 0 2024-02-15T18:26:53.942 INFO:tasks.workunit.client.0.smithi042.stdout:7/237: chown c2 1731 1 2024-02-15T18:26:53.944 INFO:tasks.workunit.client.0.smithi042.stdout:8/169: dwrite dd/de/d12/f1e [0,4194304] 0 2024-02-15T18:26:53.944 INFO:tasks.workunit.client.0.smithi042.stdout:9/206: symlink d6/da/d1d/d2d/d40/d48/l4b 0 2024-02-15T18:26:53.953 INFO:tasks.workunit.client.0.smithi042.stdout:1/226: mkdir de/d15/d2c/d51 0 2024-02-15T18:26:53.953 INFO:tasks.workunit.client.0.smithi042.stdout:8/170: dread dd/de/f10 [4194304,4194304] 0 2024-02-15T18:26:53.953 INFO:tasks.workunit.client.0.smithi042.stdout:8/171: chown l7 14 1 2024-02-15T18:26:53.953 INFO:tasks.workunit.client.0.smithi042.stdout:8/172: write dd/d23/f28 [1605107,78825] 0 2024-02-15T18:26:53.953 INFO:tasks.workunit.client.0.smithi042.stdout:6/273: rename d9/d11/d19/d36 to d9/d11/d19/d1c/d31/d53 0 2024-02-15T18:26:53.956 INFO:tasks.workunit.client.0.smithi042.stdout:8/173: read dd/d23/f28 [669295,21720] 0 2024-02-15T18:26:53.958 INFO:tasks.workunit.client.0.smithi042.stdout:7/238: dread d10/d12/d23/f42 [0,4194304] 0 2024-02-15T18:26:53.958 INFO:tasks.workunit.client.0.smithi042.stdout:4/193: dwrite da/d11/d1b/f1e [0,4194304] 0 2024-02-15T18:26:53.958 INFO:tasks.workunit.client.0.smithi042.stdout:4/194: stat da/d11/d29/c3d 0 2024-02-15T18:26:53.959 INFO:tasks.workunit.client.0.smithi042.stdout:4/195: read da/d11/d1b/f3e [591035,118448] 0 2024-02-15T18:26:53.959 INFO:tasks.workunit.client.0.smithi042.stdout:4/196: truncate da/f1a 789364 0 2024-02-15T18:26:53.966 INFO:tasks.workunit.client.0.smithi042.stdout:9/207: rmdir d6/dd 39 2024-02-15T18:26:53.969 INFO:tasks.workunit.client.0.smithi042.stdout:8/174: creat dd/f29 x:0 0 0 2024-02-15T18:26:53.969 INFO:tasks.workunit.client.0.smithi042.stdout:8/175: truncate dd/de/f10 9370370 0 2024-02-15T18:26:53.974 INFO:tasks.workunit.client.0.smithi042.stdout:9/208: mknod d6/da/d1d/c4c 0 2024-02-15T18:26:53.974 INFO:tasks.workunit.client.0.smithi042.stdout:1/227: mkdir de/d15/d2e/d52 0 2024-02-15T18:26:53.974 INFO:tasks.workunit.client.0.smithi042.stdout:1/228: stat de/d15/d2c/f4b 0 2024-02-15T18:26:53.974 INFO:tasks.workunit.client.0.smithi042.stdout:1/229: chown de/d15/d1b/f25 29609 1 2024-02-15T18:26:53.974 INFO:tasks.workunit.client.0.smithi042.stdout:1/230: fsync de/d15/d1b/f25 0 2024-02-15T18:26:53.974 INFO:tasks.workunit.client.0.smithi042.stdout:1/231: readlink de/d15/d2c/l3c 0 2024-02-15T18:26:53.975 INFO:tasks.workunit.client.0.smithi042.stdout:8/176: creat dd/de/f2a x:0 0 0 2024-02-15T18:26:53.975 INFO:tasks.workunit.client.0.smithi042.stdout:8/177: fdatasync dd/f13 0 2024-02-15T18:26:53.976 INFO:tasks.workunit.client.0.smithi042.stdout:9/209: mknod d6/dd/d9/c4d 0 2024-02-15T18:26:53.977 INFO:tasks.workunit.client.0.smithi042.stdout:9/210: read - d6/dd/f19 zero size 2024-02-15T18:26:53.978 INFO:tasks.workunit.client.0.smithi042.stdout:1/232: rename de/d15/d2c/f35 to de/f53 0 2024-02-15T18:26:53.980 INFO:tasks.workunit.client.0.smithi042.stdout:6/274: dwrite d9/f14 [0,4194304] 0 2024-02-15T18:26:53.981 INFO:tasks.workunit.client.0.smithi042.stdout:6/275: read d9/f2d [355235,22783] 0 2024-02-15T18:26:53.981 INFO:tasks.workunit.client.0.smithi042.stdout:9/211: symlink d6/dd/d9/l4e 0 2024-02-15T18:26:53.981 INFO:tasks.workunit.client.0.smithi042.stdout:9/212: write d6/dd/d9/f14 [3868019,29254] 0 2024-02-15T18:26:53.982 INFO:tasks.workunit.client.0.smithi042.stdout:8/178: write dd/de/d12/f1e [4501144,8026] 0 2024-02-15T18:26:53.983 INFO:tasks.workunit.client.0.smithi042.stdout:4/197: dwrite da/d11/d27/f3f [0,4194304] 0 2024-02-15T18:26:53.984 INFO:tasks.workunit.client.0.smithi042.stdout:1/233: mkdir de/d54 0 2024-02-15T18:26:53.984 INFO:tasks.workunit.client.0.smithi042.stdout:1/234: readlink de/d15/d1b/l2f 0 2024-02-15T18:26:53.984 INFO:tasks.workunit.client.0.smithi042.stdout:4/198: write da/d11/f1d [1338921,124931] 0 2024-02-15T18:26:53.987 INFO:tasks.workunit.client.0.smithi042.stdout:7/239: dwrite d10/d12/d23/f27 [0,4194304] 0 2024-02-15T18:26:53.988 INFO:tasks.workunit.client.0.smithi042.stdout:7/240: write d10/d12/d23/f4a [789574,94588] 0 2024-02-15T18:26:53.988 INFO:tasks.workunit.client.0.smithi042.stdout:7/241: chown d10/c32 9220911 1 2024-02-15T18:26:53.989 INFO:tasks.workunit.client.0.smithi042.stdout:6/276: symlink d9/d11/d19/d1c/d31/d4f/l54 0 2024-02-15T18:26:53.990 INFO:tasks.workunit.client.0.smithi042.stdout:9/213: mkdir d6/dd/d4f 0 2024-02-15T18:26:53.999 INFO:tasks.workunit.client.0.smithi042.stdout:1/235: creat de/d15/d2c/d51/f55 x:0 0 0 2024-02-15T18:26:53.999 INFO:tasks.workunit.client.0.smithi042.stdout:1/236: chown de/d15/d1b/f49 92 1 2024-02-15T18:26:53.999 INFO:tasks.workunit.client.0.smithi042.stdout:1/237: fdatasync de/d15/d2e/f34 0 2024-02-15T18:26:54.001 INFO:tasks.workunit.client.0.smithi042.stdout:8/179: dwrite fb [0,4194304] 0 2024-02-15T18:26:54.001 INFO:tasks.workunit.client.0.smithi042.stdout:4/199: mkdir da/d11/d1b/d40 0 2024-02-15T18:26:54.002 INFO:tasks.workunit.client.0.smithi042.stdout:4/200: write da/d11/fe [3614633,72582] 0 2024-02-15T18:26:54.002 INFO:tasks.workunit.client.0.smithi042.stdout:4/201: read da/d11/d1b/f3e [929383,28841] 0 2024-02-15T18:26:54.016 INFO:tasks.workunit.client.0.smithi042.stdout:7/242: creat d10/d12/d23/f55 x:0 0 0 2024-02-15T18:26:54.018 INFO:tasks.workunit.client.0.smithi042.stdout:6/277: mkdir d9/d11/d19/d35/d55 0 2024-02-15T18:26:54.018 INFO:tasks.workunit.client.0.smithi042.stdout:6/278: fdatasync d9/d11/d19/d1c/d31/f3a 0 2024-02-15T18:26:54.018 INFO:tasks.workunit.client.0.smithi042.stdout:6/279: chown d9/d11/d19 26 1 2024-02-15T18:26:54.018 INFO:tasks.workunit.client.0.smithi042.stdout:6/280: readlink d9/d11/l23 0 2024-02-15T18:26:54.018 INFO:tasks.workunit.client.0.smithi042.stdout:9/214: mknod d6/da/d1d/d2d/c50 0 2024-02-15T18:26:54.019 INFO:tasks.workunit.client.0.smithi042.stdout:1/238: mknod de/d54/c56 0 2024-02-15T18:26:54.020 INFO:tasks.workunit.client.0.smithi042.stdout:8/180: creat dd/de/d22/f2b x:0 0 0 2024-02-15T18:26:54.021 INFO:tasks.workunit.client.0.smithi042.stdout:7/243: dread d10/d15/f1d [0,4194304] 0 2024-02-15T18:26:54.022 INFO:tasks.workunit.client.0.smithi042.stdout:4/202: rmdir da/d2b 39 2024-02-15T18:26:54.024 INFO:tasks.workunit.client.0.smithi042.stdout:0/241: sync 2024-02-15T18:26:54.024 INFO:tasks.workunit.client.0.smithi042.stdout:0/242: readlink d4/d16/d2e/l52 0 2024-02-15T18:26:54.025 INFO:tasks.workunit.client.0.smithi042.stdout:0/243: stat d4/d9/d15/f27 0 2024-02-15T18:26:54.025 INFO:tasks.workunit.client.0.smithi042.stdout:0/244: fdatasync d4/d9/d34/f40 0 2024-02-15T18:26:54.025 INFO:tasks.workunit.client.0.smithi042.stdout:0/245: chown d4/d9/d34/f36 225 1 2024-02-15T18:26:54.025 INFO:tasks.workunit.client.0.smithi042.stdout:0/246: fsync d4/fe 0 2024-02-15T18:26:54.025 INFO:tasks.workunit.client.0.smithi042.stdout:6/281: getdents d9/d11/d19/d35/d49 0 2024-02-15T18:26:54.025 INFO:tasks.workunit.client.0.smithi042.stdout:6/282: fsync d9/d11/d19/d1c/f27 0 2024-02-15T18:26:54.025 INFO:tasks.workunit.client.0.smithi042.stdout:8/181: dread dd/d11/f15 [4194304,4194304] 0 2024-02-15T18:26:54.026 INFO:tasks.workunit.client.0.smithi042.stdout:9/215: link d6/da/d1d/c4c d6/dd/d9/c51 0 2024-02-15T18:26:54.026 INFO:tasks.workunit.client.0.smithi042.stdout:8/182: write dd/de/f1a [3442360,8023] 0 2024-02-15T18:26:54.027 INFO:tasks.workunit.client.0.smithi042.stdout:7/244: dread d10/f2f [0,4194304] 0 2024-02-15T18:26:54.027 INFO:tasks.workunit.client.0.smithi042.stdout:7/245: truncate d10/f21 279522 0 2024-02-15T18:26:54.029 INFO:tasks.workunit.client.0.smithi042.stdout:1/239: creat de/d15/d1b/f57 x:0 0 0 2024-02-15T18:26:54.029 INFO:tasks.workunit.client.0.smithi042.stdout:1/240: write de/d15/d1b/f49 [645670,34578] 0 2024-02-15T18:26:54.029 INFO:tasks.workunit.client.0.smithi042.stdout:1/241: stat de/d54 0 2024-02-15T18:26:54.032 INFO:tasks.workunit.client.0.smithi042.stdout:4/203: mkdir da/d3b/d41 0 2024-02-15T18:26:54.036 INFO:tasks.workunit.client.0.smithi042.stdout:1/242: dread de/d15/f20 [0,4194304] 0 2024-02-15T18:26:54.036 INFO:tasks.workunit.client.0.smithi042.stdout:1/243: write de/d15/d2c/f41 [47367,75443] 0 2024-02-15T18:26:54.037 INFO:tasks.workunit.client.0.smithi042.stdout:0/247: creat d4/d16/d2e/d4c/f53 x:0 0 0 2024-02-15T18:26:54.039 INFO:tasks.workunit.client.0.smithi042.stdout:6/283: creat d9/d11/d19/d1c/d31/f56 x:0 0 0 2024-02-15T18:26:54.041 INFO:tasks.workunit.client.0.smithi042.stdout:8/183: creat dd/f2c x:0 0 0 2024-02-15T18:26:54.042 INFO:tasks.workunit.client.0.smithi042.stdout:4/204: creat da/f42 x:0 0 0 2024-02-15T18:26:54.044 INFO:tasks.workunit.client.0.smithi042.stdout:0/248: truncate d4/f7 513292 0 2024-02-15T18:26:54.044 INFO:tasks.workunit.client.0.smithi042.stdout:0/249: stat d4/d38 0 2024-02-15T18:26:54.044 INFO:tasks.workunit.client.0.smithi042.stdout:0/250: fsync d4/f6 0 2024-02-15T18:26:54.046 INFO:tasks.workunit.client.0.smithi042.stdout:6/284: rename d9/d11/d19/c28 to d9/d11/d19/d1c/d31/c57 0 2024-02-15T18:26:54.047 INFO:tasks.workunit.client.0.smithi042.stdout:6/285: write d9/f21 [2088343,68606] 0 2024-02-15T18:26:54.048 INFO:tasks.workunit.client.0.smithi042.stdout:8/184: creat dd/de/d22/f2d x:0 0 0 2024-02-15T18:26:54.048 INFO:tasks.workunit.client.0.smithi042.stdout:8/185: readlink dd/l16 0 2024-02-15T18:26:54.049 INFO:tasks.workunit.client.0.smithi042.stdout:0/251: symlink d4/d9/d34/d4e/l54 0 2024-02-15T18:26:54.050 INFO:tasks.workunit.client.0.smithi042.stdout:0/252: fdatasync d4/d16/d2e/d4c/f53 0 2024-02-15T18:26:54.051 INFO:tasks.workunit.client.0.smithi042.stdout:6/286: mknod d9/d11/d19/d35/d55/c58 0 2024-02-15T18:26:54.051 INFO:tasks.workunit.client.0.smithi042.stdout:7/246: dwrite d10/d12/f3f [0,4194304] 0 2024-02-15T18:26:54.051 INFO:tasks.workunit.client.0.smithi042.stdout:9/216: dwrite d6/da/f23 [0,4194304] 0 2024-02-15T18:26:54.052 INFO:tasks.workunit.client.0.smithi042.stdout:0/253: chown d4/d16/d2e/c2b 138 1 2024-02-15T18:26:54.053 INFO:tasks.workunit.client.0.smithi042.stdout:0/254: chown d4/f4f 27 1 2024-02-15T18:26:54.059 INFO:tasks.workunit.client.0.smithi042.stdout:3/265: sync 2024-02-15T18:26:54.059 INFO:tasks.workunit.client.0.smithi042.stdout:5/225: sync 2024-02-15T18:26:54.059 INFO:tasks.workunit.client.0.smithi042.stdout:2/194: sync 2024-02-15T18:26:54.059 INFO:tasks.workunit.client.0.smithi042.stdout:3/266: write d1/d11/d14/f43 [444799,41195] 0 2024-02-15T18:26:54.059 INFO:tasks.workunit.client.0.smithi042.stdout:3/267: fsync d1/d11/d14/d4d/f4f 0 2024-02-15T18:26:54.065 INFO:tasks.workunit.client.0.smithi042.stdout:6/287: rename d9/d11/l1e to d9/d11/d19/d1c/l59 0 2024-02-15T18:26:54.065 INFO:tasks.workunit.client.0.smithi042.stdout:6/288: dread - d9/d11/d19/d35/d49/f52 zero size 2024-02-15T18:26:54.066 INFO:tasks.workunit.client.0.smithi042.stdout:7/247: creat d10/d15/d39/f56 x:0 0 0 2024-02-15T18:26:54.067 INFO:tasks.workunit.client.0.smithi042.stdout:7/248: chown d10/f26 48802031 1 2024-02-15T18:26:54.070 INFO:tasks.workunit.client.0.smithi042.stdout:9/217: unlink d6/da/l3b 0 2024-02-15T18:26:54.072 INFO:tasks.workunit.client.0.smithi042.stdout:0/255: mknod d4/d9/d34/d4d/c55 0 2024-02-15T18:26:54.074 INFO:tasks.workunit.client.0.smithi042.stdout:3/268: symlink d1/de/d19/l56 0 2024-02-15T18:26:54.080 INFO:tasks.workunit.client.0.smithi042.stdout:2/195: rename d0/d8/d20/d2f to d0/d8/d1b/d3c 0 2024-02-15T18:26:54.082 INFO:tasks.workunit.client.0.smithi042.stdout:7/249: mkdir d10/d43/d57 0 2024-02-15T18:26:54.083 INFO:tasks.workunit.client.0.smithi042.stdout:9/218: mknod d6/dd/d9/c52 0 2024-02-15T18:26:54.088 INFO:tasks.workunit.client.0.smithi042.stdout:6/289: rename d9/d11/d1d/f41 to d9/d11/d19/f5a 0 2024-02-15T18:26:54.088 INFO:tasks.workunit.client.0.smithi042.stdout:6/290: chown d9/d11 1274588 1 2024-02-15T18:26:54.090 INFO:tasks.workunit.client.0.smithi042.stdout:7/250: unlink d10/d15/c4d 0 2024-02-15T18:26:54.091 INFO:tasks.workunit.client.0.smithi042.stdout:9/219: symlink d6/da/d1d/d2d/d40/l53 0 2024-02-15T18:26:54.092 INFO:tasks.workunit.client.0.smithi042.stdout:9/220: chown f3 11 1 2024-02-15T18:26:54.092 INFO:tasks.workunit.client.0.smithi042.stdout:9/221: stat d6/da/d1d/d2d/c37 0 2024-02-15T18:26:54.094 INFO:tasks.workunit.client.0.smithi042.stdout:6/291: rename d9/d11/d19/f48 to d9/d11/d19/d1c/d31/f5b 0 2024-02-15T18:26:54.094 INFO:tasks.workunit.client.0.smithi042.stdout:8/186: dwrite dd/de/d22/f2d [0,4194304] 0 2024-02-15T18:26:54.095 INFO:tasks.workunit.client.0.smithi042.stdout:7/251: link d10/d12/f3f d10/d43/f58 0 2024-02-15T18:26:54.096 INFO:tasks.workunit.client.0.smithi042.stdout:9/222: mkdir d6/da/d2f/d54 0 2024-02-15T18:26:54.097 INFO:tasks.workunit.client.0.smithi042.stdout:6/292: creat d9/d11/d19/d1c/d31/f5c x:0 0 0 2024-02-15T18:26:54.099 INFO:tasks.workunit.client.0.smithi042.stdout:1/244: dwrite de/d15/d2e/f26 [0,4194304] 0 2024-02-15T18:26:54.099 INFO:tasks.workunit.client.0.smithi042.stdout:1/245: chown de/d15/d2e/d52 1925959175 1 2024-02-15T18:26:54.099 INFO:tasks.workunit.client.0.smithi042.stdout:4/205: dwrite da/f24 [0,4194304] 0 2024-02-15T18:26:54.100 INFO:tasks.workunit.client.0.smithi042.stdout:4/206: readlink da/d11/l19 0 2024-02-15T18:26:54.103 INFO:tasks.workunit.client.0.smithi042.stdout:8/187: creat dd/de/d22/f2e x:0 0 0 2024-02-15T18:26:54.105 INFO:tasks.workunit.client.0.smithi042.stdout:9/223: rename d6/dd/d3c to d6/da/d2f/d55 0 2024-02-15T18:26:54.106 INFO:tasks.workunit.client.0.smithi042.stdout:6/293: write d9/f21 [4464391,72655] 0 2024-02-15T18:26:54.106 INFO:tasks.workunit.client.0.smithi042.stdout:6/294: fsync d9/d11/f20 0 2024-02-15T18:26:54.110 INFO:tasks.workunit.client.0.smithi042.stdout:4/207: link da/d11/d29/c3d da/d11/d27/c43 0 2024-02-15T18:26:54.112 INFO:tasks.workunit.client.0.smithi042.stdout:9/224: creat d6/da/d2f/f56 x:0 0 0 2024-02-15T18:26:54.112 INFO:tasks.workunit.client.0.smithi042.stdout:9/225: dread - d6/da/d2f/d55/f44 zero size 2024-02-15T18:26:54.114 INFO:tasks.workunit.client.0.smithi042.stdout:5/226: dwrite d0/fa [0,4194304] 0 2024-02-15T18:26:54.114 INFO:tasks.workunit.client.0.smithi042.stdout:5/227: dread - d0/db/d41/f4c zero size 2024-02-15T18:26:54.114 INFO:tasks.workunit.client.0.smithi042.stdout:0/256: dwrite d4/d9/d15/f21 [0,4194304] 0 2024-02-15T18:26:54.117 INFO:tasks.workunit.client.0.smithi042.stdout:3/269: dwrite d1/de/d19/f1f [0,4194304] 0 2024-02-15T18:26:54.118 INFO:tasks.workunit.client.0.smithi042.stdout:2/196: dwrite d0/f34 [0,4194304] 0 2024-02-15T18:26:54.120 INFO:tasks.workunit.client.0.smithi042.stdout:1/246: dwrite de/d15/d1b/f36 [0,4194304] 0 2024-02-15T18:26:54.120 INFO:tasks.workunit.client.0.smithi042.stdout:1/247: fdatasync de/d15/f4c 0 2024-02-15T18:26:54.120 INFO:tasks.workunit.client.0.smithi042.stdout:6/295: mkdir d9/d11/d19/d35/d55/d5d 0 2024-02-15T18:26:54.121 INFO:tasks.workunit.client.0.smithi042.stdout:7/252: dwrite d10/d12/f20 [0,4194304] 0 2024-02-15T18:26:54.124 INFO:tasks.workunit.client.0.smithi042.stdout:0/257: dread d4/fe [4194304,4194304] 0 2024-02-15T18:26:54.125 INFO:tasks.workunit.client.0.smithi042.stdout:8/188: dread dd/d23/f28 [0,4194304] 0 2024-02-15T18:26:54.125 INFO:tasks.workunit.client.0.smithi042.stdout:8/189: fdatasync fc 0 2024-02-15T18:26:54.125 INFO:tasks.workunit.client.0.smithi042.stdout:4/208: creat da/d3a/f44 x:0 0 0 2024-02-15T18:26:54.133 INFO:tasks.workunit.client.0.smithi042.stdout:5/228: rename d0/d22/l30 to d0/db/d41/d52/l58 0 2024-02-15T18:26:54.133 INFO:tasks.workunit.client.0.smithi042.stdout:5/229: readlink d0/d22/d2d/d42/l44 0 2024-02-15T18:26:54.134 INFO:tasks.workunit.client.0.smithi042.stdout:5/230: fsync d0/db/f15 0 2024-02-15T18:26:54.135 INFO:tasks.workunit.client.0.smithi042.stdout:3/270: dwrite d1/d11/d14/f53 [0,4194304] 0 2024-02-15T18:26:54.135 INFO:tasks.workunit.client.0.smithi042.stdout:3/271: fsync d1/de/f10 0 2024-02-15T18:26:54.148 INFO:tasks.workunit.client.0.smithi042.stdout:3/272: dwrite d1/de/f10 [0,4194304] 0 2024-02-15T18:26:54.148 INFO:tasks.workunit.client.0.smithi042.stdout:2/197: mknod d0/d8/d20/c3d 0 2024-02-15T18:26:54.149 INFO:tasks.workunit.client.0.smithi042.stdout:2/198: write d0/d8/f39 [379851,46109] 0 2024-02-15T18:26:54.155 INFO:tasks.workunit.client.0.smithi042.stdout:1/248: unlink fb 0 2024-02-15T18:26:54.155 INFO:tasks.workunit.client.0.smithi042.stdout:1/249: chown de/d15/d1b/f25 4088786 1 2024-02-15T18:26:54.159 INFO:tasks.workunit.client.0.smithi042.stdout:6/296: creat d9/d11/d1d/f5e x:0 0 0 2024-02-15T18:26:54.159 INFO:tasks.workunit.client.0.smithi042.stdout:6/297: fdatasync d9/f32 0 2024-02-15T18:26:54.159 INFO:tasks.workunit.client.0.smithi042.stdout:6/298: chown d9/d11 5 1 2024-02-15T18:26:54.168 INFO:tasks.workunit.client.0.smithi042.stdout:0/258: rmdir d4/d38 39 2024-02-15T18:26:54.168 INFO:tasks.workunit.client.0.smithi042.stdout:0/259: dread - d4/f4f zero size 2024-02-15T18:26:54.168 INFO:tasks.workunit.client.0.smithi042.stdout:0/260: chown d4/d16/d2e/f32 4954 1 2024-02-15T18:26:54.170 INFO:tasks.workunit.client.0.smithi042.stdout:4/209: getdents da/d11 0 2024-02-15T18:26:54.175 INFO:tasks.workunit.client.0.smithi042.stdout:5/231: link d0/d22/l2f d0/db/d41/d52/l59 0 2024-02-15T18:26:54.183 INFO:tasks.workunit.client.0.smithi042.stdout:2/199: creat d0/d8/db/f3e x:0 0 0 2024-02-15T18:26:54.184 INFO:tasks.workunit.client.0.smithi042.stdout:1/250: creat de/f58 x:0 0 0 2024-02-15T18:26:54.186 INFO:tasks.workunit.client.0.smithi042.stdout:6/299: symlink d9/d11/d19/d1c/d31/d4f/l5f 0 2024-02-15T18:26:54.188 INFO:tasks.workunit.client.0.smithi042.stdout:6/300: dread d9/d11/d19/f38 [0,4194304] 0 2024-02-15T18:26:54.189 INFO:tasks.workunit.client.0.smithi042.stdout:2/200: dread d0/f3 [4194304,4194304] 0 2024-02-15T18:26:54.191 INFO:tasks.workunit.client.0.smithi042.stdout:0/261: getdents d4/d9/d34 0 2024-02-15T18:26:54.192 INFO:tasks.workunit.client.0.smithi042.stdout:4/210: mknod da/d3b/d41/c45 0 2024-02-15T18:26:54.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:52.680200+0000 mgr.smithi042.kfoihb (mgr.24435) 34 : cluster [DBG] pgmap v25: 65 pgs: 65 active+clean; 380 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 26 MiB/s wr, 603 op/s 2024-02-15T18:26:54.196 INFO:tasks.workunit.client.0.smithi042.stdout:7/253: rename d10/d12/l13 to d10/d12/d23/l59 0 2024-02-15T18:26:54.197 INFO:tasks.workunit.client.0.smithi042.stdout:7/254: readlink d10/d15/d39/l52 0 2024-02-15T18:26:54.197 INFO:tasks.workunit.client.0.smithi042.stdout:6/301: dread d9/f21 [4194304,4194304] 0 2024-02-15T18:26:54.202 INFO:tasks.workunit.client.0.smithi042.stdout:5/232: getdents d0/d12 0 2024-02-15T18:26:54.202 INFO:tasks.workunit.client.0.smithi042.stdout:5/233: write d0/d22/d2d/f56 [923032,45633] 0 2024-02-15T18:26:54.204 INFO:tasks.workunit.client.0.smithi042.stdout:0/262: unlink d4/d16/f24 0 2024-02-15T18:26:54.204 INFO:tasks.workunit.client.0.smithi042.stdout:0/263: truncate d4/d9/d34/f4b 976748 0 2024-02-15T18:26:54.205 INFO:tasks.workunit.client.0.smithi042.stdout:4/211: symlink da/d3b/d41/l46 0 2024-02-15T18:26:54.207 INFO:tasks.workunit.client.0.smithi042.stdout:8/190: rename fb to dd/f2f 0 2024-02-15T18:26:54.207 INFO:tasks.workunit.client.0.smithi042.stdout:3/273: rename d1/d11 to d1/d11/d2e/d4c/d57 22 2024-02-15T18:26:54.208 INFO:tasks.workunit.client.0.smithi042.stdout:9/226: sync 2024-02-15T18:26:54.208 INFO:tasks.workunit.client.0.smithi042.stdout:6/302: creat d9/d11/d19/d35/d55/d5d/f60 x:0 0 0 2024-02-15T18:26:54.209 INFO:tasks.workunit.client.0.smithi042.stdout:2/201: dwrite d0/f34 [0,4194304] 0 2024-02-15T18:26:54.213 INFO:tasks.workunit.client.0.smithi042.stdout:5/234: creat d0/d38/f5a x:0 0 0 2024-02-15T18:26:54.213 INFO:tasks.workunit.client.0.smithi042.stdout:5/235: stat d0/db/d41/d52 0 2024-02-15T18:26:54.213 INFO:tasks.workunit.client.0.smithi042.stdout:5/236: fdatasync d0/db/f10 0 2024-02-15T18:26:54.213 INFO:tasks.workunit.client.0.smithi042.stdout:1/251: creat de/d15/d2e/d46/f59 x:0 0 0 2024-02-15T18:26:54.213 INFO:tasks.workunit.client.0.smithi042.stdout:1/252: fdatasync de/d15/d1b/f25 0 2024-02-15T18:26:54.215 INFO:tasks.workunit.client.0.smithi042.stdout:0/264: symlink d4/d9/d34/d4e/l56 0 2024-02-15T18:26:54.219 INFO:tasks.workunit.client.0.smithi042.stdout:4/212: dwrite da/d11/d27/f3f [0,4194304] 0 2024-02-15T18:26:54.220 INFO:tasks.workunit.client.0.smithi042.stdout:7/255: rename d10/d12/d23/f42 to d10/d15/d39/f5a 0 2024-02-15T18:26:54.221 INFO:tasks.workunit.client.0.smithi042.stdout:6/303: rmdir d9/d11/d19/d1c/d31/d4f 39 2024-02-15T18:26:54.221 INFO:tasks.workunit.client.0.smithi042.stdout:7/256: chown d10/d12/d23/d51 10439 1 2024-02-15T18:26:54.221 INFO:tasks.workunit.client.0.smithi042.stdout:6/304: chown d9/d11/d19/d1c/d31/d4f/l54 16728783 1 2024-02-15T18:26:54.221 INFO:tasks.workunit.client.0.smithi042.stdout:3/274: write d1/d11/d14/f47 [984897,114308] 0 2024-02-15T18:26:54.221 INFO:tasks.workunit.client.0.smithi042.stdout:2/202: chown d0/d8/db/de/d1a/d2e 22667 1 2024-02-15T18:26:54.222 INFO:tasks.workunit.client.0.smithi042.stdout:5/237: mkdir d0/d22/d5b 0 2024-02-15T18:26:54.223 INFO:tasks.workunit.client.0.smithi042.stdout:1/253: getdents de/d54 0 2024-02-15T18:26:54.224 INFO:tasks.workunit.client.0.smithi042.stdout:3/275: read d1/d11/f2c [1092306,60936] 0 2024-02-15T18:26:54.224 INFO:tasks.workunit.client.0.smithi042.stdout:7/257: write d10/d12/d23/f27 [629139,57597] 0 2024-02-15T18:26:54.224 INFO:tasks.workunit.client.0.smithi042.stdout:3/276: stat d1/de/d1a 0 2024-02-15T18:26:54.226 INFO:tasks.workunit.client.0.smithi042.stdout:8/191: rename c3 to dd/d23/c30 0 2024-02-15T18:26:54.227 INFO:tasks.workunit.client.0.smithi042.stdout:6/305: link d9/d3e/f3b d9/d11/d19/d35/f61 0 2024-02-15T18:26:54.227 INFO:tasks.workunit.client.0.smithi042.stdout:6/306: read - d9/d11/d1d/f5e zero size 2024-02-15T18:26:54.227 INFO:tasks.workunit.client.0.smithi042.stdout:6/307: truncate d9/d11/f2b 6315691 0 2024-02-15T18:26:54.228 INFO:tasks.workunit.client.0.smithi042.stdout:0/265: dwrite d4/d9/f46 [0,4194304] 0 2024-02-15T18:26:54.229 INFO:tasks.workunit.client.0.smithi042.stdout:2/203: rmdir d0/d8/db/de/d36 0 2024-02-15T18:26:54.231 INFO:tasks.workunit.client.0.smithi042.stdout:1/254: creat de/d15/d2c/f5a x:0 0 0 2024-02-15T18:26:54.231 INFO:tasks.workunit.client.0.smithi042.stdout:1/255: dread - de/d15/d2c/f4b zero size 2024-02-15T18:26:54.232 INFO:tasks.workunit.client.0.smithi042.stdout:5/238: dread d0/f3 [0,4194304] 0 2024-02-15T18:26:54.233 INFO:tasks.workunit.client.0.smithi042.stdout:0/266: dread d4/d38/f3c [0,4194304] 0 2024-02-15T18:26:54.234 INFO:tasks.workunit.client.0.smithi042.stdout:3/277: creat d1/d11/d2e/d4c/f58 x:0 0 0 2024-02-15T18:26:54.237 INFO:tasks.workunit.client.0.smithi042.stdout:9/227: rename f4 to d6/da/d2f/f57 0 2024-02-15T18:26:54.237 INFO:tasks.workunit.client.0.smithi042.stdout:9/228: readlink d6/dd/l26 0 2024-02-15T18:26:54.239 INFO:tasks.workunit.client.0.smithi042.stdout:6/308: mknod d9/d3e/c62 0 2024-02-15T18:26:54.249 INFO:tasks.workunit.client.0.smithi042.stdout:1/256: rename de/f53 to de/f5b 0 2024-02-15T18:26:54.251 INFO:tasks.workunit.client.0.smithi042.stdout:5/239: symlink d0/d22/d2d/d42/l5c 0 2024-02-15T18:26:54.251 INFO:tasks.workunit.client.0.smithi042.stdout:5/240: dread - d0/d14/f2e zero size 2024-02-15T18:26:54.251 INFO:tasks.workunit.client.0.smithi042.stdout:5/241: write d0/d14/f2e [683907,45602] 0 2024-02-15T18:26:54.253 INFO:tasks.workunit.client.0.smithi042.stdout:0/267: mkdir d4/d9/d34/d57 0 2024-02-15T18:26:54.257 INFO:tasks.workunit.client.0.smithi042.stdout:8/192: dwrite dd/de/d22/f2d [0,4194304] 0 2024-02-15T18:26:54.257 INFO:tasks.workunit.client.0.smithi042.stdout:2/204: dread d0/d8/f28 [0,4194304] 0 2024-02-15T18:26:54.257 INFO:tasks.workunit.client.0.smithi042.stdout:8/193: write dd/d11/f21 [226791,89024] 0 2024-02-15T18:26:54.261 INFO:tasks.workunit.client.0.smithi042.stdout:6/309: write d9/d11/f2b [3681528,44147] 0 2024-02-15T18:26:54.261 INFO:tasks.workunit.client.0.smithi042.stdout:6/310: chown d9/d11/f20 13 1 2024-02-15T18:26:54.266 INFO:tasks.workunit.client.0.smithi042.stdout:9/229: dwrite d6/dd/f4a [0,4194304] 0 2024-02-15T18:26:54.269 INFO:tasks.workunit.client.0.smithi042.stdout:3/278: dwrite d1/de/f3a [0,4194304] 0 2024-02-15T18:26:54.269 INFO:tasks.workunit.client.0.smithi042.stdout:3/279: chown d1/de/d19/l1b 92 1 2024-02-15T18:26:54.273 INFO:tasks.workunit.client.0.smithi042.stdout:2/205: dread d0/f3 [0,4194304] 0 2024-02-15T18:26:54.276 INFO:tasks.workunit.client.0.smithi042.stdout:4/213: rename da/f26 to da/d11/d1b/d40/f47 0 2024-02-15T18:26:54.279 INFO:tasks.workunit.client.0.smithi042.stdout:9/230: dread d6/da/f1c [0,4194304] 0 2024-02-15T18:26:54.279 INFO:tasks.workunit.client.0.smithi042.stdout:5/242: mknod d0/d22/d2d/c5d 0 2024-02-15T18:26:54.281 INFO:tasks.workunit.client.0.smithi042.stdout:0/268: truncate d4/d9/d15/f18 2185473 0 2024-02-15T18:26:54.286 INFO:tasks.workunit.client.0.smithi042.stdout:8/194: dwrite dd/de/d22/f2e [0,4194304] 0 2024-02-15T18:26:54.289 INFO:tasks.workunit.client.0.smithi042.stdout:2/206: rename d0/d8/d20/d38 to d0/d8/d20/d3f 0 2024-02-15T18:26:54.291 INFO:tasks.workunit.client.0.smithi042.stdout:4/214: unlink da/d11/f1d 0 2024-02-15T18:26:54.291 INFO:tasks.workunit.client.0.smithi042.stdout:4/215: fdatasync da/d11/d27/f3f 0 2024-02-15T18:26:54.292 INFO:tasks.workunit.client.0.smithi042.stdout:4/216: fdatasync da/f24 0 2024-02-15T18:26:54.292 INFO:tasks.workunit.client.0.smithi042.stdout:9/231: creat d6/da/d1d/f58 x:0 0 0 2024-02-15T18:26:54.293 INFO:tasks.workunit.client.0.smithi042.stdout:9/232: stat d6/dd/d9/f18 0 2024-02-15T18:26:54.293 INFO:tasks.workunit.client.0.smithi042.stdout:9/233: fsync d6/dd/f19 0 2024-02-15T18:26:54.294 INFO:tasks.workunit.client.0.smithi042.stdout:5/243: mknod d0/d22/d5b/c5e 0 2024-02-15T18:26:54.296 INFO:tasks.workunit.client.0.smithi042.stdout:3/280: dwrite d1/de/d19/f44 [0,4194304] 0 2024-02-15T18:26:54.297 INFO:tasks.workunit.client.0.smithi042.stdout:1/257: dwrite de/d15/d2e/f3a [0,4194304] 0 2024-02-15T18:26:54.297 INFO:tasks.workunit.client.0.smithi042.stdout:6/311: dwrite d9/d11/d19/d1c/d31/f5b [0,4194304] 0 2024-02-15T18:26:54.298 INFO:tasks.workunit.client.0.smithi042.stdout:6/312: write d9/d11/d46/f4b [3202773,32992] 0 2024-02-15T18:26:54.300 INFO:tasks.workunit.client.0.smithi042.stdout:0/269: mknod d4/d9/d34/c58 0 2024-02-15T18:26:54.310 INFO:tasks.workunit.client.0.smithi042.stdout:8/195: link l8 dd/l31 0 2024-02-15T18:26:54.313 INFO:tasks.workunit.client.0.smithi042.stdout:3/281: dwrite d1/de/d19/f44 [0,4194304] 0 2024-02-15T18:26:54.313 INFO:tasks.workunit.client.0.smithi042.stdout:2/207: mkdir d0/d40 0 2024-02-15T18:26:54.317 INFO:tasks.workunit.client.0.smithi042.stdout:4/217: creat da/d11/d1b/d40/f48 x:0 0 0 2024-02-15T18:26:54.317 INFO:tasks.workunit.client.0.smithi042.stdout:4/218: fdatasync da/d11/fe 0 2024-02-15T18:26:54.326 INFO:tasks.workunit.client.0.smithi042.stdout:9/234: unlink d6/da/f13 0 2024-02-15T18:26:54.326 INFO:tasks.workunit.client.0.smithi042.stdout:9/235: write d6/da/d1d/f58 [1040921,50743] 0 2024-02-15T18:26:54.326 INFO:tasks.workunit.client.0.smithi042.stdout:9/236: write d6/da/f25 [778683,26839] 0 2024-02-15T18:26:54.326 INFO:tasks.workunit.client.0.smithi042.stdout:9/237: truncate d6/da/f25 1426058 0 2024-02-15T18:26:54.326 INFO:tasks.workunit.client.0.smithi042.stdout:9/238: chown d6/da/d1d/f38 117377953 1 2024-02-15T18:26:54.326 INFO:tasks.workunit.client.0.smithi042.stdout:9/239: fsync d6/dd/f19 0 2024-02-15T18:26:54.329 INFO:tasks.workunit.client.0.smithi042.stdout:5/244: creat d0/d50/f5f x:0 0 0 2024-02-15T18:26:54.329 INFO:tasks.workunit.client.0.smithi042.stdout:5/245: fsync d0/fa 0 2024-02-15T18:26:54.334 INFO:tasks.workunit.client.0.smithi042.stdout:1/258: mknod de/d15/d2c/d51/c5c 0 2024-02-15T18:26:54.336 INFO:tasks.workunit.client.0.smithi042.stdout:6/313: creat d9/d3e/f63 x:0 0 0 2024-02-15T18:26:54.337 INFO:tasks.workunit.client.0.smithi042.stdout:0/270: truncate d4/d9/f46 3187065 0 2024-02-15T18:26:54.337 INFO:tasks.workunit.client.0.smithi042.stdout:0/271: fdatasync d4/d9/f31 0 2024-02-15T18:26:54.338 INFO:tasks.workunit.client.0.smithi042.stdout:3/282: mknod d1/c59 0 2024-02-15T18:26:54.339 INFO:tasks.workunit.client.0.smithi042.stdout:3/283: truncate d1/de/d19/f4b 233015 0 2024-02-15T18:26:54.339 INFO:tasks.workunit.client.0.smithi042.stdout:3/284: dread - d1/de/d1a/f2f zero size 2024-02-15T18:26:54.340 INFO:tasks.workunit.client.0.smithi042.stdout:4/219: creat da/d3b/d41/f49 x:0 0 0 2024-02-15T18:26:54.340 INFO:tasks.workunit.client.0.smithi042.stdout:4/220: write da/f21 [630816,128985] 0 2024-02-15T18:26:54.342 INFO:tasks.workunit.client.0.smithi042.stdout:2/208: dwrite d0/d8/f11 [4194304,4194304] 0 2024-02-15T18:26:54.342 INFO:tasks.workunit.client.0.smithi042.stdout:5/246: dread d0/db/f15 [0,4194304] 0 2024-02-15T18:26:54.342 INFO:tasks.workunit.client.0.smithi042.stdout:8/196: dwrite dd/de/d22/f2d [0,4194304] 0 2024-02-15T18:26:54.344 INFO:tasks.workunit.client.0.smithi042.stdout:8/197: write dd/de/f1a [2711760,10620] 0 2024-02-15T18:26:54.349 INFO:tasks.workunit.client.0.smithi042.stdout:9/240: mkdir d6/dd/d9/d59 0 2024-02-15T18:26:54.353 INFO:tasks.workunit.client.0.smithi042.stdout:6/314: link d9/d11/d19/d35/f51 d9/d3e/f64 0 2024-02-15T18:26:54.354 INFO:tasks.workunit.client.0.smithi042.stdout:6/315: stat d9/d11/d19/d35/d55 0 2024-02-15T18:26:54.356 INFO:tasks.workunit.client.0.smithi042.stdout:3/285: creat d1/de/d1a/f5a x:0 0 0 2024-02-15T18:26:54.357 INFO:tasks.workunit.client.0.smithi042.stdout:4/221: dwrite da/f25 [0,4194304] 0 2024-02-15T18:26:54.360 INFO:tasks.workunit.client.0.smithi042.stdout:2/209: mkdir d0/d8/d20/d41 0 2024-02-15T18:26:54.360 INFO:tasks.workunit.client.0.smithi042.stdout:2/210: readlink d0/l23 0 2024-02-15T18:26:54.364 INFO:tasks.workunit.client.0.smithi042.stdout:5/247: mknod d0/db/d41/c60 0 2024-02-15T18:26:54.364 INFO:tasks.workunit.client.0.smithi042.stdout:5/248: write d0/d50/f5f [591938,80695] 0 2024-02-15T18:26:54.364 INFO:tasks.workunit.client.0.smithi042.stdout:5/249: write d0/f4d [475283,43802] 0 2024-02-15T18:26:54.364 INFO:tasks.workunit.client.0.smithi042.stdout:5/250: fsync d0/d38/f5a 0 2024-02-15T18:26:54.364 INFO:tasks.workunit.client.0.smithi042.stdout:5/251: truncate d0/d14/f28 1213749 0 2024-02-15T18:26:54.367 INFO:tasks.workunit.client.0.smithi042.stdout:8/198: mkdir dd/de/d12/d32 0 2024-02-15T18:26:54.368 INFO:tasks.workunit.client.0.smithi042.stdout:9/241: mknod d6/da/c5a 0 2024-02-15T18:26:54.369 INFO:tasks.workunit.client.0.smithi042.stdout:2/211: dread d0/f7 [0,4194304] 0 2024-02-15T18:26:54.372 INFO:tasks.workunit.client.0.smithi042.stdout:6/316: dwrite d9/d11/d19/f38 [0,4194304] 0 2024-02-15T18:26:54.372 INFO:tasks.workunit.client.0.smithi042.stdout:1/259: symlink de/d15/d2e/d52/l5d 0 2024-02-15T18:26:54.376 INFO:tasks.workunit.client.0.smithi042.stdout:0/272: mknod d4/d16/d2e/c59 0 2024-02-15T18:26:54.378 INFO:tasks.workunit.client.0.smithi042.stdout:3/286: truncate d1/de/d19/f1f 2939655 0 2024-02-15T18:26:54.378 INFO:tasks.workunit.client.0.smithi042.stdout:0/273: write d4/fe [11131606,68061] 0 2024-02-15T18:26:54.388 INFO:tasks.workunit.client.0.smithi042.stdout:1/260: dwrite de/d15/d2c/d51/f55 [0,4194304] 0 2024-02-15T18:26:54.395 INFO:tasks.workunit.client.0.smithi042.stdout:4/222: mkdir da/d11/d29/d4a 0 2024-02-15T18:26:54.395 INFO:tasks.workunit.client.0.smithi042.stdout:4/223: fsync da/d11/d29/f2c 0 2024-02-15T18:26:54.396 INFO:tasks.workunit.client.0.smithi042.stdout:5/252: rmdir d0/d22/d2d/d42 39 2024-02-15T18:26:54.396 INFO:tasks.workunit.client.0.smithi042.stdout:5/253: chown d0/db/d41/f4c 6522 1 2024-02-15T18:26:54.399 INFO:tasks.workunit.client.0.smithi042.stdout:8/199: unlink l1 0 2024-02-15T18:26:54.404 INFO:tasks.workunit.client.0.smithi042.stdout:1/261: dwrite de/d15/d1b/f25 [0,4194304] 0 2024-02-15T18:26:54.404 INFO:tasks.workunit.client.0.smithi042.stdout:1/262: stat de/c1a 0 2024-02-15T18:26:54.409 INFO:tasks.workunit.client.0.smithi042.stdout:2/212: creat d0/d8/db/f42 x:0 0 0 2024-02-15T18:26:54.409 INFO:tasks.workunit.client.0.smithi042.stdout:2/213: write d0/d8/db/f3e [635315,109322] 0 2024-02-15T18:26:54.409 INFO:tasks.workunit.client.0.smithi042.stdout:2/214: stat d0 0 2024-02-15T18:26:54.410 INFO:tasks.workunit.client.0.smithi042.stdout:5/254: dwrite d0/d22/d2d/f32 [0,4194304] 0 2024-02-15T18:26:54.411 INFO:tasks.workunit.client.0.smithi042.stdout:8/200: dread dd/d11/f21 [0,4194304] 0 2024-02-15T18:26:54.412 INFO:tasks.workunit.client.0.smithi042.stdout:8/201: chown dd/de/ff 14943 1 2024-02-15T18:26:54.413 INFO:tasks.workunit.client.0.smithi042.stdout:6/317: rename d9/d11/d19/d1c/d31/d53/f43 to d9/d11/d19/d35/d55/f65 0 2024-02-15T18:26:54.415 INFO:tasks.workunit.client.0.smithi042.stdout:3/287: truncate d1/de/f2d 4639028 0 2024-02-15T18:26:54.415 INFO:tasks.workunit.client.0.smithi042.stdout:3/288: chown d1/d11/d14/f43 2005924285 1 2024-02-15T18:26:54.416 INFO:tasks.workunit.client.0.smithi042.stdout:0/274: mkdir d4/d9/d5a 0 2024-02-15T18:26:54.417 INFO:tasks.workunit.client.0.smithi042.stdout:7/258: sync 2024-02-15T18:26:54.418 INFO:tasks.workunit.client.0.smithi042.stdout:7/259: write d10/d43/f4f [380795,59898] 0 2024-02-15T18:26:54.418 INFO:tasks.workunit.client.0.smithi042.stdout:7/260: write d10/d43/f4e [444777,39497] 0 2024-02-15T18:26:54.419 INFO:tasks.workunit.client.0.smithi042.stdout:7/261: write d10/d12/d23/f27 [3082748,124028] 0 2024-02-15T18:26:54.420 INFO:tasks.workunit.client.0.smithi042.stdout:4/224: mkdir da/d11/d29/d4b 0 2024-02-15T18:26:54.423 INFO:tasks.workunit.client.0.smithi042.stdout:1/263: creat de/d54/f5e x:0 0 0 2024-02-15T18:26:54.424 INFO:tasks.workunit.client.0.smithi042.stdout:7/262: read d10/d12/f3f [4009648,112315] 0 2024-02-15T18:26:54.430 INFO:tasks.workunit.client.0.smithi042.stdout:3/289: dread d1/d11/d2e/f37 [0,4194304] 0 2024-02-15T18:26:54.430 INFO:tasks.workunit.client.0.smithi042.stdout:2/215: dread d0/d8/db/de/f18 [0,4194304] 0 2024-02-15T18:26:54.434 INFO:tasks.workunit.client.0.smithi042.stdout:5/255: mkdir d0/d12/d61 0 2024-02-15T18:26:54.434 INFO:tasks.workunit.client.0.smithi042.stdout:9/242: dwrite d6/da/d1d/d2d/f39 [0,4194304] 0 2024-02-15T18:26:54.434 INFO:tasks.workunit.client.0.smithi042.stdout:8/202: mkdir dd/de/d33 0 2024-02-15T18:26:54.435 INFO:tasks.workunit.client.0.smithi042.stdout:5/256: dread d0/f4d [0,4194304] 0 2024-02-15T18:26:54.435 INFO:tasks.workunit.client.0.smithi042.stdout:5/257: fdatasync d0/d12/d2a/f3b 0 2024-02-15T18:26:54.435 INFO:tasks.workunit.client.0.smithi042.stdout:5/258: write d0/f47 [849649,116528] 0 2024-02-15T18:26:54.441 INFO:tasks.workunit.client.0.smithi042.stdout:6/318: creat d9/f66 x:0 0 0 2024-02-15T18:26:54.443 INFO:tasks.workunit.client.0.smithi042.stdout:0/275: mknod d4/d9/d15/c5b 0 2024-02-15T18:26:54.447 INFO:tasks.workunit.client.0.smithi042.stdout:2/216: dwrite d0/d8/d1b/d3c/f30 [4194304,4194304] 0 2024-02-15T18:26:54.448 INFO:tasks.workunit.client.0.smithi042.stdout:1/264: dwrite de/d15/d2c/f4a [0,4194304] 0 2024-02-15T18:26:54.448 INFO:tasks.workunit.client.0.smithi042.stdout:1/265: dread - de/d15/d2e/d46/f59 zero size 2024-02-15T18:26:54.450 INFO:tasks.workunit.client.0.smithi042.stdout:3/290: link d1/d11/d2e/f54 d1/d11/d2e/d4c/f5b 0 2024-02-15T18:26:54.458 INFO:tasks.workunit.client.0.smithi042.stdout:8/203: creat dd/de/d12/d32/f34 x:0 0 0 2024-02-15T18:26:54.463 INFO:tasks.workunit.client.0.smithi042.stdout:5/259: mknod d0/d22/c62 0 2024-02-15T18:26:54.469 INFO:tasks.workunit.client.0.smithi042.stdout:5/260: dread d0/d14/f2e [0,4194304] 0 2024-02-15T18:26:54.470 INFO:tasks.workunit.client.0.smithi042.stdout:0/276: mkdir d4/d38/d5c 0 2024-02-15T18:26:54.470 INFO:tasks.workunit.client.0.smithi042.stdout:0/277: write d4/d38/f3c [1079100,99340] 0 2024-02-15T18:26:54.476 INFO:tasks.workunit.client.0.smithi042.stdout:2/217: dwrite d0/f19 [0,4194304] 0 2024-02-15T18:26:54.476 INFO:tasks.workunit.client.0.smithi042.stdout:2/218: write d0/f3 [9383673,96345] 0 2024-02-15T18:26:54.479 INFO:tasks.workunit.client.0.smithi042.stdout:6/319: link d9/d11/f20 d9/f67 0 2024-02-15T18:26:54.479 INFO:tasks.workunit.client.0.smithi042.stdout:1/266: dwrite de/d15/d1b/f57 [0,4194304] 0 2024-02-15T18:26:54.480 INFO:tasks.workunit.client.0.smithi042.stdout:5/261: truncate d0/d14/f1c 998223 0 2024-02-15T18:26:54.480 INFO:tasks.workunit.client.0.smithi042.stdout:3/291: dwrite d1/de/f2d [4194304,4194304] 0 2024-02-15T18:26:54.480 INFO:tasks.workunit.client.0.smithi042.stdout:6/320: write d9/f2f [4094376,11882] 0 2024-02-15T18:26:54.481 INFO:tasks.workunit.client.0.smithi042.stdout:0/278: symlink d4/d9/d34/d4d/l5d 0 2024-02-15T18:26:54.483 INFO:tasks.workunit.client.0.smithi042.stdout:8/204: truncate dd/d11/f15 9189640 0 2024-02-15T18:26:54.483 INFO:tasks.workunit.client.0.smithi042.stdout:8/205: readlink dd/de/d12/l26 0 2024-02-15T18:26:54.483 INFO:tasks.workunit.client.0.smithi042.stdout:8/206: fdatasync dd/f2f 0 2024-02-15T18:26:54.483 INFO:tasks.workunit.client.0.smithi042.stdout:8/207: truncate dd/f29 387897 0 2024-02-15T18:26:54.488 INFO:tasks.workunit.client.0.smithi042.stdout:2/219: dread d0/f19 [0,4194304] 0 2024-02-15T18:26:54.495 INFO:tasks.workunit.client.0.smithi042.stdout:1/267: creat de/d15/d2c/f5f x:0 0 0 2024-02-15T18:26:54.497 INFO:tasks.workunit.client.0.smithi042.stdout:5/262: creat d0/d14/f63 x:0 0 0 2024-02-15T18:26:54.497 INFO:tasks.workunit.client.0.smithi042.stdout:6/321: dwrite d9/d3e/f63 [0,4194304] 0 2024-02-15T18:26:54.497 INFO:tasks.workunit.client.0.smithi042.stdout:6/322: dread - d9/f66 zero size 2024-02-15T18:26:54.502 INFO:tasks.workunit.client.0.smithi042.stdout:3/292: fdatasync d1/d11/d14/f39 0 2024-02-15T18:26:54.505 INFO:tasks.workunit.client.0.smithi042.stdout:6/323: dread d9/d11/d19/f2a [0,4194304] 0 2024-02-15T18:26:54.507 INFO:tasks.workunit.client.0.smithi042.stdout:0/279: mkdir d4/d9/d34/d57/d5e 0 2024-02-15T18:26:54.508 INFO:tasks.workunit.client.0.smithi042.stdout:8/208: chown l8 2 1 2024-02-15T18:26:54.510 INFO:tasks.workunit.client.0.smithi042.stdout:2/220: unlink d0/f19 0 2024-02-15T18:26:54.511 INFO:tasks.workunit.client.0.smithi042.stdout:5/263: truncate d0/fa 63774 0 2024-02-15T18:26:54.512 INFO:tasks.workunit.client.0.smithi042.stdout:8/209: dread dd/f13 [0,4194304] 0 2024-02-15T18:26:54.515 INFO:tasks.workunit.client.0.smithi042.stdout:1/268: dwrite de/f21 [0,4194304] 0 2024-02-15T18:26:54.516 INFO:tasks.workunit.client.0.smithi042.stdout:1/269: readlink de/d15/l37 0 2024-02-15T18:26:54.518 INFO:tasks.workunit.client.0.smithi042.stdout:6/324: creat d9/d11/d19/d1c/d31/d4f/f68 x:0 0 0 2024-02-15T18:26:54.520 INFO:tasks.workunit.client.0.smithi042.stdout:0/280: symlink d4/d9/d34/d4d/l5f 0 2024-02-15T18:26:54.522 INFO:tasks.workunit.client.0.smithi042.stdout:3/293: dwrite d1/d11/d14/f39 [0,4194304] 0 2024-02-15T18:26:54.525 INFO:tasks.workunit.client.0.smithi042.stdout:5/264: mkdir d0/d38/d53/d64 0 2024-02-15T18:26:54.526 INFO:tasks.workunit.client.0.smithi042.stdout:8/210: mknod dd/d11/c35 0 2024-02-15T18:26:54.529 INFO:tasks.workunit.client.0.smithi042.stdout:1/270: unlink de/d15/d2c/f4a 0 2024-02-15T18:26:54.532 INFO:tasks.workunit.client.0.smithi042.stdout:6/325: rename d9/d11/d19/d1c/d31/c57 to d9/d11/d19/d1c/c69 0 2024-02-15T18:26:54.532 INFO:tasks.workunit.client.0.smithi042.stdout:2/221: dwrite d0/f34 [0,4194304] 0 2024-02-15T18:26:54.535 INFO:tasks.workunit.client.0.smithi042.stdout:3/294: truncate d1/d11/d14/f43 101337 0 2024-02-15T18:26:54.537 INFO:tasks.workunit.client.0.smithi042.stdout:5/265: mkdir d0/d12/d65 0 2024-02-15T18:26:54.537 INFO:tasks.workunit.client.0.smithi042.stdout:5/266: fdatasync d0/fa 0 2024-02-15T18:26:54.539 INFO:tasks.workunit.client.0.smithi042.stdout:8/211: symlink dd/de/l36 0 2024-02-15T18:26:54.539 INFO:tasks.workunit.client.0.smithi042.stdout:0/281: dwrite d4/d9/d15/f27 [0,4194304] 0 2024-02-15T18:26:54.543 INFO:tasks.workunit.client.0.smithi042.stdout:1/271: mkdir de/d15/d2c/d51/d60 0 2024-02-15T18:26:54.549 INFO:tasks.workunit.client.0.smithi042.stdout:6/326: rename d9/d3e/c62 to d9/d11/d19/d35/c6a 0 2024-02-15T18:26:54.551 INFO:tasks.workunit.client.0.smithi042.stdout:5/267: creat d0/d12/d61/f66 x:0 0 0 2024-02-15T18:26:54.552 INFO:tasks.workunit.client.0.smithi042.stdout:3/295: dwrite d1/de/d1a/f2b [0,4194304] 0 2024-02-15T18:26:54.563 INFO:tasks.workunit.client.0.smithi042.stdout:8/212: truncate fc 1435328 0 2024-02-15T18:26:54.566 INFO:tasks.workunit.client.0.smithi042.stdout:0/282: mkdir d4/d9/d5a/d60 0 2024-02-15T18:26:54.568 INFO:tasks.workunit.client.0.smithi042.stdout:6/327: dwrite d9/d11/f2b [0,4194304] 0 2024-02-15T18:26:54.568 INFO:tasks.workunit.client.0.smithi042.stdout:6/328: readlink d9/lb 0 2024-02-15T18:26:54.569 INFO:tasks.workunit.client.0.smithi042.stdout:6/329: fdatasync d9/f17 0 2024-02-15T18:26:54.569 INFO:tasks.workunit.client.0.smithi042.stdout:6/330: fsync d9/d11/d19/d1c/d31/d4f/f68 0 2024-02-15T18:26:54.572 INFO:tasks.workunit.client.0.smithi042.stdout:2/222: rename d0/d8/db/de/f18 to d0/d8/db/de/d1a/f43 0 2024-02-15T18:26:54.577 INFO:tasks.workunit.client.0.smithi042.stdout:2/223: write d0/f7 [6213775,33028] 0 2024-02-15T18:26:54.577 INFO:tasks.workunit.client.0.smithi042.stdout:7/263: sync 2024-02-15T18:26:54.577 INFO:tasks.workunit.client.0.smithi042.stdout:9/243: sync 2024-02-15T18:26:54.577 INFO:tasks.workunit.client.0.smithi042.stdout:4/225: sync 2024-02-15T18:26:54.578 INFO:tasks.workunit.client.0.smithi042.stdout:5/268: link d0/l1d d0/d12/d65/l67 0 2024-02-15T18:26:54.578 INFO:tasks.workunit.client.0.smithi042.stdout:3/296: rmdir d1 39 2024-02-15T18:26:54.581 INFO:tasks.workunit.client.0.smithi042.stdout:1/272: dwrite de/d15/d2e/f3b [0,4194304] 0 2024-02-15T18:26:54.582 INFO:tasks.workunit.client.0.smithi042.stdout:8/213: creat dd/de/d33/f37 x:0 0 0 2024-02-15T18:26:54.584 INFO:tasks.workunit.client.0.smithi042.stdout:0/283: rmdir d4/d9/d34/d57/d5e 0 2024-02-15T18:26:54.584 INFO:tasks.workunit.client.0.smithi042.stdout:0/284: chown d4/f41 1779706586 1 2024-02-15T18:26:54.588 INFO:tasks.workunit.client.0.smithi042.stdout:2/224: mkdir d0/d8/db/d44 0 2024-02-15T18:26:54.591 INFO:tasks.workunit.client.0.smithi042.stdout:7/264: creat d10/f5b x:0 0 0 2024-02-15T18:26:54.591 INFO:tasks.workunit.client.0.smithi042.stdout:7/265: stat d10/d15/d39/f56 0 2024-02-15T18:26:54.592 INFO:tasks.workunit.client.0.smithi042.stdout:4/226: mknod da/d11/d27/c4c 0 2024-02-15T18:26:54.594 INFO:tasks.workunit.client.0.smithi042.stdout:5/269: unlink d0/d22/d5b/c5e 0 2024-02-15T18:26:54.599 INFO:tasks.workunit.client.0.smithi042.stdout:6/331: dwrite d9/f14 [0,4194304] 0 2024-02-15T18:26:54.599 INFO:tasks.workunit.client.0.smithi042.stdout:8/214: unlink dd/de/f10 0 2024-02-15T18:26:54.599 INFO:tasks.workunit.client.0.smithi042.stdout:3/297: dwrite d1/d11/f45 [4194304,4194304] 0 2024-02-15T18:26:54.602 INFO:tasks.workunit.client.0.smithi042.stdout:0/285: truncate d4/d38/f3a 293244 0 2024-02-15T18:26:54.602 INFO:tasks.workunit.client.0.smithi042.stdout:0/286: chown d4/d38/f3a 323206076 1 2024-02-15T18:26:54.602 INFO:tasks.workunit.client.0.smithi042.stdout:0/287: truncate d4/d9/d15/f27 5181386 0 2024-02-15T18:26:54.603 INFO:tasks.workunit.client.0.smithi042.stdout:2/225: creat d0/d8/d1b/f45 x:0 0 0 2024-02-15T18:26:54.606 INFO:tasks.workunit.client.0.smithi042.stdout:7/266: unlink d10/d15/l4c 0 2024-02-15T18:26:54.606 INFO:tasks.workunit.client.0.smithi042.stdout:9/244: dwrite d6/dd/d9/f41 [0,4194304] 0 2024-02-15T18:26:54.606 INFO:tasks.workunit.client.0.smithi042.stdout:6/332: dread d9/f21 [0,4194304] 0 2024-02-15T18:26:54.613 INFO:tasks.workunit.client.0.smithi042.stdout:2/226: dread d0/d8/db/f3e [0,4194304] 0 2024-02-15T18:26:54.613 INFO:tasks.workunit.client.0.smithi042.stdout:0/288: write d4/f6 [800915,49164] 0 2024-02-15T18:26:54.616 INFO:tasks.workunit.client.0.smithi042.stdout:4/227: rename da/d11/d1b/d40 to da/d11/d1b/d2f/d4d 0 2024-02-15T18:26:54.620 INFO:tasks.workunit.client.0.smithi042.stdout:5/270: mkdir d0/d38/d53/d68 0 2024-02-15T18:26:54.623 INFO:tasks.workunit.client.0.smithi042.stdout:1/273: unlink de/d15/d2e/f18 0 2024-02-15T18:26:54.626 INFO:tasks.workunit.client.0.smithi042.stdout:7/267: truncate d10/d12/d23/f4a 559243 0 2024-02-15T18:26:54.628 INFO:tasks.workunit.client.0.smithi042.stdout:0/289: dread d4/d9/f22 [0,4194304] 0 2024-02-15T18:26:54.632 INFO:tasks.workunit.client.0.smithi042.stdout:6/333: rename d9/d11/d19/d35/l44 to d9/d11/d19/d1c/d31/l6b 0 2024-02-15T18:26:54.633 INFO:tasks.workunit.client.0.smithi042.stdout:5/271: dread d0/db/f1f [0,4194304] 0 2024-02-15T18:26:54.634 INFO:tasks.workunit.client.0.smithi042.stdout:4/228: symlink da/d3a/l4e 0 2024-02-15T18:26:54.634 INFO:tasks.workunit.client.0.smithi042.stdout:3/298: dwrite d1/d11/f45 [4194304,4194304] 0 2024-02-15T18:26:54.634 INFO:tasks.workunit.client.0.smithi042.stdout:3/299: fsync d1/de/d1a/f2f 0 2024-02-15T18:26:54.634 INFO:tasks.workunit.client.0.smithi042.stdout:9/245: write d6/f2e [127581,68535] 0 2024-02-15T18:26:54.634 INFO:tasks.workunit.client.0.smithi042.stdout:8/215: dwrite dd/de/ff [0,4194304] 0 2024-02-15T18:26:54.634 INFO:tasks.workunit.client.0.smithi042.stdout:8/216: read dd/d23/f28 [1356873,42860] 0 2024-02-15T18:26:54.634 INFO:tasks.workunit.client.0.smithi042.stdout:8/217: read - dd/de/d33/f37 zero size 2024-02-15T18:26:54.645 INFO:tasks.workunit.client.0.smithi042.stdout:1/274: write de/d15/d2e/f2a [265837,87701] 0 2024-02-15T18:26:54.649 INFO:tasks.workunit.client.0.smithi042.stdout:7/268: mknod d10/d49/c5c 0 2024-02-15T18:26:54.651 INFO:tasks.workunit.client.0.smithi042.stdout:8/218: dwrite dd/f29 [0,4194304] 0 2024-02-15T18:26:54.651 INFO:tasks.workunit.client.0.smithi042.stdout:0/290: creat d4/d38/d5c/f61 x:0 0 0 2024-02-15T18:26:54.655 INFO:tasks.workunit.client.0.smithi042.stdout:6/334: rename d9/d11/d19/d1c/f27 to d9/d11/d19/d1c/f6c 0 2024-02-15T18:26:54.655 INFO:tasks.workunit.client.0.smithi042.stdout:6/335: dread - d9/d11/d19/d35/d49/f52 zero size 2024-02-15T18:26:54.656 INFO:tasks.workunit.client.0.smithi042.stdout:5/272: mknod d0/d12/c69 0 2024-02-15T18:26:54.656 INFO:tasks.workunit.client.0.smithi042.stdout:5/273: write d0/d22/d2d/f56 [1917742,67458] 0 2024-02-15T18:26:54.656 INFO:tasks.workunit.client.0.smithi042.stdout:5/274: truncate d0/d14/f28 2182927 0 2024-02-15T18:26:54.657 INFO:tasks.workunit.client.0.smithi042.stdout:5/275: dread - d0/d38/f5a zero size 2024-02-15T18:26:54.658 INFO:tasks.workunit.client.0.smithi042.stdout:4/229: mkdir da/d11/d29/d4a/d4f 0 2024-02-15T18:26:54.658 INFO:tasks.workunit.client.0.smithi042.stdout:4/230: fsync da/d3b/d41/f49 0 2024-02-15T18:26:54.658 INFO:tasks.workunit.client.0.smithi042.stdout:4/231: truncate da/d2b/f37 372255 0 2024-02-15T18:26:54.659 INFO:tasks.workunit.client.0.smithi042.stdout:3/300: link d1/de/f42 d1/de/d1a/f5c 0 2024-02-15T18:26:54.661 INFO:tasks.workunit.client.0.smithi042.stdout:9/246: creat d6/dd/d4f/f5b x:0 0 0 2024-02-15T18:26:54.664 INFO:tasks.workunit.client.0.smithi042.stdout:1/275: mknod de/d15/d1b/c61 0 2024-02-15T18:26:54.664 INFO:tasks.workunit.client.0.smithi042.stdout:1/276: dread - de/d15/f43 zero size 2024-02-15T18:26:54.666 INFO:tasks.workunit.client.0.smithi042.stdout:7/269: link d10/d43/f4f d10/d15/d39/f5d 0 2024-02-15T18:26:54.666 INFO:tasks.workunit.client.0.smithi042.stdout:7/270: write d10/d12/f3f [4957504,117694] 0 2024-02-15T18:26:54.670 INFO:tasks.workunit.client.0.smithi042.stdout:8/219: symlink dd/de/d22/l38 0 2024-02-15T18:26:54.670 INFO:tasks.workunit.client.0.smithi042.stdout:9/247: dread d6/da/d1d/d2d/f39 [0,4194304] 0 2024-02-15T18:26:54.670 INFO:tasks.workunit.client.0.smithi042.stdout:9/248: fdatasync d6/da/d2f/d55/f44 0 2024-02-15T18:26:54.671 INFO:tasks.workunit.client.0.smithi042.stdout:2/227: rename d0/d8/db/de/c2a to d0/d8/db/c46 0 2024-02-15T18:26:54.671 INFO:tasks.workunit.client.0.smithi042.stdout:2/228: read d0/d8/db/de/d1a/f43 [440923,10791] 0 2024-02-15T18:26:54.672 INFO:tasks.workunit.client.0.smithi042.stdout:6/336: symlink d9/d11/d19/d1c/d31/d4f/l6d 0 2024-02-15T18:26:54.673 INFO:tasks.workunit.client.0.smithi042.stdout:4/232: unlink da/d11/l19 0 2024-02-15T18:26:54.675 INFO:tasks.workunit.client.0.smithi042.stdout:3/301: mkdir d1/d11/d5d 0 2024-02-15T18:26:54.676 INFO:tasks.workunit.client.0.smithi042.stdout:1/277: stat de/d15/c30 0 2024-02-15T18:26:54.676 INFO:tasks.workunit.client.0.smithi042.stdout:1/278: chown c3 114 1 2024-02-15T18:26:54.676 INFO:tasks.workunit.client.0.smithi042.stdout:1/279: write de/d15/d2c/f41 [408460,65786] 0 2024-02-15T18:26:54.676 INFO:tasks.workunit.client.0.smithi042.stdout:9/249: stat d6/f17 0 2024-02-15T18:26:54.678 INFO:tasks.workunit.client.0.smithi042.stdout:5/276: dwrite d0/db/fd [0,4194304] 0 2024-02-15T18:26:54.679 INFO:tasks.workunit.client.0.smithi042.stdout:0/291: rename d4/d16/d2e/d4c to d4/d9/d62 0 2024-02-15T18:26:54.681 INFO:tasks.workunit.client.0.smithi042.stdout:2/229: mknod d0/d8/d20/c47 0 2024-02-15T18:26:54.683 INFO:tasks.workunit.client.0.smithi042.stdout:6/337: symlink d9/d11/d19/d35/d49/l6e 0 2024-02-15T18:26:54.683 INFO:tasks.workunit.client.0.smithi042.stdout:8/220: dread dd/de/f1a [0,4194304] 0 2024-02-15T18:26:54.683 INFO:tasks.workunit.client.0.smithi042.stdout:8/221: chown dd/de/d12/f24 775006 1 2024-02-15T18:26:54.690 INFO:tasks.workunit.client.0.smithi042.stdout:2/230: dread d0/d8/f28 [0,4194304] 0 2024-02-15T18:26:54.690 INFO:tasks.workunit.client.0.smithi042.stdout:2/231: readlink d0/d8/d1b/l22 0 2024-02-15T18:26:54.690 INFO:tasks.workunit.client.0.smithi042.stdout:2/232: readlink d0/d8/d1b/l26 0 2024-02-15T18:26:54.690 INFO:tasks.workunit.client.0.smithi042.stdout:2/233: fsync d0/d8/f11 0 2024-02-15T18:26:54.697 INFO:tasks.workunit.client.0.smithi042.stdout:4/233: dread da/d11/d29/f32 [0,4194304] 0 2024-02-15T18:26:54.697 INFO:tasks.workunit.client.0.smithi042.stdout:1/280: symlink de/d15/d1b/d44/l62 0 2024-02-15T18:26:54.698 INFO:tasks.workunit.client.0.smithi042.stdout:5/277: dread d0/d50/f5f [0,4194304] 0 2024-02-15T18:26:54.698 INFO:tasks.workunit.client.0.smithi042.stdout:5/278: stat d0/d14/f2e 0 2024-02-15T18:26:54.698 INFO:tasks.workunit.client.0.smithi042.stdout:5/279: stat d0/db/d41/c60 0 2024-02-15T18:26:54.699 INFO:tasks.workunit.client.0.smithi042.stdout:9/250: mkdir d6/dd/d9/d59/d5c 0 2024-02-15T18:26:54.699 INFO:tasks.workunit.client.0.smithi042.stdout:9/251: write f2 [2135760,53889] 0 2024-02-15T18:26:54.700 INFO:tasks.workunit.client.0.smithi042.stdout:9/252: fdatasync d6/da/f29 0 2024-02-15T18:26:54.701 INFO:tasks.workunit.client.0.smithi042.stdout:6/338: dwrite d9/f2d [0,4194304] 0 2024-02-15T18:26:54.702 INFO:tasks.workunit.client.0.smithi042.stdout:7/271: rename d10/d15/d39/f56 to d10/d43/d57/f5e 0 2024-02-15T18:26:54.702 INFO:tasks.workunit.client.0.smithi042.stdout:7/272: chown d10/c1e 16246466 1 2024-02-15T18:26:54.704 INFO:tasks.workunit.client.0.smithi042.stdout:0/292: rmdir d4/d9/d15 39 2024-02-15T18:26:54.717 INFO:tasks.workunit.client.0.smithi042.stdout:2/234: link d0/d8/d1b/l26 d0/d8/db/de/d1a/l48 0 2024-02-15T18:26:54.718 INFO:tasks.workunit.client.0.smithi042.stdout:4/234: unlink da/d11/d27/c43 0 2024-02-15T18:26:54.719 INFO:tasks.workunit.client.0.smithi042.stdout:2/235: dread d0/d8/f39 [0,4194304] 0 2024-02-15T18:26:54.719 INFO:tasks.workunit.client.0.smithi042.stdout:2/236: write d0/d8/f39 [352228,60573] 0 2024-02-15T18:26:54.720 INFO:tasks.workunit.client.0.smithi042.stdout:1/281: rmdir de/d15/d2c/d51/d60 0 2024-02-15T18:26:54.721 INFO:tasks.workunit.client.0.smithi042.stdout:1/282: write de/d15/d2c/f5a [488945,114782] 0 2024-02-15T18:26:54.722 INFO:tasks.workunit.client.0.smithi042.stdout:5/280: rmdir d0/d39 39 2024-02-15T18:26:54.726 INFO:tasks.workunit.client.0.smithi042.stdout:8/222: dwrite dd/d11/f15 [0,4194304] 0 2024-02-15T18:26:54.726 INFO:tasks.workunit.client.0.smithi042.stdout:8/223: chown l8 406827 1 2024-02-15T18:26:54.727 INFO:tasks.workunit.client.0.smithi042.stdout:6/339: rmdir d9/d11/d19/d1c 39 2024-02-15T18:26:54.728 INFO:tasks.workunit.client.0.smithi042.stdout:3/302: rename d1/d11/d14/c4a to d1/d11/d2e/c5e 0 2024-02-15T18:26:54.730 INFO:tasks.workunit.client.0.smithi042.stdout:7/273: getdents d10 0 2024-02-15T18:26:54.732 INFO:tasks.workunit.client.0.smithi042.stdout:3/303: dread d1/d11/d14/f47 [0,4194304] 0 2024-02-15T18:26:54.732 INFO:tasks.workunit.client.0.smithi042.stdout:0/293: write d4/f7 [558906,48535] 0 2024-02-15T18:26:54.741 INFO:tasks.workunit.client.0.smithi042.stdout:4/235: creat da/d11/d29/d4a/d4f/f50 x:0 0 0 2024-02-15T18:26:54.743 INFO:tasks.workunit.client.0.smithi042.stdout:2/237: unlink d0/d8/f11 0 2024-02-15T18:26:54.746 INFO:tasks.workunit.client.0.smithi042.stdout:7/274: dwrite d10/f26 [0,4194304] 0 2024-02-15T18:26:54.746 INFO:tasks.workunit.client.0.smithi042.stdout:6/340: creat d9/d11/d19/d1c/d31/f6f x:0 0 0 2024-02-15T18:26:54.748 INFO:tasks.workunit.client.0.smithi042.stdout:0/294: dread d4/f6 [0,4194304] 0 2024-02-15T18:26:54.748 INFO:tasks.workunit.client.0.smithi042.stdout:9/253: rename d6/da/d2f/f57 to d6/dd/f5d 0 2024-02-15T18:26:54.749 INFO:tasks.workunit.client.0.smithi042.stdout:9/254: read f3 [1056510,411] 0 2024-02-15T18:26:54.750 INFO:tasks.workunit.client.0.smithi042.stdout:4/236: creat da/d11/d29/d4b/f51 x:0 0 0 2024-02-15T18:26:54.750 INFO:tasks.workunit.client.0.smithi042.stdout:4/237: truncate da/d11/fe 4380741 0 2024-02-15T18:26:54.752 INFO:tasks.workunit.client.0.smithi042.stdout:2/238: rename d0/f34 to d0/f49 0 2024-02-15T18:26:54.752 INFO:tasks.workunit.client.0.smithi042.stdout:2/239: write d0/d8/f28 [4899039,118674] 0 2024-02-15T18:26:54.753 INFO:tasks.workunit.client.0.smithi042.stdout:1/283: mkdir de/d15/d2e/d63 0 2024-02-15T18:26:54.753 INFO:tasks.workunit.client.0.smithi042.stdout:1/284: chown de/d15/d1b 525410961 1 2024-02-15T18:26:54.756 INFO:tasks.workunit.client.0.smithi042.stdout:8/224: link dd/f2f dd/d11/f39 0 2024-02-15T18:26:54.760 INFO:tasks.workunit.client.0.smithi042.stdout:6/341: mkdir d9/d11/d70 0 2024-02-15T18:26:54.763 INFO:tasks.workunit.client.0.smithi042.stdout:5/281: rename d0/d22/d2d to d0/d38/d53/d68/d6a 0 2024-02-15T18:26:54.763 INFO:tasks.workunit.client.0.smithi042.stdout:5/282: readlink d0/d12/d65/l67 0 2024-02-15T18:26:54.766 INFO:tasks.workunit.client.0.smithi042.stdout:9/255: unlink d6/da/d1d/d2d/c37 0 2024-02-15T18:26:54.767 INFO:tasks.workunit.client.0.smithi042.stdout:4/238: creat da/d2b/f52 x:0 0 0 2024-02-15T18:26:54.769 INFO:tasks.workunit.client.0.smithi042.stdout:1/285: link de/d15/c4d de/d15/c64 0 2024-02-15T18:26:54.769 INFO:tasks.workunit.client.0.smithi042.stdout:0/295: dwrite d4/d9/d34/f4b [0,4194304] 0 2024-02-15T18:26:54.772 INFO:tasks.workunit.client.0.smithi042.stdout:9/256: write d6/dd/d9/f45 [2522823,61480] 0 2024-02-15T18:26:54.777 INFO:tasks.workunit.client.0.smithi042.stdout:8/225: creat dd/de/d22/f3a x:0 0 0 2024-02-15T18:26:54.783 INFO:tasks.workunit.client.0.smithi042.stdout:0/296: dwrite d4/d9/d15/f21 [4194304,4194304] 0 2024-02-15T18:26:54.783 INFO:tasks.workunit.client.0.smithi042.stdout:0/297: read - d4/d9/d62/f53 zero size 2024-02-15T18:26:54.791 INFO:tasks.workunit.client.0.smithi042.stdout:6/342: mknod d9/d11/d19/d35/d55/c71 0 2024-02-15T18:26:54.797 INFO:tasks.workunit.client.0.smithi042.stdout:3/304: rename d1/d11/d14/l49 to d1/l5f 0 2024-02-15T18:26:54.797 INFO:tasks.workunit.client.0.smithi042.stdout:3/305: truncate d1/f7 3363457 0 2024-02-15T18:26:54.797 INFO:tasks.workunit.client.0.smithi042.stdout:3/306: chown d1/de 847127 1 2024-02-15T18:26:54.814 INFO:tasks.workunit.client.0.smithi042.stdout:1/286: dwrite de/d15/d2e/f32 [0,4194304] 0 2024-02-15T18:26:54.816 INFO:tasks.workunit.client.0.smithi042.stdout:1/287: dread de/f29 [0,4194304] 0 2024-02-15T18:26:54.827 INFO:tasks.workunit.client.0.smithi042.stdout:9/257: creat d6/dd/d9/d59/f5e x:0 0 0 2024-02-15T18:26:54.831 INFO:tasks.workunit.client.0.smithi042.stdout:8/226: mkdir dd/de/d33/d3b 0 2024-02-15T18:26:54.836 INFO:tasks.workunit.client.0.smithi042.stdout:6/343: creat d9/f72 x:0 0 0 2024-02-15T18:26:54.838 INFO:tasks.workunit.client.0.smithi042.stdout:7/275: rename d10/f26 to d10/d12/f5f 0 2024-02-15T18:26:54.838 INFO:tasks.workunit.client.0.smithi042.stdout:9/258: dwrite d6/da/d2f/d55/f44 [0,4194304] 0 2024-02-15T18:26:54.842 INFO:tasks.workunit.client.0.smithi042.stdout:1/288: symlink de/d54/l65 0 2024-02-15T18:26:54.842 INFO:tasks.workunit.client.0.smithi042.stdout:1/289: chown de/d15/c4d 0 1 2024-02-15T18:26:54.842 INFO:tasks.workunit.client.0.smithi042.stdout:1/290: write de/f4e [861200,60492] 0 2024-02-15T18:26:54.850 INFO:tasks.workunit.client.0.smithi042.stdout:7/276: dread d10/d15/d39/f3a [0,4194304] 0 2024-02-15T18:26:54.850 INFO:tasks.workunit.client.0.smithi042.stdout:7/277: stat d10/d15/l3e 0 2024-02-15T18:26:54.850 INFO:tasks.workunit.client.0.smithi042.stdout:7/278: fdatasync d10/d15/f34 0 2024-02-15T18:26:54.850 INFO:tasks.workunit.client.0.smithi042.stdout:7/279: truncate d10/d12/d23/f27 4870173 0 2024-02-15T18:26:54.850 INFO:tasks.workunit.client.0.smithi042.stdout:8/227: mkdir dd/de/d3c 0 2024-02-15T18:26:54.850 INFO:tasks.workunit.client.0.smithi042.stdout:7/280: read d10/d15/d39/f3a [2410833,46559] 0 2024-02-15T18:26:54.850 INFO:tasks.workunit.client.0.smithi042.stdout:0/298: symlink d4/d16/d2e/l63 0 2024-02-15T18:26:54.854 INFO:tasks.workunit.client.0.smithi042.stdout:6/344: creat d9/d11/d19/d35/f73 x:0 0 0 2024-02-15T18:26:54.855 INFO:tasks.workunit.client.0.smithi042.stdout:6/345: write d9/d11/f20 [1222732,66097] 0 2024-02-15T18:26:54.855 INFO:tasks.workunit.client.0.smithi042.stdout:6/346: truncate d9/f66 876298 0 2024-02-15T18:26:54.856 INFO:tasks.workunit.client.0.smithi042.stdout:5/283: rename d0/d38/d53/d68/d6a/d42/l5c to d0/d38/d53/l6b 0 2024-02-15T18:26:54.856 INFO:tasks.workunit.client.0.smithi042.stdout:1/291: dwrite de/d15/f31 [0,4194304] 0 2024-02-15T18:26:54.859 INFO:tasks.workunit.client.0.smithi042.stdout:5/284: dread d0/fe [0,4194304] 0 2024-02-15T18:26:54.860 INFO:tasks.workunit.client.0.smithi042.stdout:3/307: link d1/de/d1a/f5c d1/d11/d14/f60 0 2024-02-15T18:26:54.861 INFO:tasks.workunit.client.0.smithi042.stdout:4/239: sync 2024-02-15T18:26:54.861 INFO:tasks.workunit.client.0.smithi042.stdout:2/240: sync 2024-02-15T18:26:54.861 INFO:tasks.workunit.client.0.smithi042.stdout:4/240: chown da/d11/d27/c28 8422185 1 2024-02-15T18:26:54.862 INFO:tasks.workunit.client.0.smithi042.stdout:9/259: mkdir d6/da/d1d/d2d/d40/d5f 0 2024-02-15T18:26:54.863 INFO:tasks.workunit.client.0.smithi042.stdout:1/292: dread de/d15/d2e/f3b [0,4194304] 0 2024-02-15T18:26:54.866 INFO:tasks.workunit.client.0.smithi042.stdout:0/299: creat d4/d9/d5a/d60/f64 x:0 0 0 2024-02-15T18:26:54.868 INFO:tasks.workunit.client.0.smithi042.stdout:6/347: mknod d9/d11/d46/c74 0 2024-02-15T18:26:54.870 INFO:tasks.workunit.client.0.smithi042.stdout:8/228: rename dd/de/c1c to dd/de/d33/d3b/c3d 0 2024-02-15T18:26:54.871 INFO:tasks.workunit.client.0.smithi042.stdout:7/281: dwrite d10/d12/d23/f4a [0,4194304] 0 2024-02-15T18:26:54.871 INFO:tasks.workunit.client.0.smithi042.stdout:7/282: fsync d10/d12/d23/f55 0 2024-02-15T18:26:54.873 INFO:tasks.workunit.client.0.smithi042.stdout:5/285: symlink d0/d39/l6c 0 2024-02-15T18:26:54.878 INFO:tasks.workunit.client.0.smithi042.stdout:3/308: unlink d1/de/d1a/f2b 0 2024-02-15T18:26:54.880 INFO:tasks.workunit.client.0.smithi042.stdout:9/260: dread f3 [0,4194304] 0 2024-02-15T18:26:54.882 INFO:tasks.workunit.client.0.smithi042.stdout:4/241: unlink da/d11/d29/d4a/d4f/f50 0 2024-02-15T18:26:54.884 INFO:tasks.workunit.client.0.smithi042.stdout:2/241: creat d0/d8/db/de/d1a/d2e/f4a x:0 0 0 2024-02-15T18:26:54.886 INFO:tasks.workunit.client.0.smithi042.stdout:8/229: creat dd/f3e x:0 0 0 2024-02-15T18:26:54.889 INFO:tasks.workunit.client.0.smithi042.stdout:7/283: symlink d10/d15/d39/l60 0 2024-02-15T18:26:54.889 INFO:tasks.workunit.client.0.smithi042.stdout:7/284: fdatasync d10/f21 0 2024-02-15T18:26:54.889 INFO:tasks.workunit.client.0.smithi042.stdout:7/285: chown d10/d12/f3f 45200 1 2024-02-15T18:26:54.897 INFO:tasks.workunit.client.0.smithi042.stdout:5/286: creat d0/db/d41/f6d x:0 0 0 2024-02-15T18:26:54.898 INFO:tasks.workunit.client.0.smithi042.stdout:5/287: dread d0/d22/f4a [0,4194304] 0 2024-02-15T18:26:54.898 INFO:tasks.workunit.client.0.smithi042.stdout:5/288: chown d0/d38/d53/d68/d6a/c5d 656 1 2024-02-15T18:26:54.898 INFO:tasks.workunit.client.0.smithi042.stdout:3/309: symlink d1/d11/l61 0 2024-02-15T18:26:54.899 INFO:tasks.workunit.client.0.smithi042.stdout:4/242: mkdir da/d11/d1b/d53 0 2024-02-15T18:26:54.899 INFO:tasks.workunit.client.0.smithi042.stdout:4/243: readlink da/l18 0 2024-02-15T18:26:54.900 INFO:tasks.workunit.client.0.smithi042.stdout:2/242: mknod d0/d8/d10/c4b 0 2024-02-15T18:26:54.901 INFO:tasks.workunit.client.0.smithi042.stdout:1/293: mkdir de/d15/d2e/d66 0 2024-02-15T18:26:54.905 INFO:tasks.workunit.client.0.smithi042.stdout:5/289: rename d0/fa to d0/d38/d53/d68/f6e 0 2024-02-15T18:26:54.905 INFO:tasks.workunit.client.0.smithi042.stdout:5/290: dread - d0/d12/d61/f66 zero size 2024-02-15T18:26:54.907 INFO:tasks.workunit.client.0.smithi042.stdout:4/244: mknod da/d3b/c54 0 2024-02-15T18:26:54.908 INFO:tasks.workunit.client.0.smithi042.stdout:2/243: creat d0/d8/db/de/d1a/f4c x:0 0 0 2024-02-15T18:26:54.908 INFO:tasks.workunit.client.0.smithi042.stdout:2/244: stat d0/d8/db/de/d1a/d2e/f4a 0 2024-02-15T18:26:54.909 INFO:tasks.workunit.client.0.smithi042.stdout:2/245: write d0/d8/db/f3e [289647,3981] 0 2024-02-15T18:26:54.910 INFO:tasks.workunit.client.0.smithi042.stdout:1/294: unlink de/d15/l33 0 2024-02-15T18:26:54.912 INFO:tasks.workunit.client.0.smithi042.stdout:5/291: symlink d0/db/d41/d52/l6f 0 2024-02-15T18:26:54.914 INFO:tasks.workunit.client.0.smithi042.stdout:4/245: symlink da/d11/d1b/d53/l55 0 2024-02-15T18:26:54.914 INFO:tasks.workunit.client.0.smithi042.stdout:3/310: dwrite d1/de/d19/f4b [0,4194304] 0 2024-02-15T18:26:54.914 INFO:tasks.workunit.client.0.smithi042.stdout:3/311: dread - d1/d11/d2e/f54 zero size 2024-02-15T18:26:54.915 INFO:tasks.workunit.client.0.smithi042.stdout:7/286: getdents d10/d15/d39 0 2024-02-15T18:26:54.919 INFO:tasks.workunit.client.0.smithi042.stdout:2/246: unlink d0/d8/d20/l2d 0 2024-02-15T18:26:54.920 INFO:tasks.workunit.client.0.smithi042.stdout:2/247: dread d0/d8/db/de/d1a/f43 [0,4194304] 0 2024-02-15T18:26:54.920 INFO:tasks.workunit.client.0.smithi042.stdout:2/248: chown d0/d8/d1b/l22 384292455 1 2024-02-15T18:26:54.922 INFO:tasks.workunit.client.0.smithi042.stdout:5/292: link d0/c8 d0/d22/c70 0 2024-02-15T18:26:54.923 INFO:tasks.workunit.client.0.smithi042.stdout:4/246: mkdir da/d11/d29/d56 0 2024-02-15T18:26:54.924 INFO:tasks.workunit.client.0.smithi042.stdout:5/293: dread d0/d14/f2e [0,4194304] 0 2024-02-15T18:26:54.925 INFO:tasks.workunit.client.0.smithi042.stdout:4/247: dread da/d11/d1b/f3e [0,4194304] 0 2024-02-15T18:26:54.925 INFO:tasks.workunit.client.0.smithi042.stdout:4/248: read da/d11/d1b/d2f/d4d/f47 [208323,41541] 0 2024-02-15T18:26:54.928 INFO:tasks.workunit.client.0.smithi042.stdout:3/312: creat d1/d11/d5d/f62 x:0 0 0 2024-02-15T18:26:54.930 INFO:tasks.workunit.client.0.smithi042.stdout:0/300: dwrite d4/f7 [0,4194304] 0 2024-02-15T18:26:54.930 INFO:tasks.workunit.client.0.smithi042.stdout:0/301: write d4/d9/f31 [310234,43693] 0 2024-02-15T18:26:54.934 INFO:tasks.workunit.client.0.smithi042.stdout:2/249: dread d0/d8/db/f3e [0,4194304] 0 2024-02-15T18:26:54.934 INFO:tasks.workunit.client.0.smithi042.stdout:0/302: write d4/d9/d15/f21 [1039078,73685] 0 2024-02-15T18:26:54.934 INFO:tasks.workunit.client.0.smithi042.stdout:2/250: read d0/d8/d10/f17 [42862,46748] 0 2024-02-15T18:26:54.935 INFO:tasks.workunit.client.0.smithi042.stdout:2/251: dread - d0/d8/db/f42 zero size 2024-02-15T18:26:54.936 INFO:tasks.workunit.client.0.smithi042.stdout:1/295: dwrite de/d15/d2e/f3f [0,4194304] 0 2024-02-15T18:26:54.936 INFO:tasks.workunit.client.0.smithi042.stdout:5/294: creat d0/d38/d53/d68/d6a/d42/f71 x:0 0 0 2024-02-15T18:26:54.938 INFO:tasks.workunit.client.0.smithi042.stdout:3/313: mknod d1/de/d19/c63 0 2024-02-15T18:26:54.938 INFO:tasks.workunit.client.0.smithi042.stdout:5/295: write d0/db/f10 [3542382,21967] 0 2024-02-15T18:26:54.938 INFO:tasks.workunit.client.0.smithi042.stdout:3/314: dread d1/d11/d14/f35 [0,4194304] 0 2024-02-15T18:26:54.940 INFO:tasks.workunit.client.0.smithi042.stdout:0/303: symlink d4/d9/d62/l65 0 2024-02-15T18:26:54.942 INFO:tasks.workunit.client.0.smithi042.stdout:7/287: link d10/d12/d23/f3b d10/d12/d23/f61 0 2024-02-15T18:26:54.951 INFO:tasks.workunit.client.0.smithi042.stdout:2/252: link d0/d8/f28 d0/d8/db/de/f4d 0 2024-02-15T18:26:54.953 INFO:tasks.workunit.client.0.smithi042.stdout:7/288: write d10/d15/d39/f5d [139080,62324] 0 2024-02-15T18:26:54.956 INFO:tasks.workunit.client.0.smithi042.stdout:0/304: mknod d4/d16/d2e/c66 0 2024-02-15T18:26:54.958 INFO:tasks.workunit.client.0.smithi042.stdout:7/289: getdents d10/d43 0 2024-02-15T18:26:54.958 INFO:tasks.workunit.client.0.smithi042.stdout:9/261: dwrite f2 [0,4194304] 0 2024-02-15T18:26:54.960 INFO:tasks.workunit.client.0.smithi042.stdout:7/290: mknod d10/d12/d23/d51/c62 0 2024-02-15T18:26:54.960 INFO:tasks.workunit.client.0.smithi042.stdout:7/291: chown d10/d43/f4e 144 1 2024-02-15T18:26:54.961 INFO:tasks.workunit.client.0.smithi042.stdout:9/262: getdents d6/da/d1d 0 2024-02-15T18:26:54.965 INFO:tasks.workunit.client.0.smithi042.stdout:7/292: link d10/d15/d39/f3a d10/f63 0 2024-02-15T18:26:54.965 INFO:tasks.workunit.client.0.smithi042.stdout:8/230: dwrite dd/de/f1a [4194304,4194304] 0 2024-02-15T18:26:54.966 INFO:tasks.workunit.client.0.smithi042.stdout:8/231: write dd/de/d22/f2b [794769,45304] 0 2024-02-15T18:26:54.970 INFO:tasks.workunit.client.0.smithi042.stdout:8/232: creat dd/de/d12/d32/f3f x:0 0 0 2024-02-15T18:26:54.973 INFO:tasks.workunit.client.0.smithi042.stdout:7/293: rename d10/f22 to d10/d12/d23/f64 0 2024-02-15T18:26:54.974 INFO:tasks.workunit.client.0.smithi042.stdout:7/294: write d10/d12/d23/f61 [154340,119195] 0 2024-02-15T18:26:54.976 INFO:tasks.workunit.client.0.smithi042.stdout:8/233: creat dd/de/d33/f40 x:0 0 0 2024-02-15T18:26:54.976 INFO:tasks.workunit.client.0.smithi042.stdout:1/296: dwrite de/d15/d2e/f3f [0,4194304] 0 2024-02-15T18:26:54.978 INFO:tasks.workunit.client.0.smithi042.stdout:7/295: truncate d10/d12/f20 3194499 0 2024-02-15T18:26:54.979 INFO:tasks.workunit.client.0.smithi042.stdout:8/234: symlink dd/de/d22/l41 0 2024-02-15T18:26:54.983 INFO:tasks.workunit.client.0.smithi042.stdout:7/296: write d10/d43/f58 [927765,50685] 0 2024-02-15T18:26:54.983 INFO:tasks.workunit.client.0.smithi042.stdout:7/297: readlink d10/d15/d39/l52 0 2024-02-15T18:26:54.983 INFO:tasks.workunit.client.0.smithi042.stdout:7/298: chown d10/d15/l3e 0 1 2024-02-15T18:26:54.983 INFO:tasks.workunit.client.0.smithi042.stdout:7/299: write d10/d12/d23/f27 [1851054,45622] 0 2024-02-15T18:26:54.984 INFO:tasks.workunit.client.0.smithi042.stdout:1/297: rename de/d15/d2e to de/d15/d67 0 2024-02-15T18:26:54.986 INFO:tasks.workunit.client.0.smithi042.stdout:8/235: dread dd/d11/f21 [0,4194304] 0 2024-02-15T18:26:54.987 INFO:tasks.workunit.client.0.smithi042.stdout:8/236: fdatasync dd/de/d22/f2d 0 2024-02-15T18:26:54.987 INFO:tasks.workunit.client.0.smithi042.stdout:8/237: write dd/f13 [486091,12039] 0 2024-02-15T18:26:54.987 INFO:tasks.workunit.client.0.smithi042.stdout:5/296: dwrite d0/db/f1f [0,4194304] 0 2024-02-15T18:26:54.988 INFO:tasks.workunit.client.0.smithi042.stdout:2/253: dwrite d0/f7 [0,4194304] 0 2024-02-15T18:26:54.988 INFO:tasks.workunit.client.0.smithi042.stdout:3/315: dwrite d1/f7 [0,4194304] 0 2024-02-15T18:26:54.988 INFO:tasks.workunit.client.0.smithi042.stdout:0/305: dwrite d4/f41 [0,4194304] 0 2024-02-15T18:26:54.989 INFO:tasks.workunit.client.0.smithi042.stdout:7/300: mkdir d10/d12/d65 0 2024-02-15T18:26:54.989 INFO:tasks.workunit.client.0.smithi042.stdout:1/298: dread de/d15/d2c/f5a [0,4194304] 0 2024-02-15T18:26:54.990 INFO:tasks.workunit.client.0.smithi042.stdout:9/263: dwrite d6/dd/d9/f18 [0,4194304] 0 2024-02-15T18:26:54.990 INFO:tasks.workunit.client.0.smithi042.stdout:8/238: truncate dd/de/d22/f2d 1185489 0 2024-02-15T18:26:54.994 INFO:tasks.workunit.client.0.smithi042.stdout:8/239: fsync dd/de/d22/f2b 0 2024-02-15T18:26:55.010 INFO:tasks.workunit.client.0.smithi042.stdout:5/297: creat d0/db/d41/f72 x:0 0 0 2024-02-15T18:26:55.010 INFO:tasks.workunit.client.0.smithi042.stdout:5/298: fdatasync d0/db/d41/f4c 0 2024-02-15T18:26:55.021 INFO:tasks.workunit.client.0.smithi042.stdout:2/254: mknod d0/d8/d1b/d3c/c4e 0 2024-02-15T18:26:55.022 INFO:tasks.workunit.client.0.smithi042.stdout:2/255: write d0/d8/d1b/d3c/f30 [3051381,108028] 0 2024-02-15T18:26:55.022 INFO:tasks.workunit.client.0.smithi042.stdout:2/256: chown d0/d8/db/de/d1a 97378 1 2024-02-15T18:26:55.022 INFO:tasks.workunit.client.0.smithi042.stdout:2/257: write d0/d8/d10/f17 [683665,50981] 0 2024-02-15T18:26:55.022 INFO:tasks.workunit.client.0.smithi042.stdout:2/258: chown d0/f49 6328216 1 2024-02-15T18:26:55.022 INFO:tasks.workunit.client.0.smithi042.stdout:3/316: mknod d1/d11/d2e/d4c/c64 0 2024-02-15T18:26:55.062 INFO:tasks.workunit.client.0.smithi042.stdout:0/306: creat d4/d9/d34/d4d/f67 x:0 0 0 2024-02-15T18:26:55.063 INFO:tasks.workunit.client.0.smithi042.stdout:7/301: creat d10/d43/d57/f66 x:0 0 0 2024-02-15T18:26:55.064 INFO:tasks.workunit.client.0.smithi042.stdout:7/302: write d10/d15/f36 [1496066,119082] 0 2024-02-15T18:26:55.064 INFO:tasks.workunit.client.0.smithi042.stdout:1/299: creat de/d15/d67/d46/f68 x:0 0 0 2024-02-15T18:26:55.065 INFO:tasks.workunit.client.0.smithi042.stdout:1/300: write de/f5b [2598203,121424] 0 2024-02-15T18:26:55.066 INFO:tasks.workunit.client.0.smithi042.stdout:7/303: dread d10/d15/d39/f5d [0,4194304] 0 2024-02-15T18:26:55.066 INFO:tasks.workunit.client.0.smithi042.stdout:7/304: chown d10/l2b 50524784 1 2024-02-15T18:26:55.066 INFO:tasks.workunit.client.0.smithi042.stdout:8/240: rmdir dd/d23 39 2024-02-15T18:26:55.066 INFO:tasks.workunit.client.0.smithi042.stdout:8/241: stat dd/de/d12/l27 0 2024-02-15T18:26:55.067 INFO:tasks.workunit.client.0.smithi042.stdout:5/299: link d0/d39/c4f d0/d38/c73 0 2024-02-15T18:26:55.068 INFO:tasks.workunit.client.0.smithi042.stdout:3/317: rmdir d1/d11/d14/d4d 39 2024-02-15T18:26:55.074 INFO:tasks.workunit.client.0.smithi042.stdout:1/301: dread de/f5b [0,4194304] 0 2024-02-15T18:26:55.075 INFO:tasks.workunit.client.0.smithi042.stdout:0/307: truncate d4/d9/d15/f18 1743730 0 2024-02-15T18:26:55.078 INFO:tasks.workunit.client.0.smithi042.stdout:9/264: truncate d6/f21 2476477 0 2024-02-15T18:26:55.080 INFO:tasks.workunit.client.0.smithi042.stdout:7/305: creat d10/d49/f67 x:0 0 0 2024-02-15T18:26:55.080 INFO:tasks.workunit.client.0.smithi042.stdout:7/306: getdents d10/d12/d65 0 2024-02-15T18:26:55.089 INFO:tasks.workunit.client.0.smithi042.stdout:3/318: dwrite d1/d11/d5d/f62 [0,4194304] 0 2024-02-15T18:26:55.089 INFO:tasks.workunit.client.0.smithi042.stdout:3/319: chown d1/de/f42 1817780 1 2024-02-15T18:26:55.089 INFO:tasks.workunit.client.0.smithi042.stdout:6/348: sync 2024-02-15T18:26:55.089 INFO:tasks.workunit.client.0.smithi042.stdout:4/249: sync 2024-02-15T18:26:55.092 INFO:tasks.workunit.client.0.smithi042.stdout:8/242: creat dd/de/d33/f42 x:0 0 0 2024-02-15T18:26:55.094 INFO:tasks.workunit.client.0.smithi042.stdout:8/243: dread dd/d11/f21 [0,4194304] 0 2024-02-15T18:26:55.094 INFO:tasks.workunit.client.0.smithi042.stdout:8/244: read fc [175706,5634] 0 2024-02-15T18:26:55.095 INFO:tasks.workunit.client.0.smithi042.stdout:5/300: rename d0/db/d41 to d0/d12/d65/d74 0 2024-02-15T18:26:55.098 INFO:tasks.workunit.client.0.smithi042.stdout:7/307: mkdir d10/d49/d68 0 2024-02-15T18:26:55.098 INFO:tasks.workunit.client.0.smithi042.stdout:7/308: read - d10/d43/d57/f66 zero size 2024-02-15T18:26:55.104 INFO:tasks.workunit.client.0.smithi042.stdout:3/320: rmdir d1/d11/d2e 39 2024-02-15T18:26:55.104 INFO:tasks.workunit.client.0.smithi042.stdout:3/321: chown d1/de/d19/f4b 5 1 2024-02-15T18:26:55.104 INFO:tasks.workunit.client.0.smithi042.stdout:8/245: creat dd/de/d22/f43 x:0 0 0 2024-02-15T18:26:55.105 INFO:tasks.workunit.client.0.smithi042.stdout:6/349: dread d9/d3e/f63 [0,4194304] 0 2024-02-15T18:26:55.105 INFO:tasks.workunit.client.0.smithi042.stdout:7/309: creat d10/f69 x:0 0 0 2024-02-15T18:26:55.105 INFO:tasks.workunit.client.0.smithi042.stdout:7/310: write d10/d43/f50 [632716,89179] 0 2024-02-15T18:26:55.106 INFO:tasks.workunit.client.0.smithi042.stdout:3/322: creat d1/d11/f65 x:0 0 0 2024-02-15T18:26:55.108 INFO:tasks.workunit.client.0.smithi042.stdout:8/246: creat dd/de/d33/f44 x:0 0 0 2024-02-15T18:26:55.109 INFO:tasks.workunit.client.0.smithi042.stdout:6/350: creat d9/d3e/f75 x:0 0 0 2024-02-15T18:26:55.109 INFO:tasks.workunit.client.0.smithi042.stdout:5/301: dread d0/d12/f21 [0,4194304] 0 2024-02-15T18:26:55.111 INFO:tasks.workunit.client.0.smithi042.stdout:8/247: creat dd/de/d12/f45 x:0 0 0 2024-02-15T18:26:55.114 INFO:tasks.workunit.client.0.smithi042.stdout:6/351: rmdir d9/d11/d70 0 2024-02-15T18:26:55.114 INFO:tasks.workunit.client.0.smithi042.stdout:6/352: readlink d9/d11/d19/d1c/l4d 0 2024-02-15T18:26:55.116 INFO:tasks.workunit.client.0.smithi042.stdout:5/302: creat d0/d39/f75 x:0 0 0 2024-02-15T18:26:55.117 INFO:tasks.workunit.client.0.smithi042.stdout:7/311: mknod d10/d12/d23/c6a 0 2024-02-15T18:26:55.117 INFO:tasks.workunit.client.0.smithi042.stdout:7/312: write d10/d12/d23/f61 [1354334,118344] 0 2024-02-15T18:26:55.118 INFO:tasks.workunit.client.0.smithi042.stdout:7/313: fsync d10/f1b 0 2024-02-15T18:26:55.118 INFO:tasks.workunit.client.0.smithi042.stdout:7/314: chown d10/d15/f3d 22 1 2024-02-15T18:26:55.118 INFO:tasks.workunit.client.0.smithi042.stdout:8/248: creat dd/de/d33/d3b/f46 x:0 0 0 2024-02-15T18:26:55.119 INFO:tasks.workunit.client.0.smithi042.stdout:8/249: read dd/f13 [2539592,85878] 0 2024-02-15T18:26:55.119 INFO:tasks.workunit.client.0.smithi042.stdout:9/265: dwrite d6/da/f25 [0,4194304] 0 2024-02-15T18:26:55.121 INFO:tasks.workunit.client.0.smithi042.stdout:5/303: link d0/db/f1f d0/d12/f76 0 2024-02-15T18:26:55.122 INFO:tasks.workunit.client.0.smithi042.stdout:1/302: dwrite de/d15/d67/f32 [0,4194304] 0 2024-02-15T18:26:55.122 INFO:tasks.workunit.client.0.smithi042.stdout:0/308: dwrite d4/d38/f3c [4194304,4194304] 0 2024-02-15T18:26:55.124 INFO:tasks.workunit.client.0.smithi042.stdout:4/250: dwrite da/d11/d1b/f3e [0,4194304] 0 2024-02-15T18:26:55.125 INFO:tasks.workunit.client.0.smithi042.stdout:9/266: dread d6/f21 [0,4194304] 0 2024-02-15T18:26:55.125 INFO:tasks.workunit.client.0.smithi042.stdout:9/267: dread - d6/da/d1d/d2d/d40/f49 zero size 2024-02-15T18:26:55.125 INFO:tasks.workunit.client.0.smithi042.stdout:9/268: truncate d6/da/d2f/d55/f44 4591713 0 2024-02-15T18:26:55.125 INFO:tasks.workunit.client.0.smithi042.stdout:8/250: mknod dd/de/d12/c47 0 2024-02-15T18:26:55.129 INFO:tasks.workunit.client.0.smithi042.stdout:3/323: dwrite d1/f34 [0,4194304] 0 2024-02-15T18:26:55.134 INFO:tasks.workunit.client.0.smithi042.stdout:6/353: rename d9/d11/d46/f4b to d9/f76 0 2024-02-15T18:26:55.142 INFO:tasks.workunit.client.0.smithi042.stdout:5/304: getdents d0/d12/d2a 0 2024-02-15T18:26:55.143 INFO:tasks.workunit.client.0.smithi042.stdout:5/305: write d0/d12/f21 [757702,37] 0 2024-02-15T18:26:55.143 INFO:tasks.workunit.client.0.smithi042.stdout:5/306: chown d0/d38/d53/d68/d6a/d42/l44 3951836 1 2024-02-15T18:26:55.143 INFO:tasks.workunit.client.0.smithi042.stdout:5/307: chown d0/d22/d5b 567713156 1 2024-02-15T18:26:55.145 INFO:tasks.workunit.client.0.smithi042.stdout:0/309: creat d4/f68 x:0 0 0 2024-02-15T18:26:55.145 INFO:tasks.workunit.client.0.smithi042.stdout:0/310: write d4/d9/d15/f42 [4649996,31809] 0 2024-02-15T18:26:55.148 INFO:tasks.workunit.client.0.smithi042.stdout:4/251: stat da/d11/c13 0 2024-02-15T18:26:55.148 INFO:tasks.workunit.client.0.smithi042.stdout:4/252: stat da/f21 0 2024-02-15T18:26:55.148 INFO:tasks.workunit.client.0.smithi042.stdout:4/253: chown da/d11/d1b 238235679 1 2024-02-15T18:26:55.148 INFO:tasks.workunit.client.0.smithi042.stdout:1/303: dwrite f2 [0,4194304] 0 2024-02-15T18:26:55.149 INFO:tasks.workunit.client.0.smithi042.stdout:9/269: dwrite d6/dd/f19 [0,4194304] 0 2024-02-15T18:26:55.149 INFO:tasks.workunit.client.0.smithi042.stdout:8/251: getdents dd/de/d12/d32 0 2024-02-15T18:26:55.149 INFO:tasks.workunit.client.0.smithi042.stdout:8/252: write dd/de/d33/d3b/f46 [712159,36824] 0 2024-02-15T18:26:55.162 INFO:tasks.workunit.client.0.smithi042.stdout:7/315: rename d10/d15/f36 to d10/d12/f6b 0 2024-02-15T18:26:55.166 INFO:tasks.workunit.client.0.smithi042.stdout:6/354: getdents d9/d11/d19/d35/d55 0 2024-02-15T18:26:55.167 INFO:tasks.workunit.client.0.smithi042.stdout:5/308: unlink d0/d12/d2a/f3b 0 2024-02-15T18:26:55.167 INFO:tasks.workunit.client.0.smithi042.stdout:6/355: dread d9/d11/f2b [4194304,4194304] 0 2024-02-15T18:26:55.168 INFO:tasks.workunit.client.0.smithi042.stdout:4/254: symlink da/d3b/l57 0 2024-02-15T18:26:55.168 INFO:tasks.workunit.client.0.smithi042.stdout:4/255: chown da/d11/f39 15596 1 2024-02-15T18:26:55.171 INFO:tasks.workunit.client.0.smithi042.stdout:2/259: sync 2024-02-15T18:26:55.174 INFO:tasks.workunit.client.0.smithi042.stdout:8/253: creat dd/f48 x:0 0 0 2024-02-15T18:26:55.174 INFO:tasks.workunit.client.0.smithi042.stdout:8/254: stat dd/f2f 0 2024-02-15T18:26:55.174 INFO:tasks.workunit.client.0.smithi042.stdout:8/255: fsync dd/f2c 0 2024-02-15T18:26:55.174 INFO:tasks.workunit.client.0.smithi042.stdout:8/256: readlink dd/l25 0 2024-02-15T18:26:55.176 INFO:tasks.workunit.client.0.smithi042.stdout:6/356: dread d9/f17 [0,4194304] 0 2024-02-15T18:26:55.176 INFO:tasks.workunit.client.0.smithi042.stdout:6/357: chown d9/d11/d19/d1c/d31/f3a 1023664908 1 2024-02-15T18:26:55.178 INFO:tasks.workunit.client.0.smithi042.stdout:7/316: mkdir d10/d15/d39/d6c 0 2024-02-15T18:26:55.179 INFO:tasks.workunit.client.0.smithi042.stdout:7/317: dread d10/d15/d39/f5d [0,4194304] 0 2024-02-15T18:26:55.181 INFO:tasks.workunit.client.0.smithi042.stdout:3/324: rename d1/d11/d5d/f62 to d1/d11/f66 0 2024-02-15T18:26:55.181 INFO:tasks.workunit.client.0.smithi042.stdout:3/325: stat d1/de/d19/c40 0 2024-02-15T18:26:55.181 INFO:tasks.workunit.client.0.smithi042.stdout:3/326: chown d1/de/d1a/f2f 593913 1 2024-02-15T18:26:55.181 INFO:tasks.workunit.client.0.smithi042.stdout:5/309: creat d0/d14/f77 x:0 0 0 2024-02-15T18:26:55.183 INFO:tasks.workunit.client.0.smithi042.stdout:4/256: mknod da/d3b/c58 0 2024-02-15T18:26:55.184 INFO:tasks.workunit.client.0.smithi042.stdout:4/257: write da/d11/d1b/d2f/d4d/f47 [1120451,33707] 0 2024-02-15T18:26:55.185 INFO:tasks.workunit.client.0.smithi042.stdout:2/260: link d0/d8/db/de/d1a/l48 d0/d8/db/l4f 0 2024-02-15T18:26:55.186 INFO:tasks.workunit.client.0.smithi042.stdout:7/318: dread d10/d15/f1d [0,4194304] 0 2024-02-15T18:26:55.187 INFO:tasks.workunit.client.0.smithi042.stdout:7/319: dread d10/d15/d39/f5a [0,4194304] 0 2024-02-15T18:26:55.187 INFO:tasks.workunit.client.0.smithi042.stdout:7/320: chown d10/d12/d23/f55 12 1 2024-02-15T18:26:55.187 INFO:tasks.workunit.client.0.smithi042.stdout:8/257: dwrite dd/de/d22/f2b [0,4194304] 0 2024-02-15T18:26:55.188 INFO:tasks.workunit.client.0.smithi042.stdout:6/358: creat d9/d11/d46/f77 x:0 0 0 2024-02-15T18:26:55.189 INFO:tasks.workunit.client.0.smithi042.stdout:0/311: rename d4/d9/d34/d4e/l56 to d4/d9/l69 0 2024-02-15T18:26:55.191 INFO:tasks.workunit.client.0.smithi042.stdout:4/258: mknod da/d2b/c59 0 2024-02-15T18:26:55.193 INFO:tasks.workunit.client.0.smithi042.stdout:2/261: creat d0/d8/d10/d31/d3a/f50 x:0 0 0 2024-02-15T18:26:55.194 INFO:tasks.workunit.client.0.smithi042.stdout:2/262: chown d0/d8/d10/f17 368 1 2024-02-15T18:26:55.199 INFO:tasks.workunit.client.0.smithi042.stdout:1/304: rename de/d15/d67/f32 to de/d15/d67/d63/f69 0 2024-02-15T18:26:55.199 INFO:tasks.workunit.client.0.smithi042.stdout:1/305: stat de/d54/l65 0 2024-02-15T18:26:55.200 INFO:tasks.workunit.client.0.smithi042.stdout:1/306: write de/f5b [2444777,107328] 0 2024-02-15T18:26:55.208 INFO:tasks.workunit.client.0.smithi042.stdout:1/307: read de/d15/d2c/f5a [396774,3430] 0 2024-02-15T18:26:55.211 INFO:tasks.workunit.client.0.smithi042.stdout:0/312: creat d4/d38/d5c/f6a x:0 0 0 2024-02-15T18:26:55.215 INFO:tasks.workunit.client.0.smithi042.stdout:1/308: mkdir de/d15/d67/d46/d6a 0 2024-02-15T18:26:55.215 INFO:tasks.workunit.client.0.smithi042.stdout:1/309: fdatasync de/d15/d2c/f4f 0 2024-02-15T18:26:55.215 INFO:tasks.workunit.client.0.smithi042.stdout:0/313: truncate d4/d9/d34/f4b 5923603 0 2024-02-15T18:26:55.219 INFO:tasks.workunit.client.0.smithi042.stdout:9/270: rename d6/da/cf to d6/c60 0 2024-02-15T18:26:55.221 INFO:tasks.workunit.client.0.smithi042.stdout:9/271: mknod d6/da/d1d/d2d/d40/c61 0 2024-02-15T18:26:55.226 INFO:tasks.workunit.client.0.smithi042.stdout:3/327: rename d1/d11/d2e/d4c/c4e to d1/d11/d5d/c67 0 2024-02-15T18:26:55.229 INFO:tasks.workunit.client.0.smithi042.stdout:2/263: dwrite d0/d8/f28 [4194304,4194304] 0 2024-02-15T18:26:55.229 INFO:tasks.workunit.client.0.smithi042.stdout:5/310: rename d0/d22/l45 to d0/d39/l78 0 2024-02-15T18:26:55.230 INFO:tasks.workunit.client.0.smithi042.stdout:3/328: link d1/d11/d14/d4d/l55 d1/d11/d14/d4d/l68 0 2024-02-15T18:26:55.231 INFO:tasks.workunit.client.0.smithi042.stdout:8/258: dwrite dd/de/d12/f24 [0,4194304] 0 2024-02-15T18:26:55.232 INFO:tasks.workunit.client.0.smithi042.stdout:2/264: creat d0/f51 x:0 0 0 2024-02-15T18:26:55.234 INFO:tasks.workunit.client.0.smithi042.stdout:7/321: dwrite d10/d12/d23/f4a [0,4194304] 0 2024-02-15T18:26:55.235 INFO:tasks.workunit.client.0.smithi042.stdout:9/272: dwrite d6/da/d1d/f38 [0,4194304] 0 2024-02-15T18:26:55.236 INFO:tasks.workunit.client.0.smithi042.stdout:6/359: rename d9/l42 to d9/d11/d1d/l78 0 2024-02-15T18:26:55.236 INFO:tasks.workunit.client.0.smithi042.stdout:6/360: truncate d9/d11/d19/d1c/d31/f56 966387 0 2024-02-15T18:26:55.236 INFO:tasks.workunit.client.0.smithi042.stdout:6/361: fdatasync d9/d3e/f3b 0 2024-02-15T18:26:55.236 INFO:tasks.workunit.client.0.smithi042.stdout:6/362: write d9/d11/d46/f77 [539358,115265] 0 2024-02-15T18:26:55.236 INFO:tasks.workunit.client.0.smithi042.stdout:6/363: chown d9/d11/d19/f2a 67 1 2024-02-15T18:26:55.236 INFO:tasks.workunit.client.0.smithi042.stdout:4/259: dwrite da/f42 [0,4194304] 0 2024-02-15T18:26:55.242 INFO:tasks.workunit.client.0.smithi042.stdout:7/322: dread d10/d12/d23/f4a [0,4194304] 0 2024-02-15T18:26:55.243 INFO:tasks.workunit.client.0.smithi042.stdout:3/329: mknod d1/d11/c69 0 2024-02-15T18:26:55.245 INFO:tasks.workunit.client.0.smithi042.stdout:3/330: dread d1/de/d19/f48 [0,4194304] 0 2024-02-15T18:26:55.251 INFO:tasks.workunit.client.0.smithi042.stdout:8/259: symlink dd/de/d22/l49 0 2024-02-15T18:26:55.252 INFO:tasks.workunit.client.0.smithi042.stdout:2/265: mkdir d0/d8/d52 0 2024-02-15T18:26:55.260 INFO:tasks.workunit.client.0.smithi042.stdout:4/260: dread f9 [0,4194304] 0 2024-02-15T18:26:55.261 INFO:tasks.workunit.client.0.smithi042.stdout:4/261: fdatasync da/d3a/f44 0 2024-02-15T18:26:55.262 INFO:tasks.workunit.client.0.smithi042.stdout:1/310: rename de/d15/d67/d52 to de/d15/d67/d46/d6b 0 2024-02-15T18:26:55.276 INFO:tasks.workunit.client.0.smithi042.stdout:5/311: dwrite d0/d38/d53/d68/f6e [0,4194304] 0 2024-02-15T18:26:55.277 INFO:tasks.workunit.client.0.smithi042.stdout:9/273: dwrite d6/da/d2f/f3d [0,4194304] 0 2024-02-15T18:26:55.277 INFO:tasks.workunit.client.0.smithi042.stdout:7/323: dwrite d10/f5b [0,4194304] 0 2024-02-15T18:26:55.278 INFO:tasks.workunit.client.0.smithi042.stdout:5/312: dread d0/d38/f3c [0,4194304] 0 2024-02-15T18:26:55.278 INFO:tasks.workunit.client.0.smithi042.stdout:5/313: write d0/d12/f21 [271668,94064] 0 2024-02-15T18:26:55.287 INFO:tasks.workunit.client.0.smithi042.stdout:4/262: getdents da/d11/d29/d4a/d4f 0 2024-02-15T18:26:55.292 INFO:tasks.workunit.client.0.smithi042.stdout:6/364: rename d9/d11/d19/d35/d55/d5d to d9/d11/d19/d35/d49/d79 0 2024-02-15T18:26:55.293 INFO:tasks.workunit.client.0.smithi042.stdout:1/311: symlink de/d54/l6c 0 2024-02-15T18:26:55.297 INFO:tasks.workunit.client.0.smithi042.stdout:9/274: mknod d6/da/d1d/d2d/d40/d48/c62 0 2024-02-15T18:26:55.300 INFO:tasks.workunit.client.0.smithi042.stdout:7/324: rmdir d10/d12 39 2024-02-15T18:26:55.301 INFO:tasks.workunit.client.0.smithi042.stdout:4/263: dwrite da/d11/d27/f3f [0,4194304] 0 2024-02-15T18:26:55.307 INFO:tasks.workunit.client.0.smithi042.stdout:8/260: link dd/d11/c35 dd/d23/c4a 0 2024-02-15T18:26:55.307 INFO:tasks.workunit.client.0.smithi042.stdout:8/261: write dd/de/f2a [866804,105084] 0 2024-02-15T18:26:55.312 INFO:tasks.workunit.client.0.smithi042.stdout:2/266: rename d0/d8/d1b to d0/d8/d10/d31/d3a/d53 0 2024-02-15T18:26:55.312 INFO:tasks.workunit.client.0.smithi042.stdout:2/267: write d0/d8/db/de/d1a/f2b [1055937,30756] 0 2024-02-15T18:26:55.312 INFO:tasks.workunit.client.0.smithi042.stdout:2/268: dread - d0/d8/d10/d31/d3a/d53/f45 zero size 2024-02-15T18:26:55.312 INFO:tasks.workunit.client.0.smithi042.stdout:2/269: stat d0/d8/f39 0 2024-02-15T18:26:55.314 INFO:tasks.workunit.client.0.smithi042.stdout:6/365: getdents d9/d11/d19 0 2024-02-15T18:26:55.317 INFO:tasks.workunit.client.0.smithi042.stdout:1/312: rmdir de/d15/d67/d63 39 2024-02-15T18:26:55.318 INFO:tasks.workunit.client.0.smithi042.stdout:9/275: dwrite d6/da/f1c [0,4194304] 0 2024-02-15T18:26:55.318 INFO:tasks.workunit.client.0.smithi042.stdout:9/276: write d6/da/d2f/d55/f47 [707328,128592] 0 2024-02-15T18:26:55.319 INFO:tasks.workunit.client.0.smithi042.stdout:7/325: mkdir d10/d49/d6d 0 2024-02-15T18:26:55.320 INFO:tasks.workunit.client.0.smithi042.stdout:4/264: mknod da/d3b/c5a 0 2024-02-15T18:26:55.321 INFO:tasks.workunit.client.0.smithi042.stdout:4/265: write da/d11/d27/f3f [4518931,22466] 0 2024-02-15T18:26:55.321 INFO:tasks.workunit.client.0.smithi042.stdout:8/262: mkdir dd/de/d3c/d4b 0 2024-02-15T18:26:55.322 INFO:tasks.workunit.client.0.smithi042.stdout:0/314: sync 2024-02-15T18:26:55.322 INFO:tasks.workunit.client.0.smithi042.stdout:0/315: truncate d4/d38/d5c/f61 939369 0 2024-02-15T18:26:55.323 INFO:tasks.workunit.client.0.smithi042.stdout:2/270: creat d0/d8/d10/f54 x:0 0 0 2024-02-15T18:26:55.323 INFO:tasks.workunit.client.0.smithi042.stdout:2/271: chown d0/d8/d10/d31 1821 1 2024-02-15T18:26:55.323 INFO:tasks.workunit.client.0.smithi042.stdout:2/272: chown d0/d8/db/c1e 767 1 2024-02-15T18:26:55.323 INFO:tasks.workunit.client.0.smithi042.stdout:2/273: dread - d0/d8/db/f42 zero size 2024-02-15T18:26:55.323 INFO:tasks.workunit.client.0.smithi042.stdout:2/274: dread - d0/d8/db/f42 zero size 2024-02-15T18:26:55.324 INFO:tasks.workunit.client.0.smithi042.stdout:6/366: creat d9/d11/d19/d35/d49/d79/f7a x:0 0 0 2024-02-15T18:26:55.325 INFO:tasks.workunit.client.0.smithi042.stdout:1/313: mkdir de/d6d 0 2024-02-15T18:26:55.327 INFO:tasks.workunit.client.0.smithi042.stdout:9/277: creat d6/dd/d9/d59/f63 x:0 0 0 2024-02-15T18:26:55.327 INFO:tasks.workunit.client.0.smithi042.stdout:9/278: fsync d6/da/f3e 0 2024-02-15T18:26:55.328 INFO:tasks.workunit.client.0.smithi042.stdout:7/326: mkdir d10/d49/d6d/d6e 0 2024-02-15T18:26:55.329 INFO:tasks.workunit.client.0.smithi042.stdout:4/266: symlink da/d11/d29/l5b 0 2024-02-15T18:26:55.330 INFO:tasks.workunit.client.0.smithi042.stdout:8/263: creat dd/de/d12/f4c x:0 0 0 2024-02-15T18:26:55.333 INFO:tasks.workunit.client.0.smithi042.stdout:0/316: creat d4/d9/d5a/f6b x:0 0 0 2024-02-15T18:26:55.333 INFO:tasks.workunit.client.0.smithi042.stdout:0/317: fsync d4/d9/d15/f42 0 2024-02-15T18:26:55.335 INFO:tasks.workunit.client.0.smithi042.stdout:2/275: mknod d0/d8/db/de/d1a/c55 0 2024-02-15T18:26:55.338 INFO:tasks.workunit.client.0.smithi042.stdout:6/367: link d9/d11/d19/d1c/d31/d4f/l5f d9/d11/d19/d1c/d31/d4f/l7b 0 2024-02-15T18:26:55.338 INFO:tasks.workunit.client.0.smithi042.stdout:2/276: dread d0/d8/f39 [0,4194304] 0 2024-02-15T18:26:55.338 INFO:tasks.workunit.client.0.smithi042.stdout:6/368: chown d9/d11/d19/d35/f51 4 1 2024-02-15T18:26:55.340 INFO:tasks.workunit.client.0.smithi042.stdout:1/314: symlink de/l6e 0 2024-02-15T18:26:55.340 INFO:tasks.workunit.client.0.smithi042.stdout:1/315: write de/d15/d2c/f5f [611998,39163] 0 2024-02-15T18:26:55.340 INFO:tasks.workunit.client.0.smithi042.stdout:1/316: write de/d15/d1b/f36 [5178906,56447] 0 2024-02-15T18:26:55.340 INFO:tasks.workunit.client.0.smithi042.stdout:1/317: chown de/d15/d2c/f4f 48972 1 2024-02-15T18:26:55.341 INFO:tasks.workunit.client.0.smithi042.stdout:0/318: dread d4/f3b [0,4194304] 0 2024-02-15T18:26:55.342 INFO:tasks.workunit.client.0.smithi042.stdout:7/327: creat d10/d15/d39/d6c/f6f x:0 0 0 2024-02-15T18:26:55.345 INFO:tasks.workunit.client.0.smithi042.stdout:2/277: unlink d0/d8/db/de/d1a/c55 0 2024-02-15T18:26:55.347 INFO:tasks.workunit.client.0.smithi042.stdout:1/318: creat de/d15/d2c/d51/f6f x:0 0 0 2024-02-15T18:26:55.349 INFO:tasks.workunit.client.0.smithi042.stdout:0/319: mkdir d4/d9/d34/d4e/d6c 0 2024-02-15T18:26:55.351 INFO:tasks.workunit.client.0.smithi042.stdout:7/328: rename d10/d15/l3e to d10/d49/d68/l70 0 2024-02-15T18:26:55.351 INFO:tasks.workunit.client.0.smithi042.stdout:7/329: chown d10/d15/d39/d6c 1068254857 1 2024-02-15T18:26:55.352 INFO:tasks.workunit.client.0.smithi042.stdout:2/278: mkdir d0/d40/d56 0 2024-02-15T18:26:55.353 INFO:tasks.workunit.client.0.smithi042.stdout:0/320: getdents d4/d9/d62 0 2024-02-15T18:26:55.354 INFO:tasks.workunit.client.0.smithi042.stdout:1/319: write de/d15/d67/d63/f69 [2306996,109069] 0 2024-02-15T18:26:55.355 INFO:tasks.workunit.client.0.smithi042.stdout:7/330: unlink d10/d12/d23/f4a 0 2024-02-15T18:26:55.355 INFO:tasks.workunit.client.0.smithi042.stdout:7/331: stat d10/d49/d6d 0 2024-02-15T18:26:55.357 INFO:tasks.workunit.client.0.smithi042.stdout:2/279: mknod d0/d8/db/de/d1a/d2e/c57 0 2024-02-15T18:26:55.358 INFO:tasks.workunit.client.0.smithi042.stdout:0/321: symlink d4/d9/d34/l6d 0 2024-02-15T18:26:55.359 INFO:tasks.workunit.client.0.smithi042.stdout:1/320: mkdir de/d15/d2c/d51/d70 0 2024-02-15T18:26:55.360 INFO:tasks.workunit.client.0.smithi042.stdout:2/280: rmdir d0/d8/d52 0 2024-02-15T18:26:55.360 INFO:tasks.workunit.client.0.smithi042.stdout:2/281: readlink d0/d8/db/l4f 0 2024-02-15T18:26:55.360 INFO:tasks.workunit.client.0.smithi042.stdout:2/282: chown d0/d8/db/de/d1a/l48 130751 1 2024-02-15T18:26:55.362 INFO:tasks.workunit.client.0.smithi042.stdout:0/322: creat d4/d38/d39/f6e x:0 0 0 2024-02-15T18:26:55.363 INFO:tasks.workunit.client.0.smithi042.stdout:2/283: unlink d0/d8/c13 0 2024-02-15T18:26:55.364 INFO:tasks.workunit.client.0.smithi042.stdout:4/267: dwrite da/d2b/f37 [0,4194304] 0 2024-02-15T18:26:55.365 INFO:tasks.workunit.client.0.smithi042.stdout:9/279: dwrite d6/da/d2f/d55/f47 [0,4194304] 0 2024-02-15T18:26:55.365 INFO:tasks.workunit.client.0.smithi042.stdout:1/321: dread f2 [0,4194304] 0 2024-02-15T18:26:55.366 INFO:tasks.workunit.client.0.smithi042.stdout:7/332: dread d10/d12/f20 [0,4194304] 0 2024-02-15T18:26:55.366 INFO:tasks.workunit.client.0.smithi042.stdout:7/333: write d10/d49/f67 [145446,58382] 0 2024-02-15T18:26:55.367 INFO:tasks.workunit.client.0.smithi042.stdout:6/369: dwrite d9/f2d [4194304,4194304] 0 2024-02-15T18:26:55.368 INFO:tasks.workunit.client.0.smithi042.stdout:0/323: write d4/d9/d15/f21 [351884,60108] 0 2024-02-15T18:26:55.370 INFO:tasks.workunit.client.0.smithi042.stdout:8/264: dwrite dd/de/d12/f1e [0,4194304] 0 2024-02-15T18:26:55.370 INFO:tasks.workunit.client.0.smithi042.stdout:8/265: truncate dd/de/d12/d32/f34 41225 0 2024-02-15T18:26:55.372 INFO:tasks.workunit.client.0.smithi042.stdout:1/322: creat de/d15/d67/d46/f71 x:0 0 0 2024-02-15T18:26:55.372 INFO:tasks.workunit.client.0.smithi042.stdout:8/266: write dd/de/f1a [2942241,23474] 0 2024-02-15T18:26:55.373 INFO:tasks.workunit.client.0.smithi042.stdout:7/334: creat d10/d15/d39/d6c/f71 x:0 0 0 2024-02-15T18:26:55.379 INFO:tasks.workunit.client.0.smithi042.stdout:6/370: dread d9/d11/d19/d1c/d31/f3a [0,4194304] 0 2024-02-15T18:26:55.379 INFO:tasks.workunit.client.0.smithi042.stdout:6/371: rename d9/d11/d19/d1c to d9/d11/d19/d1c/d31/d7c 22 2024-02-15T18:26:55.379 INFO:tasks.workunit.client.0.smithi042.stdout:6/372: fdatasync d9/d11/d19/d35/f73 0 2024-02-15T18:26:55.390 INFO:tasks.workunit.client.0.smithi042.stdout:2/284: dwrite d0/d8/db/de/f4d [4194304,4194304] 0 2024-02-15T18:26:55.390 INFO:tasks.workunit.client.0.smithi042.stdout:2/285: chown d0/d8/d10/d31/d3a/d53/d3c/c4e 0 1 2024-02-15T18:26:55.401 INFO:tasks.workunit.client.0.smithi042.stdout:2/286: dread d0/f49 [0,4194304] 0 2024-02-15T18:26:55.402 INFO:tasks.workunit.client.0.smithi042.stdout:0/324: link d4/f7 d4/d9/d34/d57/f6f 0 2024-02-15T18:26:55.405 INFO:tasks.workunit.client.0.smithi042.stdout:2/287: read d0/f7 [4567406,15011] 0 2024-02-15T18:26:55.405 INFO:tasks.workunit.client.0.smithi042.stdout:2/288: write d0/d8/d10/d31/d3a/d53/f45 [645387,76845] 0 2024-02-15T18:26:55.407 INFO:tasks.workunit.client.0.smithi042.stdout:8/267: creat dd/f4d x:0 0 0 2024-02-15T18:26:55.407 INFO:tasks.workunit.client.0.smithi042.stdout:7/335: rmdir d10/d49/d68 39 2024-02-15T18:26:55.411 INFO:tasks.workunit.client.0.smithi042.stdout:2/289: unlink d0/d8/db/de/d1a/d2e/f4a 0 2024-02-15T18:26:55.411 INFO:tasks.workunit.client.0.smithi042.stdout:2/290: write d0/d8/f28 [8458283,88415] 0 2024-02-15T18:26:55.411 INFO:tasks.workunit.client.0.smithi042.stdout:2/291: dread d0/d8/db/de/d1a/f43 [0,4194304] 0 2024-02-15T18:26:55.412 INFO:tasks.workunit.client.0.smithi042.stdout:7/336: truncate d10/d12/f20 1838575 0 2024-02-15T18:26:55.412 INFO:tasks.workunit.client.0.smithi042.stdout:7/337: rename d10/d49 to d10/d49/d72 22 2024-02-15T18:26:55.414 INFO:tasks.workunit.client.0.smithi042.stdout:9/280: dwrite d6/dd/d9/f41 [0,4194304] 0 2024-02-15T18:26:55.414 INFO:tasks.workunit.client.0.smithi042.stdout:2/292: getdents d0/d8/d10/d31/d3a/d53/d3c 0 2024-02-15T18:26:55.414 INFO:tasks.workunit.client.0.smithi042.stdout:8/268: dread dd/de/d22/f2b [0,4194304] 0 2024-02-15T18:26:55.415 INFO:tasks.workunit.client.0.smithi042.stdout:1/323: dwrite de/d15/f20 [4194304,4194304] 0 2024-02-15T18:26:55.415 INFO:tasks.workunit.client.0.smithi042.stdout:7/338: unlink d10/d15/c48 0 2024-02-15T18:26:55.416 INFO:tasks.workunit.client.0.smithi042.stdout:6/373: dwrite d9/d3e/f3b [0,4194304] 0 2024-02-15T18:26:55.416 INFO:tasks.workunit.client.0.smithi042.stdout:4/268: dwrite da/d11/d27/f3f [4194304,4194304] 0 2024-02-15T18:26:55.418 INFO:tasks.workunit.client.0.smithi042.stdout:2/293: creat d0/d8/d10/d31/d3a/d53/f58 x:0 0 0 2024-02-15T18:26:55.418 INFO:tasks.workunit.client.0.smithi042.stdout:2/294: truncate d0/f51 223517 0 2024-02-15T18:26:55.418 INFO:tasks.workunit.client.0.smithi042.stdout:2/295: getdents d0/d8/db/d44 0 2024-02-15T18:26:55.418 INFO:tasks.workunit.client.0.smithi042.stdout:9/281: dread d6/f2e [0,4194304] 0 2024-02-15T18:26:55.418 INFO:tasks.workunit.client.0.smithi042.stdout:9/282: write d6/da/f1c [4262333,12636] 0 2024-02-15T18:26:55.419 INFO:tasks.workunit.client.0.smithi042.stdout:8/269: rmdir dd/de/d33 39 2024-02-15T18:26:55.423 INFO:tasks.workunit.client.0.smithi042.stdout:1/324: creat de/f72 x:0 0 0 2024-02-15T18:26:55.423 INFO:tasks.workunit.client.0.smithi042.stdout:1/325: stat de/d54/l6c 0 2024-02-15T18:26:55.426 INFO:tasks.workunit.client.0.smithi042.stdout:0/325: dwrite d4/d9/d34/f36 [0,4194304] 0 2024-02-15T18:26:55.427 INFO:tasks.workunit.client.0.smithi042.stdout:6/374: link d9/d11/d19/d1c/d31/d4f/l5f d9/d11/d19/d35/d49/d79/l7d 0 2024-02-15T18:26:55.427 INFO:tasks.workunit.client.0.smithi042.stdout:6/375: write d9/d11/d19/d35/f73 [597912,51573] 0 2024-02-15T18:26:55.428 INFO:tasks.workunit.client.0.smithi042.stdout:4/269: dread f9 [0,4194304] 0 2024-02-15T18:26:55.433 INFO:tasks.workunit.client.0.smithi042.stdout:9/283: dwrite d6/da/d2f/f56 [0,4194304] 0 2024-02-15T18:26:55.433 INFO:tasks.workunit.client.0.smithi042.stdout:7/339: dwrite d10/d43/d57/f5e [0,4194304] 0 2024-02-15T18:26:55.441 INFO:tasks.workunit.client.0.smithi042.stdout:7/340: dread d10/d15/d39/f3a [0,4194304] 0 2024-02-15T18:26:55.441 INFO:tasks.workunit.client.0.smithi042.stdout:7/341: fdatasync d10/d12/d23/f3b 0 2024-02-15T18:26:55.442 INFO:tasks.workunit.client.0.smithi042.stdout:4/270: dread da/f25 [0,4194304] 0 2024-02-15T18:26:55.442 INFO:tasks.workunit.client.0.smithi042.stdout:4/271: chown da/d3b/c58 9737 1 2024-02-15T18:26:55.445 INFO:tasks.workunit.client.0.smithi042.stdout:9/284: dread d6/dd/f19 [0,4194304] 0 2024-02-15T18:26:55.464 INFO:tasks.workunit.client.0.smithi042.stdout:2/296: mknod d0/d8/d10/d31/d3a/d53/d3c/c59 0 2024-02-15T18:26:55.471 INFO:tasks.workunit.client.0.smithi042.stdout:3/331: sync 2024-02-15T18:26:55.471 INFO:tasks.workunit.client.0.smithi042.stdout:5/314: sync 2024-02-15T18:26:55.471 INFO:tasks.workunit.client.0.smithi042.stdout:3/332: write d1/de/d1a/f2f [504379,73590] 0 2024-02-15T18:26:55.471 INFO:tasks.workunit.client.0.smithi042.stdout:8/270: rename dd/d23 to dd/de/d12/d32/d4e 0 2024-02-15T18:26:55.472 INFO:tasks.workunit.client.0.smithi042.stdout:8/271: readlink dd/de/d22/l41 0 2024-02-15T18:26:55.473 INFO:tasks.workunit.client.0.smithi042.stdout:1/326: mknod de/d15/d2c/c73 0 2024-02-15T18:26:55.476 INFO:tasks.workunit.client.0.smithi042.stdout:0/326: mkdir d4/d9/d34/d4e/d6c/d70 0 2024-02-15T18:26:55.477 INFO:tasks.workunit.client.0.smithi042.stdout:2/297: dwrite d0/f51 [0,4194304] 0 2024-02-15T18:26:55.484 INFO:tasks.workunit.client.0.smithi042.stdout:4/272: mkdir da/d11/d29/d4b/d5c 0 2024-02-15T18:26:55.486 INFO:tasks.workunit.client.0.smithi042.stdout:9/285: creat d6/da/d2f/d54/f64 x:0 0 0 2024-02-15T18:26:55.491 INFO:tasks.workunit.client.0.smithi042.stdout:5/315: mknod d0/d39/c79 0 2024-02-15T18:26:55.493 INFO:tasks.workunit.client.0.smithi042.stdout:3/333: creat d1/d11/d2e/d4c/f6a x:0 0 0 2024-02-15T18:26:55.494 INFO:tasks.workunit.client.0.smithi042.stdout:6/376: rename d9/d11/d19/d1c/d31/d4f/l7b to d9/d11/d19/d1c/l7e 0 2024-02-15T18:26:55.495 INFO:tasks.workunit.client.0.smithi042.stdout:6/377: write d9/f76 [3603993,112481] 0 2024-02-15T18:26:55.495 INFO:tasks.workunit.client.0.smithi042.stdout:6/378: chown d9/d11/d19/d1c 1151 1 2024-02-15T18:26:55.495 INFO:tasks.workunit.client.0.smithi042.stdout:8/272: creat dd/de/d3c/d4b/f4f x:0 0 0 2024-02-15T18:26:55.495 INFO:tasks.workunit.client.0.smithi042.stdout:8/273: readlink dd/de/d12/l27 0 2024-02-15T18:26:55.500 INFO:tasks.workunit.client.0.smithi042.stdout:2/298: rmdir d0/d8/d10/d31/d3a/d53 39 2024-02-15T18:26:55.500 INFO:tasks.workunit.client.0.smithi042.stdout:2/299: chown d0/d8/d20/c32 2062013921 1 2024-02-15T18:26:55.509 INFO:tasks.workunit.client.0.smithi042.stdout:4/273: mknod da/d11/d29/d4b/c5d 0 2024-02-15T18:26:55.513 INFO:tasks.workunit.client.0.smithi042.stdout:9/286: creat d6/da/f65 x:0 0 0 2024-02-15T18:26:55.513 INFO:tasks.workunit.client.0.smithi042.stdout:9/287: stat d6/da/f3e 0 2024-02-15T18:26:55.514 INFO:tasks.workunit.client.0.smithi042.stdout:5/316: creat d0/d38/d53/f7a x:0 0 0 2024-02-15T18:26:55.514 INFO:tasks.workunit.client.0.smithi042.stdout:9/288: dread d6/da/d1d/d2d/d40/f43 [0,4194304] 0 2024-02-15T18:26:55.515 INFO:tasks.workunit.client.0.smithi042.stdout:9/289: readlink d6/dd/l27 0 2024-02-15T18:26:55.517 INFO:tasks.workunit.client.0.smithi042.stdout:5/317: dread d0/d14/f28 [0,4194304] 0 2024-02-15T18:26:55.517 INFO:tasks.workunit.client.0.smithi042.stdout:3/334: symlink d1/de/d19/l6b 0 2024-02-15T18:26:55.519 INFO:tasks.workunit.client.0.smithi042.stdout:7/342: rename d10/d15/d39/f3a to d10/f73 0 2024-02-15T18:26:55.520 INFO:tasks.workunit.client.0.smithi042.stdout:6/379: symlink d9/d11/d19/d35/d55/l7f 0 2024-02-15T18:26:55.521 INFO:tasks.workunit.client.0.smithi042.stdout:8/274: symlink dd/de/d3c/d4b/l50 0 2024-02-15T18:26:55.521 INFO:tasks.workunit.client.0.smithi042.stdout:4/274: dwrite da/d3b/d41/f49 [0,4194304] 0 2024-02-15T18:26:55.523 INFO:tasks.workunit.client.0.smithi042.stdout:1/327: mkdir de/d15/d67/d74 0 2024-02-15T18:26:55.523 INFO:tasks.workunit.client.0.smithi042.stdout:1/328: stat de/d15/f43 0 2024-02-15T18:26:55.524 INFO:tasks.workunit.client.0.smithi042.stdout:2/300: mkdir d0/d8/d10/d5a 0 2024-02-15T18:26:55.524 INFO:tasks.workunit.client.0.smithi042.stdout:6/380: dread d9/d11/d19/d35/d49/f4c [0,4194304] 0 2024-02-15T18:26:55.524 INFO:tasks.workunit.client.0.smithi042.stdout:9/290: symlink d6/da/d1d/d2d/d40/d48/l66 0 2024-02-15T18:26:55.526 INFO:tasks.workunit.client.0.smithi042.stdout:3/335: mkdir d1/d11/d14/d4d/d6c 0 2024-02-15T18:26:55.529 INFO:tasks.workunit.client.0.smithi042.stdout:5/318: rename d0/d12/d61 to d0/d22/d7b 0 2024-02-15T18:26:55.531 INFO:tasks.workunit.client.0.smithi042.stdout:7/343: creat d10/d12/d65/f74 x:0 0 0 2024-02-15T18:26:55.531 INFO:tasks.workunit.client.0.smithi042.stdout:7/344: rename d10 to d10/d15/d39/d75 22 2024-02-15T18:26:55.531 INFO:tasks.workunit.client.0.smithi042.stdout:6/381: dread d9/f40 [0,4194304] 0 2024-02-15T18:26:55.533 INFO:tasks.workunit.client.0.smithi042.stdout:8/275: getdents dd/de/d33 0 2024-02-15T18:26:55.537 INFO:tasks.workunit.client.0.smithi042.stdout:8/276: dread dd/de/d12/d32/d4e/f28 [0,4194304] 0 2024-02-15T18:26:55.539 INFO:tasks.workunit.client.0.smithi042.stdout:8/277: write dd/d11/f39 [69579,92244] 0 2024-02-15T18:26:55.539 INFO:tasks.workunit.client.0.smithi042.stdout:4/275: dwrite da/d3a/f44 [0,4194304] 0 2024-02-15T18:26:55.539 INFO:tasks.workunit.client.0.smithi042.stdout:4/276: stat da/d11/l2e 0 2024-02-15T18:26:55.550 INFO:tasks.workunit.client.0.smithi042.stdout:9/291: rmdir d6/da/d2f 39 2024-02-15T18:26:55.553 INFO:tasks.workunit.client.0.smithi042.stdout:3/336: mknod d1/c6d 0 2024-02-15T18:26:55.553 INFO:tasks.workunit.client.0.smithi042.stdout:3/337: chown d1/c59 6699432 1 2024-02-15T18:26:55.553 INFO:tasks.workunit.client.0.smithi042.stdout:3/338: chown d1/d11/f45 47149978 1 2024-02-15T18:26:55.553 INFO:tasks.workunit.client.0.smithi042.stdout:6/382: creat d9/d11/d19/d35/f80 x:0 0 0 2024-02-15T18:26:55.553 INFO:tasks.workunit.client.0.smithi042.stdout:6/383: chown d9/d11/d19/d35 63903979 1 2024-02-15T18:26:55.553 INFO:tasks.workunit.client.0.smithi042.stdout:6/384: fsync d9/d11/d19/d35/d49/f4c 0 2024-02-15T18:26:55.553 INFO:tasks.workunit.client.0.smithi042.stdout:3/339: write d1/d11/d14/d4d/f4f [861028,117741] 0 2024-02-15T18:26:55.553 INFO:tasks.workunit.client.0.smithi042.stdout:3/340: fdatasync d1/d11/d14/f43 0 2024-02-15T18:26:55.553 INFO:tasks.workunit.client.0.smithi042.stdout:3/341: fsync d1/d11/d14/f35 0 2024-02-15T18:26:55.556 INFO:tasks.workunit.client.0.smithi042.stdout:4/277: mkdir da/d11/d29/d4b/d5e 0 2024-02-15T18:26:55.559 INFO:tasks.workunit.client.0.smithi042.stdout:6/385: symlink d9/d11/d19/d35/d55/l81 0 2024-02-15T18:26:55.561 INFO:tasks.workunit.client.0.smithi042.stdout:3/342: link d1/de/d1a/f5c d1/de/f6e 0 2024-02-15T18:26:55.563 INFO:tasks.workunit.client.0.smithi042.stdout:4/278: mknod da/d11/d29/d4b/d5c/c5f 0 2024-02-15T18:26:55.563 INFO:tasks.workunit.client.0.smithi042.stdout:4/279: dread - da/d11/d1b/d2f/f34 zero size 2024-02-15T18:26:55.565 INFO:tasks.workunit.client.0.smithi042.stdout:3/343: link d1/d11/d2e/d4c/f58 d1/de/d1a/f6f 0 2024-02-15T18:26:55.566 INFO:tasks.workunit.client.0.smithi042.stdout:4/280: creat da/d11/d1b/d53/f60 x:0 0 0 2024-02-15T18:26:55.568 INFO:tasks.workunit.client.0.smithi042.stdout:3/344: mknod d1/d11/d2e/d4c/c70 0 2024-02-15T18:26:55.570 INFO:tasks.workunit.client.0.smithi042.stdout:4/281: creat da/d11/d27/f61 x:0 0 0 2024-02-15T18:26:55.570 INFO:tasks.workunit.client.0.smithi042.stdout:6/386: dread d9/d11/d19/d35/f61 [0,4194304] 0 2024-02-15T18:26:55.572 INFO:tasks.workunit.client.0.smithi042.stdout:6/387: mknod d9/d11/d19/c82 0 2024-02-15T18:26:55.573 INFO:tasks.workunit.client.0.smithi042.stdout:6/388: fdatasync d9/d11/d19/d1c/d31/d4f/f68 0 2024-02-15T18:26:55.577 INFO:tasks.workunit.client.0.smithi042.stdout:5/319: dwrite d0/f23 [0,4194304] 0 2024-02-15T18:26:55.577 INFO:tasks.workunit.client.0.smithi042.stdout:8/278: dwrite fc [0,4194304] 0 2024-02-15T18:26:55.577 INFO:tasks.workunit.client.0.smithi042.stdout:8/279: dread - dd/de/d12/f45 zero size 2024-02-15T18:26:55.579 INFO:tasks.workunit.client.0.smithi042.stdout:7/345: dwrite d10/d43/f4e [0,4194304] 0 2024-02-15T18:26:55.579 INFO:tasks.workunit.client.0.smithi042.stdout:7/346: chown d10/d12/d23/f3b 1 1 2024-02-15T18:26:55.579 INFO:tasks.workunit.client.0.smithi042.stdout:3/345: dread d1/d11/d2e/f37 [0,4194304] 0 2024-02-15T18:26:55.580 INFO:tasks.workunit.client.0.smithi042.stdout:9/292: dwrite d6/da/f3e [0,4194304] 0 2024-02-15T18:26:55.580 INFO:tasks.workunit.client.0.smithi042.stdout:1/329: dwrite de/d15/d67/f26 [0,4194304] 0 2024-02-15T18:26:55.580 INFO:tasks.workunit.client.0.smithi042.stdout:6/389: write d9/f2f [2723743,83729] 0 2024-02-15T18:26:55.584 INFO:tasks.workunit.client.0.smithi042.stdout:4/282: dwrite da/d11/d27/f3f [8388608,4194304] 0 2024-02-15T18:26:55.584 INFO:tasks.workunit.client.0.smithi042.stdout:4/283: chown da/d3b/c5a 65016806 1 2024-02-15T18:26:55.585 INFO:tasks.workunit.client.0.smithi042.stdout:4/284: chown da/d11/f22 4554 1 2024-02-15T18:26:55.585 INFO:tasks.workunit.client.0.smithi042.stdout:4/285: readlink da/d11/l2e 0 2024-02-15T18:26:55.585 INFO:tasks.workunit.client.0.smithi042.stdout:4/286: stat da/d3b/d41/c45 0 2024-02-15T18:26:55.586 INFO:tasks.workunit.client.0.smithi042.stdout:5/320: mknod d0/d38/d53/d64/c7c 0 2024-02-15T18:26:55.596 INFO:tasks.workunit.client.0.smithi042.stdout:8/280: rename dd/de/d33/f37 to dd/de/d3c/f51 0 2024-02-15T18:26:55.600 INFO:tasks.workunit.client.0.smithi042.stdout:8/281: write dd/f29 [346716,92615] 0 2024-02-15T18:26:55.603 INFO:tasks.workunit.client.0.smithi042.stdout:3/346: creat d1/d11/d2e/f71 x:0 0 0 2024-02-15T18:26:55.606 INFO:tasks.workunit.client.0.smithi042.stdout:7/347: dwrite fa [0,4194304] 0 2024-02-15T18:26:55.608 INFO:tasks.workunit.client.0.smithi042.stdout:7/348: dread d10/d12/f20 [0,4194304] 0 2024-02-15T18:26:55.608 INFO:tasks.workunit.client.0.smithi042.stdout:9/293: dwrite d6/dd/d9/f32 [4194304,4194304] 0 2024-02-15T18:26:55.610 INFO:tasks.workunit.client.0.smithi042.stdout:1/330: creat de/d15/d67/d74/f75 x:0 0 0 2024-02-15T18:26:55.610 INFO:tasks.workunit.client.0.smithi042.stdout:1/331: write de/d15/d1b/f25 [62005,128367] 0 2024-02-15T18:26:55.613 INFO:tasks.workunit.client.0.smithi042.stdout:6/390: link d9/d11/d19/d35/d49/l4a d9/d11/d19/d1c/d31/d4f/l83 0 2024-02-15T18:26:55.615 INFO:tasks.workunit.client.0.smithi042.stdout:4/287: unlink da/d11/fe 0 2024-02-15T18:26:55.615 INFO:tasks.workunit.client.0.smithi042.stdout:6/391: dread d9/d11/d46/f77 [0,4194304] 0 2024-02-15T18:26:55.618 INFO:tasks.workunit.client.0.smithi042.stdout:5/321: mkdir d0/d38/d53/d68/d6a/d42/d7d 0 2024-02-15T18:26:55.620 INFO:tasks.workunit.client.0.smithi042.stdout:8/282: rmdir dd 39 2024-02-15T18:26:55.622 INFO:tasks.workunit.client.0.smithi042.stdout:3/347: readlink d1/d11/d14/l30 0 2024-02-15T18:26:55.626 INFO:tasks.workunit.client.0.smithi042.stdout:7/349: truncate d10/d12/f5f 523532 0 2024-02-15T18:26:55.634 INFO:tasks.workunit.client.0.smithi042.stdout:1/332: symlink de/l76 0 2024-02-15T18:26:55.636 INFO:tasks.workunit.client.0.smithi042.stdout:4/288: creat da/d11/d29/d4b/f62 x:0 0 0 2024-02-15T18:26:55.639 INFO:tasks.workunit.client.0.smithi042.stdout:5/322: dwrite d0/d14/f28 [0,4194304] 0 2024-02-15T18:26:55.641 INFO:tasks.workunit.client.0.smithi042.stdout:5/323: dread d0/d38/d53/d68/d6a/f56 [0,4194304] 0 2024-02-15T18:26:55.649 INFO:tasks.workunit.client.0.smithi042.stdout:3/348: truncate d1/f9 2637581 0 2024-02-15T18:26:55.650 INFO:tasks.workunit.client.0.smithi042.stdout:7/350: unlink d10/d12/f20 0 2024-02-15T18:26:55.652 INFO:tasks.workunit.client.0.smithi042.stdout:9/294: rename d6/da/d1d/f46 to d6/f67 0 2024-02-15T18:26:55.653 INFO:tasks.workunit.client.0.smithi042.stdout:1/333: mknod de/d15/d67/d63/c77 0 2024-02-15T18:26:55.654 INFO:tasks.workunit.client.0.smithi042.stdout:4/289: unlink da/d3b/d41/f49 0 2024-02-15T18:26:55.655 INFO:tasks.workunit.client.0.smithi042.stdout:8/283: creat dd/de/d12/d32/f52 x:0 0 0 2024-02-15T18:26:55.658 INFO:tasks.workunit.client.0.smithi042.stdout:3/349: mknod d1/d11/c72 0 2024-02-15T18:26:55.659 INFO:tasks.workunit.client.0.smithi042.stdout:7/351: unlink d10/d12/l2e 0 2024-02-15T18:26:55.661 INFO:tasks.workunit.client.0.smithi042.stdout:5/324: dwrite d0/d38/d53/d68/d6a/f56 [0,4194304] 0 2024-02-15T18:26:55.661 INFO:tasks.workunit.client.0.smithi042.stdout:6/392: rename d9/d11/d1d/f5e to d9/d11/d19/d1c/d31/d53/f84 0 2024-02-15T18:26:55.661 INFO:tasks.workunit.client.0.smithi042.stdout:6/393: stat d9/d11/d19/d35/d49/d79/f60 0 2024-02-15T18:26:55.665 INFO:tasks.workunit.client.0.smithi042.stdout:9/295: symlink d6/dd/d4f/l68 0 2024-02-15T18:26:55.666 INFO:tasks.workunit.client.0.smithi042.stdout:9/296: read d6/da/f23 [2503512,114537] 0 2024-02-15T18:26:55.667 INFO:tasks.workunit.client.0.smithi042.stdout:3/350: getdents d1/d11/d2e 0 2024-02-15T18:26:55.671 INFO:tasks.workunit.client.0.smithi042.stdout:5/325: link d0/d12/d65/d74/d52/l59 d0/d39/l7e 0 2024-02-15T18:26:55.671 INFO:tasks.workunit.client.0.smithi042.stdout:5/326: dread - d0/d12/d65/d74/f72 zero size 2024-02-15T18:26:55.672 INFO:tasks.workunit.client.0.smithi042.stdout:3/351: write d1/de/f3a [1165004,119960] 0 2024-02-15T18:26:55.673 INFO:tasks.workunit.client.0.smithi042.stdout:8/284: dwrite dd/de/d12/d32/d4e/f28 [0,4194304] 0 2024-02-15T18:26:55.674 INFO:tasks.workunit.client.0.smithi042.stdout:1/334: rename de/d15/d67/f1d to de/d15/d2c/d51/d70/f78 0 2024-02-15T18:26:55.674 INFO:tasks.workunit.client.0.smithi042.stdout:3/352: mkdir d1/d11/d14/d4d/d73 0 2024-02-15T18:26:55.683 INFO:tasks.workunit.client.0.smithi042.stdout:4/290: rename da/d11/d27/f61 to da/f63 0 2024-02-15T18:26:55.685 INFO:tasks.workunit.client.0.smithi042.stdout:9/297: dread d6/da/f1c [0,4194304] 0 2024-02-15T18:26:55.685 INFO:tasks.workunit.client.0.smithi042.stdout:6/394: rename d9/d11/d19/d1c/d31/d4f/l83 to d9/d11/d1d/l85 0 2024-02-15T18:26:55.686 INFO:tasks.workunit.client.0.smithi042.stdout:5/327: dwrite d0/f4d [0,4194304] 0 2024-02-15T18:26:55.687 INFO:tasks.workunit.client.0.smithi042.stdout:5/328: fsync d0/d22/d7b/f66 0 2024-02-15T18:26:55.687 INFO:tasks.workunit.client.0.smithi042.stdout:4/291: getdents da/d3a 0 2024-02-15T18:26:55.687 INFO:tasks.workunit.client.0.smithi042.stdout:4/292: truncate da/f63 337740 0 2024-02-15T18:26:55.687 INFO:tasks.workunit.client.0.smithi042.stdout:7/352: dwrite d10/d15/f34 [0,4194304] 0 2024-02-15T18:26:55.687 INFO:tasks.workunit.client.0.smithi042.stdout:7/353: readlink d10/d15/d39/l60 0 2024-02-15T18:26:55.687 INFO:tasks.workunit.client.0.smithi042.stdout:7/354: chown d10/d12/f5f 58 1 2024-02-15T18:26:55.688 INFO:tasks.workunit.client.0.smithi042.stdout:8/285: dread dd/f29 [0,4194304] 0 2024-02-15T18:26:55.689 INFO:tasks.workunit.client.0.smithi042.stdout:3/353: rename d1/d11/d5d to d1/de/d1a/d74 0 2024-02-15T18:26:55.689 INFO:tasks.workunit.client.0.smithi042.stdout:3/354: dread - d1/d11/d2e/d4c/f58 zero size 2024-02-15T18:26:55.692 INFO:tasks.workunit.client.0.smithi042.stdout:6/395: symlink d9/d11/d19/d35/d55/l86 0 2024-02-15T18:26:55.697 INFO:tasks.workunit.client.0.smithi042.stdout:5/329: symlink d0/d12/d65/d74/d52/l7f 0 2024-02-15T18:26:55.698 INFO:tasks.workunit.client.0.smithi042.stdout:4/293: mknod da/d11/c64 0 2024-02-15T18:26:55.700 INFO:tasks.workunit.client.0.smithi042.stdout:7/355: mknod d10/d12/d23/d51/c76 0 2024-02-15T18:26:55.700 INFO:tasks.workunit.client.0.smithi042.stdout:7/356: write d10/d12/d23/f55 [831831,5384] 0 2024-02-15T18:26:55.700 INFO:tasks.workunit.client.0.smithi042.stdout:8/286: creat dd/de/d33/f53 x:0 0 0 2024-02-15T18:26:55.700 INFO:tasks.workunit.client.0.smithi042.stdout:0/327: sync 2024-02-15T18:26:55.700 INFO:tasks.workunit.client.0.smithi042.stdout:0/328: write d4/d9/d34/d4d/f67 [305722,72371] 0 2024-02-15T18:26:55.700 INFO:tasks.workunit.client.0.smithi042.stdout:0/329: truncate d4/d9/d62/f53 878969 0 2024-02-15T18:26:55.702 INFO:tasks.workunit.client.0.smithi042.stdout:9/298: rename d6/dd/d4f to d6/da/d2f/d69 0 2024-02-15T18:26:55.704 INFO:tasks.workunit.client.0.smithi042.stdout:1/335: dwrite de/d15/d67/f45 [0,4194304] 0 2024-02-15T18:26:55.704 INFO:tasks.workunit.client.0.smithi042.stdout:4/294: link da/d3a/f44 da/d11/d29/d4b/d5c/f65 0 2024-02-15T18:26:55.706 INFO:tasks.workunit.client.0.smithi042.stdout:7/357: mknod d10/d12/c77 0 2024-02-15T18:26:55.707 INFO:tasks.workunit.client.0.smithi042.stdout:6/396: dwrite d9/d11/f20 [0,4194304] 0 2024-02-15T18:26:55.707 INFO:tasks.workunit.client.0.smithi042.stdout:0/330: creat d4/d9/d34/d4e/d6c/f71 x:0 0 0 2024-02-15T18:26:55.709 INFO:tasks.workunit.client.0.smithi042.stdout:2/301: sync 2024-02-15T18:26:55.709 INFO:tasks.workunit.client.0.smithi042.stdout:2/302: write d0/d8/d10/d31/d3a/f50 [617011,129781] 0 2024-02-15T18:26:55.709 INFO:tasks.workunit.client.0.smithi042.stdout:0/331: write d4/d16/d2e/f32 [3075596,101063] 0 2024-02-15T18:26:55.709 INFO:tasks.workunit.client.0.smithi042.stdout:0/332: chown d4/d38/c47 30097 1 2024-02-15T18:26:55.710 INFO:tasks.workunit.client.0.smithi042.stdout:2/303: write d0/d8/d10/d31/d3a/d53/d3c/f30 [6589220,15460] 0 2024-02-15T18:26:55.712 INFO:tasks.workunit.client.0.smithi042.stdout:3/355: rename d1/f32 to d1/d11/d14/d4d/d6c/f75 0 2024-02-15T18:26:55.713 INFO:tasks.workunit.client.0.smithi042.stdout:9/299: creat d6/da/d2f/f6a x:0 0 0 2024-02-15T18:26:55.717 INFO:tasks.workunit.client.0.smithi042.stdout:1/336: symlink de/d15/d1b/l79 0 2024-02-15T18:26:55.717 INFO:tasks.workunit.client.0.smithi042.stdout:5/330: dwrite d0/d39/f75 [0,4194304] 0 2024-02-15T18:26:55.718 INFO:tasks.workunit.client.0.smithi042.stdout:5/331: dread - d0/d38/d53/d68/d6a/f40 zero size 2024-02-15T18:26:55.719 INFO:tasks.workunit.client.0.smithi042.stdout:4/295: mkdir da/d11/d29/d4b/d5c/d66 0 2024-02-15T18:26:55.720 INFO:tasks.workunit.client.0.smithi042.stdout:7/358: mkdir d10/d49/d6d/d78 0 2024-02-15T18:26:55.721 INFO:tasks.workunit.client.0.smithi042.stdout:6/397: symlink d9/d11/d19/l87 0 2024-02-15T18:26:55.722 INFO:tasks.workunit.client.0.smithi042.stdout:3/356: getdents d1/d11 0 2024-02-15T18:26:55.723 INFO:tasks.workunit.client.0.smithi042.stdout:9/300: rmdir d6/dd/d9 39 2024-02-15T18:26:55.723 INFO:tasks.workunit.client.0.smithi042.stdout:9/301: write d6/da/f29 [912273,49203] 0 2024-02-15T18:26:55.724 INFO:tasks.workunit.client.0.smithi042.stdout:1/337: rmdir de/d15/d67/d63 39 2024-02-15T18:26:55.724 INFO:tasks.workunit.client.0.smithi042.stdout:1/338: fsync de/f21 0 2024-02-15T18:26:55.726 INFO:tasks.workunit.client.0.smithi042.stdout:8/287: dwrite dd/de/d12/f24 [0,4194304] 0 2024-02-15T18:26:55.726 INFO:tasks.workunit.client.0.smithi042.stdout:8/288: dread - dd/de/d12/d32/f3f zero size 2024-02-15T18:26:55.726 INFO:tasks.workunit.client.0.smithi042.stdout:8/289: truncate dd/de/f2a 1444071 0 2024-02-15T18:26:55.727 INFO:tasks.workunit.client.0.smithi042.stdout:7/359: creat d10/d43/d57/f79 x:0 0 0 2024-02-15T18:26:55.727 INFO:tasks.workunit.client.0.smithi042.stdout:6/398: creat d9/d11/d1d/f88 x:0 0 0 2024-02-15T18:26:55.728 INFO:tasks.workunit.client.0.smithi042.stdout:3/357: rmdir d1/de/d19 39 2024-02-15T18:26:55.730 INFO:tasks.workunit.client.0.smithi042.stdout:9/302: mknod d6/da/d1d/d2d/c6b 0 2024-02-15T18:26:55.731 INFO:tasks.workunit.client.0.smithi042.stdout:7/360: chown d10/d12/d23/l2c 0 1 2024-02-15T18:26:55.731 INFO:tasks.workunit.client.0.smithi042.stdout:3/358: rmdir d1/d11/d2e 39 2024-02-15T18:26:55.733 INFO:tasks.workunit.client.0.smithi042.stdout:9/303: mkdir d6/da/d2f/d6c 0 2024-02-15T18:26:55.733 INFO:tasks.workunit.client.0.smithi042.stdout:9/304: write d6/f67 [1193541,125328] 0 2024-02-15T18:26:55.734 INFO:tasks.workunit.client.0.smithi042.stdout:3/359: creat d1/d11/d2e/d4c/f76 x:0 0 0 2024-02-15T18:26:55.735 INFO:tasks.workunit.client.0.smithi042.stdout:3/360: write d1/d11/d2e/d4c/f5b [240919,120938] 0 2024-02-15T18:26:55.735 INFO:tasks.workunit.client.0.smithi042.stdout:3/361: chown d1/d11/d2e/f37 91834 1 2024-02-15T18:26:55.766 INFO:tasks.workunit.client.0.smithi042.stdout:2/304: dwrite d0/d8/d10/f54 [0,4194304] 0 2024-02-15T18:26:55.766 INFO:tasks.workunit.client.0.smithi042.stdout:0/333: dwrite d4/d9/d62/f53 [0,4194304] 0 2024-02-15T18:26:55.767 INFO:tasks.workunit.client.0.smithi042.stdout:2/305: write d0/d8/d10/f17 [167651,19447] 0 2024-02-15T18:26:55.772 INFO:tasks.workunit.client.0.smithi042.stdout:4/296: rename da/d11/d29/d4b to da/d11/d67 0 2024-02-15T18:26:55.772 INFO:tasks.workunit.client.0.smithi042.stdout:4/297: rename da/d11 to da/d11/d1b/d68 22 2024-02-15T18:26:55.772 INFO:tasks.workunit.client.0.smithi042.stdout:4/298: chown da/d11/d29/l5b 485905148 1 2024-02-15T18:26:55.772 INFO:tasks.workunit.client.0.smithi042.stdout:5/332: dwrite d0/d38/f3c [0,4194304] 0 2024-02-15T18:26:55.772 INFO:tasks.workunit.client.0.smithi042.stdout:5/333: truncate d0/db/f35 917195 0 2024-02-15T18:26:55.772 INFO:tasks.workunit.client.0.smithi042.stdout:5/334: chown d0/d38/d53/d68/d6a/c3d 94 1 2024-02-15T18:26:55.772 INFO:tasks.workunit.client.0.smithi042.stdout:5/335: chown d0/c2c 1080419 1 2024-02-15T18:26:55.775 INFO:tasks.workunit.client.0.smithi042.stdout:2/306: symlink d0/d40/d56/l5b 0 2024-02-15T18:26:55.775 INFO:tasks.workunit.client.0.smithi042.stdout:4/299: creat da/d11/d27/f69 x:0 0 0 2024-02-15T18:26:55.775 INFO:tasks.workunit.client.0.smithi042.stdout:4/300: stat da/d3b/c54 0 2024-02-15T18:26:55.775 INFO:tasks.workunit.client.0.smithi042.stdout:4/301: write da/d11/d67/f62 [293414,105724] 0 2024-02-15T18:26:55.777 INFO:tasks.workunit.client.0.smithi042.stdout:2/307: symlink d0/d8/l5c 0 2024-02-15T18:26:55.777 INFO:tasks.workunit.client.0.smithi042.stdout:2/308: read d0/f3 [8821742,90162] 0 2024-02-15T18:26:55.779 INFO:tasks.workunit.client.0.smithi042.stdout:7/361: dwrite d10/f53 [0,4194304] 0 2024-02-15T18:26:55.779 INFO:tasks.workunit.client.0.smithi042.stdout:7/362: chown d10/d49/d6d 2073174 1 2024-02-15T18:26:55.780 INFO:tasks.workunit.client.0.smithi042.stdout:1/339: dwrite de/d15/d1b/f49 [0,4194304] 0 2024-02-15T18:26:55.781 INFO:tasks.workunit.client.0.smithi042.stdout:0/334: read d4/d16/d2e/f32 [1966045,109631] 0 2024-02-15T18:26:55.781 INFO:tasks.workunit.client.0.smithi042.stdout:9/305: dwrite d6/dd/d9/f14 [0,4194304] 0 2024-02-15T18:26:55.783 INFO:tasks.workunit.client.0.smithi042.stdout:6/399: dwrite d9/d11/d46/f77 [0,4194304] 0 2024-02-15T18:26:55.783 INFO:tasks.workunit.client.0.smithi042.stdout:2/309: dread d0/f49 [0,4194304] 0 2024-02-15T18:26:55.783 INFO:tasks.workunit.client.0.smithi042.stdout:8/290: dwrite dd/f2c [0,4194304] 0 2024-02-15T18:26:55.783 INFO:tasks.workunit.client.0.smithi042.stdout:6/400: chown d9/d11/d19/d1c/d31 894257039 1 2024-02-15T18:26:55.783 INFO:tasks.workunit.client.0.smithi042.stdout:1/340: dread de/f29 [0,4194304] 0 2024-02-15T18:26:55.787 INFO:tasks.workunit.client.0.smithi042.stdout:5/336: dread d0/db/fd [0,4194304] 0 2024-02-15T18:26:55.789 INFO:tasks.workunit.client.0.smithi042.stdout:4/302: dwrite da/d2b/f52 [0,4194304] 0 2024-02-15T18:26:55.789 INFO:tasks.workunit.client.0.smithi042.stdout:4/303: fsync da/d11/d67/d5c/f65 0 2024-02-15T18:26:55.790 INFO:tasks.workunit.client.0.smithi042.stdout:0/335: mknod d4/d9/d34/d4e/c72 0 2024-02-15T18:26:55.790 INFO:tasks.workunit.client.0.smithi042.stdout:0/336: dread - d4/f68 zero size 2024-02-15T18:26:55.790 INFO:tasks.workunit.client.0.smithi042.stdout:4/304: write da/f63 [129034,100048] 0 2024-02-15T18:26:55.790 INFO:tasks.workunit.client.0.smithi042.stdout:4/305: fsync f9 0 2024-02-15T18:26:55.790 INFO:tasks.workunit.client.0.smithi042.stdout:0/337: dread d4/d38/d5c/f61 [0,4194304] 0 2024-02-15T18:26:55.793 INFO:tasks.workunit.client.0.smithi042.stdout:9/306: rename d6/f67 to d6/da/d2f/d55/f6d 0 2024-02-15T18:26:55.797 INFO:tasks.workunit.client.0.smithi042.stdout:2/310: creat d0/d8/d10/d31/d3a/d53/f5d x:0 0 0 2024-02-15T18:26:55.798 INFO:tasks.workunit.client.0.smithi042.stdout:8/291: creat dd/de/d3c/f54 x:0 0 0 2024-02-15T18:26:55.799 INFO:tasks.workunit.client.0.smithi042.stdout:6/401: mkdir d9/d11/d46/d89 0 2024-02-15T18:26:55.801 INFO:tasks.workunit.client.0.smithi042.stdout:1/341: mknod de/d6d/c7a 0 2024-02-15T18:26:55.801 INFO:tasks.workunit.client.0.smithi042.stdout:1/342: fdatasync de/d15/d2c/d51/d70/f78 0 2024-02-15T18:26:55.802 INFO:tasks.workunit.client.0.smithi042.stdout:7/363: symlink d10/d12/d23/l7a 0 2024-02-15T18:26:55.804 INFO:tasks.workunit.client.0.smithi042.stdout:7/364: dread d10/d12/d23/f3b [0,4194304] 0 2024-02-15T18:26:55.805 INFO:tasks.workunit.client.1.smithi191.stdout:3/5: sync 2024-02-15T18:26:55.806 INFO:tasks.workunit.client.0.smithi042.stdout:5/337: symlink d0/d39/l80 0 2024-02-15T18:26:55.807 INFO:tasks.workunit.client.1.smithi191.stdout:3/6: symlink l0 0 2024-02-15T18:26:55.810 INFO:tasks.workunit.client.0.smithi042.stdout:1/343: dread de/d15/d67/f26 [0,4194304] 0 2024-02-15T18:26:55.810 INFO:tasks.workunit.client.0.smithi042.stdout:1/344: dread - de/d15/f43 zero size 2024-02-15T18:26:55.812 INFO:tasks.workunit.client.1.smithi191.stdout:3/7: mknod c1 0 2024-02-15T18:26:55.814 INFO:tasks.workunit.client.0.smithi042.stdout:4/306: getdents da/d11/d67 0 2024-02-15T18:26:55.814 INFO:tasks.workunit.client.0.smithi042.stdout:4/307: getdents da/d11/d67 0 2024-02-15T18:26:55.815 INFO:tasks.workunit.client.0.smithi042.stdout:0/338: rmdir d4/d16 39 2024-02-15T18:26:55.817 INFO:tasks.workunit.client.1.smithi191.stdout:3/8: creat f2 x:0 0 0 2024-02-15T18:26:55.817 INFO:tasks.workunit.client.1.smithi191.stdout:3/9: readlink l0 0 2024-02-15T18:26:55.817 INFO:tasks.workunit.client.1.smithi191.stdout:3/10: write f2 [97953,81634] 0 2024-02-15T18:26:55.819 INFO:tasks.workunit.client.0.smithi042.stdout:9/307: link d6/dd/l27 d6/da/d2f/d69/l6e 0 2024-02-15T18:26:55.820 INFO:tasks.workunit.client.0.smithi042.stdout:2/311: mkdir d0/d40/d5e 0 2024-02-15T18:26:55.820 INFO:tasks.workunit.client.0.smithi042.stdout:2/312: truncate d0/d8/d10/d31/d3a/d53/f58 699580 0 2024-02-15T18:26:55.821 INFO:tasks.workunit.client.0.smithi042.stdout:2/313: read d0/d8/d10/d31/d3a/f50 [274207,46409] 0 2024-02-15T18:26:55.821 INFO:tasks.workunit.client.0.smithi042.stdout:8/292: symlink dd/de/d33/l55 0 2024-02-15T18:26:55.821 INFO:tasks.workunit.client.0.smithi042.stdout:8/293: write dd/de/d3c/d4b/f4f [409924,85883] 0 2024-02-15T18:26:55.821 INFO:tasks.workunit.client.0.smithi042.stdout:8/294: chown dd/de/f2a 11658 1 2024-02-15T18:26:55.821 INFO:tasks.workunit.client.0.smithi042.stdout:6/402: link d9/d11/d19/d1c/l7e d9/d11/d19/d35/l8a 0 2024-02-15T18:26:55.823 INFO:tasks.workunit.client.0.smithi042.stdout:1/345: mknod de/c7b 0 2024-02-15T18:26:55.828 INFO:tasks.workunit.client.0.smithi042.stdout:9/308: dread d6/da/f25 [0,4194304] 0 2024-02-15T18:26:55.830 INFO:tasks.workunit.client.0.smithi042.stdout:5/338: dwrite d0/d14/f77 [0,4194304] 0 2024-02-15T18:26:55.830 INFO:tasks.workunit.client.0.smithi042.stdout:5/339: fsync d0/d14/f77 0 2024-02-15T18:26:55.840 INFO:tasks.workunit.client.0.smithi042.stdout:6/403: dwrite d9/d11/d19/d35/d49/f52 [0,4194304] 0 2024-02-15T18:26:55.844 INFO:tasks.workunit.client.0.smithi042.stdout:1/346: dwrite de/f29 [0,4194304] 0 2024-02-15T18:26:55.844 INFO:tasks.workunit.client.0.smithi042.stdout:9/309: dwrite d6/dd/d9/d59/f63 [0,4194304] 0 2024-02-15T18:26:55.921 INFO:tasks.workunit.client.1.smithi191.stdout:7/17: sync 2024-02-15T18:26:55.922 INFO:tasks.workunit.client.1.smithi191.stdout:5/1: sync 2024-02-15T18:26:55.922 INFO:tasks.workunit.client.1.smithi191.stdout:5/2: dread - no filename 2024-02-15T18:26:55.922 INFO:tasks.workunit.client.1.smithi191.stdout:0/0: sync 2024-02-15T18:26:55.937 INFO:tasks.workunit.client.0.smithi042.stdout:0/339: creat d4/d9/f73 x:0 0 0 2024-02-15T18:26:55.940 INFO:tasks.workunit.client.0.smithi042.stdout:2/314: creat d0/d8/f5f x:0 0 0 2024-02-15T18:26:55.940 INFO:tasks.workunit.client.0.smithi042.stdout:2/315: write d0/d8/db/f42 [317695,61036] 0 2024-02-15T18:26:55.942 INFO:tasks.workunit.client.0.smithi042.stdout:8/295: creat dd/de/d33/d3b/f56 x:0 0 0 2024-02-15T18:26:55.942 INFO:tasks.workunit.client.0.smithi042.stdout:8/296: chown dd/f2c 204294 1 2024-02-15T18:26:55.943 INFO:tasks.workunit.client.1.smithi191.stdout:5/3: mkdir d0 0 2024-02-15T18:26:55.943 INFO:tasks.workunit.client.1.smithi191.stdout:5/4: dread - no filename 2024-02-15T18:26:55.943 INFO:tasks.workunit.client.1.smithi191.stdout:5/5: fsync - no filename 2024-02-15T18:26:55.943 INFO:tasks.workunit.client.1.smithi191.stdout:5/6: dwrite - no filename 2024-02-15T18:26:55.946 INFO:tasks.workunit.client.0.smithi042.stdout:8/297: dread dd/de/d22/f2d [0,4194304] 0 2024-02-15T18:26:55.947 INFO:tasks.workunit.client.1.smithi191.stdout:7/18: mknod d0/c6 0 2024-02-15T18:26:55.947 INFO:tasks.workunit.client.1.smithi191.stdout:7/19: write d0/d1/d3/f4 [96554,119419] 0 2024-02-15T18:26:55.947 INFO:tasks.workunit.client.1.smithi191.stdout:7/20: dread - d0/f5 zero size 2024-02-15T18:26:55.948 INFO:tasks.workunit.client.0.smithi042.stdout:5/340: creat d0/d38/d53/d64/f81 x:0 0 0 2024-02-15T18:26:55.948 INFO:tasks.workunit.client.0.smithi042.stdout:5/341: fdatasync d0/d14/f1c 0 2024-02-15T18:26:55.950 INFO:tasks.workunit.client.0.smithi042.stdout:6/404: creat d9/d3e/f8b x:0 0 0 2024-02-15T18:26:55.952 INFO:tasks.workunit.client.0.smithi042.stdout:1/347: unlink de/d15/l17 0 2024-02-15T18:26:55.955 INFO:tasks.workunit.client.0.smithi042.stdout:0/340: mkdir d4/d9/d62/d74 0 2024-02-15T18:26:55.956 INFO:tasks.workunit.client.0.smithi042.stdout:6/405: write d9/d11/d19/d1c/d31/f5b [2263861,45305] 0 2024-02-15T18:26:55.956 INFO:tasks.workunit.client.0.smithi042.stdout:6/406: fdatasync d9/d3e/f3b 0 2024-02-15T18:26:55.956 INFO:tasks.workunit.client.0.smithi042.stdout:2/316: truncate d0/d8/db/de/f4d 4425838 0 2024-02-15T18:26:55.956 INFO:tasks.workunit.client.0.smithi042.stdout:9/310: write d6/f8 [2392285,124627] 0 2024-02-15T18:26:55.957 INFO:tasks.workunit.client.0.smithi042.stdout:8/298: mknod dd/de/c57 0 2024-02-15T18:26:55.957 INFO:tasks.workunit.client.0.smithi042.stdout:5/342: mknod d0/d12/c82 0 2024-02-15T18:26:55.957 INFO:tasks.workunit.client.0.smithi042.stdout:0/341: truncate d4/d38/d5c/f61 82572 0 2024-02-15T18:26:55.959 INFO:tasks.workunit.client.0.smithi042.stdout:2/317: creat d0/d8/d10/d31/d3a/d53/d3c/f60 x:0 0 0 2024-02-15T18:26:55.959 INFO:tasks.workunit.client.0.smithi042.stdout:2/318: write d0/d8/db/de/d1a/f2b [540433,25259] 0 2024-02-15T18:26:55.960 INFO:tasks.workunit.client.0.smithi042.stdout:9/311: symlink d6/da/d2f/d6c/l6f 0 2024-02-15T18:26:55.960 INFO:tasks.workunit.client.0.smithi042.stdout:8/299: symlink dd/de/d33/l58 0 2024-02-15T18:26:55.962 INFO:tasks.workunit.client.0.smithi042.stdout:5/343: symlink d0/d22/l83 0 2024-02-15T18:26:55.962 INFO:tasks.workunit.client.0.smithi042.stdout:5/344: readlink d0/d12/l3f 0 2024-02-15T18:26:55.963 INFO:tasks.workunit.client.0.smithi042.stdout:5/345: rename d0 to d0/d12/d65/d74/d52/d84 22 2024-02-15T18:26:55.963 INFO:tasks.workunit.client.0.smithi042.stdout:0/342: unlink d4/d9/f73 0 2024-02-15T18:26:55.963 INFO:tasks.workunit.client.0.smithi042.stdout:0/343: readlink d4/d16/d2e/l45 0 2024-02-15T18:26:55.964 INFO:tasks.workunit.client.0.smithi042.stdout:6/407: symlink d9/l8c 0 2024-02-15T18:26:55.965 INFO:tasks.workunit.client.0.smithi042.stdout:2/319: dread d0/d8/db/de/f14 [0,4194304] 0 2024-02-15T18:26:55.965 INFO:tasks.workunit.client.0.smithi042.stdout:9/312: creat d6/da/d2f/d69/f70 x:0 0 0 2024-02-15T18:26:55.966 INFO:tasks.workunit.client.0.smithi042.stdout:0/344: mknod d4/d9/d5a/c75 0 2024-02-15T18:26:55.967 INFO:tasks.workunit.client.0.smithi042.stdout:2/320: mkdir d0/d8/d10/d61 0 2024-02-15T18:26:55.968 INFO:tasks.workunit.client.1.smithi191.stdout:7/21: rmdir d0 39 2024-02-15T18:26:55.972 INFO:tasks.workunit.client.0.smithi042.stdout:7/365: sync 2024-02-15T18:26:55.973 INFO:tasks.workunit.client.0.smithi042.stdout:4/308: sync 2024-02-15T18:26:55.973 INFO:tasks.workunit.client.0.smithi042.stdout:3/362: sync 2024-02-15T18:26:55.979 INFO:tasks.workunit.client.0.smithi042.stdout:9/313: rmdir d6/da/d1d/d2d 39 2024-02-15T18:26:55.979 INFO:tasks.workunit.client.0.smithi042.stdout:9/314: chown f3 2193 1 2024-02-15T18:26:55.980 INFO:tasks.workunit.client.0.smithi042.stdout:0/345: truncate d4/f6 291544 0 2024-02-15T18:26:55.980 INFO:tasks.workunit.client.0.smithi042.stdout:0/346: fsync d4/d9/d15/f18 0 2024-02-15T18:26:55.980 INFO:tasks.workunit.client.0.smithi042.stdout:0/347: chown d4/d9/f46 3 1 2024-02-15T18:26:55.980 INFO:tasks.workunit.client.0.smithi042.stdout:8/300: dwrite dd/de/d3c/f51 [0,4194304] 0 2024-02-15T18:26:55.981 INFO:tasks.workunit.client.0.smithi042.stdout:7/366: creat d10/f7b x:0 0 0 2024-02-15T18:26:55.981 INFO:tasks.workunit.client.0.smithi042.stdout:4/309: symlink da/d3b/l6a 0 2024-02-15T18:26:55.982 INFO:tasks.workunit.client.0.smithi042.stdout:3/363: creat d1/de/d1a/f77 x:0 0 0 2024-02-15T18:26:55.985 INFO:tasks.workunit.client.0.smithi042.stdout:9/315: rename d6/da/f3e to d6/da/d2f/d6c/f71 0 2024-02-15T18:26:55.985 INFO:tasks.workunit.client.0.smithi042.stdout:9/316: readlink d6/da/d1d/d2d/l36 0 2024-02-15T18:26:55.985 INFO:tasks.workunit.client.0.smithi042.stdout:9/317: truncate d6/dd/d9/f14 4299150 0 2024-02-15T18:26:55.986 INFO:tasks.workunit.client.0.smithi042.stdout:8/301: getdents dd/de/d33/d3b 0 2024-02-15T18:26:55.986 INFO:tasks.workunit.client.0.smithi042.stdout:8/302: write dd/de/d12/f4c [109797,74877] 0 2024-02-15T18:26:55.986 INFO:tasks.workunit.client.0.smithi042.stdout:8/303: chown dd/de/d33 30795 1 2024-02-15T18:26:55.986 INFO:tasks.workunit.client.0.smithi042.stdout:7/367: creat d10/d49/d6d/f7c x:0 0 0 2024-02-15T18:26:55.987 INFO:tasks.workunit.client.0.smithi042.stdout:8/304: read dd/de/d3c/f51 [2642479,73718] 0 2024-02-15T18:26:55.989 INFO:tasks.workunit.client.0.smithi042.stdout:4/310: rename da/f21 to da/d11/d29/d4a/f6b 0 2024-02-15T18:26:55.990 INFO:tasks.workunit.client.0.smithi042.stdout:9/318: truncate d6/dd/d9/f32 3826105 0 2024-02-15T18:26:55.992 INFO:tasks.workunit.client.0.smithi042.stdout:9/319: creat d6/dd/d9/f72 x:0 0 0 2024-02-15T18:26:55.993 INFO:tasks.workunit.client.0.smithi042.stdout:9/320: mkdir d6/da/d2f/d55/d73 0 2024-02-15T18:26:55.998 INFO:tasks.workunit.client.1.smithi191.stdout:7/22: dwrite d0/d1/d3/f4 [0,4194304] 0 2024-02-15T18:26:55.999 INFO:tasks.workunit.client.1.smithi191.stdout:7/23: write d0/f5 [683806,32175] 0 2024-02-15T18:26:56.002 INFO:tasks.workunit.client.0.smithi042.stdout:9/321: dread d6/da/d2f/d6c/f71 [0,4194304] 0 2024-02-15T18:26:56.005 INFO:tasks.workunit.client.0.smithi042.stdout:7/368: write d10/d12/f3f [1738731,101407] 0 2024-02-15T18:26:56.005 INFO:tasks.workunit.client.0.smithi042.stdout:5/346: dwrite d0/d38/d53/d68/f6e [0,4194304] 0 2024-02-15T18:26:56.005 INFO:tasks.workunit.client.0.smithi042.stdout:8/305: dread dd/de/ff [0,4194304] 0 2024-02-15T18:26:56.005 INFO:tasks.workunit.client.0.smithi042.stdout:7/369: dread - d10/d43/d57/f79 zero size 2024-02-15T18:26:56.006 INFO:tasks.workunit.client.0.smithi042.stdout:3/364: dwrite d1/de/d1a/f5a [0,4194304] 0 2024-02-15T18:26:56.007 INFO:tasks.workunit.client.0.smithi042.stdout:3/365: readlink d1/d11/d14/d4d/l68 0 2024-02-15T18:26:56.008 INFO:tasks.workunit.client.0.smithi042.stdout:7/370: write d10/d12/d23/f27 [3445713,120904] 0 2024-02-15T18:26:56.010 INFO:tasks.workunit.client.1.smithi191.stdout:7/24: dread d0/d1/d3/f4 [0,4194304] 0 2024-02-15T18:26:56.013 INFO:tasks.workunit.client.0.smithi042.stdout:8/306: write dd/f2f [262550,39035] 0 2024-02-15T18:26:56.013 INFO:tasks.workunit.client.0.smithi042.stdout:4/311: dwrite da/d11/d1b/f1c [4194304,4194304] 0 2024-02-15T18:26:56.013 INFO:tasks.workunit.client.0.smithi042.stdout:4/312: dread - da/d11/d1b/d2f/f34 zero size 2024-02-15T18:26:56.013 INFO:tasks.workunit.client.0.smithi042.stdout:8/307: read dd/d11/f15 [8457745,94932] 0 2024-02-15T18:26:56.013 INFO:tasks.workunit.client.0.smithi042.stdout:8/308: write dd/de/d33/f53 [831736,27774] 0 2024-02-15T18:26:56.013 INFO:tasks.workunit.client.0.smithi042.stdout:5/347: creat d0/d38/d53/d68/d6a/f85 x:0 0 0 2024-02-15T18:26:56.014 INFO:tasks.workunit.client.1.smithi191.stdout:7/25: mknod d0/d1/c7 0 2024-02-15T18:26:56.014 INFO:tasks.workunit.client.1.smithi191.stdout:7/26: write d0/f5 [1638108,16834] 0 2024-02-15T18:26:56.014 INFO:tasks.workunit.client.0.smithi042.stdout:5/348: dread d0/f47 [0,4194304] 0 2024-02-15T18:26:56.018 INFO:tasks.workunit.client.1.smithi191.stdout:7/27: dread d0/d1/d3/f4 [0,4194304] 0 2024-02-15T18:26:56.018 INFO:tasks.workunit.client.0.smithi042.stdout:9/322: dwrite f2 [0,4194304] 0 2024-02-15T18:26:56.019 INFO:tasks.workunit.client.1.smithi191.stdout:7/28: write d0/f5 [454242,45890] 0 2024-02-15T18:26:56.019 INFO:tasks.workunit.client.0.smithi042.stdout:3/366: creat d1/de/d1a/d74/f78 x:0 0 0 2024-02-15T18:26:56.021 INFO:tasks.workunit.client.1.smithi191.stdout:1/23: sync 2024-02-15T18:26:56.022 INFO:tasks.workunit.client.1.smithi191.stdout:2/17: sync 2024-02-15T18:26:56.022 INFO:tasks.workunit.client.1.smithi191.stdout:4/10: sync 2024-02-15T18:26:56.022 INFO:tasks.workunit.client.1.smithi191.stdout:6/30: sync 2024-02-15T18:26:56.022 INFO:tasks.workunit.client.1.smithi191.stdout:6/31: write d8/d9/fa [403575,103746] 0 2024-02-15T18:26:56.022 INFO:tasks.workunit.client.1.smithi191.stdout:6/32: fsync f5 0 2024-02-15T18:26:56.022 INFO:tasks.workunit.client.1.smithi191.stdout:8/167: sync 2024-02-15T18:26:56.022 INFO:tasks.workunit.client.1.smithi191.stdout:9/12: sync 2024-02-15T18:26:56.022 INFO:tasks.workunit.client.0.smithi042.stdout:7/371: symlink d10/d15/d39/l7d 0 2024-02-15T18:26:56.023 INFO:tasks.workunit.client.1.smithi191.stdout:7/29: link d0/f5 d0/f8 0 2024-02-15T18:26:56.024 INFO:tasks.workunit.client.1.smithi191.stdout:4/11: rmdir d2 39 2024-02-15T18:26:56.025 INFO:tasks.workunit.client.0.smithi042.stdout:5/349: dread d0/f3 [0,4194304] 0 2024-02-15T18:26:56.025 INFO:tasks.workunit.client.0.smithi042.stdout:4/313: rmdir da/d11/d67/d5e 0 2024-02-15T18:26:56.025 INFO:tasks.workunit.client.0.smithi042.stdout:4/314: write da/d11/f22 [831445,79639] 0 2024-02-15T18:26:56.027 INFO:tasks.workunit.client.1.smithi191.stdout:2/18: rename l3 to l5 0 2024-02-15T18:26:56.030 INFO:tasks.workunit.client.0.smithi042.stdout:8/309: rmdir dd 39 2024-02-15T18:26:56.031 INFO:tasks.workunit.client.1.smithi191.stdout:1/24: mkdir d0/db 0 2024-02-15T18:26:56.033 INFO:tasks.workunit.client.1.smithi191.stdout:6/33: symlink d8/d9/lb 0 2024-02-15T18:26:56.034 INFO:tasks.workunit.client.1.smithi191.stdout:6/34: dread f0 [0,4194304] 0 2024-02-15T18:26:56.035 INFO:tasks.workunit.client.0.smithi042.stdout:9/323: link d6/da/d1d/d2d/d40/f49 d6/da/d2f/d69/f74 0 2024-02-15T18:26:56.036 INFO:tasks.workunit.client.0.smithi042.stdout:7/372: creat d10/f7e x:0 0 0 2024-02-15T18:26:56.036 INFO:tasks.workunit.client.1.smithi191.stdout:8/168: truncate d3/d2c/f1d 1926365 0 2024-02-15T18:26:56.038 INFO:tasks.workunit.client.0.smithi042.stdout:5/350: mkdir d0/d12/d65/d74/d86 0 2024-02-15T18:26:56.040 INFO:tasks.workunit.client.0.smithi042.stdout:7/373: dread d10/f2f [0,4194304] 0 2024-02-15T18:26:56.041 INFO:tasks.workunit.client.0.smithi042.stdout:7/374: rename d10/d49 to d10/d49/d6d/d6e/d7f 22 2024-02-15T18:26:56.041 INFO:tasks.workunit.client.0.smithi042.stdout:8/310: truncate dd/f13 109627 0 2024-02-15T18:26:56.041 INFO:tasks.workunit.client.0.smithi042.stdout:8/311: readlink dd/l16 0 2024-02-15T18:26:56.041 INFO:tasks.workunit.client.0.smithi042.stdout:7/375: getdents d10/d49/d6d/d78 0 2024-02-15T18:26:56.041 INFO:tasks.workunit.client.1.smithi191.stdout:9/13: link d0/c2 d0/c5 0 2024-02-15T18:26:56.041 INFO:tasks.workunit.client.1.smithi191.stdout:9/14: fsync d0/f3 0 2024-02-15T18:26:56.042 INFO:tasks.workunit.client.1.smithi191.stdout:7/30: creat d0/d1/f9 x:0 0 0 2024-02-15T18:26:56.043 INFO:tasks.workunit.client.0.smithi042.stdout:4/315: dread da/d11/d67/d5c/f65 [0,4194304] 0 2024-02-15T18:26:56.043 INFO:tasks.workunit.client.0.smithi042.stdout:9/324: unlink d6/da/f29 0 2024-02-15T18:26:56.043 INFO:tasks.workunit.client.1.smithi191.stdout:4/12: chown d2/f4 0 1 2024-02-15T18:26:56.044 INFO:tasks.workunit.client.0.smithi042.stdout:5/351: rmdir d0/d12 39 2024-02-15T18:26:56.044 INFO:tasks.workunit.client.0.smithi042.stdout:5/352: readlink d0/d38/d53/d68/d6a/d42/l44 0 2024-02-15T18:26:56.045 INFO:tasks.workunit.client.0.smithi042.stdout:8/312: read dd/d11/f39 [1194901,16417] 0 2024-02-15T18:26:56.045 INFO:tasks.workunit.client.0.smithi042.stdout:8/313: readlink dd/de/d22/l49 0 2024-02-15T18:26:56.045 INFO:tasks.workunit.client.0.smithi042.stdout:8/314: chown dd/de/d33 1535643 1 2024-02-15T18:26:56.046 INFO:tasks.workunit.client.0.smithi042.stdout:3/367: dwrite d1/d11/d2e/f37 [0,4194304] 0 2024-02-15T18:26:56.047 INFO:tasks.workunit.client.1.smithi191.stdout:1/25: rename d0/la to d0/db/lc 0 2024-02-15T18:26:56.048 INFO:tasks.workunit.client.0.smithi042.stdout:7/376: truncate d10/d12/d23/f55 616385 0 2024-02-15T18:26:56.048 INFO:tasks.workunit.client.1.smithi191.stdout:2/19: dwrite f1 [0,4194304] 0 2024-02-15T18:26:56.050 INFO:tasks.workunit.client.0.smithi042.stdout:4/316: creat da/d11/d29/d4a/d4f/f6c x:0 0 0 2024-02-15T18:26:56.050 INFO:tasks.workunit.client.0.smithi042.stdout:4/317: fdatasync da/d11/d29/d4a/d4f/f6c 0 2024-02-15T18:26:56.050 INFO:tasks.workunit.client.0.smithi042.stdout:4/318: write da/d11/d29/f30 [197306,5605] 0 2024-02-15T18:26:56.050 INFO:tasks.workunit.client.0.smithi042.stdout:4/319: write da/d11/d67/f51 [213392,92022] 0 2024-02-15T18:26:56.051 INFO:tasks.workunit.client.1.smithi191.stdout:7/31: mknod d0/d1/ca 0 2024-02-15T18:26:56.051 INFO:tasks.workunit.client.0.smithi042.stdout:9/325: mkdir d6/da/d75 0 2024-02-15T18:26:56.051 INFO:tasks.workunit.client.0.smithi042.stdout:9/326: chown d6/da/d1d/d2d 1398 1 2024-02-15T18:26:56.052 INFO:tasks.workunit.client.1.smithi191.stdout:4/13: rmdir d2 39 2024-02-15T18:26:56.055 INFO:tasks.workunit.client.1.smithi191.stdout:1/26: creat d0/d8/fd x:0 0 0 2024-02-15T18:26:56.056 INFO:tasks.workunit.client.0.smithi042.stdout:8/315: creat dd/de/d33/d3b/f59 x:0 0 0 2024-02-15T18:26:56.056 INFO:tasks.workunit.client.0.smithi042.stdout:5/353: creat d0/d14/f87 x:0 0 0 2024-02-15T18:26:56.056 INFO:tasks.workunit.client.0.smithi042.stdout:5/354: chown d0/d39/l78 16558127 1 2024-02-15T18:26:56.056 INFO:tasks.workunit.client.0.smithi042.stdout:3/368: creat d1/d11/d2e/f79 x:0 0 0 2024-02-15T18:26:56.057 INFO:tasks.workunit.client.1.smithi191.stdout:6/35: dwrite f6 [4194304,4194304] 0 2024-02-15T18:26:56.058 INFO:tasks.workunit.client.0.smithi042.stdout:3/369: write d1/de/f42 [82684,15340] 0 2024-02-15T18:26:56.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:54.680555+0000 mgr.smithi042.kfoihb (mgr.24435) 35 : cluster [DBG] pgmap v26: 65 pgs: 65 active+clean; 380 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 275 op/s 2024-02-15T18:26:56.060 INFO:tasks.workunit.client.0.smithi042.stdout:9/327: dread d6/dd/d9/f45 [0,4194304] 0 2024-02-15T18:26:56.068 INFO:tasks.workunit.client.0.smithi042.stdout:7/377: symlink d10/d49/d6d/l80 0 2024-02-15T18:26:56.069 INFO:tasks.workunit.client.0.smithi042.stdout:4/320: creat da/d11/f6d x:0 0 0 2024-02-15T18:26:56.069 INFO:tasks.workunit.client.0.smithi042.stdout:4/321: fsync da/d11/d1b/f1c 0 2024-02-15T18:26:56.070 INFO:tasks.workunit.client.1.smithi191.stdout:9/15: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:26:56.070 INFO:tasks.workunit.client.1.smithi191.stdout:9/16: chown d0/f3 21577 1 2024-02-15T18:26:56.070 INFO:tasks.workunit.client.1.smithi191.stdout:2/20: symlink l6 0 2024-02-15T18:26:56.072 INFO:tasks.workunit.client.0.smithi042.stdout:4/322: dread da/d11/d1b/d2f/d4d/f47 [0,4194304] 0 2024-02-15T18:26:56.072 INFO:tasks.workunit.client.0.smithi042.stdout:4/323: dread - da/d11/d27/f69 zero size 2024-02-15T18:26:56.072 INFO:tasks.workunit.client.0.smithi042.stdout:4/324: truncate da/d11/f22 1247712 0 2024-02-15T18:26:56.072 INFO:tasks.workunit.client.0.smithi042.stdout:4/325: stat da/d11/d29/l5b 0 2024-02-15T18:26:56.072 INFO:tasks.workunit.client.0.smithi042.stdout:8/316: unlink dd/de/d3c/d4b/f4f 0 2024-02-15T18:26:56.073 INFO:tasks.workunit.client.1.smithi191.stdout:8/169: dwrite d3/f39 [0,4194304] 0 2024-02-15T18:26:56.073 INFO:tasks.workunit.client.1.smithi191.stdout:7/32: mkdir d0/db 0 2024-02-15T18:26:56.073 INFO:tasks.workunit.client.0.smithi042.stdout:5/355: mkdir d0/d14/d88 0 2024-02-15T18:26:56.074 INFO:tasks.workunit.client.1.smithi191.stdout:4/14: rmdir d2 39 2024-02-15T18:26:56.076 INFO:tasks.workunit.client.1.smithi191.stdout:1/27: creat d0/db/fe x:0 0 0 2024-02-15T18:26:56.076 INFO:tasks.workunit.client.1.smithi191.stdout:1/28: fdatasync d0/f3 0 2024-02-15T18:26:56.079 INFO:tasks.workunit.client.1.smithi191.stdout:2/21: dread f1 [0,4194304] 0 2024-02-15T18:26:56.079 INFO:tasks.workunit.client.0.smithi042.stdout:3/370: mkdir d1/d11/d7a 0 2024-02-15T18:26:56.082 INFO:tasks.workunit.client.1.smithi191.stdout:7/33: dread d0/f8 [0,4194304] 0 2024-02-15T18:26:56.082 INFO:tasks.workunit.client.1.smithi191.stdout:7/34: fdatasync d0/d1/f9 0 2024-02-15T18:26:56.084 INFO:tasks.workunit.client.1.smithi191.stdout:6/36: dwrite f1 [0,4194304] 0 2024-02-15T18:26:56.085 INFO:tasks.workunit.client.1.smithi191.stdout:7/35: dread d0/d1/d3/f4 [0,4194304] 0 2024-02-15T18:26:56.085 INFO:tasks.workunit.client.1.smithi191.stdout:9/17: unlink d0/f3 0 2024-02-15T18:26:56.087 INFO:tasks.workunit.client.0.smithi042.stdout:9/328: chown d6/f31 135 1 2024-02-15T18:26:56.094 INFO:tasks.workunit.client.1.smithi191.stdout:8/170: creat d3/d2c/d1a/d23/d2e/f3a x:0 0 0 2024-02-15T18:26:56.098 INFO:tasks.workunit.client.1.smithi191.stdout:1/29: mkdir d0/d8/df 0 2024-02-15T18:26:56.101 INFO:tasks.workunit.client.1.smithi191.stdout:2/22: unlink l4 0 2024-02-15T18:26:56.103 INFO:tasks.workunit.client.1.smithi191.stdout:7/36: dwrite d0/d1/d3/f4 [0,4194304] 0 2024-02-15T18:26:56.108 INFO:tasks.workunit.client.0.smithi042.stdout:7/378: symlink d10/d12/d23/d51/l81 0 2024-02-15T18:26:56.108 INFO:tasks.workunit.client.1.smithi191.stdout:6/37: creat d8/d9/fc x:0 0 0 2024-02-15T18:26:56.108 INFO:tasks.workunit.client.1.smithi191.stdout:6/38: truncate d8/d9/fc 1011004 0 2024-02-15T18:26:56.108 INFO:tasks.workunit.client.1.smithi191.stdout:6/39: stat l7 0 2024-02-15T18:26:56.109 INFO:tasks.workunit.client.1.smithi191.stdout:6/40: readlink d8/d9/lb 0 2024-02-15T18:26:56.109 INFO:tasks.workunit.client.0.smithi042.stdout:4/326: truncate da/f63 104966 0 2024-02-15T18:26:56.109 INFO:tasks.workunit.client.0.smithi042.stdout:4/327: truncate da/d11/d29/d4a/d4f/f6c 993644 0 2024-02-15T18:26:56.110 INFO:tasks.workunit.client.1.smithi191.stdout:9/18: symlink d0/l6 0 2024-02-15T18:26:56.110 INFO:tasks.workunit.client.1.smithi191.stdout:9/19: write d0/f4 [703085,50717] 0 2024-02-15T18:26:56.111 INFO:tasks.workunit.client.0.smithi042.stdout:5/356: mkdir d0/d89 0 2024-02-15T18:26:56.114 INFO:tasks.workunit.client.1.smithi191.stdout:9/20: write d0/f4 [371190,9961] 0 2024-02-15T18:26:56.117 INFO:tasks.workunit.client.0.smithi042.stdout:9/329: symlink d6/da/d1d/d2d/d40/l76 0 2024-02-15T18:26:56.117 INFO:tasks.workunit.client.0.smithi042.stdout:9/330: fsync d6/da/d2f/d55/f47 0 2024-02-15T18:26:56.117 INFO:tasks.workunit.client.0.smithi042.stdout:9/331: chown d6/da/f10 32 1 2024-02-15T18:26:56.118 INFO:tasks.workunit.client.1.smithi191.stdout:4/15: dwrite d2/f3 [0,4194304] 0 2024-02-15T18:26:56.118 INFO:tasks.workunit.client.1.smithi191.stdout:8/171: dwrite d3/d2c/d1a/f2d [4194304,4194304] 0 2024-02-15T18:26:56.118 INFO:tasks.workunit.client.1.smithi191.stdout:8/172: fdatasync d3/d2c/d1a/d23/d2a/f35 0 2024-02-15T18:26:56.118 INFO:tasks.workunit.client.0.smithi042.stdout:5/357: mknod d0/d38/d53/d68/d6a/c8a 0 2024-02-15T18:26:56.119 INFO:tasks.workunit.client.1.smithi191.stdout:2/23: creat f7 x:0 0 0 2024-02-15T18:26:56.119 INFO:tasks.workunit.client.1.smithi191.stdout:2/24: dread - f7 zero size 2024-02-15T18:26:56.119 INFO:tasks.workunit.client.1.smithi191.stdout:7/37: mkdir d0/d1/d3/dc 0 2024-02-15T18:26:56.121 INFO:tasks.workunit.client.0.smithi042.stdout:9/332: creat d6/da/d2f/d55/d73/f77 x:0 0 0 2024-02-15T18:26:56.122 INFO:tasks.workunit.client.0.smithi042.stdout:9/333: write f3 [2517840,43006] 0 2024-02-15T18:26:56.122 INFO:tasks.workunit.client.0.smithi042.stdout:9/334: fdatasync d6/da/d2f/f3d 0 2024-02-15T18:26:56.123 INFO:tasks.workunit.client.1.smithi191.stdout:6/41: creat d8/d9/fd x:0 0 0 2024-02-15T18:26:56.124 INFO:tasks.workunit.client.0.smithi042.stdout:9/335: write d6/da/d2f/d55/f47 [3250499,89521] 0 2024-02-15T18:26:56.126 INFO:tasks.workunit.client.1.smithi191.stdout:1/30: dwrite d0/f5 [0,4194304] 0 2024-02-15T18:26:56.130 INFO:tasks.workunit.client.0.smithi042.stdout:5/358: dread d0/d12/f76 [0,4194304] 0 2024-02-15T18:26:56.130 INFO:tasks.workunit.client.0.smithi042.stdout:5/359: write d0/d38/d53/f57 [238986,70746] 0 2024-02-15T18:26:56.130 INFO:tasks.workunit.client.0.smithi042.stdout:5/360: fdatasync d0/db/f35 0 2024-02-15T18:26:56.134 INFO:tasks.workunit.client.0.smithi042.stdout:9/336: rename d6/da/d2f/d55/f44 to d6/da/d1d/d2d/d40/f78 0 2024-02-15T18:26:56.134 INFO:tasks.workunit.client.0.smithi042.stdout:9/337: write d6/f8 [4676567,72143] 0 2024-02-15T18:26:56.138 INFO:tasks.workunit.client.0.smithi042.stdout:9/338: link d6/da/d1d/d2d/c6b d6/da/d2f/c79 0 2024-02-15T18:26:56.138 INFO:tasks.workunit.client.0.smithi042.stdout:9/339: read d6/da/d2f/d55/f47 [2009123,98144] 0 2024-02-15T18:26:56.140 INFO:tasks.workunit.client.1.smithi191.stdout:2/25: link f1 f8 0 2024-02-15T18:26:56.140 INFO:tasks.workunit.client.0.smithi042.stdout:9/340: mknod d6/dd/d9/d59/d5c/c7a 0 2024-02-15T18:26:56.141 INFO:tasks.workunit.client.1.smithi191.stdout:6/42: creat d8/d9/fe x:0 0 0 2024-02-15T18:26:56.143 INFO:tasks.workunit.client.0.smithi042.stdout:5/361: rmdir d0/d22/d5b 0 2024-02-15T18:26:56.144 INFO:tasks.workunit.client.1.smithi191.stdout:8/173: dread d3/f39 [0,4194304] 0 2024-02-15T18:26:56.144 INFO:tasks.workunit.client.1.smithi191.stdout:8/174: readlink d3/l8 0 2024-02-15T18:26:56.144 INFO:tasks.workunit.client.1.smithi191.stdout:2/26: dread f1 [0,4194304] 0 2024-02-15T18:26:56.144 INFO:tasks.workunit.client.1.smithi191.stdout:9/21: dread d0/f1 [0,4194304] 0 2024-02-15T18:26:56.145 INFO:tasks.workunit.client.1.smithi191.stdout:2/27: write f1 [4152373,50047] 0 2024-02-15T18:26:56.146 INFO:tasks.workunit.client.0.smithi042.stdout:9/341: mkdir d6/da/d7b 0 2024-02-15T18:26:56.146 INFO:tasks.workunit.client.0.smithi042.stdout:9/342: write d6/dd/d9/f14 [5692632,43842] 0 2024-02-15T18:26:56.147 INFO:tasks.workunit.client.1.smithi191.stdout:1/31: mkdir d0/db/d10 0 2024-02-15T18:26:56.147 INFO:tasks.workunit.client.1.smithi191.stdout:1/32: chown d0/db 5079 1 2024-02-15T18:26:56.148 INFO:tasks.workunit.client.0.smithi042.stdout:9/343: dread d6/da/d1d/f58 [0,4194304] 0 2024-02-15T18:26:56.148 INFO:tasks.workunit.client.1.smithi191.stdout:4/16: mkdir d2/d5 0 2024-02-15T18:26:56.150 INFO:tasks.workunit.client.0.smithi042.stdout:9/344: rmdir d6/da/d1d/d2d/d40/d48 39 2024-02-15T18:26:56.150 INFO:tasks.workunit.client.0.smithi042.stdout:9/345: stat d6/da/d2f/d55/f6d 0 2024-02-15T18:26:56.150 INFO:tasks.workunit.client.1.smithi191.stdout:7/38: dwrite d0/d1/d3/f4 [0,4194304] 0 2024-02-15T18:26:56.151 INFO:tasks.workunit.client.1.smithi191.stdout:7/39: write d0/f5 [1933023,85543] 0 2024-02-15T18:26:56.151 INFO:tasks.workunit.client.1.smithi191.stdout:9/22: mkdir d0/d7 0 2024-02-15T18:26:56.151 INFO:tasks.workunit.client.0.smithi042.stdout:7/379: dwrite d10/d15/d39/d6c/f71 [0,4194304] 0 2024-02-15T18:26:56.152 INFO:tasks.workunit.client.1.smithi191.stdout:2/28: creat f9 x:0 0 0 2024-02-15T18:26:56.152 INFO:tasks.workunit.client.1.smithi191.stdout:2/29: stat f8 0 2024-02-15T18:26:56.152 INFO:tasks.workunit.client.0.smithi042.stdout:7/380: write d10/d15/f3d [1020192,119046] 0 2024-02-15T18:26:56.152 INFO:tasks.workunit.client.0.smithi042.stdout:4/328: dwrite da/f24 [0,4194304] 0 2024-02-15T18:26:56.152 INFO:tasks.workunit.client.0.smithi042.stdout:4/329: truncate da/d11/d27/f69 720071 0 2024-02-15T18:26:56.153 INFO:tasks.workunit.client.1.smithi191.stdout:4/17: write d2/f4 [2003082,74641] 0 2024-02-15T18:26:56.153 INFO:tasks.workunit.client.1.smithi191.stdout:4/18: write f1 [528834,85297] 0 2024-02-15T18:26:56.164 INFO:tasks.workunit.client.1.smithi191.stdout:7/40: rename d0/d1/c7 to d0/cd 0 2024-02-15T18:26:56.167 INFO:tasks.workunit.client.1.smithi191.stdout:6/43: getdents d8/d9 0 2024-02-15T18:26:56.167 INFO:tasks.workunit.client.1.smithi191.stdout:6/44: truncate f5 471107 0 2024-02-15T18:26:56.168 INFO:tasks.workunit.client.1.smithi191.stdout:6/45: chown d8/d9 37283176 1 2024-02-15T18:26:56.168 INFO:tasks.workunit.client.0.smithi042.stdout:9/346: dread d6/da/d1d/f24 [0,4194304] 0 2024-02-15T18:26:56.169 INFO:tasks.workunit.client.1.smithi191.stdout:9/23: mkdir d0/d8 0 2024-02-15T18:26:56.170 INFO:tasks.workunit.client.0.smithi042.stdout:7/381: mknod d10/d43/d57/c82 0 2024-02-15T18:26:56.171 INFO:tasks.workunit.client.0.smithi042.stdout:4/330: mkdir da/d2b/d6e 0 2024-02-15T18:26:56.173 INFO:tasks.workunit.client.1.smithi191.stdout:7/41: mknod d0/d1/d3/ce 0 2024-02-15T18:26:56.173 INFO:tasks.workunit.client.1.smithi191.stdout:7/42: truncate d0/f5 2514263 0 2024-02-15T18:26:56.173 INFO:tasks.workunit.client.1.smithi191.stdout:7/43: chown d0/cd 1130169 1 2024-02-15T18:26:56.173 INFO:tasks.workunit.client.1.smithi191.stdout:7/44: truncate d0/d1/d3/f4 4794404 0 2024-02-15T18:26:56.173 INFO:tasks.workunit.client.1.smithi191.stdout:7/45: write d0/f5 [2775378,69050] 0 2024-02-15T18:26:56.173 INFO:tasks.workunit.client.1.smithi191.stdout:7/46: write d0/f5 [2384171,98539] 0 2024-02-15T18:26:56.175 INFO:tasks.workunit.client.1.smithi191.stdout:6/46: unlink d8/d9/fa 0 2024-02-15T18:26:56.176 INFO:tasks.workunit.client.1.smithi191.stdout:4/19: symlink d2/l6 0 2024-02-15T18:26:56.178 INFO:tasks.workunit.client.1.smithi191.stdout:7/47: mkdir d0/d1/df 0 2024-02-15T18:26:56.178 INFO:tasks.workunit.client.1.smithi191.stdout:7/48: readlink - no filename 2024-02-15T18:26:56.179 INFO:tasks.workunit.client.1.smithi191.stdout:4/20: mkdir d2/d5/d7 0 2024-02-15T18:26:56.179 INFO:tasks.workunit.client.1.smithi191.stdout:4/21: rename d2/d5 to d2/d5/d7/d8 22 2024-02-15T18:26:56.180 INFO:tasks.workunit.client.1.smithi191.stdout:6/47: mknod d8/d9/cf 0 2024-02-15T18:26:56.181 INFO:tasks.workunit.client.1.smithi191.stdout:8/175: dwrite d3/f18 [0,4194304] 0 2024-02-15T18:26:56.181 INFO:tasks.workunit.client.1.smithi191.stdout:8/176: chown d3/cb 320340708 1 2024-02-15T18:26:56.181 INFO:tasks.workunit.client.1.smithi191.stdout:7/49: write d0/d1/d3/f4 [1911911,79578] 0 2024-02-15T18:26:56.183 INFO:tasks.workunit.client.1.smithi191.stdout:6/48: write f1 [3002069,42774] 0 2024-02-15T18:26:56.184 INFO:tasks.workunit.client.1.smithi191.stdout:1/33: write d0/f1 [1477351,80887] 0 2024-02-15T18:26:56.184 INFO:tasks.workunit.client.1.smithi191.stdout:2/30: dwrite f9 [0,4194304] 0 2024-02-15T18:26:56.184 INFO:tasks.workunit.client.1.smithi191.stdout:1/34: truncate d0/d8/fd 299423 0 2024-02-15T18:26:56.185 INFO:tasks.workunit.client.1.smithi191.stdout:4/22: symlink d2/l9 0 2024-02-15T18:26:56.185 INFO:tasks.workunit.client.1.smithi191.stdout:4/23: truncate f1 747695 0 2024-02-15T18:26:56.185 INFO:tasks.workunit.client.1.smithi191.stdout:4/24: readlink d2/l6 0 2024-02-15T18:26:56.185 INFO:tasks.workunit.client.1.smithi191.stdout:4/25: write d2/f4 [5087189,93172] 0 2024-02-15T18:26:56.188 INFO:tasks.workunit.client.0.smithi042.stdout:5/362: dwrite d0/d38/d53/f57 [0,4194304] 0 2024-02-15T18:26:56.188 INFO:tasks.workunit.client.0.smithi042.stdout:5/363: chown d0/d12/d65/d74/d52 7039002 1 2024-02-15T18:26:56.189 INFO:tasks.workunit.client.1.smithi191.stdout:7/50: write d0/d1/d3/f4 [1071789,23120] 0 2024-02-15T18:26:56.190 INFO:tasks.workunit.client.1.smithi191.stdout:8/177: getdents d3/d2c/d1a/d23/d2e 0 2024-02-15T18:26:56.190 INFO:tasks.workunit.client.1.smithi191.stdout:8/178: dread - d3/d2c/f1f zero size 2024-02-15T18:26:56.190 INFO:tasks.workunit.client.1.smithi191.stdout:8/179: chown d3 66 1 2024-02-15T18:26:56.190 INFO:tasks.workunit.client.1.smithi191.stdout:8/180: write d3/d2c/f1f [275293,74048] 0 2024-02-15T18:26:56.194 INFO:tasks.workunit.client.0.smithi042.stdout:5/364: truncate d0/d14/f77 2973094 0 2024-02-15T18:26:56.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:54.680555+0000 mgr.smithi042.kfoihb (mgr.24435) 35 : cluster [DBG] pgmap v26: 65 pgs: 65 active+clean; 380 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 275 op/s 2024-02-15T18:26:56.196 INFO:tasks.workunit.client.1.smithi191.stdout:1/35: rename d0/l2 to d0/d8/l11 0 2024-02-15T18:26:56.197 INFO:tasks.workunit.client.1.smithi191.stdout:8/181: link d3/fa d3/d2c/d1a/f3b 0 2024-02-15T18:26:56.198 INFO:tasks.workunit.client.1.smithi191.stdout:1/36: mkdir d0/d8/df/d12 0 2024-02-15T18:26:56.199 INFO:tasks.workunit.client.1.smithi191.stdout:8/182: mknod d3/d2c/d1a/d23/d2a/d34/c3c 0 2024-02-15T18:26:56.200 INFO:tasks.workunit.client.0.smithi042.stdout:5/365: dread d0/f4d [0,4194304] 0 2024-02-15T18:26:56.201 INFO:tasks.workunit.client.1.smithi191.stdout:8/183: symlink d3/d2c/d1a/d23/d2e/l3d 0 2024-02-15T18:26:56.202 INFO:tasks.workunit.client.0.smithi042.stdout:5/366: dread d0/db/f35 [0,4194304] 0 2024-02-15T18:26:56.203 INFO:tasks.workunit.client.0.smithi042.stdout:9/347: dwrite d6/da/d1d/f24 [0,4194304] 0 2024-02-15T18:26:56.203 INFO:tasks.workunit.client.0.smithi042.stdout:7/382: dwrite d10/d15/d39/d6c/f6f [0,4194304] 0 2024-02-15T18:26:56.204 INFO:tasks.workunit.client.0.smithi042.stdout:9/348: write d6/f17 [4053696,73270] 0 2024-02-15T18:26:56.204 INFO:tasks.workunit.client.1.smithi191.stdout:8/184: mkdir d3/d2c/d1a/d23/d3e 0 2024-02-15T18:26:56.207 INFO:tasks.workunit.client.1.smithi191.stdout:9/24: dwrite d0/f4 [0,4194304] 0 2024-02-15T18:26:56.207 INFO:tasks.workunit.client.0.smithi042.stdout:5/367: rename d0/l2 to d0/d12/d65/l8b 0 2024-02-15T18:26:56.207 INFO:tasks.workunit.client.0.smithi042.stdout:9/349: write d6/dd/f19 [978948,129819] 0 2024-02-15T18:26:56.207 INFO:tasks.workunit.client.0.smithi042.stdout:5/368: chown d0/d12/d2a 7963 1 2024-02-15T18:26:56.210 INFO:tasks.workunit.client.1.smithi191.stdout:7/51: rename d0/cd to d0/c10 0 2024-02-15T18:26:56.213 INFO:tasks.workunit.client.1.smithi191.stdout:2/31: dread f8 [0,4194304] 0 2024-02-15T18:26:56.214 INFO:tasks.workunit.client.1.smithi191.stdout:2/32: write f1 [3211199,69479] 0 2024-02-15T18:26:56.215 INFO:tasks.workunit.client.1.smithi191.stdout:2/33: readlink l5 0 2024-02-15T18:26:56.215 INFO:tasks.workunit.client.1.smithi191.stdout:2/34: chown f7 806 1 2024-02-15T18:26:56.215 INFO:tasks.workunit.client.1.smithi191.stdout:2/35: write f7 [575425,77696] 0 2024-02-15T18:26:56.215 INFO:tasks.workunit.client.1.smithi191.stdout:1/37: dwrite d0/f5 [4194304,4194304] 0 2024-02-15T18:26:56.219 INFO:tasks.workunit.client.1.smithi191.stdout:6/49: truncate f1 863134 0 2024-02-15T18:26:56.220 INFO:tasks.workunit.client.0.smithi042.stdout:1/348: sync 2024-02-15T18:26:56.220 INFO:tasks.workunit.client.0.smithi042.stdout:1/349: dread - de/d54/f5e zero size 2024-02-15T18:26:56.220 INFO:tasks.workunit.client.0.smithi042.stdout:1/350: chown de/l1e 318 1 2024-02-15T18:26:56.227 INFO:tasks.workunit.client.1.smithi191.stdout:9/25: creat d0/d7/f9 x:0 0 0 2024-02-15T18:26:56.227 INFO:tasks.workunit.client.1.smithi191.stdout:9/26: dread - d0/d7/f9 zero size 2024-02-15T18:26:56.228 INFO:tasks.workunit.client.0.smithi042.stdout:5/369: mknod d0/d38/d53/d68/d6a/d42/d7d/c8c 0 2024-02-15T18:26:56.229 INFO:tasks.workunit.client.1.smithi191.stdout:7/52: mkdir d0/d1/d3/d11 0 2024-02-15T18:26:56.229 INFO:tasks.workunit.client.1.smithi191.stdout:7/53: write d0/d1/f9 [960298,53910] 0 2024-02-15T18:26:56.229 INFO:tasks.workunit.client.1.smithi191.stdout:7/54: readlink - no filename 2024-02-15T18:26:56.231 INFO:tasks.workunit.client.1.smithi191.stdout:2/36: getdents . 0 2024-02-15T18:26:56.232 INFO:tasks.workunit.client.0.smithi042.stdout:5/370: truncate d0/f23 2652478 0 2024-02-15T18:26:56.232 INFO:tasks.workunit.client.1.smithi191.stdout:4/26: dwrite d2/f4 [4194304,4194304] 0 2024-02-15T18:26:56.232 INFO:tasks.workunit.client.1.smithi191.stdout:4/27: rename d2 to d2/d5/da 22 2024-02-15T18:26:56.232 INFO:tasks.workunit.client.1.smithi191.stdout:4/28: chown f1 27720 1 2024-02-15T18:26:56.233 INFO:tasks.workunit.client.1.smithi191.stdout:1/38: mknod d0/db/d10/c13 0 2024-02-15T18:26:56.234 INFO:tasks.workunit.client.0.smithi042.stdout:5/371: mkdir d0/d39/d8d 0 2024-02-15T18:26:56.234 INFO:tasks.workunit.client.1.smithi191.stdout:1/39: read d0/f4 [319681,53395] 0 2024-02-15T18:26:56.234 INFO:tasks.workunit.client.1.smithi191.stdout:1/40: write d0/f9 [271551,44803] 0 2024-02-15T18:26:56.235 INFO:tasks.workunit.client.1.smithi191.stdout:2/37: rename l0 to la 0 2024-02-15T18:26:56.236 INFO:tasks.workunit.client.0.smithi042.stdout:5/372: mknod d0/c8e 0 2024-02-15T18:26:56.237 INFO:tasks.workunit.client.1.smithi191.stdout:4/29: creat d2/d5/fb x:0 0 0 2024-02-15T18:26:56.238 INFO:tasks.workunit.client.1.smithi191.stdout:1/41: unlink d0/d8/l11 0 2024-02-15T18:26:56.238 INFO:tasks.workunit.client.1.smithi191.stdout:1/42: stat d0/c7 0 2024-02-15T18:26:56.238 INFO:tasks.workunit.client.1.smithi191.stdout:1/43: write d0/f4 [1430412,71229] 0 2024-02-15T18:26:56.238 INFO:tasks.workunit.client.1.smithi191.stdout:2/38: write f8 [5170847,293] 0 2024-02-15T18:26:56.240 INFO:tasks.workunit.client.1.smithi191.stdout:4/30: mkdir d2/d5/d7/dc 0 2024-02-15T18:26:56.240 INFO:tasks.workunit.client.1.smithi191.stdout:4/31: write d2/d5/fb [558389,4311] 0 2024-02-15T18:26:56.242 INFO:tasks.workunit.client.1.smithi191.stdout:2/39: creat fb x:0 0 0 2024-02-15T18:26:56.243 INFO:tasks.workunit.client.1.smithi191.stdout:4/32: creat d2/d5/d7/fd x:0 0 0 2024-02-15T18:26:56.244 INFO:tasks.workunit.client.0.smithi042.stdout:1/351: dwrite de/d15/d2c/d51/f55 [0,4194304] 0 2024-02-15T18:26:56.245 INFO:tasks.workunit.client.1.smithi191.stdout:6/50: write f1 [1799674,82222] 0 2024-02-15T18:26:56.246 INFO:tasks.workunit.client.1.smithi191.stdout:8/185: dwrite d3/d2c/d1a/f20 [4194304,4194304] 0 2024-02-15T18:26:56.246 INFO:tasks.workunit.client.0.smithi042.stdout:7/383: dwrite d10/d43/f45 [0,4194304] 0 2024-02-15T18:26:56.250 INFO:tasks.workunit.client.0.smithi042.stdout:9/350: truncate d6/dd/f19 2948366 0 2024-02-15T18:26:56.254 INFO:tasks.workunit.client.0.smithi042.stdout:1/352: mkdir de/d15/d67/d66/d7c 0 2024-02-15T18:26:56.257 INFO:tasks.workunit.client.1.smithi191.stdout:6/51: read f1 [658207,8438] 0 2024-02-15T18:26:56.259 INFO:tasks.workunit.client.1.smithi191.stdout:2/40: dread f7 [0,4194304] 0 2024-02-15T18:26:56.259 INFO:tasks.workunit.client.1.smithi191.stdout:7/55: dread d0/d1/d3/f4 [0,4194304] 0 2024-02-15T18:26:56.260 INFO:tasks.workunit.client.1.smithi191.stdout:7/56: write d0/f5 [1958970,64058] 0 2024-02-15T18:26:56.262 INFO:tasks.workunit.client.0.smithi042.stdout:1/353: rename de/d15/l37 to de/d15/d1b/d44/l7d 0 2024-02-15T18:26:56.263 INFO:tasks.workunit.client.1.smithi191.stdout:1/44: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:26:56.267 INFO:tasks.workunit.client.0.smithi042.stdout:7/384: dwrite d10/f7e [0,4194304] 0 2024-02-15T18:26:56.269 INFO:tasks.workunit.client.0.smithi042.stdout:1/354: mkdir de/d15/d67/d74/d7e 0 2024-02-15T18:26:56.269 INFO:tasks.workunit.client.0.smithi042.stdout:1/355: stat de/d15/f43 0 2024-02-15T18:26:56.275 INFO:tasks.workunit.client.0.smithi042.stdout:7/385: dread d10/d43/d57/f5e [0,4194304] 0 2024-02-15T18:26:56.276 INFO:tasks.workunit.client.0.smithi042.stdout:9/351: dwrite f2 [0,4194304] 0 2024-02-15T18:26:56.277 INFO:tasks.workunit.client.0.smithi042.stdout:9/352: fdatasync d6/dd/d9/f14 0 2024-02-15T18:26:56.277 INFO:tasks.workunit.client.0.smithi042.stdout:9/353: stat d6/da/d1d/d2d/d40/l76 0 2024-02-15T18:26:56.277 INFO:tasks.workunit.client.0.smithi042.stdout:7/386: dread d10/d12/d23/f55 [0,4194304] 0 2024-02-15T18:26:56.277 INFO:tasks.workunit.client.1.smithi191.stdout:9/27: dwrite d0/f4 [4194304,4194304] 0 2024-02-15T18:26:56.277 INFO:tasks.workunit.client.1.smithi191.stdout:9/28: fsync d0/f1 0 2024-02-15T18:26:56.277 INFO:tasks.workunit.client.1.smithi191.stdout:9/29: fdatasync d0/f1 0 2024-02-15T18:26:56.278 INFO:tasks.workunit.client.1.smithi191.stdout:9/30: write d0/f1 [3225845,66385] 0 2024-02-15T18:26:56.278 INFO:tasks.workunit.client.1.smithi191.stdout:9/31: write d0/d7/f9 [843982,18189] 0 2024-02-15T18:26:56.279 INFO:tasks.workunit.client.0.smithi042.stdout:9/354: read d6/da/d1d/d2d/f39 [1128773,115547] 0 2024-02-15T18:26:56.280 INFO:tasks.workunit.client.0.smithi042.stdout:9/355: write f3 [362272,32283] 0 2024-02-15T18:26:56.280 INFO:tasks.workunit.client.0.smithi042.stdout:1/356: mknod de/d15/d67/c7f 0 2024-02-15T18:26:56.282 INFO:tasks.workunit.client.0.smithi042.stdout:7/387: rmdir d10/d12/d65 39 2024-02-15T18:26:56.284 INFO:tasks.workunit.client.0.smithi042.stdout:9/356: mkdir d6/da/d2f/d7c 0 2024-02-15T18:26:56.286 INFO:tasks.workunit.client.0.smithi042.stdout:2/321: sync 2024-02-15T18:26:56.286 INFO:tasks.workunit.client.0.smithi042.stdout:6/408: sync 2024-02-15T18:26:56.286 INFO:tasks.workunit.client.0.smithi042.stdout:0/348: sync 2024-02-15T18:26:56.286 INFO:tasks.workunit.client.0.smithi042.stdout:7/388: dread d10/d15/f3d [0,4194304] 0 2024-02-15T18:26:56.286 INFO:tasks.workunit.client.0.smithi042.stdout:7/389: write d10/d15/d39/f5d [1077446,36000] 0 2024-02-15T18:26:56.286 INFO:tasks.workunit.client.0.smithi042.stdout:1/357: symlink de/d15/d2c/d51/l80 0 2024-02-15T18:26:56.287 INFO:tasks.workunit.client.0.smithi042.stdout:1/358: truncate de/d15/d2c/f4b 506694 0 2024-02-15T18:26:56.287 INFO:tasks.workunit.client.0.smithi042.stdout:9/357: creat d6/da/d2f/d69/f7d x:0 0 0 2024-02-15T18:26:56.290 INFO:tasks.workunit.client.0.smithi042.stdout:0/349: getdents d4/d9/d62 0 2024-02-15T18:26:56.292 INFO:tasks.workunit.client.0.smithi042.stdout:7/390: mkdir d10/d49/d6d/d78/d83 0 2024-02-15T18:26:56.292 INFO:tasks.workunit.client.0.smithi042.stdout:7/391: dread - d10/d43/d57/f79 zero size 2024-02-15T18:26:56.293 INFO:tasks.workunit.client.1.smithi191.stdout:4/33: symlink d2/le 0 2024-02-15T18:26:56.293 INFO:tasks.workunit.client.1.smithi191.stdout:4/34: stat d2/d5 0 2024-02-15T18:26:56.294 INFO:tasks.workunit.client.0.smithi042.stdout:1/359: mknod de/d15/d67/d66/d7c/c81 0 2024-02-15T18:26:56.294 INFO:tasks.workunit.client.0.smithi042.stdout:1/360: chown de/d15/d1b/c61 14924328 1 2024-02-15T18:26:56.295 INFO:tasks.workunit.client.0.smithi042.stdout:9/358: getdents d6/da/d1d/d2d/d40/d48 0 2024-02-15T18:26:56.296 INFO:tasks.workunit.client.1.smithi191.stdout:6/52: dread d8/d9/fc [0,4194304] 0 2024-02-15T18:26:56.296 INFO:tasks.workunit.client.1.smithi191.stdout:6/53: fdatasync f0 0 2024-02-15T18:26:56.297 INFO:tasks.workunit.client.0.smithi042.stdout:9/359: dread d6/dd/d9/f14 [4194304,4194304] 0 2024-02-15T18:26:56.300 INFO:tasks.workunit.client.0.smithi042.stdout:6/409: mkdir d9/d8d 0 2024-02-15T18:26:56.300 INFO:tasks.workunit.client.0.smithi042.stdout:6/410: fdatasync d9/d11/d19/d35/f80 0 2024-02-15T18:26:56.300 INFO:tasks.workunit.client.0.smithi042.stdout:2/322: dwrite d0/d8/d10/d31/d3a/f50 [0,4194304] 0 2024-02-15T18:26:56.300 INFO:tasks.workunit.client.0.smithi042.stdout:2/323: truncate d0/d8/f5f 989656 0 2024-02-15T18:26:56.300 INFO:tasks.workunit.client.0.smithi042.stdout:2/324: getdents d0/d8/d20/d3f 0 2024-02-15T18:26:56.301 INFO:tasks.workunit.client.0.smithi042.stdout:0/350: mknod d4/d9/d34/d4d/c76 0 2024-02-15T18:26:56.301 INFO:tasks.workunit.client.1.smithi191.stdout:7/57: mkdir d0/d1/d3/dc/d12 0 2024-02-15T18:26:56.301 INFO:tasks.workunit.client.1.smithi191.stdout:7/58: readlink - no filename 2024-02-15T18:26:56.302 INFO:tasks.workunit.client.0.smithi042.stdout:7/392: creat d10/d43/f84 x:0 0 0 2024-02-15T18:26:56.303 INFO:tasks.workunit.client.0.smithi042.stdout:6/411: write d9/f2d [2125119,70127] 0 2024-02-15T18:26:56.305 INFO:tasks.workunit.client.1.smithi191.stdout:1/45: creat d0/db/f14 x:0 0 0 2024-02-15T18:26:56.305 INFO:tasks.workunit.client.1.smithi191.stdout:1/46: truncate d0/f9 977801 0 2024-02-15T18:26:56.305 INFO:tasks.workunit.client.1.smithi191.stdout:5/7: sync 2024-02-15T18:26:56.305 INFO:tasks.workunit.client.1.smithi191.stdout:0/1: sync 2024-02-15T18:26:56.305 INFO:tasks.workunit.client.1.smithi191.stdout:3/11: sync 2024-02-15T18:26:56.305 INFO:tasks.workunit.client.1.smithi191.stdout:5/8: dwrite - no filename 2024-02-15T18:26:56.306 INFO:tasks.workunit.client.0.smithi042.stdout:1/361: mknod de/d15/d67/d46/d6a/c82 0 2024-02-15T18:26:56.306 INFO:tasks.workunit.client.0.smithi042.stdout:1/362: truncate de/d15/d67/f2a 942450 0 2024-02-15T18:26:56.308 INFO:tasks.workunit.client.0.smithi042.stdout:9/360: stat d6/l34 0 2024-02-15T18:26:56.310 INFO:tasks.workunit.client.1.smithi191.stdout:9/32: rename d0/d8 to d0/d7/da 0 2024-02-15T18:26:56.313 INFO:tasks.workunit.client.1.smithi191.stdout:9/33: write d0/f4 [8384214,48314] 0 2024-02-15T18:26:56.313 INFO:tasks.workunit.client.1.smithi191.stdout:9/34: readlink d0/l6 0 2024-02-15T18:26:56.313 INFO:tasks.workunit.client.1.smithi191.stdout:9/35: readlink d0/l6 0 2024-02-15T18:26:56.314 INFO:tasks.workunit.client.1.smithi191.stdout:9/36: write d0/f4 [3715652,9009] 0 2024-02-15T18:26:56.315 INFO:tasks.workunit.client.0.smithi042.stdout:2/325: mknod d0/d8/db/de/d1a/c62 0 2024-02-15T18:26:56.315 INFO:tasks.workunit.client.0.smithi042.stdout:7/393: dwrite d10/f7b [0,4194304] 0 2024-02-15T18:26:56.316 INFO:tasks.workunit.client.0.smithi042.stdout:0/351: rename d4/d16 to d4/d38/d77 0 2024-02-15T18:26:56.316 INFO:tasks.workunit.client.0.smithi042.stdout:2/326: write d0/d8/db/de/d1a/f43 [815116,71606] 0 2024-02-15T18:26:56.316 INFO:tasks.workunit.client.0.smithi042.stdout:2/327: fdatasync d0/d8/d10/d31/d3a/d53/d3c/f60 0 2024-02-15T18:26:56.317 INFO:tasks.workunit.client.1.smithi191.stdout:6/54: symlink d8/l10 0 2024-02-15T18:26:56.318 INFO:tasks.workunit.client.1.smithi191.stdout:2/41: truncate f7 399711 0 2024-02-15T18:26:56.319 INFO:tasks.workunit.client.0.smithi042.stdout:6/412: creat d9/f8e x:0 0 0 2024-02-15T18:26:56.319 INFO:tasks.workunit.client.0.smithi042.stdout:1/363: symlink de/d15/d2c/d51/d70/l83 0 2024-02-15T18:26:56.320 INFO:tasks.workunit.client.1.smithi191.stdout:7/59: link d0/f5 d0/f13 0 2024-02-15T18:26:56.323 INFO:tasks.workunit.client.1.smithi191.stdout:3/12: creat f3 x:0 0 0 2024-02-15T18:26:56.323 INFO:tasks.workunit.client.1.smithi191.stdout:3/13: write f2 [591829,64878] 0 2024-02-15T18:26:56.324 INFO:tasks.workunit.client.1.smithi191.stdout:0/2: symlink l0 0 2024-02-15T18:26:56.325 INFO:tasks.workunit.client.1.smithi191.stdout:0/3: dread - no filename 2024-02-15T18:26:56.325 INFO:tasks.workunit.client.1.smithi191.stdout:0/4: chown l0 0 1 2024-02-15T18:26:56.325 INFO:tasks.workunit.client.0.smithi042.stdout:7/394: dread d10/d15/d39/d6c/f6f [0,4194304] 0 2024-02-15T18:26:56.326 INFO:tasks.workunit.client.0.smithi042.stdout:0/352: creat d4/d9/d15/f78 x:0 0 0 2024-02-15T18:26:56.326 INFO:tasks.workunit.client.0.smithi042.stdout:0/353: stat d4/f20 0 2024-02-15T18:26:56.326 INFO:tasks.workunit.client.0.smithi042.stdout:9/361: dwrite d6/da/d2f/d55/d73/f77 [0,4194304] 0 2024-02-15T18:26:56.326 INFO:tasks.workunit.client.1.smithi191.stdout:5/9: symlink d0/l1 0 2024-02-15T18:26:56.327 INFO:tasks.workunit.client.0.smithi042.stdout:2/328: creat d0/f63 x:0 0 0 2024-02-15T18:26:56.327 INFO:tasks.workunit.client.0.smithi042.stdout:2/329: chown d0/d8/db/f3e 21680408 1 2024-02-15T18:26:56.327 INFO:tasks.workunit.client.0.smithi042.stdout:0/354: write d4/d9/f22 [3824516,9580] 0 2024-02-15T18:26:56.328 INFO:tasks.workunit.client.0.smithi042.stdout:0/355: readlink d4/d38/d77/d2e/l52 0 2024-02-15T18:26:56.328 INFO:tasks.workunit.client.0.smithi042.stdout:6/413: mkdir d9/d11/d46/d89/d8f 0 2024-02-15T18:26:56.329 INFO:tasks.workunit.client.1.smithi191.stdout:4/35: symlink d2/lf 0 2024-02-15T18:26:56.330 INFO:tasks.workunit.client.0.smithi042.stdout:1/364: creat de/d15/d1b/d44/f84 x:0 0 0 2024-02-15T18:26:56.330 INFO:tasks.workunit.client.0.smithi042.stdout:1/365: dread - de/f48 zero size 2024-02-15T18:26:56.330 INFO:tasks.workunit.client.0.smithi042.stdout:1/366: chown de/d15/d67/d46/f50 13438825 1 2024-02-15T18:26:56.330 INFO:tasks.workunit.client.0.smithi042.stdout:6/414: write d9/d11/f2b [1066029,82014] 0 2024-02-15T18:26:56.331 INFO:tasks.workunit.client.1.smithi191.stdout:1/47: dwrite d0/db/f14 [0,4194304] 0 2024-02-15T18:26:56.332 INFO:tasks.workunit.client.0.smithi042.stdout:2/330: dread d0/d8/d10/d31/d3a/d53/f45 [0,4194304] 0 2024-02-15T18:26:56.332 INFO:tasks.workunit.client.0.smithi042.stdout:2/331: truncate d0/d8/d10/d31/d3a/d53/f58 1396579 0 2024-02-15T18:26:56.332 INFO:tasks.workunit.client.0.smithi042.stdout:2/332: chown d0/d8/d20/c47 25 1 2024-02-15T18:26:56.332 INFO:tasks.workunit.client.0.smithi042.stdout:2/333: write d0/d8/db/de/d1a/f4c [930462,99608] 0 2024-02-15T18:26:56.333 INFO:tasks.workunit.client.0.smithi042.stdout:7/395: unlink d10/d49/d6d/f7c 0 2024-02-15T18:26:56.333 INFO:tasks.workunit.client.0.smithi042.stdout:7/396: chown d10/d12/c18 1003 1 2024-02-15T18:26:56.336 INFO:tasks.workunit.client.1.smithi191.stdout:9/37: rmdir d0/d7 39 2024-02-15T18:26:56.340 INFO:tasks.workunit.client.1.smithi191.stdout:1/48: dread d0/f5 [0,4194304] 0 2024-02-15T18:26:56.341 INFO:tasks.workunit.client.1.smithi191.stdout:2/42: dread f1 [0,4194304] 0 2024-02-15T18:26:56.341 INFO:tasks.workunit.client.1.smithi191.stdout:6/55: dwrite f0 [0,4194304] 0 2024-02-15T18:26:56.342 INFO:tasks.workunit.client.1.smithi191.stdout:3/14: creat f4 x:0 0 0 2024-02-15T18:26:56.344 INFO:tasks.workunit.client.1.smithi191.stdout:1/49: dread d0/f9 [0,4194304] 0 2024-02-15T18:26:56.344 INFO:tasks.workunit.client.1.smithi191.stdout:1/50: fdatasync d0/f1 0 2024-02-15T18:26:56.346 INFO:tasks.workunit.client.1.smithi191.stdout:0/5: link l0 l1 0 2024-02-15T18:26:56.346 INFO:tasks.workunit.client.1.smithi191.stdout:0/6: write - no filename 2024-02-15T18:26:56.346 INFO:tasks.workunit.client.1.smithi191.stdout:0/7: chown l0 39 1 2024-02-15T18:26:56.346 INFO:tasks.workunit.client.1.smithi191.stdout:0/8: read - no filename 2024-02-15T18:26:56.349 INFO:tasks.workunit.client.0.smithi042.stdout:9/362: creat d6/dd/d9/d59/d5c/f7e x:0 0 0 2024-02-15T18:26:56.350 INFO:tasks.workunit.client.1.smithi191.stdout:5/10: symlink d0/l2 0 2024-02-15T18:26:56.350 INFO:tasks.workunit.client.1.smithi191.stdout:5/11: dwrite - no filename 2024-02-15T18:26:56.350 INFO:tasks.workunit.client.1.smithi191.stdout:4/36: symlink d2/d5/l10 0 2024-02-15T18:26:56.351 INFO:tasks.workunit.client.0.smithi042.stdout:6/415: dwrite d9/d11/f15 [0,4194304] 0 2024-02-15T18:26:56.351 INFO:tasks.workunit.client.0.smithi042.stdout:6/416: fsync d9/d11/d19/d35/f73 0 2024-02-15T18:26:56.352 INFO:tasks.workunit.client.0.smithi042.stdout:6/417: chown d9/d11/l2e 2629196 1 2024-02-15T18:26:56.352 INFO:tasks.workunit.client.0.smithi042.stdout:1/367: creat de/d15/d1b/d44/f85 x:0 0 0 2024-02-15T18:26:56.352 INFO:tasks.workunit.client.0.smithi042.stdout:1/368: readlink de/l6e 0 2024-02-15T18:26:56.354 INFO:tasks.workunit.client.0.smithi042.stdout:2/334: unlink d0/d8/db/c46 0 2024-02-15T18:26:56.357 INFO:tasks.workunit.client.0.smithi042.stdout:7/397: creat d10/d49/d6d/f85 x:0 0 0 2024-02-15T18:26:56.358 INFO:tasks.workunit.client.1.smithi191.stdout:9/38: symlink d0/d7/lb 0 2024-02-15T18:26:56.362 INFO:tasks.workunit.client.1.smithi191.stdout:2/43: truncate f8 4347620 0 2024-02-15T18:26:56.367 INFO:tasks.workunit.client.1.smithi191.stdout:6/56: mknod d8/c11 0 2024-02-15T18:26:56.368 INFO:tasks.workunit.client.1.smithi191.stdout:3/15: creat f5 x:0 0 0 2024-02-15T18:26:56.373 INFO:tasks.workunit.client.1.smithi191.stdout:1/51: symlink d0/d8/df/d12/l15 0 2024-02-15T18:26:56.375 INFO:tasks.workunit.client.1.smithi191.stdout:0/9: rename l1 to l2 0 2024-02-15T18:26:56.375 INFO:tasks.workunit.client.1.smithi191.stdout:0/10: dwrite - no filename 2024-02-15T18:26:56.375 INFO:tasks.workunit.client.1.smithi191.stdout:0/11: dwrite - no filename 2024-02-15T18:26:56.376 INFO:tasks.workunit.client.0.smithi042.stdout:9/363: symlink d6/da/d2f/d6c/l7f 0 2024-02-15T18:26:56.377 INFO:tasks.workunit.client.1.smithi191.stdout:5/12: unlink d0/l2 0 2024-02-15T18:26:56.377 INFO:tasks.workunit.client.1.smithi191.stdout:5/13: fdatasync - no filename 2024-02-15T18:26:56.378 INFO:tasks.workunit.client.0.smithi042.stdout:0/356: link d4/l2c d4/d38/d77/d2e/l79 0 2024-02-15T18:26:56.379 INFO:tasks.workunit.client.1.smithi191.stdout:4/37: link d2/l9 d2/d5/l11 0 2024-02-15T18:26:56.380 INFO:tasks.workunit.client.0.smithi042.stdout:6/418: rmdir d9/d11/d46/d89 39 2024-02-15T18:26:56.380 INFO:tasks.workunit.client.0.smithi042.stdout:6/419: write d9/d11/d19/d35/f80 [1024150,59833] 0 2024-02-15T18:26:56.381 INFO:tasks.workunit.client.0.smithi042.stdout:2/335: dwrite d0/d8/d10/d31/d3a/d53/d3c/f60 [0,4194304] 0 2024-02-15T18:26:56.381 INFO:tasks.workunit.client.0.smithi042.stdout:1/369: creat de/d15/d67/d63/f86 x:0 0 0 2024-02-15T18:26:56.385 INFO:tasks.workunit.client.1.smithi191.stdout:9/39: dwrite d0/f4 [4194304,4194304] 0 2024-02-15T18:26:56.385 INFO:tasks.workunit.client.0.smithi042.stdout:7/398: creat d10/d49/d6d/f86 x:0 0 0 2024-02-15T18:26:56.386 INFO:tasks.workunit.client.0.smithi042.stdout:7/399: chown d10/c32 13857516 1 2024-02-15T18:26:56.391 INFO:tasks.workunit.client.0.smithi042.stdout:3/371: sync 2024-02-15T18:26:56.391 INFO:tasks.workunit.client.0.smithi042.stdout:8/317: sync 2024-02-15T18:26:56.393 INFO:tasks.workunit.client.0.smithi042.stdout:9/364: rmdir d6/dd/d9/d59/d5c 39 2024-02-15T18:26:56.394 INFO:tasks.workunit.client.1.smithi191.stdout:6/57: truncate d8/d9/fd 750866 0 2024-02-15T18:26:56.395 INFO:tasks.workunit.client.1.smithi191.stdout:6/58: stat d8/d9 0 2024-02-15T18:26:56.396 INFO:tasks.workunit.client.0.smithi042.stdout:0/357: rename d4/d9/d5a/c75 to d4/d9/d34/d57/c7a 0 2024-02-15T18:26:56.396 INFO:tasks.workunit.client.0.smithi042.stdout:0/358: chown l3 1327006 1 2024-02-15T18:26:56.397 INFO:tasks.workunit.client.1.smithi191.stdout:3/16: mkdir d6 0 2024-02-15T18:26:56.397 INFO:tasks.workunit.client.1.smithi191.stdout:3/17: getdents d6 0 2024-02-15T18:26:56.399 INFO:tasks.workunit.client.0.smithi042.stdout:6/420: unlink d9/d11/d19/d1c/d31/d53/l3d 0 2024-02-15T18:26:56.399 INFO:tasks.workunit.client.1.smithi191.stdout:4/38: dwrite d2/d5/fb [0,4194304] 0 2024-02-15T18:26:56.399 INFO:tasks.workunit.client.1.smithi191.stdout:4/39: readlink d2/d5/l10 0 2024-02-15T18:26:56.399 INFO:tasks.workunit.client.1.smithi191.stdout:1/52: truncate d0/d8/fd 54938 0 2024-02-15T18:26:56.400 INFO:tasks.workunit.client.1.smithi191.stdout:0/12: stat l2 0 2024-02-15T18:26:56.401 INFO:tasks.workunit.client.1.smithi191.stdout:5/14: mknod d0/c3 0 2024-02-15T18:26:56.402 INFO:tasks.workunit.client.1.smithi191.stdout:2/44: mknod cc 0 2024-02-15T18:26:56.402 INFO:tasks.workunit.client.1.smithi191.stdout:2/45: dread - fb zero size 2024-02-15T18:26:56.402 INFO:tasks.workunit.client.1.smithi191.stdout:2/46: readlink la 0 2024-02-15T18:26:56.403 INFO:tasks.workunit.client.1.smithi191.stdout:6/59: readlink d8/d9/lb 0 2024-02-15T18:26:56.404 INFO:tasks.workunit.client.0.smithi042.stdout:1/370: getdents de/d15/d67/d46/d6a 0 2024-02-15T18:26:56.406 INFO:tasks.workunit.client.1.smithi191.stdout:3/18: mknod d6/c7 0 2024-02-15T18:26:56.406 INFO:tasks.workunit.client.1.smithi191.stdout:3/19: dread - f3 zero size 2024-02-15T18:26:56.406 INFO:tasks.workunit.client.1.smithi191.stdout:9/40: dwrite d0/d7/f9 [0,4194304] 0 2024-02-15T18:26:56.407 INFO:tasks.workunit.client.0.smithi042.stdout:8/318: dread dd/de/d12/d32/d4e/f28 [0,4194304] 0 2024-02-15T18:26:56.407 INFO:tasks.workunit.client.0.smithi042.stdout:8/319: chown dd/de/d22/f2d 214903936 1 2024-02-15T18:26:56.407 INFO:tasks.workunit.client.0.smithi042.stdout:8/320: dread - dd/de/d33/d3b/f59 zero size 2024-02-15T18:26:56.407 INFO:tasks.workunit.client.0.smithi042.stdout:7/400: creat d10/d49/d6d/d78/d83/f87 x:0 0 0 2024-02-15T18:26:56.410 INFO:tasks.workunit.client.0.smithi042.stdout:0/359: dread d4/d9/d15/f27 [0,4194304] 0 2024-02-15T18:26:56.410 INFO:tasks.workunit.client.0.smithi042.stdout:3/372: getdents d1/d11/d2e 0 2024-02-15T18:26:56.410 INFO:tasks.workunit.client.0.smithi042.stdout:3/373: chown d1/de/d1a/l36 34598124 1 2024-02-15T18:26:56.410 INFO:tasks.workunit.client.0.smithi042.stdout:3/374: dread - d1/d11/f65 zero size 2024-02-15T18:26:56.417 INFO:tasks.workunit.client.1.smithi191.stdout:4/40: rename d2/d5/d7/fd to d2/d5/f12 0 2024-02-15T18:26:56.417 INFO:tasks.workunit.client.1.smithi191.stdout:5/15: rename d0 to d0/d4 22 2024-02-15T18:26:56.418 INFO:tasks.workunit.client.1.smithi191.stdout:1/53: rmdir d0/db/d10 39 2024-02-15T18:26:56.418 INFO:tasks.workunit.client.1.smithi191.stdout:1/54: fsync d0/f4 0 2024-02-15T18:26:56.418 INFO:tasks.workunit.client.1.smithi191.stdout:1/55: chown d0/f1 64 1 2024-02-15T18:26:56.418 INFO:tasks.workunit.client.1.smithi191.stdout:1/56: readlink d0/db/lc 0 2024-02-15T18:26:56.418 INFO:tasks.workunit.client.1.smithi191.stdout:1/57: chown d0/f9 358 1 2024-02-15T18:26:56.420 INFO:tasks.workunit.client.0.smithi042.stdout:2/336: rename d0/d40/d5e to d0/d35/d64 0 2024-02-15T18:26:56.420 INFO:tasks.workunit.client.0.smithi042.stdout:7/401: creat d10/d12/d65/f88 x:0 0 0 2024-02-15T18:26:56.420 INFO:tasks.workunit.client.0.smithi042.stdout:7/402: dread - d10/d12/d65/f74 zero size 2024-02-15T18:26:56.420 INFO:tasks.workunit.client.0.smithi042.stdout:7/403: fsync d10/d15/d39/d6c/f71 0 2024-02-15T18:26:56.420 INFO:tasks.workunit.client.1.smithi191.stdout:0/13: mknod c3 0 2024-02-15T18:26:56.421 INFO:tasks.workunit.client.1.smithi191.stdout:0/14: write - no filename 2024-02-15T18:26:56.421 INFO:tasks.workunit.client.0.smithi042.stdout:8/321: dread dd/de/d22/f2e [0,4194304] 0 2024-02-15T18:26:56.421 INFO:tasks.workunit.client.0.smithi042.stdout:7/404: write d10/f73 [3049742,99213] 0 2024-02-15T18:26:56.422 INFO:tasks.workunit.client.0.smithi042.stdout:4/331: sync 2024-02-15T18:26:56.425 INFO:tasks.workunit.client.1.smithi191.stdout:2/47: unlink la 0 2024-02-15T18:26:56.427 INFO:tasks.workunit.client.0.smithi042.stdout:1/371: dread de/d15/d67/f45 [0,4194304] 0 2024-02-15T18:26:56.428 INFO:tasks.workunit.client.1.smithi191.stdout:3/20: creat d6/f8 x:0 0 0 2024-02-15T18:26:56.429 INFO:tasks.workunit.client.0.smithi042.stdout:3/375: rmdir d1/d11 39 2024-02-15T18:26:56.429 INFO:tasks.workunit.client.0.smithi042.stdout:9/365: getdents d6/dd/d9/d59/d5c 0 2024-02-15T18:26:56.429 INFO:tasks.workunit.client.0.smithi042.stdout:9/366: chown d6/da/d2f/f56 135181 1 2024-02-15T18:26:56.429 INFO:tasks.workunit.client.0.smithi042.stdout:7/405: write d10/f5b [2386869,92667] 0 2024-02-15T18:26:56.431 INFO:tasks.workunit.client.1.smithi191.stdout:6/60: dwrite f6 [4194304,4194304] 0 2024-02-15T18:26:56.431 INFO:tasks.workunit.client.1.smithi191.stdout:9/41: mkdir d0/d7/dc 0 2024-02-15T18:26:56.433 INFO:tasks.workunit.client.0.smithi042.stdout:8/322: mkdir dd/de/d22/d5a 0 2024-02-15T18:26:56.433 INFO:tasks.workunit.client.0.smithi042.stdout:8/323: dread - dd/de/d12/f45 zero size 2024-02-15T18:26:56.433 INFO:tasks.workunit.client.0.smithi042.stdout:8/324: readlink dd/l16 0 2024-02-15T18:26:56.433 INFO:tasks.workunit.client.1.smithi191.stdout:5/16: mknod d0/c5 0 2024-02-15T18:26:56.434 INFO:tasks.workunit.client.0.smithi042.stdout:6/421: dwrite d9/d3e/f75 [0,4194304] 0 2024-02-15T18:26:56.435 INFO:tasks.workunit.client.1.smithi191.stdout:1/58: mknod d0/c16 0 2024-02-15T18:26:56.435 INFO:tasks.workunit.client.0.smithi042.stdout:4/332: creat da/d11/d67/f6f x:0 0 0 2024-02-15T18:26:56.437 INFO:tasks.workunit.client.1.smithi191.stdout:0/15: link l0 l4 0 2024-02-15T18:26:56.437 INFO:tasks.workunit.client.1.smithi191.stdout:0/16: dread - no filename 2024-02-15T18:26:56.437 INFO:tasks.workunit.client.1.smithi191.stdout:0/17: read - no filename 2024-02-15T18:26:56.437 INFO:tasks.workunit.client.1.smithi191.stdout:0/18: dwrite - no filename 2024-02-15T18:26:56.437 INFO:tasks.workunit.client.1.smithi191.stdout:9/42: dread d0/f1 [0,4194304] 0 2024-02-15T18:26:56.438 INFO:tasks.workunit.client.1.smithi191.stdout:9/43: chown d0/c5 315653 1 2024-02-15T18:26:56.438 INFO:tasks.workunit.client.0.smithi042.stdout:5/373: sync 2024-02-15T18:26:56.439 INFO:tasks.workunit.client.0.smithi042.stdout:1/372: getdents de/d15/d2c/d51 0 2024-02-15T18:26:56.440 INFO:tasks.workunit.client.1.smithi191.stdout:3/21: link c1 d6/c9 0 2024-02-15T18:26:56.440 INFO:tasks.workunit.client.1.smithi191.stdout:3/22: dread - d6/f8 zero size 2024-02-15T18:26:56.441 INFO:tasks.workunit.client.1.smithi191.stdout:4/41: dwrite d2/f3 [0,4194304] 0 2024-02-15T18:26:56.442 INFO:tasks.workunit.client.1.smithi191.stdout:6/61: dread f1 [0,4194304] 0 2024-02-15T18:26:56.442 INFO:tasks.workunit.client.1.smithi191.stdout:6/62: chown d8/d9/fd 5 1 2024-02-15T18:26:56.442 INFO:tasks.workunit.client.0.smithi042.stdout:3/376: unlink d1/d11/d2e/f71 0 2024-02-15T18:26:56.442 INFO:tasks.workunit.client.1.smithi191.stdout:2/48: dwrite f7 [0,4194304] 0 2024-02-15T18:26:56.443 INFO:tasks.workunit.client.1.smithi191.stdout:5/17: mknod d0/c6 0 2024-02-15T18:26:56.447 INFO:tasks.workunit.client.1.smithi191.stdout:1/59: creat d0/d8/f17 x:0 0 0 2024-02-15T18:26:56.448 INFO:tasks.workunit.client.0.smithi042.stdout:7/406: dread d10/f73 [0,4194304] 0 2024-02-15T18:26:56.448 INFO:tasks.workunit.client.0.smithi042.stdout:7/407: dread - d10/d49/d6d/f86 zero size 2024-02-15T18:26:56.450 INFO:tasks.workunit.client.0.smithi042.stdout:9/367: rename d6/da/d2f/d69/l68 to d6/da/d2f/l80 0 2024-02-15T18:26:56.454 INFO:tasks.workunit.client.1.smithi191.stdout:0/19: creat f5 x:0 0 0 2024-02-15T18:26:56.454 INFO:tasks.workunit.client.1.smithi191.stdout:0/20: chown l2 15386504 1 2024-02-15T18:26:56.454 INFO:tasks.workunit.client.1.smithi191.stdout:0/21: rmdir - no directory 2024-02-15T18:26:56.454 INFO:tasks.workunit.client.1.smithi191.stdout:0/22: chown l0 694051941 1 2024-02-15T18:26:56.454 INFO:tasks.workunit.client.1.smithi191.stdout:2/49: dread f7 [0,4194304] 0 2024-02-15T18:26:56.455 INFO:tasks.workunit.client.0.smithi042.stdout:1/373: dwrite de/d15/d1b/d44/f84 [0,4194304] 0 2024-02-15T18:26:56.455 INFO:tasks.workunit.client.0.smithi042.stdout:1/374: chown de/d15/d2c/d51/d70 155 1 2024-02-15T18:26:56.458 INFO:tasks.workunit.client.0.smithi042.stdout:8/325: creat dd/de/d12/d32/f5b x:0 0 0 2024-02-15T18:26:56.458 INFO:tasks.workunit.client.0.smithi042.stdout:8/326: dread - dd/de/d22/f3a zero size 2024-02-15T18:26:56.459 INFO:tasks.workunit.client.0.smithi042.stdout:8/327: write dd/de/d33/f53 [1400906,61928] 0 2024-02-15T18:26:56.459 INFO:tasks.workunit.client.1.smithi191.stdout:9/44: creat d0/d7/fd x:0 0 0 2024-02-15T18:26:56.461 INFO:tasks.workunit.client.1.smithi191.stdout:3/23: rmdir d6 39 2024-02-15T18:26:56.461 INFO:tasks.workunit.client.1.smithi191.stdout:3/24: write f5 [812903,110887] 0 2024-02-15T18:26:56.461 INFO:tasks.workunit.client.1.smithi191.stdout:3/25: write d6/f8 [125398,2241] 0 2024-02-15T18:26:56.461 INFO:tasks.workunit.client.1.smithi191.stdout:3/26: truncate d6/f8 745241 0 2024-02-15T18:26:56.462 INFO:tasks.workunit.client.1.smithi191.stdout:4/42: creat d2/d5/f13 x:0 0 0 2024-02-15T18:26:56.463 INFO:tasks.workunit.client.1.smithi191.stdout:4/43: chown d2 7215 1 2024-02-15T18:26:56.463 INFO:tasks.workunit.client.1.smithi191.stdout:4/44: write d2/d5/f12 [878829,23320] 0 2024-02-15T18:26:56.463 INFO:tasks.workunit.client.0.smithi042.stdout:6/422: creat d9/d11/d1d/f90 x:0 0 0 2024-02-15T18:26:56.466 INFO:tasks.workunit.client.1.smithi191.stdout:6/63: creat d8/f12 x:0 0 0 2024-02-15T18:26:56.467 INFO:tasks.workunit.client.0.smithi042.stdout:4/333: creat da/d11/d29/f70 x:0 0 0 2024-02-15T18:26:56.469 INFO:tasks.workunit.client.1.smithi191.stdout:4/45: dread d2/f3 [0,4194304] 0 2024-02-15T18:26:56.469 INFO:tasks.workunit.client.0.smithi042.stdout:5/374: rename d0/f47 to d0/f8f 0 2024-02-15T18:26:56.471 INFO:tasks.workunit.client.1.smithi191.stdout:5/18: rename d0/l1 to d0/l7 0 2024-02-15T18:26:56.471 INFO:tasks.workunit.client.1.smithi191.stdout:5/19: write - no filename 2024-02-15T18:26:56.471 INFO:tasks.workunit.client.1.smithi191.stdout:5/20: read - no filename 2024-02-15T18:26:56.471 INFO:tasks.workunit.client.1.smithi191.stdout:5/21: dwrite - no filename 2024-02-15T18:26:56.474 INFO:tasks.workunit.client.0.smithi042.stdout:7/408: symlink d10/d43/d57/l89 0 2024-02-15T18:26:56.474 INFO:tasks.workunit.client.0.smithi042.stdout:3/377: rename d1/de/d1a/d74 to d1/d11/d14/d4d/d6c/d7b 0 2024-02-15T18:26:56.474 INFO:tasks.workunit.client.0.smithi042.stdout:3/378: chown d1/de/d19/f44 294 1 2024-02-15T18:26:56.474 INFO:tasks.workunit.client.0.smithi042.stdout:3/379: fdatasync d1/de/d19/f1f 0 2024-02-15T18:26:56.475 INFO:tasks.workunit.client.1.smithi191.stdout:0/23: creat f6 x:0 0 0 2024-02-15T18:26:56.477 INFO:tasks.workunit.client.1.smithi191.stdout:2/50: readlink l6 0 2024-02-15T18:26:56.477 INFO:tasks.workunit.client.0.smithi042.stdout:9/368: creat d6/da/d1d/d2d/d40/d48/f81 x:0 0 0 2024-02-15T18:26:56.477 INFO:tasks.workunit.client.0.smithi042.stdout:9/369: write d6/da/f65 [186141,108412] 0 2024-02-15T18:26:56.478 INFO:tasks.workunit.client.1.smithi191.stdout:9/45: symlink d0/d7/dc/le 0 2024-02-15T18:26:56.478 INFO:tasks.workunit.client.0.smithi042.stdout:5/375: dread d0/db/f1f [0,4194304] 0 2024-02-15T18:26:56.478 INFO:tasks.workunit.client.0.smithi042.stdout:5/376: stat d0/d38/f5a 0 2024-02-15T18:26:56.479 INFO:tasks.workunit.client.0.smithi042.stdout:5/377: fsync d0/d22/f4a 0 2024-02-15T18:26:56.479 INFO:tasks.workunit.client.1.smithi191.stdout:3/27: creat d6/fa x:0 0 0 2024-02-15T18:26:56.480 INFO:tasks.workunit.client.0.smithi042.stdout:4/334: dwrite da/d11/d29/f30 [0,4194304] 0 2024-02-15T18:26:56.480 INFO:tasks.workunit.client.0.smithi042.stdout:4/335: chown da/d2b/f37 5492467 1 2024-02-15T18:26:56.481 INFO:tasks.workunit.client.1.smithi191.stdout:2/51: dread f1 [0,4194304] 0 2024-02-15T18:26:56.481 INFO:tasks.workunit.client.1.smithi191.stdout:2/52: chown cc 5 1 2024-02-15T18:26:56.481 INFO:tasks.workunit.client.0.smithi042.stdout:1/375: unlink de/d15/d67/f45 0 2024-02-15T18:26:56.482 INFO:tasks.workunit.client.0.smithi042.stdout:1/376: read - de/d54/f5e zero size 2024-02-15T18:26:56.487 INFO:tasks.workunit.client.0.smithi042.stdout:6/423: creat d9/d11/d46/f91 x:0 0 0 2024-02-15T18:26:56.489 INFO:tasks.workunit.client.1.smithi191.stdout:4/46: mkdir d2/d5/d14 0 2024-02-15T18:26:56.489 INFO:tasks.workunit.client.0.smithi042.stdout:7/409: symlink d10/d15/d39/l8a 0 2024-02-15T18:26:56.490 INFO:tasks.workunit.client.1.smithi191.stdout:5/22: rename d0/c6 to d0/c8 0 2024-02-15T18:26:56.490 INFO:tasks.workunit.client.1.smithi191.stdout:5/23: chown d0/l7 87158411 1 2024-02-15T18:26:56.492 INFO:tasks.workunit.client.0.smithi042.stdout:3/380: symlink d1/d11/d7a/l7c 0 2024-02-15T18:26:56.493 INFO:tasks.workunit.client.1.smithi191.stdout:0/24: unlink f5 0 2024-02-15T18:26:56.493 INFO:tasks.workunit.client.1.smithi191.stdout:0/25: write f6 [810408,90910] 0 2024-02-15T18:26:56.494 INFO:tasks.workunit.client.0.smithi042.stdout:9/370: creat d6/da/d1d/d2d/d40/d5f/f82 x:0 0 0 2024-02-15T18:26:56.494 INFO:tasks.workunit.client.0.smithi042.stdout:9/371: dread - d6/dd/d9/d59/d5c/f7e zero size 2024-02-15T18:26:56.495 INFO:tasks.workunit.client.1.smithi191.stdout:9/46: creat d0/d7/ff x:0 0 0 2024-02-15T18:26:56.495 INFO:tasks.workunit.client.1.smithi191.stdout:9/47: write d0/f4 [5972711,8707] 0 2024-02-15T18:26:56.495 INFO:tasks.workunit.client.1.smithi191.stdout:9/48: chown d0/f1 1 1 2024-02-15T18:26:56.497 INFO:tasks.workunit.client.1.smithi191.stdout:3/28: rename f2 to d6/fb 0 2024-02-15T18:26:56.497 INFO:tasks.workunit.client.1.smithi191.stdout:3/29: write d6/f8 [1207742,15034] 0 2024-02-15T18:26:56.498 INFO:tasks.workunit.client.1.smithi191.stdout:2/53: dwrite f1 [0,4194304] 0 2024-02-15T18:26:56.499 INFO:tasks.workunit.client.0.smithi042.stdout:4/336: rename da/d11/d1b/d53/f60 to da/d2b/d6e/f71 0 2024-02-15T18:26:56.499 INFO:tasks.workunit.client.0.smithi042.stdout:5/378: dwrite d0/d38/d53/d68/d6a/f32 [0,4194304] 0 2024-02-15T18:26:56.499 INFO:tasks.workunit.client.0.smithi042.stdout:5/379: truncate d0/d38/f3c 4602093 0 2024-02-15T18:26:56.500 INFO:tasks.workunit.client.1.smithi191.stdout:4/47: creat d2/d5/d7/f15 x:0 0 0 2024-02-15T18:26:56.500 INFO:tasks.workunit.client.1.smithi191.stdout:4/48: chown d2/d5 80479 1 2024-02-15T18:26:56.500 INFO:tasks.workunit.client.1.smithi191.stdout:4/49: readlink d2/l6 0 2024-02-15T18:26:56.501 INFO:tasks.workunit.client.0.smithi042.stdout:5/380: read d0/db/fd [654721,94982] 0 2024-02-15T18:26:56.501 INFO:tasks.workunit.client.0.smithi042.stdout:5/381: getdents d0/d89 0 2024-02-15T18:26:56.501 INFO:tasks.workunit.client.1.smithi191.stdout:2/54: read f1 [2917131,92149] 0 2024-02-15T18:26:56.506 INFO:tasks.workunit.client.1.smithi191.stdout:6/64: truncate d8/d9/fd 736542 0 2024-02-15T18:26:56.507 INFO:tasks.workunit.client.1.smithi191.stdout:6/65: fdatasync f5 0 2024-02-15T18:26:56.508 INFO:tasks.workunit.client.0.smithi042.stdout:1/377: symlink de/d15/d1b/d44/l87 0 2024-02-15T18:26:56.508 INFO:tasks.workunit.client.0.smithi042.stdout:1/378: read de/f5b [2571526,22445] 0 2024-02-15T18:26:56.509 INFO:tasks.workunit.client.1.smithi191.stdout:5/24: unlink d0/c8 0 2024-02-15T18:26:56.509 INFO:tasks.workunit.client.1.smithi191.stdout:5/25: chown d0/c3 1914030 1 2024-02-15T18:26:56.511 INFO:tasks.workunit.client.0.smithi042.stdout:6/424: mkdir d9/d11/d19/d35/d55/d92 0 2024-02-15T18:26:56.511 INFO:tasks.workunit.client.1.smithi191.stdout:0/26: mkdir d7 0 2024-02-15T18:26:56.517 INFO:tasks.workunit.client.1.smithi191.stdout:9/49: mknod d0/d7/dc/c10 0 2024-02-15T18:26:56.518 INFO:tasks.workunit.client.1.smithi191.stdout:3/30: rmdir d6 39 2024-02-15T18:26:56.518 INFO:tasks.workunit.client.1.smithi191.stdout:3/31: dread - f4 zero size 2024-02-15T18:26:56.519 INFO:tasks.workunit.client.0.smithi042.stdout:9/372: creat d6/da/d1d/d2d/f83 x:0 0 0 2024-02-15T18:26:56.519 INFO:tasks.workunit.client.0.smithi042.stdout:3/381: getdents d1 0 2024-02-15T18:26:56.521 INFO:tasks.workunit.client.1.smithi191.stdout:4/50: symlink d2/d5/d14/l16 0 2024-02-15T18:26:56.521 INFO:tasks.workunit.client.1.smithi191.stdout:4/51: write d2/d5/d7/f15 [966830,32340] 0 2024-02-15T18:26:56.530 INFO:tasks.workunit.client.0.smithi042.stdout:6/425: dwrite d9/d11/d19/d35/f51 [0,4194304] 0 2024-02-15T18:26:56.530 INFO:tasks.workunit.client.0.smithi042.stdout:6/426: readlink d9/d11/d19/l1f 0 2024-02-15T18:26:56.530 INFO:tasks.workunit.client.0.smithi042.stdout:6/427: stat d9/d11/f20 0 2024-02-15T18:26:56.534 INFO:tasks.workunit.client.1.smithi191.stdout:4/52: dread d2/d5/fb [0,4194304] 0 2024-02-15T18:26:56.547 INFO:tasks.workunit.client.0.smithi042.stdout:5/382: rename d0/d38/d53/d68/d6a/f40 to d0/d14/d88/f90 0 2024-02-15T18:26:56.548 INFO:tasks.workunit.client.1.smithi191.stdout:2/55: dwrite f7 [0,4194304] 0 2024-02-15T18:26:56.548 INFO:tasks.workunit.client.1.smithi191.stdout:6/66: truncate d8/d9/fc 148294 0 2024-02-15T18:26:56.549 INFO:tasks.workunit.client.1.smithi191.stdout:6/67: chown d8/c11 1004860 1 2024-02-15T18:26:56.549 INFO:tasks.workunit.client.1.smithi191.stdout:6/68: chown d8/d9/fc 14247 1 2024-02-15T18:26:56.549 INFO:tasks.workunit.client.1.smithi191.stdout:6/69: write f5 [958401,83608] 0 2024-02-15T18:26:56.551 INFO:tasks.workunit.client.0.smithi042.stdout:4/337: dwrite da/d11/d1b/f1c [0,4194304] 0 2024-02-15T18:26:56.551 INFO:tasks.workunit.client.0.smithi042.stdout:4/338: readlink da/d3b/l6a 0 2024-02-15T18:26:56.551 INFO:tasks.workunit.client.0.smithi042.stdout:1/379: dwrite de/d15/d1b/f25 [0,4194304] 0 2024-02-15T18:26:56.555 INFO:tasks.workunit.client.0.smithi042.stdout:7/410: mknod d10/d12/d23/c8b 0 2024-02-15T18:26:56.556 INFO:tasks.workunit.client.1.smithi191.stdout:5/26: symlink d0/l9 0 2024-02-15T18:26:56.557 INFO:tasks.workunit.client.1.smithi191.stdout:0/27: mknod d7/c8 0 2024-02-15T18:26:56.557 INFO:tasks.workunit.client.1.smithi191.stdout:0/28: stat f6 0 2024-02-15T18:26:56.558 INFO:tasks.workunit.client.1.smithi191.stdout:9/50: creat d0/d7/f11 x:0 0 0 2024-02-15T18:26:56.558 INFO:tasks.workunit.client.1.smithi191.stdout:9/51: dread - d0/d7/fd zero size 2024-02-15T18:26:56.559 INFO:tasks.workunit.client.1.smithi191.stdout:9/52: dread - d0/d7/fd zero size 2024-02-15T18:26:56.560 INFO:tasks.workunit.client.1.smithi191.stdout:3/32: link f5 d6/fc 0 2024-02-15T18:26:56.561 INFO:tasks.workunit.client.1.smithi191.stdout:4/53: symlink d2/d5/l17 0 2024-02-15T18:26:56.562 INFO:tasks.workunit.client.0.smithi042.stdout:6/428: unlink d9/d11/d19/d35/c6a 0 2024-02-15T18:26:56.564 INFO:tasks.workunit.client.0.smithi042.stdout:3/382: rename d1/d11/d2e/d4c/c51 to d1/de/d19/c7d 0 2024-02-15T18:26:56.565 INFO:tasks.workunit.client.1.smithi191.stdout:2/56: unlink f8 0 2024-02-15T18:26:56.567 INFO:tasks.workunit.client.1.smithi191.stdout:6/70: getdents d8/d9 0 2024-02-15T18:26:56.569 INFO:tasks.workunit.client.0.smithi042.stdout:4/339: unlink da/d11/d29/d4a/f6b 0 2024-02-15T18:26:56.569 INFO:tasks.workunit.client.0.smithi042.stdout:4/340: chown da/d11/d67/d5c/f65 44858066 1 2024-02-15T18:26:56.569 INFO:tasks.workunit.client.0.smithi042.stdout:4/341: fdatasync da/d11/f6d 0 2024-02-15T18:26:56.569 INFO:tasks.workunit.client.1.smithi191.stdout:5/27: mknod d0/ca 0 2024-02-15T18:26:56.570 INFO:tasks.workunit.client.1.smithi191.stdout:5/28: write - no filename 2024-02-15T18:26:56.570 INFO:tasks.workunit.client.0.smithi042.stdout:5/383: dwrite d0/d38/f3c [0,4194304] 0 2024-02-15T18:26:56.570 INFO:tasks.workunit.client.0.smithi042.stdout:5/384: truncate d0/d14/f16 4995346 0 2024-02-15T18:26:56.572 INFO:tasks.workunit.client.1.smithi191.stdout:3/33: creat d6/fd x:0 0 0 2024-02-15T18:26:56.575 INFO:tasks.workunit.client.0.smithi042.stdout:1/380: fdatasync de/d15/d67/f34 0 2024-02-15T18:26:56.575 INFO:tasks.workunit.client.1.smithi191.stdout:9/53: fsync d0/d7/f11 0 2024-02-15T18:26:56.576 INFO:tasks.workunit.client.1.smithi191.stdout:4/54: stat d2/d5/l11 0 2024-02-15T18:26:56.577 INFO:tasks.workunit.client.0.smithi042.stdout:7/411: creat d10/d12/f8c x:0 0 0 2024-02-15T18:26:56.577 INFO:tasks.workunit.client.0.smithi042.stdout:7/412: chown d10/d49/d6d/f86 32 1 2024-02-15T18:26:56.581 INFO:tasks.workunit.client.0.smithi042.stdout:6/429: dread d9/f21 [4194304,4194304] 0 2024-02-15T18:26:56.585 INFO:tasks.workunit.client.0.smithi042.stdout:9/373: symlink d6/l84 0 2024-02-15T18:26:56.585 INFO:tasks.workunit.client.0.smithi042.stdout:9/374: chown d6/da/d2f/d7c 233249 1 2024-02-15T18:26:56.585 INFO:tasks.workunit.client.1.smithi191.stdout:6/71: mknod d8/d9/c13 0 2024-02-15T18:26:56.585 INFO:tasks.workunit.client.1.smithi191.stdout:6/72: chown d8/l10 19345 1 2024-02-15T18:26:56.585 INFO:tasks.workunit.client.1.smithi191.stdout:6/73: dread - d8/f12 zero size 2024-02-15T18:26:56.586 INFO:tasks.workunit.client.0.smithi042.stdout:3/383: creat d1/d11/d14/d4d/f7e x:0 0 0 2024-02-15T18:26:56.586 INFO:tasks.workunit.client.0.smithi042.stdout:0/360: sync 2024-02-15T18:26:56.587 INFO:tasks.workunit.client.0.smithi042.stdout:2/337: sync 2024-02-15T18:26:56.587 INFO:tasks.workunit.client.0.smithi042.stdout:8/328: sync 2024-02-15T18:26:56.587 INFO:tasks.workunit.client.1.smithi191.stdout:5/29: rename d0/l9 to d0/lb 0 2024-02-15T18:26:56.587 INFO:tasks.workunit.client.1.smithi191.stdout:5/30: readlink d0/l7 0 2024-02-15T18:26:56.587 INFO:tasks.workunit.client.1.smithi191.stdout:5/31: write - no filename 2024-02-15T18:26:56.587 INFO:tasks.workunit.client.1.smithi191.stdout:5/32: chown d0/ca 0 1 2024-02-15T18:26:56.588 INFO:tasks.workunit.client.0.smithi042.stdout:4/342: creat da/d3b/d41/f72 x:0 0 0 2024-02-15T18:26:56.588 INFO:tasks.workunit.client.0.smithi042.stdout:4/343: fsync da/f63 0 2024-02-15T18:26:56.588 INFO:tasks.workunit.client.1.smithi191.stdout:3/34: mkdir d6/de 0 2024-02-15T18:26:56.590 INFO:tasks.workunit.client.0.smithi042.stdout:1/381: mkdir de/d15/d67/d46/d6b/d88 0 2024-02-15T18:26:56.591 INFO:tasks.workunit.client.1.smithi191.stdout:9/54: creat d0/f12 x:0 0 0 2024-02-15T18:26:56.591 INFO:tasks.workunit.client.0.smithi042.stdout:7/413: creat d10/d15/d39/d6c/f8d x:0 0 0 2024-02-15T18:26:56.592 INFO:tasks.workunit.client.1.smithi191.stdout:9/55: read d0/f4 [5198335,116786] 0 2024-02-15T18:26:56.592 INFO:tasks.workunit.client.0.smithi042.stdout:7/414: dread d10/d12/d23/f27 [4194304,4194304] 0 2024-02-15T18:26:56.592 INFO:tasks.workunit.client.0.smithi042.stdout:7/415: truncate d10/d12/f5f 1358363 0 2024-02-15T18:26:56.593 INFO:tasks.workunit.client.0.smithi042.stdout:7/416: chown d10/f21 7642064 1 2024-02-15T18:26:56.595 INFO:tasks.workunit.client.1.smithi191.stdout:0/29: dwrite f6 [0,4194304] 0 2024-02-15T18:26:56.596 INFO:tasks.workunit.client.1.smithi191.stdout:4/55: rename f1 to d2/d5/d7/f18 0 2024-02-15T18:26:56.596 INFO:tasks.workunit.client.1.smithi191.stdout:2/57: link f1 fd 0 2024-02-15T18:26:56.596 INFO:tasks.workunit.client.0.smithi042.stdout:6/430: creat d9/d11/d19/f93 x:0 0 0 2024-02-15T18:26:56.596 INFO:tasks.workunit.client.0.smithi042.stdout:6/431: chown d9 324809 1 2024-02-15T18:26:56.598 INFO:tasks.workunit.client.0.smithi042.stdout:5/385: dwrite d0/d38/d53/d68/d6a/d42/f71 [0,4194304] 0 2024-02-15T18:26:56.598 INFO:tasks.workunit.client.0.smithi042.stdout:5/386: write d0/d38/d53/d64/f81 [95754,3368] 0 2024-02-15T18:26:56.599 INFO:tasks.workunit.client.1.smithi191.stdout:6/74: creat d8/d9/f14 x:0 0 0 2024-02-15T18:26:56.599 INFO:tasks.workunit.client.0.smithi042.stdout:2/338: dread d0/d8/db/de/d1a/f43 [0,4194304] 0 2024-02-15T18:26:56.600 INFO:tasks.workunit.client.0.smithi042.stdout:9/375: mknod d6/dd/d9/c85 0 2024-02-15T18:26:56.602 INFO:tasks.workunit.client.1.smithi191.stdout:5/33: creat d0/fc x:0 0 0 2024-02-15T18:26:56.602 INFO:tasks.workunit.client.1.smithi191.stdout:5/34: chown d0/fc 1 1 2024-02-15T18:26:56.603 INFO:tasks.workunit.client.0.smithi042.stdout:4/344: mknod da/d11/d29/d4a/d4f/c73 0 2024-02-15T18:26:56.603 INFO:tasks.workunit.client.0.smithi042.stdout:9/376: write d6/dd/d9/f14 [5409251,52743] 0 2024-02-15T18:26:56.603 INFO:tasks.workunit.client.0.smithi042.stdout:9/377: readlink d6/da/d1d/l30 0 2024-02-15T18:26:56.604 INFO:tasks.workunit.client.1.smithi191.stdout:0/30: dread f6 [0,4194304] 0 2024-02-15T18:26:56.604 INFO:tasks.workunit.client.1.smithi191.stdout:3/35: creat d6/de/ff x:0 0 0 2024-02-15T18:26:56.607 INFO:tasks.workunit.client.1.smithi191.stdout:2/58: creat fe x:0 0 0 2024-02-15T18:26:56.607 INFO:tasks.workunit.client.1.smithi191.stdout:2/59: readlink l6 0 2024-02-15T18:26:56.607 INFO:tasks.workunit.client.1.smithi191.stdout:2/60: chown fe 320025715 1 2024-02-15T18:26:56.609 INFO:tasks.workunit.client.0.smithi042.stdout:5/387: mkdir d0/d38/d53/d91 0 2024-02-15T18:26:56.610 INFO:tasks.workunit.client.1.smithi191.stdout:6/75: creat d8/f15 x:0 0 0 2024-02-15T18:26:56.611 INFO:tasks.workunit.client.0.smithi042.stdout:1/382: mkdir de/d15/d67/d89 0 2024-02-15T18:26:56.612 INFO:tasks.workunit.client.0.smithi042.stdout:1/383: truncate de/d15/d1b/d44/f85 119809 0 2024-02-15T18:26:56.612 INFO:tasks.workunit.client.1.smithi191.stdout:2/61: read f1 [882325,38574] 0 2024-02-15T18:26:56.613 INFO:tasks.workunit.client.1.smithi191.stdout:3/36: creat d6/f10 x:0 0 0 2024-02-15T18:26:56.613 INFO:tasks.workunit.client.1.smithi191.stdout:3/37: dread - d6/fd zero size 2024-02-15T18:26:56.613 INFO:tasks.workunit.client.1.smithi191.stdout:3/38: fdatasync f3 0 2024-02-15T18:26:56.614 INFO:tasks.workunit.client.0.smithi042.stdout:9/378: symlink d6/da/d2f/d69/l86 0 2024-02-15T18:26:56.614 INFO:tasks.workunit.client.0.smithi042.stdout:9/379: dread - d6/dd/d9/d59/f5e zero size 2024-02-15T18:26:56.614 INFO:tasks.workunit.client.0.smithi042.stdout:9/380: chown d6/dd/d9/d59/d5c 482 1 2024-02-15T18:26:56.614 INFO:tasks.workunit.client.0.smithi042.stdout:4/345: dread da/f24 [0,4194304] 0 2024-02-15T18:26:56.615 INFO:tasks.workunit.client.1.smithi191.stdout:6/76: stat d8/d9/cf 0 2024-02-15T18:26:56.615 INFO:tasks.workunit.client.0.smithi042.stdout:3/384: dread d1/f34 [0,4194304] 0 2024-02-15T18:26:56.615 INFO:tasks.workunit.client.0.smithi042.stdout:7/417: dread d10/d43/f50 [0,4194304] 0 2024-02-15T18:26:56.616 INFO:tasks.workunit.client.1.smithi191.stdout:2/62: truncate f7 1624594 0 2024-02-15T18:26:56.619 INFO:tasks.workunit.client.1.smithi191.stdout:6/77: rename l7 to d8/l16 0 2024-02-15T18:26:56.620 INFO:tasks.workunit.client.1.smithi191.stdout:4/56: dread d2/d5/d7/f15 [0,4194304] 0 2024-02-15T18:26:56.622 INFO:tasks.workunit.client.0.smithi042.stdout:0/361: dwrite d4/f20 [0,4194304] 0 2024-02-15T18:26:56.623 INFO:tasks.workunit.client.0.smithi042.stdout:9/381: dread d6/da/d2f/d55/d73/f77 [0,4194304] 0 2024-02-15T18:26:56.623 INFO:tasks.workunit.client.0.smithi042.stdout:9/382: chown d6/da/d2f/d69/f7d 8072245 1 2024-02-15T18:26:56.623 INFO:tasks.workunit.client.0.smithi042.stdout:9/383: readlink d6/dd/l26 0 2024-02-15T18:26:56.623 INFO:tasks.workunit.client.0.smithi042.stdout:9/384: stat d6/f21 0 2024-02-15T18:26:56.623 INFO:tasks.workunit.client.0.smithi042.stdout:4/346: unlink da/d11/d27/f3f 0 2024-02-15T18:26:56.624 INFO:tasks.workunit.client.1.smithi191.stdout:4/57: mknod d2/d5/d14/c19 0 2024-02-15T18:26:56.624 INFO:tasks.workunit.client.1.smithi191.stdout:4/58: chown d2 39 1 2024-02-15T18:26:56.624 INFO:tasks.workunit.client.0.smithi042.stdout:3/385: rename d1/de/d1a/f5c to d1/de/d1a/f7f 0 2024-02-15T18:26:56.624 INFO:tasks.workunit.client.0.smithi042.stdout:6/432: dwrite d9/f32 [0,4194304] 0 2024-02-15T18:26:56.627 INFO:tasks.workunit.client.0.smithi042.stdout:2/339: dwrite d0/d8/f28 [4194304,4194304] 0 2024-02-15T18:26:56.627 INFO:tasks.workunit.client.1.smithi191.stdout:3/39: dread d6/fc [0,4194304] 0 2024-02-15T18:26:56.629 INFO:tasks.workunit.client.1.smithi191.stdout:4/59: fsync d2/d5/d7/f18 0 2024-02-15T18:26:56.632 INFO:tasks.workunit.client.0.smithi042.stdout:7/418: creat d10/d49/f8e x:0 0 0 2024-02-15T18:26:56.634 INFO:tasks.workunit.client.0.smithi042.stdout:0/362: truncate d4/f7 446243 0 2024-02-15T18:26:56.634 INFO:tasks.workunit.client.0.smithi042.stdout:0/363: dread - d4/d9/d34/d4e/d6c/f71 zero size 2024-02-15T18:26:56.635 INFO:tasks.workunit.client.1.smithi191.stdout:6/78: symlink d8/d9/l17 0 2024-02-15T18:26:56.635 INFO:tasks.workunit.client.1.smithi191.stdout:6/79: truncate f0 4254841 0 2024-02-15T18:26:56.636 INFO:tasks.workunit.client.0.smithi042.stdout:9/385: mknod d6/dd/d9/c87 0 2024-02-15T18:26:56.636 INFO:tasks.workunit.client.0.smithi042.stdout:9/386: chown d6/da/d1d/d2d/c3a 424 1 2024-02-15T18:26:56.636 INFO:tasks.workunit.client.0.smithi042.stdout:1/384: dwrite de/d15/d1b/f57 [0,4194304] 0 2024-02-15T18:26:56.637 INFO:tasks.workunit.client.0.smithi042.stdout:9/387: write d6/f2e [98083,11763] 0 2024-02-15T18:26:56.638 INFO:tasks.workunit.client.0.smithi042.stdout:4/347: rename da/c2d to da/d11/d1b/d2f/d4d/c74 0 2024-02-15T18:26:56.640 INFO:tasks.workunit.client.1.smithi191.stdout:3/40: creat d6/de/f11 x:0 0 0 2024-02-15T18:26:56.640 INFO:tasks.workunit.client.0.smithi042.stdout:6/433: creat d9/d11/d46/f94 x:0 0 0 2024-02-15T18:26:56.641 INFO:tasks.workunit.client.1.smithi191.stdout:4/60: link d2/d5/f12 d2/d5/f1a 0 2024-02-15T18:26:56.642 INFO:tasks.workunit.client.0.smithi042.stdout:2/340: creat d0/d8/f65 x:0 0 0 2024-02-15T18:26:56.642 INFO:tasks.workunit.client.1.smithi191.stdout:9/56: dwrite d0/f4 [4194304,4194304] 0 2024-02-15T18:26:56.643 INFO:tasks.workunit.client.0.smithi042.stdout:7/419: mkdir d10/d15/d8f 0 2024-02-15T18:26:56.645 INFO:tasks.workunit.client.1.smithi191.stdout:6/80: mknod d8/c18 0 2024-02-15T18:26:56.645 INFO:tasks.workunit.client.1.smithi191.stdout:6/81: dread - d8/f15 zero size 2024-02-15T18:26:56.645 INFO:tasks.workunit.client.0.smithi042.stdout:9/388: dread f2 [0,4194304] 0 2024-02-15T18:26:56.646 INFO:tasks.workunit.client.0.smithi042.stdout:3/386: dwrite d1/d11/d2e/f3b [0,4194304] 0 2024-02-15T18:26:56.648 INFO:tasks.workunit.client.0.smithi042.stdout:6/434: dread d9/d11/d19/f2a [0,4194304] 0 2024-02-15T18:26:56.649 INFO:tasks.workunit.client.1.smithi191.stdout:2/63: fsync f7 0 2024-02-15T18:26:56.649 INFO:tasks.workunit.client.1.smithi191.stdout:2/64: write fb [469872,46835] 0 2024-02-15T18:26:56.650 INFO:tasks.workunit.client.0.smithi042.stdout:0/364: symlink d4/d9/d5a/l7b 0 2024-02-15T18:26:56.652 INFO:tasks.workunit.client.0.smithi042.stdout:0/365: write d4/d9/f22 [2986066,38788] 0 2024-02-15T18:26:56.652 INFO:tasks.workunit.client.1.smithi191.stdout:3/41: unlink d6/fc 0 2024-02-15T18:26:56.657 INFO:tasks.workunit.client.0.smithi042.stdout:1/385: rename de/d15/d67/d66/d7c/c81 to de/d15/d67/d46/d6b/c8a 0 2024-02-15T18:26:56.657 INFO:tasks.workunit.client.1.smithi191.stdout:4/61: symlink d2/d5/d7/l1b 0 2024-02-15T18:26:56.659 INFO:tasks.workunit.client.0.smithi042.stdout:2/341: creat d0/d8/d10/d31/d3a/d53/f66 x:0 0 0 2024-02-15T18:26:56.661 INFO:tasks.workunit.client.0.smithi042.stdout:0/366: dread d4/d9/d62/f53 [0,4194304] 0 2024-02-15T18:26:56.664 INFO:tasks.workunit.client.0.smithi042.stdout:0/367: write d4/fe [2950112,90307] 0 2024-02-15T18:26:56.664 INFO:tasks.workunit.client.0.smithi042.stdout:0/368: chown d4/l10 507 1 2024-02-15T18:26:56.667 INFO:tasks.workunit.client.1.smithi191.stdout:6/82: dread - d8/d9/fe zero size 2024-02-15T18:26:56.667 INFO:tasks.workunit.client.1.smithi191.stdout:6/83: fdatasync d8/d9/f14 0 2024-02-15T18:26:56.668 INFO:tasks.workunit.client.1.smithi191.stdout:9/57: dwrite d0/f1 [4194304,4194304] 0 2024-02-15T18:26:56.668 INFO:tasks.workunit.client.1.smithi191.stdout:9/58: chown d0/d7 120 1 2024-02-15T18:26:56.668 INFO:tasks.workunit.client.0.smithi042.stdout:9/389: symlink d6/dd/d9/d59/l88 0 2024-02-15T18:26:56.668 INFO:tasks.workunit.client.0.smithi042.stdout:9/390: chown d6/da/d2f/d6c 113 1 2024-02-15T18:26:56.671 INFO:tasks.workunit.client.1.smithi191.stdout:4/62: mknod d2/d5/c1c 0 2024-02-15T18:26:56.672 INFO:tasks.workunit.client.0.smithi042.stdout:3/387: getdents d1/d11/d7a 0 2024-02-15T18:26:56.672 INFO:tasks.workunit.client.0.smithi042.stdout:6/435: creat d9/d11/d19/d35/d55/d92/f95 x:0 0 0 2024-02-15T18:26:56.672 INFO:tasks.workunit.client.0.smithi042.stdout:7/420: dwrite d10/d43/f4f [0,4194304] 0 2024-02-15T18:26:56.672 INFO:tasks.workunit.client.1.smithi191.stdout:3/42: dwrite d6/de/f11 [0,4194304] 0 2024-02-15T18:26:56.673 INFO:tasks.workunit.client.0.smithi042.stdout:1/386: creat de/d15/d67/d63/f8b x:0 0 0 2024-02-15T18:26:56.674 INFO:tasks.workunit.client.1.smithi191.stdout:6/84: dread d8/d9/fd [0,4194304] 0 2024-02-15T18:26:56.674 INFO:tasks.workunit.client.1.smithi191.stdout:6/85: stat f6 0 2024-02-15T18:26:56.674 INFO:tasks.workunit.client.1.smithi191.stdout:6/86: chown d8/d9/cf 11688664 1 2024-02-15T18:26:56.677 INFO:tasks.workunit.client.0.smithi042.stdout:4/348: rename da/d11/d1b/f1e to da/d11/d1b/d2f/d4d/f75 0 2024-02-15T18:26:56.678 INFO:tasks.workunit.client.1.smithi191.stdout:9/59: mknod d0/d7/dc/c13 0 2024-02-15T18:26:56.679 INFO:tasks.workunit.client.1.smithi191.stdout:9/60: chown d0/f1 2748820 1 2024-02-15T18:26:56.679 INFO:tasks.workunit.client.1.smithi191.stdout:9/61: readlink d0/d7/dc/le 0 2024-02-15T18:26:56.680 INFO:tasks.workunit.client.0.smithi042.stdout:0/369: creat d4/d38/d77/f7c x:0 0 0 2024-02-15T18:26:56.681 INFO:tasks.workunit.client.0.smithi042.stdout:9/391: symlink d6/dd/d9/d59/d5c/l89 0 2024-02-15T18:26:56.683 INFO:tasks.workunit.client.0.smithi042.stdout:6/436: creat d9/d3e/f96 x:0 0 0 2024-02-15T18:26:56.685 INFO:tasks.workunit.client.1.smithi191.stdout:3/43: symlink d6/l12 0 2024-02-15T18:26:56.687 INFO:tasks.workunit.client.1.smithi191.stdout:9/62: creat d0/d7/da/f14 x:0 0 0 2024-02-15T18:26:56.687 INFO:tasks.workunit.client.1.smithi191.stdout:9/63: fdatasync d0/d7/f11 0 2024-02-15T18:26:56.688 INFO:tasks.workunit.client.0.smithi042.stdout:7/421: getdents d10/d12/d65 0 2024-02-15T18:26:56.688 INFO:tasks.workunit.client.0.smithi042.stdout:7/422: readlink d10/d49/d68/l70 0 2024-02-15T18:26:56.689 INFO:tasks.workunit.client.1.smithi191.stdout:4/63: rename d2/l6 to d2/l1d 0 2024-02-15T18:26:56.690 INFO:tasks.workunit.client.0.smithi042.stdout:9/392: write d6/dd/f4a [2126618,59861] 0 2024-02-15T18:26:56.690 INFO:tasks.workunit.client.0.smithi042.stdout:1/387: readlink l4 0 2024-02-15T18:26:56.691 INFO:tasks.workunit.client.0.smithi042.stdout:7/423: read d10/d43/f4e [4006771,50989] 0 2024-02-15T18:26:56.692 INFO:tasks.workunit.client.1.smithi191.stdout:3/44: link d6/fa d6/de/f13 0 2024-02-15T18:26:56.693 INFO:tasks.workunit.client.0.smithi042.stdout:3/388: rename d1/d11/d14/d4d/d6c/d7b/c67 to d1/d11/c80 0 2024-02-15T18:26:56.694 INFO:tasks.workunit.client.1.smithi191.stdout:9/64: creat d0/d7/dc/f15 x:0 0 0 2024-02-15T18:26:56.696 INFO:tasks.workunit.client.1.smithi191.stdout:4/64: symlink d2/d5/l1e 0 2024-02-15T18:26:56.699 INFO:tasks.workunit.client.0.smithi042.stdout:1/388: link de/d15/d1b/d44/l7d de/d15/d67/d46/d6b/d88/l8c 0 2024-02-15T18:26:56.700 INFO:tasks.workunit.client.0.smithi042.stdout:6/437: dread d9/d11/d46/f77 [0,4194304] 0 2024-02-15T18:26:56.701 INFO:tasks.workunit.client.1.smithi191.stdout:9/65: rename d0/d7/lb to d0/l16 0 2024-02-15T18:26:56.701 INFO:tasks.workunit.client.1.smithi191.stdout:9/66: stat d0/d7 0 2024-02-15T18:26:56.705 INFO:tasks.workunit.client.0.smithi042.stdout:0/370: rename d4/d9/d34/c58 to d4/d9/d62/c7d 0 2024-02-15T18:26:56.705 INFO:tasks.workunit.client.1.smithi191.stdout:9/67: mkdir d0/d7/dc/d17 0 2024-02-15T18:26:56.706 INFO:tasks.workunit.client.0.smithi042.stdout:3/389: mkdir d1/de/d19/d81 0 2024-02-15T18:26:56.706 INFO:tasks.workunit.client.0.smithi042.stdout:3/390: write d1/de/d1a/f6f [555865,84985] 0 2024-02-15T18:26:56.709 INFO:tasks.workunit.client.0.smithi042.stdout:1/389: unlink de/f21 0 2024-02-15T18:26:56.709 INFO:tasks.workunit.client.0.smithi042.stdout:1/390: getdents de/d15/d67/d74/d7e 0 2024-02-15T18:26:56.709 INFO:tasks.workunit.client.0.smithi042.stdout:6/438: mkdir d9/d11/d97 0 2024-02-15T18:26:56.710 INFO:tasks.workunit.client.0.smithi042.stdout:0/371: mknod d4/d9/d34/d4e/d6c/d70/c7e 0 2024-02-15T18:26:56.711 INFO:tasks.workunit.client.0.smithi042.stdout:3/391: creat d1/d11/d14/d4d/d73/f82 x:0 0 0 2024-02-15T18:26:56.711 INFO:tasks.workunit.client.0.smithi042.stdout:0/372: write d4/d9/d62/f53 [3112696,55439] 0 2024-02-15T18:26:56.711 INFO:tasks.workunit.client.0.smithi042.stdout:0/373: dread - d4/d9/d34/d4e/d6c/f71 zero size 2024-02-15T18:26:56.711 INFO:tasks.workunit.client.0.smithi042.stdout:1/391: creat de/d15/d67/d89/f8d x:0 0 0 2024-02-15T18:26:56.712 INFO:tasks.workunit.client.0.smithi042.stdout:6/439: creat d9/d11/d19/d1c/d31/d4f/f98 x:0 0 0 2024-02-15T18:26:56.713 INFO:tasks.workunit.client.0.smithi042.stdout:3/392: creat d1/d11/d14/d4d/d6c/f83 x:0 0 0 2024-02-15T18:26:56.713 INFO:tasks.workunit.client.0.smithi042.stdout:3/393: read d1/de/d1a/f6f [228913,110860] 0 2024-02-15T18:26:56.713 INFO:tasks.workunit.client.0.smithi042.stdout:0/374: rmdir d4/d9/d34/d57 39 2024-02-15T18:26:56.713 INFO:tasks.workunit.client.0.smithi042.stdout:0/375: truncate d4/d9/d34/d57/f6f 759197 0 2024-02-15T18:26:56.714 INFO:tasks.workunit.client.0.smithi042.stdout:2/342: dwrite d0/d8/db/fd [0,4194304] 0 2024-02-15T18:26:56.715 INFO:tasks.workunit.client.1.smithi191.stdout:2/65: dwrite f9 [0,4194304] 0 2024-02-15T18:26:56.716 INFO:tasks.workunit.client.0.smithi042.stdout:2/343: dread d0/d8/d10/d31/d3a/d53/f45 [0,4194304] 0 2024-02-15T18:26:56.716 INFO:tasks.workunit.client.0.smithi042.stdout:4/349: dwrite f9 [0,4194304] 0 2024-02-15T18:26:56.716 INFO:tasks.workunit.client.0.smithi042.stdout:2/344: chown d0/d8/f39 836 1 2024-02-15T18:26:56.716 INFO:tasks.workunit.client.0.smithi042.stdout:4/350: truncate da/d3b/d41/f72 215529 0 2024-02-15T18:26:56.716 INFO:tasks.workunit.client.0.smithi042.stdout:6/440: getdents d9 0 2024-02-15T18:26:56.716 INFO:tasks.workunit.client.0.smithi042.stdout:6/441: fdatasync d9/d11/f20 0 2024-02-15T18:26:56.721 INFO:tasks.workunit.client.1.smithi191.stdout:6/87: dwrite f6 [0,4194304] 0 2024-02-15T18:26:56.721 INFO:tasks.workunit.client.1.smithi191.stdout:6/88: truncate f6 8866927 0 2024-02-15T18:26:56.721 INFO:tasks.workunit.client.1.smithi191.stdout:4/65: dwrite d2/d5/d7/f15 [0,4194304] 0 2024-02-15T18:26:56.722 INFO:tasks.workunit.client.0.smithi042.stdout:9/393: dwrite d6/da/d2f/d69/f70 [0,4194304] 0 2024-02-15T18:26:56.722 INFO:tasks.workunit.client.0.smithi042.stdout:9/394: fsync d6/f17 0 2024-02-15T18:26:56.723 INFO:tasks.workunit.client.0.smithi042.stdout:0/376: creat d4/d9/d34/d57/f7f x:0 0 0 2024-02-15T18:26:56.729 INFO:tasks.workunit.client.0.smithi042.stdout:0/377: dread d4/d9/d15/f21 [4194304,4194304] 0 2024-02-15T18:26:56.729 INFO:tasks.workunit.client.0.smithi042.stdout:0/378: fsync d4/d9/d5a/f6b 0 2024-02-15T18:26:56.736 INFO:tasks.workunit.client.1.smithi191.stdout:9/68: rmdir d0/d7 39 2024-02-15T18:26:56.736 INFO:tasks.workunit.client.1.smithi191.stdout:9/69: readlink d0/d7/dc/le 0 2024-02-15T18:26:56.736 INFO:tasks.workunit.client.1.smithi191.stdout:9/70: truncate d0/f12 342847 0 2024-02-15T18:26:56.736 INFO:tasks.workunit.client.0.smithi042.stdout:3/394: getdents d1/d11/d14/d4d/d73 0 2024-02-15T18:26:56.736 INFO:tasks.workunit.client.0.smithi042.stdout:3/395: write d1/f15 [245705,120304] 0 2024-02-15T18:26:56.737 INFO:tasks.workunit.client.0.smithi042.stdout:3/396: write d1/d11/d14/d4d/f4f [839451,77122] 0 2024-02-15T18:26:56.737 INFO:tasks.workunit.client.0.smithi042.stdout:1/392: mkdir de/d15/d67/d8e 0 2024-02-15T18:26:56.737 INFO:tasks.workunit.client.0.smithi042.stdout:1/393: dread - de/d15/d2c/d51/f6f zero size 2024-02-15T18:26:56.737 INFO:tasks.workunit.client.0.smithi042.stdout:1/394: write de/d15/d67/d46/f50 [74684,33122] 0 2024-02-15T18:26:56.738 INFO:tasks.workunit.client.1.smithi191.stdout:6/89: getdents d8 0 2024-02-15T18:26:56.740 INFO:tasks.workunit.client.1.smithi191.stdout:2/66: mkdir df 0 2024-02-15T18:26:56.746 INFO:tasks.workunit.client.1.smithi191.stdout:6/90: dread f6 [0,4194304] 0 2024-02-15T18:26:56.749 INFO:tasks.workunit.client.1.smithi191.stdout:9/71: creat d0/f18 x:0 0 0 2024-02-15T18:26:56.753 INFO:tasks.workunit.client.0.smithi042.stdout:2/345: symlink d0/d8/d10/d31/d3a/l67 0 2024-02-15T18:26:56.753 INFO:tasks.workunit.client.1.smithi191.stdout:6/91: mkdir d8/d19 0 2024-02-15T18:26:56.754 INFO:tasks.workunit.client.0.smithi042.stdout:4/351: rename c3 to da/d11/d67/c76 0 2024-02-15T18:26:56.756 INFO:tasks.workunit.client.1.smithi191.stdout:4/66: mknod d2/c1f 0 2024-02-15T18:26:56.756 INFO:tasks.workunit.client.1.smithi191.stdout:4/67: fsync d2/d5/d7/f15 0 2024-02-15T18:26:56.757 INFO:tasks.workunit.client.0.smithi042.stdout:6/442: unlink d9/d11/d19/d1c/l59 0 2024-02-15T18:26:56.758 INFO:tasks.workunit.client.1.smithi191.stdout:9/72: creat d0/d7/f19 x:0 0 0 2024-02-15T18:26:56.759 INFO:tasks.workunit.client.1.smithi191.stdout:2/67: truncate fb 435804 0 2024-02-15T18:26:56.759 INFO:tasks.workunit.client.1.smithi191.stdout:2/68: write fe [677428,10566] 0 2024-02-15T18:26:56.760 INFO:tasks.workunit.client.1.smithi191.stdout:6/92: stat d8/d9/fc 0 2024-02-15T18:26:56.760 INFO:tasks.workunit.client.1.smithi191.stdout:6/93: truncate d8/d9/fe 269181 0 2024-02-15T18:26:56.760 INFO:tasks.workunit.client.1.smithi191.stdout:2/69: write f9 [4054456,119424] 0 2024-02-15T18:26:56.760 INFO:tasks.workunit.client.1.smithi191.stdout:2/70: truncate fe 1211772 0 2024-02-15T18:26:56.762 INFO:tasks.workunit.client.1.smithi191.stdout:4/68: rmdir d2/d5 39 2024-02-15T18:26:56.763 INFO:tasks.workunit.client.1.smithi191.stdout:8/186: sync 2024-02-15T18:26:56.763 INFO:tasks.workunit.client.1.smithi191.stdout:7/60: sync 2024-02-15T18:26:56.763 INFO:tasks.workunit.client.1.smithi191.stdout:4/69: write d2/d5/f12 [630249,67108] 0 2024-02-15T18:26:56.769 INFO:tasks.workunit.client.1.smithi191.stdout:7/61: write d0/d1/f9 [377353,46802] 0 2024-02-15T18:26:56.770 INFO:tasks.workunit.client.1.smithi191.stdout:9/73: mknod d0/c1a 0 2024-02-15T18:26:56.771 INFO:tasks.workunit.client.0.smithi042.stdout:9/395: creat d6/da/d2f/d69/f8a x:0 0 0 2024-02-15T18:26:56.771 INFO:tasks.workunit.client.1.smithi191.stdout:6/94: symlink d8/d19/l1a 0 2024-02-15T18:26:56.772 INFO:tasks.workunit.client.0.smithi042.stdout:9/396: write f3 [2385177,16113] 0 2024-02-15T18:26:56.772 INFO:tasks.workunit.client.0.smithi042.stdout:9/397: write d6/dd/d9/f72 [80998,90873] 0 2024-02-15T18:26:56.772 INFO:tasks.workunit.client.0.smithi042.stdout:0/379: symlink d4/d9/d34/d4e/d6c/d70/l80 0 2024-02-15T18:26:56.773 INFO:tasks.workunit.client.1.smithi191.stdout:8/187: mknod d3/d2c/d1a/d23/d2e/c3f 0 2024-02-15T18:26:56.774 INFO:tasks.workunit.client.1.smithi191.stdout:4/70: mknod d2/d5/d7/dc/c20 0 2024-02-15T18:26:56.775 INFO:tasks.workunit.client.0.smithi042.stdout:3/397: mkdir d1/d11/d84 0 2024-02-15T18:26:56.775 INFO:tasks.workunit.client.1.smithi191.stdout:4/71: write d2/d5/fb [783358,69974] 0 2024-02-15T18:26:56.775 INFO:tasks.workunit.client.0.smithi042.stdout:1/395: creat de/d15/d67/d63/f8f x:0 0 0 2024-02-15T18:26:56.776 INFO:tasks.workunit.client.0.smithi042.stdout:1/396: stat de/d15/d2c/f5f 0 2024-02-15T18:26:56.777 INFO:tasks.workunit.client.0.smithi042.stdout:2/346: rename d0/d8/db/de/d1a/c21 to d0/d8/db/de/d1a/d2e/c68 0 2024-02-15T18:26:56.777 INFO:tasks.workunit.client.0.smithi042.stdout:2/347: getdents d0/d8/d10/d5a 0 2024-02-15T18:26:56.778 INFO:tasks.workunit.client.0.smithi042.stdout:6/443: dwrite d9/d11/d19/f93 [0,4194304] 0 2024-02-15T18:26:56.778 INFO:tasks.workunit.client.0.smithi042.stdout:6/444: readlink d9/d11/d19/d1c/l4d 0 2024-02-15T18:26:56.778 INFO:tasks.workunit.client.0.smithi042.stdout:6/445: stat d9/d11/d19/d1c/f6c 0 2024-02-15T18:26:56.779 INFO:tasks.workunit.client.0.smithi042.stdout:4/352: symlink da/d11/d1b/d53/l77 0 2024-02-15T18:26:56.784 INFO:tasks.workunit.client.0.smithi042.stdout:8/329: sync 2024-02-15T18:26:56.784 INFO:tasks.workunit.client.0.smithi042.stdout:9/398: creat d6/da/d2f/d69/f8b x:0 0 0 2024-02-15T18:26:56.787 INFO:tasks.workunit.client.0.smithi042.stdout:0/380: symlink d4/d9/d34/d57/l81 0 2024-02-15T18:26:56.789 INFO:tasks.workunit.client.0.smithi042.stdout:3/398: truncate d1/f7 2805890 0 2024-02-15T18:26:56.791 INFO:tasks.workunit.client.0.smithi042.stdout:1/397: creat de/d15/d67/d74/f90 x:0 0 0 2024-02-15T18:26:56.791 INFO:tasks.workunit.client.1.smithi191.stdout:7/62: creat d0/f14 x:0 0 0 2024-02-15T18:26:56.791 INFO:tasks.workunit.client.1.smithi191.stdout:9/74: creat d0/d7/f1b x:0 0 0 2024-02-15T18:26:56.793 INFO:tasks.workunit.client.0.smithi042.stdout:2/348: unlink d0/d8/d10/d31/d3a/f50 0 2024-02-15T18:26:56.793 INFO:tasks.workunit.client.0.smithi042.stdout:2/349: fdatasync d0/d8/db/de/d1a/f4c 0 2024-02-15T18:26:56.794 INFO:tasks.workunit.client.1.smithi191.stdout:6/95: link d8/d9/l17 d8/d19/l1b 0 2024-02-15T18:26:56.798 INFO:tasks.workunit.client.1.smithi191.stdout:7/63: mkdir d0/d1/d15 0 2024-02-15T18:26:56.798 INFO:tasks.workunit.client.1.smithi191.stdout:7/64: fdatasync d0/f14 0 2024-02-15T18:26:56.800 INFO:tasks.workunit.client.1.smithi191.stdout:9/75: creat d0/d7/da/f1c x:0 0 0 2024-02-15T18:26:56.800 INFO:tasks.workunit.client.1.smithi191.stdout:9/76: dread - d0/d7/f1b zero size 2024-02-15T18:26:56.801 INFO:tasks.workunit.client.0.smithi042.stdout:4/353: dwrite da/d11/d1b/f3e [0,4194304] 0 2024-02-15T18:26:56.803 INFO:tasks.workunit.client.1.smithi191.stdout:6/96: unlink f5 0 2024-02-15T18:26:56.803 INFO:tasks.workunit.client.1.smithi191.stdout:6/97: dread - d8/f12 zero size 2024-02-15T18:26:56.803 INFO:tasks.workunit.client.1.smithi191.stdout:6/98: truncate d8/d9/f14 359578 0 2024-02-15T18:26:56.803 INFO:tasks.workunit.client.1.smithi191.stdout:6/99: dread - d8/f15 zero size 2024-02-15T18:26:56.805 INFO:tasks.workunit.client.1.smithi191.stdout:8/188: symlink d3/l40 0 2024-02-15T18:26:56.806 INFO:tasks.workunit.client.0.smithi042.stdout:6/446: creat d9/d11/d19/f99 x:0 0 0 2024-02-15T18:26:56.806 INFO:tasks.workunit.client.0.smithi042.stdout:6/447: stat d9/d11/d19/d35/d49/d79/f60 0 2024-02-15T18:26:56.806 INFO:tasks.workunit.client.0.smithi042.stdout:6/448: getdents d9/d11/d46/d89/d8f 0 2024-02-15T18:26:56.806 INFO:tasks.workunit.client.0.smithi042.stdout:6/449: write d9/d3e/f8b [946430,9116] 0 2024-02-15T18:26:56.808 INFO:tasks.workunit.client.1.smithi191.stdout:4/72: rename d2/d5/d7/dc/c20 to d2/c21 0 2024-02-15T18:26:56.810 INFO:tasks.workunit.client.0.smithi042.stdout:8/330: symlink dd/de/d12/l5c 0 2024-02-15T18:26:56.814 INFO:tasks.workunit.client.1.smithi191.stdout:9/77: mkdir d0/d1d 0 2024-02-15T18:26:56.815 INFO:tasks.workunit.client.0.smithi042.stdout:9/399: creat d6/da/d1d/d2d/d40/f8c x:0 0 0 2024-02-15T18:26:56.816 INFO:tasks.workunit.client.0.smithi042.stdout:9/400: fsync d6/dd/f2b 0 2024-02-15T18:26:56.816 INFO:tasks.workunit.client.1.smithi191.stdout:8/189: truncate d3/d2c/f1f 212592 0 2024-02-15T18:26:56.816 INFO:tasks.workunit.client.1.smithi191.stdout:8/190: write d3/f21 [5152924,57292] 0 2024-02-15T18:26:56.818 INFO:tasks.workunit.client.0.smithi042.stdout:7/424: sync 2024-02-15T18:26:56.818 INFO:tasks.workunit.client.0.smithi042.stdout:5/388: sync 2024-02-15T18:26:56.820 INFO:tasks.workunit.client.0.smithi042.stdout:3/399: symlink d1/d11/d14/d4d/d6c/d7b/l85 0 2024-02-15T18:26:56.820 INFO:tasks.workunit.client.1.smithi191.stdout:9/78: rmdir d0/d7 39 2024-02-15T18:26:56.821 INFO:tasks.workunit.client.0.smithi042.stdout:3/400: write d1/d11/d14/f53 [3655444,94628] 0 2024-02-15T18:26:56.822 INFO:tasks.workunit.client.0.smithi042.stdout:1/398: creat de/d15/d67/d46/d6b/f91 x:0 0 0 2024-02-15T18:26:56.823 INFO:tasks.workunit.client.0.smithi042.stdout:1/399: stat de/d15/d67/d46/d6b/f91 0 2024-02-15T18:26:56.823 INFO:tasks.workunit.client.1.smithi191.stdout:8/191: rmdir d3/d2c/d1a/d23/d2a/d34 39 2024-02-15T18:26:56.823 INFO:tasks.workunit.client.1.smithi191.stdout:8/192: truncate d3/d2c/f26 642330 0 2024-02-15T18:26:56.824 INFO:tasks.workunit.client.0.smithi042.stdout:2/350: creat d0/d8/db/d44/f69 x:0 0 0 2024-02-15T18:26:56.825 INFO:tasks.workunit.client.1.smithi191.stdout:7/65: dwrite d0/d1/d3/f4 [4194304,4194304] 0 2024-02-15T18:26:56.825 INFO:tasks.workunit.client.1.smithi191.stdout:9/79: unlink d0/d7/da/f14 0 2024-02-15T18:26:56.825 INFO:tasks.workunit.client.0.smithi042.stdout:1/400: dread de/f4e [0,4194304] 0 2024-02-15T18:26:56.827 INFO:tasks.workunit.client.1.smithi191.stdout:6/100: dwrite d8/f12 [0,4194304] 0 2024-02-15T18:26:56.827 INFO:tasks.workunit.client.0.smithi042.stdout:4/354: rename da/d11/d1b to da/d11/d67/d5c/d66/d78 0 2024-02-15T18:26:56.828 INFO:tasks.workunit.client.1.smithi191.stdout:6/101: write f1 [124033,79948] 0 2024-02-15T18:26:56.828 INFO:tasks.workunit.client.1.smithi191.stdout:4/73: dwrite d2/f4 [0,4194304] 0 2024-02-15T18:26:56.828 INFO:tasks.workunit.client.1.smithi191.stdout:4/74: chown d2/f3 5964436 1 2024-02-15T18:26:56.828 INFO:tasks.workunit.client.0.smithi042.stdout:4/355: dread da/f63 [0,4194304] 0 2024-02-15T18:26:56.828 INFO:tasks.workunit.client.0.smithi042.stdout:4/356: read da/d11/d29/f2c [31505,66159] 0 2024-02-15T18:26:56.829 INFO:tasks.workunit.client.1.smithi191.stdout:8/193: symlink d3/d2c/d1a/d23/l41 0 2024-02-15T18:26:56.829 INFO:tasks.workunit.client.1.smithi191.stdout:8/194: chown d3/f15 12516384 1 2024-02-15T18:26:56.829 INFO:tasks.workunit.client.1.smithi191.stdout:6/102: write f1 [1100464,104995] 0 2024-02-15T18:26:56.834 INFO:tasks.workunit.client.0.smithi042.stdout:6/450: mknod d9/c9a 0 2024-02-15T18:26:56.840 INFO:tasks.workunit.client.1.smithi191.stdout:7/66: unlink d0/c6 0 2024-02-15T18:26:56.841 INFO:tasks.workunit.client.0.smithi042.stdout:9/401: mkdir d6/da/d2f/d55/d73/d8d 0 2024-02-15T18:26:56.841 INFO:tasks.workunit.client.1.smithi191.stdout:9/80: mknod d0/d7/dc/c1e 0 2024-02-15T18:26:56.842 INFO:tasks.workunit.client.0.smithi042.stdout:7/425: truncate d10/d43/d57/f5e 4170102 0 2024-02-15T18:26:56.843 INFO:tasks.workunit.client.0.smithi042.stdout:5/389: mknod d0/d38/d53/d68/d6a/d42/d7d/c92 0 2024-02-15T18:26:56.845 INFO:tasks.workunit.client.0.smithi042.stdout:3/401: rmdir d1/de/d19/d81 0 2024-02-15T18:26:56.845 INFO:tasks.workunit.client.0.smithi042.stdout:6/451: dread d9/d11/d19/d1c/d31/f5b [0,4194304] 0 2024-02-15T18:26:56.846 INFO:tasks.workunit.client.1.smithi191.stdout:6/103: dwrite d8/f15 [0,4194304] 0 2024-02-15T18:26:56.849 INFO:tasks.workunit.client.0.smithi042.stdout:4/357: dwrite da/d11/d67/d5c/d66/d78/d2f/f34 [0,4194304] 0 2024-02-15T18:26:56.868 INFO:tasks.workunit.client.0.smithi042.stdout:9/402: dwrite d6/da/d1d/f38 [0,4194304] 0 2024-02-15T18:26:56.869 INFO:tasks.workunit.client.0.smithi042.stdout:9/403: dread - d6/da/d2f/d69/f7d zero size 2024-02-15T18:26:56.869 INFO:tasks.workunit.client.0.smithi042.stdout:2/351: symlink d0/d8/d10/d61/l6a 0 2024-02-15T18:26:56.872 INFO:tasks.workunit.client.1.smithi191.stdout:4/75: link d2/d5/d7/l1b d2/d5/d7/l22 0 2024-02-15T18:26:56.873 INFO:tasks.workunit.client.0.smithi042.stdout:1/401: symlink de/d15/l92 0 2024-02-15T18:26:56.873 INFO:tasks.workunit.client.0.smithi042.stdout:1/402: dread - de/d15/d2c/d51/f6f zero size 2024-02-15T18:26:56.874 INFO:tasks.workunit.client.1.smithi191.stdout:8/195: creat d3/d2c/d1a/d23/f42 x:0 0 0 2024-02-15T18:26:56.875 INFO:tasks.workunit.client.1.smithi191.stdout:7/67: creat d0/d1/df/f16 x:0 0 0 2024-02-15T18:26:56.878 INFO:tasks.workunit.client.1.smithi191.stdout:9/81: mkdir d0/d7/dc/d17/d1f 0 2024-02-15T18:26:56.881 INFO:tasks.workunit.client.0.smithi042.stdout:8/331: symlink dd/d11/l5d 0 2024-02-15T18:26:56.881 INFO:tasks.workunit.client.0.smithi042.stdout:8/332: write dd/de/d12/f4c [486268,128257] 0 2024-02-15T18:26:56.881 INFO:tasks.workunit.client.0.smithi042.stdout:8/333: chown dd/de/d12/f4c 589 1 2024-02-15T18:26:56.882 INFO:tasks.workunit.client.0.smithi042.stdout:7/426: mknod d10/d12/d23/d51/c90 0 2024-02-15T18:26:56.886 INFO:tasks.workunit.client.0.smithi042.stdout:5/390: getdents d0/d14 0 2024-02-15T18:26:56.889 INFO:tasks.workunit.client.0.smithi042.stdout:6/452: mkdir d9/d11/d19/d1c/d31/d9b 0 2024-02-15T18:26:56.889 INFO:tasks.workunit.client.0.smithi042.stdout:6/453: chown d9/d11/d46/d89 113908154 1 2024-02-15T18:26:56.892 INFO:tasks.workunit.client.1.smithi191.stdout:6/104: rename f0 to d8/f1c 0 2024-02-15T18:26:56.893 INFO:tasks.workunit.client.1.smithi191.stdout:6/105: read f6 [6874746,45029] 0 2024-02-15T18:26:56.893 INFO:tasks.workunit.client.1.smithi191.stdout:6/106: stat d8/d9/c13 0 2024-02-15T18:26:56.894 INFO:tasks.workunit.client.0.smithi042.stdout:4/358: link da/d11/c15 da/d2b/d6e/c79 0 2024-02-15T18:26:56.894 INFO:tasks.workunit.client.1.smithi191.stdout:4/76: unlink d2/d5/d7/l22 0 2024-02-15T18:26:56.897 INFO:tasks.workunit.client.1.smithi191.stdout:9/82: dwrite d0/f4 [0,4194304] 0 2024-02-15T18:26:56.897 INFO:tasks.workunit.client.1.smithi191.stdout:8/196: creat d3/d2c/f43 x:0 0 0 2024-02-15T18:26:56.898 INFO:tasks.workunit.client.1.smithi191.stdout:8/197: read d3/f21 [3617052,88594] 0 2024-02-15T18:26:56.898 INFO:tasks.workunit.client.1.smithi191.stdout:8/198: chown d3/l22 11784 1 2024-02-15T18:26:56.899 INFO:tasks.workunit.client.0.smithi042.stdout:9/404: mknod d6/dd/d9/d59/d5c/c8e 0 2024-02-15T18:26:56.899 INFO:tasks.workunit.client.0.smithi042.stdout:9/405: write d6/da/d2f/d6c/f71 [4586705,80092] 0 2024-02-15T18:26:56.901 INFO:tasks.workunit.client.1.smithi191.stdout:7/68: truncate d0/f13 504765 0 2024-02-15T18:26:56.902 INFO:tasks.workunit.client.0.smithi042.stdout:2/352: creat d0/d8/db/de/d1a/f6b x:0 0 0 2024-02-15T18:26:56.904 INFO:tasks.workunit.client.1.smithi191.stdout:6/107: dread d8/f12 [0,4194304] 0 2024-02-15T18:26:56.904 INFO:tasks.workunit.client.1.smithi191.stdout:4/77: unlink d2/d5/f12 0 2024-02-15T18:26:56.904 INFO:tasks.workunit.client.1.smithi191.stdout:6/108: write d8/d9/fe [211312,100569] 0 2024-02-15T18:26:56.905 INFO:tasks.workunit.client.0.smithi042.stdout:1/403: creat de/d15/d67/f93 x:0 0 0 2024-02-15T18:26:56.906 INFO:tasks.workunit.client.1.smithi191.stdout:8/199: creat d3/d2c/d1a/f44 x:0 0 0 2024-02-15T18:26:56.908 INFO:tasks.workunit.client.1.smithi191.stdout:9/83: dread d0/d7/f9 [0,4194304] 0 2024-02-15T18:26:56.908 INFO:tasks.workunit.client.1.smithi191.stdout:7/69: rename d0/d1/d3/f4 to d0/d1/df/f17 0 2024-02-15T18:26:56.908 INFO:tasks.workunit.client.1.smithi191.stdout:7/70: chown d0/d1/d3/d11 1842004 1 2024-02-15T18:26:56.911 INFO:tasks.workunit.client.0.smithi042.stdout:5/391: dwrite d0/f23 [0,4194304] 0 2024-02-15T18:26:56.911 INFO:tasks.workunit.client.0.smithi042.stdout:5/392: stat d0/d12/d65/d74/f4c 0 2024-02-15T18:26:56.912 INFO:tasks.workunit.client.1.smithi191.stdout:7/71: write d0/d1/df/f17 [1374773,34789] 0 2024-02-15T18:26:56.912 INFO:tasks.workunit.client.0.smithi042.stdout:1/404: write de/d15/d67/f3f [2981054,92800] 0 2024-02-15T18:26:56.914 INFO:tasks.workunit.client.1.smithi191.stdout:4/78: write d2/d5/f1a [1798634,81184] 0 2024-02-15T18:26:56.918 INFO:tasks.workunit.client.0.smithi042.stdout:7/427: creat d10/d49/d6d/d78/f91 x:0 0 0 2024-02-15T18:26:56.918 INFO:tasks.workunit.client.0.smithi042.stdout:7/428: dread - d10/d49/d6d/f85 zero size 2024-02-15T18:26:56.919 INFO:tasks.workunit.client.1.smithi191.stdout:6/109: creat d8/d19/f1d x:0 0 0 2024-02-15T18:26:56.920 INFO:tasks.workunit.client.1.smithi191.stdout:9/84: mknod d0/d7/da/c20 0 2024-02-15T18:26:56.921 INFO:tasks.workunit.client.1.smithi191.stdout:6/110: fsync d8/d9/fe 0 2024-02-15T18:26:56.921 INFO:tasks.workunit.client.1.smithi191.stdout:6/111: chown f6 4 1 2024-02-15T18:26:56.921 INFO:tasks.workunit.client.1.smithi191.stdout:6/112: write d8/d9/f14 [434970,57785] 0 2024-02-15T18:26:56.922 INFO:tasks.workunit.client.0.smithi042.stdout:3/402: rename d1/d11/c3c to d1/d11/d14/c86 0 2024-02-15T18:26:56.923 INFO:tasks.workunit.client.1.smithi191.stdout:8/200: rename d3/d2c/f26 to d3/d2c/d1a/f45 0 2024-02-15T18:26:56.923 INFO:tasks.workunit.client.1.smithi191.stdout:6/113: rename d8/d19 to d8/d19/d1e 22 2024-02-15T18:26:56.925 INFO:tasks.workunit.client.1.smithi191.stdout:6/114: dread d8/d9/fe [0,4194304] 0 2024-02-15T18:26:56.925 INFO:tasks.workunit.client.0.smithi042.stdout:6/454: creat d9/d11/d19/d1c/d31/d9b/f9c x:0 0 0 2024-02-15T18:26:56.925 INFO:tasks.workunit.client.0.smithi042.stdout:6/455: dread - d9/d11/d1d/f88 zero size 2024-02-15T18:26:56.929 INFO:tasks.workunit.client.1.smithi191.stdout:7/72: creat d0/d1/d3/d11/f18 x:0 0 0 2024-02-15T18:26:56.930 INFO:tasks.workunit.client.0.smithi042.stdout:4/359: truncate da/d2b/f52 1886942 0 2024-02-15T18:26:56.931 INFO:tasks.workunit.client.1.smithi191.stdout:4/79: mknod d2/d5/c23 0 2024-02-15T18:26:56.934 INFO:tasks.workunit.client.1.smithi191.stdout:9/85: rmdir d0/d7/dc/d17/d1f 0 2024-02-15T18:26:56.934 INFO:tasks.workunit.client.1.smithi191.stdout:9/86: fdatasync d0/d7/f9 0 2024-02-15T18:26:56.935 INFO:tasks.workunit.client.0.smithi042.stdout:2/353: link d0/d8/db/de/d1a/l48 d0/d8/d10/d31/d3a/l6c 0 2024-02-15T18:26:56.937 INFO:tasks.workunit.client.0.smithi042.stdout:5/393: creat d0/d38/d53/d64/f93 x:0 0 0 2024-02-15T18:26:56.937 INFO:tasks.workunit.client.0.smithi042.stdout:5/394: chown d0/d12/c69 1 1 2024-02-15T18:26:56.937 INFO:tasks.workunit.client.0.smithi042.stdout:5/395: chown d0/db/c3a 110 1 2024-02-15T18:26:56.937 INFO:tasks.workunit.client.0.smithi042.stdout:5/396: write d0/db/f1f [5246365,110757] 0 2024-02-15T18:26:56.937 INFO:tasks.workunit.client.0.smithi042.stdout:5/397: fdatasync d0/f23 0 2024-02-15T18:26:56.939 INFO:tasks.workunit.client.0.smithi042.stdout:1/405: unlink de/d15/c4d 0 2024-02-15T18:26:56.939 INFO:tasks.workunit.client.0.smithi042.stdout:1/406: fsync de/d15/d67/f93 0 2024-02-15T18:26:56.940 INFO:tasks.workunit.client.1.smithi191.stdout:6/115: truncate d8/f1c 2932755 0 2024-02-15T18:26:56.940 INFO:tasks.workunit.client.1.smithi191.stdout:8/201: dwrite d3/fc [0,4194304] 0 2024-02-15T18:26:56.941 INFO:tasks.workunit.client.1.smithi191.stdout:8/202: rename d3 to d3/d2c/d1a/d23/d2a/d34/d46 22 2024-02-15T18:26:56.941 INFO:tasks.workunit.client.0.smithi042.stdout:7/429: rmdir d10/d49/d68 39 2024-02-15T18:26:56.941 INFO:tasks.workunit.client.0.smithi042.stdout:7/430: chown d10/f63 19142 1 2024-02-15T18:26:56.941 INFO:tasks.workunit.client.0.smithi042.stdout:7/431: write d10/f7e [4287605,17389] 0 2024-02-15T18:26:56.943 INFO:tasks.workunit.client.1.smithi191.stdout:6/116: write d8/d9/fd [358458,104543] 0 2024-02-15T18:26:56.943 INFO:tasks.workunit.client.1.smithi191.stdout:4/80: unlink d2/c1f 0 2024-02-15T18:26:56.947 INFO:tasks.workunit.client.0.smithi042.stdout:3/403: creat d1/d11/d14/d4d/d73/f87 x:0 0 0 2024-02-15T18:26:56.947 INFO:tasks.workunit.client.0.smithi042.stdout:3/404: chown d1/d11/d2e/d4c/c64 233549518 1 2024-02-15T18:26:56.956 INFO:tasks.workunit.client.0.smithi042.stdout:6/456: link d9/d11/d19/d1c/d31/l6b d9/l9d 0 2024-02-15T18:26:56.958 INFO:tasks.workunit.client.0.smithi042.stdout:2/354: dwrite d0/f49 [0,4194304] 0 2024-02-15T18:26:56.959 INFO:tasks.workunit.client.0.smithi042.stdout:2/355: write d0/d8/db/de/f4d [7067104,55848] 0 2024-02-15T18:26:56.960 INFO:tasks.workunit.client.1.smithi191.stdout:1/60: sync 2024-02-15T18:26:56.960 INFO:tasks.workunit.client.0.smithi042.stdout:4/360: mknod da/d11/d67/d5c/d66/d78/d2f/c7a 0 2024-02-15T18:26:56.960 INFO:tasks.workunit.client.0.smithi042.stdout:4/361: dread - da/d11/f6d zero size 2024-02-15T18:26:56.961 INFO:tasks.workunit.client.1.smithi191.stdout:9/87: getdents d0/d7/da 0 2024-02-15T18:26:56.961 INFO:tasks.workunit.client.1.smithi191.stdout:9/88: readlink d0/l16 0 2024-02-15T18:26:56.963 INFO:tasks.workunit.client.0.smithi042.stdout:5/398: mkdir d0/d38/d53/d91/d94 0 2024-02-15T18:26:56.964 INFO:tasks.workunit.client.0.smithi042.stdout:5/399: dread d0/f8f [0,4194304] 0 2024-02-15T18:26:56.964 INFO:tasks.workunit.client.0.smithi042.stdout:5/400: fdatasync d0/fe 0 2024-02-15T18:26:56.964 INFO:tasks.workunit.client.0.smithi042.stdout:1/407: creat de/d15/d67/d74/d7e/f94 x:0 0 0 2024-02-15T18:26:56.966 INFO:tasks.workunit.client.1.smithi191.stdout:6/117: dwrite d8/d9/f14 [0,4194304] 0 2024-02-15T18:26:56.966 INFO:tasks.workunit.client.1.smithi191.stdout:1/61: dread d0/f5 [0,4194304] 0 2024-02-15T18:26:56.966 INFO:tasks.workunit.client.1.smithi191.stdout:1/62: readlink d0/d8/df/d12/l15 0 2024-02-15T18:26:56.966 INFO:tasks.workunit.client.1.smithi191.stdout:1/63: truncate d0/f3 4847829 0 2024-02-15T18:26:56.966 INFO:tasks.workunit.client.1.smithi191.stdout:1/64: write d0/db/f14 [4580199,58382] 0 2024-02-15T18:26:56.967 INFO:tasks.workunit.client.0.smithi042.stdout:7/432: mkdir d10/d15/d39/d6c/d92 0 2024-02-15T18:26:56.971 INFO:tasks.workunit.client.0.smithi042.stdout:3/405: rename d1/de/f3a to d1/d11/d14/d4d/d6c/d7b/f88 0 2024-02-15T18:26:56.972 INFO:tasks.workunit.client.1.smithi191.stdout:8/203: mkdir d3/d2c/d1a/d23/d2e/d47 0 2024-02-15T18:26:56.973 INFO:tasks.workunit.client.0.smithi042.stdout:6/457: dread d9/f2f [0,4194304] 0 2024-02-15T18:26:56.973 INFO:tasks.workunit.client.1.smithi191.stdout:8/204: read d3/f12 [495155,60643] 0 2024-02-15T18:26:56.977 INFO:tasks.workunit.client.1.smithi191.stdout:4/81: unlink d2/l9 0 2024-02-15T18:26:56.981 INFO:tasks.workunit.client.0.smithi042.stdout:6/458: dread d9/f2d [4194304,4194304] 0 2024-02-15T18:26:56.981 INFO:tasks.workunit.client.0.smithi042.stdout:6/459: stat d9/f2d 0 2024-02-15T18:26:56.982 INFO:tasks.workunit.client.0.smithi042.stdout:5/401: creat d0/d38/d53/d68/d6a/d42/f95 x:0 0 0 2024-02-15T18:26:56.983 INFO:tasks.workunit.client.0.smithi042.stdout:4/362: dread da/d11/d67/d5c/d66/d78/f1c [4194304,4194304] 0 2024-02-15T18:26:56.986 INFO:tasks.workunit.client.1.smithi191.stdout:9/89: creat d0/d7/f21 x:0 0 0 2024-02-15T18:26:56.986 INFO:tasks.workunit.client.1.smithi191.stdout:9/90: readlink d0/d7/dc/le 0 2024-02-15T18:26:56.988 INFO:tasks.workunit.client.0.smithi042.stdout:3/406: mkdir d1/d11/d14/d4d/d73/d89 0 2024-02-15T18:26:56.990 INFO:tasks.workunit.client.0.smithi042.stdout:2/356: unlink d0/d8/db/de/l2c 0 2024-02-15T18:26:56.994 INFO:tasks.workunit.client.1.smithi191.stdout:6/118: dwrite f1 [0,4194304] 0 2024-02-15T18:26:56.998 INFO:tasks.workunit.client.1.smithi191.stdout:1/65: creat d0/d8/f18 x:0 0 0 2024-02-15T18:26:57.011 INFO:tasks.workunit.client.1.smithi191.stdout:6/119: unlink d8/l10 0 2024-02-15T18:26:57.011 INFO:tasks.workunit.client.1.smithi191.stdout:4/82: chown d2/d5/l1e 6466715 1 2024-02-15T18:26:57.011 INFO:tasks.workunit.client.1.smithi191.stdout:9/91: mknod d0/d7/da/c22 0 2024-02-15T18:26:57.011 INFO:tasks.workunit.client.1.smithi191.stdout:9/92: dread - d0/d7/f11 zero size 2024-02-15T18:26:57.011 INFO:tasks.workunit.client.1.smithi191.stdout:9/93: truncate d0/d7/ff 83379 0 2024-02-15T18:26:57.014 INFO:tasks.workunit.client.0.smithi042.stdout:3/407: rename d1/de/d1a/c28 to d1/d11/d14/d4d/d73/c8a 0 2024-02-15T18:26:57.014 INFO:tasks.workunit.client.0.smithi042.stdout:4/363: unlink da/d11/d29/d4a/d4f/c73 0 2024-02-15T18:26:57.014 INFO:tasks.workunit.client.0.smithi042.stdout:2/357: mknod d0/d8/db/de/d1a/c6d 0 2024-02-15T18:26:57.014 INFO:tasks.workunit.client.0.smithi042.stdout:3/408: dread - d1/d11/f65 zero size 2024-02-15T18:26:57.014 INFO:tasks.workunit.client.0.smithi042.stdout:2/358: truncate d0/d8/db/de/d1a/f4c 2041693 0 2024-02-15T18:26:57.015 INFO:tasks.workunit.client.1.smithi191.stdout:9/94: stat d0/c2 0 2024-02-15T18:26:57.015 INFO:tasks.workunit.client.0.smithi042.stdout:3/409: read d1/de/d1a/f7f [1404710,29347] 0 2024-02-15T18:26:57.015 INFO:tasks.workunit.client.1.smithi191.stdout:4/83: mknod d2/d5/d14/c24 0 2024-02-15T18:26:57.018 INFO:tasks.workunit.client.1.smithi191.stdout:9/95: mknod d0/d7/dc/c23 0 2024-02-15T18:26:57.018 INFO:tasks.workunit.client.0.smithi042.stdout:4/364: mknod da/d11/d29/c7b 0 2024-02-15T18:26:57.019 INFO:tasks.workunit.client.0.smithi042.stdout:2/359: link d0/d8/db/d44/f69 d0/d40/f6e 0 2024-02-15T18:26:57.024 INFO:tasks.workunit.client.0.smithi042.stdout:3/410: creat d1/d11/d14/d4d/d73/f8b x:0 0 0 2024-02-15T18:26:57.025 INFO:tasks.workunit.client.1.smithi191.stdout:9/96: mknod d0/d7/dc/c24 0 2024-02-15T18:26:57.028 INFO:tasks.workunit.client.0.smithi042.stdout:4/365: rename da/d11/d67/f62 to da/f7c 0 2024-02-15T18:26:57.028 INFO:tasks.workunit.client.0.smithi042.stdout:4/366: dread da/d11/d29/d4a/d4f/f6c [0,4194304] 0 2024-02-15T18:26:57.030 INFO:tasks.workunit.client.0.smithi042.stdout:3/411: rmdir d1/d11/d84 0 2024-02-15T18:26:57.032 INFO:tasks.workunit.client.0.smithi042.stdout:3/412: creat d1/de/d19/f8c x:0 0 0 2024-02-15T18:26:57.032 INFO:tasks.workunit.client.0.smithi042.stdout:3/413: readlink d1/l5f 0 2024-02-15T18:26:57.035 INFO:tasks.workunit.client.1.smithi191.stdout:8/205: dwrite d3/d2c/d1a/d23/d2e/f3a [0,4194304] 0 2024-02-15T18:26:57.035 INFO:tasks.workunit.client.1.smithi191.stdout:8/206: read - d3/d2c/d1a/d23/d2a/f35 zero size 2024-02-15T18:26:57.035 INFO:tasks.workunit.client.0.smithi042.stdout:3/414: symlink d1/de/d1a/l8d 0 2024-02-15T18:26:57.035 INFO:tasks.workunit.client.0.smithi042.stdout:3/415: truncate d1/de/d19/f44 4624519 0 2024-02-15T18:26:57.039 INFO:tasks.workunit.client.0.smithi042.stdout:4/367: dread da/d11/d29/f32 [0,4194304] 0 2024-02-15T18:26:57.040 INFO:tasks.workunit.client.1.smithi191.stdout:1/66: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:26:57.040 INFO:tasks.workunit.client.1.smithi191.stdout:8/207: creat d3/d2c/f48 x:0 0 0 2024-02-15T18:26:57.041 INFO:tasks.workunit.client.0.smithi042.stdout:5/402: dwrite d0/f23 [0,4194304] 0 2024-02-15T18:26:57.042 INFO:tasks.workunit.client.0.smithi042.stdout:6/460: dwrite d9/d11/f20 [4194304,4194304] 0 2024-02-15T18:26:57.042 INFO:tasks.workunit.client.0.smithi042.stdout:6/461: chown d9 6 1 2024-02-15T18:26:57.042 INFO:tasks.workunit.client.0.smithi042.stdout:6/462: dread - d9/d3e/f96 zero size 2024-02-15T18:26:57.042 INFO:tasks.workunit.client.0.smithi042.stdout:5/403: read d0/d14/f16 [4714107,105183] 0 2024-02-15T18:26:57.042 INFO:tasks.workunit.client.0.smithi042.stdout:3/416: mknod d1/d11/d14/d4d/d73/c8e 0 2024-02-15T18:26:57.042 INFO:tasks.workunit.client.0.smithi042.stdout:1/408: dwrite de/d15/d67/d63/f69 [4194304,4194304] 0 2024-02-15T18:26:57.042 INFO:tasks.workunit.client.0.smithi042.stdout:1/409: chown de/d15/d67/d46/d6b/l5d 7062 1 2024-02-15T18:26:57.042 INFO:tasks.workunit.client.0.smithi042.stdout:1/410: write de/f58 [703160,91650] 0 2024-02-15T18:26:57.042 INFO:tasks.workunit.client.0.smithi042.stdout:2/360: dwrite d0/d8/db/f3e [0,4194304] 0 2024-02-15T18:26:57.043 INFO:tasks.workunit.client.1.smithi191.stdout:8/208: write d3/f21 [4204818,94719] 0 2024-02-15T18:26:57.043 INFO:tasks.workunit.client.1.smithi191.stdout:8/209: fsync d3/d2c/d1a/d23/d2e/f3a 0 2024-02-15T18:26:57.043 INFO:tasks.workunit.client.1.smithi191.stdout:8/210: write d3/f18 [1631858,10753] 0 2024-02-15T18:26:57.043 INFO:tasks.workunit.client.0.smithi042.stdout:4/368: symlink da/d11/d29/l7d 0 2024-02-15T18:26:57.044 INFO:tasks.workunit.client.0.smithi042.stdout:0/381: sync 2024-02-15T18:26:57.044 INFO:tasks.workunit.client.0.smithi042.stdout:0/382: write d4/d9/d34/d4d/f67 [943236,52545] 0 2024-02-15T18:26:57.046 INFO:tasks.workunit.client.0.smithi042.stdout:4/369: read da/d2b/f37 [88401,88131] 0 2024-02-15T18:26:57.047 INFO:tasks.workunit.client.1.smithi191.stdout:9/97: dwrite d0/d7/f21 [0,4194304] 0 2024-02-15T18:26:57.048 INFO:tasks.workunit.client.1.smithi191.stdout:6/120: rmdir d8 39 2024-02-15T18:26:57.048 INFO:tasks.workunit.client.1.smithi191.stdout:9/98: write d0/f1 [1391116,112513] 0 2024-02-15T18:26:57.052 INFO:tasks.workunit.client.1.smithi191.stdout:4/84: unlink d2/d5/fb 0 2024-02-15T18:26:57.055 INFO:tasks.workunit.client.0.smithi042.stdout:6/463: mkdir d9/d11/d19/d35/d55/d92/d9e 0 2024-02-15T18:26:57.056 INFO:tasks.workunit.client.1.smithi191.stdout:1/67: creat d0/db/d10/f19 x:0 0 0 2024-02-15T18:26:57.059 INFO:tasks.workunit.client.0.smithi042.stdout:5/404: creat d0/d39/f96 x:0 0 0 2024-02-15T18:26:57.059 INFO:tasks.workunit.client.0.smithi042.stdout:5/405: fdatasync d0/db/fd 0 2024-02-15T18:26:57.060 INFO:tasks.workunit.client.0.smithi042.stdout:3/417: rmdir d1/d11 39 2024-02-15T18:26:57.060 INFO:tasks.workunit.client.0.smithi042.stdout:3/418: write d1/d11/d14/d4d/d6c/d7b/f78 [19024,86714] 0 2024-02-15T18:26:57.061 INFO:tasks.workunit.client.0.smithi042.stdout:1/411: readlink de/d15/d2c/d51/d70/l83 0 2024-02-15T18:26:57.062 INFO:tasks.workunit.client.0.smithi042.stdout:2/361: dwrite d0/d8/db/de/d1a/f6b [0,4194304] 0 2024-02-15T18:26:57.062 INFO:tasks.workunit.client.0.smithi042.stdout:2/362: getdents d0/d8/d20/d41 0 2024-02-15T18:26:57.063 INFO:tasks.workunit.client.1.smithi191.stdout:8/211: fsync d3/d2c/f48 0 2024-02-15T18:26:57.063 INFO:tasks.workunit.client.1.smithi191.stdout:8/212: readlink d3/l40 0 2024-02-15T18:26:57.067 INFO:tasks.workunit.client.1.smithi191.stdout:6/121: rename d8/d9/fe to d8/d19/f1f 0 2024-02-15T18:26:57.068 INFO:tasks.workunit.client.1.smithi191.stdout:6/122: readlink d8/l16 0 2024-02-15T18:26:57.068 INFO:tasks.workunit.client.1.smithi191.stdout:5/35: sync 2024-02-15T18:26:57.070 INFO:tasks.workunit.client.1.smithi191.stdout:9/99: creat d0/d7/dc/d17/f25 x:0 0 0 2024-02-15T18:26:57.070 INFO:tasks.workunit.client.1.smithi191.stdout:9/100: dread - d0/f18 zero size 2024-02-15T18:26:57.071 INFO:tasks.workunit.client.1.smithi191.stdout:9/101: rename d0 to d0/d7/da/d26 22 2024-02-15T18:26:57.072 INFO:tasks.workunit.client.0.smithi042.stdout:0/383: symlink d4/d9/d34/d4d/l82 0 2024-02-15T18:26:57.072 INFO:tasks.workunit.client.1.smithi191.stdout:9/102: dread d0/f12 [0,4194304] 0 2024-02-15T18:26:57.072 INFO:tasks.workunit.client.1.smithi191.stdout:9/103: chown d0/d7/da 19858435 1 2024-02-15T18:26:57.073 INFO:tasks.workunit.client.0.smithi042.stdout:4/370: mkdir da/d2b/d7e 0 2024-02-15T18:26:57.075 INFO:tasks.workunit.client.0.smithi042.stdout:5/406: dwrite d0/d12/d65/d74/f72 [0,4194304] 0 2024-02-15T18:26:57.075 INFO:tasks.workunit.client.0.smithi042.stdout:5/407: dread - d0/d38/d53/d68/d6a/f85 zero size 2024-02-15T18:26:57.075 INFO:tasks.workunit.client.0.smithi042.stdout:6/464: mknod d9/d11/d19/d35/d49/d79/c9f 0 2024-02-15T18:26:57.075 INFO:tasks.workunit.client.1.smithi191.stdout:1/68: creat d0/db/d10/f1a x:0 0 0 2024-02-15T18:26:57.076 INFO:tasks.workunit.client.1.smithi191.stdout:1/69: readlink d0/db/lc 0 2024-02-15T18:26:57.076 INFO:tasks.workunit.client.0.smithi042.stdout:3/419: creat d1/d11/f8f x:0 0 0 2024-02-15T18:26:57.076 INFO:tasks.workunit.client.1.smithi191.stdout:1/70: dread d0/d8/fd [0,4194304] 0 2024-02-15T18:26:57.077 INFO:tasks.workunit.client.1.smithi191.stdout:1/71: readlink d0/db/lc 0 2024-02-15T18:26:57.079 INFO:tasks.workunit.client.1.smithi191.stdout:8/213: creat d3/d2c/d1a/d23/d2a/f49 x:0 0 0 2024-02-15T18:26:57.082 INFO:tasks.workunit.client.0.smithi042.stdout:2/363: unlink d0/d8/d10/d31/d3a/d53/d3c/c4e 0 2024-02-15T18:26:57.083 INFO:tasks.workunit.client.1.smithi191.stdout:5/36: mknod d0/cd 0 2024-02-15T18:26:57.083 INFO:tasks.workunit.client.1.smithi191.stdout:5/37: chown d0/l7 200677 1 2024-02-15T18:26:57.084 INFO:tasks.workunit.client.0.smithi042.stdout:0/384: creat d4/d9/d15/f83 x:0 0 0 2024-02-15T18:26:57.084 INFO:tasks.workunit.client.0.smithi042.stdout:0/385: stat d4/d38/d5c 0 2024-02-15T18:26:57.085 INFO:tasks.workunit.client.1.smithi191.stdout:4/85: creat d2/d5/d7/f25 x:0 0 0 2024-02-15T18:26:57.087 INFO:tasks.workunit.client.0.smithi042.stdout:3/420: dread d1/de/d1a/f5a [0,4194304] 0 2024-02-15T18:26:57.087 INFO:tasks.workunit.client.0.smithi042.stdout:4/371: mknod da/d2b/d6e/c7f 0 2024-02-15T18:26:57.088 INFO:tasks.workunit.client.1.smithi191.stdout:8/214: symlink d3/d2c/d1a/d23/d2a/d34/l4a 0 2024-02-15T18:26:57.088 INFO:tasks.workunit.client.1.smithi191.stdout:8/215: dread - d3/d2c/d1a/d23/d2a/f49 zero size 2024-02-15T18:26:57.089 INFO:tasks.workunit.client.1.smithi191.stdout:6/123: symlink d8/d9/l20 0 2024-02-15T18:26:57.090 INFO:tasks.workunit.client.1.smithi191.stdout:6/124: read f1 [214132,15198] 0 2024-02-15T18:26:57.091 INFO:tasks.workunit.client.1.smithi191.stdout:6/125: write d8/f15 [5203011,86662] 0 2024-02-15T18:26:57.091 INFO:tasks.workunit.client.1.smithi191.stdout:6/126: chown d8/d19/l1b 9590 1 2024-02-15T18:26:57.091 INFO:tasks.workunit.client.1.smithi191.stdout:6/127: stat d8/f15 0 2024-02-15T18:26:57.092 INFO:tasks.workunit.client.1.smithi191.stdout:5/38: unlink d0/c5 0 2024-02-15T18:26:57.092 INFO:tasks.workunit.client.1.smithi191.stdout:5/39: chown d0/c3 14681026 1 2024-02-15T18:26:57.092 INFO:tasks.workunit.client.1.smithi191.stdout:5/40: write d0/fc [933727,1042] 0 2024-02-15T18:26:57.092 INFO:tasks.workunit.client.1.smithi191.stdout:5/41: chown d0/l7 7 1 2024-02-15T18:26:57.097 INFO:tasks.workunit.client.0.smithi042.stdout:5/408: unlink d0/db/c3a 0 2024-02-15T18:26:57.098 INFO:tasks.workunit.client.1.smithi191.stdout:4/86: symlink d2/d5/d7/dc/l26 0 2024-02-15T18:26:57.098 INFO:tasks.workunit.client.0.smithi042.stdout:6/465: mknod d9/d11/d19/d1c/d31/d4f/ca0 0 2024-02-15T18:26:57.098 INFO:tasks.workunit.client.0.smithi042.stdout:6/466: fdatasync d9/d11/d19/d1c/d31/d9b/f9c 0 2024-02-15T18:26:57.099 INFO:tasks.workunit.client.0.smithi042.stdout:6/467: chown d9/d11/d19/d1c/l7e 695 1 2024-02-15T18:26:57.101 INFO:tasks.workunit.client.1.smithi191.stdout:6/128: mknod d8/d9/c21 0 2024-02-15T18:26:57.105 INFO:tasks.workunit.client.0.smithi042.stdout:1/412: getdents de/d15/d1b 0 2024-02-15T18:26:57.106 INFO:tasks.workunit.client.0.smithi042.stdout:4/372: dwrite da/d2b/d6e/f71 [0,4194304] 0 2024-02-15T18:26:57.106 INFO:tasks.workunit.client.0.smithi042.stdout:4/373: read da/d11/d27/f69 [361710,128703] 0 2024-02-15T18:26:57.107 INFO:tasks.workunit.client.1.smithi191.stdout:9/104: dwrite d0/d7/ff [0,4194304] 0 2024-02-15T18:26:57.108 INFO:tasks.workunit.client.1.smithi191.stdout:1/72: dwrite d0/db/d10/f19 [0,4194304] 0 2024-02-15T18:26:57.108 INFO:tasks.workunit.client.1.smithi191.stdout:1/73: stat d0/db 0 2024-02-15T18:26:57.110 INFO:tasks.workunit.client.0.smithi042.stdout:2/364: rename d0/f51 to d0/d8/db/de/d1a/f6f 0 2024-02-15T18:26:57.112 INFO:tasks.workunit.client.0.smithi042.stdout:0/386: symlink d4/d9/d62/d74/l84 0 2024-02-15T18:26:57.112 INFO:tasks.workunit.client.0.smithi042.stdout:0/387: write d4/d9/d34/f40 [912732,67653] 0 2024-02-15T18:26:57.112 INFO:tasks.workunit.client.1.smithi191.stdout:5/42: dread d0/fc [0,4194304] 0 2024-02-15T18:26:57.113 INFO:tasks.workunit.client.1.smithi191.stdout:4/87: dread d2/f4 [4194304,4194304] 0 2024-02-15T18:26:57.117 INFO:tasks.workunit.client.0.smithi042.stdout:3/421: mkdir d1/d11/d14/d4d/d73/d89/d90 0 2024-02-15T18:26:57.123 INFO:tasks.workunit.client.0.smithi042.stdout:5/409: creat d0/d89/f97 x:0 0 0 2024-02-15T18:26:57.125 INFO:tasks.workunit.client.0.smithi042.stdout:6/468: creat d9/d11/d46/d89/fa1 x:0 0 0 2024-02-15T18:26:57.125 INFO:tasks.workunit.client.0.smithi042.stdout:3/422: dread d1/de/d1a/f1e [0,4194304] 0 2024-02-15T18:26:57.129 INFO:tasks.workunit.client.1.smithi191.stdout:5/43: rename d0/fc to d0/fe 0 2024-02-15T18:26:57.130 INFO:tasks.workunit.client.1.smithi191.stdout:6/129: dwrite d8/f12 [0,4194304] 0 2024-02-15T18:26:57.130 INFO:tasks.workunit.client.1.smithi191.stdout:1/74: dwrite d0/f3 [4194304,4194304] 0 2024-02-15T18:26:57.130 INFO:tasks.workunit.client.1.smithi191.stdout:4/88: symlink d2/d5/d7/l27 0 2024-02-15T18:26:57.131 INFO:tasks.workunit.client.0.smithi042.stdout:4/374: getdents da/d2b/d6e 0 2024-02-15T18:26:57.131 INFO:tasks.workunit.client.1.smithi191.stdout:4/89: write d2/d5/d7/f15 [1872150,70812] 0 2024-02-15T18:26:57.131 INFO:tasks.workunit.client.1.smithi191.stdout:1/75: dread d0/f9 [0,4194304] 0 2024-02-15T18:26:57.132 INFO:tasks.workunit.client.1.smithi191.stdout:5/44: link d0/c3 d0/cf 0 2024-02-15T18:26:57.132 INFO:tasks.workunit.client.0.smithi042.stdout:0/388: dwrite d4/f68 [0,4194304] 0 2024-02-15T18:26:57.133 INFO:tasks.workunit.client.0.smithi042.stdout:2/365: creat d0/d8/f70 x:0 0 0 2024-02-15T18:26:57.136 INFO:tasks.workunit.client.0.smithi042.stdout:8/334: sync 2024-02-15T18:26:57.136 INFO:tasks.workunit.client.0.smithi042.stdout:9/406: sync 2024-02-15T18:26:57.136 INFO:tasks.workunit.client.0.smithi042.stdout:7/433: sync 2024-02-15T18:26:57.136 INFO:tasks.workunit.client.0.smithi042.stdout:9/407: read d6/f8 [5053022,53439] 0 2024-02-15T18:26:57.137 INFO:tasks.workunit.client.0.smithi042.stdout:9/408: dread - d6/da/d2f/d69/f8b zero size 2024-02-15T18:26:57.137 INFO:tasks.workunit.client.0.smithi042.stdout:5/410: mkdir d0/d50/d98 0 2024-02-15T18:26:57.137 INFO:tasks.workunit.client.0.smithi042.stdout:5/411: chown d0/d22/d7b 45390659 1 2024-02-15T18:26:57.140 INFO:tasks.workunit.client.1.smithi191.stdout:6/130: rename d8/d9/fd to d8/d19/f22 0 2024-02-15T18:26:57.140 INFO:tasks.workunit.client.1.smithi191.stdout:6/131: chown d8/d19 3 1 2024-02-15T18:26:57.140 INFO:tasks.workunit.client.0.smithi042.stdout:6/469: symlink d9/d11/d19/d35/d49/la2 0 2024-02-15T18:26:57.140 INFO:tasks.workunit.client.0.smithi042.stdout:6/470: fdatasync d9/f40 0 2024-02-15T18:26:57.141 INFO:tasks.workunit.client.0.smithi042.stdout:6/471: write d9/d3e/f63 [5114801,83346] 0 2024-02-15T18:26:57.141 INFO:tasks.workunit.client.1.smithi191.stdout:4/90: getdents d2/d5 0 2024-02-15T18:26:57.142 INFO:tasks.workunit.client.0.smithi042.stdout:3/423: mknod d1/d11/c91 0 2024-02-15T18:26:57.142 INFO:tasks.workunit.client.0.smithi042.stdout:3/424: stat d1/f34 0 2024-02-15T18:26:57.142 INFO:tasks.workunit.client.0.smithi042.stdout:3/425: write d1/d11/d2e/d4c/f6a [178369,63465] 0 2024-02-15T18:26:57.143 INFO:tasks.workunit.client.0.smithi042.stdout:3/426: chown d1/d11/d2e/d4c/f76 3 1 2024-02-15T18:26:57.144 INFO:tasks.workunit.client.0.smithi042.stdout:1/413: mkdir de/d15/d95 0 2024-02-15T18:26:57.144 INFO:tasks.workunit.client.0.smithi042.stdout:1/414: readlink de/d15/d1b/l2f 0 2024-02-15T18:26:57.144 INFO:tasks.workunit.client.0.smithi042.stdout:1/415: write de/d15/d67/d89/f8d [1002396,69963] 0 2024-02-15T18:26:57.144 INFO:tasks.workunit.client.1.smithi191.stdout:5/45: symlink d0/l10 0 2024-02-15T18:26:57.144 INFO:tasks.workunit.client.1.smithi191.stdout:5/46: chown d0/lb 984 1 2024-02-15T18:26:57.148 INFO:tasks.workunit.client.0.smithi042.stdout:4/375: mkdir da/d11/d29/d4a/d4f/d80 0 2024-02-15T18:26:57.148 INFO:tasks.workunit.client.0.smithi042.stdout:4/376: truncate da/d3b/d41/f72 700191 0 2024-02-15T18:26:57.149 INFO:tasks.workunit.client.1.smithi191.stdout:6/132: dread f6 [4194304,4194304] 0 2024-02-15T18:26:57.149 INFO:tasks.workunit.client.1.smithi191.stdout:1/76: unlink d0/db/d10/f19 0 2024-02-15T18:26:57.151 INFO:tasks.workunit.client.1.smithi191.stdout:5/47: creat d0/f11 x:0 0 0 2024-02-15T18:26:57.153 INFO:tasks.workunit.client.1.smithi191.stdout:1/77: creat d0/f1b x:0 0 0 2024-02-15T18:26:57.153 INFO:tasks.workunit.client.1.smithi191.stdout:1/78: chown d0/db/lc 0 1 2024-02-15T18:26:57.154 INFO:tasks.workunit.client.0.smithi042.stdout:0/389: link d4/d9/d34/d57/l81 d4/d38/d77/l85 0 2024-02-15T18:26:57.154 INFO:tasks.workunit.client.0.smithi042.stdout:0/390: readlink d4/d38/d77/d2e/l63 0 2024-02-15T18:26:57.155 INFO:tasks.workunit.client.1.smithi191.stdout:5/48: mknod d0/c12 0 2024-02-15T18:26:57.155 INFO:tasks.workunit.client.1.smithi191.stdout:5/49: dread - d0/f11 zero size 2024-02-15T18:26:57.155 INFO:tasks.workunit.client.0.smithi042.stdout:0/391: read d4/d9/d34/f36 [4165435,70172] 0 2024-02-15T18:26:57.155 INFO:tasks.workunit.client.0.smithi042.stdout:2/366: creat d0/f71 x:0 0 0 2024-02-15T18:26:57.156 INFO:tasks.workunit.client.1.smithi191.stdout:1/79: creat d0/d8/f1c x:0 0 0 2024-02-15T18:26:57.156 INFO:tasks.workunit.client.0.smithi042.stdout:2/367: read d0/d8/d10/d31/d3a/d53/f58 [983091,80154] 0 2024-02-15T18:26:57.158 INFO:tasks.workunit.client.0.smithi042.stdout:8/335: unlink dd/de/d3c/f51 0 2024-02-15T18:26:57.158 INFO:tasks.workunit.client.1.smithi191.stdout:5/50: rename d0/fe to d0/f13 0 2024-02-15T18:26:57.160 INFO:tasks.workunit.client.0.smithi042.stdout:7/434: rename d10/f1b to d10/d49/d6d/f93 0 2024-02-15T18:26:57.161 INFO:tasks.workunit.client.1.smithi191.stdout:1/80: link d0/f9 d0/db/f1d 0 2024-02-15T18:26:57.161 INFO:tasks.workunit.client.0.smithi042.stdout:9/409: rmdir d6/da/d1d 39 2024-02-15T18:26:57.163 INFO:tasks.workunit.client.0.smithi042.stdout:5/412: symlink d0/l99 0 2024-02-15T18:26:57.164 INFO:tasks.workunit.client.1.smithi191.stdout:1/81: rename d0/d8/df/d12/l15 to d0/l1e 0 2024-02-15T18:26:57.164 INFO:tasks.workunit.client.0.smithi042.stdout:6/472: creat d9/d3e/fa3 x:0 0 0 2024-02-15T18:26:57.166 INFO:tasks.workunit.client.1.smithi191.stdout:4/91: dwrite d2/f3 [0,4194304] 0 2024-02-15T18:26:57.166 INFO:tasks.workunit.client.1.smithi191.stdout:4/92: stat d2/d5/d7/dc 0 2024-02-15T18:26:57.166 INFO:tasks.workunit.client.0.smithi042.stdout:3/427: link d1/d11/d14/d4d/l55 d1/d11/d14/d4d/d6c/l92 0 2024-02-15T18:26:57.168 INFO:tasks.workunit.client.0.smithi042.stdout:1/416: link de/d15/d67/d46/d6b/d88/l8c de/d15/d2c/l96 0 2024-02-15T18:26:57.169 INFO:tasks.workunit.client.0.smithi042.stdout:4/377: link da/d11/d29/f70 da/d3b/f81 0 2024-02-15T18:26:57.170 INFO:tasks.workunit.client.1.smithi191.stdout:6/133: dwrite f6 [4194304,4194304] 0 2024-02-15T18:26:57.172 INFO:tasks.workunit.client.1.smithi191.stdout:4/93: creat d2/f28 x:0 0 0 2024-02-15T18:26:57.172 INFO:tasks.workunit.client.1.smithi191.stdout:4/94: write d2/d5/d7/f25 [236810,100027] 0 2024-02-15T18:26:57.174 INFO:tasks.workunit.client.0.smithi042.stdout:8/336: dread dd/f2f [0,4194304] 0 2024-02-15T18:26:57.174 INFO:tasks.workunit.client.0.smithi042.stdout:0/392: symlink d4/d9/d62/l86 0 2024-02-15T18:26:57.179 INFO:tasks.workunit.client.0.smithi042.stdout:2/368: unlink d0/d40/f6e 0 2024-02-15T18:26:57.179 INFO:tasks.workunit.client.0.smithi042.stdout:2/369: write d0/d8/d10/f17 [943757,62974] 0 2024-02-15T18:26:57.180 INFO:tasks.workunit.client.0.smithi042.stdout:7/435: mknod d10/d43/d57/c94 0 2024-02-15T18:26:57.181 INFO:tasks.workunit.client.1.smithi191.stdout:1/82: fsync d0/d8/f1c 0 2024-02-15T18:26:57.181 INFO:tasks.workunit.client.1.smithi191.stdout:1/83: dread - d0/d8/f18 zero size 2024-02-15T18:26:57.181 INFO:tasks.workunit.client.1.smithi191.stdout:1/84: read - d0/d8/f17 zero size 2024-02-15T18:26:57.181 INFO:tasks.workunit.client.1.smithi191.stdout:1/85: write d0/f1b [587525,67875] 0 2024-02-15T18:26:57.181 INFO:tasks.workunit.client.0.smithi042.stdout:9/410: creat d6/dd/d9/d59/d5c/f8f x:0 0 0 2024-02-15T18:26:57.182 INFO:tasks.workunit.client.0.smithi042.stdout:9/411: chown d6/da/d2f/d55 59 1 2024-02-15T18:26:57.184 INFO:tasks.workunit.client.1.smithi191.stdout:1/86: dread d0/f4 [0,4194304] 0 2024-02-15T18:26:57.184 INFO:tasks.workunit.client.0.smithi042.stdout:5/413: unlink d0/d38/d53/d68/d6a/c5d 0 2024-02-15T18:26:57.186 INFO:tasks.workunit.client.0.smithi042.stdout:3/428: unlink d1/de/d19/f8c 0 2024-02-15T18:26:57.186 INFO:tasks.workunit.client.1.smithi191.stdout:5/51: fsync d0/f13 0 2024-02-15T18:26:57.186 INFO:tasks.workunit.client.1.smithi191.stdout:5/52: chown d0/l7 0 1 2024-02-15T18:26:57.186 INFO:tasks.workunit.client.1.smithi191.stdout:5/53: fsync d0/f11 0 2024-02-15T18:26:57.187 INFO:tasks.workunit.client.1.smithi191.stdout:6/134: dwrite d8/f12 [0,4194304] 0 2024-02-15T18:26:57.190 INFO:tasks.workunit.client.0.smithi042.stdout:0/393: dwrite d4/d9/f46 [0,4194304] 0 2024-02-15T18:26:57.190 INFO:tasks.workunit.client.0.smithi042.stdout:1/417: symlink de/d15/d95/l97 0 2024-02-15T18:26:57.190 INFO:tasks.workunit.client.0.smithi042.stdout:1/418: dread - de/d15/d67/d46/d6b/f91 zero size 2024-02-15T18:26:57.195 INFO:tasks.workunit.client.1.smithi191.stdout:4/95: getdents d2 0 2024-02-15T18:26:57.195 INFO:tasks.workunit.client.1.smithi191.stdout:4/96: dread - d2/d5/f13 zero size 2024-02-15T18:26:57.196 INFO:tasks.workunit.client.1.smithi191.stdout:1/87: getdents d0/db/d10 0 2024-02-15T18:26:57.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:56.680907+0000 mgr.smithi042.kfoihb (mgr.24435) 36 : cluster [DBG] pgmap v27: 65 pgs: 65 active+clean; 380 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 275 op/s 2024-02-15T18:26:57.198 INFO:tasks.workunit.client.1.smithi191.stdout:6/135: dread d8/f15 [0,4194304] 0 2024-02-15T18:26:57.198 INFO:tasks.workunit.client.1.smithi191.stdout:5/54: mkdir d0/d14 0 2024-02-15T18:26:57.198 INFO:tasks.workunit.client.0.smithi042.stdout:4/378: mkdir da/d11/d67/d5c/d66/d78/d53/d82 0 2024-02-15T18:26:57.199 INFO:tasks.workunit.client.1.smithi191.stdout:4/97: creat d2/d5/d7/f29 x:0 0 0 2024-02-15T18:26:57.201 INFO:tasks.workunit.client.1.smithi191.stdout:1/88: unlink d0/d8/f1c 0 2024-02-15T18:26:57.201 INFO:tasks.workunit.client.1.smithi191.stdout:1/89: truncate d0/d8/f17 212575 0 2024-02-15T18:26:57.201 INFO:tasks.workunit.client.1.smithi191.stdout:1/90: fdatasync d0/f4 0 2024-02-15T18:26:57.201 INFO:tasks.workunit.client.1.smithi191.stdout:1/91: truncate d0/db/f14 5522648 0 2024-02-15T18:26:57.201 INFO:tasks.workunit.client.0.smithi042.stdout:1/419: dread de/d15/d1b/f49 [0,4194304] 0 2024-02-15T18:26:57.202 INFO:tasks.workunit.client.0.smithi042.stdout:1/420: chown de/d15/d1b/d44 2 1 2024-02-15T18:26:57.202 INFO:tasks.workunit.client.0.smithi042.stdout:5/414: dwrite d0/db/f35 [0,4194304] 0 2024-02-15T18:26:57.202 INFO:tasks.workunit.client.1.smithi191.stdout:6/136: rename d8/c11 to d8/c23 0 2024-02-15T18:26:57.202 INFO:tasks.workunit.client.1.smithi191.stdout:6/137: write d8/f12 [4852894,28262] 0 2024-02-15T18:26:57.207 INFO:tasks.workunit.client.1.smithi191.stdout:5/55: mknod d0/c15 0 2024-02-15T18:26:57.209 INFO:tasks.workunit.client.0.smithi042.stdout:8/337: symlink dd/de/l5e 0 2024-02-15T18:26:57.210 INFO:tasks.workunit.client.1.smithi191.stdout:4/98: rename d2/lf to d2/d5/d14/l2a 0 2024-02-15T18:26:57.211 INFO:tasks.workunit.client.0.smithi042.stdout:2/370: symlink d0/d8/d10/d31/l72 0 2024-02-15T18:26:57.212 INFO:tasks.workunit.client.0.smithi042.stdout:7/436: mknod d10/d12/d23/d51/c95 0 2024-02-15T18:26:57.213 INFO:tasks.workunit.client.1.smithi191.stdout:4/99: link d2/f28 d2/f2b 0 2024-02-15T18:26:57.214 INFO:tasks.workunit.client.0.smithi042.stdout:9/412: symlink d6/da/d7b/l90 0 2024-02-15T18:26:57.214 INFO:tasks.workunit.client.0.smithi042.stdout:9/413: chown d6/c60 13 1 2024-02-15T18:26:57.214 INFO:tasks.workunit.client.0.smithi042.stdout:9/414: readlink d6/da/d1d/l1e 0 2024-02-15T18:26:57.215 INFO:tasks.workunit.client.0.smithi042.stdout:3/429: getdents d1/d11/d14/d4d/d73 0 2024-02-15T18:26:57.217 INFO:tasks.workunit.client.1.smithi191.stdout:4/100: creat d2/f2c x:0 0 0 2024-02-15T18:26:57.222 INFO:tasks.workunit.client.0.smithi042.stdout:4/379: creat da/d11/f83 x:0 0 0 2024-02-15T18:26:57.224 INFO:tasks.workunit.client.1.smithi191.stdout:6/138: dwrite f1 [4194304,4194304] 0 2024-02-15T18:26:57.224 INFO:tasks.workunit.client.1.smithi191.stdout:6/139: fdatasync f6 0 2024-02-15T18:26:57.224 INFO:tasks.workunit.client.0.smithi042.stdout:1/421: stat de/d15/d2c/l96 0 2024-02-15T18:26:57.226 INFO:tasks.workunit.client.0.smithi042.stdout:2/371: creat d0/d40/d56/f73 x:0 0 0 2024-02-15T18:26:57.226 INFO:tasks.workunit.client.0.smithi042.stdout:7/437: creat d10/d12/f96 x:0 0 0 2024-02-15T18:26:57.226 INFO:tasks.workunit.client.0.smithi042.stdout:2/372: chown d0/d35/d64 1210557 1 2024-02-15T18:26:57.230 INFO:tasks.workunit.client.1.smithi191.stdout:6/140: dread d8/d19/f22 [0,4194304] 0 2024-02-15T18:26:57.230 INFO:tasks.workunit.client.0.smithi042.stdout:9/415: link d6/dd/d9/f32 d6/da/d1d/d2d/d40/d5f/f91 0 2024-02-15T18:26:57.231 INFO:tasks.workunit.client.0.smithi042.stdout:9/416: truncate d6/dd/f5d 4297309 0 2024-02-15T18:26:57.231 INFO:tasks.workunit.client.0.smithi042.stdout:4/380: rename da/d2b/f52 to da/d11/d29/d4a/f84 0 2024-02-15T18:26:57.232 INFO:tasks.workunit.client.0.smithi042.stdout:2/373: creat d0/d8/db/de/d1a/d2e/f74 x:0 0 0 2024-02-15T18:26:57.234 INFO:tasks.workunit.client.0.smithi042.stdout:9/417: rename d6/da/d1d/d2d/d40/d5f/f91 to d6/da/d7b/f92 0 2024-02-15T18:26:57.234 INFO:tasks.workunit.client.0.smithi042.stdout:2/374: dread d0/d8/d10/d31/d3a/d53/f45 [0,4194304] 0 2024-02-15T18:26:57.235 INFO:tasks.workunit.client.0.smithi042.stdout:4/381: truncate da/d11/d67/d5c/d66/d78/d2f/d4d/f75 2480277 0 2024-02-15T18:26:57.235 INFO:tasks.workunit.client.0.smithi042.stdout:4/382: chown da/d11/d67/d5c/d66/d78/d2f/f34 179731 1 2024-02-15T18:26:57.235 INFO:tasks.workunit.client.0.smithi042.stdout:4/383: fsync da/d11/d67/d5c/d66/d78/d2f/f34 0 2024-02-15T18:26:57.236 INFO:tasks.workunit.client.0.smithi042.stdout:9/418: creat d6/da/d75/f93 x:0 0 0 2024-02-15T18:26:57.236 INFO:tasks.workunit.client.0.smithi042.stdout:9/419: fdatasync d6/da/d2f/d69/f8a 0 2024-02-15T18:26:57.237 INFO:tasks.workunit.client.0.smithi042.stdout:4/384: mkdir da/d11/d85 0 2024-02-15T18:26:57.237 INFO:tasks.workunit.client.0.smithi042.stdout:4/385: chown da/d2b 422264359 1 2024-02-15T18:26:57.238 INFO:tasks.workunit.client.0.smithi042.stdout:9/420: truncate d6/da/d2f/d6c/f71 3994029 0 2024-02-15T18:26:57.238 INFO:tasks.workunit.client.0.smithi042.stdout:9/421: chown d6/da/d1d/d2d/c6b 14308 1 2024-02-15T18:26:57.239 INFO:tasks.workunit.client.0.smithi042.stdout:4/386: creat da/d11/d27/f86 x:0 0 0 2024-02-15T18:26:57.239 INFO:tasks.workunit.client.0.smithi042.stdout:4/387: dread - da/d11/f83 zero size 2024-02-15T18:26:57.239 INFO:tasks.workunit.client.0.smithi042.stdout:4/388: fdatasync da/d11/d67/d5c/d66/d78/f3e 0 2024-02-15T18:26:57.240 INFO:tasks.workunit.client.0.smithi042.stdout:5/415: dwrite d0/db/f15 [0,4194304] 0 2024-02-15T18:26:57.242 INFO:tasks.workunit.client.0.smithi042.stdout:9/422: link d6/f31 d6/da/d2f/d6c/f94 0 2024-02-15T18:26:57.244 INFO:tasks.workunit.client.0.smithi042.stdout:9/423: mkdir d6/da/d1d/d2d/d40/d95 0 2024-02-15T18:26:57.244 INFO:tasks.workunit.client.0.smithi042.stdout:9/424: stat d6/da/d2f/f3d 0 2024-02-15T18:26:57.245 INFO:tasks.workunit.client.0.smithi042.stdout:9/425: getdents d6/da/d2f/d7c 0 2024-02-15T18:26:57.256 INFO:tasks.workunit.client.1.smithi191.stdout:1/92: dwrite d0/db/f14 [0,4194304] 0 2024-02-15T18:26:57.262 INFO:tasks.workunit.client.1.smithi191.stdout:1/93: mkdir d0/db/d10/d1f 0 2024-02-15T18:26:57.263 INFO:tasks.workunit.client.1.smithi191.stdout:1/94: chown d0/db/f1d 1 1 2024-02-15T18:26:57.263 INFO:tasks.workunit.client.1.smithi191.stdout:1/95: symlink d0/l20 0 2024-02-15T18:26:57.264 INFO:tasks.workunit.client.0.smithi042.stdout:9/426: dread d6/f31 [0,4194304] 0 2024-02-15T18:26:57.264 INFO:tasks.workunit.client.1.smithi191.stdout:1/96: mknod d0/d8/c21 0 2024-02-15T18:26:57.266 INFO:tasks.workunit.client.0.smithi042.stdout:9/427: mkdir d6/da/d2f/d54/d96 0 2024-02-15T18:26:57.267 INFO:tasks.workunit.client.1.smithi191.stdout:1/97: unlink d0/f1 0 2024-02-15T18:26:57.269 INFO:tasks.workunit.client.0.smithi042.stdout:9/428: mknod d6/da/d2f/d54/d96/c97 0 2024-02-15T18:26:57.269 INFO:tasks.workunit.client.0.smithi042.stdout:9/429: truncate d6/da/d2f/d69/f7d 641782 0 2024-02-15T18:26:57.269 INFO:tasks.workunit.client.0.smithi042.stdout:9/430: truncate d6/dd/fe 4325417 0 2024-02-15T18:26:57.270 INFO:tasks.workunit.client.0.smithi042.stdout:9/431: creat d6/da/d2f/d7c/f98 x:0 0 0 2024-02-15T18:26:57.271 INFO:tasks.workunit.client.0.smithi042.stdout:9/432: symlink d6/da/d1d/d2d/d40/d5f/l99 0 2024-02-15T18:26:57.277 INFO:tasks.workunit.client.1.smithi191.stdout:5/56: dwrite d0/f13 [0,4194304] 0 2024-02-15T18:26:57.277 INFO:tasks.workunit.client.1.smithi191.stdout:5/57: readlink d0/l10 0 2024-02-15T18:26:57.278 INFO:tasks.workunit.client.0.smithi042.stdout:9/433: dread d6/da/d2f/d55/f6d [0,4194304] 0 2024-02-15T18:26:57.280 INFO:tasks.workunit.client.0.smithi042.stdout:8/338: dwrite dd/de/d12/f45 [0,4194304] 0 2024-02-15T18:26:57.281 INFO:tasks.workunit.client.0.smithi042.stdout:3/430: dwrite d1/d11/d14/f35 [0,4194304] 0 2024-02-15T18:26:57.281 INFO:tasks.workunit.client.0.smithi042.stdout:3/431: chown d1/d11/d14/d4d/d73/c8a 30 1 2024-02-15T18:26:57.282 INFO:tasks.workunit.client.0.smithi042.stdout:5/416: dwrite d0/d38/f5a [0,4194304] 0 2024-02-15T18:26:57.282 INFO:tasks.workunit.client.0.smithi042.stdout:4/389: dwrite da/d11/d67/d5c/d66/d78/d2f/f34 [4194304,4194304] 0 2024-02-15T18:26:57.282 INFO:tasks.workunit.client.0.smithi042.stdout:5/417: fdatasync d0/d39/f75 0 2024-02-15T18:26:57.282 INFO:tasks.workunit.client.0.smithi042.stdout:5/418: chown d0/d39/f75 1 1 2024-02-15T18:26:57.282 INFO:tasks.workunit.client.0.smithi042.stdout:5/419: chown d0/d12/d65/d74/d52/l58 801931 1 2024-02-15T18:26:57.284 INFO:tasks.workunit.client.1.smithi191.stdout:6/141: dwrite f6 [0,4194304] 0 2024-02-15T18:26:57.284 INFO:tasks.workunit.client.1.smithi191.stdout:6/142: truncate d8/d19/f1d 389281 0 2024-02-15T18:26:57.284 INFO:tasks.workunit.client.0.smithi042.stdout:1/422: dwrite f2 [4194304,4194304] 0 2024-02-15T18:26:57.285 INFO:tasks.workunit.client.0.smithi042.stdout:1/423: write de/d15/d67/f3a [458936,98380] 0 2024-02-15T18:26:57.288 INFO:tasks.workunit.client.1.smithi191.stdout:4/101: truncate d2/d5/d7/f15 37343 0 2024-02-15T18:26:57.291 INFO:tasks.workunit.client.0.smithi042.stdout:4/390: fsync da/d11/d29/d4a/f84 0 2024-02-15T18:26:57.292 INFO:tasks.workunit.client.1.smithi191.stdout:4/102: write d2/f4 [1294111,105577] 0 2024-02-15T18:26:57.300 INFO:tasks.workunit.client.1.smithi191.stdout:5/58: mknod d0/d14/c16 0 2024-02-15T18:26:57.307 INFO:tasks.workunit.client.1.smithi191.stdout:5/59: creat d0/d14/f17 x:0 0 0 2024-02-15T18:26:57.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:56.680907+0000 mgr.smithi042.kfoihb (mgr.24435) 36 : cluster [DBG] pgmap v27: 65 pgs: 65 active+clean; 380 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 6.4 MiB/s rd, 18 MiB/s wr, 275 op/s 2024-02-15T18:26:57.309 INFO:tasks.workunit.client.0.smithi042.stdout:3/432: symlink d1/d11/d2e/d4c/l93 0 2024-02-15T18:26:57.310 INFO:tasks.workunit.client.0.smithi042.stdout:3/433: write d1/d11/f66 [4295559,31256] 0 2024-02-15T18:26:57.310 INFO:tasks.workunit.client.0.smithi042.stdout:3/434: readlink d1/de/d19/l56 0 2024-02-15T18:26:57.310 INFO:tasks.workunit.client.0.smithi042.stdout:3/435: read d1/d11/d2e/d4c/f6a [149686,125325] 0 2024-02-15T18:26:57.312 INFO:tasks.workunit.client.1.smithi191.stdout:5/60: mkdir d0/d18 0 2024-02-15T18:26:57.312 INFO:tasks.workunit.client.1.smithi191.stdout:5/61: write d0/f11 [117984,1849] 0 2024-02-15T18:26:57.312 INFO:tasks.workunit.client.1.smithi191.stdout:5/62: stat d0/f11 0 2024-02-15T18:26:57.314 INFO:tasks.workunit.client.1.smithi191.stdout:5/63: link d0/l7 d0/l19 0 2024-02-15T18:26:57.315 INFO:tasks.workunit.client.0.smithi042.stdout:5/420: mknod d0/d12/c9a 0 2024-02-15T18:26:57.316 INFO:tasks.workunit.client.1.smithi191.stdout:5/64: creat d0/f1a x:0 0 0 2024-02-15T18:26:57.317 INFO:tasks.workunit.client.0.smithi042.stdout:1/424: mknod de/d6d/c98 0 2024-02-15T18:26:57.318 INFO:tasks.workunit.client.0.smithi042.stdout:4/391: mknod da/d11/d67/c87 0 2024-02-15T18:26:57.319 INFO:tasks.workunit.client.0.smithi042.stdout:5/421: mkdir d0/d39/d8d/d9b 0 2024-02-15T18:26:57.319 INFO:tasks.workunit.client.0.smithi042.stdout:5/422: readlink d0/l1d 0 2024-02-15T18:26:57.320 INFO:tasks.workunit.client.0.smithi042.stdout:1/425: creat de/d15/d95/f99 x:0 0 0 2024-02-15T18:26:57.320 INFO:tasks.workunit.client.0.smithi042.stdout:5/423: write d0/db/f1f [4842912,128204] 0 2024-02-15T18:26:57.321 INFO:tasks.workunit.client.1.smithi191.stdout:4/103: dwrite d2/d5/f1a [0,4194304] 0 2024-02-15T18:26:57.321 INFO:tasks.workunit.client.1.smithi191.stdout:4/104: write d2/f2b [243065,47375] 0 2024-02-15T18:26:57.321 INFO:tasks.workunit.client.1.smithi191.stdout:4/105: dread - d2/f2c zero size 2024-02-15T18:26:57.322 INFO:tasks.workunit.client.0.smithi042.stdout:4/392: link da/d11/c15 da/d2b/c88 0 2024-02-15T18:26:57.329 INFO:tasks.workunit.client.0.smithi042.stdout:1/426: mkdir de/d54/d9a 0 2024-02-15T18:26:57.330 INFO:tasks.workunit.client.0.smithi042.stdout:4/393: creat da/d11/d27/f89 x:0 0 0 2024-02-15T18:26:57.330 INFO:tasks.workunit.client.0.smithi042.stdout:4/394: fdatasync da/f63 0 2024-02-15T18:26:57.331 INFO:tasks.workunit.client.0.smithi042.stdout:1/427: mknod de/d15/d67/d74/c9b 0 2024-02-15T18:26:57.334 INFO:tasks.workunit.client.0.smithi042.stdout:1/428: creat de/d15/d67/d66/d7c/f9c x:0 0 0 2024-02-15T18:26:57.335 INFO:tasks.workunit.client.0.smithi042.stdout:1/429: link de/d15/d2c/f4b de/d15/d1b/f9d 0 2024-02-15T18:26:57.337 INFO:tasks.workunit.client.0.smithi042.stdout:1/430: rmdir de/d15/d67/d46/d6a 39 2024-02-15T18:26:57.338 INFO:tasks.workunit.client.0.smithi042.stdout:1/431: write de/d15/d2c/f5a [20081,27264] 0 2024-02-15T18:26:57.339 INFO:tasks.workunit.client.0.smithi042.stdout:1/432: read f6 [2797638,92098] 0 2024-02-15T18:26:57.340 INFO:tasks.workunit.client.0.smithi042.stdout:5/424: dwrite d0/db/fd [0,4194304] 0 2024-02-15T18:26:57.340 INFO:tasks.workunit.client.0.smithi042.stdout:1/433: mkdir de/d6d/d9e 0 2024-02-15T18:26:57.340 INFO:tasks.workunit.client.0.smithi042.stdout:1/434: chown de/d15/d67/d89 7373 1 2024-02-15T18:26:57.340 INFO:tasks.workunit.client.0.smithi042.stdout:1/435: stat de/d15/d1b/f25 0 2024-02-15T18:26:57.341 INFO:tasks.workunit.client.1.smithi191.stdout:3/45: sync 2024-02-15T18:26:57.341 INFO:tasks.workunit.client.1.smithi191.stdout:0/31: sync 2024-02-15T18:26:57.342 INFO:tasks.workunit.client.0.smithi042.stdout:5/425: rename d0/d38/d53/d68/d6a/d42/f71 to d0/d14/f9c 0 2024-02-15T18:26:57.342 INFO:tasks.workunit.client.0.smithi042.stdout:5/426: fsync d0/d12/f21 0 2024-02-15T18:26:57.342 INFO:tasks.workunit.client.0.smithi042.stdout:7/438: dwrite d10/d49/d6d/d78/f91 [0,4194304] 0 2024-02-15T18:26:57.344 INFO:tasks.workunit.client.0.smithi042.stdout:1/436: symlink de/d15/d67/d74/d7e/l9f 0 2024-02-15T18:26:57.344 INFO:tasks.workunit.client.1.smithi191.stdout:3/46: symlink d6/de/l14 0 2024-02-15T18:26:57.345 INFO:tasks.workunit.client.1.smithi191.stdout:3/47: read d6/fb [308721,75270] 0 2024-02-15T18:26:57.346 INFO:tasks.workunit.client.0.smithi042.stdout:5/427: creat d0/f9d x:0 0 0 2024-02-15T18:26:57.346 INFO:tasks.workunit.client.0.smithi042.stdout:5/428: chown d0/d14/f9c 252106497 1 2024-02-15T18:26:57.347 INFO:tasks.workunit.client.1.smithi191.stdout:4/106: dwrite d2/d5/d7/f25 [0,4194304] 0 2024-02-15T18:26:57.347 INFO:tasks.workunit.client.1.smithi191.stdout:4/107: chown d2/d5/d7/f29 91675632 1 2024-02-15T18:26:57.347 INFO:tasks.workunit.client.1.smithi191.stdout:4/108: chown d2/d5/d14 50 1 2024-02-15T18:26:57.347 INFO:tasks.workunit.client.1.smithi191.stdout:1/98: dwrite d0/db/f1d [0,4194304] 0 2024-02-15T18:26:57.347 INFO:tasks.workunit.client.1.smithi191.stdout:1/99: truncate d0/f4 2294344 0 2024-02-15T18:26:57.347 INFO:tasks.workunit.client.1.smithi191.stdout:0/32: mkdir d7/d9 0 2024-02-15T18:26:57.348 INFO:tasks.workunit.client.1.smithi191.stdout:4/109: dread d2/d5/d7/f15 [0,4194304] 0 2024-02-15T18:26:57.348 INFO:tasks.workunit.client.1.smithi191.stdout:4/110: chown d2/d5/d7/f18 2021 1 2024-02-15T18:26:57.350 INFO:tasks.workunit.client.1.smithi191.stdout:3/48: unlink d6/f10 0 2024-02-15T18:26:57.350 INFO:tasks.workunit.client.1.smithi191.stdout:3/49: fsync d6/de/ff 0 2024-02-15T18:26:57.350 INFO:tasks.workunit.client.1.smithi191.stdout:3/50: dread - d6/fa zero size 2024-02-15T18:26:57.350 INFO:tasks.workunit.client.1.smithi191.stdout:3/51: stat d6/f8 0 2024-02-15T18:26:57.350 INFO:tasks.workunit.client.1.smithi191.stdout:3/52: chown d6/de/f11 1467 1 2024-02-15T18:26:57.353 INFO:tasks.workunit.client.0.smithi042.stdout:7/439: rename d10/d49/d6d/l80 to d10/d12/d23/d51/l97 0 2024-02-15T18:26:57.357 INFO:tasks.workunit.client.0.smithi042.stdout:8/339: dwrite dd/de/d33/d3b/f56 [0,4194304] 0 2024-02-15T18:26:57.358 INFO:tasks.workunit.client.0.smithi042.stdout:7/440: symlink d10/d12/d23/d51/l98 0 2024-02-15T18:26:57.359 INFO:tasks.workunit.client.1.smithi191.stdout:1/100: creat d0/d8/df/f22 x:0 0 0 2024-02-15T18:26:57.363 INFO:tasks.workunit.client.0.smithi042.stdout:1/437: rename de/d15/l92 to de/d15/d67/d89/la0 0 2024-02-15T18:26:57.367 INFO:tasks.workunit.client.0.smithi042.stdout:7/441: dread d10/d43/f45 [0,4194304] 0 2024-02-15T18:26:57.367 INFO:tasks.workunit.client.0.smithi042.stdout:7/442: fsync d10/d12/f5f 0 2024-02-15T18:26:57.367 INFO:tasks.workunit.client.1.smithi191.stdout:4/111: mknod d2/c2d 0 2024-02-15T18:26:57.367 INFO:tasks.workunit.client.1.smithi191.stdout:4/112: stat d2/c2d 0 2024-02-15T18:26:57.367 INFO:tasks.workunit.client.1.smithi191.stdout:3/53: symlink d6/de/l15 0 2024-02-15T18:26:57.369 INFO:tasks.workunit.client.1.smithi191.stdout:6/143: dwrite f1 [0,4194304] 0 2024-02-15T18:26:57.372 INFO:tasks.workunit.client.0.smithi042.stdout:9/434: dwrite d6/da/f10 [4194304,4194304] 0 2024-02-15T18:26:57.372 INFO:tasks.workunit.client.0.smithi042.stdout:9/435: fdatasync d6/f17 0 2024-02-15T18:26:57.372 INFO:tasks.workunit.client.1.smithi191.stdout:3/54: mkdir d6/d16 0 2024-02-15T18:26:57.373 INFO:tasks.workunit.client.1.smithi191.stdout:3/55: dread - f3 zero size 2024-02-15T18:26:57.373 INFO:tasks.workunit.client.1.smithi191.stdout:3/56: fdatasync f3 0 2024-02-15T18:26:57.373 INFO:tasks.workunit.client.1.smithi191.stdout:3/57: dread - d6/de/ff zero size 2024-02-15T18:26:57.373 INFO:tasks.workunit.client.1.smithi191.stdout:3/58: truncate d6/f8 1326686 0 2024-02-15T18:26:57.373 INFO:tasks.workunit.client.0.smithi042.stdout:8/340: creat dd/de/d12/d32/f5f x:0 0 0 2024-02-15T18:26:57.376 INFO:tasks.workunit.client.1.smithi191.stdout:0/33: getdents d7 0 2024-02-15T18:26:57.378 INFO:tasks.workunit.client.0.smithi042.stdout:7/443: creat d10/d12/d23/d51/f99 x:0 0 0 2024-02-15T18:26:57.383 INFO:tasks.workunit.client.1.smithi191.stdout:6/144: rename d8/c18 to d8/d19/c24 0 2024-02-15T18:26:57.383 INFO:tasks.workunit.client.0.smithi042.stdout:4/395: dwrite da/d11/d29/f32 [0,4194304] 0 2024-02-15T18:26:57.384 INFO:tasks.workunit.client.0.smithi042.stdout:4/396: dread da/d11/d29/d4a/d4f/f6c [0,4194304] 0 2024-02-15T18:26:57.384 INFO:tasks.workunit.client.0.smithi042.stdout:4/397: truncate da/d11/d67/d5c/d66/d78/d2f/d4d/f48 382549 0 2024-02-15T18:26:57.384 INFO:tasks.workunit.client.0.smithi042.stdout:4/398: fdatasync da/d11/d67/d5c/d66/d78/d2f/d4d/f47 0 2024-02-15T18:26:57.384 INFO:tasks.workunit.client.0.smithi042.stdout:4/399: dread da/d11/d67/d5c/d66/d78/d2f/d4d/f48 [0,4194304] 0 2024-02-15T18:26:57.387 INFO:tasks.workunit.client.0.smithi042.stdout:5/429: dwrite d0/db/f35 [0,4194304] 0 2024-02-15T18:26:57.387 INFO:tasks.workunit.client.0.smithi042.stdout:5/430: chown d0/db/f1f 6732 1 2024-02-15T18:26:57.387 INFO:tasks.workunit.client.1.smithi191.stdout:1/101: dwrite d0/db/d10/f1a [0,4194304] 0 2024-02-15T18:26:57.388 INFO:tasks.workunit.client.0.smithi042.stdout:9/436: unlink d6/da/d2f/c79 0 2024-02-15T18:26:57.389 INFO:tasks.workunit.client.1.smithi191.stdout:4/113: unlink d2/d5/c23 0 2024-02-15T18:26:57.393 INFO:tasks.workunit.client.1.smithi191.stdout:3/59: mknod d6/c17 0 2024-02-15T18:26:57.393 INFO:tasks.workunit.client.1.smithi191.stdout:3/60: dread - d6/de/f13 zero size 2024-02-15T18:26:57.393 INFO:tasks.workunit.client.1.smithi191.stdout:3/61: dread - d6/fa zero size 2024-02-15T18:26:57.393 INFO:tasks.workunit.client.1.smithi191.stdout:3/62: truncate f3 155583 0 2024-02-15T18:26:57.395 INFO:tasks.workunit.client.1.smithi191.stdout:2/71: sync 2024-02-15T18:26:57.397 INFO:tasks.workunit.client.1.smithi191.stdout:2/72: dread f1 [4194304,4194304] 0 2024-02-15T18:26:57.397 INFO:tasks.workunit.client.1.smithi191.stdout:2/73: stat fd 0 2024-02-15T18:26:57.397 INFO:tasks.workunit.client.1.smithi191.stdout:2/74: chown fb 5 1 2024-02-15T18:26:57.402 INFO:tasks.workunit.client.1.smithi191.stdout:4/114: dread d2/d5/d7/f25 [0,4194304] 0 2024-02-15T18:26:57.402 INFO:tasks.workunit.client.1.smithi191.stdout:4/115: fdatasync d2/d5/d7/f29 0 2024-02-15T18:26:57.402 INFO:tasks.workunit.client.1.smithi191.stdout:4/116: write d2/d5/f1a [4203218,119510] 0 2024-02-15T18:26:57.403 INFO:tasks.workunit.client.1.smithi191.stdout:0/34: rmdir d7 39 2024-02-15T18:26:57.403 INFO:tasks.workunit.client.1.smithi191.stdout:0/35: write f6 [5217389,45402] 0 2024-02-15T18:26:57.405 INFO:tasks.workunit.client.0.smithi042.stdout:8/341: mknod dd/de/d12/d32/c60 0 2024-02-15T18:26:57.405 INFO:tasks.workunit.client.0.smithi042.stdout:9/437: dwrite d6/dd/d9/d59/d5c/f8f [0,4194304] 0 2024-02-15T18:26:57.405 INFO:tasks.workunit.client.0.smithi042.stdout:9/438: chown d6/dd/f2b 11418970 1 2024-02-15T18:26:57.405 INFO:tasks.workunit.client.0.smithi042.stdout:9/439: stat d6/dd/c3f 0 2024-02-15T18:26:57.406 INFO:tasks.workunit.client.0.smithi042.stdout:1/438: link de/d15/f20 de/d15/d67/d89/fa1 0 2024-02-15T18:26:57.406 INFO:tasks.workunit.client.0.smithi042.stdout:1/439: dread - de/d15/d67/d74/f90 zero size 2024-02-15T18:26:57.406 INFO:tasks.workunit.client.0.smithi042.stdout:1/440: write de/d15/d2c/f4b [815821,18897] 0 2024-02-15T18:26:57.410 INFO:tasks.workunit.client.1.smithi191.stdout:6/145: truncate d8/d9/f14 1200516 0 2024-02-15T18:26:57.415 INFO:tasks.workunit.client.0.smithi042.stdout:6/473: sync 2024-02-15T18:26:57.417 INFO:tasks.workunit.client.0.smithi042.stdout:0/394: sync 2024-02-15T18:26:57.417 INFO:tasks.workunit.client.0.smithi042.stdout:0/395: dread - d4/d9/d5a/d60/f64 zero size 2024-02-15T18:26:57.417 INFO:tasks.workunit.client.0.smithi042.stdout:0/396: chown d4/d9/d34/f40 5041139 1 2024-02-15T18:26:57.419 INFO:tasks.workunit.client.0.smithi042.stdout:4/400: getdents da/d11/d67/d5c/d66 0 2024-02-15T18:26:57.419 INFO:tasks.workunit.client.0.smithi042.stdout:7/444: unlink d10/d49/d68/l70 0 2024-02-15T18:26:57.419 INFO:tasks.workunit.client.0.smithi042.stdout:7/445: write d10/d43/f4f [4595301,26003] 0 2024-02-15T18:26:57.422 INFO:tasks.workunit.client.1.smithi191.stdout:1/102: rename d0/db/d10/f1a to d0/d8/df/d12/f23 0 2024-02-15T18:26:57.422 INFO:tasks.workunit.client.1.smithi191.stdout:3/63: mknod d6/c18 0 2024-02-15T18:26:57.422 INFO:tasks.workunit.client.1.smithi191.stdout:3/64: write d6/fa [689746,4866] 0 2024-02-15T18:26:57.422 INFO:tasks.workunit.client.0.smithi042.stdout:5/431: unlink d0/d14/f16 0 2024-02-15T18:26:57.427 INFO:tasks.workunit.client.0.smithi042.stdout:7/446: read d10/d12/f6b [4153525,120274] 0 2024-02-15T18:26:57.428 INFO:tasks.workunit.client.1.smithi191.stdout:4/117: creat d2/d5/d14/f2e x:0 0 0 2024-02-15T18:26:57.435 INFO:tasks.workunit.client.1.smithi191.stdout:0/36: rmdir d7/d9 0 2024-02-15T18:26:57.438 INFO:tasks.workunit.client.0.smithi042.stdout:6/474: link d9/d11/d1d/l85 d9/d11/d46/la4 0 2024-02-15T18:26:57.438 INFO:tasks.workunit.client.0.smithi042.stdout:6/475: write d9/d11/d19/d1c/d31/f6f [69685,66993] 0 2024-02-15T18:26:57.438 INFO:tasks.workunit.client.0.smithi042.stdout:6/476: fsync d9/f2f 0 2024-02-15T18:26:57.438 INFO:tasks.workunit.client.1.smithi191.stdout:6/146: truncate d8/f15 1102288 0 2024-02-15T18:26:57.439 INFO:tasks.workunit.client.1.smithi191.stdout:6/147: stat d8/d9/f14 0 2024-02-15T18:26:57.440 INFO:tasks.workunit.client.0.smithi042.stdout:0/397: truncate d4/d9/d15/f18 350014 0 2024-02-15T18:26:57.446 INFO:tasks.workunit.client.1.smithi191.stdout:2/75: rename f1 to df/f10 0 2024-02-15T18:26:57.449 INFO:tasks.workunit.client.0.smithi042.stdout:4/401: unlink da/d11/d67/d5c/d66/d78/d2f/c7a 0 2024-02-15T18:26:57.449 INFO:tasks.workunit.client.0.smithi042.stdout:2/375: sync 2024-02-15T18:26:57.450 INFO:tasks.workunit.client.1.smithi191.stdout:1/103: dwrite d0/d8/df/d12/f23 [0,4194304] 0 2024-02-15T18:26:57.450 INFO:tasks.workunit.client.1.smithi191.stdout:1/104: stat d0/db/d10/d1f 0 2024-02-15T18:26:57.450 INFO:tasks.workunit.client.1.smithi191.stdout:3/65: mknod d6/de/c19 0 2024-02-15T18:26:57.450 INFO:tasks.workunit.client.0.smithi042.stdout:9/440: dwrite d6/da/d2f/d69/f8b [0,4194304] 0 2024-02-15T18:26:57.451 INFO:tasks.workunit.client.1.smithi191.stdout:3/66: write d6/fb [284068,128751] 0 2024-02-15T18:26:57.451 INFO:tasks.workunit.client.1.smithi191.stdout:4/118: mkdir d2/d5/d7/d2f 0 2024-02-15T18:26:57.452 INFO:tasks.workunit.client.1.smithi191.stdout:4/119: readlink d2/le 0 2024-02-15T18:26:57.452 INFO:tasks.workunit.client.1.smithi191.stdout:4/120: chown d2/d5/l1e 1569 1 2024-02-15T18:26:57.452 INFO:tasks.workunit.client.1.smithi191.stdout:4/121: getdents d2/d5/d7/d2f 0 2024-02-15T18:26:57.454 INFO:tasks.workunit.client.0.smithi042.stdout:5/432: link d0/db/fd d0/d12/d65/d74/d52/f9e 0 2024-02-15T18:26:57.454 INFO:tasks.workunit.client.0.smithi042.stdout:5/433: write d0/d89/f97 [245270,67757] 0 2024-02-15T18:26:57.454 INFO:tasks.workunit.client.0.smithi042.stdout:1/441: dwrite de/d15/d67/d46/f71 [0,4194304] 0 2024-02-15T18:26:57.454 INFO:tasks.workunit.client.0.smithi042.stdout:7/447: dwrite d10/d49/d6d/f86 [0,4194304] 0 2024-02-15T18:26:57.455 INFO:tasks.workunit.client.1.smithi191.stdout:1/105: write d0/d8/df/d12/f23 [2103906,36841] 0 2024-02-15T18:26:57.456 INFO:tasks.workunit.client.0.smithi042.stdout:0/398: link d4/d38/f3a d4/d9/d34/d4e/f87 0 2024-02-15T18:26:57.457 INFO:tasks.workunit.client.1.smithi191.stdout:6/148: getdents d8/d19 0 2024-02-15T18:26:57.458 INFO:tasks.workunit.client.1.smithi191.stdout:1/106: write d0/f4 [212814,67149] 0 2024-02-15T18:26:57.459 INFO:tasks.workunit.client.1.smithi191.stdout:0/37: dread f6 [4194304,4194304] 0 2024-02-15T18:26:57.461 INFO:tasks.workunit.client.0.smithi042.stdout:3/436: sync 2024-02-15T18:26:57.461 INFO:tasks.workunit.client.0.smithi042.stdout:3/437: chown d1/de/d19/f4b 14 1 2024-02-15T18:26:57.461 INFO:tasks.workunit.client.0.smithi042.stdout:6/477: dwrite d9/d11/d19/d35/f51 [0,4194304] 0 2024-02-15T18:26:57.464 INFO:tasks.workunit.client.0.smithi042.stdout:1/442: dread de/d15/d1b/d44/f85 [0,4194304] 0 2024-02-15T18:26:57.466 INFO:tasks.workunit.client.1.smithi191.stdout:0/38: dread f6 [0,4194304] 0 2024-02-15T18:26:57.467 INFO:tasks.workunit.client.0.smithi042.stdout:4/402: creat da/d11/d67/d5c/d66/f8a x:0 0 0 2024-02-15T18:26:57.467 INFO:tasks.workunit.client.0.smithi042.stdout:4/403: chown da/d11/d67/d5c 1064 1 2024-02-15T18:26:57.467 INFO:tasks.workunit.client.0.smithi042.stdout:4/404: write da/d11/d29/d4a/f84 [1876910,91648] 0 2024-02-15T18:26:57.468 INFO:tasks.workunit.client.1.smithi191.stdout:2/76: truncate f7 11055 0 2024-02-15T18:26:57.472 INFO:tasks.workunit.client.1.smithi191.stdout:4/122: mkdir d2/d5/d7/dc/d30 0 2024-02-15T18:26:57.475 INFO:tasks.workunit.client.0.smithi042.stdout:9/441: creat d6/da/f9a x:0 0 0 2024-02-15T18:26:57.475 INFO:tasks.workunit.client.0.smithi042.stdout:9/442: chown f3 25 1 2024-02-15T18:26:57.476 INFO:tasks.workunit.client.1.smithi191.stdout:1/107: link d0/c6 d0/d8/df/d12/c24 0 2024-02-15T18:26:57.477 INFO:tasks.workunit.client.1.smithi191.stdout:0/39: mkdir d7/da 0 2024-02-15T18:26:57.479 INFO:tasks.workunit.client.0.smithi042.stdout:5/434: link d0/db/f15 d0/d38/d53/d64/f9f 0 2024-02-15T18:26:57.480 INFO:tasks.workunit.client.0.smithi042.stdout:2/376: dwrite d0/f71 [0,4194304] 0 2024-02-15T18:26:57.481 INFO:tasks.workunit.client.1.smithi191.stdout:2/77: truncate f9 1432292 0 2024-02-15T18:26:57.483 INFO:tasks.workunit.client.1.smithi191.stdout:1/108: rmdir d0/db/d10/d1f 0 2024-02-15T18:26:57.483 INFO:tasks.workunit.client.1.smithi191.stdout:1/109: readlink d0/db/lc 0 2024-02-15T18:26:57.483 INFO:tasks.workunit.client.1.smithi191.stdout:1/110: fsync d0/db/f1d 0 2024-02-15T18:26:57.484 INFO:tasks.workunit.client.1.smithi191.stdout:4/123: creat d2/d5/d7/d2f/f31 x:0 0 0 2024-02-15T18:26:57.484 INFO:tasks.workunit.client.0.smithi042.stdout:7/448: creat d10/d15/d39/d6c/f9a x:0 0 0 2024-02-15T18:26:57.484 INFO:tasks.workunit.client.1.smithi191.stdout:2/78: fsync fd 0 2024-02-15T18:26:57.486 INFO:tasks.workunit.client.1.smithi191.stdout:3/67: getdents d6/de 0 2024-02-15T18:26:57.486 INFO:tasks.workunit.client.1.smithi191.stdout:3/68: write d6/de/f13 [672666,31386] 0 2024-02-15T18:26:57.491 INFO:tasks.workunit.client.1.smithi191.stdout:6/149: dwrite d8/f15 [0,4194304] 0 2024-02-15T18:26:57.491 INFO:tasks.workunit.client.1.smithi191.stdout:0/40: creat d7/fb x:0 0 0 2024-02-15T18:26:57.492 INFO:tasks.workunit.client.0.smithi042.stdout:3/438: mknod d1/d11/d2e/c94 0 2024-02-15T18:26:57.493 INFO:tasks.workunit.client.0.smithi042.stdout:6/478: symlink d9/d11/d19/d35/d49/la5 0 2024-02-15T18:26:57.497 INFO:tasks.workunit.client.0.smithi042.stdout:0/399: dwrite d4/d9/d34/d4d/f67 [0,4194304] 0 2024-02-15T18:26:57.497 INFO:tasks.workunit.client.0.smithi042.stdout:1/443: symlink de/d15/d67/d46/d6b/la2 0 2024-02-15T18:26:57.499 INFO:tasks.workunit.client.0.smithi042.stdout:5/435: dread d0/d38/d53/f57 [0,4194304] 0 2024-02-15T18:26:57.500 INFO:tasks.workunit.client.1.smithi191.stdout:1/111: unlink d0/d8/df/f22 0 2024-02-15T18:26:57.500 INFO:tasks.workunit.client.1.smithi191.stdout:4/124: symlink d2/d5/d7/d2f/l32 0 2024-02-15T18:26:57.500 INFO:tasks.workunit.client.1.smithi191.stdout:4/125: stat d2/d5/d7/dc/l26 0 2024-02-15T18:26:57.500 INFO:tasks.workunit.client.1.smithi191.stdout:4/126: readlink d2/d5/l1e 0 2024-02-15T18:26:57.500 INFO:tasks.workunit.client.1.smithi191.stdout:4/127: chown d2/c21 30807101 1 2024-02-15T18:26:57.500 INFO:tasks.workunit.client.0.smithi042.stdout:9/443: symlink d6/dd/d9/d59/d5c/l9b 0 2024-02-15T18:26:57.501 INFO:tasks.workunit.client.0.smithi042.stdout:9/444: truncate d6/dd/d9/d59/d5c/f7e 934729 0 2024-02-15T18:26:57.506 INFO:tasks.workunit.client.0.smithi042.stdout:1/444: dread de/d15/d67/f3a [0,4194304] 0 2024-02-15T18:26:57.506 INFO:tasks.workunit.client.0.smithi042.stdout:1/445: dread - de/d54/f5e zero size 2024-02-15T18:26:57.508 INFO:tasks.workunit.client.1.smithi191.stdout:2/79: chown l5 197706 1 2024-02-15T18:26:57.509 INFO:tasks.workunit.client.1.smithi191.stdout:3/69: creat d6/f1a x:0 0 0 2024-02-15T18:26:57.509 INFO:tasks.workunit.client.1.smithi191.stdout:3/70: fdatasync d6/de/f11 0 2024-02-15T18:26:57.515 INFO:tasks.workunit.client.1.smithi191.stdout:6/150: unlink d8/d9/l17 0 2024-02-15T18:26:57.516 INFO:tasks.workunit.client.1.smithi191.stdout:0/41: symlink d7/lc 0 2024-02-15T18:26:57.523 INFO:tasks.workunit.client.0.smithi042.stdout:2/377: symlink d0/d35/d64/l75 0 2024-02-15T18:26:57.523 INFO:tasks.workunit.client.0.smithi042.stdout:7/449: link d10/d12/d23/l2c d10/l9b 0 2024-02-15T18:26:57.527 INFO:tasks.workunit.client.1.smithi191.stdout:2/80: dread df/f10 [0,4194304] 0 2024-02-15T18:26:57.531 INFO:tasks.workunit.client.1.smithi191.stdout:4/128: mknod d2/d5/d7/d2f/c33 0 2024-02-15T18:26:57.531 INFO:tasks.workunit.client.1.smithi191.stdout:4/129: fdatasync d2/f4 0 2024-02-15T18:26:57.532 INFO:tasks.workunit.client.0.smithi042.stdout:3/439: symlink d1/l95 0 2024-02-15T18:26:57.533 INFO:tasks.workunit.client.0.smithi042.stdout:6/479: symlink d9/d11/d19/d35/d55/la6 0 2024-02-15T18:26:57.534 INFO:tasks.workunit.client.1.smithi191.stdout:3/71: rename d6/d16 to d6/d1b 0 2024-02-15T18:26:57.535 INFO:tasks.workunit.client.1.smithi191.stdout:2/81: dread fd [0,4194304] 0 2024-02-15T18:26:57.536 INFO:tasks.workunit.client.1.smithi191.stdout:2/82: fsync f7 0 2024-02-15T18:26:57.536 INFO:tasks.workunit.client.1.smithi191.stdout:2/83: write fe [1283855,129473] 0 2024-02-15T18:26:57.536 INFO:tasks.workunit.client.1.smithi191.stdout:6/151: symlink d8/l25 0 2024-02-15T18:26:57.538 INFO:tasks.workunit.client.0.smithi042.stdout:1/446: getdents de/d15 0 2024-02-15T18:26:57.539 INFO:tasks.workunit.client.0.smithi042.stdout:9/445: creat d6/da/d1d/f9c x:0 0 0 2024-02-15T18:26:57.539 INFO:tasks.workunit.client.1.smithi191.stdout:4/130: creat d2/f34 x:0 0 0 2024-02-15T18:26:57.540 INFO:tasks.workunit.client.1.smithi191.stdout:4/131: truncate d2/d5/d7/f15 436618 0 2024-02-15T18:26:57.540 INFO:tasks.workunit.client.1.smithi191.stdout:4/132: stat d2/d5 0 2024-02-15T18:26:57.540 INFO:tasks.workunit.client.0.smithi042.stdout:0/400: dwrite d4/d9/d34/d4e/f87 [0,4194304] 0 2024-02-15T18:26:57.541 INFO:tasks.workunit.client.1.smithi191.stdout:1/112: dwrite d0/f5 [4194304,4194304] 0 2024-02-15T18:26:57.541 INFO:tasks.workunit.client.1.smithi191.stdout:3/72: mkdir d6/d1b/d1c 0 2024-02-15T18:26:57.543 INFO:tasks.workunit.client.1.smithi191.stdout:6/152: creat d8/d19/f26 x:0 0 0 2024-02-15T18:26:57.545 INFO:tasks.workunit.client.1.smithi191.stdout:0/42: dwrite f6 [0,4194304] 0 2024-02-15T18:26:57.545 INFO:tasks.workunit.client.0.smithi042.stdout:7/450: rmdir d10/d49 39 2024-02-15T18:26:57.547 INFO:tasks.workunit.client.0.smithi042.stdout:5/436: dwrite d0/d38/d53/d64/f93 [0,4194304] 0 2024-02-15T18:26:57.551 INFO:tasks.workunit.client.0.smithi042.stdout:5/437: write d0/d12/d65/d74/d52/f9e [300648,5904] 0 2024-02-15T18:26:57.551 INFO:tasks.workunit.client.1.smithi191.stdout:4/133: symlink d2/d5/d7/dc/d30/l35 0 2024-02-15T18:26:57.554 INFO:tasks.workunit.client.1.smithi191.stdout:2/84: dwrite f7 [0,4194304] 0 2024-02-15T18:26:57.557 INFO:tasks.workunit.client.0.smithi042.stdout:5/438: dread d0/fe [0,4194304] 0 2024-02-15T18:26:57.557 INFO:tasks.workunit.client.0.smithi042.stdout:5/439: write d0/d14/f63 [1020644,47374] 0 2024-02-15T18:26:57.558 INFO:tasks.workunit.client.0.smithi042.stdout:2/378: symlink d0/d8/db/d44/l76 0 2024-02-15T18:26:57.562 INFO:tasks.workunit.client.1.smithi191.stdout:1/113: dwrite d0/f4 [0,4194304] 0 2024-02-15T18:26:57.562 INFO:tasks.workunit.client.1.smithi191.stdout:1/114: read - d0/db/fe zero size 2024-02-15T18:26:57.562 INFO:tasks.workunit.client.1.smithi191.stdout:1/115: chown d0/l1e 794323562 1 2024-02-15T18:26:57.565 INFO:tasks.workunit.client.0.smithi042.stdout:6/480: truncate d9/d11/d19/f38 2007856 0 2024-02-15T18:26:57.565 INFO:tasks.workunit.client.0.smithi042.stdout:6/481: fdatasync d9/f17 0 2024-02-15T18:26:57.565 INFO:tasks.workunit.client.1.smithi191.stdout:3/73: rename d6/fb to d6/d1b/f1d 0 2024-02-15T18:26:57.566 INFO:tasks.workunit.client.0.smithi042.stdout:1/447: rmdir de/d15/d95 39 2024-02-15T18:26:57.570 INFO:tasks.workunit.client.1.smithi191.stdout:0/43: getdents d7 0 2024-02-15T18:26:57.571 INFO:tasks.workunit.client.0.smithi042.stdout:0/401: rmdir d4/d9/d34/d57 39 2024-02-15T18:26:57.571 INFO:tasks.workunit.client.0.smithi042.stdout:0/402: dread - d4/d9/d5a/d60/f64 zero size 2024-02-15T18:26:57.571 INFO:tasks.workunit.client.0.smithi042.stdout:0/403: write d4/d9/d5a/f6b [358130,22663] 0 2024-02-15T18:26:57.572 INFO:tasks.workunit.client.0.smithi042.stdout:0/404: dread - d4/d9/d15/f83 zero size 2024-02-15T18:26:57.572 INFO:tasks.workunit.client.0.smithi042.stdout:0/405: fdatasync d4/d9/d15/f78 0 2024-02-15T18:26:57.574 INFO:tasks.workunit.client.0.smithi042.stdout:7/451: mknod d10/d49/d6d/d78/c9c 0 2024-02-15T18:26:57.574 INFO:tasks.workunit.client.0.smithi042.stdout:7/452: chown d10/c4b 0 1 2024-02-15T18:26:57.574 INFO:tasks.workunit.client.0.smithi042.stdout:7/453: write d10/d43/d57/f66 [978501,10170] 0 2024-02-15T18:26:57.579 INFO:tasks.workunit.client.1.smithi191.stdout:4/134: rename d2/d5/d14/c24 to d2/d5/d7/dc/c36 0 2024-02-15T18:26:57.580 INFO:tasks.workunit.client.0.smithi042.stdout:2/379: stat d0/d8/db/de/d1a/c29 0 2024-02-15T18:26:57.582 INFO:tasks.workunit.client.1.smithi191.stdout:3/74: rename d6/c17 to d6/c1e 0 2024-02-15T18:26:57.582 INFO:tasks.workunit.client.1.smithi191.stdout:3/75: chown d6/f1a 110183 1 2024-02-15T18:26:57.582 INFO:tasks.workunit.client.0.smithi042.stdout:6/482: symlink d9/d11/d19/d35/d49/d79/la7 0 2024-02-15T18:26:57.584 INFO:tasks.workunit.client.0.smithi042.stdout:0/406: mkdir d4/d9/d34/d57/d88 0 2024-02-15T18:26:57.584 INFO:tasks.workunit.client.0.smithi042.stdout:0/407: chown d4/d9/d34/l6d 815878909 1 2024-02-15T18:26:57.586 INFO:tasks.workunit.client.1.smithi191.stdout:4/135: symlink d2/l37 0 2024-02-15T18:26:57.587 INFO:tasks.workunit.client.1.smithi191.stdout:1/116: dread d0/f9 [0,4194304] 0 2024-02-15T18:26:57.587 INFO:tasks.workunit.client.1.smithi191.stdout:1/117: truncate d0/db/fe 117359 0 2024-02-15T18:26:57.588 INFO:tasks.workunit.client.1.smithi191.stdout:4/136: dread d2/f2b [0,4194304] 0 2024-02-15T18:26:57.588 INFO:tasks.workunit.client.0.smithi042.stdout:7/454: unlink d10/d12/f8c 0 2024-02-15T18:26:57.589 INFO:tasks.workunit.client.1.smithi191.stdout:3/76: mkdir d6/d1b/d1f 0 2024-02-15T18:26:57.590 INFO:tasks.workunit.client.0.smithi042.stdout:2/380: mknod d0/d8/db/c77 0 2024-02-15T18:26:57.591 INFO:tasks.workunit.client.0.smithi042.stdout:2/381: chown d0/d8/d10/d31/d3a 295 1 2024-02-15T18:26:57.591 INFO:tasks.workunit.client.0.smithi042.stdout:2/382: dread - d0/d8/d10/d31/d3a/d53/f5d zero size 2024-02-15T18:26:57.592 INFO:tasks.workunit.client.0.smithi042.stdout:6/483: creat d9/d11/d19/d35/d49/d79/fa8 x:0 0 0 2024-02-15T18:26:57.592 INFO:tasks.workunit.client.1.smithi191.stdout:2/85: dwrite fb [0,4194304] 0 2024-02-15T18:26:57.592 INFO:tasks.workunit.client.1.smithi191.stdout:2/86: readlink l5 0 2024-02-15T18:26:57.593 INFO:tasks.workunit.client.1.smithi191.stdout:4/137: creat d2/d5/d7/dc/d30/f38 x:0 0 0 2024-02-15T18:26:57.594 INFO:tasks.workunit.client.0.smithi042.stdout:2/383: dread d0/d8/db/de/f14 [0,4194304] 0 2024-02-15T18:26:57.594 INFO:tasks.workunit.client.0.smithi042.stdout:2/384: read - d0/d8/f65 zero size 2024-02-15T18:26:57.594 INFO:tasks.workunit.client.0.smithi042.stdout:0/408: creat d4/d38/d5c/f89 x:0 0 0 2024-02-15T18:26:57.595 INFO:tasks.workunit.client.1.smithi191.stdout:6/153: dwrite f1 [0,4194304] 0 2024-02-15T18:26:57.595 INFO:tasks.workunit.client.1.smithi191.stdout:3/77: symlink d6/d1b/d1c/l20 0 2024-02-15T18:26:57.596 INFO:tasks.workunit.client.0.smithi042.stdout:7/455: mknod d10/c9d 0 2024-02-15T18:26:57.599 INFO:tasks.workunit.client.0.smithi042.stdout:5/440: dwrite d0/d12/d65/d74/d52/f9e [0,4194304] 0 2024-02-15T18:26:57.600 INFO:tasks.workunit.client.1.smithi191.stdout:0/44: truncate f6 2370323 0 2024-02-15T18:26:57.600 INFO:tasks.workunit.client.1.smithi191.stdout:0/45: getdents d7/da 0 2024-02-15T18:26:57.600 INFO:tasks.workunit.client.1.smithi191.stdout:0/46: fdatasync d7/fb 0 2024-02-15T18:26:57.600 INFO:tasks.workunit.client.1.smithi191.stdout:0/47: write d7/fb [654482,8735] 0 2024-02-15T18:26:57.601 INFO:tasks.workunit.client.1.smithi191.stdout:7/73: sync 2024-02-15T18:26:57.601 INFO:tasks.workunit.client.1.smithi191.stdout:5/65: sync 2024-02-15T18:26:57.601 INFO:tasks.workunit.client.1.smithi191.stdout:9/105: sync 2024-02-15T18:26:57.602 INFO:tasks.workunit.client.1.smithi191.stdout:9/106: dread - d0/d7/f1b zero size 2024-02-15T18:26:57.602 INFO:tasks.workunit.client.1.smithi191.stdout:8/216: sync 2024-02-15T18:26:57.605 INFO:tasks.workunit.client.0.smithi042.stdout:1/448: dwrite de/d15/d67/d74/d7e/f94 [0,4194304] 0 2024-02-15T18:26:57.606 INFO:tasks.workunit.client.0.smithi042.stdout:0/409: mknod d4/d9/d34/d4e/d6c/d70/c8a 0 2024-02-15T18:26:57.606 INFO:tasks.workunit.client.1.smithi191.stdout:4/138: dread d2/f3 [0,4194304] 0 2024-02-15T18:26:57.606 INFO:tasks.workunit.client.1.smithi191.stdout:4/139: rename d2 to d2/d5/d7/dc/d30/d39 22 2024-02-15T18:26:57.608 INFO:tasks.workunit.client.1.smithi191.stdout:2/87: creat df/f11 x:0 0 0 2024-02-15T18:26:57.608 INFO:tasks.workunit.client.1.smithi191.stdout:2/88: chown df/f11 355578067 1 2024-02-15T18:26:57.609 INFO:tasks.workunit.client.0.smithi042.stdout:5/441: mkdir d0/d12/d65/d74/d52/da0 0 2024-02-15T18:26:57.611 INFO:tasks.workunit.client.1.smithi191.stdout:1/118: dwrite d0/d8/f18 [0,4194304] 0 2024-02-15T18:26:57.611 INFO:tasks.workunit.client.1.smithi191.stdout:1/119: chown d0/d8/df/d12 3 1 2024-02-15T18:26:57.618 INFO:tasks.workunit.client.1.smithi191.stdout:3/78: write f5 [754210,98333] 0 2024-02-15T18:26:57.618 INFO:tasks.workunit.client.1.smithi191.stdout:0/48: mkdir d7/dd 0 2024-02-15T18:26:57.623 INFO:tasks.workunit.client.1.smithi191.stdout:9/107: creat d0/d7/da/f27 x:0 0 0 2024-02-15T18:26:57.624 INFO:tasks.workunit.client.1.smithi191.stdout:9/108: chown d0/d7/dc 1071383160 1 2024-02-15T18:26:57.624 INFO:tasks.workunit.client.1.smithi191.stdout:9/109: read d0/d7/f21 [2686676,43216] 0 2024-02-15T18:26:57.625 INFO:tasks.workunit.client.1.smithi191.stdout:9/110: fsync d0/d7/dc/f15 0 2024-02-15T18:26:57.625 INFO:tasks.workunit.client.0.smithi042.stdout:7/456: dwrite d10/d12/d23/f3b [0,4194304] 0 2024-02-15T18:26:57.625 INFO:tasks.workunit.client.0.smithi042.stdout:6/484: dwrite d9/d11/d19/d1c/d31/f6f [0,4194304] 0 2024-02-15T18:26:57.625 INFO:tasks.workunit.client.0.smithi042.stdout:6/485: write d9/f17 [5137384,89792] 0 2024-02-15T18:26:57.626 INFO:tasks.workunit.client.1.smithi191.stdout:4/140: creat d2/d5/d14/f3a x:0 0 0 2024-02-15T18:26:57.628 INFO:tasks.workunit.client.1.smithi191.stdout:2/89: mkdir df/d12 0 2024-02-15T18:26:57.629 INFO:tasks.workunit.client.1.smithi191.stdout:6/154: mknod d8/d9/c27 0 2024-02-15T18:26:57.630 INFO:tasks.workunit.client.0.smithi042.stdout:0/410: dwrite d4/d38/d39/f6e [0,4194304] 0 2024-02-15T18:26:57.632 INFO:tasks.workunit.client.0.smithi042.stdout:7/457: rename d10/f2f to d10/d12/f9e 0 2024-02-15T18:26:57.634 INFO:tasks.workunit.client.1.smithi191.stdout:0/49: dread f6 [0,4194304] 0 2024-02-15T18:26:57.636 INFO:tasks.workunit.client.1.smithi191.stdout:1/120: creat d0/db/d10/f25 x:0 0 0 2024-02-15T18:26:57.637 INFO:tasks.workunit.client.1.smithi191.stdout:1/121: dread d0/d8/fd [0,4194304] 0 2024-02-15T18:26:57.638 INFO:tasks.workunit.client.1.smithi191.stdout:1/122: write d0/d8/fd [592866,49409] 0 2024-02-15T18:26:57.639 INFO:tasks.workunit.client.1.smithi191.stdout:3/79: mkdir d6/d1b/d1f/d21 0 2024-02-15T18:26:57.639 INFO:tasks.workunit.client.0.smithi042.stdout:6/486: mknod d9/d11/ca9 0 2024-02-15T18:26:57.640 INFO:tasks.workunit.client.1.smithi191.stdout:9/111: creat d0/d7/f28 x:0 0 0 2024-02-15T18:26:57.641 INFO:tasks.workunit.client.1.smithi191.stdout:9/112: truncate d0/d7/dc/f15 29874 0 2024-02-15T18:26:57.643 INFO:tasks.workunit.client.1.smithi191.stdout:4/141: unlink d2/le 0 2024-02-15T18:26:57.644 INFO:tasks.workunit.client.0.smithi042.stdout:0/411: mkdir d4/d9/d34/d8b 0 2024-02-15T18:26:57.645 INFO:tasks.workunit.client.1.smithi191.stdout:9/113: write d0/d7/f21 [3507271,41415] 0 2024-02-15T18:26:57.646 INFO:tasks.workunit.client.1.smithi191.stdout:2/90: mkdir df/d12/d13 0 2024-02-15T18:26:57.647 INFO:tasks.workunit.client.0.smithi042.stdout:6/487: rename d9/d11/f20 to d9/d11/d19/d35/d55/d92/d9e/faa 0 2024-02-15T18:26:57.648 INFO:tasks.workunit.client.0.smithi042.stdout:6/488: chown d9/f2f 7 1 2024-02-15T18:26:57.649 INFO:tasks.workunit.client.1.smithi191.stdout:0/50: symlink d7/da/le 0 2024-02-15T18:26:57.651 INFO:tasks.workunit.client.0.smithi042.stdout:6/489: rename d9/d11/d19/f38 to d9/d11/d46/d89/fab 0 2024-02-15T18:26:57.652 INFO:tasks.workunit.client.0.smithi042.stdout:0/412: dread d4/d9/f46 [0,4194304] 0 2024-02-15T18:26:57.652 INFO:tasks.workunit.client.0.smithi042.stdout:0/413: write d4/d38/f3c [6570316,38230] 0 2024-02-15T18:26:57.653 INFO:tasks.workunit.client.1.smithi191.stdout:0/51: write f6 [1513959,81740] 0 2024-02-15T18:26:57.653 INFO:tasks.workunit.client.1.smithi191.stdout:0/52: fsync d7/fb 0 2024-02-15T18:26:57.654 INFO:tasks.workunit.client.1.smithi191.stdout:3/80: unlink f3 0 2024-02-15T18:26:57.656 INFO:tasks.workunit.client.1.smithi191.stdout:4/142: creat d2/d5/d7/f3b x:0 0 0 2024-02-15T18:26:57.656 INFO:tasks.workunit.client.1.smithi191.stdout:4/143: chown d2/f2c 486067042 1 2024-02-15T18:26:57.656 INFO:tasks.workunit.client.0.smithi042.stdout:6/490: truncate d9/f2d 4697974 0 2024-02-15T18:26:57.657 INFO:tasks.workunit.client.1.smithi191.stdout:3/81: dread d6/de/f13 [0,4194304] 0 2024-02-15T18:26:57.658 INFO:tasks.workunit.client.0.smithi042.stdout:0/414: rename d4/d9/d15/f42 to d4/d9/d15/f8c 0 2024-02-15T18:26:57.660 INFO:tasks.workunit.client.1.smithi191.stdout:9/114: mkdir d0/d29 0 2024-02-15T18:26:57.660 INFO:tasks.workunit.client.1.smithi191.stdout:9/115: stat d0/d7/f1b 0 2024-02-15T18:26:57.660 INFO:tasks.workunit.client.1.smithi191.stdout:9/116: getdents d0/d29 0 2024-02-15T18:26:57.660 INFO:tasks.workunit.client.1.smithi191.stdout:9/117: fdatasync d0/d7/ff 0 2024-02-15T18:26:57.662 INFO:tasks.workunit.client.1.smithi191.stdout:6/155: symlink d8/d19/l28 0 2024-02-15T18:26:57.666 INFO:tasks.workunit.client.1.smithi191.stdout:3/82: symlink d6/d1b/d1c/l22 0 2024-02-15T18:26:57.666 INFO:tasks.workunit.client.1.smithi191.stdout:9/118: mknod d0/c2a 0 2024-02-15T18:26:57.666 INFO:tasks.workunit.client.1.smithi191.stdout:9/119: chown d0/d7/f11 88 1 2024-02-15T18:26:57.666 INFO:tasks.workunit.client.1.smithi191.stdout:9/120: write d0/d7/da/f27 [298420,113952] 0 2024-02-15T18:26:57.666 INFO:tasks.workunit.client.1.smithi191.stdout:9/121: chown d0/d7/f9 4057 1 2024-02-15T18:26:57.666 INFO:tasks.workunit.client.1.smithi191.stdout:9/122: readlink d0/l16 0 2024-02-15T18:26:57.668 INFO:tasks.workunit.client.1.smithi191.stdout:9/123: creat d0/d7/dc/f2b x:0 0 0 2024-02-15T18:26:57.668 INFO:tasks.workunit.client.1.smithi191.stdout:9/124: chown d0/c1a 967043 1 2024-02-15T18:26:57.670 INFO:tasks.workunit.client.1.smithi191.stdout:9/125: rename d0/d7/ff to d0/d1d/f2c 0 2024-02-15T18:26:57.673 INFO:tasks.workunit.client.0.smithi042.stdout:5/442: dwrite d0/d38/d53/d64/f9f [0,4194304] 0 2024-02-15T18:26:57.673 INFO:tasks.workunit.client.0.smithi042.stdout:5/443: chown d0/d22/d7b/f66 2108 1 2024-02-15T18:26:57.677 INFO:tasks.workunit.client.0.smithi042.stdout:5/444: write d0/db/f1f [1640086,130242] 0 2024-02-15T18:26:57.677 INFO:tasks.workunit.client.0.smithi042.stdout:5/445: stat d0/d38/d53/d68/d6a/d42/d7d/c92 0 2024-02-15T18:26:57.679 INFO:tasks.workunit.client.1.smithi191.stdout:8/217: dwrite d3/d2c/d1a/f1b [4194304,4194304] 0 2024-02-15T18:26:57.682 INFO:tasks.workunit.client.1.smithi191.stdout:8/218: creat d3/d2c/d1a/d23/d2e/f4b x:0 0 0 2024-02-15T18:26:57.682 INFO:tasks.workunit.client.1.smithi191.stdout:8/219: readlink d3/l30 0 2024-02-15T18:26:57.682 INFO:tasks.workunit.client.1.smithi191.stdout:8/220: dread - d3/d2c/f43 zero size 2024-02-15T18:26:57.682 INFO:tasks.workunit.client.1.smithi191.stdout:8/221: truncate d3/d2c/d1a/d23/d2e/f4b 155932 0 2024-02-15T18:26:57.682 INFO:tasks.workunit.client.1.smithi191.stdout:8/222: stat d3/d2c/d1a/d23/d2e/d38 0 2024-02-15T18:26:57.682 INFO:tasks.workunit.client.0.smithi042.stdout:5/446: mkdir d0/d14/d88/da1 0 2024-02-15T18:26:57.692 INFO:tasks.workunit.client.1.smithi191.stdout:8/223: dread d3/d2c/d1a/d23/d2e/f3a [0,4194304] 0 2024-02-15T18:26:57.696 INFO:tasks.workunit.client.1.smithi191.stdout:8/224: write d3/f15 [2510772,106047] 0 2024-02-15T18:26:57.696 INFO:tasks.workunit.client.0.smithi042.stdout:5/447: dwrite d0/db/f10 [0,4194304] 0 2024-02-15T18:26:57.696 INFO:tasks.workunit.client.0.smithi042.stdout:5/448: chown d0/d22/d7b 20168 1 2024-02-15T18:26:57.697 INFO:tasks.workunit.client.1.smithi191.stdout:8/225: creat d3/d2c/d1a/d23/f4c x:0 0 0 2024-02-15T18:26:57.700 INFO:tasks.workunit.client.0.smithi042.stdout:5/449: dread d0/d38/d53/d64/f81 [0,4194304] 0 2024-02-15T18:26:57.700 INFO:tasks.workunit.client.0.smithi042.stdout:5/450: mknod d0/d12/ca2 0 2024-02-15T18:26:57.703 INFO:tasks.workunit.client.0.smithi042.stdout:7/458: dwrite d10/d12/d23/f55 [0,4194304] 0 2024-02-15T18:26:57.703 INFO:tasks.workunit.client.0.smithi042.stdout:1/449: dwrite de/d15/d95/f99 [0,4194304] 0 2024-02-15T18:26:57.703 INFO:tasks.workunit.client.1.smithi191.stdout:2/91: dwrite fd [0,4194304] 0 2024-02-15T18:26:57.704 INFO:tasks.workunit.client.0.smithi042.stdout:6/491: dwrite d9/d11/d19/f5a [0,4194304] 0 2024-02-15T18:26:57.706 INFO:tasks.workunit.client.1.smithi191.stdout:6/156: dwrite f6 [8388608,4194304] 0 2024-02-15T18:26:57.707 INFO:tasks.workunit.client.1.smithi191.stdout:3/83: dwrite d6/d1b/f1d [0,4194304] 0 2024-02-15T18:26:57.708 INFO:tasks.workunit.client.1.smithi191.stdout:4/144: dwrite d2/d5/d7/dc/d30/f38 [0,4194304] 0 2024-02-15T18:26:57.708 INFO:tasks.workunit.client.1.smithi191.stdout:4/145: dread - d2/d5/d14/f2e zero size 2024-02-15T18:26:57.708 INFO:tasks.workunit.client.1.smithi191.stdout:4/146: readlink d2/l37 0 2024-02-15T18:26:57.708 INFO:tasks.workunit.client.1.smithi191.stdout:4/147: stat d2/d5 0 2024-02-15T18:26:57.711 INFO:tasks.workunit.client.1.smithi191.stdout:4/148: dread d2/d5/d7/f18 [0,4194304] 0 2024-02-15T18:26:57.711 INFO:tasks.workunit.client.1.smithi191.stdout:4/149: chown d2/d5/l11 60060 1 2024-02-15T18:26:57.712 INFO:tasks.workunit.client.1.smithi191.stdout:4/150: write d2/f3 [2450774,120196] 0 2024-02-15T18:26:57.721 INFO:tasks.workunit.client.0.smithi042.stdout:0/415: fsync d4/f41 0 2024-02-15T18:26:57.721 INFO:tasks.workunit.client.0.smithi042.stdout:0/416: stat d4/c23 0 2024-02-15T18:26:57.721 INFO:tasks.workunit.client.0.smithi042.stdout:0/417: write d4/d9/d34/d57/f7f [461950,26010] 0 2024-02-15T18:26:57.721 INFO:tasks.workunit.client.1.smithi191.stdout:0/53: truncate f6 513474 0 2024-02-15T18:26:57.722 INFO:tasks.workunit.client.1.smithi191.stdout:0/54: truncate d7/fb 728021 0 2024-02-15T18:26:57.726 INFO:tasks.workunit.client.1.smithi191.stdout:8/226: write d3/d2c/d1a/f45 [388731,52297] 0 2024-02-15T18:26:57.727 INFO:tasks.workunit.client.1.smithi191.stdout:9/126: rmdir d0/d7 39 2024-02-15T18:26:57.728 INFO:tasks.workunit.client.0.smithi042.stdout:0/418: dread d4/f3b [0,4194304] 0 2024-02-15T18:26:57.730 INFO:tasks.workunit.client.1.smithi191.stdout:6/157: creat d8/d9/f29 x:0 0 0 2024-02-15T18:26:57.730 INFO:tasks.workunit.client.1.smithi191.stdout:9/127: write d0/f4 [6975019,89864] 0 2024-02-15T18:26:57.730 INFO:tasks.workunit.client.1.smithi191.stdout:9/128: write d0/d7/da/f1c [224574,38470] 0 2024-02-15T18:26:57.733 INFO:tasks.workunit.client.1.smithi191.stdout:3/84: truncate d6/de/f11 1404843 0 2024-02-15T18:26:57.733 INFO:tasks.workunit.client.1.smithi191.stdout:9/129: write d0/f4 [5039021,124408] 0 2024-02-15T18:26:57.734 INFO:tasks.workunit.client.1.smithi191.stdout:6/158: write d8/f12 [1412104,79063] 0 2024-02-15T18:26:57.736 INFO:tasks.workunit.client.1.smithi191.stdout:9/130: write d0/d1d/f2c [1963137,68041] 0 2024-02-15T18:26:57.736 INFO:tasks.workunit.client.1.smithi191.stdout:4/151: rename d2/d5/d7/dc/d30/l35 to d2/d5/d14/l3c 0 2024-02-15T18:26:57.738 INFO:tasks.workunit.client.1.smithi191.stdout:8/227: mknod d3/d2c/d1a/d23/d2e/c4d 0 2024-02-15T18:26:57.738 INFO:tasks.workunit.client.1.smithi191.stdout:8/228: chown d3/cb 23 1 2024-02-15T18:26:57.740 INFO:tasks.workunit.client.1.smithi191.stdout:3/85: unlink d6/c9 0 2024-02-15T18:26:57.741 INFO:tasks.workunit.client.0.smithi042.stdout:0/419: mkdir d4/d9/d8d 0 2024-02-15T18:26:57.741 INFO:tasks.workunit.client.0.smithi042.stdout:0/420: chown d4/d38/d77/d2e/f32 1961 1 2024-02-15T18:26:57.743 INFO:tasks.workunit.client.1.smithi191.stdout:9/131: creat d0/d7/dc/d17/f2d x:0 0 0 2024-02-15T18:26:57.746 INFO:tasks.workunit.client.0.smithi042.stdout:0/421: rename d4/d9/d34/d4d to d4/d38/d39/d8e 0 2024-02-15T18:26:57.746 INFO:tasks.workunit.client.0.smithi042.stdout:0/422: read - d4/f4f zero size 2024-02-15T18:26:57.747 INFO:tasks.workunit.client.1.smithi191.stdout:8/229: mknod d3/d2c/d1a/d23/d2e/c4e 0 2024-02-15T18:26:57.747 INFO:tasks.workunit.client.1.smithi191.stdout:8/230: chown d3/d2c/d1a/f44 6240044 1 2024-02-15T18:26:57.747 INFO:tasks.workunit.client.1.smithi191.stdout:8/231: write d3/f31 [621478,77661] 0 2024-02-15T18:26:57.747 INFO:tasks.workunit.client.1.smithi191.stdout:8/232: getdents d3/d2c/d1a/d36 0 2024-02-15T18:26:57.748 INFO:tasks.workunit.client.1.smithi191.stdout:8/233: readlink d3/le 0 2024-02-15T18:26:57.748 INFO:tasks.workunit.client.1.smithi191.stdout:3/86: getdents d6/d1b/d1c 0 2024-02-15T18:26:57.748 INFO:tasks.workunit.client.1.smithi191.stdout:9/132: symlink d0/l2e 0 2024-02-15T18:26:57.750 INFO:tasks.workunit.client.1.smithi191.stdout:8/234: mkdir d3/d2c/d1a/d23/d2e/d38/d4f 0 2024-02-15T18:26:57.750 INFO:tasks.workunit.client.1.smithi191.stdout:8/235: fsync d3/f18 0 2024-02-15T18:26:57.751 INFO:tasks.workunit.client.1.smithi191.stdout:3/87: mknod d6/d1b/c23 0 2024-02-15T18:26:57.751 INFO:tasks.workunit.client.1.smithi191.stdout:3/88: truncate d6/de/ff 646271 0 2024-02-15T18:26:57.751 INFO:tasks.workunit.client.1.smithi191.stdout:3/89: chown d6/fa 49800243 1 2024-02-15T18:26:57.751 INFO:tasks.workunit.client.1.smithi191.stdout:3/90: getdents d6/de 0 2024-02-15T18:26:57.751 INFO:tasks.workunit.client.1.smithi191.stdout:9/133: mknod d0/d1d/c2f 0 2024-02-15T18:26:57.753 INFO:tasks.workunit.client.1.smithi191.stdout:8/236: rename d3/f39 to d3/d2c/d1a/d23/d2e/d38/f50 0 2024-02-15T18:26:57.754 INFO:tasks.workunit.client.1.smithi191.stdout:8/237: creat d3/d2c/d1a/f51 x:0 0 0 2024-02-15T18:26:57.758 INFO:tasks.workunit.client.0.smithi042.stdout:7/459: dwrite d10/f69 [0,4194304] 0 2024-02-15T18:26:57.758 INFO:tasks.workunit.client.0.smithi042.stdout:7/460: stat d10/d12/d23/d51/c95 0 2024-02-15T18:26:57.763 INFO:tasks.workunit.client.1.smithi191.stdout:2/92: dwrite fd [0,4194304] 0 2024-02-15T18:26:57.763 INFO:tasks.workunit.client.1.smithi191.stdout:2/93: truncate df/f11 194702 0 2024-02-15T18:26:57.766 INFO:tasks.workunit.client.0.smithi042.stdout:7/461: creat d10/d15/d8f/f9f x:0 0 0 2024-02-15T18:26:57.767 INFO:tasks.workunit.client.0.smithi042.stdout:5/451: dwrite d0/d38/d53/d64/f9f [0,4194304] 0 2024-02-15T18:26:57.770 INFO:tasks.workunit.client.1.smithi191.stdout:8/238: dread d3/f18 [0,4194304] 0 2024-02-15T18:26:57.770 INFO:tasks.workunit.client.0.smithi042.stdout:5/452: getdents d0/d12 0 2024-02-15T18:26:57.770 INFO:tasks.workunit.client.0.smithi042.stdout:5/453: readlink d0/d12/d65/d74/d52/l7f 0 2024-02-15T18:26:57.771 INFO:tasks.workunit.client.0.smithi042.stdout:6/492: dwrite d9/d11/d19/d35/f61 [0,4194304] 0 2024-02-15T18:26:57.771 INFO:tasks.workunit.client.0.smithi042.stdout:6/493: dread - d9/d11/d19/d35/d49/d79/fa8 zero size 2024-02-15T18:26:57.771 INFO:tasks.workunit.client.0.smithi042.stdout:6/494: stat d9/d11/d19/d1c/d31/f3a 0 2024-02-15T18:26:57.771 INFO:tasks.workunit.client.0.smithi042.stdout:1/450: dwrite de/d54/f5e [0,4194304] 0 2024-02-15T18:26:57.778 INFO:tasks.workunit.client.0.smithi042.stdout:8/342: sync 2024-02-15T18:26:57.779 INFO:tasks.workunit.client.0.smithi042.stdout:3/440: sync 2024-02-15T18:26:57.779 INFO:tasks.workunit.client.0.smithi042.stdout:4/405: sync 2024-02-15T18:26:57.779 INFO:tasks.workunit.client.0.smithi042.stdout:4/406: chown da/d11/d67/d5c/d66/d78/d2f/f34 2661224 1 2024-02-15T18:26:57.781 INFO:tasks.workunit.client.1.smithi191.stdout:3/91: unlink d6/de/f11 0 2024-02-15T18:26:57.785 INFO:tasks.workunit.client.0.smithi042.stdout:6/495: link d9/d11/d19/d35/d55/d92/d9e/faa d9/d11/d19/d35/d55/fac 0 2024-02-15T18:26:57.786 INFO:tasks.workunit.client.0.smithi042.stdout:8/343: dread fc [0,4194304] 0 2024-02-15T18:26:57.786 INFO:tasks.workunit.client.0.smithi042.stdout:8/344: truncate dd/de/d33/f44 416951 0 2024-02-15T18:26:57.787 INFO:tasks.workunit.client.0.smithi042.stdout:1/451: creat de/d6d/fa3 x:0 0 0 2024-02-15T18:26:57.788 INFO:tasks.workunit.client.0.smithi042.stdout:3/441: mknod d1/c96 0 2024-02-15T18:26:57.789 INFO:tasks.workunit.client.0.smithi042.stdout:3/442: dread - d1/d11/f8f zero size 2024-02-15T18:26:57.789 INFO:tasks.workunit.client.0.smithi042.stdout:3/443: stat d1/d11/d2e/d4c/f76 0 2024-02-15T18:26:57.789 INFO:tasks.workunit.client.0.smithi042.stdout:3/444: write d1/d11/d14/d4d/d73/f87 [50088,59846] 0 2024-02-15T18:26:57.790 INFO:tasks.workunit.client.0.smithi042.stdout:4/407: creat da/d11/d67/d5c/f8b x:0 0 0 2024-02-15T18:26:57.790 INFO:tasks.workunit.client.0.smithi042.stdout:6/496: creat d9/d11/d19/d1c/fad x:0 0 0 2024-02-15T18:26:57.791 INFO:tasks.workunit.client.0.smithi042.stdout:5/454: dread d0/db/fd [0,4194304] 0 2024-02-15T18:26:57.791 INFO:tasks.workunit.client.0.smithi042.stdout:5/455: write d0/d50/f5f [1658921,70817] 0 2024-02-15T18:26:57.791 INFO:tasks.workunit.client.0.smithi042.stdout:8/345: truncate dd/de/d22/f2d 68475 0 2024-02-15T18:26:57.792 INFO:tasks.workunit.client.0.smithi042.stdout:8/346: readlink dd/de/l5e 0 2024-02-15T18:26:57.792 INFO:tasks.workunit.client.0.smithi042.stdout:4/408: mknod da/d3b/c8c 0 2024-02-15T18:26:57.792 INFO:tasks.workunit.client.0.smithi042.stdout:1/452: write de/d15/d67/d74/d7e/f94 [1876305,69391] 0 2024-02-15T18:26:57.793 INFO:tasks.workunit.client.1.smithi191.stdout:4/152: dwrite d2/d5/d7/f18 [0,4194304] 0 2024-02-15T18:26:57.793 INFO:tasks.workunit.client.1.smithi191.stdout:4/153: truncate d2/f2b 714424 0 2024-02-15T18:26:57.793 INFO:tasks.workunit.client.0.smithi042.stdout:6/497: link d9/d11/d19/d1c/d31/d4f/l5f d9/d8d/lae 0 2024-02-15T18:26:57.793 INFO:tasks.workunit.client.0.smithi042.stdout:6/498: truncate d9/d11/d19/d35/f80 1706015 0 2024-02-15T18:26:57.794 INFO:tasks.workunit.client.0.smithi042.stdout:6/499: write d9/d11/d19/d35/d49/f52 [3950096,9660] 0 2024-02-15T18:26:57.794 INFO:tasks.workunit.client.0.smithi042.stdout:6/500: chown d9/d11/d19/f99 9190 1 2024-02-15T18:26:57.794 INFO:tasks.workunit.client.0.smithi042.stdout:5/456: rename d0/d38/d53/f57 to d0/d14/fa3 0 2024-02-15T18:26:57.795 INFO:tasks.workunit.client.0.smithi042.stdout:6/501: write d9/d11/d19/d35/d55/fac [49856,57898] 0 2024-02-15T18:26:57.795 INFO:tasks.workunit.client.0.smithi042.stdout:6/502: chown d9/d11/d19/d1c/d31/d4f/f68 3621 1 2024-02-15T18:26:57.796 INFO:tasks.workunit.client.1.smithi191.stdout:9/134: dread d0/d7/da/f1c [0,4194304] 0 2024-02-15T18:26:57.798 INFO:tasks.workunit.client.0.smithi042.stdout:4/409: mkdir da/d3b/d41/d8d 0 2024-02-15T18:26:57.804 INFO:tasks.workunit.client.0.smithi042.stdout:5/457: dread d0/f23 [0,4194304] 0 2024-02-15T18:26:57.807 INFO:tasks.workunit.client.0.smithi042.stdout:1/453: mknod de/d6d/d9e/ca4 0 2024-02-15T18:26:57.808 INFO:tasks.workunit.client.1.smithi191.stdout:9/135: mkdir d0/d7/dc/d30 0 2024-02-15T18:26:57.809 INFO:tasks.workunit.client.1.smithi191.stdout:4/154: symlink d2/d5/l3d 0 2024-02-15T18:26:57.810 INFO:tasks.workunit.client.0.smithi042.stdout:5/458: mknod d0/d38/d53/d91/ca4 0 2024-02-15T18:26:57.811 INFO:tasks.workunit.client.0.smithi042.stdout:5/459: stat d0/d22/d7b 0 2024-02-15T18:26:57.811 INFO:tasks.workunit.client.0.smithi042.stdout:5/460: chown d0/d14/f9c 413805 1 2024-02-15T18:26:57.812 INFO:tasks.workunit.client.1.smithi191.stdout:9/136: link d0/d1d/c2f d0/d7/dc/d17/c31 0 2024-02-15T18:26:57.813 INFO:tasks.workunit.client.0.smithi042.stdout:6/503: dread d9/d3e/f75 [0,4194304] 0 2024-02-15T18:26:57.813 INFO:tasks.workunit.client.1.smithi191.stdout:4/155: mkdir d2/d5/d7/d3e 0 2024-02-15T18:26:57.814 INFO:tasks.workunit.client.0.smithi042.stdout:5/461: mknod d0/d38/d53/d91/ca5 0 2024-02-15T18:26:57.815 INFO:tasks.workunit.client.0.smithi042.stdout:1/454: rename de/d15/d67/d63/c77 to de/d15/d1b/ca5 0 2024-02-15T18:26:57.816 INFO:tasks.workunit.client.0.smithi042.stdout:0/423: dwrite d4/d38/d39/d8e/f67 [0,4194304] 0 2024-02-15T18:26:57.816 INFO:tasks.workunit.client.1.smithi191.stdout:0/55: dwrite f6 [0,4194304] 0 2024-02-15T18:26:57.817 INFO:tasks.workunit.client.0.smithi042.stdout:6/504: read d9/d11/d19/f93 [2263229,69323] 0 2024-02-15T18:26:57.817 INFO:tasks.workunit.client.0.smithi042.stdout:0/424: write d4/d9/f46 [2006417,56008] 0 2024-02-15T18:26:57.817 INFO:tasks.workunit.client.0.smithi042.stdout:5/462: unlink d0/d12/d65/d74/d52/l59 0 2024-02-15T18:26:57.818 INFO:tasks.workunit.client.1.smithi191.stdout:4/156: symlink d2/d5/d7/d2f/l3f 0 2024-02-15T18:26:57.819 INFO:tasks.workunit.client.0.smithi042.stdout:1/455: creat de/d15/d95/fa6 x:0 0 0 2024-02-15T18:26:57.820 INFO:tasks.workunit.client.1.smithi191.stdout:0/56: write f6 [3205161,20077] 0 2024-02-15T18:26:57.822 INFO:tasks.workunit.client.0.smithi042.stdout:6/505: symlink d9/d11/d19/d35/d49/d79/laf 0 2024-02-15T18:26:57.822 INFO:tasks.workunit.client.0.smithi042.stdout:6/506: stat d9/f2d 0 2024-02-15T18:26:57.825 INFO:tasks.workunit.client.0.smithi042.stdout:0/425: mkdir d4/d38/d39/d8e/d8f 0 2024-02-15T18:26:57.826 INFO:tasks.workunit.client.0.smithi042.stdout:5/463: dread d0/d14/f77 [0,4194304] 0 2024-02-15T18:26:57.826 INFO:tasks.workunit.client.0.smithi042.stdout:5/464: truncate d0/d39/f96 860414 0 2024-02-15T18:26:57.826 INFO:tasks.workunit.client.0.smithi042.stdout:5/465: chown d0/d14/f87 3406 1 2024-02-15T18:26:57.827 INFO:tasks.workunit.client.0.smithi042.stdout:1/456: creat de/d15/d67/d46/d6b/fa7 x:0 0 0 2024-02-15T18:26:57.828 INFO:tasks.workunit.client.0.smithi042.stdout:6/507: creat d9/d11/d1d/fb0 x:0 0 0 2024-02-15T18:26:57.830 INFO:tasks.workunit.client.0.smithi042.stdout:0/426: creat d4/d9/d34/d8b/f90 x:0 0 0 2024-02-15T18:26:57.830 INFO:tasks.workunit.client.0.smithi042.stdout:0/427: write d4/d38/d77/f7c [848997,20007] 0 2024-02-15T18:26:57.831 INFO:tasks.workunit.client.0.smithi042.stdout:5/466: mknod d0/d12/d2a/ca6 0 2024-02-15T18:26:57.831 INFO:tasks.workunit.client.0.smithi042.stdout:5/467: chown d0 267914 1 2024-02-15T18:26:57.831 INFO:tasks.workunit.client.0.smithi042.stdout:5/468: fsync d0/d12/f21 0 2024-02-15T18:26:57.832 INFO:tasks.workunit.client.0.smithi042.stdout:6/508: truncate d9/f76 3863355 0 2024-02-15T18:26:57.832 INFO:tasks.workunit.client.0.smithi042.stdout:6/509: truncate d9/d11/d19/d1c/d31/d4f/f68 380681 0 2024-02-15T18:26:57.833 INFO:tasks.workunit.client.1.smithi191.stdout:6/159: dwrite d8/f12 [4194304,4194304] 0 2024-02-15T18:26:57.833 INFO:tasks.workunit.client.1.smithi191.stdout:3/92: dwrite d6/de/f13 [0,4194304] 0 2024-02-15T18:26:57.833 INFO:tasks.workunit.client.0.smithi042.stdout:7/462: dwrite d10/d12/f3f [0,4194304] 0 2024-02-15T18:26:57.833 INFO:tasks.workunit.client.0.smithi042.stdout:0/428: mknod d4/c91 0 2024-02-15T18:26:57.835 INFO:tasks.workunit.client.0.smithi042.stdout:8/347: dwrite dd/de/d12/d32/f52 [0,4194304] 0 2024-02-15T18:26:57.835 INFO:tasks.workunit.client.0.smithi042.stdout:8/348: fsync dd/de/d33/f40 0 2024-02-15T18:26:57.835 INFO:tasks.workunit.client.0.smithi042.stdout:8/349: readlink l8 0 2024-02-15T18:26:57.835 INFO:tasks.workunit.client.0.smithi042.stdout:8/350: dread - dd/de/d12/d32/f5b zero size 2024-02-15T18:26:57.835 INFO:tasks.workunit.client.0.smithi042.stdout:3/445: dwrite d1/d11/d2e/f37 [4194304,4194304] 0 2024-02-15T18:26:57.836 INFO:tasks.workunit.client.0.smithi042.stdout:4/410: dwrite da/d11/d67/d5c/d66/d78/d2f/d4d/f48 [0,4194304] 0 2024-02-15T18:26:57.836 INFO:tasks.workunit.client.0.smithi042.stdout:6/510: unlink d9/d11/d19/d35/d49/la5 0 2024-02-15T18:26:57.837 INFO:tasks.workunit.client.0.smithi042.stdout:5/469: write d0/d14/fa3 [1871128,27557] 0 2024-02-15T18:26:57.837 INFO:tasks.workunit.client.0.smithi042.stdout:5/470: write d0/db/f10 [4817354,69968] 0 2024-02-15T18:26:57.842 INFO:tasks.workunit.client.0.smithi042.stdout:0/429: mkdir d4/d9/d15/d92 0 2024-02-15T18:26:57.842 INFO:tasks.workunit.client.0.smithi042.stdout:9/446: sync 2024-02-15T18:26:57.842 INFO:tasks.workunit.client.0.smithi042.stdout:2/385: sync 2024-02-15T18:26:57.845 INFO:tasks.workunit.client.0.smithi042.stdout:2/386: dread d0/d8/d10/d31/d3a/d53/f45 [0,4194304] 0 2024-02-15T18:26:57.845 INFO:tasks.workunit.client.0.smithi042.stdout:2/387: stat d0/d8/d10/f54 0 2024-02-15T18:26:57.845 INFO:tasks.workunit.client.0.smithi042.stdout:2/388: write d0/d8/f39 [798007,15883] 0 2024-02-15T18:26:57.856 INFO:tasks.workunit.client.0.smithi042.stdout:7/463: dwrite d10/d43/d57/f5e [0,4194304] 0 2024-02-15T18:26:57.863 INFO:tasks.workunit.client.1.smithi191.stdout:3/93: dwrite d6/fd [0,4194304] 0 2024-02-15T18:26:57.863 INFO:tasks.workunit.client.0.smithi042.stdout:1/457: dwrite de/d15/d67/d63/f69 [0,4194304] 0 2024-02-15T18:26:57.864 INFO:tasks.workunit.client.0.smithi042.stdout:9/447: dwrite d6/f21 [0,4194304] 0 2024-02-15T18:26:57.864 INFO:tasks.workunit.client.0.smithi042.stdout:9/448: write d6/f2e [151097,8162] 0 2024-02-15T18:26:57.865 INFO:tasks.workunit.client.0.smithi042.stdout:8/351: getdents dd/de/d22 0 2024-02-15T18:26:57.865 INFO:tasks.workunit.client.0.smithi042.stdout:8/352: chown dd/de/c57 2546430 1 2024-02-15T18:26:57.868 INFO:tasks.workunit.client.0.smithi042.stdout:0/430: dwrite d4/d38/d5c/f89 [0,4194304] 0 2024-02-15T18:26:57.868 INFO:tasks.workunit.client.0.smithi042.stdout:0/431: write d4/d9/f46 [4722240,68219] 0 2024-02-15T18:26:57.870 INFO:tasks.workunit.client.0.smithi042.stdout:3/446: getdents d1/de/d1a 0 2024-02-15T18:26:57.871 INFO:tasks.workunit.client.1.smithi191.stdout:0/57: unlink f6 0 2024-02-15T18:26:57.874 INFO:tasks.workunit.client.1.smithi191.stdout:6/160: chown d8/d19/l1b 1993 1 2024-02-15T18:26:57.875 INFO:tasks.workunit.client.0.smithi042.stdout:4/411: creat da/d11/d29/d4a/d4f/f8e x:0 0 0 2024-02-15T18:26:57.875 INFO:tasks.workunit.client.0.smithi042.stdout:4/412: getdents da/d2b/d6e 0 2024-02-15T18:26:57.878 INFO:tasks.workunit.client.0.smithi042.stdout:6/511: rename d9/d3e/l30 to d9/d11/d19/d35/d55/lb1 0 2024-02-15T18:26:57.880 INFO:tasks.workunit.client.0.smithi042.stdout:5/471: creat d0/d12/d65/d74/d52/fa7 x:0 0 0 2024-02-15T18:26:57.880 INFO:tasks.workunit.client.1.smithi191.stdout:6/161: dread d8/f1c [0,4194304] 0 2024-02-15T18:26:57.880 INFO:tasks.workunit.client.1.smithi191.stdout:6/162: readlink l2 0 2024-02-15T18:26:57.881 INFO:tasks.workunit.client.0.smithi042.stdout:2/389: mknod d0/d35/c78 0 2024-02-15T18:26:57.884 INFO:tasks.workunit.client.1.smithi191.stdout:3/94: unlink d6/d1b/d1c/l20 0 2024-02-15T18:26:57.886 INFO:tasks.workunit.client.0.smithi042.stdout:7/464: mknod d10/d15/ca0 0 2024-02-15T18:26:57.886 INFO:tasks.workunit.client.1.smithi191.stdout:4/157: symlink d2/d5/l40 0 2024-02-15T18:26:57.887 INFO:tasks.workunit.client.0.smithi042.stdout:1/458: dread - de/d15/f4c zero size 2024-02-15T18:26:57.891 INFO:tasks.workunit.client.1.smithi191.stdout:6/163: dread f1 [4194304,4194304] 0 2024-02-15T18:26:57.891 INFO:tasks.workunit.client.1.smithi191.stdout:3/95: link l0 d6/d1b/d1c/l24 0 2024-02-15T18:26:57.891 INFO:tasks.workunit.client.1.smithi191.stdout:3/96: write d6/fa [4402517,126285] 0 2024-02-15T18:26:57.892 INFO:tasks.workunit.client.0.smithi042.stdout:8/353: symlink dd/de/d22/d5a/l61 0 2024-02-15T18:26:57.893 INFO:tasks.workunit.client.1.smithi191.stdout:4/158: mkdir d2/d5/d7/d2f/d41 0 2024-02-15T18:26:57.895 INFO:tasks.workunit.client.1.smithi191.stdout:3/97: rename d6/c7 to d6/d1b/d1f/d21/c25 0 2024-02-15T18:26:57.896 INFO:tasks.workunit.client.0.smithi042.stdout:0/432: symlink d4/d9/d34/d4e/d6c/l93 0 2024-02-15T18:26:57.897 INFO:tasks.workunit.client.1.smithi191.stdout:4/159: mkdir d2/d5/d7/d3e/d42 0 2024-02-15T18:26:57.897 INFO:tasks.workunit.client.1.smithi191.stdout:4/160: write d2/f34 [376434,66996] 0 2024-02-15T18:26:57.898 INFO:tasks.workunit.client.0.smithi042.stdout:3/447: symlink d1/de/d1a/l97 0 2024-02-15T18:26:57.899 INFO:tasks.workunit.client.0.smithi042.stdout:5/472: dwrite d0/f34 [0,4194304] 0 2024-02-15T18:26:57.899 INFO:tasks.workunit.client.0.smithi042.stdout:5/473: read d0/d39/f96 [36245,73978] 0 2024-02-15T18:26:57.900 INFO:tasks.workunit.client.0.smithi042.stdout:4/413: mkdir da/d3b/d41/d8d/d8f 0 2024-02-15T18:26:57.902 INFO:tasks.workunit.client.1.smithi191.stdout:6/164: dread d8/f12 [0,4194304] 0 2024-02-15T18:26:57.904 INFO:tasks.workunit.client.0.smithi042.stdout:6/512: creat d9/fb2 x:0 0 0 2024-02-15T18:26:57.906 INFO:tasks.workunit.client.1.smithi191.stdout:4/161: mknod d2/d5/c43 0 2024-02-15T18:26:57.907 INFO:tasks.workunit.client.0.smithi042.stdout:7/465: creat d10/d12/d23/fa1 x:0 0 0 2024-02-15T18:26:57.907 INFO:tasks.workunit.client.1.smithi191.stdout:6/165: truncate d8/f15 27180 0 2024-02-15T18:26:57.907 INFO:tasks.workunit.client.1.smithi191.stdout:6/166: fdatasync d8/d9/f14 0 2024-02-15T18:26:57.908 INFO:tasks.workunit.client.0.smithi042.stdout:1/459: creat de/fa8 x:0 0 0 2024-02-15T18:26:57.910 INFO:tasks.workunit.client.1.smithi191.stdout:4/162: rename d2/d5/d7/dc to d2/d5/d14/d44 0 2024-02-15T18:26:57.910 INFO:tasks.workunit.client.1.smithi191.stdout:3/98: dwrite f5 [0,4194304] 0 2024-02-15T18:26:57.910 INFO:tasks.workunit.client.1.smithi191.stdout:3/99: fsync d6/f8 0 2024-02-15T18:26:57.919 INFO:tasks.workunit.client.0.smithi042.stdout:9/449: mkdir d6/d9d 0 2024-02-15T18:26:57.922 INFO:tasks.workunit.client.1.smithi191.stdout:4/163: rename d2/d5/d14/d44/l26 to d2/d5/l45 0 2024-02-15T18:26:57.924 INFO:tasks.workunit.client.1.smithi191.stdout:4/164: symlink d2/d5/d7/l46 0 2024-02-15T18:26:57.925 INFO:tasks.workunit.client.0.smithi042.stdout:8/354: unlink dd/de/d33/f44 0 2024-02-15T18:26:57.926 INFO:tasks.workunit.client.1.smithi191.stdout:4/165: mkdir d2/d5/d14/d47 0 2024-02-15T18:26:57.927 INFO:tasks.workunit.client.1.smithi191.stdout:4/166: truncate d2/f2c 676552 0 2024-02-15T18:26:57.927 INFO:tasks.workunit.client.0.smithi042.stdout:8/355: write dd/f13 [24274,79151] 0 2024-02-15T18:26:57.927 INFO:tasks.workunit.client.0.smithi042.stdout:8/356: chown dd/f3e 1657921712 1 2024-02-15T18:26:57.927 INFO:tasks.workunit.client.0.smithi042.stdout:8/357: dread - dd/de/d3c/f54 zero size 2024-02-15T18:26:57.927 INFO:tasks.workunit.client.1.smithi191.stdout:4/167: mkdir d2/d5/d14/d47/d48 0 2024-02-15T18:26:57.928 INFO:tasks.workunit.client.1.smithi191.stdout:4/168: truncate d2/f28 1364076 0 2024-02-15T18:26:57.928 INFO:tasks.workunit.client.0.smithi042.stdout:0/433: symlink d4/d9/d34/l94 0 2024-02-15T18:26:57.929 INFO:tasks.workunit.client.0.smithi042.stdout:3/448: link d1/d11/d14/f47 d1/d11/d14/d4d/d6c/d7b/f98 0 2024-02-15T18:26:57.930 INFO:tasks.workunit.client.0.smithi042.stdout:3/449: stat d1/d11/d2e/d4c/c70 0 2024-02-15T18:26:57.930 INFO:tasks.workunit.client.0.smithi042.stdout:9/450: dread d6/dd/d9/f41 [0,4194304] 0 2024-02-15T18:26:57.931 INFO:tasks.workunit.client.1.smithi191.stdout:3/100: dwrite d6/de/f13 [0,4194304] 0 2024-02-15T18:26:57.932 INFO:tasks.workunit.client.0.smithi042.stdout:9/451: dread d6/da/f65 [0,4194304] 0 2024-02-15T18:26:57.933 INFO:tasks.workunit.client.1.smithi191.stdout:4/169: creat d2/d5/d7/d3e/f49 x:0 0 0 2024-02-15T18:26:57.934 INFO:tasks.workunit.client.1.smithi191.stdout:6/167: dwrite d8/f15 [0,4194304] 0 2024-02-15T18:26:57.936 INFO:tasks.workunit.client.0.smithi042.stdout:4/414: mkdir da/d11/d29/d4a/d4f/d90 0 2024-02-15T18:26:57.937 INFO:tasks.workunit.client.1.smithi191.stdout:6/168: symlink d8/l2a 0 2024-02-15T18:26:57.938 INFO:tasks.workunit.client.1.smithi191.stdout:6/169: write d8/f15 [4611676,116498] 0 2024-02-15T18:26:57.938 INFO:tasks.workunit.client.1.smithi191.stdout:6/170: chown d8/d9/c27 1827379 1 2024-02-15T18:26:57.939 INFO:tasks.workunit.client.0.smithi042.stdout:6/513: truncate d9/d3e/f3b 3955763 0 2024-02-15T18:26:57.939 INFO:tasks.workunit.client.0.smithi042.stdout:7/466: mknod d10/d12/d65/ca2 0 2024-02-15T18:26:57.939 INFO:tasks.workunit.client.0.smithi042.stdout:7/467: fdatasync d10/d15/f3d 0 2024-02-15T18:26:57.939 INFO:tasks.workunit.client.0.smithi042.stdout:6/514: write d9/d11/d19/d1c/d31/d9b/f9c [952575,34514] 0 2024-02-15T18:26:57.939 INFO:tasks.workunit.client.0.smithi042.stdout:7/468: dread - d10/d43/d57/f79 zero size 2024-02-15T18:26:57.939 INFO:tasks.workunit.client.0.smithi042.stdout:6/515: write d9/d11/f15 [5152364,93907] 0 2024-02-15T18:26:57.941 INFO:tasks.workunit.client.1.smithi191.stdout:6/171: dread d8/d9/f14 [0,4194304] 0 2024-02-15T18:26:57.946 INFO:tasks.workunit.client.0.smithi042.stdout:1/460: symlink de/d15/d67/d63/la9 0 2024-02-15T18:26:57.949 INFO:tasks.workunit.client.1.smithi191.stdout:3/101: dwrite d6/f8 [0,4194304] 0 2024-02-15T18:26:57.950 INFO:tasks.workunit.client.1.smithi191.stdout:4/170: dread d2/d5/d14/d44/d30/f38 [0,4194304] 0 2024-02-15T18:26:57.950 INFO:tasks.workunit.client.1.smithi191.stdout:4/171: fdatasync d2/d5/d7/f29 0 2024-02-15T18:26:57.950 INFO:tasks.workunit.client.0.smithi042.stdout:8/358: creat dd/de/d12/f62 x:0 0 0 2024-02-15T18:26:57.950 INFO:tasks.workunit.client.0.smithi042.stdout:0/434: truncate d4/d9/d15/f21 6246527 0 2024-02-15T18:26:57.950 INFO:tasks.workunit.client.0.smithi042.stdout:0/435: stat d4/d9/d34/d4e/d6c/d70/l80 0 2024-02-15T18:26:57.951 INFO:tasks.workunit.client.0.smithi042.stdout:0/436: readlink d4/d38/d77/d2e/l45 0 2024-02-15T18:26:57.951 INFO:tasks.workunit.client.0.smithi042.stdout:0/437: chown d4/d9/f46 692 1 2024-02-15T18:26:57.951 INFO:tasks.workunit.client.0.smithi042.stdout:0/438: chown d4/d9/d34/d4e/d6c/d70/l80 210311 1 2024-02-15T18:26:57.951 INFO:tasks.workunit.client.1.smithi191.stdout:6/172: truncate d8/d19/f26 37943 0 2024-02-15T18:26:57.951 INFO:tasks.workunit.client.1.smithi191.stdout:6/173: fsync d8/d9/fc 0 2024-02-15T18:26:57.952 INFO:tasks.workunit.client.0.smithi042.stdout:3/450: rename d1/d11/d14/d4d/d73/c8a to d1/de/c99 0 2024-02-15T18:26:57.955 INFO:tasks.workunit.client.0.smithi042.stdout:9/452: link d6/dd/d9/f45 d6/da/d2f/f9e 0 2024-02-15T18:26:57.955 INFO:tasks.workunit.client.0.smithi042.stdout:9/453: write d6/da/d2f/f56 [1053750,66488] 0 2024-02-15T18:26:57.959 INFO:tasks.workunit.client.0.smithi042.stdout:7/469: creat d10/d49/d6d/d78/d83/fa3 x:0 0 0 2024-02-15T18:26:57.959 INFO:tasks.workunit.client.0.smithi042.stdout:7/470: chown d10/l2b 238811 1 2024-02-15T18:26:57.960 INFO:tasks.workunit.client.1.smithi191.stdout:3/102: symlink d6/l26 0 2024-02-15T18:26:57.961 INFO:tasks.workunit.client.1.smithi191.stdout:4/172: symlink d2/d5/d7/d3e/d42/l4a 0 2024-02-15T18:26:57.962 INFO:tasks.workunit.client.0.smithi042.stdout:6/516: creat d9/d11/d19/fb3 x:0 0 0 2024-02-15T18:26:57.963 INFO:tasks.workunit.client.1.smithi191.stdout:4/173: symlink d2/l4b 0 2024-02-15T18:26:57.965 INFO:tasks.workunit.client.1.smithi191.stdout:4/174: mknod d2/d5/d14/d44/c4c 0 2024-02-15T18:26:57.965 INFO:tasks.workunit.client.1.smithi191.stdout:4/175: chown d2/d5/d7/f25 25220 1 2024-02-15T18:26:57.966 INFO:tasks.workunit.client.1.smithi191.stdout:6/174: dwrite d8/d9/fc [0,4194304] 0 2024-02-15T18:26:57.967 INFO:tasks.workunit.client.1.smithi191.stdout:4/176: creat d2/d5/f4d x:0 0 0 2024-02-15T18:26:57.969 INFO:tasks.workunit.client.1.smithi191.stdout:3/103: dread f5 [0,4194304] 0 2024-02-15T18:26:57.974 INFO:tasks.workunit.client.1.smithi191.stdout:4/177: unlink d2/d5/l17 0 2024-02-15T18:26:57.975 INFO:tasks.workunit.client.0.smithi042.stdout:8/359: mknod dd/c63 0 2024-02-15T18:26:57.975 INFO:tasks.workunit.client.0.smithi042.stdout:8/360: read - dd/de/d33/f40 zero size 2024-02-15T18:26:57.976 INFO:tasks.workunit.client.1.smithi191.stdout:3/104: truncate d6/f8 597484 0 2024-02-15T18:26:57.977 INFO:tasks.workunit.client.1.smithi191.stdout:6/175: mkdir d8/d9/d2b 0 2024-02-15T18:26:57.980 INFO:tasks.workunit.client.0.smithi042.stdout:4/415: rename da/d11/d67/c5d to da/d2b/d7e/c91 0 2024-02-15T18:26:57.981 INFO:tasks.workunit.client.0.smithi042.stdout:4/416: dread - da/d3b/f81 zero size 2024-02-15T18:26:57.981 INFO:tasks.workunit.client.0.smithi042.stdout:4/417: chown da/d11/d67/d5c/d66/d78/f3e 14322 1 2024-02-15T18:26:57.981 INFO:tasks.workunit.client.0.smithi042.stdout:4/418: dread - da/d11/d27/f86 zero size 2024-02-15T18:26:57.981 INFO:tasks.workunit.client.0.smithi042.stdout:4/419: fsync da/d11/d67/d5c/d66/d78/f3e 0 2024-02-15T18:26:57.981 INFO:tasks.workunit.client.0.smithi042.stdout:4/420: truncate da/d11/d29/d4a/d4f/f8e 329066 0 2024-02-15T18:26:57.982 INFO:tasks.workunit.client.1.smithi191.stdout:4/178: symlink d2/d5/d14/l4e 0 2024-02-15T18:26:57.983 INFO:tasks.workunit.client.1.smithi191.stdout:4/179: dread d2/f2c [0,4194304] 0 2024-02-15T18:26:57.983 INFO:tasks.workunit.client.0.smithi042.stdout:3/451: getdents d1/de 0 2024-02-15T18:26:57.984 INFO:tasks.workunit.client.1.smithi191.stdout:3/105: mkdir d6/d27 0 2024-02-15T18:26:57.986 INFO:tasks.workunit.client.0.smithi042.stdout:9/454: creat d6/dd/d9/f9f x:0 0 0 2024-02-15T18:26:57.987 INFO:tasks.workunit.client.0.smithi042.stdout:7/471: unlink d10/d12/c77 0 2024-02-15T18:26:57.991 INFO:tasks.workunit.client.0.smithi042.stdout:6/517: mkdir d9/d11/d19/d1c/d31/d9b/db4 0 2024-02-15T18:26:57.993 INFO:tasks.workunit.client.0.smithi042.stdout:1/461: link de/d15/f31 de/d15/d67/d89/faa 0 2024-02-15T18:26:57.994 INFO:tasks.workunit.client.0.smithi042.stdout:1/462: readlink de/d15/d67/d89/la0 0 2024-02-15T18:26:57.994 INFO:tasks.workunit.client.0.smithi042.stdout:7/472: dread d10/d43/d57/f5e [0,4194304] 0 2024-02-15T18:26:57.994 INFO:tasks.workunit.client.0.smithi042.stdout:8/361: creat dd/de/d22/d5a/f64 x:0 0 0 2024-02-15T18:26:57.996 INFO:tasks.workunit.client.0.smithi042.stdout:0/439: truncate d4/d9/d15/f21 1164661 0 2024-02-15T18:26:57.996 INFO:tasks.workunit.client.0.smithi042.stdout:0/440: dread - d4/d38/f49 zero size 2024-02-15T18:26:57.997 INFO:tasks.workunit.client.1.smithi191.stdout:4/180: creat d2/d5/d14/d44/d30/f4f x:0 0 0 2024-02-15T18:26:57.997 INFO:tasks.workunit.client.1.smithi191.stdout:4/181: truncate d2/d5/d7/d2f/f31 1010633 0 2024-02-15T18:26:57.998 INFO:tasks.workunit.client.1.smithi191.stdout:3/106: mknod d6/d1b/d1f/d21/c28 0 2024-02-15T18:26:58.000 INFO:tasks.workunit.client.0.smithi042.stdout:4/421: link da/d11/d67/d5c/c5f da/d11/d27/c92 0 2024-02-15T18:26:58.004 INFO:tasks.workunit.client.0.smithi042.stdout:3/452: link d1/d11/d14/l30 d1/d11/d2e/l9a 0 2024-02-15T18:26:58.005 INFO:tasks.workunit.client.0.smithi042.stdout:3/453: dread d1/de/d19/f3e [0,4194304] 0 2024-02-15T18:26:58.006 INFO:tasks.workunit.client.0.smithi042.stdout:3/454: write d1/d11/d14/d4d/f4f [148817,47370] 0 2024-02-15T18:26:58.007 INFO:tasks.workunit.client.1.smithi191.stdout:4/182: creat d2/d5/d14/d44/f50 x:0 0 0 2024-02-15T18:26:58.009 INFO:tasks.workunit.client.0.smithi042.stdout:9/455: creat d6/da/d1d/d2d/d40/d5f/fa0 x:0 0 0 2024-02-15T18:26:58.009 INFO:tasks.workunit.client.1.smithi191.stdout:6/176: rename d8/d9/lb to d8/d19/l2c 0 2024-02-15T18:26:58.010 INFO:tasks.workunit.client.1.smithi191.stdout:6/177: chown d8/d19 603120 1 2024-02-15T18:26:58.010 INFO:tasks.workunit.client.1.smithi191.stdout:4/183: getdents d2/d5/d14 0 2024-02-15T18:26:58.010 INFO:tasks.workunit.client.1.smithi191.stdout:4/184: dread - d2/d5/d14/f2e zero size 2024-02-15T18:26:58.012 INFO:tasks.workunit.client.0.smithi042.stdout:6/518: creat d9/d11/d19/fb5 x:0 0 0 2024-02-15T18:26:58.012 INFO:tasks.workunit.client.0.smithi042.stdout:1/463: getdents de/d15 0 2024-02-15T18:26:58.013 INFO:tasks.workunit.client.1.smithi191.stdout:4/185: link d2/d5/d7/f25 d2/d5/d7/d2f/f51 0 2024-02-15T18:26:58.014 INFO:tasks.workunit.client.0.smithi042.stdout:8/362: rename dd/de/d33/d3b/f59 to dd/de/d12/f65 0 2024-02-15T18:26:58.016 INFO:tasks.workunit.client.0.smithi042.stdout:4/422: link da/d11/d67/f51 da/d3b/d41/d8d/f93 0 2024-02-15T18:26:58.017 INFO:tasks.workunit.client.0.smithi042.stdout:7/473: dwrite d10/f5b [0,4194304] 0 2024-02-15T18:26:58.017 INFO:tasks.workunit.client.0.smithi042.stdout:7/474: fsync d10/d49/d6d/d78/f91 0 2024-02-15T18:26:58.017 INFO:tasks.workunit.client.1.smithi191.stdout:4/186: mkdir d2/d5/d7/d3e/d42/d52 0 2024-02-15T18:26:58.017 INFO:tasks.workunit.client.1.smithi191.stdout:4/187: chown d2/c21 3905859 1 2024-02-15T18:26:58.017 INFO:tasks.workunit.client.1.smithi191.stdout:4/188: chown d2/d5/l10 111529819 1 2024-02-15T18:26:58.018 INFO:tasks.workunit.client.1.smithi191.stdout:3/107: dwrite f5 [0,4194304] 0 2024-02-15T18:26:58.022 INFO:tasks.workunit.client.0.smithi042.stdout:3/455: rmdir d1/d11/d2e 39 2024-02-15T18:26:58.028 INFO:tasks.workunit.client.1.smithi191.stdout:4/189: creat d2/d5/d7/d3e/d42/d52/f53 x:0 0 0 2024-02-15T18:26:58.029 INFO:tasks.workunit.client.0.smithi042.stdout:9/456: mkdir d6/dd/d9/da1 0 2024-02-15T18:26:58.031 INFO:tasks.workunit.client.1.smithi191.stdout:3/108: link d6/d1b/d1f/d21/c25 d6/de/c29 0 2024-02-15T18:26:58.032 INFO:tasks.workunit.client.0.smithi042.stdout:1/464: unlink de/d15/d2c/f4b 0 2024-02-15T18:26:58.032 INFO:tasks.workunit.client.0.smithi042.stdout:1/465: readlink de/d54/l6c 0 2024-02-15T18:26:58.032 INFO:tasks.workunit.client.0.smithi042.stdout:1/466: dread - de/d15/d67/d63/f8f zero size 2024-02-15T18:26:58.032 INFO:tasks.workunit.client.1.smithi191.stdout:4/190: creat d2/d5/d7/d3e/d42/d52/f54 x:0 0 0 2024-02-15T18:26:58.033 INFO:tasks.workunit.client.0.smithi042.stdout:0/441: rename d4/d38/d77/d2e/l63 to d4/d38/d39/l95 0 2024-02-15T18:26:58.033 INFO:tasks.workunit.client.0.smithi042.stdout:0/442: fsync d4/d9/d15/f83 0 2024-02-15T18:26:58.034 INFO:tasks.workunit.client.0.smithi042.stdout:9/457: read d6/da/d1d/d2d/f39 [1196279,121088] 0 2024-02-15T18:26:58.034 INFO:tasks.workunit.client.0.smithi042.stdout:9/458: readlink d6/l34 0 2024-02-15T18:26:58.034 INFO:tasks.workunit.client.1.smithi191.stdout:3/109: creat d6/d1b/d1c/f2a x:0 0 0 2024-02-15T18:26:58.035 INFO:tasks.workunit.client.0.smithi042.stdout:4/423: mknod da/d3b/c94 0 2024-02-15T18:26:58.035 INFO:tasks.workunit.client.1.smithi191.stdout:4/191: symlink d2/d5/d7/d2f/d41/l55 0 2024-02-15T18:26:58.036 INFO:tasks.workunit.client.1.smithi191.stdout:4/192: write d2/f3 [2917171,55188] 0 2024-02-15T18:26:58.036 INFO:tasks.workunit.client.0.smithi042.stdout:9/459: dread d6/dd/d9/f72 [0,4194304] 0 2024-02-15T18:26:58.037 INFO:tasks.workunit.client.1.smithi191.stdout:3/110: unlink f5 0 2024-02-15T18:26:58.038 INFO:tasks.workunit.client.0.smithi042.stdout:7/475: getdents d10/d49/d68 0 2024-02-15T18:26:58.038 INFO:tasks.workunit.client.0.smithi042.stdout:1/467: write de/d15/d1b/f57 [3358987,20562] 0 2024-02-15T18:26:58.039 INFO:tasks.workunit.client.1.smithi191.stdout:4/193: symlink d2/d5/d14/d44/l56 0 2024-02-15T18:26:58.040 INFO:tasks.workunit.client.0.smithi042.stdout:3/456: rmdir d1/d11/d2e 39 2024-02-15T18:26:58.040 INFO:tasks.workunit.client.0.smithi042.stdout:3/457: truncate d1/d11/d2e/f54 1160801 0 2024-02-15T18:26:58.040 INFO:tasks.workunit.client.0.smithi042.stdout:6/519: dwrite d9/d11/d19/d35/d49/d79/fa8 [0,4194304] 0 2024-02-15T18:26:58.040 INFO:tasks.workunit.client.0.smithi042.stdout:8/363: dwrite dd/de/d12/f65 [0,4194304] 0 2024-02-15T18:26:58.041 INFO:tasks.workunit.client.1.smithi191.stdout:3/111: symlink d6/d27/l2b 0 2024-02-15T18:26:58.041 INFO:tasks.workunit.client.0.smithi042.stdout:4/424: creat da/d11/d67/f95 x:0 0 0 2024-02-15T18:26:58.041 INFO:tasks.workunit.client.0.smithi042.stdout:4/425: fdatasync da/d11/d67/d5c/f65 0 2024-02-15T18:26:58.047 INFO:tasks.workunit.client.0.smithi042.stdout:0/443: dread d4/d38/d39/f6e [0,4194304] 0 2024-02-15T18:26:58.047 INFO:tasks.workunit.client.0.smithi042.stdout:0/444: fdatasync d4/d9/d15/f78 0 2024-02-15T18:26:58.047 INFO:tasks.workunit.client.0.smithi042.stdout:7/476: creat d10/d43/fa4 x:0 0 0 2024-02-15T18:26:58.047 INFO:tasks.workunit.client.0.smithi042.stdout:7/477: stat d10/d12/f96 0 2024-02-15T18:26:58.049 INFO:tasks.workunit.client.0.smithi042.stdout:1/468: rename de/d15/d1b/f9d to de/d15/d67/d74/fab 0 2024-02-15T18:26:58.050 INFO:tasks.workunit.client.1.smithi191.stdout:4/194: dread d2/d5/f1a [0,4194304] 0 2024-02-15T18:26:58.050 INFO:tasks.workunit.client.1.smithi191.stdout:4/195: readlink d2/l1d 0 2024-02-15T18:26:58.050 INFO:tasks.workunit.client.0.smithi042.stdout:1/469: read f2 [1638608,13876] 0 2024-02-15T18:26:58.051 INFO:tasks.workunit.client.1.smithi191.stdout:3/112: dread d6/fd [0,4194304] 0 2024-02-15T18:26:58.054 INFO:tasks.workunit.client.1.smithi191.stdout:3/113: creat d6/de/f2c x:0 0 0 2024-02-15T18:26:58.057 INFO:tasks.workunit.client.0.smithi042.stdout:0/445: link d4/d9/d34/d4e/d6c/f71 d4/d9/d5a/d60/f96 0 2024-02-15T18:26:58.058 INFO:tasks.workunit.client.0.smithi042.stdout:5/474: sync 2024-02-15T18:26:58.058 INFO:tasks.workunit.client.0.smithi042.stdout:5/475: chown d0/d14/f63 1584 1 2024-02-15T18:26:58.058 INFO:tasks.workunit.client.0.smithi042.stdout:5/476: chown d0/l4 30278036 1 2024-02-15T18:26:58.058 INFO:tasks.workunit.client.0.smithi042.stdout:2/390: sync 2024-02-15T18:26:58.058 INFO:tasks.workunit.client.0.smithi042.stdout:7/478: mkdir d10/d15/d8f/da5 0 2024-02-15T18:26:58.059 INFO:tasks.workunit.client.1.smithi191.stdout:3/114: read d6/d1b/f1d [3498205,130446] 0 2024-02-15T18:26:58.060 INFO:tasks.workunit.client.0.smithi042.stdout:9/460: dwrite d6/da/d2f/d55/f6d [0,4194304] 0 2024-02-15T18:26:58.060 INFO:tasks.workunit.client.0.smithi042.stdout:9/461: dread - d6/da/d1d/d2d/d40/f8c zero size 2024-02-15T18:26:58.060 INFO:tasks.workunit.client.0.smithi042.stdout:3/458: rename d1/d11/d14/d4d/d6c/d7b/f88 to d1/d11/d14/d4d/d6c/f9b 0 2024-02-15T18:26:58.060 INFO:tasks.workunit.client.0.smithi042.stdout:3/459: fdatasync d1/de/d19/f44 0 2024-02-15T18:26:58.060 INFO:tasks.workunit.client.0.smithi042.stdout:5/477: creat d0/d38/d53/d68/fa8 x:0 0 0 2024-02-15T18:26:58.061 INFO:tasks.workunit.client.0.smithi042.stdout:6/520: dread d9/d11/f15 [0,4194304] 0 2024-02-15T18:26:58.061 INFO:tasks.workunit.client.0.smithi042.stdout:6/521: write d9/f3c [1424992,7361] 0 2024-02-15T18:26:58.061 INFO:tasks.workunit.client.0.smithi042.stdout:6/522: chown d9/d11/d19/d35/d55/fac 3110972 1 2024-02-15T18:26:58.062 INFO:tasks.workunit.client.0.smithi042.stdout:9/462: read d6/da/d2f/d55/f6d [3558010,29045] 0 2024-02-15T18:26:58.063 INFO:tasks.workunit.client.0.smithi042.stdout:9/463: read f2 [947456,46183] 0 2024-02-15T18:26:58.064 INFO:tasks.workunit.client.0.smithi042.stdout:1/470: write de/d15/d95/f99 [1400836,33490] 0 2024-02-15T18:26:58.069 INFO:tasks.workunit.client.1.smithi191.stdout:4/196: rmdir d2 39 2024-02-15T18:26:58.071 INFO:tasks.workunit.client.0.smithi042.stdout:2/391: mkdir d0/d8/d10/d31/d79 0 2024-02-15T18:26:58.076 INFO:tasks.workunit.client.0.smithi042.stdout:2/392: write d0/d8/db/f3e [2865846,91908] 0 2024-02-15T18:26:58.078 INFO:tasks.workunit.client.0.smithi042.stdout:5/478: truncate d0/d38/f3c 3237818 0 2024-02-15T18:26:58.079 INFO:tasks.workunit.client.0.smithi042.stdout:5/479: read d0/d38/d53/d68/d6a/f56 [2565668,11059] 0 2024-02-15T18:26:58.079 INFO:tasks.workunit.client.0.smithi042.stdout:5/480: fsync d0/db/f10 0 2024-02-15T18:26:58.082 INFO:tasks.workunit.client.0.smithi042.stdout:8/364: rename dd/de/d12/d32 to dd/de/d33/d66 0 2024-02-15T18:26:58.084 INFO:tasks.workunit.client.1.smithi191.stdout:4/197: creat d2/d5/d7/d2f/d41/f57 x:0 0 0 2024-02-15T18:26:58.084 INFO:tasks.workunit.client.1.smithi191.stdout:4/198: chown d2/d5/d7/l46 653 1 2024-02-15T18:26:58.087 INFO:tasks.workunit.client.0.smithi042.stdout:1/471: symlink de/d15/d67/d66/lac 0 2024-02-15T18:26:58.087 INFO:tasks.workunit.client.1.smithi191.stdout:4/199: creat d2/d5/d14/f58 x:0 0 0 2024-02-15T18:26:58.088 INFO:tasks.workunit.client.1.smithi191.stdout:4/200: mkdir d2/d5/d14/d47/d48/d59 0 2024-02-15T18:26:58.088 INFO:tasks.workunit.client.1.smithi191.stdout:4/201: fsync d2/d5/d7/d3e/d42/d52/f54 0 2024-02-15T18:26:58.090 INFO:tasks.workunit.client.1.smithi191.stdout:4/202: mknod d2/d5/d7/d2f/c5a 0 2024-02-15T18:26:58.090 INFO:tasks.workunit.client.1.smithi191.stdout:4/203: write d2/d5/d14/f58 [963174,28423] 0 2024-02-15T18:26:58.090 INFO:tasks.workunit.client.1.smithi191.stdout:4/204: write d2/f34 [469316,73384] 0 2024-02-15T18:26:58.092 INFO:tasks.workunit.client.1.smithi191.stdout:4/205: symlink d2/d5/d7/d2f/d41/l5b 0 2024-02-15T18:26:58.095 INFO:tasks.workunit.client.0.smithi042.stdout:2/393: unlink d0/d8/db/de/d1a/f4c 0 2024-02-15T18:26:58.095 INFO:tasks.workunit.client.0.smithi042.stdout:2/394: fdatasync d0/d8/f28 0 2024-02-15T18:26:58.096 INFO:tasks.workunit.client.0.smithi042.stdout:6/523: dwrite d9/d11/d19/d35/d49/f4c [4194304,4194304] 0 2024-02-15T18:26:58.096 INFO:tasks.workunit.client.0.smithi042.stdout:3/460: dwrite d1/de/d19/f1f [0,4194304] 0 2024-02-15T18:26:58.096 INFO:tasks.workunit.client.0.smithi042.stdout:7/479: dwrite d10/d15/d39/d6c/f8d [0,4194304] 0 2024-02-15T18:26:58.096 INFO:tasks.workunit.client.0.smithi042.stdout:7/480: write d10/d15/d39/f5d [4978082,93715] 0 2024-02-15T18:26:58.098 INFO:tasks.workunit.client.1.smithi191.stdout:4/206: creat d2/f5c x:0 0 0 2024-02-15T18:26:58.099 INFO:tasks.workunit.client.0.smithi042.stdout:4/426: rename da/d11/f39 to da/d11/d67/d5c/d66/d78/d2f/d4d/f96 0 2024-02-15T18:26:58.099 INFO:tasks.workunit.client.0.smithi042.stdout:4/427: chown da/d3b/l57 0 1 2024-02-15T18:26:58.105 INFO:tasks.workunit.client.0.smithi042.stdout:9/464: mkdir d6/da2 0 2024-02-15T18:26:58.106 INFO:tasks.workunit.client.1.smithi191.stdout:4/207: dread d2/d5/d7/f18 [0,4194304] 0 2024-02-15T18:26:58.107 INFO:tasks.workunit.client.0.smithi042.stdout:2/395: creat d0/d8/d10/d31/f7a x:0 0 0 2024-02-15T18:26:58.107 INFO:tasks.workunit.client.0.smithi042.stdout:2/396: readlink d0/d8/db/l4f 0 2024-02-15T18:26:58.110 INFO:tasks.workunit.client.0.smithi042.stdout:2/397: write d0/f49 [3475361,127410] 0 2024-02-15T18:26:58.111 INFO:tasks.workunit.client.0.smithi042.stdout:6/524: stat d9/d11/d1d/l78 0 2024-02-15T18:26:58.114 INFO:tasks.workunit.client.0.smithi042.stdout:7/481: getdents d10/d49/d6d/d78 0 2024-02-15T18:26:58.117 INFO:tasks.workunit.client.0.smithi042.stdout:2/398: dread d0/d8/db/de/d1a/f6b [0,4194304] 0 2024-02-15T18:26:58.117 INFO:tasks.workunit.client.0.smithi042.stdout:7/482: dread d10/d12/f5f [0,4194304] 0 2024-02-15T18:26:58.117 INFO:tasks.workunit.client.0.smithi042.stdout:7/483: readlink d10/d12/d23/d51/l81 0 2024-02-15T18:26:58.119 INFO:tasks.workunit.client.0.smithi042.stdout:5/481: rename d0/d12/d65/d74/d86 to d0/d22/d7b/da9 0 2024-02-15T18:26:58.122 INFO:tasks.workunit.client.0.smithi042.stdout:9/465: rmdir d6/dd/d9 39 2024-02-15T18:26:58.122 INFO:tasks.workunit.client.1.smithi191.stdout:4/208: dwrite d2/f3 [0,4194304] 0 2024-02-15T18:26:58.123 INFO:tasks.workunit.client.0.smithi042.stdout:1/472: creat de/d15/d67/d74/d7e/fad x:0 0 0 2024-02-15T18:26:58.123 INFO:tasks.workunit.client.0.smithi042.stdout:5/482: write d0/db/f35 [2982606,120011] 0 2024-02-15T18:26:58.127 INFO:tasks.workunit.client.0.smithi042.stdout:6/525: symlink d9/d11/d19/d35/d49/d79/lb6 0 2024-02-15T18:26:58.128 INFO:tasks.workunit.client.0.smithi042.stdout:6/526: write d9/d11/d46/d89/fab [1335017,120345] 0 2024-02-15T18:26:58.129 INFO:tasks.workunit.client.0.smithi042.stdout:2/399: getdents d0/d8/d10/d61 0 2024-02-15T18:26:58.129 INFO:tasks.workunit.client.0.smithi042.stdout:1/473: dread de/d15/d67/f3a [0,4194304] 0 2024-02-15T18:26:58.129 INFO:tasks.workunit.client.0.smithi042.stdout:1/474: fsync de/d15/f4c 0 2024-02-15T18:26:58.129 INFO:tasks.workunit.client.0.smithi042.stdout:1/475: chown de/d6d/c7a 55295057 1 2024-02-15T18:26:58.130 INFO:tasks.workunit.client.0.smithi042.stdout:7/484: creat d10/d15/fa6 x:0 0 0 2024-02-15T18:26:58.136 INFO:tasks.workunit.client.0.smithi042.stdout:8/365: rename dd/de/d22 to dd/de/d3c/d4b/d67 0 2024-02-15T18:26:58.136 INFO:tasks.workunit.client.0.smithi042.stdout:8/366: readlink dd/de/d3c/d4b/d67/l41 0 2024-02-15T18:26:58.136 INFO:tasks.workunit.client.0.smithi042.stdout:9/466: creat d6/da/d2f/d55/fa3 x:0 0 0 2024-02-15T18:26:58.136 INFO:tasks.workunit.client.0.smithi042.stdout:9/467: chown d6/da/d1d/d2d/d40/f78 19527 1 2024-02-15T18:26:58.136 INFO:tasks.workunit.client.0.smithi042.stdout:9/468: fdatasync d6/dd/d9/f45 0 2024-02-15T18:26:58.137 INFO:tasks.workunit.client.1.smithi191.stdout:4/209: truncate d2/f3 3167825 0 2024-02-15T18:26:58.137 INFO:tasks.workunit.client.1.smithi191.stdout:4/210: fsync d2/d5/d14/f2e 0 2024-02-15T18:26:58.137 INFO:tasks.workunit.client.1.smithi191.stdout:4/211: truncate d2/d5/d7/f3b 79850 0 2024-02-15T18:26:58.138 INFO:tasks.workunit.client.0.smithi042.stdout:5/483: symlink d0/d14/d88/laa 0 2024-02-15T18:26:58.138 INFO:tasks.workunit.client.0.smithi042.stdout:5/484: read - d0/d12/d65/d74/f4c zero size 2024-02-15T18:26:58.138 INFO:tasks.workunit.client.0.smithi042.stdout:5/485: readlink d0/d38/d53/d68/d6a/d42/l44 0 2024-02-15T18:26:58.139 INFO:tasks.workunit.client.0.smithi042.stdout:5/486: write d0/d39/f96 [1669560,87114] 0 2024-02-15T18:26:58.141 INFO:tasks.workunit.client.1.smithi191.stdout:4/212: dread d2/d5/d7/f18 [0,4194304] 0 2024-02-15T18:26:58.145 INFO:tasks.workunit.client.0.smithi042.stdout:7/485: dwrite d10/f7b [0,4194304] 0 2024-02-15T18:26:58.203 INFO:tasks.workunit.client.0.smithi042.stdout:6/527: symlink d9/d11/d19/d1c/d31/d9b/lb7 0 2024-02-15T18:26:58.204 INFO:tasks.workunit.client.0.smithi042.stdout:6/528: chown d9/d11/d19/d1c 776039997 1 2024-02-15T18:26:58.205 INFO:tasks.workunit.client.1.smithi191.stdout:4/213: link d2/d5/l3d d2/d5/d7/l5d 0 2024-02-15T18:26:58.205 INFO:tasks.workunit.client.1.smithi191.stdout:4/214: chown d2/d5/d7/l1b 2476339 1 2024-02-15T18:26:58.205 INFO:tasks.workunit.client.0.smithi042.stdout:1/476: symlink de/d15/d67/d46/d6b/d88/lae 0 2024-02-15T18:26:58.206 INFO:tasks.workunit.client.0.smithi042.stdout:1/477: fdatasync de/d15/d2c/f41 0 2024-02-15T18:26:58.222 INFO:tasks.workunit.client.0.smithi042.stdout:6/529: dwrite d9/d11/d19/d35/f61 [0,4194304] 0 2024-02-15T18:26:58.222 INFO:tasks.workunit.client.0.smithi042.stdout:6/530: dread - d9/d11/d1d/fb0 zero size 2024-02-15T18:26:58.241 INFO:tasks.workunit.client.0.smithi042.stdout:6/531: dwrite d9/fb2 [0,4194304] 0 2024-02-15T18:26:58.253 INFO:tasks.workunit.client.0.smithi042.stdout:4/428: rename da/d11/d29/d4a/f84 to da/d11/d29/d56/f97 0 2024-02-15T18:26:58.253 INFO:tasks.workunit.client.0.smithi042.stdout:4/429: dread - da/d11/d27/f89 zero size 2024-02-15T18:26:58.259 INFO:tasks.workunit.client.0.smithi042.stdout:8/367: creat dd/de/d12/f68 x:0 0 0 2024-02-15T18:26:58.261 INFO:tasks.workunit.client.0.smithi042.stdout:6/532: dwrite d9/d11/d19/fb5 [0,4194304] 0 2024-02-15T18:26:58.276 INFO:tasks.workunit.client.0.smithi042.stdout:9/469: creat d6/da/d2f/d55/fa4 x:0 0 0 2024-02-15T18:26:58.277 INFO:tasks.workunit.client.0.smithi042.stdout:9/470: write d6/da/f25 [3606714,35182] 0 2024-02-15T18:26:58.277 INFO:tasks.workunit.client.0.smithi042.stdout:9/471: write d6/dd/d9/f14 [6716069,59118] 0 2024-02-15T18:26:58.280 INFO:tasks.workunit.client.1.smithi191.stdout:4/215: unlink d2/d5/d14/d44/f50 0 2024-02-15T18:26:58.282 INFO:tasks.workunit.client.0.smithi042.stdout:5/487: mknod d0/d38/d53/d91/d94/cab 0 2024-02-15T18:26:58.283 INFO:tasks.workunit.client.0.smithi042.stdout:7/486: mkdir d10/d12/d65/da7 0 2024-02-15T18:26:58.283 INFO:tasks.workunit.client.0.smithi042.stdout:7/487: write d10/d49/f67 [1121176,102083] 0 2024-02-15T18:26:58.352 INFO:tasks.workunit.client.1.smithi191.stdout:2/94: sync 2024-02-15T18:26:58.352 INFO:tasks.workunit.client.1.smithi191.stdout:8/239: sync 2024-02-15T18:26:58.352 INFO:tasks.workunit.client.1.smithi191.stdout:9/137: sync 2024-02-15T18:26:58.352 INFO:tasks.workunit.client.1.smithi191.stdout:1/123: sync 2024-02-15T18:26:58.352 INFO:tasks.workunit.client.1.smithi191.stdout:5/66: sync 2024-02-15T18:26:58.352 INFO:tasks.workunit.client.1.smithi191.stdout:7/74: sync 2024-02-15T18:26:58.353 INFO:tasks.workunit.client.1.smithi191.stdout:9/138: dread d0/f12 [0,4194304] 0 2024-02-15T18:26:58.354 INFO:tasks.workunit.client.1.smithi191.stdout:5/67: stat d0/l19 0 2024-02-15T18:26:58.355 INFO:tasks.workunit.client.0.smithi042.stdout:1/478: truncate de/d15/d67/d46/f71 4062519 0 2024-02-15T18:26:58.356 INFO:tasks.workunit.client.1.smithi191.stdout:8/240: unlink d3/le 0 2024-02-15T18:26:58.358 INFO:tasks.workunit.client.1.smithi191.stdout:1/124: getdents d0/d8/df 0 2024-02-15T18:26:58.362 INFO:tasks.workunit.client.0.smithi042.stdout:0/446: sync 2024-02-15T18:26:58.362 INFO:tasks.workunit.client.0.smithi042.stdout:3/461: sync 2024-02-15T18:26:58.362 INFO:tasks.workunit.client.0.smithi042.stdout:0/447: chown d4/d38/d5c/f61 3 1 2024-02-15T18:26:58.362 INFO:tasks.workunit.client.0.smithi042.stdout:3/462: chown d1/d11/d2e 6447 1 2024-02-15T18:26:58.363 INFO:tasks.workunit.client.1.smithi191.stdout:7/75: rename d0/f5 to d0/d1/df/f19 0 2024-02-15T18:26:58.364 INFO:tasks.workunit.client.1.smithi191.stdout:2/95: mknod df/d12/c14 0 2024-02-15T18:26:58.366 INFO:tasks.workunit.client.0.smithi042.stdout:2/400: rename d0/d35/d64 to d0/d8/d10/d7b 0 2024-02-15T18:26:58.368 INFO:tasks.workunit.client.1.smithi191.stdout:9/139: mkdir d0/d7/dc/d32 0 2024-02-15T18:26:58.369 INFO:tasks.workunit.client.1.smithi191.stdout:9/140: write d0/d7/f9 [1060996,127093] 0 2024-02-15T18:26:58.370 INFO:tasks.workunit.client.1.smithi191.stdout:5/68: unlink d0/lb 0 2024-02-15T18:26:58.371 INFO:tasks.workunit.client.1.smithi191.stdout:8/241: creat d3/d2c/d1a/d23/d2e/d47/f52 x:0 0 0 2024-02-15T18:26:58.371 INFO:tasks.workunit.client.1.smithi191.stdout:8/242: chown d3/f31 94817214 1 2024-02-15T18:26:58.371 INFO:tasks.workunit.client.1.smithi191.stdout:8/243: chown d3/d2c/d1a/d23/d2e/f4b 15128 1 2024-02-15T18:26:58.374 INFO:tasks.workunit.client.1.smithi191.stdout:1/125: creat d0/db/f26 x:0 0 0 2024-02-15T18:26:58.374 INFO:tasks.workunit.client.1.smithi191.stdout:1/126: readlink d0/db/lc 0 2024-02-15T18:26:58.375 INFO:tasks.workunit.client.1.smithi191.stdout:7/76: creat d0/d1/df/f1a x:0 0 0 2024-02-15T18:26:58.375 INFO:tasks.workunit.client.0.smithi042.stdout:6/533: link d9/d11/d19/d35/d49/d79/f7a d9/d11/d97/fb8 0 2024-02-15T18:26:58.376 INFO:tasks.workunit.client.0.smithi042.stdout:9/472: creat d6/dd/d9/da1/fa5 x:0 0 0 2024-02-15T18:26:58.376 INFO:tasks.workunit.client.0.smithi042.stdout:9/473: chown d6/da/d2f/d55/d73 0 1 2024-02-15T18:26:58.376 INFO:tasks.workunit.client.0.smithi042.stdout:9/474: write d6/dd/d9/d59/d5c/f7e [1140115,75419] 0 2024-02-15T18:26:58.377 INFO:tasks.workunit.client.1.smithi191.stdout:2/96: mknod df/c15 0 2024-02-15T18:26:58.377 INFO:tasks.workunit.client.0.smithi042.stdout:5/488: symlink d0/d22/d7b/da9/lac 0 2024-02-15T18:26:58.377 INFO:tasks.workunit.client.0.smithi042.stdout:5/489: read - d0/d14/f87 zero size 2024-02-15T18:26:58.378 INFO:tasks.workunit.client.0.smithi042.stdout:7/488: creat d10/d15/fa8 x:0 0 0 2024-02-15T18:26:58.379 INFO:tasks.workunit.client.1.smithi191.stdout:9/141: mkdir d0/d7/dc/d17/d33 0 2024-02-15T18:26:58.383 INFO:tasks.workunit.client.0.smithi042.stdout:1/479: unlink de/d15/c64 0 2024-02-15T18:26:58.383 INFO:tasks.workunit.client.0.smithi042.stdout:1/480: dread - de/d15/d67/d63/f8b zero size 2024-02-15T18:26:58.387 INFO:tasks.workunit.client.0.smithi042.stdout:3/463: truncate d1/d11/d14/f39 3416368 0 2024-02-15T18:26:58.387 INFO:tasks.workunit.client.0.smithi042.stdout:3/464: write d1/d11/d2e/d4c/f76 [533761,21239] 0 2024-02-15T18:26:58.389 INFO:tasks.workunit.client.0.smithi042.stdout:1/481: write f2 [11449427,97343] 0 2024-02-15T18:26:58.392 INFO:tasks.workunit.client.0.smithi042.stdout:4/430: rename da/d11/d27/c4c to da/d11/d67/d5c/d66/d78/d2f/c98 0 2024-02-15T18:26:58.392 INFO:tasks.workunit.client.1.smithi191.stdout:1/127: mkdir d0/d8/d27 0 2024-02-15T18:26:58.393 INFO:tasks.workunit.client.0.smithi042.stdout:2/401: write d0/d8/db/d44/f69 [892379,130592] 0 2024-02-15T18:26:58.393 INFO:tasks.workunit.client.1.smithi191.stdout:1/128: dread d0/db/fe [0,4194304] 0 2024-02-15T18:26:58.393 INFO:tasks.workunit.client.1.smithi191.stdout:1/129: fsync d0/db/f26 0 2024-02-15T18:26:58.394 INFO:tasks.workunit.client.1.smithi191.stdout:9/142: mknod d0/d7/dc/d17/c34 0 2024-02-15T18:26:58.395 INFO:tasks.workunit.client.0.smithi042.stdout:6/534: symlink d9/d3e/lb9 0 2024-02-15T18:26:58.396 INFO:tasks.workunit.client.1.smithi191.stdout:7/77: dwrite d0/f14 [0,4194304] 0 2024-02-15T18:26:58.402 INFO:tasks.workunit.client.1.smithi191.stdout:7/78: dread d0/f8 [0,4194304] 0 2024-02-15T18:26:58.402 INFO:tasks.workunit.client.1.smithi191.stdout:8/244: write d3/d2c/f1d [2841433,114121] 0 2024-02-15T18:26:58.404 INFO:tasks.workunit.client.1.smithi191.stdout:1/130: dread d0/d8/df/d12/f23 [0,4194304] 0 2024-02-15T18:26:58.404 INFO:tasks.workunit.client.1.smithi191.stdout:9/143: link d0/d7/dc/d17/f25 d0/d7/da/f35 0 2024-02-15T18:26:58.406 INFO:tasks.workunit.client.1.smithi191.stdout:2/97: dwrite fb [0,4194304] 0 2024-02-15T18:26:58.408 INFO:tasks.workunit.client.0.smithi042.stdout:9/475: creat d6/dd/d9/d59/d5c/fa6 x:0 0 0 2024-02-15T18:26:58.408 INFO:tasks.workunit.client.0.smithi042.stdout:9/476: read d6/da/d1d/f38 [3107252,27931] 0 2024-02-15T18:26:58.408 INFO:tasks.workunit.client.0.smithi042.stdout:9/477: dread - d6/da/d2f/d69/f8a zero size 2024-02-15T18:26:58.410 INFO:tasks.workunit.client.0.smithi042.stdout:5/490: mkdir d0/d12/d65/d74/d52/dad 0 2024-02-15T18:26:58.413 INFO:tasks.workunit.client.1.smithi191.stdout:7/79: symlink d0/d1/d3/d11/l1b 0 2024-02-15T18:26:58.414 INFO:tasks.workunit.client.1.smithi191.stdout:7/80: truncate d0/d1/df/f1a 153691 0 2024-02-15T18:26:58.415 INFO:tasks.workunit.client.0.smithi042.stdout:9/478: write d6/da/f10 [2049484,128091] 0 2024-02-15T18:26:58.416 INFO:tasks.workunit.client.0.smithi042.stdout:7/489: truncate d10/f5b 2861166 0 2024-02-15T18:26:58.416 INFO:tasks.workunit.client.0.smithi042.stdout:8/368: rename dd/d11/c35 to dd/de/d33/d66/d4e/c69 0 2024-02-15T18:26:58.417 INFO:tasks.workunit.client.1.smithi191.stdout:8/245: mknod d3/d2c/d1a/d36/c53 0 2024-02-15T18:26:58.420 INFO:tasks.workunit.client.0.smithi042.stdout:4/431: link da/d11/d29/c3d da/d3b/c99 0 2024-02-15T18:26:58.429 INFO:tasks.workunit.client.0.smithi042.stdout:6/535: symlink d9/d11/lba 0 2024-02-15T18:26:58.430 INFO:tasks.workunit.client.0.smithi042.stdout:6/536: chown d9/d11/d19/d1c/d31/d9b/db4 6978 1 2024-02-15T18:26:58.430 INFO:tasks.workunit.client.0.smithi042.stdout:2/402: mkdir d0/d8/d20/d3f/d7c 0 2024-02-15T18:26:58.432 INFO:tasks.workunit.client.1.smithi191.stdout:2/98: rename df/f11 to df/f16 0 2024-02-15T18:26:58.434 INFO:tasks.workunit.client.1.smithi191.stdout:7/81: mkdir d0/d1/d3/d11/d1c 0 2024-02-15T18:26:58.436 INFO:tasks.workunit.client.1.smithi191.stdout:8/246: rename d3/d2c/d1a/d23/f42 to d3/d2c/d1a/d23/f54 0 2024-02-15T18:26:58.437 INFO:tasks.workunit.client.1.smithi191.stdout:7/82: mkdir d0/d1/d1d 0 2024-02-15T18:26:58.437 INFO:tasks.workunit.client.1.smithi191.stdout:7/83: fdatasync d0/d1/d3/d11/f18 0 2024-02-15T18:26:58.439 INFO:tasks.workunit.client.1.smithi191.stdout:8/247: symlink d3/d2c/l55 0 2024-02-15T18:26:58.444 INFO:tasks.workunit.client.1.smithi191.stdout:7/84: rename d0/d1/d15 to d0/d1/d3/dc/d12/d1e 0 2024-02-15T18:26:58.444 INFO:tasks.workunit.client.1.smithi191.stdout:9/144: dwrite d0/d7/dc/d17/f25 [0,4194304] 0 2024-02-15T18:26:58.444 INFO:tasks.workunit.client.1.smithi191.stdout:8/248: creat d3/d2c/d1a/d23/d2e/f56 x:0 0 0 2024-02-15T18:26:58.446 INFO:tasks.workunit.client.1.smithi191.stdout:1/131: dwrite d0/f5 [4194304,4194304] 0 2024-02-15T18:26:58.446 INFO:tasks.workunit.client.1.smithi191.stdout:1/132: read d0/d8/f17 [206706,81430] 0 2024-02-15T18:26:58.448 INFO:tasks.workunit.client.0.smithi042.stdout:5/491: creat d0/d50/d98/fae x:0 0 0 2024-02-15T18:26:58.449 INFO:tasks.workunit.client.0.smithi042.stdout:0/448: rename d4/d38/d77/d2e/l45 to d4/d9/d5a/d60/l97 0 2024-02-15T18:26:58.449 INFO:tasks.workunit.client.0.smithi042.stdout:0/449: chown d4/d9/d34/d57 14172 1 2024-02-15T18:26:58.451 INFO:tasks.workunit.client.0.smithi042.stdout:4/432: mknod da/d11/d29/c9a 0 2024-02-15T18:26:58.452 INFO:tasks.workunit.client.1.smithi191.stdout:9/145: creat d0/d29/f36 x:0 0 0 2024-02-15T18:26:58.454 INFO:tasks.workunit.client.0.smithi042.stdout:2/403: unlink d0/d8/d10/c4b 0 2024-02-15T18:26:58.455 INFO:tasks.workunit.client.1.smithi191.stdout:8/249: symlink d3/d2c/d1a/d23/d3e/l57 0 2024-02-15T18:26:58.455 INFO:tasks.workunit.client.0.smithi042.stdout:9/479: dwrite d6/da/d2f/d6c/f94 [0,4194304] 0 2024-02-15T18:26:58.456 INFO:tasks.workunit.client.0.smithi042.stdout:8/369: mknod dd/d11/c6a 0 2024-02-15T18:26:58.456 INFO:tasks.workunit.client.0.smithi042.stdout:7/490: dwrite d10/d43/d57/f66 [0,4194304] 0 2024-02-15T18:26:58.457 INFO:tasks.workunit.client.0.smithi042.stdout:7/491: write d10/d49/d6d/f85 [939370,56489] 0 2024-02-15T18:26:58.459 INFO:tasks.workunit.client.1.smithi191.stdout:1/133: creat d0/d8/df/d12/f28 x:0 0 0 2024-02-15T18:26:58.460 INFO:tasks.workunit.client.1.smithi191.stdout:1/134: chown d0/f1b 709937924 1 2024-02-15T18:26:58.461 INFO:tasks.workunit.client.0.smithi042.stdout:5/492: getdents d0/d12/d2a 0 2024-02-15T18:26:58.463 INFO:tasks.workunit.client.0.smithi042.stdout:7/492: write d10/d15/f34 [248036,78268] 0 2024-02-15T18:26:58.464 INFO:tasks.workunit.client.1.smithi191.stdout:0/58: sync 2024-02-15T18:26:58.464 INFO:tasks.workunit.client.1.smithi191.stdout:0/59: stat d7/lc 0 2024-02-15T18:26:58.464 INFO:tasks.workunit.client.1.smithi191.stdout:0/60: chown l2 739489430 1 2024-02-15T18:26:58.464 INFO:tasks.workunit.client.1.smithi191.stdout:0/61: write d7/fb [1359567,56349] 0 2024-02-15T18:26:58.464 INFO:tasks.workunit.client.1.smithi191.stdout:0/62: chown d7/lc 3311 1 2024-02-15T18:26:58.465 INFO:tasks.workunit.client.1.smithi191.stdout:2/99: dwrite df/f10 [0,4194304] 0 2024-02-15T18:26:58.466 INFO:tasks.workunit.client.1.smithi191.stdout:9/146: unlink d0/c2 0 2024-02-15T18:26:58.467 INFO:tasks.workunit.client.0.smithi042.stdout:0/450: mknod d4/d38/d39/c98 0 2024-02-15T18:26:58.468 INFO:tasks.workunit.client.0.smithi042.stdout:6/537: dwrite d9/d11/d19/d1c/d31/f5b [0,4194304] 0 2024-02-15T18:26:58.470 INFO:tasks.workunit.client.1.smithi191.stdout:8/250: unlink d3/d2c/d1a/f3b 0 2024-02-15T18:26:58.472 INFO:tasks.workunit.client.0.smithi042.stdout:4/433: mknod da/d11/d29/d4a/d4f/c9b 0 2024-02-15T18:26:58.472 INFO:tasks.workunit.client.0.smithi042.stdout:4/434: write da/d11/d67/f95 [133289,115404] 0 2024-02-15T18:26:58.473 INFO:tasks.workunit.client.1.smithi191.stdout:1/135: creat d0/d8/f29 x:0 0 0 2024-02-15T18:26:58.473 INFO:tasks.workunit.client.1.smithi191.stdout:1/136: chown d0/l1e 803 1 2024-02-15T18:26:58.474 INFO:tasks.workunit.client.1.smithi191.stdout:7/85: dwrite d0/d1/df/f17 [4194304,4194304] 0 2024-02-15T18:26:58.474 INFO:tasks.workunit.client.1.smithi191.stdout:7/86: rename d0/d1 to d0/d1/d1d/d1f 22 2024-02-15T18:26:58.475 INFO:tasks.workunit.client.1.smithi191.stdout:2/100: symlink df/l17 0 2024-02-15T18:26:58.478 INFO:tasks.workunit.client.0.smithi042.stdout:2/404: mkdir d0/d8/d20/d7d 0 2024-02-15T18:26:58.480 INFO:tasks.workunit.client.1.smithi191.stdout:1/137: write d0/f5 [8298440,64234] 0 2024-02-15T18:26:58.481 INFO:tasks.workunit.client.1.smithi191.stdout:8/251: stat d3/d2c/d1a/d23/d2a/c33 0 2024-02-15T18:26:58.485 INFO:tasks.workunit.client.0.smithi042.stdout:9/480: getdents d6 0 2024-02-15T18:26:58.488 INFO:tasks.workunit.client.1.smithi191.stdout:7/87: symlink d0/d1/d3/d11/d1c/l20 0 2024-02-15T18:26:58.488 INFO:tasks.workunit.client.1.smithi191.stdout:7/88: write d0/d1/f9 [2037777,4183] 0 2024-02-15T18:26:58.491 INFO:tasks.workunit.client.1.smithi191.stdout:2/101: creat df/f18 x:0 0 0 2024-02-15T18:26:58.492 INFO:tasks.workunit.client.0.smithi042.stdout:2/405: dread d0/d8/db/f3e [0,4194304] 0 2024-02-15T18:26:58.492 INFO:tasks.workunit.client.0.smithi042.stdout:0/451: symlink d4/d9/d34/d57/l99 0 2024-02-15T18:26:58.493 INFO:tasks.workunit.client.1.smithi191.stdout:1/138: truncate d0/d8/df/d12/f23 3481257 0 2024-02-15T18:26:58.495 INFO:tasks.workunit.client.0.smithi042.stdout:4/435: creat da/f9c x:0 0 0 2024-02-15T18:26:58.495 INFO:tasks.workunit.client.0.smithi042.stdout:7/493: dwrite d10/d12/d23/f61 [0,4194304] 0 2024-02-15T18:26:58.495 INFO:tasks.workunit.client.0.smithi042.stdout:7/494: truncate d10/d49/d6d/d78/d83/f87 849553 0 2024-02-15T18:26:58.495 INFO:tasks.workunit.client.0.smithi042.stdout:5/493: dwrite d0/db/f1f [0,4194304] 0 2024-02-15T18:26:58.495 INFO:tasks.workunit.client.1.smithi191.stdout:1/139: read d0/d8/f18 [404896,98843] 0 2024-02-15T18:26:58.496 INFO:tasks.workunit.client.1.smithi191.stdout:0/63: dwrite d7/fb [0,4194304] 0 2024-02-15T18:26:58.496 INFO:tasks.workunit.client.1.smithi191.stdout:0/64: chown d7/lc 14 1 2024-02-15T18:26:58.496 INFO:tasks.workunit.client.1.smithi191.stdout:8/252: link d3/d2c/d1a/c2f d3/d2c/d1a/d23/d2a/c58 0 2024-02-15T18:26:58.497 INFO:tasks.workunit.client.1.smithi191.stdout:7/89: creat d0/d1/f21 x:0 0 0 2024-02-15T18:26:58.500 INFO:tasks.workunit.client.1.smithi191.stdout:2/102: rename df/d12/d13 to df/d12/d19 0 2024-02-15T18:26:58.500 INFO:tasks.workunit.client.1.smithi191.stdout:8/253: dread d3/d2c/f17 [0,4194304] 0 2024-02-15T18:26:58.504 INFO:tasks.workunit.client.0.smithi042.stdout:9/481: symlink d6/da/d2f/d55/d73/d8d/la7 0 2024-02-15T18:26:58.508 INFO:tasks.workunit.client.1.smithi191.stdout:0/65: symlink d7/dd/lf 0 2024-02-15T18:26:58.513 INFO:tasks.workunit.client.0.smithi042.stdout:8/370: symlink dd/d11/l6b 0 2024-02-15T18:26:58.513 INFO:tasks.workunit.client.0.smithi042.stdout:8/371: chown dd/de/d33/f53 18526949 1 2024-02-15T18:26:58.513 INFO:tasks.workunit.client.0.smithi042.stdout:8/372: write dd/de/d3c/d4b/d67/f43 [359196,101298] 0 2024-02-15T18:26:58.514 INFO:tasks.workunit.client.1.smithi191.stdout:2/103: mkdir df/d12/d19/d1a 0 2024-02-15T18:26:58.515 INFO:tasks.workunit.client.0.smithi042.stdout:2/406: rename d0/d8/d10/d61 to d0/d8/d20/d7d/d7e 0 2024-02-15T18:26:58.515 INFO:tasks.workunit.client.0.smithi042.stdout:2/407: readlink d0/d8/d10/d31/l72 0 2024-02-15T18:26:58.515 INFO:tasks.workunit.client.0.smithi042.stdout:2/408: read - d0/d8/f70 zero size 2024-02-15T18:26:58.516 INFO:tasks.workunit.client.1.smithi191.stdout:8/254: rename d3/d2c/d1a/d23/l24 to d3/d2c/d1a/d23/d3e/l59 0 2024-02-15T18:26:58.519 INFO:tasks.workunit.client.0.smithi042.stdout:7/495: creat d10/d49/fa9 x:0 0 0 2024-02-15T18:26:58.521 INFO:tasks.workunit.client.1.smithi191.stdout:2/104: rename df/f18 to df/f1b 0 2024-02-15T18:26:58.523 INFO:tasks.workunit.client.1.smithi191.stdout:0/66: rename c3 to d7/da/c10 0 2024-02-15T18:26:58.525 INFO:tasks.workunit.client.1.smithi191.stdout:8/255: symlink d3/d2c/d1a/d36/l5a 0 2024-02-15T18:26:58.525 INFO:tasks.workunit.client.1.smithi191.stdout:8/256: chown d3/d2c/d1a/d23/d2e/d38 413709137 1 2024-02-15T18:26:58.526 INFO:tasks.workunit.client.1.smithi191.stdout:2/105: creat df/d12/d19/d1a/f1c x:0 0 0 2024-02-15T18:26:58.528 INFO:tasks.workunit.client.1.smithi191.stdout:0/67: stat d7/da/c10 0 2024-02-15T18:26:58.528 INFO:tasks.workunit.client.1.smithi191.stdout:7/90: dwrite d0/d1/df/f19 [0,4194304] 0 2024-02-15T18:26:58.529 INFO:tasks.workunit.client.0.smithi042.stdout:5/494: dwrite d0/d38/d53/d68/d6a/f85 [0,4194304] 0 2024-02-15T18:26:58.530 INFO:tasks.workunit.client.1.smithi191.stdout:1/140: dwrite d0/db/f26 [0,4194304] 0 2024-02-15T18:26:58.531 INFO:tasks.workunit.client.0.smithi042.stdout:5/495: write d0/db/f10 [2447481,61194] 0 2024-02-15T18:26:58.531 INFO:tasks.workunit.client.0.smithi042.stdout:5/496: write d0/d12/d65/d74/d52/fa7 [512096,44173] 0 2024-02-15T18:26:58.531 INFO:tasks.workunit.client.0.smithi042.stdout:5/497: fsync d0/d38/d53/d68/d6a/d42/f95 0 2024-02-15T18:26:58.532 INFO:tasks.workunit.client.0.smithi042.stdout:9/482: creat d6/fa8 x:0 0 0 2024-02-15T18:26:58.532 INFO:tasks.workunit.client.1.smithi191.stdout:8/257: mkdir d3/d2c/d1a/d23/d2e/d5b 0 2024-02-15T18:26:58.533 INFO:tasks.workunit.client.1.smithi191.stdout:8/258: chown d3/fc 39088916 1 2024-02-15T18:26:58.533 INFO:tasks.workunit.client.1.smithi191.stdout:8/259: chown d3/d2c/d1a/d23/d3e 229 1 2024-02-15T18:26:58.533 INFO:tasks.workunit.client.1.smithi191.stdout:8/260: dread d3/d2c/f1f [0,4194304] 0 2024-02-15T18:26:58.534 INFO:tasks.workunit.client.0.smithi042.stdout:8/373: creat dd/de/d33/d66/d4e/f6c x:0 0 0 2024-02-15T18:26:58.535 INFO:tasks.workunit.client.1.smithi191.stdout:2/106: rmdir df/d12/d19/d1a 39 2024-02-15T18:26:58.536 INFO:tasks.workunit.client.0.smithi042.stdout:9/483: dread d6/da/d2f/d55/d73/f77 [0,4194304] 0 2024-02-15T18:26:58.536 INFO:tasks.workunit.client.0.smithi042.stdout:9/484: chown d6/da/d2f/d7c 554 1 2024-02-15T18:26:58.551 INFO:tasks.workunit.client.0.smithi042.stdout:4/436: rename f9 to da/d11/d29/d4a/d4f/f9d 0 2024-02-15T18:26:58.551 INFO:tasks.workunit.client.0.smithi042.stdout:2/409: unlink d0/d8/db/f3e 0 2024-02-15T18:26:58.551 INFO:tasks.workunit.client.1.smithi191.stdout:8/261: creat d3/d2c/d1a/d23/f5c x:0 0 0 2024-02-15T18:26:58.552 INFO:tasks.workunit.client.0.smithi042.stdout:7/496: dread d10/d15/f34 [0,4194304] 0 2024-02-15T18:26:58.552 INFO:tasks.workunit.client.0.smithi042.stdout:7/497: read - d10/d49/f8e zero size 2024-02-15T18:26:58.552 INFO:tasks.workunit.client.0.smithi042.stdout:7/498: readlink d10/d12/d23/d51/l98 0 2024-02-15T18:26:58.553 INFO:tasks.workunit.client.0.smithi042.stdout:5/498: rmdir d0/d22/d7b/da9 39 2024-02-15T18:26:58.553 INFO:tasks.workunit.client.1.smithi191.stdout:2/107: getdents df/d12/d19/d1a 0 2024-02-15T18:26:58.556 INFO:tasks.workunit.client.1.smithi191.stdout:2/108: unlink l6 0 2024-02-15T18:26:58.557 INFO:tasks.workunit.client.0.smithi042.stdout:4/437: creat da/d2b/f9e x:0 0 0 2024-02-15T18:26:58.558 INFO:tasks.workunit.client.0.smithi042.stdout:2/410: creat d0/d40/d56/f7f x:0 0 0 2024-02-15T18:26:58.558 INFO:tasks.workunit.client.1.smithi191.stdout:2/109: truncate f9 1116334 0 2024-02-15T18:26:58.559 INFO:tasks.workunit.client.1.smithi191.stdout:2/110: chown f9 7532519 1 2024-02-15T18:26:58.559 INFO:tasks.workunit.client.1.smithi191.stdout:2/111: chown df/f16 387 1 2024-02-15T18:26:58.562 INFO:tasks.workunit.client.1.smithi191.stdout:2/112: symlink df/d12/l1d 0 2024-02-15T18:26:58.562 INFO:tasks.workunit.client.1.smithi191.stdout:2/113: fsync fd 0 2024-02-15T18:26:58.562 INFO:tasks.workunit.client.0.smithi042.stdout:9/485: dwrite d6/dd/d9/f41 [0,4194304] 0 2024-02-15T18:26:58.562 INFO:tasks.workunit.client.0.smithi042.stdout:9/486: chown d6/dd 5957 1 2024-02-15T18:26:58.562 INFO:tasks.workunit.client.0.smithi042.stdout:9/487: fdatasync d6/da/f10 0 2024-02-15T18:26:58.562 INFO:tasks.workunit.client.0.smithi042.stdout:5/499: mkdir d0/d12/d65/daf 0 2024-02-15T18:26:58.562 INFO:tasks.workunit.client.0.smithi042.stdout:4/438: creat da/d11/d67/d5c/d66/d78/d53/f9f x:0 0 0 2024-02-15T18:26:58.563 INFO:tasks.workunit.client.0.smithi042.stdout:2/411: rename d0/d8/d10/d31/d3a/d53/d3c/f30 to d0/d8/db/f80 0 2024-02-15T18:26:58.564 INFO:tasks.workunit.client.1.smithi191.stdout:7/91: dwrite d0/d1/f9 [0,4194304] 0 2024-02-15T18:26:58.566 INFO:tasks.workunit.client.1.smithi191.stdout:0/68: write d7/fb [5059854,33427] 0 2024-02-15T18:26:58.566 INFO:tasks.workunit.client.0.smithi042.stdout:8/374: rmdir dd/de/d33 39 2024-02-15T18:26:58.567 INFO:tasks.workunit.client.1.smithi191.stdout:1/141: dwrite d0/d8/f18 [0,4194304] 0 2024-02-15T18:26:58.568 INFO:tasks.workunit.client.1.smithi191.stdout:2/114: rmdir df/d12/d19 39 2024-02-15T18:26:58.568 INFO:tasks.workunit.client.0.smithi042.stdout:8/375: dread dd/de/d33/d3b/f46 [0,4194304] 0 2024-02-15T18:26:58.569 INFO:tasks.workunit.client.0.smithi042.stdout:8/376: chown dd/de/d33/d66/d4e/f28 8 1 2024-02-15T18:26:58.569 INFO:tasks.workunit.client.0.smithi042.stdout:8/377: read - dd/f3e zero size 2024-02-15T18:26:58.569 INFO:tasks.workunit.client.1.smithi191.stdout:0/69: fdatasync d7/fb 0 2024-02-15T18:26:58.574 INFO:tasks.workunit.client.1.smithi191.stdout:8/262: dwrite d3/f31 [0,4194304] 0 2024-02-15T18:26:58.578 INFO:tasks.workunit.client.0.smithi042.stdout:7/499: dwrite d10/d15/d39/d6c/f6f [0,4194304] 0 2024-02-15T18:26:58.580 INFO:tasks.workunit.client.0.smithi042.stdout:3/465: sync 2024-02-15T18:26:58.580 INFO:tasks.workunit.client.0.smithi042.stdout:1/482: sync 2024-02-15T18:26:58.581 INFO:tasks.workunit.client.1.smithi191.stdout:7/92: creat d0/d1/d3/f22 x:0 0 0 2024-02-15T18:26:58.584 INFO:tasks.workunit.client.1.smithi191.stdout:0/70: dwrite d7/fb [4194304,4194304] 0 2024-02-15T18:26:58.585 INFO:tasks.workunit.client.0.smithi042.stdout:9/488: mknod d6/da/d2f/d6c/ca9 0 2024-02-15T18:26:58.585 INFO:tasks.workunit.client.0.smithi042.stdout:5/500: mkdir d0/d38/d53/d91/db0 0 2024-02-15T18:26:58.585 INFO:tasks.workunit.client.0.smithi042.stdout:6/538: sync 2024-02-15T18:26:58.585 INFO:tasks.workunit.client.0.smithi042.stdout:5/501: getdents d0/d38/d53/d91/db0 0 2024-02-15T18:26:58.585 INFO:tasks.workunit.client.0.smithi042.stdout:4/439: creat da/d11/d29/d4a/fa0 x:0 0 0 2024-02-15T18:26:58.585 INFO:tasks.workunit.client.1.smithi191.stdout:1/142: mkdir d0/d8/df/d12/d2a 0 2024-02-15T18:26:58.586 INFO:tasks.workunit.client.0.smithi042.stdout:2/412: creat d0/d35/f81 x:0 0 0 2024-02-15T18:26:58.586 INFO:tasks.workunit.client.0.smithi042.stdout:2/413: write d0/d8/db/de/d1a/f43 [983068,29672] 0 2024-02-15T18:26:58.587 INFO:tasks.workunit.client.1.smithi191.stdout:1/143: write d0/f1b [149060,41767] 0 2024-02-15T18:26:58.587 INFO:tasks.workunit.client.1.smithi191.stdout:2/115: mkdir df/d12/d1e 0 2024-02-15T18:26:58.587 INFO:tasks.workunit.client.1.smithi191.stdout:2/116: truncate fe 1480470 0 2024-02-15T18:26:58.587 INFO:tasks.workunit.client.1.smithi191.stdout:2/117: readlink df/d12/l1d 0 2024-02-15T18:26:58.587 INFO:tasks.workunit.client.0.smithi042.stdout:8/378: creat dd/de/d3c/f6d x:0 0 0 2024-02-15T18:26:58.588 INFO:tasks.workunit.client.0.smithi042.stdout:8/379: fdatasync dd/de/d33/d66/d4e/f28 0 2024-02-15T18:26:58.591 INFO:tasks.workunit.client.0.smithi042.stdout:5/502: dread d0/d14/f28 [0,4194304] 0 2024-02-15T18:26:58.600 INFO:tasks.workunit.client.0.smithi042.stdout:9/489: dwrite d6/da/d1d/d2d/d40/f43 [0,4194304] 0 2024-02-15T18:26:58.601 INFO:tasks.workunit.client.0.smithi042.stdout:9/490: chown d6/da/d2f/d6c/f71 629788 1 2024-02-15T18:26:58.601 INFO:tasks.workunit.client.0.smithi042.stdout:9/491: chown d6/da/d2f 652905 1 2024-02-15T18:26:58.618 INFO:tasks.workunit.client.0.smithi042.stdout:9/492: dwrite d6/da/d1d/f58 [0,4194304] 0 2024-02-15T18:26:58.618 INFO:tasks.workunit.client.0.smithi042.stdout:9/493: dread - d6/da/d2f/d69/f74 zero size 2024-02-15T18:26:58.618 INFO:tasks.workunit.client.0.smithi042.stdout:7/500: creat d10/d15/d8f/da5/faa x:0 0 0 2024-02-15T18:26:58.618 INFO:tasks.workunit.client.0.smithi042.stdout:7/501: write d10/d15/f34 [2792373,54457] 0 2024-02-15T18:26:58.621 INFO:tasks.workunit.client.1.smithi191.stdout:8/263: mkdir d3/d2c/d1a/d23/d2e/d38/d4f/d5d 0 2024-02-15T18:26:58.621 INFO:tasks.workunit.client.0.smithi042.stdout:3/466: mkdir d1/d9c 0 2024-02-15T18:26:58.622 INFO:tasks.workunit.client.1.smithi191.stdout:7/93: link d0/d1/d3/ce d0/d1/d1d/c23 0 2024-02-15T18:26:58.623 INFO:tasks.workunit.client.0.smithi042.stdout:1/483: creat de/d15/d2c/d51/d70/faf x:0 0 0 2024-02-15T18:26:58.624 INFO:tasks.workunit.client.0.smithi042.stdout:6/539: symlink d9/d11/d19/lbb 0 2024-02-15T18:26:58.625 INFO:tasks.workunit.client.1.smithi191.stdout:1/144: mkdir d0/d8/df/d12/d2a/d2b 0 2024-02-15T18:26:58.625 INFO:tasks.workunit.client.1.smithi191.stdout:1/145: chown d0/d8/fd 8681 1 2024-02-15T18:26:58.627 INFO:tasks.workunit.client.1.smithi191.stdout:2/118: mkdir df/d12/d1f 0 2024-02-15T18:26:58.628 INFO:tasks.workunit.client.0.smithi042.stdout:4/440: creat da/d11/d29/d4a/d4f/d80/fa1 x:0 0 0 2024-02-15T18:26:58.628 INFO:tasks.workunit.client.0.smithi042.stdout:4/441: stat da/d11/d67/f51 0 2024-02-15T18:26:58.628 INFO:tasks.workunit.client.0.smithi042.stdout:4/442: dread - da/d11/d67/d5c/d66/d78/d2f/d4d/f96 zero size 2024-02-15T18:26:58.630 INFO:tasks.workunit.client.0.smithi042.stdout:2/414: mknod d0/d8/c82 0 2024-02-15T18:26:58.631 INFO:tasks.workunit.client.0.smithi042.stdout:8/380: rmdir dd/d11 39 2024-02-15T18:26:58.631 INFO:tasks.workunit.client.0.smithi042.stdout:8/381: fdatasync dd/f13 0 2024-02-15T18:26:58.631 INFO:tasks.workunit.client.0.smithi042.stdout:8/382: readlink dd/de/d3c/d4b/d67/l38 0 2024-02-15T18:26:58.632 INFO:tasks.workunit.client.0.smithi042.stdout:8/383: read dd/de/d3c/d4b/d67/f2b [138023,63247] 0 2024-02-15T18:26:58.632 INFO:tasks.workunit.client.0.smithi042.stdout:5/503: creat d0/d14/d88/da1/fb1 x:0 0 0 2024-02-15T18:26:58.632 INFO:tasks.workunit.client.0.smithi042.stdout:5/504: chown d0/d22/c70 5245 1 2024-02-15T18:26:58.632 INFO:tasks.workunit.client.0.smithi042.stdout:5/505: chown d0/d38/d53/d91/d94 49170 1 2024-02-15T18:26:58.632 INFO:tasks.workunit.client.0.smithi042.stdout:5/506: chown d0/d12/d65/d74/d52/l58 2655667 1 2024-02-15T18:26:58.633 INFO:tasks.workunit.client.1.smithi191.stdout:7/94: mknod d0/d1/d3/d11/c24 0 2024-02-15T18:26:58.633 INFO:tasks.workunit.client.0.smithi042.stdout:5/507: write d0/d14/fa3 [2231594,62208] 0 2024-02-15T18:26:58.633 INFO:tasks.workunit.client.0.smithi042.stdout:5/508: truncate d0/d38/d53/d64/f81 418674 0 2024-02-15T18:26:58.634 INFO:tasks.workunit.client.1.smithi191.stdout:1/146: creat d0/d8/df/f2c x:0 0 0 2024-02-15T18:26:58.636 INFO:tasks.workunit.client.1.smithi191.stdout:2/119: mknod df/d12/d1e/c20 0 2024-02-15T18:26:58.636 INFO:tasks.workunit.client.1.smithi191.stdout:2/120: write df/f16 [710480,10377] 0 2024-02-15T18:26:58.640 INFO:tasks.workunit.client.1.smithi191.stdout:8/264: mkdir d3/d5e 0 2024-02-15T18:26:58.642 INFO:tasks.workunit.client.1.smithi191.stdout:7/95: rmdir d0/d1/df 39 2024-02-15T18:26:58.643 INFO:tasks.workunit.client.0.smithi042.stdout:9/494: mknod d6/da/d7b/caa 0 2024-02-15T18:26:58.643 INFO:tasks.workunit.client.1.smithi191.stdout:1/147: creat d0/db/d10/f2d x:0 0 0 2024-02-15T18:26:58.644 INFO:tasks.workunit.client.0.smithi042.stdout:9/495: write d6/da/d1d/f38 [202816,5915] 0 2024-02-15T18:26:58.645 INFO:tasks.workunit.client.0.smithi042.stdout:7/502: link c2 d10/d15/d39/d6c/cab 0 2024-02-15T18:26:58.646 INFO:tasks.workunit.client.1.smithi191.stdout:2/121: creat df/d12/d19/d1a/f21 x:0 0 0 2024-02-15T18:26:58.647 INFO:tasks.workunit.client.0.smithi042.stdout:3/467: link d1/d11/d14/d4d/d6c/f83 d1/d11/d14/d4d/d73/f9d 0 2024-02-15T18:26:58.649 INFO:tasks.workunit.client.1.smithi191.stdout:8/265: rename d3/d2c/d1a/d23/d2e/d38 to d3/d2c/d1a/d23/d2e/d47/d5f 0 2024-02-15T18:26:58.649 INFO:tasks.workunit.client.0.smithi042.stdout:1/484: creat de/d54/fb0 x:0 0 0 2024-02-15T18:26:58.650 INFO:tasks.workunit.client.0.smithi042.stdout:1/485: dread - de/f48 zero size 2024-02-15T18:26:58.651 INFO:tasks.workunit.client.1.smithi191.stdout:1/148: getdents d0/d8/df/d12 0 2024-02-15T18:26:58.651 INFO:tasks.workunit.client.1.smithi191.stdout:1/149: fdatasync d0/d8/f29 0 2024-02-15T18:26:58.653 INFO:tasks.workunit.client.1.smithi191.stdout:1/150: truncate d0/db/f1d 659760 0 2024-02-15T18:26:58.653 INFO:tasks.workunit.client.1.smithi191.stdout:1/151: truncate d0/db/fe 686526 0 2024-02-15T18:26:58.654 INFO:tasks.workunit.client.0.smithi042.stdout:4/443: rename da/d2b/d6e/c79 to da/d11/d29/d4a/ca2 0 2024-02-15T18:26:58.655 INFO:tasks.workunit.client.0.smithi042.stdout:4/444: write da/d11/f22 [1503455,9091] 0 2024-02-15T18:26:58.655 INFO:tasks.workunit.client.0.smithi042.stdout:4/445: dread - da/d2b/f9e zero size 2024-02-15T18:26:58.655 INFO:tasks.workunit.client.1.smithi191.stdout:1/152: mkdir d0/d8/df/d2e 0 2024-02-15T18:26:58.655 INFO:tasks.workunit.client.0.smithi042.stdout:4/446: write da/d11/d67/d5c/d66/d78/f1c [7231025,66213] 0 2024-02-15T18:26:58.657 INFO:tasks.workunit.client.1.smithi191.stdout:2/122: dread df/f10 [0,4194304] 0 2024-02-15T18:26:58.658 INFO:tasks.workunit.client.0.smithi042.stdout:8/384: creat dd/de/d3c/d4b/d67/f6e x:0 0 0 2024-02-15T18:26:58.661 INFO:tasks.workunit.client.1.smithi191.stdout:7/96: dwrite d0/d1/d3/f22 [0,4194304] 0 2024-02-15T18:26:58.661 INFO:tasks.workunit.client.1.smithi191.stdout:2/123: rename fe to df/d12/f22 0 2024-02-15T18:26:58.662 INFO:tasks.workunit.client.0.smithi042.stdout:9/496: symlink d6/da/d1d/d2d/d40/d5f/lab 0 2024-02-15T18:26:58.664 INFO:tasks.workunit.client.0.smithi042.stdout:5/509: dwrite d0/d38/f5a [0,4194304] 0 2024-02-15T18:26:58.666 INFO:tasks.workunit.client.0.smithi042.stdout:7/503: unlink d10/d43/f4e 0 2024-02-15T18:26:58.666 INFO:tasks.workunit.client.0.smithi042.stdout:7/504: write d10/f53 [4045506,107738] 0 2024-02-15T18:26:58.668 INFO:tasks.workunit.client.1.smithi191.stdout:1/153: dread d0/f4 [0,4194304] 0 2024-02-15T18:26:58.668 INFO:tasks.workunit.client.1.smithi191.stdout:1/154: fdatasync d0/f9 0 2024-02-15T18:26:58.668 INFO:tasks.workunit.client.1.smithi191.stdout:1/155: chown d0/d8/df 31586351 1 2024-02-15T18:26:58.668 INFO:tasks.workunit.client.1.smithi191.stdout:1/156: fdatasync d0/d8/f17 0 2024-02-15T18:26:58.673 INFO:tasks.workunit.client.0.smithi042.stdout:0/452: sync 2024-02-15T18:26:58.674 INFO:tasks.workunit.client.0.smithi042.stdout:3/468: creat d1/de/d1a/f9e x:0 0 0 2024-02-15T18:26:58.675 INFO:tasks.workunit.client.0.smithi042.stdout:1/486: fdatasync de/d15/d1b/f49 0 2024-02-15T18:26:58.676 INFO:tasks.workunit.client.0.smithi042.stdout:2/415: truncate d0/d8/db/de/d1a/f43 76250 0 2024-02-15T18:26:58.676 INFO:tasks.workunit.client.0.smithi042.stdout:2/416: stat d0/d40/d56/f7f 0 2024-02-15T18:26:58.677 INFO:tasks.workunit.client.1.smithi191.stdout:1/157: mkdir d0/d2f 0 2024-02-15T18:26:58.677 INFO:tasks.workunit.client.1.smithi191.stdout:1/158: read - d0/d8/df/d12/f28 zero size 2024-02-15T18:26:58.678 INFO:tasks.workunit.client.0.smithi042.stdout:8/385: mkdir dd/de/d3c/d4b/d67/d6f 0 2024-02-15T18:26:58.679 INFO:tasks.workunit.client.0.smithi042.stdout:9/497: creat d6/da2/fac x:0 0 0 2024-02-15T18:26:58.679 INFO:tasks.workunit.client.0.smithi042.stdout:8/386: stat dd/de/d33/d66/f3f 0 2024-02-15T18:26:58.681 INFO:tasks.workunit.client.0.smithi042.stdout:8/387: dread dd/de/d12/f4c [0,4194304] 0 2024-02-15T18:26:58.681 INFO:tasks.workunit.client.0.smithi042.stdout:8/388: fdatasync dd/d11/f15 0 2024-02-15T18:26:58.681 INFO:tasks.workunit.client.0.smithi042.stdout:5/510: symlink d0/d39/lb2 0 2024-02-15T18:26:58.681 INFO:tasks.workunit.client.0.smithi042.stdout:0/453: mknod d4/d38/d77/c9a 0 2024-02-15T18:26:58.681 INFO:tasks.workunit.client.0.smithi042.stdout:7/505: mknod d10/d15/d8f/da5/cac 0 2024-02-15T18:26:58.681 INFO:tasks.workunit.client.0.smithi042.stdout:7/506: fsync d10/d43/f84 0 2024-02-15T18:26:58.681 INFO:tasks.workunit.client.0.smithi042.stdout:5/511: read d0/db/fd [3218074,2778] 0 2024-02-15T18:26:58.683 INFO:tasks.workunit.client.0.smithi042.stdout:3/469: symlink d1/d9c/l9f 0 2024-02-15T18:26:58.683 INFO:tasks.workunit.client.0.smithi042.stdout:1/487: creat de/d15/d67/d66/d7c/fb1 x:0 0 0 2024-02-15T18:26:58.683 INFO:tasks.workunit.client.0.smithi042.stdout:3/470: readlink d1/d11/l61 0 2024-02-15T18:26:58.683 INFO:tasks.workunit.client.0.smithi042.stdout:1/488: truncate de/d6d/fa3 460772 0 2024-02-15T18:26:58.683 INFO:tasks.workunit.client.0.smithi042.stdout:2/417: mknod d0/d8/d20/c83 0 2024-02-15T18:26:58.684 INFO:tasks.workunit.client.0.smithi042.stdout:7/507: mkdir d10/d15/d8f/da5/dad 0 2024-02-15T18:26:58.684 INFO:tasks.workunit.client.0.smithi042.stdout:1/489: write de/d15/d67/f3a [3671343,42599] 0 2024-02-15T18:26:58.685 INFO:tasks.workunit.client.1.smithi191.stdout:7/97: dwrite d0/d1/f9 [4194304,4194304] 0 2024-02-15T18:26:58.685 INFO:tasks.workunit.client.0.smithi042.stdout:4/447: dwrite da/f42 [0,4194304] 0 2024-02-15T18:26:58.685 INFO:tasks.workunit.client.0.smithi042.stdout:4/448: readlink da/d3b/l6a 0 2024-02-15T18:26:58.685 INFO:tasks.workunit.client.0.smithi042.stdout:5/512: symlink d0/d12/d65/d74/d52/dad/lb3 0 2024-02-15T18:26:58.694 INFO:tasks.workunit.client.0.smithi042.stdout:0/454: dread d4/d38/f3a [0,4194304] 0 2024-02-15T18:26:58.694 INFO:tasks.workunit.client.0.smithi042.stdout:3/471: rmdir d1/d11/d14/d4d/d73 39 2024-02-15T18:26:58.695 INFO:tasks.workunit.client.0.smithi042.stdout:3/472: fsync d1/d11/d2e/f3b 0 2024-02-15T18:26:58.696 INFO:tasks.workunit.client.1.smithi191.stdout:1/159: dread d0/db/f26 [0,4194304] 0 2024-02-15T18:26:58.699 INFO:tasks.workunit.client.0.smithi042.stdout:2/418: rmdir d0/d8/d10 39 2024-02-15T18:26:58.699 INFO:tasks.workunit.client.0.smithi042.stdout:2/419: chown d0/d8/db/de/d1a/f43 17 1 2024-02-15T18:26:58.699 INFO:tasks.workunit.client.0.smithi042.stdout:2/420: write d0/d8/d10/d31/d3a/d53/f58 [1293219,96866] 0 2024-02-15T18:26:58.706 INFO:tasks.workunit.client.0.smithi042.stdout:2/421: dread d0/d8/db/de/f4d [4194304,4194304] 0 2024-02-15T18:26:58.706 INFO:tasks.workunit.client.0.smithi042.stdout:2/422: write d0/d35/f81 [508592,66472] 0 2024-02-15T18:26:58.706 INFO:tasks.workunit.client.0.smithi042.stdout:2/423: truncate d0/d40/d56/f7f 856487 0 2024-02-15T18:26:58.707 INFO:tasks.workunit.client.1.smithi191.stdout:1/160: creat d0/d8/f30 x:0 0 0 2024-02-15T18:26:58.708 INFO:tasks.workunit.client.1.smithi191.stdout:1/161: truncate d0/f4 2640738 0 2024-02-15T18:26:58.708 INFO:tasks.workunit.client.1.smithi191.stdout:1/162: readlink d0/l20 0 2024-02-15T18:26:58.709 INFO:tasks.workunit.client.0.smithi042.stdout:5/513: mkdir d0/d38/d53/db4 0 2024-02-15T18:26:58.709 INFO:tasks.workunit.client.0.smithi042.stdout:5/514: fdatasync d0/d12/d65/d74/f4c 0 2024-02-15T18:26:58.710 INFO:tasks.workunit.client.1.smithi191.stdout:1/163: symlink d0/d8/d27/l31 0 2024-02-15T18:26:58.712 INFO:tasks.workunit.client.0.smithi042.stdout:0/455: creat d4/d38/d77/f9b x:0 0 0 2024-02-15T18:26:58.713 INFO:tasks.workunit.client.0.smithi042.stdout:9/498: dwrite d6/da/d2f/d55/f47 [0,4194304] 0 2024-02-15T18:26:58.713 INFO:tasks.workunit.client.0.smithi042.stdout:5/515: dread d0/f23 [0,4194304] 0 2024-02-15T18:26:58.713 INFO:tasks.workunit.client.0.smithi042.stdout:8/389: dwrite dd/de/d3c/d4b/d67/f2e [0,4194304] 0 2024-02-15T18:26:58.714 INFO:tasks.workunit.client.1.smithi191.stdout:1/164: creat d0/d8/df/d12/f32 x:0 0 0 2024-02-15T18:26:58.716 INFO:tasks.workunit.client.0.smithi042.stdout:3/473: rename d1/d11/d2e/l9a to d1/d11/d7a/la0 0 2024-02-15T18:26:58.717 INFO:tasks.workunit.client.0.smithi042.stdout:7/508: rmdir d10/d15/d8f 39 2024-02-15T18:26:58.719 INFO:tasks.workunit.client.0.smithi042.stdout:1/490: creat de/d15/d1b/d44/fb2 x:0 0 0 2024-02-15T18:26:58.720 INFO:tasks.workunit.client.1.smithi191.stdout:6/178: sync 2024-02-15T18:26:58.721 INFO:tasks.workunit.client.1.smithi191.stdout:3/115: sync 2024-02-15T18:26:58.722 INFO:tasks.workunit.client.0.smithi042.stdout:6/540: sync 2024-02-15T18:26:58.724 INFO:tasks.workunit.client.0.smithi042.stdout:0/456: mknod d4/d38/d39/c9c 0 2024-02-15T18:26:58.724 INFO:tasks.workunit.client.0.smithi042.stdout:0/457: write d4/d38/d77/f7c [1781965,50396] 0 2024-02-15T18:26:58.724 INFO:tasks.workunit.client.1.smithi191.stdout:1/165: mkdir d0/d8/df/d12/d33 0 2024-02-15T18:26:58.725 INFO:tasks.workunit.client.0.smithi042.stdout:5/516: mkdir d0/d38/d53/d68/d6a/d42/d7d/db5 0 2024-02-15T18:26:58.725 INFO:tasks.workunit.client.0.smithi042.stdout:5/517: stat d0/d14/l27 0 2024-02-15T18:26:58.725 INFO:tasks.workunit.client.1.smithi191.stdout:3/116: stat d6/d1b/d1c/l24 0 2024-02-15T18:26:58.726 INFO:tasks.workunit.client.1.smithi191.stdout:3/117: dread d6/de/ff [0,4194304] 0 2024-02-15T18:26:58.727 INFO:tasks.workunit.client.1.smithi191.stdout:6/179: truncate f1 8196962 0 2024-02-15T18:26:58.727 INFO:tasks.workunit.client.0.smithi042.stdout:8/390: dwrite dd/de/d12/f68 [0,4194304] 0 2024-02-15T18:26:58.728 INFO:tasks.workunit.client.0.smithi042.stdout:0/458: write d4/d38/f3c [5770798,109904] 0 2024-02-15T18:26:58.729 INFO:tasks.workunit.client.0.smithi042.stdout:2/424: rename d0/d8/f28 to d0/d8/db/d44/f84 0 2024-02-15T18:26:58.729 INFO:tasks.workunit.client.1.smithi191.stdout:1/166: mknod d0/d2f/c34 0 2024-02-15T18:26:58.732 INFO:tasks.workunit.client.1.smithi191.stdout:6/180: dread d8/d19/f22 [0,4194304] 0 2024-02-15T18:26:58.732 INFO:tasks.workunit.client.1.smithi191.stdout:1/167: creat d0/d8/f35 x:0 0 0 2024-02-15T18:26:58.733 INFO:tasks.workunit.client.0.smithi042.stdout:7/509: symlink d10/d49/d6d/lae 0 2024-02-15T18:26:58.734 INFO:tasks.workunit.client.1.smithi191.stdout:1/168: write d0/db/f14 [5190151,73055] 0 2024-02-15T18:26:58.738 INFO:tasks.workunit.client.0.smithi042.stdout:6/541: mkdir d9/d11/d46/d89/dbc 0 2024-02-15T18:26:58.738 INFO:tasks.workunit.client.1.smithi191.stdout:6/181: symlink d8/d9/d2b/l2d 0 2024-02-15T18:26:58.740 INFO:tasks.workunit.client.0.smithi042.stdout:5/518: dread d0/d38/f5a [0,4194304] 0 2024-02-15T18:26:58.740 INFO:tasks.workunit.client.1.smithi191.stdout:6/182: creat d8/d9/d2b/f2e x:0 0 0 2024-02-15T18:26:58.741 INFO:tasks.workunit.client.1.smithi191.stdout:1/169: dread d0/db/f26 [0,4194304] 0 2024-02-15T18:26:58.741 INFO:tasks.workunit.client.1.smithi191.stdout:1/170: readlink d0/l1e 0 2024-02-15T18:26:58.741 INFO:tasks.workunit.client.0.smithi042.stdout:8/391: creat dd/de/d3c/f70 x:0 0 0 2024-02-15T18:26:58.742 INFO:tasks.workunit.client.0.smithi042.stdout:0/459: mknod d4/d9/d62/c9d 0 2024-02-15T18:26:58.742 INFO:tasks.workunit.client.0.smithi042.stdout:8/392: write dd/de/d3c/d4b/d67/f2b [2362493,25098] 0 2024-02-15T18:26:58.743 INFO:tasks.workunit.client.1.smithi191.stdout:3/118: dwrite d6/fd [0,4194304] 0 2024-02-15T18:26:58.743 INFO:tasks.workunit.client.1.smithi191.stdout:3/119: write d6/de/f2c [667154,61088] 0 2024-02-15T18:26:58.743 INFO:tasks.workunit.client.1.smithi191.stdout:3/120: truncate d6/de/f13 5125529 0 2024-02-15T18:26:58.746 INFO:tasks.workunit.client.0.smithi042.stdout:3/474: rename d1/d11/d14/f35 to d1/d11/d14/d4d/d6c/d7b/fa1 0 2024-02-15T18:26:58.747 INFO:tasks.workunit.client.1.smithi191.stdout:1/171: link d0/f3 d0/d8/d27/f36 0 2024-02-15T18:26:58.749 INFO:tasks.workunit.client.0.smithi042.stdout:7/510: mknod d10/d15/d39/caf 0 2024-02-15T18:26:58.750 INFO:tasks.workunit.client.0.smithi042.stdout:6/542: creat d9/d11/d19/fbd x:0 0 0 2024-02-15T18:26:58.750 INFO:tasks.workunit.client.0.smithi042.stdout:6/543: chown d9/d11/d19/fb5 15177 1 2024-02-15T18:26:58.750 INFO:tasks.workunit.client.0.smithi042.stdout:6/544: stat d9/d11/d19/f99 0 2024-02-15T18:26:58.750 INFO:tasks.workunit.client.0.smithi042.stdout:6/545: chown d9/d8d 856966 1 2024-02-15T18:26:58.750 INFO:tasks.workunit.client.0.smithi042.stdout:6/546: chown d9/d11/l2e 5 1 2024-02-15T18:26:58.751 INFO:tasks.workunit.client.1.smithi191.stdout:3/121: getdents d6 0 2024-02-15T18:26:58.751 INFO:tasks.workunit.client.1.smithi191.stdout:3/122: fsync d6/fa 0 2024-02-15T18:26:58.752 INFO:tasks.workunit.client.1.smithi191.stdout:6/183: mknod d8/d9/c2f 0 2024-02-15T18:26:58.755 INFO:tasks.workunit.client.0.smithi042.stdout:8/393: mknod dd/de/d33/d3b/c71 0 2024-02-15T18:26:58.755 INFO:tasks.workunit.client.0.smithi042.stdout:8/394: write dd/f3e [740799,50025] 0 2024-02-15T18:26:58.755 INFO:tasks.workunit.client.0.smithi042.stdout:8/395: chown dd/de/d33/f42 40131464 1 2024-02-15T18:26:58.755 INFO:tasks.workunit.client.0.smithi042.stdout:8/396: write dd/de/d33/f40 [465639,66771] 0 2024-02-15T18:26:58.756 INFO:tasks.workunit.client.1.smithi191.stdout:1/172: creat d0/d8/df/d12/d2a/d2b/f37 x:0 0 0 2024-02-15T18:26:58.756 INFO:tasks.workunit.client.1.smithi191.stdout:1/173: chown d0/d8/df/f2c 57180 1 2024-02-15T18:26:58.756 INFO:tasks.workunit.client.0.smithi042.stdout:2/425: rename d0/d8/d10/d31/f7a to d0/d8/d20/d41/f85 0 2024-02-15T18:26:58.759 INFO:tasks.workunit.client.0.smithi042.stdout:3/475: unlink d1/d11/c69 0 2024-02-15T18:26:58.759 INFO:tasks.workunit.client.0.smithi042.stdout:3/476: fdatasync d1/d11/d14/d4d/d6c/d7b/f98 0 2024-02-15T18:26:58.759 INFO:tasks.workunit.client.1.smithi191.stdout:1/174: dread d0/d8/df/d12/f23 [0,4194304] 0 2024-02-15T18:26:58.760 INFO:tasks.workunit.client.1.smithi191.stdout:3/123: unlink d6/l26 0 2024-02-15T18:26:58.767 INFO:tasks.workunit.client.1.smithi191.stdout:6/184: dread d8/d19/f26 [0,4194304] 0 2024-02-15T18:26:58.767 INFO:tasks.workunit.client.1.smithi191.stdout:6/185: write d8/d19/f22 [298494,115506] 0 2024-02-15T18:26:58.768 INFO:tasks.workunit.client.1.smithi191.stdout:3/124: truncate d6/fd 4799721 0 2024-02-15T18:26:58.770 INFO:tasks.workunit.client.1.smithi191.stdout:6/186: rmdir d8/d19 39 2024-02-15T18:26:58.770 INFO:tasks.workunit.client.0.smithi042.stdout:8/397: symlink dd/de/d12/l72 0 2024-02-15T18:26:58.770 INFO:tasks.workunit.client.0.smithi042.stdout:5/519: rename d0/d39/f96 to d0/d38/d53/d91/fb6 0 2024-02-15T18:26:58.770 INFO:tasks.workunit.client.0.smithi042.stdout:5/520: stat d0/db/f15 0 2024-02-15T18:26:58.775 INFO:tasks.workunit.client.0.smithi042.stdout:0/460: rename d4/d38/d39/d8e/l5f to d4/d38/d39/d8e/d8f/l9e 0 2024-02-15T18:26:58.776 INFO:tasks.workunit.client.1.smithi191.stdout:3/125: link d6/d27/l2b d6/d1b/d1f/l2d 0 2024-02-15T18:26:58.776 INFO:tasks.workunit.client.0.smithi042.stdout:3/477: symlink d1/d11/d14/la2 0 2024-02-15T18:26:58.777 INFO:tasks.workunit.client.0.smithi042.stdout:8/398: creat dd/de/d3c/f73 x:0 0 0 2024-02-15T18:26:58.779 INFO:tasks.workunit.client.0.smithi042.stdout:2/426: dwrite d0/d8/db/de/d1a/f2b [0,4194304] 0 2024-02-15T18:26:58.779 INFO:tasks.workunit.client.0.smithi042.stdout:2/427: fdatasync d0/f71 0 2024-02-15T18:26:58.779 INFO:tasks.workunit.client.0.smithi042.stdout:2/428: dread - d0/d40/d56/f73 zero size 2024-02-15T18:26:58.779 INFO:tasks.workunit.client.0.smithi042.stdout:2/429: dread - d0/d8/d10/d31/d3a/d53/f66 zero size 2024-02-15T18:26:58.779 INFO:tasks.workunit.client.0.smithi042.stdout:5/521: symlink d0/d38/d53/d68/d6a/lb7 0 2024-02-15T18:26:58.780 INFO:tasks.workunit.client.1.smithi191.stdout:1/175: dwrite d0/d8/f35 [0,4194304] 0 2024-02-15T18:26:58.780 INFO:tasks.workunit.client.0.smithi042.stdout:7/511: rmdir d10/d49/d68 0 2024-02-15T18:26:58.781 INFO:tasks.workunit.client.0.smithi042.stdout:7/512: fsync d10/d15/d39/d6c/f8d 0 2024-02-15T18:26:58.781 INFO:tasks.workunit.client.0.smithi042.stdout:6/547: rmdir d9/d11/d46/d89/d8f 0 2024-02-15T18:26:58.782 INFO:tasks.workunit.client.0.smithi042.stdout:5/522: dread d0/fe [0,4194304] 0 2024-02-15T18:26:58.784 INFO:tasks.workunit.client.0.smithi042.stdout:0/461: creat d4/d38/f9f x:0 0 0 2024-02-15T18:26:58.784 INFO:tasks.workunit.client.1.smithi191.stdout:1/176: getdents d0 0 2024-02-15T18:26:58.786 INFO:tasks.workunit.client.0.smithi042.stdout:3/478: mknod d1/d11/d2e/ca3 0 2024-02-15T18:26:58.786 INFO:tasks.workunit.client.0.smithi042.stdout:3/479: dread - d1/de/d1a/f9e zero size 2024-02-15T18:26:58.786 INFO:tasks.workunit.client.0.smithi042.stdout:6/548: dread d9/f3c [0,4194304] 0 2024-02-15T18:26:58.787 INFO:tasks.workunit.client.0.smithi042.stdout:8/399: creat dd/d11/f74 x:0 0 0 2024-02-15T18:26:58.787 INFO:tasks.workunit.client.0.smithi042.stdout:8/400: truncate dd/d11/f74 280504 0 2024-02-15T18:26:58.787 INFO:tasks.workunit.client.0.smithi042.stdout:8/401: chown fc 38294581 1 2024-02-15T18:26:58.787 INFO:tasks.workunit.client.1.smithi191.stdout:1/177: symlink d0/d2f/l38 0 2024-02-15T18:26:58.787 INFO:tasks.workunit.client.0.smithi042.stdout:7/513: dread d10/f14 [0,4194304] 0 2024-02-15T18:26:58.789 INFO:tasks.workunit.client.1.smithi191.stdout:1/178: write d0/f5 [2646136,76292] 0 2024-02-15T18:26:58.789 INFO:tasks.workunit.client.1.smithi191.stdout:1/179: stat d0/db/fe 0 2024-02-15T18:26:58.789 INFO:tasks.workunit.client.1.smithi191.stdout:1/180: stat d0/d8/df/d2e 0 2024-02-15T18:26:58.793 INFO:tasks.workunit.client.1.smithi191.stdout:3/126: getdents d6 0 2024-02-15T18:26:58.799 INFO:tasks.workunit.client.0.smithi042.stdout:2/430: rmdir d0/d8/db 39 2024-02-15T18:26:58.799 INFO:tasks.workunit.client.0.smithi042.stdout:2/431: truncate d0/d8/db/de/d1a/d2e/f74 882822 0 2024-02-15T18:26:58.799 INFO:tasks.workunit.client.0.smithi042.stdout:2/432: chown d0/d8/d10/d31/d3a 342838 1 2024-02-15T18:26:58.799 INFO:tasks.workunit.client.0.smithi042.stdout:2/433: readlink d0/d8/d10/d31/d3a/d53/d3c/l3b 0 2024-02-15T18:26:58.799 INFO:tasks.workunit.client.0.smithi042.stdout:2/434: dread - d0/d8/d10/d31/d3a/d53/f5d zero size 2024-02-15T18:26:58.801 INFO:tasks.workunit.client.0.smithi042.stdout:6/549: dwrite d9/d11/d1d/fb0 [0,4194304] 0 2024-02-15T18:26:58.801 INFO:tasks.workunit.client.0.smithi042.stdout:6/550: readlink d9/d11/d19/d1c/l4d 0 2024-02-15T18:26:58.806 INFO:tasks.workunit.client.0.smithi042.stdout:5/523: link d0/d38/d53/d68/d6a/f32 d0/d38/d53/d91/db0/fb8 0 2024-02-15T18:26:58.807 INFO:tasks.workunit.client.0.smithi042.stdout:0/462: creat d4/d9/d62/fa0 x:0 0 0 2024-02-15T18:26:58.808 INFO:tasks.workunit.client.0.smithi042.stdout:3/480: creat d1/de/d1a/fa4 x:0 0 0 2024-02-15T18:26:58.809 INFO:tasks.workunit.client.0.smithi042.stdout:3/481: write d1/d11/d14/d4d/d73/f9d [352506,104272] 0 2024-02-15T18:26:58.809 INFO:tasks.workunit.client.0.smithi042.stdout:8/402: getdents dd/de/d3c/d4b 0 2024-02-15T18:26:58.810 INFO:tasks.workunit.client.0.smithi042.stdout:7/514: rmdir d10/d15/d39 39 2024-02-15T18:26:58.810 INFO:tasks.workunit.client.0.smithi042.stdout:7/515: write d10/d49/d6d/d78/d83/fa3 [287383,38577] 0 2024-02-15T18:26:58.810 INFO:tasks.workunit.client.0.smithi042.stdout:7/516: chown d10/d15/d39/d6c/cab 299151286 1 2024-02-15T18:26:58.810 INFO:tasks.workunit.client.0.smithi042.stdout:7/517: dread - d10/d15/fa8 zero size 2024-02-15T18:26:58.810 INFO:tasks.workunit.client.0.smithi042.stdout:7/518: chown d10/d12/d23/f3b 986 1 2024-02-15T18:26:58.810 INFO:tasks.workunit.client.0.smithi042.stdout:7/519: stat d10/d49/d6d/f85 0 2024-02-15T18:26:58.812 INFO:tasks.workunit.client.0.smithi042.stdout:2/435: rename d0/d8/d10/d7b/l75 to d0/d8/d10/d31/d3a/d53/d3c/l86 0 2024-02-15T18:26:58.815 INFO:tasks.workunit.client.0.smithi042.stdout:4/449: sync 2024-02-15T18:26:58.817 INFO:tasks.workunit.client.0.smithi042.stdout:6/551: mkdir d9/d11/d19/d1c/d31/d53/dbe 0 2024-02-15T18:26:58.817 INFO:tasks.workunit.client.0.smithi042.stdout:6/552: stat d9/d11/d19/d35/d55/d92/f95 0 2024-02-15T18:26:58.819 INFO:tasks.workunit.client.0.smithi042.stdout:3/482: creat d1/fa5 x:0 0 0 2024-02-15T18:26:58.821 INFO:tasks.workunit.client.0.smithi042.stdout:0/463: rename d4/c13 to d4/d9/d62/d74/ca1 0 2024-02-15T18:26:58.821 INFO:tasks.workunit.client.0.smithi042.stdout:0/464: read d4/d9/d34/d57/f7f [5531,636] 0 2024-02-15T18:26:58.823 INFO:tasks.workunit.client.0.smithi042.stdout:4/450: creat da/d11/d67/d5c/d66/fa3 x:0 0 0 2024-02-15T18:26:58.823 INFO:tasks.workunit.client.0.smithi042.stdout:4/451: read - da/d11/d27/f89 zero size 2024-02-15T18:26:58.824 INFO:tasks.workunit.client.0.smithi042.stdout:4/452: write da/d11/d67/d5c/d66/d78/d2f/d4d/f96 [789968,118126] 0 2024-02-15T18:26:58.824 INFO:tasks.workunit.client.0.smithi042.stdout:4/453: write da/d11/d29/d4a/fa0 [506816,94277] 0 2024-02-15T18:26:58.824 INFO:tasks.workunit.client.0.smithi042.stdout:6/553: symlink d9/d11/d19/d35/d55/lbf 0 2024-02-15T18:26:58.825 INFO:tasks.workunit.client.0.smithi042.stdout:4/454: dread da/f63 [0,4194304] 0 2024-02-15T18:26:58.825 INFO:tasks.workunit.client.0.smithi042.stdout:0/465: truncate d4/f7 418842 0 2024-02-15T18:26:58.826 INFO:tasks.workunit.client.0.smithi042.stdout:6/554: link d9/d3e/f96 d9/d3e/fc0 0 2024-02-15T18:26:58.829 INFO:tasks.workunit.client.0.smithi042.stdout:0/466: creat d4/d38/d5c/fa2 x:0 0 0 2024-02-15T18:26:58.829 INFO:tasks.workunit.client.0.smithi042.stdout:0/467: write d4/d9/d34/d4e/d6c/f71 [949481,63267] 0 2024-02-15T18:26:58.829 INFO:tasks.workunit.client.0.smithi042.stdout:0/468: fsync d4/f3b 0 2024-02-15T18:26:58.831 INFO:tasks.workunit.client.0.smithi042.stdout:3/483: dread d1/de/d19/f3e [0,4194304] 0 2024-02-15T18:26:58.831 INFO:tasks.workunit.client.0.smithi042.stdout:3/484: chown d1/d11/d14/d4d 381063726 1 2024-02-15T18:26:58.836 INFO:tasks.workunit.client.0.smithi042.stdout:4/455: rmdir da/d11/d67/d5c 39 2024-02-15T18:26:58.836 INFO:tasks.workunit.client.0.smithi042.stdout:0/469: dread d4/d9/d15/f21 [0,4194304] 0 2024-02-15T18:26:58.840 INFO:tasks.workunit.client.0.smithi042.stdout:4/456: rename da/d3b/d41/d8d/d8f to da/d11/da4 0 2024-02-15T18:26:58.842 INFO:tasks.workunit.client.0.smithi042.stdout:4/457: mknod da/d11/d27/ca5 0 2024-02-15T18:26:58.845 INFO:tasks.workunit.client.0.smithi042.stdout:4/458: creat da/d2b/d7e/fa6 x:0 0 0 2024-02-15T18:26:58.845 INFO:tasks.workunit.client.0.smithi042.stdout:8/403: dwrite dd/de/d3c/d4b/d67/f2e [4194304,4194304] 0 2024-02-15T18:26:58.845 INFO:tasks.workunit.client.0.smithi042.stdout:8/404: chown dd/de/d3c/d4b/d67/l38 5 1 2024-02-15T18:26:58.846 INFO:tasks.workunit.client.0.smithi042.stdout:4/459: dread da/d11/d29/d4a/d4f/f8e [0,4194304] 0 2024-02-15T18:26:58.846 INFO:tasks.workunit.client.0.smithi042.stdout:3/485: dread d1/de/d19/f44 [0,4194304] 0 2024-02-15T18:26:58.846 INFO:tasks.workunit.client.0.smithi042.stdout:5/524: dwrite d0/d12/d65/d74/f6d [0,4194304] 0 2024-02-15T18:26:58.847 INFO:tasks.workunit.client.0.smithi042.stdout:8/405: getdents dd/de 0 2024-02-15T18:26:58.847 INFO:tasks.workunit.client.0.smithi042.stdout:4/460: unlink da/d3b/d41/f72 0 2024-02-15T18:26:58.848 INFO:tasks.workunit.client.1.smithi191.stdout:4/216: sync 2024-02-15T18:26:58.849 INFO:tasks.workunit.client.1.smithi191.stdout:4/217: write d2/d5/d14/f2e [449586,53364] 0 2024-02-15T18:26:58.849 INFO:tasks.workunit.client.0.smithi042.stdout:3/486: rename d1/de/c33 to d1/ca6 0 2024-02-15T18:26:58.849 INFO:tasks.workunit.client.0.smithi042.stdout:3/487: fdatasync d1/d11/d14/d4d/f4f 0 2024-02-15T18:26:58.851 INFO:tasks.workunit.client.0.smithi042.stdout:5/525: creat d0/d38/d53/d68/d6a/d42/fb9 x:0 0 0 2024-02-15T18:26:58.851 INFO:tasks.workunit.client.0.smithi042.stdout:5/526: dread - d0/f9d zero size 2024-02-15T18:26:58.855 INFO:tasks.workunit.client.0.smithi042.stdout:2/436: dwrite d0/d8/d10/f17 [0,4194304] 0 2024-02-15T18:26:58.856 INFO:tasks.workunit.client.0.smithi042.stdout:4/461: link da/d11/d29/d4a/fa0 da/fa7 0 2024-02-15T18:26:58.865 INFO:tasks.workunit.client.0.smithi042.stdout:5/527: mkdir d0/d12/d65/d74/d52/dad/dba 0 2024-02-15T18:26:58.865 INFO:tasks.workunit.client.0.smithi042.stdout:8/406: mkdir dd/d11/d75 0 2024-02-15T18:26:58.867 INFO:tasks.workunit.client.0.smithi042.stdout:4/462: symlink da/d11/d67/d5c/d66/d78/la8 0 2024-02-15T18:26:58.867 INFO:tasks.workunit.client.0.smithi042.stdout:4/463: chown da/d11/f3c 26006105 1 2024-02-15T18:26:58.867 INFO:tasks.workunit.client.0.smithi042.stdout:4/464: chown da/d11/d29/l5b 2586830 1 2024-02-15T18:26:58.867 INFO:tasks.workunit.client.0.smithi042.stdout:4/465: dread da/d11/d29/d4a/d4f/f6c [0,4194304] 0 2024-02-15T18:26:58.868 INFO:tasks.workunit.client.0.smithi042.stdout:4/466: creat da/d11/d29/d4a/d4f/d80/fa9 x:0 0 0 2024-02-15T18:26:58.869 INFO:tasks.workunit.client.0.smithi042.stdout:4/467: getdents da/d3b 0 2024-02-15T18:26:58.877 INFO:tasks.workunit.client.1.smithi191.stdout:4/218: dwrite d2/f4 [0,4194304] 0 2024-02-15T18:26:58.878 INFO:tasks.workunit.client.0.smithi042.stdout:8/407: dread dd/f2c [0,4194304] 0 2024-02-15T18:26:58.881 INFO:tasks.workunit.client.0.smithi042.stdout:8/408: creat dd/de/d3c/d4b/f76 x:0 0 0 2024-02-15T18:26:58.883 INFO:tasks.workunit.client.0.smithi042.stdout:8/409: link dd/l25 dd/de/d3c/d4b/d67/l77 0 2024-02-15T18:26:58.883 INFO:tasks.workunit.client.0.smithi042.stdout:8/410: dread - dd/de/d33/d66/f5b zero size 2024-02-15T18:26:58.883 INFO:tasks.workunit.client.0.smithi042.stdout:8/411: truncate dd/f13 196088 0 2024-02-15T18:26:58.884 INFO:tasks.workunit.client.0.smithi042.stdout:8/412: write dd/de/d12/f68 [683909,111914] 0 2024-02-15T18:26:58.884 INFO:tasks.workunit.client.0.smithi042.stdout:8/413: fsync dd/de/d3c/d4b/d67/f3a 0 2024-02-15T18:26:58.885 INFO:tasks.workunit.client.0.smithi042.stdout:8/414: mkdir dd/de/d3c/d4b/d67/d6f/d78 0 2024-02-15T18:26:58.885 INFO:tasks.workunit.client.0.smithi042.stdout:8/415: chown dd/f48 15337038 1 2024-02-15T18:26:58.885 INFO:tasks.workunit.client.0.smithi042.stdout:8/416: fsync dd/de/d33/d66/f52 0 2024-02-15T18:26:58.885 INFO:tasks.workunit.client.0.smithi042.stdout:8/417: chown dd/d11/l6b 3751387 1 2024-02-15T18:26:58.886 INFO:tasks.workunit.client.0.smithi042.stdout:8/418: symlink dd/de/d33/d3b/l79 0 2024-02-15T18:26:58.886 INFO:tasks.workunit.client.0.smithi042.stdout:8/419: write dd/de/d33/d66/f5b [429790,71775] 0 2024-02-15T18:26:58.886 INFO:tasks.workunit.client.0.smithi042.stdout:8/420: read dd/de/d33/f40 [284412,118242] 0 2024-02-15T18:26:58.893 INFO:tasks.workunit.client.0.smithi042.stdout:0/470: dwrite d4/d9/d5a/d60/f96 [0,4194304] 0 2024-02-15T18:26:58.894 INFO:tasks.workunit.client.0.smithi042.stdout:2/437: dwrite d0/d8/d10/d31/d3a/d53/f58 [0,4194304] 0 2024-02-15T18:26:58.894 INFO:tasks.workunit.client.0.smithi042.stdout:2/438: fdatasync d0/d8/db/de/f4d 0 2024-02-15T18:26:58.894 INFO:tasks.workunit.client.0.smithi042.stdout:4/468: dwrite da/d11/d29/f70 [0,4194304] 0 2024-02-15T18:26:58.894 INFO:tasks.workunit.client.0.smithi042.stdout:4/469: chown da/d3b/l6a 2273 1 2024-02-15T18:26:58.894 INFO:tasks.workunit.client.0.smithi042.stdout:4/470: write da/d11/d27/f86 [1021721,92163] 0 2024-02-15T18:26:58.895 INFO:tasks.workunit.client.1.smithi191.stdout:4/219: fdatasync d2/d5/d14/f2e 0 2024-02-15T18:26:58.896 INFO:tasks.workunit.client.0.smithi042.stdout:5/528: dwrite d0/d14/f29 [0,4194304] 0 2024-02-15T18:26:58.896 INFO:tasks.workunit.client.0.smithi042.stdout:5/529: stat d0/d38/d53/d68/d6a/d42/d7d/c8c 0 2024-02-15T18:26:58.897 INFO:tasks.workunit.client.0.smithi042.stdout:8/421: dread dd/d11/f39 [0,4194304] 0 2024-02-15T18:26:58.897 INFO:tasks.workunit.client.0.smithi042.stdout:8/422: chown dd/de/d33/d66 33637 1 2024-02-15T18:26:58.898 INFO:tasks.workunit.client.0.smithi042.stdout:0/471: dread d4/d9/d34/f40 [0,4194304] 0 2024-02-15T18:26:58.899 INFO:tasks.workunit.client.0.smithi042.stdout:8/423: dread dd/de/d12/f4c [0,4194304] 0 2024-02-15T18:26:58.899 INFO:tasks.workunit.client.0.smithi042.stdout:8/424: write dd/de/d3c/d4b/d67/f3a [1020537,7534] 0 2024-02-15T18:26:58.899 INFO:tasks.workunit.client.0.smithi042.stdout:8/425: chown dd/f17 1 1 2024-02-15T18:26:58.948 INFO:tasks.workunit.client.0.smithi042.stdout:1/491: sync 2024-02-15T18:26:58.948 INFO:tasks.workunit.client.0.smithi042.stdout:9/499: sync 2024-02-15T18:26:58.952 INFO:tasks.workunit.client.0.smithi042.stdout:2/439: unlink d0/d8/d10/f54 0 2024-02-15T18:26:58.952 INFO:tasks.workunit.client.0.smithi042.stdout:2/440: read - d0/d8/d10/d31/d3a/d53/f66 zero size 2024-02-15T18:26:58.957 INFO:tasks.workunit.client.1.smithi191.stdout:4/220: mknod d2/c5e 0 2024-02-15T18:26:58.962 INFO:tasks.workunit.client.0.smithi042.stdout:4/471: rename da/d11/d67/d5c/f8b to da/d11/da4/faa 0 2024-02-15T18:26:58.963 INFO:tasks.workunit.client.1.smithi191.stdout:4/221: mknod d2/d5/d7/c5f 0 2024-02-15T18:26:58.965 INFO:tasks.workunit.client.0.smithi042.stdout:0/472: unlink d4/d9/d34/l6d 0 2024-02-15T18:26:58.967 INFO:tasks.workunit.client.1.smithi191.stdout:4/222: mkdir d2/d5/d14/d44/d60 0 2024-02-15T18:26:58.967 INFO:tasks.workunit.client.1.smithi191.stdout:4/223: chown d2/d5/d14/f2e 5370 1 2024-02-15T18:26:58.967 INFO:tasks.workunit.client.1.smithi191.stdout:4/224: truncate d2/d5/f13 969780 0 2024-02-15T18:26:58.968 INFO:tasks.workunit.client.1.smithi191.stdout:4/225: creat d2/d5/d14/d44/d30/f61 x:0 0 0 2024-02-15T18:26:58.970 INFO:tasks.workunit.client.1.smithi191.stdout:4/226: rename d2/f2b to d2/d5/d7/d3e/d42/d52/f62 0 2024-02-15T18:26:58.970 INFO:tasks.workunit.client.1.smithi191.stdout:4/227: read - d2/d5/d14/d44/d30/f61 zero size 2024-02-15T18:26:58.972 INFO:tasks.workunit.client.0.smithi042.stdout:1/492: mkdir de/d15/d67/d66/d7c/db3 0 2024-02-15T18:26:58.973 INFO:tasks.workunit.client.0.smithi042.stdout:0/473: dread d4/d38/f3a [0,4194304] 0 2024-02-15T18:26:58.973 INFO:tasks.workunit.client.0.smithi042.stdout:9/500: mknod d6/da/d2f/d7c/cad 0 2024-02-15T18:26:58.973 INFO:tasks.workunit.client.0.smithi042.stdout:2/441: dwrite d0/d8/d10/d31/d3a/d53/f5d [0,4194304] 0 2024-02-15T18:26:58.974 INFO:tasks.workunit.client.0.smithi042.stdout:2/442: write d0/d8/d10/d31/d3a/d53/f66 [477031,44616] 0 2024-02-15T18:26:58.975 INFO:tasks.workunit.client.0.smithi042.stdout:4/472: creat da/d3b/d41/d8d/fab x:0 0 0 2024-02-15T18:26:58.977 INFO:tasks.workunit.client.0.smithi042.stdout:0/474: chown d4/d38/d77/d2e/c33 30163895 1 2024-02-15T18:26:58.979 INFO:tasks.workunit.client.0.smithi042.stdout:8/426: rename dd/de/d12/l5c to dd/d11/l7a 0 2024-02-15T18:26:58.979 INFO:tasks.workunit.client.0.smithi042.stdout:8/427: dread - dd/f4d zero size 2024-02-15T18:26:58.979 INFO:tasks.workunit.client.0.smithi042.stdout:8/428: truncate dd/f13 1047526 0 2024-02-15T18:26:58.980 INFO:tasks.workunit.client.0.smithi042.stdout:8/429: chown dd/d11/d75 1 1 2024-02-15T18:26:58.980 INFO:tasks.workunit.client.0.smithi042.stdout:8/430: fdatasync dd/de/d3c/d4b/d67/f6e 0 2024-02-15T18:26:58.984 INFO:tasks.workunit.client.0.smithi042.stdout:6/555: sync 2024-02-15T18:26:58.984 INFO:tasks.workunit.client.0.smithi042.stdout:5/530: sync 2024-02-15T18:26:58.984 INFO:tasks.workunit.client.0.smithi042.stdout:3/488: sync 2024-02-15T18:26:58.984 INFO:tasks.workunit.client.0.smithi042.stdout:7/520: sync 2024-02-15T18:26:58.984 INFO:tasks.workunit.client.0.smithi042.stdout:6/556: dread - d9/d11/d1d/f88 zero size 2024-02-15T18:26:58.985 INFO:tasks.workunit.client.0.smithi042.stdout:4/473: dread da/d11/d29/f32 [0,4194304] 0 2024-02-15T18:26:58.985 INFO:tasks.workunit.client.0.smithi042.stdout:2/443: truncate d0/d8/db/de/d1a/d2e/f74 39808 0 2024-02-15T18:26:58.987 INFO:tasks.workunit.client.0.smithi042.stdout:1/493: mknod de/d15/d67/cb4 0 2024-02-15T18:26:58.987 INFO:tasks.workunit.client.0.smithi042.stdout:1/494: chown de/c11 3605617 1 2024-02-15T18:26:58.989 INFO:tasks.workunit.client.0.smithi042.stdout:5/531: link d0/l4 d0/d38/d53/d91/db0/lbb 0 2024-02-15T18:26:58.989 INFO:tasks.workunit.client.0.smithi042.stdout:5/532: stat d0/d14 0 2024-02-15T18:26:58.989 INFO:tasks.workunit.client.0.smithi042.stdout:7/521: creat d10/d43/d57/fb0 x:0 0 0 2024-02-15T18:26:58.991 INFO:tasks.workunit.client.0.smithi042.stdout:4/474: unlink da/d11/d29/f30 0 2024-02-15T18:26:58.991 INFO:tasks.workunit.client.0.smithi042.stdout:3/489: creat d1/d11/d14/d4d/d73/d89/fa7 x:0 0 0 2024-02-15T18:26:58.991 INFO:tasks.workunit.client.0.smithi042.stdout:3/490: truncate d1/d11/d14/d4d/f7e 92141 0 2024-02-15T18:26:58.992 INFO:tasks.workunit.client.0.smithi042.stdout:9/501: dwrite d6/dd/d9/f14 [4194304,4194304] 0 2024-02-15T18:26:58.992 INFO:tasks.workunit.client.0.smithi042.stdout:2/444: creat d0/d8/d10/d31/d3a/d53/d3c/f87 x:0 0 0 2024-02-15T18:26:58.996 INFO:tasks.workunit.client.0.smithi042.stdout:3/491: read d1/d11/d2e/f3b [1500616,30536] 0 2024-02-15T18:26:58.996 INFO:tasks.workunit.client.0.smithi042.stdout:3/492: write d1/d11/d14/d4d/d6c/d7b/f98 [1418174,119847] 0 2024-02-15T18:26:58.997 INFO:tasks.workunit.client.0.smithi042.stdout:8/431: rename dd/d11/l7a to dd/de/d33/d3b/l7b 0 2024-02-15T18:26:58.998 INFO:tasks.workunit.client.0.smithi042.stdout:8/432: chown dd/de/d3c/f73 50 1 2024-02-15T18:26:58.998 INFO:tasks.workunit.client.0.smithi042.stdout:1/495: read de/d15/d1b/f25 [1245440,38307] 0 2024-02-15T18:26:59.000 INFO:tasks.workunit.client.0.smithi042.stdout:5/533: write d0/d38/f3c [1271870,15178] 0 2024-02-15T18:26:59.001 INFO:tasks.workunit.client.0.smithi042.stdout:2/445: creat d0/d8/d10/d5a/f88 x:0 0 0 2024-02-15T18:26:59.001 INFO:tasks.workunit.client.0.smithi042.stdout:3/493: mkdir d1/d11/d14/d4d/d73/d89/da8 0 2024-02-15T18:26:59.001 INFO:tasks.workunit.client.0.smithi042.stdout:0/475: dwrite d4/f3b [0,4194304] 0 2024-02-15T18:26:59.003 INFO:tasks.workunit.client.0.smithi042.stdout:3/494: dread d1/d11/d14/d4d/f7e [0,4194304] 0 2024-02-15T18:26:59.003 INFO:tasks.workunit.client.0.smithi042.stdout:0/476: write d4/d9/f31 [157918,38404] 0 2024-02-15T18:26:59.003 INFO:tasks.workunit.client.0.smithi042.stdout:8/433: rename dd/de/d3c/d4b/d67/f3a to dd/de/d33/d66/d4e/f7c 0 2024-02-15T18:26:59.003 INFO:tasks.workunit.client.0.smithi042.stdout:6/557: dwrite d9/d3e/f8b [0,4194304] 0 2024-02-15T18:26:59.003 INFO:tasks.workunit.client.0.smithi042.stdout:6/558: truncate d9/d11/d19/f5a 4903426 0 2024-02-15T18:26:59.004 INFO:tasks.workunit.client.0.smithi042.stdout:1/496: chown de/d15/d67/d74/c9b 188806020 1 2024-02-15T18:26:59.004 INFO:tasks.workunit.client.0.smithi042.stdout:7/522: dwrite d10/d15/d39/f5d [4194304,4194304] 0 2024-02-15T18:26:59.004 INFO:tasks.workunit.client.0.smithi042.stdout:6/559: write d9/f21 [7973828,68947] 0 2024-02-15T18:26:59.008 INFO:tasks.workunit.client.0.smithi042.stdout:5/534: rmdir d0/d38/d53/d68 39 2024-02-15T18:26:59.012 INFO:tasks.workunit.client.0.smithi042.stdout:9/502: dwrite d6/da/d1d/f38 [0,4194304] 0 2024-02-15T18:26:59.013 INFO:tasks.workunit.client.0.smithi042.stdout:0/477: mknod d4/d9/d8d/ca3 0 2024-02-15T18:26:59.013 INFO:tasks.workunit.client.0.smithi042.stdout:0/478: write d4/fe [13543559,28309] 0 2024-02-15T18:26:59.015 INFO:tasks.workunit.client.0.smithi042.stdout:2/446: rename d0/d8/d10/d31/d3a/l67 to d0/d8/d20/d7d/l89 0 2024-02-15T18:26:59.015 INFO:tasks.workunit.client.0.smithi042.stdout:2/447: dread - d0/d8/f65 zero size 2024-02-15T18:26:59.015 INFO:tasks.workunit.client.0.smithi042.stdout:2/448: chown d0/d8/db/de/d1a/f2b 25 1 2024-02-15T18:26:59.015 INFO:tasks.workunit.client.0.smithi042.stdout:8/434: rmdir dd/de/d33/d66/d4e 39 2024-02-15T18:26:59.015 INFO:tasks.workunit.client.0.smithi042.stdout:8/435: read dd/de/d12/f65 [1081515,58957] 0 2024-02-15T18:26:59.017 INFO:tasks.workunit.client.0.smithi042.stdout:1/497: mknod de/d15/d67/d74/d7e/cb5 0 2024-02-15T18:26:59.017 INFO:tasks.workunit.client.0.smithi042.stdout:1/498: stat de/c1a 0 2024-02-15T18:26:59.017 INFO:tasks.workunit.client.0.smithi042.stdout:7/523: mknod d10/d12/d65/da7/cb1 0 2024-02-15T18:26:59.018 INFO:tasks.workunit.client.0.smithi042.stdout:6/560: creat d9/d11/d19/fc1 x:0 0 0 2024-02-15T18:26:59.019 INFO:tasks.workunit.client.0.smithi042.stdout:5/535: creat d0/d39/fbc x:0 0 0 2024-02-15T18:26:59.021 INFO:tasks.workunit.client.0.smithi042.stdout:9/503: mknod d6/da/cae 0 2024-02-15T18:26:59.021 INFO:tasks.workunit.client.0.smithi042.stdout:9/504: fsync d6/dd/d9/f32 0 2024-02-15T18:26:59.021 INFO:tasks.workunit.client.0.smithi042.stdout:0/479: rename d4/f4f to d4/d9/d5a/fa4 0 2024-02-15T18:26:59.024 INFO:tasks.workunit.client.0.smithi042.stdout:0/480: dread d4/d9/d34/d57/f6f [0,4194304] 0 2024-02-15T18:26:59.024 INFO:tasks.workunit.client.0.smithi042.stdout:7/524: mknod d10/d43/cb2 0 2024-02-15T18:26:59.024 INFO:tasks.workunit.client.0.smithi042.stdout:2/449: truncate d0/d8/f5f 892552 0 2024-02-15T18:26:59.026 INFO:tasks.workunit.client.0.smithi042.stdout:9/505: creat d6/da/d2f/d55/d73/d8d/faf x:0 0 0 2024-02-15T18:26:59.027 INFO:tasks.workunit.client.0.smithi042.stdout:0/481: link d4/d9/d62/d74/ca1 d4/d38/d39/ca5 0 2024-02-15T18:26:59.029 INFO:tasks.workunit.client.0.smithi042.stdout:0/482: write d4/f68 [95282,117424] 0 2024-02-15T18:26:59.029 INFO:tasks.workunit.client.0.smithi042.stdout:0/483: truncate d4/d9/d34/d8b/f90 163068 0 2024-02-15T18:26:59.029 INFO:tasks.workunit.client.0.smithi042.stdout:0/484: stat d4/d9/d34/d4e/c72 0 2024-02-15T18:26:59.029 INFO:tasks.workunit.client.0.smithi042.stdout:0/485: stat d4/d38/d39/d8e/l82 0 2024-02-15T18:26:59.029 INFO:tasks.workunit.client.0.smithi042.stdout:1/499: link de/c1a de/d15/d1b/cb6 0 2024-02-15T18:26:59.031 INFO:tasks.workunit.client.0.smithi042.stdout:2/450: rename d0/d8/d20/c83 to d0/d8/c8a 0 2024-02-15T18:26:59.031 INFO:tasks.workunit.client.0.smithi042.stdout:0/486: creat d4/d9/d62/fa6 x:0 0 0 2024-02-15T18:26:59.032 INFO:tasks.workunit.client.0.smithi042.stdout:7/525: dread d10/f63 [0,4194304] 0 2024-02-15T18:26:59.032 INFO:tasks.workunit.client.0.smithi042.stdout:1/500: creat de/d15/d2c/d51/fb7 x:0 0 0 2024-02-15T18:26:59.034 INFO:tasks.workunit.client.0.smithi042.stdout:7/526: mkdir d10/d49/d6d/d6e/db3 0 2024-02-15T18:26:59.034 INFO:tasks.workunit.client.0.smithi042.stdout:7/527: dread - d10/d43/f84 zero size 2024-02-15T18:26:59.035 INFO:tasks.workunit.client.0.smithi042.stdout:7/528: symlink d10/d49/d6d/d6e/lb4 0 2024-02-15T18:26:59.035 INFO:tasks.workunit.client.0.smithi042.stdout:7/529: truncate d10/d43/d57/fb0 321303 0 2024-02-15T18:26:59.037 INFO:tasks.workunit.client.0.smithi042.stdout:7/530: unlink d10/d12/d65/f74 0 2024-02-15T18:26:59.038 INFO:tasks.workunit.client.0.smithi042.stdout:7/531: chown d10/d12/d23/f61 1241214 1 2024-02-15T18:26:59.038 INFO:tasks.workunit.client.0.smithi042.stdout:7/532: write d10/d15/d39/d6c/f9a [381709,25269] 0 2024-02-15T18:26:59.038 INFO:tasks.workunit.client.0.smithi042.stdout:7/533: write d10/d15/f34 [4356124,35737] 0 2024-02-15T18:26:59.039 INFO:tasks.workunit.client.0.smithi042.stdout:7/534: unlink d10/d15/d39/l46 0 2024-02-15T18:26:59.039 INFO:tasks.workunit.client.0.smithi042.stdout:7/535: truncate d10/d43/f84 81594 0 2024-02-15T18:26:59.039 INFO:tasks.workunit.client.0.smithi042.stdout:7/536: chown d10 7169 1 2024-02-15T18:26:59.040 INFO:tasks.workunit.client.0.smithi042.stdout:7/537: dread - d10/d15/fa6 zero size 2024-02-15T18:26:59.041 INFO:tasks.workunit.client.0.smithi042.stdout:9/506: dwrite d6/da/d2f/f56 [0,4194304] 0 2024-02-15T18:26:59.041 INFO:tasks.workunit.client.0.smithi042.stdout:9/507: truncate d6/da2/fac 866853 0 2024-02-15T18:26:59.041 INFO:tasks.workunit.client.0.smithi042.stdout:5/536: dwrite d0/d38/d53/d68/d6a/f56 [0,4194304] 0 2024-02-15T18:26:59.043 INFO:tasks.workunit.client.0.smithi042.stdout:9/508: creat d6/da/d2f/d55/fb0 x:0 0 0 2024-02-15T18:26:59.045 INFO:tasks.workunit.client.0.smithi042.stdout:9/509: getdents d6/da/d1d/d2d/d40/d48 0 2024-02-15T18:26:59.046 INFO:tasks.workunit.client.0.smithi042.stdout:9/510: write d6/da/d2f/d55/d73/f77 [862177,115387] 0 2024-02-15T18:26:59.046 INFO:tasks.workunit.client.0.smithi042.stdout:9/511: write d6/da/d2f/f6a [360520,70570] 0 2024-02-15T18:26:59.047 INFO:tasks.workunit.client.0.smithi042.stdout:9/512: mkdir d6/da2/db1 0 2024-02-15T18:26:59.063 INFO:tasks.workunit.client.1.smithi191.stdout:9/147: sync 2024-02-15T18:26:59.063 INFO:tasks.workunit.client.1.smithi191.stdout:0/71: sync 2024-02-15T18:26:59.063 INFO:tasks.workunit.client.1.smithi191.stdout:5/69: sync 2024-02-15T18:26:59.063 INFO:tasks.workunit.client.1.smithi191.stdout:5/70: chown d0/cf 704449 1 2024-02-15T18:26:59.064 INFO:tasks.workunit.client.0.smithi042.stdout:5/537: read d0/d38/d53/d64/f9f [1737371,19531] 0 2024-02-15T18:26:59.065 INFO:tasks.workunit.client.0.smithi042.stdout:9/513: dread d6/dd/f2b [0,4194304] 0 2024-02-15T18:26:59.065 INFO:tasks.workunit.client.0.smithi042.stdout:9/514: write d6/dd/d9/f9f [290525,118093] 0 2024-02-15T18:26:59.067 INFO:tasks.workunit.client.1.smithi191.stdout:0/72: getdents d7/dd 0 2024-02-15T18:26:59.067 INFO:tasks.workunit.client.1.smithi191.stdout:5/71: creat d0/f1b x:0 0 0 2024-02-15T18:26:59.071 INFO:tasks.workunit.client.1.smithi191.stdout:0/73: creat d7/dd/f11 x:0 0 0 2024-02-15T18:26:59.072 INFO:tasks.workunit.client.1.smithi191.stdout:5/72: symlink d0/d18/l1c 0 2024-02-15T18:26:59.072 INFO:tasks.workunit.client.1.smithi191.stdout:0/74: read d7/fb [3994026,47919] 0 2024-02-15T18:26:59.075 INFO:tasks.workunit.client.1.smithi191.stdout:9/148: rename d0/l2e to d0/d1d/l37 0 2024-02-15T18:26:59.078 INFO:tasks.workunit.client.0.smithi042.stdout:2/451: dread d0/d8/d10/d31/d3a/d53/d3c/f60 [0,4194304] 0 2024-02-15T18:26:59.082 INFO:tasks.workunit.client.0.smithi042.stdout:3/495: rmdir d1 39 2024-02-15T18:26:59.083 INFO:tasks.workunit.client.1.smithi191.stdout:5/73: symlink d0/l1d 0 2024-02-15T18:26:59.090 INFO:tasks.workunit.client.0.smithi042.stdout:7/538: rmdir d10/d43 39 2024-02-15T18:26:59.094 INFO:tasks.workunit.client.0.smithi042.stdout:0/487: dwrite d4/d9/d5a/fa4 [0,4194304] 0 2024-02-15T18:26:59.096 INFO:tasks.workunit.client.1.smithi191.stdout:0/75: creat d7/dd/f12 x:0 0 0 2024-02-15T18:26:59.102 INFO:tasks.workunit.client.1.smithi191.stdout:5/74: creat d0/d18/f1e x:0 0 0 2024-02-15T18:26:59.103 INFO:tasks.workunit.client.1.smithi191.stdout:0/76: rename d7/dd/f12 to d7/f13 0 2024-02-15T18:26:59.107 INFO:tasks.workunit.client.1.smithi191.stdout:0/77: rmdir d7 39 2024-02-15T18:26:59.107 INFO:tasks.workunit.client.1.smithi191.stdout:0/78: dread - d7/f13 zero size 2024-02-15T18:26:59.107 INFO:tasks.workunit.client.1.smithi191.stdout:0/79: readlink d7/da/le 0 2024-02-15T18:26:59.108 INFO:tasks.workunit.client.1.smithi191.stdout:9/149: link d0/c1a d0/d7/c38 0 2024-02-15T18:26:59.109 INFO:tasks.workunit.client.0.smithi042.stdout:1/501: dwrite de/d15/d67/d63/f69 [0,4194304] 0 2024-02-15T18:26:59.109 INFO:tasks.workunit.client.0.smithi042.stdout:1/502: write de/fa8 [95915,18981] 0 2024-02-15T18:26:59.109 INFO:tasks.workunit.client.0.smithi042.stdout:1/503: chown de/d15/f20 28 1 2024-02-15T18:26:59.110 INFO:tasks.workunit.client.1.smithi191.stdout:0/80: creat d7/da/f14 x:0 0 0 2024-02-15T18:26:59.110 INFO:tasks.workunit.client.1.smithi191.stdout:0/81: truncate d7/f13 824094 0 2024-02-15T18:26:59.110 INFO:tasks.workunit.client.1.smithi191.stdout:0/82: dread - d7/dd/f11 zero size 2024-02-15T18:26:59.110 INFO:tasks.workunit.client.1.smithi191.stdout:0/83: truncate d7/dd/f11 424403 0 2024-02-15T18:26:59.110 INFO:tasks.workunit.client.1.smithi191.stdout:0/84: truncate d7/fb 9308222 0 2024-02-15T18:26:59.111 INFO:tasks.workunit.client.1.smithi191.stdout:9/150: chown d0/f4 13642 1 2024-02-15T18:26:59.113 INFO:tasks.workunit.client.1.smithi191.stdout:9/151: read d0/f12 [64814,51218] 0 2024-02-15T18:26:59.114 INFO:tasks.workunit.client.1.smithi191.stdout:9/152: rmdir d0/d7 39 2024-02-15T18:26:59.115 INFO:tasks.workunit.client.1.smithi191.stdout:9/153: stat d0/d7/dc/d17 0 2024-02-15T18:26:59.115 INFO:tasks.workunit.client.0.smithi042.stdout:1/504: mknod de/d15/d67/d66/cb8 0 2024-02-15T18:26:59.116 INFO:tasks.workunit.client.1.smithi191.stdout:9/154: mkdir d0/d7/da/d39 0 2024-02-15T18:26:59.117 INFO:tasks.workunit.client.0.smithi042.stdout:9/515: dwrite d6/da/d1d/d2d/d40/f8c [0,4194304] 0 2024-02-15T18:26:59.117 INFO:tasks.workunit.client.0.smithi042.stdout:0/488: rename d4/d38/d77/d2e/c29 to d4/d38/d77/ca7 0 2024-02-15T18:26:59.118 INFO:tasks.workunit.client.0.smithi042.stdout:5/538: dwrite d0/d50/f5f [0,4194304] 0 2024-02-15T18:26:59.118 INFO:tasks.workunit.client.0.smithi042.stdout:2/452: dwrite d0/d8/db/fd [0,4194304] 0 2024-02-15T18:26:59.119 INFO:tasks.workunit.client.0.smithi042.stdout:3/496: dwrite d1/d11/d2e/d4c/f76 [0,4194304] 0 2024-02-15T18:26:59.119 INFO:tasks.workunit.client.0.smithi042.stdout:3/497: write d1/d11/f65 [876091,85442] 0 2024-02-15T18:26:59.120 INFO:tasks.workunit.client.0.smithi042.stdout:7/539: dwrite d10/d15/d39/d6c/f71 [0,4194304] 0 2024-02-15T18:26:59.121 INFO:tasks.workunit.client.1.smithi191.stdout:9/155: symlink d0/d29/l3a 0 2024-02-15T18:26:59.123 INFO:tasks.workunit.client.0.smithi042.stdout:1/505: rename de/d15/d67/d89/fa1 to de/d15/d95/fb9 0 2024-02-15T18:26:59.128 INFO:tasks.workunit.client.0.smithi042.stdout:9/516: write d6/da/d1d/f24 [3053795,30965] 0 2024-02-15T18:26:59.130 INFO:tasks.workunit.client.1.smithi191.stdout:5/75: rmdir d0 39 2024-02-15T18:26:59.130 INFO:tasks.workunit.client.1.smithi191.stdout:5/76: readlink d0/l10 0 2024-02-15T18:26:59.130 INFO:tasks.workunit.client.1.smithi191.stdout:5/77: write d0/f1a [400981,75953] 0 2024-02-15T18:26:59.130 INFO:tasks.workunit.client.1.smithi191.stdout:5/78: chown d0/d18 449 1 2024-02-15T18:26:59.130 INFO:tasks.workunit.client.1.smithi191.stdout:5/79: write d0/f1a [1150435,73667] 0 2024-02-15T18:26:59.130 INFO:tasks.workunit.client.1.smithi191.stdout:5/80: truncate d0/d14/f17 211352 0 2024-02-15T18:26:59.130 INFO:tasks.workunit.client.1.smithi191.stdout:5/81: chown d0/cf 4 1 2024-02-15T18:26:59.132 INFO:tasks.workunit.client.1.smithi191.stdout:9/156: unlink d0/d1d/l37 0 2024-02-15T18:26:59.137 INFO:tasks.workunit.client.0.smithi042.stdout:4/475: sync 2024-02-15T18:26:59.137 INFO:tasks.workunit.client.1.smithi191.stdout:0/85: truncate d7/f13 424213 0 2024-02-15T18:26:59.137 INFO:tasks.workunit.client.1.smithi191.stdout:0/86: readlink l4 0 2024-02-15T18:26:59.139 INFO:tasks.workunit.client.0.smithi042.stdout:2/453: mkdir d0/d40/d56/d8b 0 2024-02-15T18:26:59.139 INFO:tasks.workunit.client.1.smithi191.stdout:0/87: write d7/fb [720695,56660] 0 2024-02-15T18:26:59.139 INFO:tasks.workunit.client.1.smithi191.stdout:0/88: chown d7/lc 238 1 2024-02-15T18:26:59.139 INFO:tasks.workunit.client.1.smithi191.stdout:0/89: truncate d7/da/f14 784764 0 2024-02-15T18:26:59.139 INFO:tasks.workunit.client.1.smithi191.stdout:0/90: stat d7/da/f14 0 2024-02-15T18:26:59.140 INFO:tasks.workunit.client.1.smithi191.stdout:9/157: mkdir d0/d7/dc/d30/d3b 0 2024-02-15T18:26:59.141 INFO:tasks.workunit.client.0.smithi042.stdout:2/454: write d0/f3 [2827895,58935] 0 2024-02-15T18:26:59.145 INFO:tasks.workunit.client.0.smithi042.stdout:3/498: dwrite d1/d11/d2e/f37 [4194304,4194304] 0 2024-02-15T18:26:59.147 INFO:tasks.workunit.client.1.smithi191.stdout:9/158: creat d0/f3c x:0 0 0 2024-02-15T18:26:59.148 INFO:tasks.workunit.client.0.smithi042.stdout:7/540: creat d10/d15/d8f/fb5 x:0 0 0 2024-02-15T18:26:59.149 INFO:tasks.workunit.client.1.smithi191.stdout:9/159: creat d0/d29/f3d x:0 0 0 2024-02-15T18:26:59.154 INFO:tasks.workunit.client.0.smithi042.stdout:4/476: dwrite da/d11/d67/d5c/d66/d78/d2f/d4d/f96 [0,4194304] 0 2024-02-15T18:26:59.154 INFO:tasks.workunit.client.0.smithi042.stdout:4/477: chown da/d3b/c5a 46478 1 2024-02-15T18:26:59.155 INFO:tasks.workunit.client.0.smithi042.stdout:0/489: rename d4/d9/d15/l51 to d4/d9/d34/d8b/la8 0 2024-02-15T18:26:59.159 INFO:tasks.workunit.client.0.smithi042.stdout:9/517: creat d6/da/d2f/d55/d73/d8d/fb2 x:0 0 0 2024-02-15T18:26:59.160 INFO:tasks.workunit.client.0.smithi042.stdout:2/455: creat d0/f8c x:0 0 0 2024-02-15T18:26:59.164 INFO:tasks.workunit.client.0.smithi042.stdout:3/499: rmdir d1/d11/d2e 39 2024-02-15T18:26:59.164 INFO:tasks.workunit.client.0.smithi042.stdout:7/541: mkdir d10/d49/d6d/db6 0 2024-02-15T18:26:59.165 INFO:tasks.workunit.client.0.smithi042.stdout:4/478: symlink da/d11/d29/d4a/d4f/d90/lac 0 2024-02-15T18:26:59.167 INFO:tasks.workunit.client.1.smithi191.stdout:5/82: dwrite d0/f1a [0,4194304] 0 2024-02-15T18:26:59.167 INFO:tasks.workunit.client.1.smithi191.stdout:5/83: read d0/f1a [1101751,253] 0 2024-02-15T18:26:59.168 INFO:tasks.workunit.client.0.smithi042.stdout:6/561: sync 2024-02-15T18:26:59.168 INFO:tasks.workunit.client.0.smithi042.stdout:8/436: sync 2024-02-15T18:26:59.168 INFO:tasks.workunit.client.0.smithi042.stdout:6/562: readlink d9/d11/d19/d35/d55/lbf 0 2024-02-15T18:26:59.169 INFO:tasks.workunit.client.0.smithi042.stdout:6/563: readlink d9/d11/d19/d1c/d31/l6b 0 2024-02-15T18:26:59.169 INFO:tasks.workunit.client.0.smithi042.stdout:6/564: write d9/d11/d19/fc1 [651292,74829] 0 2024-02-15T18:26:59.171 INFO:tasks.workunit.client.0.smithi042.stdout:9/518: rename d6/da/d1d/d2d/d40/d5f/fa0 to d6/da/d1d/d2d/d40/d95/fb3 0 2024-02-15T18:26:59.171 INFO:tasks.workunit.client.0.smithi042.stdout:9/519: fsync d6/dd/d9/f41 0 2024-02-15T18:26:59.171 INFO:tasks.workunit.client.0.smithi042.stdout:3/500: write d1/de/d1a/f77 [769942,36859] 0 2024-02-15T18:26:59.176 INFO:tasks.workunit.client.1.smithi191.stdout:7/98: sync 2024-02-15T18:26:59.176 INFO:tasks.workunit.client.1.smithi191.stdout:8/266: sync 2024-02-15T18:26:59.176 INFO:tasks.workunit.client.1.smithi191.stdout:7/99: readlink d0/d1/d3/d11/l1b 0 2024-02-15T18:26:59.176 INFO:tasks.workunit.client.1.smithi191.stdout:2/124: sync 2024-02-15T18:26:59.176 INFO:tasks.workunit.client.1.smithi191.stdout:6/187: sync 2024-02-15T18:26:59.178 INFO:tasks.workunit.client.1.smithi191.stdout:2/125: dread fd [0,4194304] 0 2024-02-15T18:26:59.182 INFO:tasks.workunit.client.0.smithi042.stdout:7/542: creat d10/d15/d8f/da5/fb7 x:0 0 0 2024-02-15T18:26:59.182 INFO:tasks.workunit.client.1.smithi191.stdout:7/100: symlink d0/d1/df/l25 0 2024-02-15T18:26:59.182 INFO:tasks.workunit.client.1.smithi191.stdout:7/101: chown d0 170 1 2024-02-15T18:26:59.183 INFO:tasks.workunit.client.1.smithi191.stdout:7/102: fsync d0/d1/df/f17 0 2024-02-15T18:26:59.184 INFO:tasks.workunit.client.0.smithi042.stdout:4/479: rename da/d11/f83 to da/fad 0 2024-02-15T18:26:59.184 INFO:tasks.workunit.client.0.smithi042.stdout:4/480: chown da/d11/d67/d5c/d66/d78/d2f/c98 82 1 2024-02-15T18:26:59.185 INFO:tasks.workunit.client.1.smithi191.stdout:2/126: write f7 [2452123,40644] 0 2024-02-15T18:26:59.185 INFO:tasks.workunit.client.0.smithi042.stdout:3/501: link d1/d11/c91 d1/d11/d14/d4d/d73/d89/ca9 0 2024-02-15T18:26:59.186 INFO:tasks.workunit.client.1.smithi191.stdout:7/103: rmdir d0/d1/d1d 39 2024-02-15T18:26:59.186 INFO:tasks.workunit.client.1.smithi191.stdout:7/104: fdatasync d0/d1/df/f1a 0 2024-02-15T18:26:59.186 INFO:tasks.workunit.client.1.smithi191.stdout:7/105: dread - d0/d1/df/f16 zero size 2024-02-15T18:26:59.186 INFO:tasks.workunit.client.1.smithi191.stdout:7/106: chown d0/d1/df/f17 26 1 2024-02-15T18:26:59.186 INFO:tasks.workunit.client.0.smithi042.stdout:8/437: dread dd/f2c [0,4194304] 0 2024-02-15T18:26:59.188 INFO:tasks.workunit.client.1.smithi191.stdout:2/127: mknod df/d12/d19/d1a/c23 0 2024-02-15T18:26:59.189 INFO:tasks.workunit.client.0.smithi042.stdout:0/490: dwrite d4/f3b [0,4194304] 0 2024-02-15T18:26:59.189 INFO:tasks.workunit.client.0.smithi042.stdout:7/543: creat d10/d43/d57/fb8 x:0 0 0 2024-02-15T18:26:59.190 INFO:tasks.workunit.client.0.smithi042.stdout:7/544: chown d10/f63 1333 1 2024-02-15T18:26:59.190 INFO:tasks.workunit.client.0.smithi042.stdout:3/502: write d1/de/d19/f1f [3690293,71312] 0 2024-02-15T18:26:59.190 INFO:tasks.workunit.client.0.smithi042.stdout:1/506: dwrite de/d15/d67/d74/f90 [0,4194304] 0 2024-02-15T18:26:59.190 INFO:tasks.workunit.client.0.smithi042.stdout:9/520: rename d6/da/d2f/d69/f74 to d6/da/d1d/d2d/fb4 0 2024-02-15T18:26:59.191 INFO:tasks.workunit.client.1.smithi191.stdout:5/84: dread d0/f13 [0,4194304] 0 2024-02-15T18:26:59.191 INFO:tasks.workunit.client.1.smithi191.stdout:7/107: mkdir d0/db/d26 0 2024-02-15T18:26:59.192 INFO:tasks.workunit.client.0.smithi042.stdout:2/456: dread d0/f71 [0,4194304] 0 2024-02-15T18:26:59.192 INFO:tasks.workunit.client.0.smithi042.stdout:8/438: read dd/d11/f15 [1711379,81448] 0 2024-02-15T18:26:59.194 INFO:tasks.workunit.client.1.smithi191.stdout:2/128: symlink df/d12/d1e/l24 0 2024-02-15T18:26:59.197 INFO:tasks.workunit.client.1.smithi191.stdout:5/85: creat d0/f1f x:0 0 0 2024-02-15T18:26:59.198 INFO:tasks.workunit.client.0.smithi042.stdout:1/507: write de/d15/d1b/f25 [3449870,89998] 0 2024-02-15T18:26:59.198 INFO:tasks.workunit.client.0.smithi042.stdout:1/508: write de/d54/fb0 [965503,35508] 0 2024-02-15T18:26:59.199 INFO:tasks.workunit.client.1.smithi191.stdout:7/108: symlink d0/d1/d3/dc/d12/d1e/l27 0 2024-02-15T18:26:59.200 INFO:tasks.workunit.client.1.smithi191.stdout:2/129: rmdir df/d12 39 2024-02-15T18:26:59.203 INFO:tasks.workunit.client.0.smithi042.stdout:4/481: rename da/d11/d67/f51 to da/d11/d29/d4a/d4f/d90/fae 0 2024-02-15T18:26:59.205 INFO:tasks.workunit.client.0.smithi042.stdout:8/439: creat dd/f7d x:0 0 0 2024-02-15T18:26:59.205 INFO:tasks.workunit.client.0.smithi042.stdout:8/440: chown dd/de/d3c/f54 1023 1 2024-02-15T18:26:59.205 INFO:tasks.workunit.client.0.smithi042.stdout:8/441: truncate fc 5239944 0 2024-02-15T18:26:59.205 INFO:tasks.workunit.client.0.smithi042.stdout:8/442: dread - dd/f7d zero size 2024-02-15T18:26:59.208 INFO:tasks.workunit.client.0.smithi042.stdout:0/491: rename d4/d9/d62/d74/l84 to d4/d38/d5c/la9 0 2024-02-15T18:26:59.208 INFO:tasks.workunit.client.0.smithi042.stdout:0/492: fdatasync d4/f41 0 2024-02-15T18:26:59.208 INFO:tasks.workunit.client.1.smithi191.stdout:5/86: mknod d0/c20 0 2024-02-15T18:26:59.209 INFO:tasks.workunit.client.1.smithi191.stdout:7/109: mkdir d0/d1/d3/dc/d12/d1e/d28 0 2024-02-15T18:26:59.209 INFO:tasks.workunit.client.1.smithi191.stdout:7/110: chown d0/d1/df/f1a 113715 1 2024-02-15T18:26:59.210 INFO:tasks.workunit.client.1.smithi191.stdout:5/87: unlink d0/f11 0 2024-02-15T18:26:59.212 INFO:tasks.workunit.client.1.smithi191.stdout:7/111: truncate d0/d1/f9 444281 0 2024-02-15T18:26:59.214 INFO:tasks.workunit.client.1.smithi191.stdout:5/88: rename d0/c12 to d0/d18/c21 0 2024-02-15T18:26:59.215 INFO:tasks.workunit.client.0.smithi042.stdout:4/482: write da/d11/d67/d5c/d66/d78/d2f/d4d/f75 [1934894,124372] 0 2024-02-15T18:26:59.215 INFO:tasks.workunit.client.0.smithi042.stdout:1/509: dread de/d54/f5e [0,4194304] 0 2024-02-15T18:26:59.215 INFO:tasks.workunit.client.1.smithi191.stdout:7/112: link d0/d1/df/l25 d0/d1/d1d/l29 0 2024-02-15T18:26:59.216 INFO:tasks.workunit.client.0.smithi042.stdout:4/483: symlink da/d11/d85/laf 0 2024-02-15T18:26:59.216 INFO:tasks.workunit.client.1.smithi191.stdout:5/89: write d0/f13 [1440782,61554] 0 2024-02-15T18:26:59.218 INFO:tasks.workunit.client.0.smithi042.stdout:1/510: link de/d15/d67/d74/d7e/fad de/d15/d2c/d51/d70/fba 0 2024-02-15T18:26:59.222 INFO:tasks.workunit.client.1.smithi191.stdout:8/267: dwrite d3/d2c/f1d [0,4194304] 0 2024-02-15T18:26:59.222 INFO:tasks.workunit.client.1.smithi191.stdout:7/113: read d0/f14 [1575834,57294] 0 2024-02-15T18:26:59.223 INFO:tasks.workunit.client.0.smithi042.stdout:4/484: getdents da/d3b/d41 0 2024-02-15T18:26:59.223 INFO:tasks.workunit.client.0.smithi042.stdout:6/565: dwrite d9/f67 [4194304,4194304] 0 2024-02-15T18:26:59.223 INFO:tasks.workunit.client.0.smithi042.stdout:8/443: dread dd/de/d3c/d4b/d67/f2e [0,4194304] 0 2024-02-15T18:26:59.223 INFO:tasks.workunit.client.0.smithi042.stdout:1/511: chown ca 22 1 2024-02-15T18:26:59.223 INFO:tasks.workunit.client.0.smithi042.stdout:4/485: readlink da/d11/d67/d5c/d66/d78/la8 0 2024-02-15T18:26:59.223 INFO:tasks.workunit.client.0.smithi042.stdout:4/486: write da/d11/d67/d5c/d66/f8a [65074,79378] 0 2024-02-15T18:26:59.223 INFO:tasks.workunit.client.0.smithi042.stdout:4/487: truncate da/d11/d67/d5c/d66/d78/f3e 4625884 0 2024-02-15T18:26:59.225 INFO:tasks.workunit.client.1.smithi191.stdout:7/114: write d0/f14 [1312559,107028] 0 2024-02-15T18:26:59.231 INFO:tasks.workunit.client.0.smithi042.stdout:3/503: dread d1/de/d1a/f77 [0,4194304] 0 2024-02-15T18:26:59.231 INFO:tasks.workunit.client.0.smithi042.stdout:3/504: chown d1/d11/d14/d4d/l68 18 1 2024-02-15T18:26:59.234 INFO:tasks.workunit.client.0.smithi042.stdout:7/545: dwrite d10/f14 [0,4194304] 0 2024-02-15T18:26:59.235 INFO:tasks.workunit.client.1.smithi191.stdout:7/115: creat d0/d1/f2a x:0 0 0 2024-02-15T18:26:59.235 INFO:tasks.workunit.client.0.smithi042.stdout:9/521: dwrite d6/dd/d9/d59/d5c/f7e [0,4194304] 0 2024-02-15T18:26:59.235 INFO:tasks.workunit.client.0.smithi042.stdout:0/493: dwrite d4/d38/d77/f7c [0,4194304] 0 2024-02-15T18:26:59.235 INFO:tasks.workunit.client.0.smithi042.stdout:0/494: write d4/d9/d34/d4e/f87 [4514479,110291] 0 2024-02-15T18:26:59.236 INFO:tasks.workunit.client.1.smithi191.stdout:7/116: symlink d0/d1/l2b 0 2024-02-15T18:26:59.238 INFO:tasks.workunit.client.1.smithi191.stdout:7/117: mknod d0/db/d26/c2c 0 2024-02-15T18:26:59.240 INFO:tasks.workunit.client.0.smithi042.stdout:6/566: creat d9/d11/d19/d1c/d31/d53/fc2 x:0 0 0 2024-02-15T18:26:59.240 INFO:tasks.workunit.client.0.smithi042.stdout:6/567: stat d9/f3c 0 2024-02-15T18:26:59.240 INFO:tasks.workunit.client.0.smithi042.stdout:1/512: mknod de/cbb 0 2024-02-15T18:26:59.241 INFO:tasks.workunit.client.0.smithi042.stdout:1/513: stat de/d15/d1b/d44/l62 0 2024-02-15T18:26:59.241 INFO:tasks.workunit.client.0.smithi042.stdout:1/514: truncate de/d15/d67/f3a 4351328 0 2024-02-15T18:26:59.242 INFO:tasks.workunit.client.1.smithi191.stdout:2/130: dwrite fb [0,4194304] 0 2024-02-15T18:26:59.243 INFO:tasks.workunit.client.1.smithi191.stdout:7/118: creat d0/db/f2d x:0 0 0 2024-02-15T18:26:59.243 INFO:tasks.workunit.client.1.smithi191.stdout:7/119: chown d0/c10 115730443 1 2024-02-15T18:26:59.243 INFO:tasks.workunit.client.1.smithi191.stdout:7/120: dread - d0/d1/f2a zero size 2024-02-15T18:26:59.244 INFO:tasks.workunit.client.1.smithi191.stdout:7/121: dread - d0/d1/f21 zero size 2024-02-15T18:26:59.244 INFO:tasks.workunit.client.1.smithi191.stdout:7/122: chown d0/d1/f9 127183 1 2024-02-15T18:26:59.244 INFO:tasks.workunit.client.1.smithi191.stdout:7/123: chown d0/d1/d3/dc 7 1 2024-02-15T18:26:59.245 INFO:tasks.workunit.client.0.smithi042.stdout:8/444: creat dd/de/d3c/f7e x:0 0 0 2024-02-15T18:26:59.246 INFO:tasks.workunit.client.1.smithi191.stdout:5/90: dwrite d0/f1b [0,4194304] 0 2024-02-15T18:26:59.246 INFO:tasks.workunit.client.0.smithi042.stdout:4/488: rename da/f42 to da/d11/d67/d5c/d66/fb0 0 2024-02-15T18:26:59.247 INFO:tasks.workunit.client.1.smithi191.stdout:8/268: dwrite d3/d2c/f48 [0,4194304] 0 2024-02-15T18:26:59.253 INFO:tasks.workunit.client.0.smithi042.stdout:9/522: dread d6/dd/f4a [0,4194304] 0 2024-02-15T18:26:59.258 INFO:tasks.workunit.client.1.smithi191.stdout:7/124: dread d0/f13 [0,4194304] 0 2024-02-15T18:26:59.258 INFO:tasks.workunit.client.1.smithi191.stdout:8/269: read d3/f31 [3862441,83671] 0 2024-02-15T18:26:59.260 INFO:tasks.workunit.client.0.smithi042.stdout:9/523: dread d6/da/d1d/d2d/d40/f78 [0,4194304] 0 2024-02-15T18:26:59.261 INFO:tasks.workunit.client.0.smithi042.stdout:9/524: fsync d6/dd/d9/d59/f63 0 2024-02-15T18:26:59.266 INFO:tasks.workunit.client.0.smithi042.stdout:5/539: sync 2024-02-15T18:26:59.269 INFO:tasks.workunit.client.1.smithi191.stdout:2/131: link df/d12/d1e/c20 df/c25 0 2024-02-15T18:26:59.270 INFO:tasks.workunit.client.1.smithi191.stdout:2/132: write df/f16 [587456,87227] 0 2024-02-15T18:26:59.271 INFO:tasks.workunit.client.1.smithi191.stdout:2/133: write df/f10 [801942,2836] 0 2024-02-15T18:26:59.272 INFO:tasks.workunit.client.0.smithi042.stdout:6/568: dwrite d9/d11/d19/d1c/d31/f56 [0,4194304] 0 2024-02-15T18:26:59.272 INFO:tasks.workunit.client.0.smithi042.stdout:6/569: write d9/d11/d19/d1c/d31/d4f/f98 [76287,130845] 0 2024-02-15T18:26:59.273 INFO:tasks.workunit.client.0.smithi042.stdout:6/570: chown d9/d11/d19/d35/d49/d79/l7d 7783831 1 2024-02-15T18:26:59.273 INFO:tasks.workunit.client.0.smithi042.stdout:3/505: mknod d1/de/d19/caa 0 2024-02-15T18:26:59.273 INFO:tasks.workunit.client.0.smithi042.stdout:3/506: fdatasync d1/d11/d14/f53 0 2024-02-15T18:26:59.275 INFO:tasks.workunit.client.0.smithi042.stdout:7/546: mknod d10/d12/d65/da7/cb9 0 2024-02-15T18:26:59.275 INFO:tasks.workunit.client.0.smithi042.stdout:7/547: fsync d10/d15/f1d 0 2024-02-15T18:26:59.277 INFO:tasks.workunit.client.0.smithi042.stdout:1/515: dwrite de/d15/d1b/d44/f85 [0,4194304] 0 2024-02-15T18:26:59.279 INFO:tasks.workunit.client.1.smithi191.stdout:8/270: creat d3/d2c/d1a/d23/d2a/d34/f60 x:0 0 0 2024-02-15T18:26:59.280 INFO:tasks.workunit.client.1.smithi191.stdout:7/125: creat d0/db/f2e x:0 0 0 2024-02-15T18:26:59.280 INFO:tasks.workunit.client.1.smithi191.stdout:7/126: write d0/d1/f9 [1254062,87690] 0 2024-02-15T18:26:59.283 INFO:tasks.workunit.client.0.smithi042.stdout:6/571: dread d9/d3e/f63 [0,4194304] 0 2024-02-15T18:26:59.288 INFO:tasks.workunit.client.1.smithi191.stdout:2/134: creat df/d12/d1f/f26 x:0 0 0 2024-02-15T18:26:59.290 INFO:tasks.workunit.client.1.smithi191.stdout:5/91: truncate d0/f1b 2747954 0 2024-02-15T18:26:59.292 INFO:tasks.workunit.client.0.smithi042.stdout:0/495: link d4/d9/d5a/f6b d4/d9/d34/d4e/d6c/d70/faa 0 2024-02-15T18:26:59.292 INFO:tasks.workunit.client.0.smithi042.stdout:0/496: write d4/d38/d77/f9b [103021,62868] 0 2024-02-15T18:26:59.293 INFO:tasks.workunit.client.1.smithi191.stdout:8/271: dread d3/d2c/d1a/f20 [0,4194304] 0 2024-02-15T18:26:59.294 INFO:tasks.workunit.client.0.smithi042.stdout:8/445: creat dd/de/d33/d66/f7f x:0 0 0 2024-02-15T18:26:59.294 INFO:tasks.workunit.client.0.smithi042.stdout:8/446: stat dd/de/d33/d66 0 2024-02-15T18:26:59.294 INFO:tasks.workunit.client.1.smithi191.stdout:7/127: creat d0/db/f2f x:0 0 0 2024-02-15T18:26:59.295 INFO:tasks.workunit.client.1.smithi191.stdout:7/128: write d0/d1/f2a [677319,92351] 0 2024-02-15T18:26:59.295 INFO:tasks.workunit.client.1.smithi191.stdout:7/129: write d0/d1/f2a [195280,101483] 0 2024-02-15T18:26:59.296 INFO:tasks.workunit.client.1.smithi191.stdout:7/130: chown d0/d1/d3/ce 152458652 1 2024-02-15T18:26:59.296 INFO:tasks.workunit.client.0.smithi042.stdout:4/489: link da/d11/f22 da/d3b/d41/d8d/fb1 0 2024-02-15T18:26:59.298 INFO:tasks.workunit.client.0.smithi042.stdout:5/540: unlink d0/db/c1a 0 2024-02-15T18:26:59.298 INFO:tasks.workunit.client.0.smithi042.stdout:5/541: fsync d0/d38/d53/d68/d6a/f56 0 2024-02-15T18:26:59.298 INFO:tasks.workunit.client.0.smithi042.stdout:5/542: chown d0/d14/f63 0 1 2024-02-15T18:26:59.302 INFO:tasks.workunit.client.1.smithi191.stdout:2/135: mknod df/d12/d19/d1a/c27 0 2024-02-15T18:26:59.302 INFO:tasks.workunit.client.1.smithi191.stdout:7/131: mknod d0/d1/c30 0 2024-02-15T18:26:59.302 INFO:tasks.workunit.client.1.smithi191.stdout:8/272: rename d3/d2c/d1a/d36/c53 to d3/d2c/c61 0 2024-02-15T18:26:59.304 INFO:tasks.workunit.client.0.smithi042.stdout:3/507: unlink d1/d11/d14/d4d/d6c/d7b/fa1 0 2024-02-15T18:26:59.304 INFO:tasks.workunit.client.0.smithi042.stdout:7/548: mkdir d10/d15/d39/d6c/dba 0 2024-02-15T18:26:59.305 INFO:tasks.workunit.client.1.smithi191.stdout:2/136: mknod df/d12/d1f/c28 0 2024-02-15T18:26:59.306 INFO:tasks.workunit.client.1.smithi191.stdout:7/132: mkdir d0/d1/d3/d11/d31 0 2024-02-15T18:26:59.308 INFO:tasks.workunit.client.1.smithi191.stdout:8/273: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d5d/c62 0 2024-02-15T18:26:59.308 INFO:tasks.workunit.client.1.smithi191.stdout:8/274: dread - d3/d2c/d1a/f51 zero size 2024-02-15T18:26:59.310 INFO:tasks.workunit.client.1.smithi191.stdout:2/137: rmdir df/d12/d1e 39 2024-02-15T18:26:59.311 INFO:tasks.workunit.client.1.smithi191.stdout:7/133: creat d0/d1/d3/dc/d12/f32 x:0 0 0 2024-02-15T18:26:59.311 INFO:tasks.workunit.client.1.smithi191.stdout:7/134: stat d0/d1 0 2024-02-15T18:26:59.312 INFO:tasks.workunit.client.1.smithi191.stdout:7/135: stat d0/d1/ca 0 2024-02-15T18:26:59.312 INFO:tasks.workunit.client.0.smithi042.stdout:1/516: rename de/d15/d67/d89/faa to de/d54/d9a/fbc 0 2024-02-15T18:26:59.313 INFO:tasks.workunit.client.1.smithi191.stdout:8/275: creat d3/d2c/d1a/d23/d2e/d47/f63 x:0 0 0 2024-02-15T18:26:59.313 INFO:tasks.workunit.client.0.smithi042.stdout:0/497: mkdir d4/d9/d34/dab 0 2024-02-15T18:26:59.314 INFO:tasks.workunit.client.0.smithi042.stdout:0/498: fdatasync d4/d9/d34/d57/f6f 0 2024-02-15T18:26:59.316 INFO:tasks.workunit.client.0.smithi042.stdout:8/447: truncate dd/f29 2794196 0 2024-02-15T18:26:59.316 INFO:tasks.workunit.client.0.smithi042.stdout:8/448: fsync dd/de/d3c/d4b/f76 0 2024-02-15T18:26:59.317 INFO:tasks.workunit.client.1.smithi191.stdout:2/138: dread f9 [0,4194304] 0 2024-02-15T18:26:59.317 INFO:tasks.workunit.client.1.smithi191.stdout:7/136: creat d0/d1/d3/dc/d12/d1e/f33 x:0 0 0 2024-02-15T18:26:59.319 INFO:tasks.workunit.client.0.smithi042.stdout:4/490: creat da/d11/d29/d4a/d4f/d90/fb2 x:0 0 0 2024-02-15T18:26:59.327 INFO:tasks.workunit.client.1.smithi191.stdout:7/137: rmdir d0/d1/d3/d11/d31 0 2024-02-15T18:26:59.328 INFO:tasks.workunit.client.0.smithi042.stdout:5/543: unlink d0/d38/d53/d68/d6a/f32 0 2024-02-15T18:26:59.329 INFO:tasks.workunit.client.0.smithi042.stdout:3/508: symlink d1/de/lab 0 2024-02-15T18:26:59.329 INFO:tasks.workunit.client.0.smithi042.stdout:7/549: unlink d10/d12/f3f 0 2024-02-15T18:26:59.331 INFO:tasks.workunit.client.0.smithi042.stdout:0/499: dread d4/d9/d34/d4e/d6c/f71 [0,4194304] 0 2024-02-15T18:26:59.331 INFO:tasks.workunit.client.0.smithi042.stdout:6/572: dwrite d9/d11/d1d/f88 [0,4194304] 0 2024-02-15T18:26:59.331 INFO:tasks.workunit.client.0.smithi042.stdout:6/573: write d9/d11/d19/d1c/d31/d4f/f98 [992087,59465] 0 2024-02-15T18:26:59.331 INFO:tasks.workunit.client.0.smithi042.stdout:6/574: stat d9 0 2024-02-15T18:26:59.334 INFO:tasks.workunit.client.1.smithi191.stdout:7/138: mknod d0/d1/d3/dc/d12/c34 0 2024-02-15T18:26:59.335 INFO:tasks.workunit.client.1.smithi191.stdout:5/92: dwrite d0/d14/f17 [0,4194304] 0 2024-02-15T18:26:59.338 INFO:tasks.workunit.client.1.smithi191.stdout:7/139: dread d0/d1/f2a [0,4194304] 0 2024-02-15T18:26:59.341 INFO:tasks.workunit.client.1.smithi191.stdout:2/139: dwrite f9 [0,4194304] 0 2024-02-15T18:26:59.345 INFO:tasks.workunit.client.0.smithi042.stdout:2/457: sync 2024-02-15T18:26:59.346 INFO:tasks.workunit.client.1.smithi191.stdout:8/276: rename d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d5d to d3/d2c/d1a/d23/d2e/d64 0 2024-02-15T18:26:59.347 INFO:tasks.workunit.client.1.smithi191.stdout:8/277: chown d3/d2c/d1a/l1e 2796016 1 2024-02-15T18:26:59.347 INFO:tasks.workunit.client.1.smithi191.stdout:8/278: chown d3/d2c/f1f 1 1 2024-02-15T18:26:59.349 INFO:tasks.workunit.client.0.smithi042.stdout:8/449: dwrite dd/de/d3c/f70 [0,4194304] 0 2024-02-15T18:26:59.350 INFO:tasks.workunit.client.0.smithi042.stdout:8/450: stat dd/de/d33/d3b 0 2024-02-15T18:26:59.355 INFO:tasks.workunit.client.0.smithi042.stdout:1/517: creat de/d15/d1b/d44/fbd x:0 0 0 2024-02-15T18:26:59.355 INFO:tasks.workunit.client.0.smithi042.stdout:4/491: symlink da/d2b/lb3 0 2024-02-15T18:26:59.355 INFO:tasks.workunit.client.0.smithi042.stdout:3/509: rmdir d1/d11/d14/d4d/d73/d89 39 2024-02-15T18:26:59.357 INFO:tasks.workunit.client.0.smithi042.stdout:7/550: truncate d10/d15/d39/d6c/f8d 1064061 0 2024-02-15T18:26:59.358 INFO:tasks.workunit.client.0.smithi042.stdout:7/551: dread - d10/d12/f96 zero size 2024-02-15T18:26:59.358 INFO:tasks.workunit.client.0.smithi042.stdout:7/552: fdatasync d10/d49/d6d/d78/d83/fa3 0 2024-02-15T18:26:59.359 INFO:tasks.workunit.client.1.smithi191.stdout:5/93: link d0/f13 d0/f22 0 2024-02-15T18:26:59.360 INFO:tasks.workunit.client.0.smithi042.stdout:5/544: dwrite d0/d38/d53/d68/d6a/d42/f95 [0,4194304] 0 2024-02-15T18:26:59.362 INFO:tasks.workunit.client.1.smithi191.stdout:7/140: mkdir d0/d1/d35 0 2024-02-15T18:26:59.363 INFO:tasks.workunit.client.1.smithi191.stdout:2/140: creat df/d12/d19/d1a/f29 x:0 0 0 2024-02-15T18:26:59.365 INFO:tasks.workunit.client.0.smithi042.stdout:6/575: unlink d9/d11/d19/d35/d49/d79/laf 0 2024-02-15T18:26:59.365 INFO:tasks.workunit.client.0.smithi042.stdout:6/576: fsync d9/f8e 0 2024-02-15T18:26:59.368 INFO:tasks.workunit.client.0.smithi042.stdout:2/458: getdents d0/d40/d56 0 2024-02-15T18:26:59.368 INFO:tasks.workunit.client.1.smithi191.stdout:8/279: unlink d3/d2c/f1d 0 2024-02-15T18:26:59.370 INFO:tasks.workunit.client.0.smithi042.stdout:0/500: dwrite d4/d38/f9f [0,4194304] 0 2024-02-15T18:26:59.373 INFO:tasks.workunit.client.1.smithi191.stdout:2/141: creat df/d12/f2a x:0 0 0 2024-02-15T18:26:59.377 INFO:tasks.workunit.client.1.smithi191.stdout:1/181: sync 2024-02-15T18:26:59.377 INFO:tasks.workunit.client.1.smithi191.stdout:1/182: chown d0/d8/df/d12/f23 196516393 1 2024-02-15T18:26:59.377 INFO:tasks.workunit.client.1.smithi191.stdout:1/183: chown d0/c7 944 1 2024-02-15T18:26:59.377 INFO:tasks.workunit.client.0.smithi042.stdout:8/451: dwrite dd/de/d33/d66/d4e/f7c [0,4194304] 0 2024-02-15T18:26:59.378 INFO:tasks.workunit.client.1.smithi191.stdout:8/280: creat d3/d2c/d1a/d36/f65 x:0 0 0 2024-02-15T18:26:59.378 INFO:tasks.workunit.client.1.smithi191.stdout:8/281: readlink d3/l30 0 2024-02-15T18:26:59.381 INFO:tasks.workunit.client.1.smithi191.stdout:2/142: creat df/d12/f2b x:0 0 0 2024-02-15T18:26:59.381 INFO:tasks.workunit.client.1.smithi191.stdout:5/94: dwrite d0/d18/f1e [0,4194304] 0 2024-02-15T18:26:59.381 INFO:tasks.workunit.client.1.smithi191.stdout:1/184: write d0/d8/f18 [4088921,33173] 0 2024-02-15T18:26:59.381 INFO:tasks.workunit.client.1.smithi191.stdout:1/185: stat d0/d8 0 2024-02-15T18:26:59.382 INFO:tasks.workunit.client.0.smithi042.stdout:1/518: mknod de/d6d/d9e/cbe 0 2024-02-15T18:26:59.383 INFO:tasks.workunit.client.1.smithi191.stdout:8/282: symlink d3/d2c/d1a/d23/l66 0 2024-02-15T18:26:59.383 INFO:tasks.workunit.client.1.smithi191.stdout:8/283: write d3/d2c/d1a/d23/d2a/d34/f60 [141803,7222] 0 2024-02-15T18:26:59.384 INFO:tasks.workunit.client.1.smithi191.stdout:2/143: dread df/d12/f22 [0,4194304] 0 2024-02-15T18:26:59.384 INFO:tasks.workunit.client.1.smithi191.stdout:2/144: chown df/d12/d1e/l24 36816565 1 2024-02-15T18:26:59.384 INFO:tasks.workunit.client.0.smithi042.stdout:4/492: creat da/d11/da4/fb4 x:0 0 0 2024-02-15T18:26:59.391 INFO:tasks.workunit.client.0.smithi042.stdout:3/510: symlink d1/d11/d14/d4d/d6c/lac 0 2024-02-15T18:26:59.394 INFO:tasks.workunit.client.0.smithi042.stdout:8/452: dwrite dd/f3e [0,4194304] 0 2024-02-15T18:26:59.395 INFO:tasks.workunit.client.0.smithi042.stdout:7/553: creat d10/d49/d6d/fbb x:0 0 0 2024-02-15T18:26:59.396 INFO:tasks.workunit.client.0.smithi042.stdout:3/511: dread d1/d11/d2e/d4c/f5b [0,4194304] 0 2024-02-15T18:26:59.396 INFO:tasks.workunit.client.1.smithi191.stdout:1/186: mknod d0/d8/df/d2e/c39 0 2024-02-15T18:26:59.398 INFO:tasks.workunit.client.1.smithi191.stdout:8/284: rename d3/d2c/d1a/d23/d2a/d34 to d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67 0 2024-02-15T18:26:59.399 INFO:tasks.workunit.client.0.smithi042.stdout:5/545: mkdir d0/d12/d65/d74/d52/dad/dba/dbd 0 2024-02-15T18:26:59.399 INFO:tasks.workunit.client.0.smithi042.stdout:5/546: chown d0/d39/l6c 0 1 2024-02-15T18:26:59.399 INFO:tasks.workunit.client.1.smithi191.stdout:2/145: creat df/d12/d19/d1a/f2c x:0 0 0 2024-02-15T18:26:59.400 INFO:tasks.workunit.client.0.smithi042.stdout:6/577: truncate d9/d11/d19/d35/d55/d92/d9e/faa 6236736 0 2024-02-15T18:26:59.401 INFO:tasks.workunit.client.1.smithi191.stdout:1/187: rename d0/db/f26 to d0/d8/df/d12/d2a/f3a 0 2024-02-15T18:26:59.403 INFO:tasks.workunit.client.0.smithi042.stdout:2/459: rmdir d0/d8/d20 39 2024-02-15T18:26:59.403 INFO:tasks.workunit.client.0.smithi042.stdout:4/493: dwrite da/d11/d29/d4a/d4f/f6c [0,4194304] 0 2024-02-15T18:26:59.406 INFO:tasks.workunit.client.1.smithi191.stdout:2/146: mknod df/c2d 0 2024-02-15T18:26:59.407 INFO:tasks.workunit.client.1.smithi191.stdout:2/147: dread - df/d12/d19/d1a/f29 zero size 2024-02-15T18:26:59.407 INFO:tasks.workunit.client.1.smithi191.stdout:2/148: truncate df/d12/f2b 618964 0 2024-02-15T18:26:59.408 INFO:tasks.workunit.client.1.smithi191.stdout:2/149: dread df/d12/f22 [0,4194304] 0 2024-02-15T18:26:59.408 INFO:tasks.workunit.client.0.smithi042.stdout:8/453: dread dd/de/d12/f24 [0,4194304] 0 2024-02-15T18:26:59.409 INFO:tasks.workunit.client.0.smithi042.stdout:3/512: write d1/d11/f45 [7855097,92901] 0 2024-02-15T18:26:59.411 INFO:tasks.workunit.client.1.smithi191.stdout:1/188: rename d0/d8/f29 to d0/d8/df/d12/f3b 0 2024-02-15T18:26:59.411 INFO:tasks.workunit.client.1.smithi191.stdout:8/285: rmdir d3/d2c/d1a/d23/d2e/d47 39 2024-02-15T18:26:59.416 INFO:tasks.workunit.client.0.smithi042.stdout:9/525: sync 2024-02-15T18:26:59.417 INFO:tasks.workunit.client.1.smithi191.stdout:2/150: truncate df/f10 3648782 0 2024-02-15T18:26:59.419 INFO:tasks.workunit.client.1.smithi191.stdout:1/189: rmdir d0/d2f 39 2024-02-15T18:26:59.419 INFO:tasks.workunit.client.1.smithi191.stdout:1/190: write d0/f1b [1546908,98108] 0 2024-02-15T18:26:59.420 INFO:tasks.workunit.client.0.smithi042.stdout:3/513: dread d1/d11/d14/f39 [0,4194304] 0 2024-02-15T18:26:59.423 INFO:tasks.workunit.client.1.smithi191.stdout:1/191: mknod d0/c3c 0 2024-02-15T18:26:59.423 INFO:tasks.workunit.client.1.smithi191.stdout:1/192: write d0/db/d10/f25 [203446,111931] 0 2024-02-15T18:26:59.424 INFO:tasks.workunit.client.0.smithi042.stdout:0/501: symlink d4/d9/d5a/lac 0 2024-02-15T18:26:59.428 INFO:tasks.workunit.client.1.smithi191.stdout:1/193: creat d0/d8/df/d2e/f3d x:0 0 0 2024-02-15T18:26:59.428 INFO:tasks.workunit.client.1.smithi191.stdout:1/194: chown d0/l1e 433 1 2024-02-15T18:26:59.428 INFO:tasks.workunit.client.1.smithi191.stdout:1/195: chown d0/d2f/c34 2914778 1 2024-02-15T18:26:59.430 INFO:tasks.workunit.client.0.smithi042.stdout:1/519: mkdir de/d54/dbf 0 2024-02-15T18:26:59.434 INFO:tasks.workunit.client.0.smithi042.stdout:7/554: truncate d10/d43/f84 691825 0 2024-02-15T18:26:59.434 INFO:tasks.workunit.client.1.smithi191.stdout:1/196: getdents d0/d2f 0 2024-02-15T18:26:59.435 INFO:tasks.workunit.client.0.smithi042.stdout:5/547: creat d0/d89/fbe x:0 0 0 2024-02-15T18:26:59.436 INFO:tasks.workunit.client.1.smithi191.stdout:8/286: dwrite d3/d2c/d1a/d23/f54 [0,4194304] 0 2024-02-15T18:26:59.437 INFO:tasks.workunit.client.1.smithi191.stdout:2/151: write fd [2475500,108568] 0 2024-02-15T18:26:59.438 INFO:tasks.workunit.client.0.smithi042.stdout:9/526: dwrite d6/da/d2f/d6c/f71 [0,4194304] 0 2024-02-15T18:26:59.439 INFO:tasks.workunit.client.1.smithi191.stdout:1/197: rename d0/d2f/l38 to d0/d8/d27/l3e 0 2024-02-15T18:26:59.439 INFO:tasks.workunit.client.1.smithi191.stdout:1/198: fdatasync d0/db/f14 0 2024-02-15T18:26:59.440 INFO:tasks.workunit.client.0.smithi042.stdout:2/460: creat d0/d40/d56/f8d x:0 0 0 2024-02-15T18:26:59.440 INFO:tasks.workunit.client.0.smithi042.stdout:6/578: mkdir d9/d11/d46/dc3 0 2024-02-15T18:26:59.440 INFO:tasks.workunit.client.0.smithi042.stdout:2/461: readlink d0/d40/d56/l5b 0 2024-02-15T18:26:59.440 INFO:tasks.workunit.client.0.smithi042.stdout:2/462: chown d0/d8/d10/d31/d3a/d53/d3c/f87 28685 1 2024-02-15T18:26:59.440 INFO:tasks.workunit.client.0.smithi042.stdout:2/463: dread - d0/d8/d10/d5a/f88 zero size 2024-02-15T18:26:59.452 INFO:tasks.workunit.client.1.smithi191.stdout:1/199: dread d0/d8/f18 [0,4194304] 0 2024-02-15T18:26:59.452 INFO:tasks.workunit.client.1.smithi191.stdout:1/200: chown d0/db/lc 53283012 1 2024-02-15T18:26:59.454 INFO:tasks.workunit.client.0.smithi042.stdout:1/520: dwrite de/d15/f20 [0,4194304] 0 2024-02-15T18:26:59.455 INFO:tasks.workunit.client.1.smithi191.stdout:8/287: link d3/d2c/d1a/d23/f54 d3/d2c/d1a/d23/d3e/f68 0 2024-02-15T18:26:59.456 INFO:tasks.workunit.client.0.smithi042.stdout:4/494: mkdir da/d11/d85/db5 0 2024-02-15T18:26:59.456 INFO:tasks.workunit.client.0.smithi042.stdout:8/454: stat dd/de/d3c/d4b/d67/l77 0 2024-02-15T18:26:59.456 INFO:tasks.workunit.client.0.smithi042.stdout:3/514: truncate d1/de/d19/f48 349267 0 2024-02-15T18:26:59.456 INFO:tasks.workunit.client.0.smithi042.stdout:0/502: creat d4/d9/d34/d4e/d6c/d70/fad x:0 0 0 2024-02-15T18:26:59.458 INFO:tasks.workunit.client.1.smithi191.stdout:2/152: rmdir df/d12 39 2024-02-15T18:26:59.462 INFO:tasks.workunit.client.0.smithi042.stdout:5/548: unlink d0/d38/d53/d68/d6a/d42/d7d/c8c 0 2024-02-15T18:26:59.466 INFO:tasks.workunit.client.0.smithi042.stdout:6/579: mkdir d9/d11/d19/d35/dc4 0 2024-02-15T18:26:59.466 INFO:tasks.workunit.client.0.smithi042.stdout:6/580: fdatasync d9/f17 0 2024-02-15T18:26:59.466 INFO:tasks.workunit.client.0.smithi042.stdout:6/581: stat d9/f3c 0 2024-02-15T18:26:59.466 INFO:tasks.workunit.client.0.smithi042.stdout:6/582: readlink d9/d11/d19/d35/d49/l50 0 2024-02-15T18:26:59.466 INFO:tasks.workunit.client.0.smithi042.stdout:6/583: chown d9/d11/d19/d1c/d31 50442 1 2024-02-15T18:26:59.467 INFO:tasks.workunit.client.0.smithi042.stdout:1/521: creat de/d15/d67/d46/fc0 x:0 0 0 2024-02-15T18:26:59.469 INFO:tasks.workunit.client.1.smithi191.stdout:1/201: creat d0/d8/df/d12/f3f x:0 0 0 2024-02-15T18:26:59.469 INFO:tasks.workunit.client.0.smithi042.stdout:9/527: dwrite d6/da/d2f/d55/fa3 [0,4194304] 0 2024-02-15T18:26:59.469 INFO:tasks.workunit.client.0.smithi042.stdout:9/528: readlink d6/da/d2f/d69/l6e 0 2024-02-15T18:26:59.470 INFO:tasks.workunit.client.1.smithi191.stdout:8/288: mkdir d3/d2c/d1a/d23/d2e/d5b/d69 0 2024-02-15T18:26:59.470 INFO:tasks.workunit.client.0.smithi042.stdout:2/464: dwrite d0/d8/d10/d31/d3a/d53/f66 [0,4194304] 0 2024-02-15T18:26:59.470 INFO:tasks.workunit.client.0.smithi042.stdout:2/465: write d0/d40/d56/f73 [1046743,31389] 0 2024-02-15T18:26:59.473 INFO:tasks.workunit.client.0.smithi042.stdout:4/495: link da/d11/d27/ca5 da/d2b/cb6 0 2024-02-15T18:26:59.480 INFO:tasks.workunit.client.1.smithi191.stdout:2/153: symlink df/d12/d19/l2e 0 2024-02-15T18:26:59.480 INFO:tasks.workunit.client.1.smithi191.stdout:2/154: fdatasync df/d12/d19/d1a/f29 0 2024-02-15T18:26:59.480 INFO:tasks.workunit.client.1.smithi191.stdout:2/155: read - df/d12/d19/d1a/f21 zero size 2024-02-15T18:26:59.480 INFO:tasks.workunit.client.1.smithi191.stdout:2/156: chown df/d12/d19 594170 1 2024-02-15T18:26:59.481 INFO:tasks.workunit.client.0.smithi042.stdout:8/455: symlink dd/de/l80 0 2024-02-15T18:26:59.483 INFO:tasks.workunit.client.1.smithi191.stdout:8/289: rename d3/f21 to d3/d2c/d1a/d23/f6a 0 2024-02-15T18:26:59.484 INFO:tasks.workunit.client.0.smithi042.stdout:3/515: creat d1/d11/d7a/fad x:0 0 0 2024-02-15T18:26:59.484 INFO:tasks.workunit.client.1.smithi191.stdout:2/157: symlink df/l2f 0 2024-02-15T18:26:59.486 INFO:tasks.workunit.client.1.smithi191.stdout:2/158: write fd [3609326,5314] 0 2024-02-15T18:26:59.486 INFO:tasks.workunit.client.1.smithi191.stdout:8/290: creat d3/d2c/d1a/d36/f6b x:0 0 0 2024-02-15T18:26:59.486 INFO:tasks.workunit.client.1.smithi191.stdout:8/291: readlink d3/d2c/d1a/d23/l25 0 2024-02-15T18:26:59.486 INFO:tasks.workunit.client.1.smithi191.stdout:8/292: write d3/f29 [823599,26156] 0 2024-02-15T18:26:59.487 INFO:tasks.workunit.client.0.smithi042.stdout:7/555: mknod d10/d12/d23/cbc 0 2024-02-15T18:26:59.488 INFO:tasks.workunit.client.1.smithi191.stdout:2/159: mknod df/d12/d1f/c30 0 2024-02-15T18:26:59.488 INFO:tasks.workunit.client.1.smithi191.stdout:2/160: chown fd 7 1 2024-02-15T18:26:59.488 INFO:tasks.workunit.client.1.smithi191.stdout:2/161: dread - df/d12/d19/d1a/f21 zero size 2024-02-15T18:26:59.488 INFO:tasks.workunit.client.1.smithi191.stdout:2/162: truncate df/d12/d19/d1a/f2c 544078 0 2024-02-15T18:26:59.488 INFO:tasks.workunit.client.1.smithi191.stdout:2/163: chown df/c15 34826092 1 2024-02-15T18:26:59.489 INFO:tasks.workunit.client.0.smithi042.stdout:6/584: dwrite d9/d11/d19/d1c/d31/f56 [0,4194304] 0 2024-02-15T18:26:59.493 INFO:tasks.workunit.client.0.smithi042.stdout:5/549: rmdir d0/d39 39 2024-02-15T18:26:59.493 INFO:tasks.workunit.client.0.smithi042.stdout:5/550: fsync d0/f3 0 2024-02-15T18:26:59.493 INFO:tasks.workunit.client.0.smithi042.stdout:5/551: fsync d0/f9d 0 2024-02-15T18:26:59.494 INFO:tasks.workunit.client.1.smithi191.stdout:8/293: symlink d3/d2c/d1a/l6c 0 2024-02-15T18:26:59.494 INFO:tasks.workunit.client.1.smithi191.stdout:8/294: write d3/d2c/d1a/d23/f4c [24004,41880] 0 2024-02-15T18:26:59.495 INFO:tasks.workunit.client.0.smithi042.stdout:1/522: unlink de/c1a 0 2024-02-15T18:26:59.496 INFO:tasks.workunit.client.1.smithi191.stdout:1/202: dwrite d0/db/d10/f2d [0,4194304] 0 2024-02-15T18:26:59.497 INFO:tasks.workunit.client.1.smithi191.stdout:8/295: mkdir d3/d2c/d1a/d23/d2e/d5b/d69/d6d 0 2024-02-15T18:26:59.498 INFO:tasks.workunit.client.1.smithi191.stdout:8/296: chown d3/d2c/f17 3709 1 2024-02-15T18:26:59.499 INFO:tasks.workunit.client.0.smithi042.stdout:9/529: symlink d6/da/d75/lb5 0 2024-02-15T18:26:59.499 INFO:tasks.workunit.client.0.smithi042.stdout:9/530: truncate d6/da/d1d/f9c 317355 0 2024-02-15T18:26:59.499 INFO:tasks.workunit.client.0.smithi042.stdout:9/531: chown d6/da/d7b 3390 1 2024-02-15T18:26:59.502 INFO:tasks.workunit.client.0.smithi042.stdout:2/466: creat d0/d8/d10/d31/d79/f8e x:0 0 0 2024-02-15T18:26:59.502 INFO:tasks.workunit.client.0.smithi042.stdout:2/467: readlink d0/d8/l5c 0 2024-02-15T18:26:59.505 INFO:tasks.workunit.client.0.smithi042.stdout:4/496: getdents da/d3b/d41/d8d 0 2024-02-15T18:26:59.508 INFO:tasks.workunit.client.1.smithi191.stdout:2/164: dwrite df/d12/d1f/f26 [0,4194304] 0 2024-02-15T18:26:59.508 INFO:tasks.workunit.client.0.smithi042.stdout:8/456: mkdir dd/de/d3c/d4b/d81 0 2024-02-15T18:26:59.515 INFO:tasks.workunit.client.0.smithi042.stdout:0/503: mkdir d4/d38/d77/d2e/dae 0 2024-02-15T18:26:59.515 INFO:tasks.workunit.client.0.smithi042.stdout:0/504: write d4/d38/f3a [5557106,7911] 0 2024-02-15T18:26:59.520 INFO:tasks.workunit.client.0.smithi042.stdout:6/585: dwrite d9/d11/d19/f93 [4194304,4194304] 0 2024-02-15T18:26:59.520 INFO:tasks.workunit.client.1.smithi191.stdout:1/203: dwrite d0/db/fe [0,4194304] 0 2024-02-15T18:26:59.521 INFO:tasks.workunit.client.0.smithi042.stdout:7/556: mknod d10/d15/cbd 0 2024-02-15T18:26:59.523 INFO:tasks.workunit.client.0.smithi042.stdout:5/552: stat d0/d39/c4f 0 2024-02-15T18:26:59.523 INFO:tasks.workunit.client.0.smithi042.stdout:5/553: chown d0/d38/d53/d68/d6a/d42 9 1 2024-02-15T18:26:59.524 INFO:tasks.workunit.client.0.smithi042.stdout:1/523: symlink de/d15/lc1 0 2024-02-15T18:26:59.525 INFO:tasks.workunit.client.0.smithi042.stdout:1/524: getdents de/d15/d67/d46/d6a 0 2024-02-15T18:26:59.525 INFO:tasks.workunit.client.0.smithi042.stdout:1/525: truncate de/d15/d2c/f5f 1104354 0 2024-02-15T18:26:59.525 INFO:tasks.workunit.client.0.smithi042.stdout:8/457: dwrite dd/de/d12/f68 [0,4194304] 0 2024-02-15T18:26:59.525 INFO:tasks.workunit.client.0.smithi042.stdout:8/458: chown dd/de/d12/c47 12184301 1 2024-02-15T18:26:59.525 INFO:tasks.workunit.client.0.smithi042.stdout:9/532: mknod d6/da/d1d/d2d/d40/cb6 0 2024-02-15T18:26:59.526 INFO:tasks.workunit.client.0.smithi042.stdout:9/533: truncate d6/da/d1d/d2d/d40/f49 68996 0 2024-02-15T18:26:59.527 INFO:tasks.workunit.client.0.smithi042.stdout:2/468: symlink d0/d8/d10/d31/d3a/d53/l8f 0 2024-02-15T18:26:59.527 INFO:tasks.workunit.client.0.smithi042.stdout:2/469: write d0/d35/f81 [1099592,7168] 0 2024-02-15T18:26:59.527 INFO:tasks.workunit.client.0.smithi042.stdout:2/470: fsync d0/d8/db/fd 0 2024-02-15T18:26:59.528 INFO:tasks.workunit.client.0.smithi042.stdout:5/554: dread d0/d14/f2e [0,4194304] 0 2024-02-15T18:26:59.528 INFO:tasks.workunit.client.1.smithi191.stdout:2/165: creat df/d12/d1f/f31 x:0 0 0 2024-02-15T18:26:59.529 INFO:tasks.workunit.client.0.smithi042.stdout:4/497: link da/d11/d67/f95 da/d2b/d6e/fb7 0 2024-02-15T18:26:59.531 INFO:tasks.workunit.client.0.smithi042.stdout:4/498: dread da/d11/d67/d5c/d66/f8a [0,4194304] 0 2024-02-15T18:26:59.532 INFO:tasks.workunit.client.1.smithi191.stdout:0/91: sync 2024-02-15T18:26:59.532 INFO:tasks.workunit.client.1.smithi191.stdout:3/127: sync 2024-02-15T18:26:59.532 INFO:tasks.workunit.client.1.smithi191.stdout:0/92: write d7/da/f14 [1312666,121355] 0 2024-02-15T18:26:59.532 INFO:tasks.workunit.client.1.smithi191.stdout:0/93: fdatasync d7/fb 0 2024-02-15T18:26:59.535 INFO:tasks.workunit.client.0.smithi042.stdout:3/516: symlink d1/d11/d14/lae 0 2024-02-15T18:26:59.535 INFO:tasks.workunit.client.0.smithi042.stdout:3/517: chown d1/d9c/l9f 5829928 1 2024-02-15T18:26:59.536 INFO:tasks.workunit.client.1.smithi191.stdout:1/204: dread d0/db/d10/f2d [0,4194304] 0 2024-02-15T18:26:59.539 INFO:tasks.workunit.client.1.smithi191.stdout:3/128: mknod d6/d1b/d1f/c2e 0 2024-02-15T18:26:59.541 INFO:tasks.workunit.client.0.smithi042.stdout:0/505: rename d4/d38/d5c/f89 to d4/d9/d34/d57/faf 0 2024-02-15T18:26:59.541 INFO:tasks.workunit.client.0.smithi042.stdout:0/506: write d4/f7 [1326169,99437] 0 2024-02-15T18:26:59.542 INFO:tasks.workunit.client.1.smithi191.stdout:0/94: creat d7/dd/f15 x:0 0 0 2024-02-15T18:26:59.543 INFO:tasks.workunit.client.1.smithi191.stdout:1/205: creat d0/d2f/f40 x:0 0 0 2024-02-15T18:26:59.545 INFO:tasks.workunit.client.0.smithi042.stdout:6/586: creat d9/d11/d19/d1c/fc5 x:0 0 0 2024-02-15T18:26:59.546 INFO:tasks.workunit.client.0.smithi042.stdout:6/587: read d9/d11/d19/d1c/d31/f56 [3207765,22162] 0 2024-02-15T18:26:59.546 INFO:tasks.workunit.client.1.smithi191.stdout:3/129: mknod d6/d1b/d1f/d21/c2f 0 2024-02-15T18:26:59.547 INFO:tasks.workunit.client.1.smithi191.stdout:3/130: getdents d6 0 2024-02-15T18:26:59.548 INFO:tasks.workunit.client.0.smithi042.stdout:6/588: read d9/d11/f2b [3992325,16925] 0 2024-02-15T18:26:59.549 INFO:tasks.workunit.client.1.smithi191.stdout:1/206: unlink d0/d8/f18 0 2024-02-15T18:26:59.550 INFO:tasks.workunit.client.1.smithi191.stdout:3/131: mkdir d6/de/d30 0 2024-02-15T18:26:59.550 INFO:tasks.workunit.client.1.smithi191.stdout:2/166: dwrite df/d12/f2b [0,4194304] 0 2024-02-15T18:26:59.551 INFO:tasks.workunit.client.0.smithi042.stdout:8/459: mkdir dd/d11/d75/d82 0 2024-02-15T18:26:59.551 INFO:tasks.workunit.client.1.smithi191.stdout:1/207: getdents d0 0 2024-02-15T18:26:59.552 INFO:tasks.workunit.client.0.smithi042.stdout:9/534: dwrite d6/dd/d9/d59/d5c/fa6 [0,4194304] 0 2024-02-15T18:26:59.555 INFO:tasks.workunit.client.0.smithi042.stdout:2/471: getdents d0/d8/d20/d41 0 2024-02-15T18:26:59.555 INFO:tasks.workunit.client.1.smithi191.stdout:2/167: mkdir df/d12/d32 0 2024-02-15T18:26:59.555 INFO:tasks.workunit.client.1.smithi191.stdout:0/95: fsync d7/da/f14 0 2024-02-15T18:26:59.557 INFO:tasks.workunit.client.0.smithi042.stdout:4/499: dwrite da/d11/d29/d4a/d4f/d80/fa1 [0,4194304] 0 2024-02-15T18:26:59.567 INFO:tasks.workunit.client.1.smithi191.stdout:1/208: unlink d0/d8/df/d12/f3b 0 2024-02-15T18:26:59.567 INFO:tasks.workunit.client.1.smithi191.stdout:4/228: sync 2024-02-15T18:26:59.573 INFO:tasks.workunit.client.1.smithi191.stdout:2/168: unlink df/c15 0 2024-02-15T18:26:59.573 INFO:tasks.workunit.client.1.smithi191.stdout:2/169: write df/f16 [1229318,32955] 0 2024-02-15T18:26:59.575 INFO:tasks.workunit.client.1.smithi191.stdout:1/209: unlink d0/f4 0 2024-02-15T18:26:59.575 INFO:tasks.workunit.client.1.smithi191.stdout:1/210: readlink d0/l1e 0 2024-02-15T18:26:59.577 INFO:tasks.workunit.client.1.smithi191.stdout:4/229: mkdir d2/d5/d7/d3e/d63 0 2024-02-15T18:26:59.581 INFO:tasks.workunit.client.1.smithi191.stdout:2/170: mknod df/d12/d1e/c33 0 2024-02-15T18:26:59.582 INFO:tasks.workunit.client.1.smithi191.stdout:1/211: rename d0/d8/d27 to d0/d8/df/d41 0 2024-02-15T18:26:59.585 INFO:tasks.workunit.client.0.smithi042.stdout:8/460: dwrite dd/de/ff [0,4194304] 0 2024-02-15T18:26:59.585 INFO:tasks.workunit.client.1.smithi191.stdout:0/96: dread d7/fb [0,4194304] 0 2024-02-15T18:26:59.585 INFO:tasks.workunit.client.1.smithi191.stdout:0/97: truncate d7/f13 1200524 0 2024-02-15T18:26:59.585 INFO:tasks.workunit.client.0.smithi042.stdout:8/461: read dd/de/f2a [418672,108838] 0 2024-02-15T18:26:59.586 INFO:tasks.workunit.client.1.smithi191.stdout:2/171: creat df/d12/d1f/f34 x:0 0 0 2024-02-15T18:26:59.586 INFO:tasks.workunit.client.0.smithi042.stdout:5/555: symlink d0/d14/d88/da1/lbf 0 2024-02-15T18:26:59.587 INFO:tasks.workunit.client.0.smithi042.stdout:8/462: write dd/de/d3c/d4b/d67/f2b [3591541,98280] 0 2024-02-15T18:26:59.587 INFO:tasks.workunit.client.0.smithi042.stdout:1/526: dwrite de/d15/d1b/f49 [0,4194304] 0 2024-02-15T18:26:59.587 INFO:tasks.workunit.client.1.smithi191.stdout:1/212: creat d0/d8/df/d12/d2a/f42 x:0 0 0 2024-02-15T18:26:59.588 INFO:tasks.workunit.client.0.smithi042.stdout:3/518: fsync d1/de/d1a/f6f 0 2024-02-15T18:26:59.588 INFO:tasks.workunit.client.0.smithi042.stdout:0/507: getdents d4 0 2024-02-15T18:26:59.589 INFO:tasks.workunit.client.1.smithi191.stdout:4/230: mkdir d2/d5/d64 0 2024-02-15T18:26:59.589 INFO:tasks.workunit.client.1.smithi191.stdout:4/231: write d2/d5/f4d [812375,26890] 0 2024-02-15T18:26:59.590 INFO:tasks.workunit.client.1.smithi191.stdout:4/232: chown d2/d5/d7/d3e 7 1 2024-02-15T18:26:59.594 INFO:tasks.workunit.client.0.smithi042.stdout:6/589: truncate d9/d11/d19/d35/d49/d79/fa8 1692208 0 2024-02-15T18:26:59.595 INFO:tasks.workunit.client.1.smithi191.stdout:4/233: symlink d2/d5/d14/l65 0 2024-02-15T18:26:59.597 INFO:tasks.workunit.client.0.smithi042.stdout:9/535: rename d6/dd/d9/d59/d5c/f8f to d6/dd/d9/d59/d5c/fb7 0 2024-02-15T18:26:59.597 INFO:tasks.workunit.client.0.smithi042.stdout:9/536: chown d6/da/d2f/d6c/l7f 1 1 2024-02-15T18:26:59.597 INFO:tasks.workunit.client.0.smithi042.stdout:9/537: chown d6/dd/d9/c4d 585352544 1 2024-02-15T18:26:59.597 INFO:tasks.workunit.client.0.smithi042.stdout:9/538: dread - d6/da/d2f/d69/f8a zero size 2024-02-15T18:26:59.597 INFO:tasks.workunit.client.0.smithi042.stdout:9/539: chown d6/da/d1d/f38 1 1 2024-02-15T18:26:59.599 INFO:tasks.workunit.client.1.smithi191.stdout:4/234: mknod d2/c66 0 2024-02-15T18:26:59.599 INFO:tasks.workunit.client.1.smithi191.stdout:4/235: fdatasync d2/f5c 0 2024-02-15T18:26:59.599 INFO:tasks.workunit.client.0.smithi042.stdout:4/500: dwrite da/fad [0,4194304] 0 2024-02-15T18:26:59.601 INFO:tasks.workunit.client.0.smithi042.stdout:9/540: write d6/dd/d9/f9f [28956,4438] 0 2024-02-15T18:26:59.601 INFO:tasks.workunit.client.0.smithi042.stdout:9/541: read - d6/da/d2f/d55/fa4 zero size 2024-02-15T18:26:59.602 INFO:tasks.workunit.client.1.smithi191.stdout:4/236: creat d2/d5/d7/f67 x:0 0 0 2024-02-15T18:26:59.603 INFO:tasks.workunit.client.1.smithi191.stdout:2/172: dread df/d12/d1f/f26 [0,4194304] 0 2024-02-15T18:26:59.603 INFO:tasks.workunit.client.0.smithi042.stdout:5/556: creat d0/d38/d53/d91/d94/fc0 x:0 0 0 2024-02-15T18:26:59.604 INFO:tasks.workunit.client.1.smithi191.stdout:4/237: creat d2/d5/d14/d44/d30/f68 x:0 0 0 2024-02-15T18:26:59.606 INFO:tasks.workunit.client.0.smithi042.stdout:1/527: creat de/d15/d95/fc2 x:0 0 0 2024-02-15T18:26:59.606 INFO:tasks.workunit.client.0.smithi042.stdout:3/519: creat d1/d11/faf x:0 0 0 2024-02-15T18:26:59.607 INFO:tasks.workunit.client.0.smithi042.stdout:8/463: dwrite dd/de/d12/f68 [0,4194304] 0 2024-02-15T18:26:59.607 INFO:tasks.workunit.client.0.smithi042.stdout:8/464: truncate dd/de/d33/d66/d4e/f6c 318804 0 2024-02-15T18:26:59.608 INFO:tasks.workunit.client.0.smithi042.stdout:0/508: unlink d4/fe 0 2024-02-15T18:26:59.608 INFO:tasks.workunit.client.1.smithi191.stdout:1/213: dwrite d0/d8/df/d12/d2a/f42 [0,4194304] 0 2024-02-15T18:26:59.609 INFO:tasks.workunit.client.0.smithi042.stdout:6/590: creat d9/d11/d19/d1c/fc6 x:0 0 0 2024-02-15T18:26:59.611 INFO:tasks.workunit.client.1.smithi191.stdout:3/132: dwrite d6/d1b/f1d [4194304,4194304] 0 2024-02-15T18:26:59.611 INFO:tasks.workunit.client.1.smithi191.stdout:4/238: symlink d2/d5/d14/d44/l69 0 2024-02-15T18:26:59.611 INFO:tasks.workunit.client.1.smithi191.stdout:4/239: chown d2/d5/l3d 29797945 1 2024-02-15T18:26:59.613 INFO:tasks.workunit.client.0.smithi042.stdout:5/557: mknod d0/d38/d53/d68/d6a/d42/d7d/cc1 0 2024-02-15T18:26:59.615 INFO:tasks.workunit.client.0.smithi042.stdout:1/528: getdents de/d15/d67/d74/d7e 0 2024-02-15T18:26:59.616 INFO:tasks.workunit.client.1.smithi191.stdout:4/240: dread d2/f28 [0,4194304] 0 2024-02-15T18:26:59.617 INFO:tasks.workunit.client.1.smithi191.stdout:3/133: unlink d6/f1a 0 2024-02-15T18:26:59.619 INFO:tasks.workunit.client.0.smithi042.stdout:3/520: mkdir d1/d11/d2e/d4c/db0 0 2024-02-15T18:26:59.619 INFO:tasks.workunit.client.0.smithi042.stdout:8/465: rmdir dd/de/d33/d66 39 2024-02-15T18:26:59.621 INFO:tasks.workunit.client.1.smithi191.stdout:3/134: truncate d6/fa 124559 0 2024-02-15T18:26:59.622 INFO:tasks.workunit.client.0.smithi042.stdout:5/558: rename d0/d39/fbc to d0/d38/d53/d68/d6a/d42/d7d/fc2 0 2024-02-15T18:26:59.622 INFO:tasks.workunit.client.0.smithi042.stdout:3/521: dread d1/d11/d14/d4d/f7e [0,4194304] 0 2024-02-15T18:26:59.623 INFO:tasks.workunit.client.1.smithi191.stdout:3/135: getdents d6/d1b 0 2024-02-15T18:26:59.623 INFO:tasks.workunit.client.1.smithi191.stdout:3/136: dread - f4 zero size 2024-02-15T18:26:59.623 INFO:tasks.workunit.client.1.smithi191.stdout:3/137: chown l0 8 1 2024-02-15T18:26:59.625 INFO:tasks.workunit.client.0.smithi042.stdout:0/509: write d4/f3b [3792555,52485] 0 2024-02-15T18:26:59.625 INFO:tasks.workunit.client.0.smithi042.stdout:1/529: symlink de/d15/d67/d46/d6a/lc3 0 2024-02-15T18:26:59.627 INFO:tasks.workunit.client.0.smithi042.stdout:8/466: symlink dd/de/d33/l83 0 2024-02-15T18:26:59.627 INFO:tasks.workunit.client.0.smithi042.stdout:8/467: write dd/de/d33/d66/f5f [349471,46369] 0 2024-02-15T18:26:59.627 INFO:tasks.workunit.client.0.smithi042.stdout:8/468: readlink dd/de/d3c/d4b/d67/l41 0 2024-02-15T18:26:59.627 INFO:tasks.workunit.client.1.smithi191.stdout:3/138: creat d6/d1b/d1c/f31 x:0 0 0 2024-02-15T18:26:59.632 INFO:tasks.workunit.client.0.smithi042.stdout:4/501: dwrite da/d11/d67/f6f [0,4194304] 0 2024-02-15T18:26:59.632 INFO:tasks.workunit.client.0.smithi042.stdout:4/502: truncate da/d11/d29/d4a/d4f/d80/fa9 668219 0 2024-02-15T18:26:59.632 INFO:tasks.workunit.client.0.smithi042.stdout:4/503: read da/d3b/d41/d8d/fb1 [1493252,110850] 0 2024-02-15T18:26:59.632 INFO:tasks.workunit.client.0.smithi042.stdout:5/559: symlink d0/d12/d65/d74/d52/dad/dba/lc3 0 2024-02-15T18:26:59.634 INFO:tasks.workunit.client.0.smithi042.stdout:3/522: rename d1/d11/d2e/c94 to d1/d11/d2e/d4c/cb1 0 2024-02-15T18:26:59.636 INFO:tasks.workunit.client.1.smithi191.stdout:3/139: rename d6/d1b/f1d to d6/d27/f32 0 2024-02-15T18:26:59.642 INFO:tasks.workunit.client.1.smithi191.stdout:0/98: dwrite d7/dd/f11 [0,4194304] 0 2024-02-15T18:26:59.642 INFO:tasks.workunit.client.1.smithi191.stdout:3/140: symlink d6/d1b/d1c/l33 0 2024-02-15T18:26:59.643 INFO:tasks.workunit.client.1.smithi191.stdout:3/141: dread d6/de/ff [0,4194304] 0 2024-02-15T18:26:59.646 INFO:tasks.workunit.client.0.smithi042.stdout:9/542: dwrite d6/da/d1d/d2d/d40/f43 [0,4194304] 0 2024-02-15T18:26:59.646 INFO:tasks.workunit.client.1.smithi191.stdout:4/241: dwrite d2/d5/d7/f67 [0,4194304] 0 2024-02-15T18:26:59.646 INFO:tasks.workunit.client.1.smithi191.stdout:4/242: chown d2/f4 9326199 1 2024-02-15T18:26:59.647 INFO:tasks.workunit.client.1.smithi191.stdout:1/214: dwrite d0/d8/df/d41/f36 [8388608,4194304] 0 2024-02-15T18:26:59.647 INFO:tasks.workunit.client.0.smithi042.stdout:8/469: unlink dd/de/d33/d66/f3f 0 2024-02-15T18:26:59.656 INFO:tasks.workunit.client.1.smithi191.stdout:0/99: rmdir d7/da 39 2024-02-15T18:26:59.659 INFO:tasks.workunit.client.1.smithi191.stdout:4/243: write d2/d5/f4d [261259,87501] 0 2024-02-15T18:26:59.664 INFO:tasks.workunit.client.1.smithi191.stdout:3/142: dwrite d6/d1b/d1c/f31 [0,4194304] 0 2024-02-15T18:26:59.667 INFO:tasks.workunit.client.0.smithi042.stdout:4/504: mkdir da/d11/d29/db8 0 2024-02-15T18:26:59.667 INFO:tasks.workunit.client.0.smithi042.stdout:4/505: chown da/d2b/d6e 496529028 1 2024-02-15T18:26:59.667 INFO:tasks.workunit.client.0.smithi042.stdout:4/506: dread - da/d3b/d41/d8d/fab zero size 2024-02-15T18:26:59.668 INFO:tasks.workunit.client.0.smithi042.stdout:5/560: unlink d0/f34 0 2024-02-15T18:26:59.668 INFO:tasks.workunit.client.0.smithi042.stdout:5/561: chown d0/d12/d65/d74/f72 23 1 2024-02-15T18:26:59.669 INFO:tasks.workunit.client.1.smithi191.stdout:1/215: symlink d0/db/l43 0 2024-02-15T18:26:59.669 INFO:tasks.workunit.client.0.smithi042.stdout:5/562: write d0/d14/fa3 [4140893,83968] 0 2024-02-15T18:26:59.670 INFO:tasks.workunit.client.0.smithi042.stdout:5/563: stat d0/db/f35 0 2024-02-15T18:26:59.670 INFO:tasks.workunit.client.0.smithi042.stdout:3/523: symlink d1/d11/d2e/d4c/db0/lb2 0 2024-02-15T18:26:59.671 INFO:tasks.workunit.client.1.smithi191.stdout:0/100: creat d7/da/f16 x:0 0 0 2024-02-15T18:26:59.672 INFO:tasks.workunit.client.0.smithi042.stdout:0/510: mkdir d4/d9/d34/dab/db0 0 2024-02-15T18:26:59.673 INFO:tasks.workunit.client.0.smithi042.stdout:1/530: rename de/d15/d67/f3f to de/d15/d67/fc4 0 2024-02-15T18:26:59.676 INFO:tasks.workunit.client.0.smithi042.stdout:9/543: link d6/da/d1d/d2d/d40/d48/c62 d6/da2/cb8 0 2024-02-15T18:26:59.676 INFO:tasks.workunit.client.0.smithi042.stdout:9/544: getdents d6/d9d 0 2024-02-15T18:26:59.677 INFO:tasks.workunit.client.0.smithi042.stdout:8/470: symlink dd/de/d33/d66/d4e/l84 0 2024-02-15T18:26:59.684 INFO:tasks.workunit.client.1.smithi191.stdout:3/143: creat d6/d1b/d1f/d21/f34 x:0 0 0 2024-02-15T18:26:59.684 INFO:tasks.workunit.client.1.smithi191.stdout:1/216: creat d0/d8/df/d12/d2a/d2b/f44 x:0 0 0 2024-02-15T18:26:59.684 INFO:tasks.workunit.client.1.smithi191.stdout:3/144: write f4 [793902,78557] 0 2024-02-15T18:26:59.685 INFO:tasks.workunit.client.1.smithi191.stdout:0/101: mkdir d7/da/d17 0 2024-02-15T18:26:59.685 INFO:tasks.workunit.client.1.smithi191.stdout:0/102: fsync d7/dd/f15 0 2024-02-15T18:26:59.691 INFO:tasks.workunit.client.0.smithi042.stdout:5/564: dwrite d0/d38/d53/d68/d6a/d42/d7d/fc2 [0,4194304] 0 2024-02-15T18:26:59.691 INFO:tasks.workunit.client.0.smithi042.stdout:5/565: write d0/fe [5008303,81639] 0 2024-02-15T18:26:59.693 INFO:tasks.workunit.client.1.smithi191.stdout:6/188: sync 2024-02-15T18:26:59.694 INFO:tasks.workunit.client.1.smithi191.stdout:9/160: sync 2024-02-15T18:26:59.696 INFO:tasks.workunit.client.1.smithi191.stdout:3/145: truncate d6/de/ff 276098 0 2024-02-15T18:26:59.697 INFO:tasks.workunit.client.1.smithi191.stdout:1/217: rmdir d0/d8/df 39 2024-02-15T18:26:59.697 INFO:tasks.workunit.client.1.smithi191.stdout:1/218: fsync d0/d8/fd 0 2024-02-15T18:26:59.699 INFO:tasks.workunit.client.0.smithi042.stdout:4/507: rename da/d11/da4/faa to da/d11/d29/db8/fb9 0 2024-02-15T18:26:59.700 INFO:tasks.workunit.client.1.smithi191.stdout:0/103: creat d7/da/f18 x:0 0 0 2024-02-15T18:26:59.702 INFO:tasks.workunit.client.1.smithi191.stdout:6/189: creat d8/f30 x:0 0 0 2024-02-15T18:26:59.703 INFO:tasks.workunit.client.1.smithi191.stdout:6/190: read d8/d9/f14 [154730,89254] 0 2024-02-15T18:26:59.704 INFO:tasks.workunit.client.1.smithi191.stdout:9/161: rename d0/d7/f1b to d0/d7/da/f3e 0 2024-02-15T18:26:59.707 INFO:tasks.workunit.client.0.smithi042.stdout:1/531: rmdir de/d15/d67/d46/d6b/d88 39 2024-02-15T18:26:59.707 INFO:tasks.workunit.client.0.smithi042.stdout:1/532: fdatasync de/d15/d2c/d51/d70/f78 0 2024-02-15T18:26:59.708 INFO:tasks.workunit.client.1.smithi191.stdout:3/146: symlink d6/d1b/d1f/d21/l35 0 2024-02-15T18:26:59.709 INFO:tasks.workunit.client.0.smithi042.stdout:9/545: creat d6/da/d2f/d55/d73/fb9 x:0 0 0 2024-02-15T18:26:59.710 INFO:tasks.workunit.client.1.smithi191.stdout:1/219: truncate d0/db/d10/f2d 1570129 0 2024-02-15T18:26:59.711 INFO:tasks.workunit.client.1.smithi191.stdout:1/220: write d0/d8/df/d12/f23 [314735,44632] 0 2024-02-15T18:26:59.712 INFO:tasks.workunit.client.1.smithi191.stdout:1/221: chown d0/d8/df/d12/d2a 10 1 2024-02-15T18:26:59.712 INFO:tasks.workunit.client.1.smithi191.stdout:1/222: chown d0/d8/df/d12/d2a/d2b/f44 1 1 2024-02-15T18:26:59.712 INFO:tasks.workunit.client.1.smithi191.stdout:0/104: symlink d7/dd/l19 0 2024-02-15T18:26:59.715 INFO:tasks.workunit.client.1.smithi191.stdout:1/223: mknod d0/d8/df/d12/d2a/c45 0 2024-02-15T18:26:59.716 INFO:tasks.workunit.client.0.smithi042.stdout:2/472: sync 2024-02-15T18:26:59.716 INFO:tasks.workunit.client.0.smithi042.stdout:7/557: sync 2024-02-15T18:26:59.717 INFO:tasks.workunit.client.0.smithi042.stdout:7/558: stat d10/d15/fa6 0 2024-02-15T18:26:59.717 INFO:tasks.workunit.client.0.smithi042.stdout:2/473: chown d0/d8/d20/d3f/d7c 495553 1 2024-02-15T18:26:59.718 INFO:tasks.workunit.client.1.smithi191.stdout:1/224: symlink d0/d8/df/d2e/l46 0 2024-02-15T18:26:59.721 INFO:tasks.workunit.client.1.smithi191.stdout:1/225: mknod d0/d8/df/c47 0 2024-02-15T18:26:59.722 INFO:tasks.workunit.client.0.smithi042.stdout:5/566: creat d0/d12/d65/d74/fc4 x:0 0 0 2024-02-15T18:26:59.722 INFO:tasks.workunit.client.0.smithi042.stdout:5/567: fsync d0/d50/d98/fae 0 2024-02-15T18:26:59.722 INFO:tasks.workunit.client.0.smithi042.stdout:5/568: write d0/d14/f1c [1459321,93425] 0 2024-02-15T18:26:59.722 INFO:tasks.workunit.client.0.smithi042.stdout:5/569: chown d0/d14/d88/f90 4895 1 2024-02-15T18:26:59.722 INFO:tasks.workunit.client.0.smithi042.stdout:4/508: mknod da/d3b/d41/d8d/cba 0 2024-02-15T18:26:59.723 INFO:tasks.workunit.client.0.smithi042.stdout:2/474: creat d0/d8/d10/d5a/f90 x:0 0 0 2024-02-15T18:26:59.723 INFO:tasks.workunit.client.0.smithi042.stdout:0/511: dwrite d4/d38/d77/d2e/f32 [0,4194304] 0 2024-02-15T18:26:59.723 INFO:tasks.workunit.client.0.smithi042.stdout:9/546: dread d6/dd/d9/d59/f63 [0,4194304] 0 2024-02-15T18:26:59.725 INFO:tasks.workunit.client.0.smithi042.stdout:5/570: creat d0/d38/d53/db4/fc5 x:0 0 0 2024-02-15T18:26:59.725 INFO:tasks.workunit.client.0.smithi042.stdout:5/571: fdatasync d0/d14/f87 0 2024-02-15T18:26:59.725 INFO:tasks.workunit.client.1.smithi191.stdout:0/105: write d7/dd/f11 [2717194,58705] 0 2024-02-15T18:26:59.728 INFO:tasks.workunit.client.0.smithi042.stdout:3/524: rename d1/de/f6e to d1/de/d1a/fb3 0 2024-02-15T18:26:59.728 INFO:tasks.workunit.client.0.smithi042.stdout:3/525: truncate d1/d11/d14/f43 390250 0 2024-02-15T18:26:59.728 INFO:tasks.workunit.client.0.smithi042.stdout:3/526: fdatasync d1/fa5 0 2024-02-15T18:26:59.728 INFO:tasks.workunit.client.0.smithi042.stdout:2/475: mknod d0/c91 0 2024-02-15T18:26:59.728 INFO:tasks.workunit.client.0.smithi042.stdout:2/476: readlink d0/d8/d20/d7d/l89 0 2024-02-15T18:26:59.729 INFO:tasks.workunit.client.1.smithi191.stdout:0/106: mkdir d7/da/d1a 0 2024-02-15T18:26:59.731 INFO:tasks.workunit.client.1.smithi191.stdout:9/162: dwrite d0/d7/dc/f2b [0,4194304] 0 2024-02-15T18:26:59.731 INFO:tasks.workunit.client.0.smithi042.stdout:9/547: creat d6/da/d1d/d2d/d40/d48/fba x:0 0 0 2024-02-15T18:26:59.731 INFO:tasks.workunit.client.0.smithi042.stdout:5/572: link d0/d38/f5a d0/d39/d8d/fc6 0 2024-02-15T18:26:59.731 INFO:tasks.workunit.client.0.smithi042.stdout:1/533: dwrite de/d15/d2c/d51/d70/fba [0,4194304] 0 2024-02-15T18:26:59.733 INFO:tasks.workunit.client.1.smithi191.stdout:0/107: mkdir d7/d1b 0 2024-02-15T18:26:59.737 INFO:tasks.workunit.client.0.smithi042.stdout:4/509: dread da/d3b/f81 [0,4194304] 0 2024-02-15T18:26:59.738 INFO:tasks.workunit.client.0.smithi042.stdout:8/471: rename dd/f2c to dd/d11/d75/f85 0 2024-02-15T18:26:59.740 INFO:tasks.workunit.client.1.smithi191.stdout:3/147: dwrite d6/de/ff [0,4194304] 0 2024-02-15T18:26:59.741 INFO:tasks.workunit.client.0.smithi042.stdout:3/527: creat d1/de/d19/fb4 x:0 0 0 2024-02-15T18:26:59.742 INFO:tasks.workunit.client.0.smithi042.stdout:5/573: write d0/db/fd [3751947,36312] 0 2024-02-15T18:26:59.743 INFO:tasks.workunit.client.0.smithi042.stdout:2/477: creat d0/d8/db/de/d1a/d2e/f92 x:0 0 0 2024-02-15T18:26:59.745 INFO:tasks.workunit.client.1.smithi191.stdout:9/163: mkdir d0/d1d/d3f 0 2024-02-15T18:26:59.747 INFO:tasks.workunit.client.0.smithi042.stdout:0/512: truncate d4/f41 970404 0 2024-02-15T18:26:59.747 INFO:tasks.workunit.client.0.smithi042.stdout:2/478: dread d0/d8/db/de/f4d [4194304,4194304] 0 2024-02-15T18:26:59.749 INFO:tasks.workunit.client.1.smithi191.stdout:1/226: dwrite d0/d8/df/f2c [0,4194304] 0 2024-02-15T18:26:59.750 INFO:tasks.workunit.client.0.smithi042.stdout:9/548: dwrite d6/dd/d9/f41 [0,4194304] 0 2024-02-15T18:26:59.751 INFO:tasks.workunit.client.1.smithi191.stdout:3/148: mknod d6/de/d30/c36 0 2024-02-15T18:26:59.752 INFO:tasks.workunit.client.1.smithi191.stdout:3/149: write d6/fa [982497,27872] 0 2024-02-15T18:26:59.758 INFO:tasks.workunit.client.1.smithi191.stdout:1/227: mkdir d0/d8/d48 0 2024-02-15T18:26:59.758 INFO:tasks.workunit.client.1.smithi191.stdout:1/228: chown d0/d8/df/d41/l31 1036597 1 2024-02-15T18:26:59.762 INFO:tasks.workunit.client.1.smithi191.stdout:3/150: truncate d6/de/f2c 582377 0 2024-02-15T18:26:59.762 INFO:tasks.workunit.client.0.smithi042.stdout:1/534: mknod de/d15/d67/d89/cc5 0 2024-02-15T18:26:59.762 INFO:tasks.workunit.client.0.smithi042.stdout:7/559: rename d10/d15/cbd to d10/d43/d57/cbe 0 2024-02-15T18:26:59.762 INFO:tasks.workunit.client.0.smithi042.stdout:7/560: chown d10/d49/d6d/lae 30 1 2024-02-15T18:26:59.763 INFO:tasks.workunit.client.1.smithi191.stdout:9/164: dread d0/d7/dc/f2b [0,4194304] 0 2024-02-15T18:26:59.763 INFO:tasks.workunit.client.0.smithi042.stdout:3/528: truncate d1/d11/d14/f39 2650907 0 2024-02-15T18:26:59.766 INFO:tasks.workunit.client.1.smithi191.stdout:1/229: mkdir d0/d8/df/d12/d33/d49 0 2024-02-15T18:26:59.767 INFO:tasks.workunit.client.0.smithi042.stdout:8/472: dwrite dd/d11/f15 [4194304,4194304] 0 2024-02-15T18:26:59.772 INFO:tasks.workunit.client.0.smithi042.stdout:8/473: dread dd/de/d3c/d4b/d67/f2d [0,4194304] 0 2024-02-15T18:26:59.772 INFO:tasks.workunit.client.0.smithi042.stdout:8/474: chown dd/de/d33/d66/d4e/c4a 0 1 2024-02-15T18:26:59.772 INFO:tasks.workunit.client.0.smithi042.stdout:8/475: fsync dd/de/d3c/d4b/d67/d5a/f64 0 2024-02-15T18:26:59.777 INFO:tasks.workunit.client.0.smithi042.stdout:9/549: unlink d6/dd/d9/d59/d5c/l89 0 2024-02-15T18:26:59.777 INFO:tasks.workunit.client.0.smithi042.stdout:9/550: fsync d6/da/f65 0 2024-02-15T18:26:59.778 INFO:tasks.workunit.client.1.smithi191.stdout:3/151: mknod d6/d1b/d1f/c37 0 2024-02-15T18:26:59.778 INFO:tasks.workunit.client.1.smithi191.stdout:3/152: stat d6/d1b/d1c/f2a 0 2024-02-15T18:26:59.779 INFO:tasks.workunit.client.1.smithi191.stdout:1/230: rmdir d0/d2f 39 2024-02-15T18:26:59.779 INFO:tasks.workunit.client.1.smithi191.stdout:1/231: stat d0/d8/df/d41 0 2024-02-15T18:26:59.782 INFO:tasks.workunit.client.0.smithi042.stdout:5/574: rename d0/d12/d65/d74 to d0/d38/d53/d68/d6a/dc7 0 2024-02-15T18:26:59.783 INFO:tasks.workunit.client.0.smithi042.stdout:1/535: mknod de/d15/d67/d74/d7e/cc6 0 2024-02-15T18:26:59.784 INFO:tasks.workunit.client.0.smithi042.stdout:3/529: creat d1/d11/d14/d4d/d6c/d7b/fb5 x:0 0 0 2024-02-15T18:26:59.785 INFO:tasks.workunit.client.0.smithi042.stdout:8/476: mknod dd/de/d12/c86 0 2024-02-15T18:26:59.785 INFO:tasks.workunit.client.0.smithi042.stdout:9/551: creat d6/da/d1d/d2d/d40/d5f/fbb x:0 0 0 2024-02-15T18:26:59.786 INFO:tasks.workunit.client.0.smithi042.stdout:5/575: mknod d0/d12/d65/cc8 0 2024-02-15T18:26:59.786 INFO:tasks.workunit.client.0.smithi042.stdout:5/576: fsync d0/d14/f9c 0 2024-02-15T18:26:59.788 INFO:tasks.workunit.client.0.smithi042.stdout:1/536: unlink de/d15/d2c/d51/fb7 0 2024-02-15T18:26:59.791 INFO:tasks.workunit.client.0.smithi042.stdout:5/577: creat d0/d12/d65/daf/fc9 x:0 0 0 2024-02-15T18:26:59.791 INFO:tasks.workunit.client.0.smithi042.stdout:8/477: creat dd/de/d33/d66/d4e/f87 x:0 0 0 2024-02-15T18:26:59.791 INFO:tasks.workunit.client.0.smithi042.stdout:8/478: readlink dd/d11/l6b 0 2024-02-15T18:26:59.793 INFO:tasks.workunit.client.0.smithi042.stdout:5/578: creat d0/d38/d53/db4/fca x:0 0 0 2024-02-15T18:26:59.795 INFO:tasks.workunit.client.0.smithi042.stdout:8/479: creat dd/de/d3c/d4b/d81/f88 x:0 0 0 2024-02-15T18:26:59.796 INFO:tasks.workunit.client.0.smithi042.stdout:9/552: dread d6/da/d2f/f9e [0,4194304] 0 2024-02-15T18:26:59.796 INFO:tasks.workunit.client.0.smithi042.stdout:9/553: fdatasync d6/dd/d9/d59/d5c/f7e 0 2024-02-15T18:26:59.797 INFO:tasks.workunit.client.0.smithi042.stdout:5/579: link d0/d12/l3f d0/d38/d53/d68/d6a/lcb 0 2024-02-15T18:26:59.799 INFO:tasks.workunit.client.0.smithi042.stdout:9/554: creat d6/da/fbc x:0 0 0 2024-02-15T18:26:59.799 INFO:tasks.workunit.client.0.smithi042.stdout:9/555: write d6/da/d2f/d69/f5b [382007,114418] 0 2024-02-15T18:26:59.802 INFO:tasks.workunit.client.0.smithi042.stdout:0/513: dwrite d4/d9/d34/f4b [0,4194304] 0 2024-02-15T18:26:59.802 INFO:tasks.workunit.client.0.smithi042.stdout:4/510: dwrite da/d11/d67/d5c/d66/d78/d2f/d4d/f96 [0,4194304] 0 2024-02-15T18:26:59.802 INFO:tasks.workunit.client.1.smithi191.stdout:9/165: dwrite d0/d29/f3d [0,4194304] 0 2024-02-15T18:26:59.803 INFO:tasks.workunit.client.1.smithi191.stdout:9/166: write d0/d7/dc/f15 [787435,6983] 0 2024-02-15T18:26:59.803 INFO:tasks.workunit.client.0.smithi042.stdout:7/561: dwrite d10/d15/d39/f5a [0,4194304] 0 2024-02-15T18:26:59.804 INFO:tasks.workunit.client.0.smithi042.stdout:2/479: dwrite d0/d8/db/de/d1a/f6f [0,4194304] 0 2024-02-15T18:26:59.805 INFO:tasks.workunit.client.0.smithi042.stdout:6/591: sync 2024-02-15T18:26:59.809 INFO:tasks.workunit.client.1.smithi191.stdout:1/232: dwrite d0/db/d10/f25 [0,4194304] 0 2024-02-15T18:26:59.809 INFO:tasks.workunit.client.1.smithi191.stdout:1/233: write d0/db/f1d [1677624,14667] 0 2024-02-15T18:26:59.810 INFO:tasks.workunit.client.1.smithi191.stdout:1/234: chown d0/d8/df/d12/f28 37 1 2024-02-15T18:26:59.810 INFO:tasks.workunit.client.1.smithi191.stdout:1/235: chown d0/d8/df/d2e/l46 178 1 2024-02-15T18:26:59.810 INFO:tasks.workunit.client.1.smithi191.stdout:1/236: write d0/d8/df/d12/f28 [86644,51114] 0 2024-02-15T18:26:59.818 INFO:tasks.workunit.client.1.smithi191.stdout:9/167: rmdir d0/d7/dc/d30 39 2024-02-15T18:26:59.819 INFO:tasks.workunit.client.0.smithi042.stdout:5/580: dread d0/d38/d53/d91/fb6 [0,4194304] 0 2024-02-15T18:26:59.819 INFO:tasks.workunit.client.0.smithi042.stdout:5/581: write d0/d14/d88/da1/fb1 [902209,80299] 0 2024-02-15T18:26:59.823 INFO:tasks.workunit.client.0.smithi042.stdout:4/511: mkdir da/d3a/dbb 0 2024-02-15T18:26:59.828 INFO:tasks.workunit.client.0.smithi042.stdout:5/582: creat d0/d38/d53/d68/d6a/dc7/fcc x:0 0 0 2024-02-15T18:26:59.828 INFO:tasks.workunit.client.0.smithi042.stdout:6/592: creat d9/d11/d19/d1c/d31/fc7 x:0 0 0 2024-02-15T18:26:59.829 INFO:tasks.workunit.client.0.smithi042.stdout:5/583: chown d0/d38/d53/d68/d6a/dc7/d52/dad/dba/dbd 86413 1 2024-02-15T18:26:59.829 INFO:tasks.workunit.client.0.smithi042.stdout:5/584: fdatasync d0/d38/d53/d68/d6a/dc7/f4c 0 2024-02-15T18:26:59.829 INFO:tasks.workunit.client.0.smithi042.stdout:4/512: symlink da/lbc 0 2024-02-15T18:26:59.829 INFO:tasks.workunit.client.0.smithi042.stdout:2/480: dread d0/f71 [0,4194304] 0 2024-02-15T18:26:59.830 INFO:tasks.workunit.client.0.smithi042.stdout:5/585: creat d0/d38/d53/d68/d6a/dc7/d52/fcd x:0 0 0 2024-02-15T18:26:59.831 INFO:tasks.workunit.client.0.smithi042.stdout:4/513: write da/d11/d29/d4a/d4f/f6c [4087631,58578] 0 2024-02-15T18:26:59.831 INFO:tasks.workunit.client.0.smithi042.stdout:5/586: creat d0/fce x:0 0 0 2024-02-15T18:26:59.832 INFO:tasks.workunit.client.0.smithi042.stdout:5/587: write d0/fce [1005378,3868] 0 2024-02-15T18:26:59.838 INFO:tasks.workunit.client.0.smithi042.stdout:6/593: dread d9/f40 [0,4194304] 0 2024-02-15T18:26:59.838 INFO:tasks.workunit.client.0.smithi042.stdout:6/594: chown d9/d11/d46/d89/fab 106248 1 2024-02-15T18:26:59.838 INFO:tasks.workunit.client.0.smithi042.stdout:6/595: fdatasync d9/d11/d19/d35/f51 0 2024-02-15T18:26:59.838 INFO:tasks.workunit.client.0.smithi042.stdout:6/596: fsync d9/d11/d19/d1c/d31/f5b 0 2024-02-15T18:26:59.840 INFO:tasks.workunit.client.0.smithi042.stdout:6/597: creat d9/d11/d19/d1c/d31/d4f/fc8 x:0 0 0 2024-02-15T18:26:59.840 INFO:tasks.workunit.client.0.smithi042.stdout:6/598: chown d9/d11/d19/f2a 11834385 1 2024-02-15T18:26:59.842 INFO:tasks.workunit.client.0.smithi042.stdout:6/599: symlink d9/d11/d46/d89/lc9 0 2024-02-15T18:26:59.842 INFO:tasks.workunit.client.0.smithi042.stdout:6/600: chown d9/f2f 103 1 2024-02-15T18:26:59.846 INFO:tasks.workunit.client.0.smithi042.stdout:6/601: symlink d9/d11/d19/d35/d55/d92/d9e/lca 0 2024-02-15T18:26:59.847 INFO:tasks.workunit.client.0.smithi042.stdout:6/602: truncate d9/f8e 530803 0 2024-02-15T18:26:59.847 INFO:tasks.workunit.client.0.smithi042.stdout:6/603: dread - d9/d11/d46/f91 zero size 2024-02-15T18:26:59.848 INFO:tasks.workunit.client.0.smithi042.stdout:6/604: mknod d9/d11/d19/d35/d55/d92/ccb 0 2024-02-15T18:26:59.848 INFO:tasks.workunit.client.0.smithi042.stdout:0/514: dwrite d4/d9/d15/f27 [4194304,4194304] 0 2024-02-15T18:26:59.848 INFO:tasks.workunit.client.0.smithi042.stdout:6/605: write d9/d11/d19/d1c/d31/d4f/f98 [2090635,101919] 0 2024-02-15T18:26:59.849 INFO:tasks.workunit.client.1.smithi191.stdout:3/153: dwrite d6/d27/f32 [4194304,4194304] 0 2024-02-15T18:26:59.849 INFO:tasks.workunit.client.0.smithi042.stdout:6/606: write d9/d11/d19/d35/d55/fac [247344,120760] 0 2024-02-15T18:26:59.849 INFO:tasks.workunit.client.0.smithi042.stdout:6/607: stat d9/d3e/f64 0 2024-02-15T18:26:59.849 INFO:tasks.workunit.client.0.smithi042.stdout:7/562: dwrite d10/f7b [0,4194304] 0 2024-02-15T18:26:59.851 INFO:tasks.workunit.client.0.smithi042.stdout:2/481: dread d0/d8/d10/d31/d3a/d53/f66 [0,4194304] 0 2024-02-15T18:26:59.851 INFO:tasks.workunit.client.0.smithi042.stdout:3/530: dwrite d1/de/d1a/f5a [0,4194304] 0 2024-02-15T18:26:59.851 INFO:tasks.workunit.client.1.smithi191.stdout:3/154: link d6/d1b/d1f/d21/c28 d6/de/d30/c38 0 2024-02-15T18:26:59.851 INFO:tasks.workunit.client.1.smithi191.stdout:3/155: dread - d6/d1b/d1f/d21/f34 zero size 2024-02-15T18:26:59.853 INFO:tasks.workunit.client.0.smithi042.stdout:1/537: dwrite de/d15/d67/d46/f59 [0,4194304] 0 2024-02-15T18:26:59.853 INFO:tasks.workunit.client.0.smithi042.stdout:1/538: chown de/d15/d67/d66/d7c/fb1 12 1 2024-02-15T18:26:59.858 INFO:tasks.workunit.client.0.smithi042.stdout:6/608: mknod d9/d11/d19/d1c/d31/ccc 0 2024-02-15T18:26:59.858 INFO:tasks.workunit.client.0.smithi042.stdout:6/609: truncate d9/d3e/fa3 683153 0 2024-02-15T18:26:59.860 INFO:tasks.workunit.client.0.smithi042.stdout:7/563: creat d10/fbf x:0 0 0 2024-02-15T18:26:59.860 INFO:tasks.workunit.client.0.smithi042.stdout:7/564: chown d10/f73 187 1 2024-02-15T18:26:59.860 INFO:tasks.workunit.client.0.smithi042.stdout:7/565: write d10/f21 [694661,45732] 0 2024-02-15T18:26:59.861 INFO:tasks.workunit.client.0.smithi042.stdout:0/515: symlink d4/lb1 0 2024-02-15T18:26:59.863 INFO:tasks.workunit.client.0.smithi042.stdout:6/610: write d9/d3e/f3b [3229439,70482] 0 2024-02-15T18:26:59.863 INFO:tasks.workunit.client.0.smithi042.stdout:6/611: dread - d9/d11/d19/d1c/d31/d53/fc2 zero size 2024-02-15T18:26:59.863 INFO:tasks.workunit.client.0.smithi042.stdout:6/612: chown d9/d11/d19/d1c/fad 14323569 1 2024-02-15T18:26:59.863 INFO:tasks.workunit.client.0.smithi042.stdout:6/613: stat d9/d11/d19/d35/dc4 0 2024-02-15T18:26:59.864 INFO:tasks.workunit.client.1.smithi191.stdout:1/237: dwrite d0/db/d10/f2d [0,4194304] 0 2024-02-15T18:26:59.864 INFO:tasks.workunit.client.0.smithi042.stdout:8/480: dwrite dd/de/d12/f45 [0,4194304] 0 2024-02-15T18:26:59.865 INFO:tasks.workunit.client.0.smithi042.stdout:3/531: rmdir d1/d11/d14/d4d/d73/d89/d90 0 2024-02-15T18:26:59.865 INFO:tasks.workunit.client.0.smithi042.stdout:3/532: chown d1/ca6 23725 1 2024-02-15T18:26:59.865 INFO:tasks.workunit.client.0.smithi042.stdout:8/481: dread dd/d11/f74 [0,4194304] 0 2024-02-15T18:26:59.865 INFO:tasks.workunit.client.0.smithi042.stdout:8/482: readlink dd/de/d33/d66/d4e/l84 0 2024-02-15T18:26:59.865 INFO:tasks.workunit.client.0.smithi042.stdout:8/483: readlink dd/de/l5e 0 2024-02-15T18:26:59.869 INFO:tasks.workunit.client.0.smithi042.stdout:4/514: dwrite da/d11/d67/d5c/d66/d78/d2f/f34 [4194304,4194304] 0 2024-02-15T18:26:59.869 INFO:tasks.workunit.client.0.smithi042.stdout:4/515: readlink da/d3b/d41/l46 0 2024-02-15T18:26:59.869 INFO:tasks.workunit.client.0.smithi042.stdout:4/516: write da/d11/d27/f69 [41651,88043] 0 2024-02-15T18:26:59.870 INFO:tasks.workunit.client.0.smithi042.stdout:2/482: dwrite d0/d8/d10/d31/d79/f8e [0,4194304] 0 2024-02-15T18:26:59.871 INFO:tasks.workunit.client.0.smithi042.stdout:2/483: fsync d0/d8/db/de/d1a/f2b 0 2024-02-15T18:26:59.871 INFO:tasks.workunit.client.1.smithi191.stdout:1/238: link d0/d8/df/d12/d2a/f42 d0/d8/df/d12/d33/d49/f4a 0 2024-02-15T18:26:59.873 INFO:tasks.workunit.client.0.smithi042.stdout:5/588: dwrite d0/db/f1f [0,4194304] 0 2024-02-15T18:26:59.874 INFO:tasks.workunit.client.0.smithi042.stdout:4/517: dread da/f7c [0,4194304] 0 2024-02-15T18:26:59.880 INFO:tasks.workunit.client.0.smithi042.stdout:4/518: write da/d2b/f37 [1417773,19533] 0 2024-02-15T18:26:59.886 INFO:tasks.workunit.client.0.smithi042.stdout:0/516: dwrite d4/d9/d34/d4e/d6c/f71 [0,4194304] 0 2024-02-15T18:26:59.886 INFO:tasks.workunit.client.1.smithi191.stdout:1/239: mknod d0/d8/df/d12/d33/c4b 0 2024-02-15T18:26:59.886 INFO:tasks.workunit.client.1.smithi191.stdout:1/240: chown d0/f9 968896429 1 2024-02-15T18:26:59.886 INFO:tasks.workunit.client.1.smithi191.stdout:1/241: chown d0/db/l43 7 1 2024-02-15T18:26:59.886 INFO:tasks.workunit.client.1.smithi191.stdout:9/168: dwrite d0/d7/dc/f15 [0,4194304] 0 2024-02-15T18:26:59.889 INFO:tasks.workunit.client.1.smithi191.stdout:1/242: mknod d0/db/c4c 0 2024-02-15T18:26:59.890 INFO:tasks.workunit.client.1.smithi191.stdout:9/169: rename d0/d1d/f2c to d0/d7/dc/d17/d33/f40 0 2024-02-15T18:26:59.893 INFO:tasks.workunit.client.1.smithi191.stdout:1/243: unlink d0/d8/df/d12/d2a/d2b/f44 0 2024-02-15T18:26:59.894 INFO:tasks.workunit.client.0.smithi042.stdout:7/566: symlink d10/d49/lc0 0 2024-02-15T18:26:59.894 INFO:tasks.workunit.client.0.smithi042.stdout:7/567: fsync d10/d15/f34 0 2024-02-15T18:26:59.894 INFO:tasks.workunit.client.0.smithi042.stdout:7/568: chown d10/d49/lc0 20 1 2024-02-15T18:26:59.895 INFO:tasks.workunit.client.0.smithi042.stdout:7/569: write d10/d12/f9e [2376883,103639] 0 2024-02-15T18:26:59.895 INFO:tasks.workunit.client.0.smithi042.stdout:7/570: fsync d10/d12/d23/fa1 0 2024-02-15T18:26:59.901 INFO:tasks.workunit.client.1.smithi191.stdout:9/170: truncate d0/f18 794050 0 2024-02-15T18:26:59.902 INFO:tasks.workunit.client.0.smithi042.stdout:6/614: mknod d9/d8d/ccd 0 2024-02-15T18:26:59.905 INFO:tasks.workunit.client.0.smithi042.stdout:3/533: mknod d1/d11/d14/d4d/d73/cb6 0 2024-02-15T18:26:59.906 INFO:tasks.workunit.client.0.smithi042.stdout:8/484: symlink dd/de/d33/d66/l89 0 2024-02-15T18:26:59.908 INFO:tasks.workunit.client.0.smithi042.stdout:2/484: creat d0/d8/db/de/f93 x:0 0 0 2024-02-15T18:26:59.908 INFO:tasks.workunit.client.0.smithi042.stdout:2/485: dread - d0/d8/d10/d31/d3a/d53/d3c/f87 zero size 2024-02-15T18:26:59.909 INFO:tasks.workunit.client.1.smithi191.stdout:1/244: dwrite d0/d8/df/d12/d33/d49/f4a [0,4194304] 0 2024-02-15T18:26:59.910 INFO:tasks.workunit.client.0.smithi042.stdout:5/589: mknod d0/d38/d53/d68/d6a/d42/ccf 0 2024-02-15T18:26:59.911 INFO:tasks.workunit.client.0.smithi042.stdout:4/519: truncate da/fad 3793629 0 2024-02-15T18:26:59.912 INFO:tasks.workunit.client.0.smithi042.stdout:6/615: dread d9/d11/d19/d35/f80 [0,4194304] 0 2024-02-15T18:26:59.915 INFO:tasks.workunit.client.0.smithi042.stdout:2/486: write d0/d8/d10/d31/d79/f8e [4990455,2554] 0 2024-02-15T18:26:59.916 INFO:tasks.workunit.client.1.smithi191.stdout:1/245: link d0/d8/df/d12/d2a/f42 d0/d8/d48/f4d 0 2024-02-15T18:26:59.917 INFO:tasks.workunit.client.0.smithi042.stdout:2/487: dread d0/d8/db/de/f14 [0,4194304] 0 2024-02-15T18:26:59.918 INFO:tasks.workunit.client.1.smithi191.stdout:1/246: read d0/d8/df/f2c [3424279,66254] 0 2024-02-15T18:26:59.925 INFO:tasks.workunit.client.0.smithi042.stdout:7/571: truncate d10/d49/d6d/d78/f91 204025 0 2024-02-15T18:26:59.926 INFO:tasks.workunit.client.0.smithi042.stdout:3/534: mknod d1/d11/d14/d4d/d73/d89/cb7 0 2024-02-15T18:26:59.927 INFO:tasks.workunit.client.0.smithi042.stdout:5/590: rmdir d0/d12/d2a 39 2024-02-15T18:26:59.929 INFO:tasks.workunit.client.0.smithi042.stdout:4/520: creat da/d11/d67/d5c/d66/d78/fbd x:0 0 0 2024-02-15T18:26:59.929 INFO:tasks.workunit.client.0.smithi042.stdout:4/521: dread - da/d11/d27/f89 zero size 2024-02-15T18:26:59.929 INFO:tasks.workunit.client.0.smithi042.stdout:4/522: truncate da/d11/da4/fb4 726437 0 2024-02-15T18:26:59.931 INFO:tasks.workunit.client.0.smithi042.stdout:9/556: sync 2024-02-15T18:26:59.931 INFO:tasks.workunit.client.1.smithi191.stdout:9/171: dwrite d0/d7/f19 [0,4194304] 0 2024-02-15T18:26:59.934 INFO:tasks.workunit.client.0.smithi042.stdout:1/539: rename de/d15/d1b/d44/f84 to de/d15/d1b/fc7 0 2024-02-15T18:26:59.934 INFO:tasks.workunit.client.0.smithi042.stdout:1/540: chown l4 292 1 2024-02-15T18:26:59.934 INFO:tasks.workunit.client.0.smithi042.stdout:7/572: truncate d10/f63 2388123 0 2024-02-15T18:26:59.938 INFO:tasks.workunit.client.0.smithi042.stdout:3/535: truncate d1/f9 44402 0 2024-02-15T18:26:59.938 INFO:tasks.workunit.client.0.smithi042.stdout:5/591: truncate d0/d14/f9c 1359907 0 2024-02-15T18:26:59.939 INFO:tasks.workunit.client.0.smithi042.stdout:9/557: dread d6/dd/f4a [0,4194304] 0 2024-02-15T18:26:59.940 INFO:tasks.workunit.client.1.smithi191.stdout:9/172: unlink d0/d29/f36 0 2024-02-15T18:26:59.941 INFO:tasks.workunit.client.0.smithi042.stdout:4/523: unlink da/d3b/c5a 0 2024-02-15T18:26:59.943 INFO:tasks.workunit.client.0.smithi042.stdout:6/616: rename d9/d11/lba to d9/d11/d19/d1c/lce 0 2024-02-15T18:26:59.943 INFO:tasks.workunit.client.1.smithi191.stdout:1/247: dwrite d0/d8/df/f2c [0,4194304] 0 2024-02-15T18:26:59.944 INFO:tasks.workunit.client.0.smithi042.stdout:2/488: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:26:59.945 INFO:tasks.workunit.client.0.smithi042.stdout:1/541: rmdir de/d15/d2c/d51/d70 39 2024-02-15T18:26:59.955 INFO:tasks.workunit.client.0.smithi042.stdout:7/573: creat d10/d43/d57/fc1 x:0 0 0 2024-02-15T18:26:59.955 INFO:tasks.workunit.client.0.smithi042.stdout:7/574: chown d10/d15/d8f/da5/cac 6 1 2024-02-15T18:26:59.964 INFO:tasks.workunit.client.0.smithi042.stdout:9/558: unlink d6/da/d2f/d55/f47 0 2024-02-15T18:26:59.964 INFO:tasks.workunit.client.0.smithi042.stdout:9/559: stat d6/f17 0 2024-02-15T18:26:59.964 INFO:tasks.workunit.client.0.smithi042.stdout:1/542: mkdir de/d54/d9a/dc8 0 2024-02-15T18:26:59.966 INFO:tasks.workunit.client.0.smithi042.stdout:7/575: rmdir d10/d15/d8f 39 2024-02-15T18:26:59.968 INFO:tasks.workunit.client.0.smithi042.stdout:5/592: mknod d0/d38/d53/d91/db0/cd0 0 2024-02-15T18:26:59.970 INFO:tasks.workunit.client.0.smithi042.stdout:9/560: symlink d6/da/d2f/d54/lbd 0 2024-02-15T18:26:59.971 INFO:tasks.workunit.client.0.smithi042.stdout:1/543: rmdir de/d15/d67/d74 39 2024-02-15T18:26:59.972 INFO:tasks.workunit.client.0.smithi042.stdout:7/576: dread d10/d15/f3d [0,4194304] 0 2024-02-15T18:26:59.973 INFO:tasks.workunit.client.0.smithi042.stdout:5/593: link d0/d38/d53/l6b d0/d38/d53/d68/d6a/d42/d7d/ld1 0 2024-02-15T18:26:59.974 INFO:tasks.workunit.client.0.smithi042.stdout:9/561: creat d6/dd/d9/d59/fbe x:0 0 0 2024-02-15T18:26:59.975 INFO:tasks.workunit.client.0.smithi042.stdout:1/544: write de/d15/d67/d74/fab [1426613,90818] 0 2024-02-15T18:26:59.975 INFO:tasks.workunit.client.0.smithi042.stdout:1/545: chown de/d54/d9a/dc8 23656933 1 2024-02-15T18:26:59.975 INFO:tasks.workunit.client.0.smithi042.stdout:1/546: stat f6 0 2024-02-15T18:26:59.976 INFO:tasks.workunit.client.0.smithi042.stdout:9/562: rmdir d6/da/d1d/d2d/d40/d95 39 2024-02-15T18:26:59.976 INFO:tasks.workunit.client.0.smithi042.stdout:9/563: stat d6/da/d2f/d55/d73/d8d/fb2 0 2024-02-15T18:26:59.976 INFO:tasks.workunit.client.0.smithi042.stdout:9/564: fsync d6/dd/d9/f41 0 2024-02-15T18:26:59.976 INFO:tasks.workunit.client.0.smithi042.stdout:7/577: write d10/d43/f50 [471881,52393] 0 2024-02-15T18:26:59.976 INFO:tasks.workunit.client.0.smithi042.stdout:1/547: getdents de/d6d 0 2024-02-15T18:26:59.978 INFO:tasks.workunit.client.0.smithi042.stdout:9/565: mkdir d6/dd/d9/d59/dbf 0 2024-02-15T18:26:59.978 INFO:tasks.workunit.client.0.smithi042.stdout:9/566: fdatasync d6/dd/d9/f9f 0 2024-02-15T18:26:59.979 INFO:tasks.workunit.client.0.smithi042.stdout:7/578: mkdir d10/d15/d39/dc2 0 2024-02-15T18:26:59.979 INFO:tasks.workunit.client.0.smithi042.stdout:7/579: dread d10/d43/d57/fb0 [0,4194304] 0 2024-02-15T18:26:59.980 INFO:tasks.workunit.client.0.smithi042.stdout:1/548: rename de/d15/d67/d46/d6b/c8a to de/d15/d67/d89/cc9 0 2024-02-15T18:26:59.981 INFO:tasks.workunit.client.0.smithi042.stdout:1/549: mkdir de/d15/d67/d46/d6b/dca 0 2024-02-15T18:26:59.981 INFO:tasks.workunit.client.0.smithi042.stdout:1/550: fsync de/d15/f20 0 2024-02-15T18:26:59.981 INFO:tasks.workunit.client.0.smithi042.stdout:1/551: truncate de/d15/d67/d63/f8b 436532 0 2024-02-15T18:26:59.982 INFO:tasks.workunit.client.0.smithi042.stdout:1/552: mknod de/d6d/d9e/ccb 0 2024-02-15T18:26:59.983 INFO:tasks.workunit.client.0.smithi042.stdout:1/553: creat de/d15/d2c/d51/fcc x:0 0 0 2024-02-15T18:26:59.984 INFO:tasks.workunit.client.0.smithi042.stdout:1/554: mkdir de/d6d/d9e/dcd 0 2024-02-15T18:26:59.985 INFO:tasks.workunit.client.0.smithi042.stdout:1/555: mkdir de/d15/d67/d46/dce 0 2024-02-15T18:26:59.985 INFO:tasks.workunit.client.0.smithi042.stdout:9/567: dread d6/da/d2f/d69/f8b [0,4194304] 0 2024-02-15T18:26:59.986 INFO:tasks.workunit.client.0.smithi042.stdout:7/580: dread d10/d49/f67 [0,4194304] 0 2024-02-15T18:26:59.987 INFO:tasks.workunit.client.0.smithi042.stdout:7/581: symlink d10/d43/d57/lc3 0 2024-02-15T18:26:59.990 INFO:tasks.workunit.client.0.smithi042.stdout:9/568: dread d6/dd/d9/f9f [0,4194304] 0 2024-02-15T18:26:59.990 INFO:tasks.workunit.client.0.smithi042.stdout:9/569: chown d6/f31 897 1 2024-02-15T18:26:59.990 INFO:tasks.workunit.client.0.smithi042.stdout:7/582: mkdir d10/d49/dc4 0 2024-02-15T18:26:59.991 INFO:tasks.workunit.client.0.smithi042.stdout:7/583: read - d10/d49/f8e zero size 2024-02-15T18:26:59.993 INFO:tasks.workunit.client.0.smithi042.stdout:3/536: dwrite d1/de/d1a/f77 [0,4194304] 0 2024-02-15T18:26:59.993 INFO:tasks.workunit.client.0.smithi042.stdout:3/537: chown d1/de/l3f 2113198532 1 2024-02-15T18:26:59.993 INFO:tasks.workunit.client.0.smithi042.stdout:3/538: chown d1/d11/d14 3477 1 2024-02-15T18:26:59.993 INFO:tasks.workunit.client.0.smithi042.stdout:3/539: chown d1/c96 825 1 2024-02-15T18:26:59.994 INFO:tasks.workunit.client.0.smithi042.stdout:4/524: dwrite da/d11/d27/f89 [0,4194304] 0 2024-02-15T18:26:59.994 INFO:tasks.workunit.client.0.smithi042.stdout:6/617: dwrite d9/d11/d19/d1c/d31/d9b/f9c [0,4194304] 0 2024-02-15T18:26:59.994 INFO:tasks.workunit.client.0.smithi042.stdout:2/489: dwrite d0/d8/f5f [0,4194304] 0 2024-02-15T18:26:59.995 INFO:tasks.workunit.client.1.smithi191.stdout:9/173: dwrite d0/d7/dc/f15 [0,4194304] 0 2024-02-15T18:26:59.995 INFO:tasks.workunit.client.0.smithi042.stdout:5/594: dwrite d0/d14/d88/da1/fb1 [0,4194304] 0 2024-02-15T18:26:59.995 INFO:tasks.workunit.client.0.smithi042.stdout:5/595: dread - d0/d38/d53/db4/fc5 zero size 2024-02-15T18:26:59.995 INFO:tasks.workunit.client.0.smithi042.stdout:5/596: chown d0/d12/d65/l8b 1 1 2024-02-15T18:27:00.003 INFO:tasks.workunit.client.0.smithi042.stdout:1/556: dwrite de/d15/d2c/d51/f55 [4194304,4194304] 0 2024-02-15T18:27:00.012 INFO:tasks.workunit.client.0.smithi042.stdout:3/540: rename d1/d11/d14/d4d/d6c/d7b/l85 to d1/de/d19/lb8 0 2024-02-15T18:27:00.014 INFO:tasks.workunit.client.1.smithi191.stdout:9/174: dread d0/d7/f19 [0,4194304] 0 2024-02-15T18:27:00.015 INFO:tasks.workunit.client.0.smithi042.stdout:4/525: creat da/d11/d29/d4a/fbe x:0 0 0 2024-02-15T18:27:00.015 INFO:tasks.workunit.client.0.smithi042.stdout:4/526: fsync da/fa7 0 2024-02-15T18:27:00.015 INFO:tasks.workunit.client.0.smithi042.stdout:4/527: read - da/d11/f3c zero size 2024-02-15T18:27:00.017 INFO:tasks.workunit.client.0.smithi042.stdout:6/618: mkdir d9/d11/d19/d1c/dcf 0 2024-02-15T18:27:00.017 INFO:tasks.workunit.client.0.smithi042.stdout:6/619: chown d9/d3e/f75 34789945 1 2024-02-15T18:27:00.023 INFO:tasks.workunit.client.0.smithi042.stdout:1/557: creat de/d15/d2c/d51/fcf x:0 0 0 2024-02-15T18:27:00.024 INFO:tasks.workunit.client.0.smithi042.stdout:3/541: symlink d1/d11/d2e/d4c/db0/lb9 0 2024-02-15T18:27:00.024 INFO:tasks.workunit.client.0.smithi042.stdout:3/542: chown d1/d11 19 1 2024-02-15T18:27:00.025 INFO:tasks.workunit.client.0.smithi042.stdout:3/543: write d1/de/f42 [3541931,9355] 0 2024-02-15T18:27:00.025 INFO:tasks.workunit.client.0.smithi042.stdout:3/544: chown d1/d11/d14/d4d/d73/f8b 826 1 2024-02-15T18:27:00.026 INFO:tasks.workunit.client.1.smithi191.stdout:9/175: dread d0/d7/dc/f15 [0,4194304] 0 2024-02-15T18:27:00.026 INFO:tasks.workunit.client.0.smithi042.stdout:6/620: mknod d9/d11/d97/cd0 0 2024-02-15T18:27:00.026 INFO:tasks.workunit.client.0.smithi042.stdout:6/621: dread - d9/d11/d19/f99 zero size 2024-02-15T18:27:00.026 INFO:tasks.workunit.client.0.smithi042.stdout:1/558: truncate de/d54/d9a/fbc 1945822 0 2024-02-15T18:27:00.027 INFO:tasks.workunit.client.0.smithi042.stdout:2/490: dwrite d0/d8/db/de/d1a/f6f [0,4194304] 0 2024-02-15T18:27:00.027 INFO:tasks.workunit.client.0.smithi042.stdout:2/491: write d0/d8/db/de/d1a/d2e/f74 [779474,49244] 0 2024-02-15T18:27:00.027 INFO:tasks.workunit.client.0.smithi042.stdout:2/492: truncate d0/d40/d56/f8d 169241 0 2024-02-15T18:27:00.027 INFO:tasks.workunit.client.0.smithi042.stdout:2/493: chown d0/d8/d10/d31/d3a/d53/d3c/l86 161059308 1 2024-02-15T18:27:00.028 INFO:tasks.workunit.client.0.smithi042.stdout:3/545: link d1/d11/d2e/d4c/c70 d1/d9c/cba 0 2024-02-15T18:27:00.028 INFO:tasks.workunit.client.0.smithi042.stdout:5/597: dwrite d0/d89/fbe [0,4194304] 0 2024-02-15T18:27:00.029 INFO:tasks.workunit.client.1.smithi191.stdout:9/176: link d0/l16 d0/d7/dc/d30/d3b/l41 0 2024-02-15T18:27:00.030 INFO:tasks.workunit.client.0.smithi042.stdout:5/598: write d0/d38/d53/d68/d6a/f56 [2100543,11789] 0 2024-02-15T18:27:00.030 INFO:tasks.workunit.client.0.smithi042.stdout:6/622: truncate d9/d11/d19/d1c/d31/d4f/f98 829159 0 2024-02-15T18:27:00.031 INFO:tasks.workunit.client.0.smithi042.stdout:3/546: getdents d1/d11/d2e/d4c 0 2024-02-15T18:27:00.032 INFO:tasks.workunit.client.1.smithi191.stdout:9/177: dread d0/d7/da/f1c [0,4194304] 0 2024-02-15T18:27:00.033 INFO:tasks.workunit.client.0.smithi042.stdout:2/494: read d0/d8/db/fd [3890844,91633] 0 2024-02-15T18:27:00.033 INFO:tasks.workunit.client.0.smithi042.stdout:6/623: creat d9/d11/d1d/fd1 x:0 0 0 2024-02-15T18:27:00.034 INFO:tasks.workunit.client.1.smithi191.stdout:9/178: mkdir d0/d7/d42 0 2024-02-15T18:27:00.035 INFO:tasks.workunit.client.0.smithi042.stdout:6/624: mkdir d9/d11/d19/d1c/d31/dd2 0 2024-02-15T18:27:00.036 INFO:tasks.workunit.client.0.smithi042.stdout:6/625: symlink d9/d11/d46/dc3/ld3 0 2024-02-15T18:27:00.036 INFO:tasks.workunit.client.0.smithi042.stdout:6/626: stat d9/d11/d1d/l78 0 2024-02-15T18:27:00.036 INFO:tasks.workunit.client.0.smithi042.stdout:7/584: dwrite d10/d15/d39/d6c/f71 [0,4194304] 0 2024-02-15T18:27:00.037 INFO:tasks.workunit.client.0.smithi042.stdout:6/627: symlink d9/d11/d46/ld4 0 2024-02-15T18:27:00.038 INFO:tasks.workunit.client.1.smithi191.stdout:9/179: write d0/d7/f21 [2668532,86693] 0 2024-02-15T18:27:00.039 INFO:tasks.workunit.client.1.smithi191.stdout:9/180: chown d0/l16 16899550 1 2024-02-15T18:27:00.041 INFO:tasks.workunit.client.1.smithi191.stdout:9/181: write d0/d7/da/f27 [102934,42655] 0 2024-02-15T18:27:00.042 INFO:tasks.workunit.client.1.smithi191.stdout:9/182: write d0/d7/fd [110978,34692] 0 2024-02-15T18:27:00.042 INFO:tasks.workunit.client.1.smithi191.stdout:9/183: write d0/d7/f9 [4814300,93589] 0 2024-02-15T18:27:00.042 INFO:tasks.workunit.client.0.smithi042.stdout:1/559: dwrite de/d15/f43 [0,4194304] 0 2024-02-15T18:27:00.042 INFO:tasks.workunit.client.0.smithi042.stdout:1/560: write de/d15/d1b/f49 [4194675,112375] 0 2024-02-15T18:27:00.046 INFO:tasks.workunit.client.0.smithi042.stdout:0/517: sync 2024-02-15T18:27:00.047 INFO:tasks.workunit.client.0.smithi042.stdout:4/528: dwrite da/d11/d29/d4a/d4f/f9d [0,4194304] 0 2024-02-15T18:27:00.047 INFO:tasks.workunit.client.0.smithi042.stdout:3/547: dread d1/d11/f2c [4194304,4194304] 0 2024-02-15T18:27:00.048 INFO:tasks.workunit.client.0.smithi042.stdout:4/529: stat da/d11/d29/d56 0 2024-02-15T18:27:00.052 INFO:tasks.workunit.client.1.smithi191.stdout:9/184: link d0/d7/da/f3e d0/d7/dc/d17/d33/f43 0 2024-02-15T18:27:00.053 INFO:tasks.workunit.client.0.smithi042.stdout:2/495: dwrite d0/f49 [0,4194304] 0 2024-02-15T18:27:00.054 INFO:tasks.workunit.client.0.smithi042.stdout:7/585: symlink d10/d12/d65/lc5 0 2024-02-15T18:27:00.054 INFO:tasks.workunit.client.0.smithi042.stdout:7/586: write d10/d15/d39/d6c/f6f [5059717,62170] 0 2024-02-15T18:27:00.055 INFO:tasks.workunit.client.1.smithi191.stdout:9/185: mkdir d0/d7/dc/d17/d44 0 2024-02-15T18:27:00.056 INFO:tasks.workunit.client.0.smithi042.stdout:6/628: creat d9/d11/d19/d35/dc4/fd5 x:0 0 0 2024-02-15T18:27:00.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:26:58.681398+0000 mgr.smithi042.kfoihb (mgr.24435) 37 : cluster [DBG] pgmap v28: 65 pgs: 65 active+clean; 1.2 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 67 MiB/s rd, 143 MiB/s wr, 449 op/s 2024-02-15T18:27:00.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:59.216194+0000 mon.smithi042 (mon.0) 10 : audit [INF] from='mgr.24435 ' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:00.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:59.235249+0000 mon.smithi191 (mon.1) 72 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:27:00.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:59.237479+0000 mon.smithi191 (mon.1) 73 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:27:00.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:59.251742+0000 mon.smithi042 (mon.0) 11 : audit [INF] from='mgr.24435 ' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:00.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:59.264953+0000 mon.smithi191 (mon.1) 74 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:27:00.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:59.271019+0000 mon.smithi191 (mon.1) 75 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:00.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:26:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:26:59.272292+0000 mon.smithi191 (mon.1) 76 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-15T18:27:00.061 INFO:tasks.workunit.client.0.smithi042.stdout:8/485: sync 2024-02-15T18:27:00.062 INFO:tasks.workunit.client.0.smithi042.stdout:8/486: fdatasync dd/de/d3c/f6d 0 2024-02-15T18:27:00.062 INFO:tasks.workunit.client.0.smithi042.stdout:8/487: fsync dd/f17 0 2024-02-15T18:27:00.062 INFO:tasks.workunit.client.0.smithi042.stdout:8/488: write dd/f2f [3206452,91406] 0 2024-02-15T18:27:00.063 INFO:tasks.workunit.client.1.smithi191.stdout:9/186: symlink d0/d7/dc/d17/d33/l45 0 2024-02-15T18:27:00.064 INFO:tasks.workunit.client.1.smithi191.stdout:9/187: mknod d0/d7/dc/d30/c46 0 2024-02-15T18:27:00.065 INFO:tasks.workunit.client.0.smithi042.stdout:1/561: write de/d15/d1b/f36 [3784460,47877] 0 2024-02-15T18:27:00.065 INFO:tasks.workunit.client.0.smithi042.stdout:0/518: unlink d4/d38/d77/f9b 0 2024-02-15T18:27:00.067 INFO:tasks.workunit.client.1.smithi191.stdout:9/188: link d0/d7/dc/le d0/d7/l47 0 2024-02-15T18:27:00.068 INFO:tasks.workunit.client.0.smithi042.stdout:3/548: rmdir d1 39 2024-02-15T18:27:00.069 INFO:tasks.workunit.client.1.smithi191.stdout:9/189: mkdir d0/d29/d48 0 2024-02-15T18:27:00.074 INFO:tasks.workunit.client.1.smithi191.stdout:7/141: sync 2024-02-15T18:27:00.074 INFO:tasks.workunit.client.0.smithi042.stdout:4/530: rename da/d3b/d41/d8d/fab to da/d11/d67/d5c/d66/fbf 0 2024-02-15T18:27:00.074 INFO:tasks.workunit.client.1.smithi191.stdout:9/190: creat d0/d7/dc/d17/f49 x:0 0 0 2024-02-15T18:27:00.075 INFO:tasks.workunit.client.1.smithi191.stdout:7/142: unlink d0/d1/d1d/c23 0 2024-02-15T18:27:00.076 INFO:tasks.workunit.client.0.smithi042.stdout:7/587: truncate d10/d43/fa4 897184 0 2024-02-15T18:27:00.076 INFO:tasks.workunit.client.1.smithi191.stdout:9/191: rmdir d0/d7/dc/d17/d33 39 2024-02-15T18:27:00.077 INFO:tasks.workunit.client.0.smithi042.stdout:6/629: creat d9/d11/d19/d35/d49/fd6 x:0 0 0 2024-02-15T18:27:00.078 INFO:tasks.workunit.client.1.smithi191.stdout:7/143: mknod d0/d1/c36 0 2024-02-15T18:27:00.078 INFO:tasks.workunit.client.1.smithi191.stdout:7/144: truncate d0/d1/f2a 1485760 0 2024-02-15T18:27:00.078 INFO:tasks.workunit.client.0.smithi042.stdout:0/519: dwrite d4/d38/d39/f6e [4194304,4194304] 0 2024-02-15T18:27:00.079 INFO:tasks.workunit.client.0.smithi042.stdout:8/489: chown l7 179441744 1 2024-02-15T18:27:00.079 INFO:tasks.workunit.client.0.smithi042.stdout:8/490: dread - dd/de/d3c/d4b/f76 zero size 2024-02-15T18:27:00.079 INFO:tasks.workunit.client.1.smithi191.stdout:9/192: rename d0/d7/dc/d30/d3b/l41 to d0/d7/dc/d17/l4a 0 2024-02-15T18:27:00.079 INFO:tasks.workunit.client.1.smithi191.stdout:9/193: write d0/d7/da/f3e [493154,36890] 0 2024-02-15T18:27:00.080 INFO:tasks.workunit.client.0.smithi042.stdout:1/562: stat de/d15/d1b/cb6 0 2024-02-15T18:27:00.080 INFO:tasks.workunit.client.1.smithi191.stdout:7/145: mknod d0/db/c37 0 2024-02-15T18:27:00.081 INFO:tasks.workunit.client.0.smithi042.stdout:3/549: creat d1/d11/d14/d4d/d6c/d7b/fbb x:0 0 0 2024-02-15T18:27:00.082 INFO:tasks.workunit.client.0.smithi042.stdout:3/550: dread d1/de/d1a/f6f [0,4194304] 0 2024-02-15T18:27:00.082 INFO:tasks.workunit.client.0.smithi042.stdout:3/551: chown d1/d11/d2e/f3b 16 1 2024-02-15T18:27:00.089 INFO:tasks.workunit.client.0.smithi042.stdout:3/552: dread d1/d11/d14/f60 [0,4194304] 0 2024-02-15T18:27:00.090 INFO:tasks.workunit.client.0.smithi042.stdout:7/588: dwrite d10/f7e [0,4194304] 0 2024-02-15T18:27:00.100 INFO:tasks.workunit.client.0.smithi042.stdout:3/553: dread d1/f34 [0,4194304] 0 2024-02-15T18:27:00.101 INFO:tasks.workunit.client.0.smithi042.stdout:3/554: dread d1/d11/d2e/d4c/f5b [0,4194304] 0 2024-02-15T18:27:00.102 INFO:tasks.workunit.client.0.smithi042.stdout:3/555: dread d1/de/d19/f3e [0,4194304] 0 2024-02-15T18:27:00.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:26:58.681398+0000 mgr.smithi042.kfoihb (mgr.24435) 37 : cluster [DBG] pgmap v28: 65 pgs: 65 active+clean; 1.2 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 67 MiB/s rd, 143 MiB/s wr, 449 op/s 2024-02-15T18:27:00.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:59.216194+0000 mon.smithi042 (mon.0) 10 : audit [INF] from='mgr.24435 ' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:00.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:59.235249+0000 mon.smithi191 (mon.1) 72 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:27:00.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:59.237479+0000 mon.smithi191 (mon.1) 73 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:27:00.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:59.251742+0000 mon.smithi042 (mon.0) 11 : audit [INF] from='mgr.24435 ' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:00.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:59.264953+0000 mon.smithi191 (mon.1) 74 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:27:00.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:59.271019+0000 mon.smithi191 (mon.1) 75 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:00.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:26:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:26:59.272292+0000 mon.smithi191 (mon.1) 76 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-15T18:27:00.220 INFO:tasks.workunit.client.1.smithi191.stdout:9/194: mknod d0/d7/dc/d30/d3b/c4b 0 2024-02-15T18:27:00.223 INFO:tasks.workunit.client.0.smithi042.stdout:2/496: rename d0/d8/d10/d31/d3a/d53/d3c to d0/d8/d20/d7d/d94 0 2024-02-15T18:27:00.223 INFO:tasks.workunit.client.1.smithi191.stdout:7/146: symlink d0/d1/l38 0 2024-02-15T18:27:00.224 INFO:tasks.workunit.client.0.smithi042.stdout:4/531: mkdir da/d11/d67/d5c/d66/d78/d53/d82/dc0 0 2024-02-15T18:27:00.230 INFO:tasks.workunit.client.0.smithi042.stdout:6/630: stat d9/d11/d19/d35/d49/l4a 0 2024-02-15T18:27:00.232 INFO:tasks.workunit.client.0.smithi042.stdout:0/520: rmdir d4/d9/d34/d8b 39 2024-02-15T18:27:00.234 INFO:tasks.workunit.client.0.smithi042.stdout:8/491: mkdir dd/de/d3c/d4b/d67/d8a 0 2024-02-15T18:27:00.238 INFO:tasks.workunit.client.0.smithi042.stdout:7/589: symlink d10/d49/d6d/d6e/lc6 0 2024-02-15T18:27:00.238 INFO:tasks.workunit.client.0.smithi042.stdout:7/590: fdatasync d10/f73 0 2024-02-15T18:27:00.238 INFO:tasks.workunit.client.0.smithi042.stdout:7/591: readlink d10/d12/d65/lc5 0 2024-02-15T18:27:00.238 INFO:tasks.workunit.client.0.smithi042.stdout:7/592: chown d10/d49/d6d/d6e/lc6 12441122 1 2024-02-15T18:27:00.242 INFO:tasks.workunit.client.1.smithi191.stdout:7/147: unlink d0/d1/d3/d11/d1c/l20 0 2024-02-15T18:27:00.242 INFO:tasks.workunit.client.1.smithi191.stdout:7/148: fdatasync d0/d1/d3/dc/d12/f32 0 2024-02-15T18:27:00.242 INFO:tasks.workunit.client.1.smithi191.stdout:7/149: write d0/db/f2e [436871,94634] 0 2024-02-15T18:27:00.242 INFO:tasks.workunit.client.1.smithi191.stdout:7/150: dread - d0/d1/d3/d11/f18 zero size 2024-02-15T18:27:00.245 INFO:tasks.workunit.client.1.smithi191.stdout:9/195: mknod d0/c4c 0 2024-02-15T18:27:00.248 INFO:tasks.workunit.client.1.smithi191.stdout:7/151: link d0/db/f2d d0/d1/d3/dc/f39 0 2024-02-15T18:27:00.249 INFO:tasks.workunit.client.1.smithi191.stdout:9/196: chown d0/l6 9 1 2024-02-15T18:27:00.252 INFO:tasks.workunit.client.1.smithi191.stdout:7/152: rmdir d0/d1/d3/dc/d12/d1e/d28 0 2024-02-15T18:27:00.253 INFO:tasks.workunit.client.1.smithi191.stdout:9/197: truncate d0/d29/f3d 1316449 0 2024-02-15T18:27:00.254 INFO:tasks.workunit.client.0.smithi042.stdout:3/556: write d1/de/d1a/f7f [1178697,71961] 0 2024-02-15T18:27:00.254 INFO:tasks.workunit.client.0.smithi042.stdout:3/557: chown d1/d11/d2e/d4c/db0/lb9 5611761 1 2024-02-15T18:27:00.254 INFO:tasks.workunit.client.0.smithi042.stdout:8/492: dwrite dd/de/d33/d66/d4e/f28 [0,4194304] 0 2024-02-15T18:27:00.254 INFO:tasks.workunit.client.0.smithi042.stdout:2/497: rmdir d0/d8/d10/d31/d3a 39 2024-02-15T18:27:00.256 INFO:tasks.workunit.client.1.smithi191.stdout:7/153: rename d0/f13 to d0/f3a 0 2024-02-15T18:27:00.259 INFO:tasks.workunit.client.0.smithi042.stdout:1/563: rename de/d15/d67/d66 to de/d54/dbf/dd0 0 2024-02-15T18:27:00.259 INFO:tasks.workunit.client.0.smithi042.stdout:1/564: write de/d54/dbf/dd0/d7c/fb1 [396495,123357] 0 2024-02-15T18:27:00.262 INFO:tasks.workunit.client.0.smithi042.stdout:5/599: sync 2024-02-15T18:27:00.262 INFO:tasks.workunit.client.0.smithi042.stdout:9/570: sync 2024-02-15T18:27:00.263 INFO:tasks.workunit.client.1.smithi191.stdout:9/198: rmdir d0/d7/d42 0 2024-02-15T18:27:00.267 INFO:tasks.workunit.client.0.smithi042.stdout:4/532: mknod da/d11/d67/d5c/d66/d78/d2f/d4d/cc1 0 2024-02-15T18:27:00.267 INFO:tasks.workunit.client.0.smithi042.stdout:4/533: write da/d11/d67/d5c/d66/fbf [758,128671] 0 2024-02-15T18:27:00.269 INFO:tasks.workunit.client.1.smithi191.stdout:7/154: rename d0/c10 to d0/d1/c3b 0 2024-02-15T18:27:00.271 INFO:tasks.workunit.client.0.smithi042.stdout:6/631: unlink d9/d11/d19/d1c/d31/d9b/f9c 0 2024-02-15T18:27:00.271 INFO:tasks.workunit.client.1.smithi191.stdout:7/155: dread d0/d1/f9 [0,4194304] 0 2024-02-15T18:27:00.273 INFO:tasks.workunit.client.0.smithi042.stdout:0/521: mkdir d4/d38/d39/d8e/d8f/db2 0 2024-02-15T18:27:00.275 INFO:tasks.workunit.client.0.smithi042.stdout:7/593: creat d10/d49/d6d/d6e/db3/fc7 x:0 0 0 2024-02-15T18:27:00.275 INFO:tasks.workunit.client.1.smithi191.stdout:7/156: creat d0/db/d26/f3c x:0 0 0 2024-02-15T18:27:00.279 INFO:tasks.workunit.client.1.smithi191.stdout:7/157: dread d0/f8 [0,4194304] 0 2024-02-15T18:27:00.360 INFO:tasks.workunit.client.1.smithi191.stdout:5/95: sync 2024-02-15T18:27:00.361 INFO:tasks.workunit.client.1.smithi191.stdout:5/96: write d0/d14/f17 [4452710,98876] 0 2024-02-15T18:27:00.370 INFO:tasks.workunit.client.1.smithi191.stdout:8/297: sync 2024-02-15T18:27:00.370 INFO:tasks.workunit.client.1.smithi191.stdout:2/173: sync 2024-02-15T18:27:00.371 INFO:tasks.workunit.client.1.smithi191.stdout:2/174: dread - df/d12/f2a zero size 2024-02-15T18:27:00.371 INFO:tasks.workunit.client.1.smithi191.stdout:2/175: read df/d12/d19/d1a/f2c [34314,50773] 0 2024-02-15T18:27:00.371 INFO:tasks.workunit.client.0.smithi042.stdout:8/493: rmdir dd/d11/d75/d82 0 2024-02-15T18:27:00.372 INFO:tasks.workunit.client.1.smithi191.stdout:7/158: truncate d0/d1/f9 578670 0 2024-02-15T18:27:00.374 INFO:tasks.workunit.client.0.smithi042.stdout:3/558: rename d1/de/d19/c7d to d1/de/cbc 0 2024-02-15T18:27:00.375 INFO:tasks.workunit.client.1.smithi191.stdout:5/97: unlink d0/d18/c21 0 2024-02-15T18:27:00.376 INFO:tasks.workunit.client.0.smithi042.stdout:1/565: unlink de/d54/d9a/fbc 0 2024-02-15T18:27:00.380 INFO:tasks.workunit.client.1.smithi191.stdout:4/244: sync 2024-02-15T18:27:00.381 INFO:tasks.workunit.client.1.smithi191.stdout:8/298: creat d3/d2c/d1a/d23/d2a/f6e x:0 0 0 2024-02-15T18:27:00.383 INFO:tasks.workunit.client.1.smithi191.stdout:2/176: mkdir df/d12/d19/d1a/d35 0 2024-02-15T18:27:00.384 INFO:tasks.workunit.client.1.smithi191.stdout:2/177: write df/f1b [591163,86550] 0 2024-02-15T18:27:00.384 INFO:tasks.workunit.client.1.smithi191.stdout:2/178: read df/d12/f22 [225276,42428] 0 2024-02-15T18:27:00.385 INFO:tasks.workunit.client.1.smithi191.stdout:7/159: symlink d0/d1/d3/d11/d1c/l3d 0 2024-02-15T18:27:00.385 INFO:tasks.workunit.client.1.smithi191.stdout:7/160: write d0/db/f2e [713064,30066] 0 2024-02-15T18:27:00.386 INFO:tasks.workunit.client.0.smithi042.stdout:9/571: rmdir d6/dd/d9/d59/d5c 39 2024-02-15T18:27:00.387 INFO:tasks.workunit.client.1.smithi191.stdout:5/98: mkdir d0/d23 0 2024-02-15T18:27:00.388 INFO:tasks.workunit.client.1.smithi191.stdout:5/99: read d0/d18/f1e [3698559,58255] 0 2024-02-15T18:27:00.389 INFO:tasks.workunit.client.1.smithi191.stdout:5/100: read d0/f13 [3784520,19681] 0 2024-02-15T18:27:00.391 INFO:tasks.workunit.client.0.smithi042.stdout:5/600: mkdir d0/d39/d8d/dd2 0 2024-02-15T18:27:00.392 INFO:tasks.workunit.client.1.smithi191.stdout:4/245: creat d2/d5/d14/d44/d30/f6a x:0 0 0 2024-02-15T18:27:00.394 INFO:tasks.workunit.client.0.smithi042.stdout:4/534: truncate da/f25 3428162 0 2024-02-15T18:27:00.396 INFO:tasks.workunit.client.1.smithi191.stdout:8/299: rename d3/d2c/l55 to d3/d2c/d1a/d23/d2e/d47/d5f/d4f/l6f 0 2024-02-15T18:27:00.397 INFO:tasks.workunit.client.0.smithi042.stdout:6/632: creat d9/d11/d19/d1c/d31/d53/fd7 x:0 0 0 2024-02-15T18:27:00.397 INFO:tasks.workunit.client.0.smithi042.stdout:5/601: dread d0/d14/d88/da1/fb1 [0,4194304] 0 2024-02-15T18:27:00.398 INFO:tasks.workunit.client.1.smithi191.stdout:2/179: creat df/d12/f36 x:0 0 0 2024-02-15T18:27:00.398 INFO:tasks.workunit.client.1.smithi191.stdout:2/180: rename df/d12/d19/d1a/d35 to df/d12/d19/d1a/d35/d37 22 2024-02-15T18:27:00.399 INFO:tasks.workunit.client.0.smithi042.stdout:0/522: mknod d4/d38/d39/cb3 0 2024-02-15T18:27:00.400 INFO:tasks.workunit.client.1.smithi191.stdout:7/161: creat d0/d1/d3/f3e x:0 0 0 2024-02-15T18:27:00.400 INFO:tasks.workunit.client.1.smithi191.stdout:7/162: chown d0/d1/df/f17 971814354 1 2024-02-15T18:27:00.401 INFO:tasks.workunit.client.0.smithi042.stdout:7/594: creat d10/d12/d65/da7/fc8 x:0 0 0 2024-02-15T18:27:00.401 INFO:tasks.workunit.client.0.smithi042.stdout:7/595: fdatasync d10/fbf 0 2024-02-15T18:27:00.403 INFO:tasks.workunit.client.1.smithi191.stdout:5/101: mknod d0/d18/c24 0 2024-02-15T18:27:00.405 INFO:tasks.workunit.client.1.smithi191.stdout:7/163: dread d0/f3a [0,4194304] 0 2024-02-15T18:27:00.408 INFO:tasks.workunit.client.1.smithi191.stdout:8/300: symlink d3/d2c/d1a/d23/d2e/d5b/d69/l70 0 2024-02-15T18:27:00.408 INFO:tasks.workunit.client.1.smithi191.stdout:8/301: fsync d3/d2c/d1a/d23/d3e/f68 0 2024-02-15T18:27:00.408 INFO:tasks.workunit.client.0.smithi042.stdout:8/494: creat dd/f8b x:0 0 0 2024-02-15T18:27:00.410 INFO:tasks.workunit.client.1.smithi191.stdout:2/181: unlink df/d12/d19/d1a/f1c 0 2024-02-15T18:27:00.411 INFO:tasks.workunit.client.1.smithi191.stdout:5/102: dread d0/f22 [0,4194304] 0 2024-02-15T18:27:00.411 INFO:tasks.workunit.client.1.smithi191.stdout:4/246: dwrite d2/d5/d7/d3e/f49 [0,4194304] 0 2024-02-15T18:27:00.417 INFO:tasks.workunit.client.0.smithi042.stdout:2/498: sync 2024-02-15T18:27:00.417 INFO:tasks.workunit.client.0.smithi042.stdout:3/559: unlink d1/d11/faf 0 2024-02-15T18:27:00.417 INFO:tasks.workunit.client.1.smithi191.stdout:8/302: dread d3/d2c/d1a/d23/d2e/d47/d5f/f50 [0,4194304] 0 2024-02-15T18:27:00.417 INFO:tasks.workunit.client.1.smithi191.stdout:8/303: readlink d3/d2c/d1a/d23/d2e/l32 0 2024-02-15T18:27:00.418 INFO:tasks.workunit.client.1.smithi191.stdout:7/164: mkdir d0/d1/d3f 0 2024-02-15T18:27:00.419 INFO:tasks.workunit.client.0.smithi042.stdout:2/499: dread d0/d40/d56/f73 [0,4194304] 0 2024-02-15T18:27:00.419 INFO:tasks.workunit.client.0.smithi042.stdout:1/566: rename de/d15/c27 to de/d54/dbf/dd0/cd1 0 2024-02-15T18:27:00.424 INFO:tasks.workunit.client.1.smithi191.stdout:2/182: rmdir df/d12 39 2024-02-15T18:27:00.425 INFO:tasks.workunit.client.0.smithi042.stdout:9/572: creat d6/da/d2f/d7c/fc0 x:0 0 0 2024-02-15T18:27:00.425 INFO:tasks.workunit.client.1.smithi191.stdout:5/103: mkdir d0/d14/d25 0 2024-02-15T18:27:00.425 INFO:tasks.workunit.client.1.smithi191.stdout:5/104: chown d0/d23 3444374 1 2024-02-15T18:27:00.428 INFO:tasks.workunit.client.0.smithi042.stdout:4/535: creat da/d3b/fc2 x:0 0 0 2024-02-15T18:27:00.429 INFO:tasks.workunit.client.0.smithi042.stdout:6/633: mkdir d9/d11/d97/dd8 0 2024-02-15T18:27:00.429 INFO:tasks.workunit.client.0.smithi042.stdout:6/634: write d9/d11/d19/d35/d49/d79/f60 [559154,86582] 0 2024-02-15T18:27:00.431 INFO:tasks.workunit.client.1.smithi191.stdout:4/247: creat d2/d5/d7/d2f/f6b x:0 0 0 2024-02-15T18:27:00.432 INFO:tasks.workunit.client.0.smithi042.stdout:5/602: creat d0/d39/d8d/fd3 x:0 0 0 2024-02-15T18:27:00.433 INFO:tasks.workunit.client.0.smithi042.stdout:0/523: creat d4/d38/d77/fb4 x:0 0 0 2024-02-15T18:27:00.433 INFO:tasks.workunit.client.1.smithi191.stdout:7/165: mknod d0/d1/d3/c40 0 2024-02-15T18:27:00.434 INFO:tasks.workunit.client.1.smithi191.stdout:7/166: stat d0/d1/f9 0 2024-02-15T18:27:00.435 INFO:tasks.workunit.client.1.smithi191.stdout:4/248: creat d2/d5/d7/f6c x:0 0 0 2024-02-15T18:27:00.435 INFO:tasks.workunit.client.0.smithi042.stdout:7/596: mkdir d10/d12/d23/d51/dc9 0 2024-02-15T18:27:00.436 INFO:tasks.workunit.client.0.smithi042.stdout:8/495: unlink dd/de/d33/d66/f5b 0 2024-02-15T18:27:00.438 INFO:tasks.workunit.client.0.smithi042.stdout:8/496: read dd/de/f2a [1308289,68950] 0 2024-02-15T18:27:00.438 INFO:tasks.workunit.client.0.smithi042.stdout:8/497: stat dd/de/d33/d3b/f56 0 2024-02-15T18:27:00.438 INFO:tasks.workunit.client.0.smithi042.stdout:8/498: fdatasync dd/d11/f21 0 2024-02-15T18:27:00.438 INFO:tasks.workunit.client.0.smithi042.stdout:8/499: chown dd/de/d12/c47 598 1 2024-02-15T18:27:00.438 INFO:tasks.workunit.client.1.smithi191.stdout:8/304: rename d3/d2c/d1a/d23/d2e/l3d to d3/l71 0 2024-02-15T18:27:00.439 INFO:tasks.workunit.client.1.smithi191.stdout:8/305: dread d3/d2c/f17 [0,4194304] 0 2024-02-15T18:27:00.441 INFO:tasks.workunit.client.0.smithi042.stdout:3/560: symlink d1/d11/d14/d4d/d73/d89/da8/lbd 0 2024-02-15T18:27:00.441 INFO:tasks.workunit.client.0.smithi042.stdout:3/561: fsync d1/d11/d14/d4d/f7e 0 2024-02-15T18:27:00.442 INFO:tasks.workunit.client.0.smithi042.stdout:2/500: creat d0/d35/f95 x:0 0 0 2024-02-15T18:27:00.442 INFO:tasks.workunit.client.0.smithi042.stdout:2/501: chown d0/d8/db/de/c37 11294611 1 2024-02-15T18:27:00.442 INFO:tasks.workunit.client.0.smithi042.stdout:2/502: readlink d0/d8/db/de/d1a/l48 0 2024-02-15T18:27:00.442 INFO:tasks.workunit.client.0.smithi042.stdout:2/503: chown d0/d8/d20/d41 4 1 2024-02-15T18:27:00.443 INFO:tasks.workunit.client.1.smithi191.stdout:8/306: symlink d3/d2c/d1a/d23/d2e/d5b/d69/l72 0 2024-02-15T18:27:00.444 INFO:tasks.workunit.client.0.smithi042.stdout:1/567: rename de/d15/d67/d63/la9 to de/d15/d67/d8e/ld2 0 2024-02-15T18:27:00.445 INFO:tasks.workunit.client.1.smithi191.stdout:2/183: dwrite df/d12/f2a [0,4194304] 0 2024-02-15T18:27:00.445 INFO:tasks.workunit.client.1.smithi191.stdout:2/184: truncate df/d12/d19/d1a/f29 164880 0 2024-02-15T18:27:00.445 INFO:tasks.workunit.client.0.smithi042.stdout:1/568: write de/d15/d1b/f36 [3643082,120232] 0 2024-02-15T18:27:00.445 INFO:tasks.workunit.client.0.smithi042.stdout:9/573: link d6/da/d1d/d2d/d40/d48/c62 d6/da/d2f/d54/cc1 0 2024-02-15T18:27:00.446 INFO:tasks.workunit.client.1.smithi191.stdout:8/307: rename d3/d2c/d1a/d23/d2e/f4b to d3/d5e/f73 0 2024-02-15T18:27:00.447 INFO:tasks.workunit.client.0.smithi042.stdout:6/635: mkdir d9/d3e/dd9 0 2024-02-15T18:27:00.452 INFO:tasks.workunit.client.0.smithi042.stdout:9/574: dread d6/f2e [0,4194304] 0 2024-02-15T18:27:00.452 INFO:tasks.workunit.client.0.smithi042.stdout:0/524: mkdir d4/d9/d5a/db5 0 2024-02-15T18:27:00.452 INFO:tasks.workunit.client.0.smithi042.stdout:7/597: symlink d10/d49/d6d/d6e/lca 0 2024-02-15T18:27:00.453 INFO:tasks.workunit.client.1.smithi191.stdout:8/308: symlink d3/d2c/d1a/d23/d2e/d47/d5f/l74 0 2024-02-15T18:27:00.453 INFO:tasks.workunit.client.1.smithi191.stdout:8/309: chown d3/d2c/d1a/f45 1 1 2024-02-15T18:27:00.455 INFO:tasks.workunit.client.1.smithi191.stdout:5/105: dwrite d0/f1b [0,4194304] 0 2024-02-15T18:27:00.455 INFO:tasks.workunit.client.0.smithi042.stdout:8/500: rmdir dd/de/d3c/d4b/d67/d8a 0 2024-02-15T18:27:00.456 INFO:tasks.workunit.client.0.smithi042.stdout:0/525: write d4/d9/d34/f36 [530059,10047] 0 2024-02-15T18:27:00.456 INFO:tasks.workunit.client.0.smithi042.stdout:0/526: chown d4/d9/d8d 50340 1 2024-02-15T18:27:00.457 INFO:tasks.workunit.client.1.smithi191.stdout:8/310: creat d3/d2c/f75 x:0 0 0 2024-02-15T18:27:00.458 INFO:tasks.workunit.client.0.smithi042.stdout:3/562: creat d1/fbe x:0 0 0 2024-02-15T18:27:00.458 INFO:tasks.workunit.client.0.smithi042.stdout:3/563: fsync d1/d11/d14/f53 0 2024-02-15T18:27:00.458 INFO:tasks.workunit.client.1.smithi191.stdout:8/311: write d3/f12 [98212,17564] 0 2024-02-15T18:27:00.459 INFO:tasks.workunit.client.0.smithi042.stdout:2/504: mknod d0/d8/d10/d31/d3a/d53/c96 0 2024-02-15T18:27:00.459 INFO:tasks.workunit.client.0.smithi042.stdout:2/505: readlink d0/d8/d20/d7d/d94/l86 0 2024-02-15T18:27:00.460 INFO:tasks.workunit.client.1.smithi191.stdout:5/106: creat d0/d14/d25/f26 x:0 0 0 2024-02-15T18:27:00.460 INFO:tasks.workunit.client.0.smithi042.stdout:6/636: write d9/d11/f15 [3149748,76662] 0 2024-02-15T18:27:00.462 INFO:tasks.workunit.client.0.smithi042.stdout:4/536: rename da/d11/d67/d5c/d66/d78/d2f/d4d/f75 to da/d3a/dbb/fc3 0 2024-02-15T18:27:00.463 INFO:tasks.workunit.client.0.smithi042.stdout:2/506: write d0/d8/db/de/d1a/f6f [2519587,102086] 0 2024-02-15T18:27:00.464 INFO:tasks.workunit.client.1.smithi191.stdout:5/107: unlink d0/l7 0 2024-02-15T18:27:00.464 INFO:tasks.workunit.client.1.smithi191.stdout:5/108: dread - d0/f1f zero size 2024-02-15T18:27:00.466 INFO:tasks.workunit.client.0.smithi042.stdout:5/603: mkdir d0/d38/d53/d91/db0/dd4 0 2024-02-15T18:27:00.467 INFO:tasks.workunit.client.1.smithi191.stdout:5/109: read d0/d14/f17 [182966,62902] 0 2024-02-15T18:27:00.467 INFO:tasks.workunit.client.1.smithi191.stdout:4/249: dwrite d2/d5/f13 [0,4194304] 0 2024-02-15T18:27:00.468 INFO:tasks.workunit.client.0.smithi042.stdout:9/575: symlink d6/da/d2f/d54/lc2 0 2024-02-15T18:27:00.468 INFO:tasks.workunit.client.0.smithi042.stdout:9/576: write d6/da/d2f/d55/d73/d8d/faf [828844,48519] 0 2024-02-15T18:27:00.468 INFO:tasks.workunit.client.0.smithi042.stdout:9/577: fsync d6/da/d1d/d2d/d40/f49 0 2024-02-15T18:27:00.468 INFO:tasks.workunit.client.1.smithi191.stdout:4/250: write d2/d5/d7/d2f/f51 [2974815,13762] 0 2024-02-15T18:27:00.469 INFO:tasks.workunit.client.0.smithi042.stdout:8/501: creat dd/de/d3c/d4b/d81/f8c x:0 0 0 2024-02-15T18:27:00.471 INFO:tasks.workunit.client.0.smithi042.stdout:1/569: dread de/d15/f20 [4194304,4194304] 0 2024-02-15T18:27:00.471 INFO:tasks.workunit.client.1.smithi191.stdout:5/110: unlink d0/d14/f17 0 2024-02-15T18:27:00.475 INFO:tasks.workunit.client.0.smithi042.stdout:6/637: symlink d9/d11/d46/d89/dbc/lda 0 2024-02-15T18:27:00.476 INFO:tasks.workunit.client.1.smithi191.stdout:4/251: creat d2/d5/d7/d2f/f6d x:0 0 0 2024-02-15T18:27:00.477 INFO:tasks.workunit.client.0.smithi042.stdout:4/537: symlink da/d2b/lc4 0 2024-02-15T18:27:00.477 INFO:tasks.workunit.client.0.smithi042.stdout:2/507: symlink d0/d8/d20/l97 0 2024-02-15T18:27:00.479 INFO:tasks.workunit.client.0.smithi042.stdout:0/527: dread d4/d9/d5a/fa4 [0,4194304] 0 2024-02-15T18:27:00.480 INFO:tasks.workunit.client.1.smithi191.stdout:4/252: rename d2/c5e to d2/d5/d14/c6e 0 2024-02-15T18:27:00.480 INFO:tasks.workunit.client.1.smithi191.stdout:4/253: fsync d2/d5/d7/d3e/f49 0 2024-02-15T18:27:00.483 INFO:tasks.workunit.client.0.smithi042.stdout:9/578: creat d6/dd/d9/fc3 x:0 0 0 2024-02-15T18:27:00.483 INFO:tasks.workunit.client.0.smithi042.stdout:8/502: creat dd/de/d3c/d4b/f8d x:0 0 0 2024-02-15T18:27:00.483 INFO:tasks.workunit.client.0.smithi042.stdout:6/638: creat d9/d11/d19/d1c/fdb x:0 0 0 2024-02-15T18:27:00.483 INFO:tasks.workunit.client.0.smithi042.stdout:8/503: fdatasync dd/de/d3c/f7e 0 2024-02-15T18:27:00.484 INFO:tasks.workunit.client.0.smithi042.stdout:4/538: symlink da/d11/d67/d5c/d66/lc5 0 2024-02-15T18:27:00.484 INFO:tasks.workunit.client.1.smithi191.stdout:2/185: dwrite df/f16 [0,4194304] 0 2024-02-15T18:27:00.485 INFO:tasks.workunit.client.0.smithi042.stdout:2/508: mknod d0/d8/d10/d7b/c98 0 2024-02-15T18:27:00.488 INFO:tasks.workunit.client.0.smithi042.stdout:0/528: unlink d4/d9/d34/l3f 0 2024-02-15T18:27:00.491 INFO:tasks.workunit.client.0.smithi042.stdout:2/509: write d0/f3 [4072860,54741] 0 2024-02-15T18:27:00.491 INFO:tasks.workunit.client.0.smithi042.stdout:9/579: mknod d6/cc4 0 2024-02-15T18:27:00.492 INFO:tasks.workunit.client.0.smithi042.stdout:6/639: creat d9/d3e/dd9/fdc x:0 0 0 2024-02-15T18:27:00.493 INFO:tasks.workunit.client.0.smithi042.stdout:8/504: rename dd/de/l36 to dd/de/d33/l8e 0 2024-02-15T18:27:00.494 INFO:tasks.workunit.client.0.smithi042.stdout:0/529: creat d4/d38/d39/d8e/d8f/db2/fb6 x:0 0 0 2024-02-15T18:27:00.495 INFO:tasks.workunit.client.0.smithi042.stdout:2/510: link d0/d8/d10/d31/d3a/d53/f45 d0/d8/d20/d7d/d94/f99 0 2024-02-15T18:27:00.496 INFO:tasks.workunit.client.0.smithi042.stdout:9/580: symlink d6/da2/db1/lc5 0 2024-02-15T18:27:00.496 INFO:tasks.workunit.client.0.smithi042.stdout:6/640: creat d9/d11/d19/d1c/dcf/fdd x:0 0 0 2024-02-15T18:27:00.497 INFO:tasks.workunit.client.0.smithi042.stdout:8/505: creat dd/de/d3c/d4b/d67/d5a/f8f x:0 0 0 2024-02-15T18:27:00.498 INFO:tasks.workunit.client.0.smithi042.stdout:8/506: write dd/de/d33/d3b/f46 [114505,79590] 0 2024-02-15T18:27:00.498 INFO:tasks.workunit.client.1.smithi191.stdout:7/167: dwrite d0/db/d26/f3c [0,4194304] 0 2024-02-15T18:27:00.498 INFO:tasks.workunit.client.0.smithi042.stdout:0/530: link d4/d38/d39/cb3 d4/d9/d34/d57/cb7 0 2024-02-15T18:27:00.500 INFO:tasks.workunit.client.0.smithi042.stdout:2/511: mknod d0/c9a 0 2024-02-15T18:27:00.500 INFO:tasks.workunit.client.0.smithi042.stdout:2/512: read - d0/d8/db/de/d1a/d2e/f92 zero size 2024-02-15T18:27:00.500 INFO:tasks.workunit.client.0.smithi042.stdout:2/513: chown d0/d8/l5c 1854 1 2024-02-15T18:27:00.500 INFO:tasks.workunit.client.0.smithi042.stdout:2/514: readlink d0/d8/l5c 0 2024-02-15T18:27:00.501 INFO:tasks.workunit.client.0.smithi042.stdout:9/581: mknod d6/da/cc6 0 2024-02-15T18:27:00.502 INFO:tasks.workunit.client.1.smithi191.stdout:7/168: rename d0/d1/d3/d11/d1c/l3d to d0/db/l41 0 2024-02-15T18:27:00.503 INFO:tasks.workunit.client.0.smithi042.stdout:6/641: getdents d9/d11/d19/d1c/d31/d53 0 2024-02-15T18:27:00.503 INFO:tasks.workunit.client.0.smithi042.stdout:9/582: write d6/da/d1d/d2d/d40/f43 [2795586,11331] 0 2024-02-15T18:27:00.505 INFO:tasks.workunit.client.0.smithi042.stdout:0/531: unlink d4/d9/d15/f27 0 2024-02-15T18:27:00.506 INFO:tasks.workunit.client.0.smithi042.stdout:0/532: write d4/d9/d34/d57/f7f [25565,32662] 0 2024-02-15T18:27:00.507 INFO:tasks.workunit.client.1.smithi191.stdout:7/169: mkdir d0/d1/d35/d42 0 2024-02-15T18:27:00.507 INFO:tasks.workunit.client.1.smithi191.stdout:7/170: fdatasync d0/d1/d3/d11/f18 0 2024-02-15T18:27:00.509 INFO:tasks.workunit.client.0.smithi042.stdout:2/515: symlink d0/l9b 0 2024-02-15T18:27:00.511 INFO:tasks.workunit.client.1.smithi191.stdout:7/171: rmdir d0/d1/df 39 2024-02-15T18:27:00.511 INFO:tasks.workunit.client.1.smithi191.stdout:7/172: readlink d0/d1/d1d/l29 0 2024-02-15T18:27:00.512 INFO:tasks.workunit.client.1.smithi191.stdout:7/173: write d0/d1/df/f19 [2689261,110736] 0 2024-02-15T18:27:00.513 INFO:tasks.workunit.client.0.smithi042.stdout:7/598: dwrite d10/d49/d6d/d78/f91 [0,4194304] 0 2024-02-15T18:27:00.513 INFO:tasks.workunit.client.0.smithi042.stdout:0/533: rename d4/d9/d5a/d60/f96 to d4/d38/fb8 0 2024-02-15T18:27:00.514 INFO:tasks.workunit.client.0.smithi042.stdout:3/564: dwrite d1/d11/d14/d4d/d6c/f75 [0,4194304] 0 2024-02-15T18:27:00.514 INFO:tasks.workunit.client.0.smithi042.stdout:8/507: dread dd/de/d12/f1e [0,4194304] 0 2024-02-15T18:27:00.515 INFO:tasks.workunit.client.1.smithi191.stdout:4/254: read d2/d5/d7/f25 [4096112,128495] 0 2024-02-15T18:27:00.518 INFO:tasks.workunit.client.1.smithi191.stdout:8/312: dwrite d3/d2c/f43 [0,4194304] 0 2024-02-15T18:27:00.521 INFO:tasks.workunit.client.0.smithi042.stdout:5/604: dwrite d0/d14/f63 [0,4194304] 0 2024-02-15T18:27:00.521 INFO:tasks.workunit.client.1.smithi191.stdout:8/313: write d3/d2c/d1a/f1b [2378730,95289] 0 2024-02-15T18:27:00.522 INFO:tasks.workunit.client.0.smithi042.stdout:3/565: dread d1/d11/d14/d4d/d6c/d7b/f98 [0,4194304] 0 2024-02-15T18:27:00.523 INFO:tasks.workunit.client.0.smithi042.stdout:4/539: dwrite da/d11/d27/f69 [0,4194304] 0 2024-02-15T18:27:00.523 INFO:tasks.workunit.client.1.smithi191.stdout:2/186: dwrite df/d12/d19/d1a/f21 [0,4194304] 0 2024-02-15T18:27:00.528 INFO:tasks.workunit.client.0.smithi042.stdout:9/583: dwrite d6/da/d2f/d6c/f94 [0,4194304] 0 2024-02-15T18:27:00.528 INFO:tasks.workunit.client.0.smithi042.stdout:7/599: mkdir d10/d15/d39/d6c/dcb 0 2024-02-15T18:27:00.530 INFO:tasks.workunit.client.0.smithi042.stdout:0/534: dwrite d4/d9/d62/fa0 [0,4194304] 0 2024-02-15T18:27:00.530 INFO:tasks.workunit.client.0.smithi042.stdout:3/566: fsync d1/fbe 0 2024-02-15T18:27:00.539 INFO:tasks.workunit.client.1.smithi191.stdout:7/174: dwrite d0/d1/df/f19 [4194304,4194304] 0 2024-02-15T18:27:00.540 INFO:tasks.workunit.client.1.smithi191.stdout:7/175: chown d0 114516 1 2024-02-15T18:27:00.545 INFO:tasks.workunit.client.1.smithi191.stdout:4/255: rmdir d2/d5/d7/d2f 39 2024-02-15T18:27:00.553 INFO:tasks.workunit.client.0.smithi042.stdout:2/516: rename d0/d8/db/de/d1a/d2e/c57 to d0/d8/d20/d41/c9c 0 2024-02-15T18:27:00.553 INFO:tasks.workunit.client.0.smithi042.stdout:2/517: write d0/d8/db/d44/f69 [1150531,71626] 0 2024-02-15T18:27:00.557 INFO:tasks.workunit.client.1.smithi191.stdout:7/176: dwrite d0/f3a [0,4194304] 0 2024-02-15T18:27:00.557 INFO:tasks.workunit.client.1.smithi191.stdout:7/177: fdatasync d0/d1/df/f1a 0 2024-02-15T18:27:00.557 INFO:tasks.workunit.client.1.smithi191.stdout:7/178: chown d0/d1/d35/d42 0 1 2024-02-15T18:27:00.557 INFO:tasks.workunit.client.1.smithi191.stdout:8/314: truncate d3/d2c/d1a/f20 5376071 0 2024-02-15T18:27:00.557 INFO:tasks.workunit.client.1.smithi191.stdout:8/315: chown d3/d2c/d1a/l6c 1433 1 2024-02-15T18:27:00.560 INFO:tasks.workunit.client.0.smithi042.stdout:8/508: truncate dd/de/d3c/d4b/d67/f2b 1017996 0 2024-02-15T18:27:00.566 INFO:tasks.workunit.client.1.smithi191.stdout:2/187: rename l5 to df/l38 0 2024-02-15T18:27:00.570 INFO:tasks.workunit.client.1.smithi191.stdout:4/256: rmdir d2/d5/d14/d47/d48/d59 0 2024-02-15T18:27:00.578 INFO:tasks.workunit.client.1.smithi191.stdout:7/179: symlink d0/d1/d35/d42/l43 0 2024-02-15T18:27:00.582 INFO:tasks.workunit.client.1.smithi191.stdout:2/188: link df/d12/d19/d1a/f29 df/d12/f39 0 2024-02-15T18:27:00.582 INFO:tasks.workunit.client.1.smithi191.stdout:2/189: fsync df/d12/f2b 0 2024-02-15T18:27:00.582 INFO:tasks.workunit.client.1.smithi191.stdout:2/190: readlink df/d12/d19/l2e 0 2024-02-15T18:27:00.585 INFO:tasks.workunit.client.1.smithi191.stdout:4/257: creat d2/d5/d7/d3e/d42/f6f x:0 0 0 2024-02-15T18:27:00.587 INFO:tasks.workunit.client.1.smithi191.stdout:7/180: rename d0/d1/ca to d0/d1/d3/d11/d1c/c44 0 2024-02-15T18:27:00.590 INFO:tasks.workunit.client.0.smithi042.stdout:4/540: rmdir da/d2b/d6e 39 2024-02-15T18:27:00.590 INFO:tasks.workunit.client.1.smithi191.stdout:2/191: truncate df/d12/f2a 159181 0 2024-02-15T18:27:00.594 INFO:tasks.workunit.client.1.smithi191.stdout:4/258: symlink d2/d5/d14/d47/l70 0 2024-02-15T18:27:00.594 INFO:tasks.workunit.client.1.smithi191.stdout:4/259: truncate d2/d5/d14/d44/d30/f6a 363588 0 2024-02-15T18:27:00.595 INFO:tasks.workunit.client.0.smithi042.stdout:9/584: symlink d6/da/d2f/d6c/lc7 0 2024-02-15T18:27:00.595 INFO:tasks.workunit.client.0.smithi042.stdout:9/585: chown d6/da/d1d/d2d/c50 24550 1 2024-02-15T18:27:00.595 INFO:tasks.workunit.client.1.smithi191.stdout:2/192: mknod df/d12/d1f/c3a 0 2024-02-15T18:27:00.596 INFO:tasks.workunit.client.0.smithi042.stdout:7/600: mknod d10/d49/d6d/ccc 0 2024-02-15T18:27:00.597 INFO:tasks.workunit.client.1.smithi191.stdout:2/193: mknod df/d12/d19/d1a/c3b 0 2024-02-15T18:27:00.598 INFO:tasks.workunit.client.1.smithi191.stdout:4/260: dread d2/d5/d14/f58 [0,4194304] 0 2024-02-15T18:27:00.599 INFO:tasks.workunit.client.0.smithi042.stdout:0/535: unlink d4/d9/d5a/fa4 0 2024-02-15T18:27:00.602 INFO:tasks.workunit.client.0.smithi042.stdout:9/586: dread d6/da/d2f/d55/d73/f77 [0,4194304] 0 2024-02-15T18:27:00.607 INFO:tasks.workunit.client.0.smithi042.stdout:9/587: dread d6/dd/d9/f14 [0,4194304] 0 2024-02-15T18:27:00.608 INFO:tasks.workunit.client.1.smithi191.stdout:7/181: dwrite d0/f14 [4194304,4194304] 0 2024-02-15T18:27:00.608 INFO:tasks.workunit.client.1.smithi191.stdout:7/182: dread - d0/d1/d3/dc/f39 zero size 2024-02-15T18:27:00.609 INFO:tasks.workunit.client.1.smithi191.stdout:8/316: dwrite d3/fa [4194304,4194304] 0 2024-02-15T18:27:00.609 INFO:tasks.workunit.client.1.smithi191.stdout:8/317: read d3/d5e/f73 [82908,2698] 0 2024-02-15T18:27:00.638 INFO:tasks.workunit.client.1.smithi191.stdout:2/194: unlink df/l17 0 2024-02-15T18:27:00.641 INFO:tasks.workunit.client.0.smithi042.stdout:2/518: truncate d0/d8/db/fd 1145589 0 2024-02-15T18:27:00.642 INFO:tasks.workunit.client.1.smithi191.stdout:4/261: stat d2/d5/l3d 0 2024-02-15T18:27:00.645 INFO:tasks.workunit.client.0.smithi042.stdout:8/509: getdents dd/de/d33/d66/d4e 0 2024-02-15T18:27:00.646 INFO:tasks.workunit.client.1.smithi191.stdout:4/262: dread d2/f34 [0,4194304] 0 2024-02-15T18:27:00.648 INFO:tasks.workunit.client.1.smithi191.stdout:4/263: dread d2/d5/d14/d44/d30/f6a [0,4194304] 0 2024-02-15T18:27:00.648 INFO:tasks.workunit.client.1.smithi191.stdout:4/264: chown d2/d5/d14/d44/c4c 33775487 1 2024-02-15T18:27:00.648 INFO:tasks.workunit.client.1.smithi191.stdout:4/265: chown d2/d5/d7/c5f 6317 1 2024-02-15T18:27:00.653 INFO:tasks.workunit.client.0.smithi042.stdout:5/605: symlink d0/d38/d53/d91/db0/ld5 0 2024-02-15T18:27:00.654 INFO:tasks.workunit.client.0.smithi042.stdout:4/541: rmdir da/d11/d29/db8 39 2024-02-15T18:27:00.654 INFO:tasks.workunit.client.0.smithi042.stdout:4/542: readlink da/d11/d29/l7d 0 2024-02-15T18:27:00.657 INFO:tasks.workunit.client.0.smithi042.stdout:7/601: creat d10/d15/d39/d6c/dba/fcd x:0 0 0 2024-02-15T18:27:00.657 INFO:tasks.workunit.client.0.smithi042.stdout:7/602: write d10/d12/f96 [872399,715] 0 2024-02-15T18:27:00.659 INFO:tasks.workunit.client.0.smithi042.stdout:0/536: rmdir d4/d38/d77/d2e 39 2024-02-15T18:27:00.659 INFO:tasks.workunit.client.0.smithi042.stdout:0/537: fdatasync d4/d9/d34/f36 0 2024-02-15T18:27:00.660 INFO:tasks.workunit.client.0.smithi042.stdout:9/588: mknod d6/cc8 0 2024-02-15T18:27:00.661 INFO:tasks.workunit.client.0.smithi042.stdout:9/589: chown d6/dd/d9/f45 6778 1 2024-02-15T18:27:00.661 INFO:tasks.workunit.client.0.smithi042.stdout:2/519: dwrite d0/f8c [0,4194304] 0 2024-02-15T18:27:00.661 INFO:tasks.workunit.client.1.smithi191.stdout:7/183: creat d0/d1/d3/d11/d1c/f45 x:0 0 0 2024-02-15T18:27:00.661 INFO:tasks.workunit.client.1.smithi191.stdout:7/184: chown d0/db/d26 22463063 1 2024-02-15T18:27:00.663 INFO:tasks.workunit.client.0.smithi042.stdout:4/543: creat da/d11/d67/fc6 x:0 0 0 2024-02-15T18:27:00.665 INFO:tasks.workunit.client.1.smithi191.stdout:8/318: getdents d3/d2c/d1a/d23/d2e/d64 0 2024-02-15T18:27:00.665 INFO:tasks.workunit.client.1.smithi191.stdout:8/319: chown d3/d2c/d1a/d23 306785 1 2024-02-15T18:27:00.666 INFO:tasks.workunit.client.0.smithi042.stdout:1/570: sync 2024-02-15T18:27:00.666 INFO:tasks.workunit.client.0.smithi042.stdout:0/538: mknod d4/d38/d39/d8e/d8f/db2/cb9 0 2024-02-15T18:27:00.666 INFO:tasks.workunit.client.0.smithi042.stdout:1/571: dread - de/d15/d1b/d44/fb2 zero size 2024-02-15T18:27:00.666 INFO:tasks.workunit.client.1.smithi191.stdout:2/195: mknod df/d12/d19/d1a/d35/c3c 0 2024-02-15T18:27:00.668 INFO:tasks.workunit.client.0.smithi042.stdout:1/572: write de/d15/d67/d46/f59 [3399035,74113] 0 2024-02-15T18:27:00.668 INFO:tasks.workunit.client.0.smithi042.stdout:9/590: creat d6/da/d1d/d2d/d40/fc9 x:0 0 0 2024-02-15T18:27:00.668 INFO:tasks.workunit.client.1.smithi191.stdout:4/266: mkdir d2/d5/d14/d71 0 2024-02-15T18:27:00.669 INFO:tasks.workunit.client.0.smithi042.stdout:9/591: write d6/dd/f20 [3333263,125291] 0 2024-02-15T18:27:00.669 INFO:tasks.workunit.client.0.smithi042.stdout:8/510: dwrite dd/de/d12/f1e [0,4194304] 0 2024-02-15T18:27:00.669 INFO:tasks.workunit.client.0.smithi042.stdout:6/642: sync 2024-02-15T18:27:00.669 INFO:tasks.workunit.client.0.smithi042.stdout:3/567: sync 2024-02-15T18:27:00.671 INFO:tasks.workunit.client.0.smithi042.stdout:7/603: rmdir d10/d15/d39/d6c/dba 39 2024-02-15T18:27:00.671 INFO:tasks.workunit.client.0.smithi042.stdout:7/604: write d10/d15/d39/d6c/f71 [4867619,13538] 0 2024-02-15T18:27:00.671 INFO:tasks.workunit.client.0.smithi042.stdout:7/605: fdatasync d10/d12/f9e 0 2024-02-15T18:27:00.677 INFO:tasks.workunit.client.1.smithi191.stdout:1/248: sync 2024-02-15T18:27:00.677 INFO:tasks.workunit.client.1.smithi191.stdout:3/156: sync 2024-02-15T18:27:00.677 INFO:tasks.workunit.client.1.smithi191.stdout:3/157: fdatasync d6/de/f13 0 2024-02-15T18:27:00.677 INFO:tasks.workunit.client.1.smithi191.stdout:6/191: sync 2024-02-15T18:27:00.680 INFO:tasks.workunit.client.1.smithi191.stdout:0/108: sync 2024-02-15T18:27:00.680 INFO:tasks.workunit.client.1.smithi191.stdout:0/109: dread - d7/da/f18 zero size 2024-02-15T18:27:00.680 INFO:tasks.workunit.client.1.smithi191.stdout:7/185: rename d0/d1/d3/dc/d12 to d0/d1/d35/d42/d46 0 2024-02-15T18:27:00.682 INFO:tasks.workunit.client.0.smithi042.stdout:2/520: mknod d0/d8/d20/d3f/d7c/c9d 0 2024-02-15T18:27:00.683 INFO:tasks.workunit.client.0.smithi042.stdout:4/544: dwrite da/d11/f23 [0,4194304] 0 2024-02-15T18:27:00.683 INFO:tasks.workunit.client.1.smithi191.stdout:8/320: creat d3/d2c/d1a/d23/d2e/d64/f76 x:0 0 0 2024-02-15T18:27:00.685 INFO:tasks.workunit.client.1.smithi191.stdout:2/196: rmdir df/d12/d19 39 2024-02-15T18:27:00.686 INFO:tasks.workunit.client.0.smithi042.stdout:0/539: rename d4/d38/d39/c98 to d4/d9/d34/dab/cba 0 2024-02-15T18:27:00.688 INFO:tasks.workunit.client.1.smithi191.stdout:3/158: creat d6/d1b/f39 x:0 0 0 2024-02-15T18:27:00.688 INFO:tasks.workunit.client.1.smithi191.stdout:4/267: dwrite d2/f4 [4194304,4194304] 0 2024-02-15T18:27:00.691 INFO:tasks.workunit.client.0.smithi042.stdout:4/545: dread da/d11/d29/d4a/d4f/f6c [0,4194304] 0 2024-02-15T18:27:00.691 INFO:tasks.workunit.client.0.smithi042.stdout:7/606: dwrite d10/f53 [0,4194304] 0 2024-02-15T18:27:00.760 INFO:tasks.workunit.client.1.smithi191.stdout:6/192: mknod d8/d9/d2b/c31 0 2024-02-15T18:27:00.760 INFO:tasks.workunit.client.1.smithi191.stdout:6/193: truncate d8/f30 965155 0 2024-02-15T18:27:00.760 INFO:tasks.workunit.client.1.smithi191.stdout:6/194: write d8/d9/fc [1733080,105340] 0 2024-02-15T18:27:00.762 INFO:tasks.workunit.client.0.smithi042.stdout:1/573: mkdir de/d54/d9a/dc8/dd3 0 2024-02-15T18:27:00.762 INFO:tasks.workunit.client.0.smithi042.stdout:1/574: chown de/d15/d1b/f49 2283348 1 2024-02-15T18:27:00.762 INFO:tasks.workunit.client.1.smithi191.stdout:0/110: symlink d7/dd/l1c 0 2024-02-15T18:27:00.763 INFO:tasks.workunit.client.0.smithi042.stdout:9/592: symlink d6/da/d7b/lca 0 2024-02-15T18:27:00.764 INFO:tasks.workunit.client.1.smithi191.stdout:1/249: rename d0/d8/c21 to d0/d8/df/d41/c4e 0 2024-02-15T18:27:00.765 INFO:tasks.workunit.client.0.smithi042.stdout:8/511: symlink dd/de/d33/d3b/l90 0 2024-02-15T18:27:00.766 INFO:tasks.workunit.client.1.smithi191.stdout:3/159: symlink d6/d1b/d1f/l3a 0 2024-02-15T18:27:00.766 INFO:tasks.workunit.client.1.smithi191.stdout:3/160: readlink d6/de/l15 0 2024-02-15T18:27:00.768 INFO:tasks.workunit.client.1.smithi191.stdout:4/268: symlink d2/d5/d7/d2f/d41/l72 0 2024-02-15T18:27:00.768 INFO:tasks.workunit.client.0.smithi042.stdout:6/643: creat d9/d11/d19/d1c/d31/d4f/fde x:0 0 0 2024-02-15T18:27:00.768 INFO:tasks.workunit.client.0.smithi042.stdout:6/644: dread - d9/d11/d19/fb3 zero size 2024-02-15T18:27:00.769 INFO:tasks.workunit.client.1.smithi191.stdout:6/195: chown d8/d19/f1f 24 1 2024-02-15T18:27:00.769 INFO:tasks.workunit.client.0.smithi042.stdout:2/521: truncate d0/d8/db/f80 4028697 0 2024-02-15T18:27:00.770 INFO:tasks.workunit.client.1.smithi191.stdout:0/111: mknod d7/da/d17/c1d 0 2024-02-15T18:27:00.773 INFO:tasks.workunit.client.1.smithi191.stdout:7/186: rename d0/d1/c30 to d0/d1/df/c47 0 2024-02-15T18:27:00.773 INFO:tasks.workunit.client.1.smithi191.stdout:7/187: chown d0/d1/d3/f22 13 1 2024-02-15T18:27:00.774 INFO:tasks.workunit.client.0.smithi042.stdout:2/522: write d0/d8/db/d44/f84 [3798006,84579] 0 2024-02-15T18:27:00.775 INFO:tasks.workunit.client.1.smithi191.stdout:6/196: unlink d8/d9/f14 0 2024-02-15T18:27:00.776 INFO:tasks.workunit.client.1.smithi191.stdout:6/197: dread d8/f30 [0,4194304] 0 2024-02-15T18:27:00.777 INFO:tasks.workunit.client.0.smithi042.stdout:0/540: link d4/d9/d34/d4e/d6c/d70/faa d4/d38/d39/d8e/fbb 0 2024-02-15T18:27:00.779 INFO:tasks.workunit.client.1.smithi191.stdout:0/112: unlink l0 0 2024-02-15T18:27:00.780 INFO:tasks.workunit.client.0.smithi042.stdout:4/546: creat da/d2b/d7e/fc7 x:0 0 0 2024-02-15T18:27:00.782 INFO:tasks.workunit.client.1.smithi191.stdout:2/197: rename df/d12/f2a to df/d12/d19/d1a/f3d 0 2024-02-15T18:27:00.784 INFO:tasks.workunit.client.1.smithi191.stdout:7/188: creat d0/d1/d35/f48 x:0 0 0 2024-02-15T18:27:00.784 INFO:tasks.workunit.client.1.smithi191.stdout:7/189: dread - d0/d1/d35/f48 zero size 2024-02-15T18:27:00.785 INFO:tasks.workunit.client.0.smithi042.stdout:7/607: symlink d10/d15/d39/d6c/lce 0 2024-02-15T18:27:00.786 INFO:tasks.workunit.client.1.smithi191.stdout:0/113: link d7/da/d17/c1d d7/c1e 0 2024-02-15T18:27:00.788 INFO:tasks.workunit.client.1.smithi191.stdout:4/269: rename d2/d5/d7/d2f/f6d to d2/d5/d7/d2f/d41/f73 0 2024-02-15T18:27:00.789 INFO:tasks.workunit.client.1.smithi191.stdout:2/198: link df/d12/l1d df/l3e 0 2024-02-15T18:27:00.791 INFO:tasks.workunit.client.1.smithi191.stdout:0/114: truncate d7/dd/f11 1442137 0 2024-02-15T18:27:00.791 INFO:tasks.workunit.client.1.smithi191.stdout:1/250: dwrite d0/d8/df/d12/f32 [0,4194304] 0 2024-02-15T18:27:00.791 INFO:tasks.workunit.client.1.smithi191.stdout:1/251: chown d0/d8/df/d41/f36 7172535 1 2024-02-15T18:27:00.791 INFO:tasks.workunit.client.1.smithi191.stdout:0/115: dread d7/f13 [0,4194304] 0 2024-02-15T18:27:00.792 INFO:tasks.workunit.client.1.smithi191.stdout:3/161: dwrite d6/fa [0,4194304] 0 2024-02-15T18:27:00.793 INFO:tasks.workunit.client.1.smithi191.stdout:6/198: dwrite d8/d19/f1f [0,4194304] 0 2024-02-15T18:27:00.794 INFO:tasks.workunit.client.1.smithi191.stdout:7/190: rename d0/d1/c3b to d0/d1/d35/d42/d46/c49 0 2024-02-15T18:27:00.796 INFO:tasks.workunit.client.1.smithi191.stdout:0/116: dread d7/da/f14 [0,4194304] 0 2024-02-15T18:27:00.806 INFO:tasks.workunit.client.0.smithi042.stdout:5/606: sync 2024-02-15T18:27:00.808 INFO:tasks.workunit.client.0.smithi042.stdout:8/512: unlink dd/f4d 0 2024-02-15T18:27:00.810 INFO:tasks.workunit.client.1.smithi191.stdout:4/270: truncate d2/d5/d7/f25 2425928 0 2024-02-15T18:27:00.810 INFO:tasks.workunit.client.1.smithi191.stdout:4/271: fsync d2/d5/d14/d44/d30/f6a 0 2024-02-15T18:27:00.812 INFO:tasks.workunit.client.0.smithi042.stdout:3/568: mkdir d1/d11/d14/dbf 0 2024-02-15T18:27:00.812 INFO:tasks.workunit.client.0.smithi042.stdout:3/569: chown d1/de/d19/f3e 25 1 2024-02-15T18:27:00.812 INFO:tasks.workunit.client.0.smithi042.stdout:3/570: chown d1/l5f 62347 1 2024-02-15T18:27:00.812 INFO:tasks.workunit.client.0.smithi042.stdout:3/571: chown d1/d11/d14/d4d/d73/d89/fa7 7 1 2024-02-15T18:27:00.812 INFO:tasks.workunit.client.0.smithi042.stdout:3/572: write d1/d11/d14/f39 [3201880,117409] 0 2024-02-15T18:27:00.812 INFO:tasks.workunit.client.0.smithi042.stdout:3/573: write d1/d11/d14/d4d/d6c/d7b/fb5 [761944,32165] 0 2024-02-15T18:27:00.812 INFO:tasks.workunit.client.0.smithi042.stdout:3/574: fsync d1/de/f2d 0 2024-02-15T18:27:00.813 INFO:tasks.workunit.client.1.smithi191.stdout:2/199: symlink df/d12/d19/d1a/d35/l3f 0 2024-02-15T18:27:00.813 INFO:tasks.workunit.client.0.smithi042.stdout:6/645: symlink d9/d11/d19/d1c/d31/d53/dbe/ldf 0 2024-02-15T18:27:00.815 INFO:tasks.workunit.client.0.smithi042.stdout:2/523: mkdir d0/d35/d9e 0 2024-02-15T18:27:00.815 INFO:tasks.workunit.client.0.smithi042.stdout:2/524: dread - d0/d8/d20/d7d/d94/f87 zero size 2024-02-15T18:27:00.815 INFO:tasks.workunit.client.1.smithi191.stdout:0/117: dwrite d7/fb [4194304,4194304] 0 2024-02-15T18:27:00.817 INFO:tasks.workunit.client.1.smithi191.stdout:1/252: creat d0/d8/df/d2e/f4f x:0 0 0 2024-02-15T18:27:00.819 INFO:tasks.workunit.client.0.smithi042.stdout:6/646: dread d9/d11/d19/d35/d49/f52 [0,4194304] 0 2024-02-15T18:27:00.821 INFO:tasks.workunit.client.0.smithi042.stdout:0/541: creat d4/d38/d39/d8e/fbc x:0 0 0 2024-02-15T18:27:00.823 INFO:tasks.workunit.client.1.smithi191.stdout:3/162: rename d6/de to d6/d3b 0 2024-02-15T18:27:00.824 INFO:tasks.workunit.client.0.smithi042.stdout:2/525: dread d0/d8/db/de/d1a/f2b [0,4194304] 0 2024-02-15T18:27:00.826 INFO:tasks.workunit.client.0.smithi042.stdout:7/608: mknod d10/d49/d6d/d6e/db3/ccf 0 2024-02-15T18:27:00.829 INFO:tasks.workunit.client.0.smithi042.stdout:1/575: rename de/d15/d1b/f57 to de/d54/dbf/dd0/fd4 0 2024-02-15T18:27:00.830 INFO:tasks.workunit.client.0.smithi042.stdout:1/576: chown de/d54/l6c 12 1 2024-02-15T18:27:00.831 INFO:tasks.workunit.client.1.smithi191.stdout:6/199: creat d8/d9/d2b/f32 x:0 0 0 2024-02-15T18:27:00.831 INFO:tasks.workunit.client.1.smithi191.stdout:6/200: readlink d8/d9/l20 0 2024-02-15T18:27:00.833 INFO:tasks.workunit.client.0.smithi042.stdout:5/607: symlink d0/d22/d7b/da9/ld6 0 2024-02-15T18:27:00.833 INFO:tasks.workunit.client.0.smithi042.stdout:5/608: fsync d0/d38/d53/d68/d6a/dc7/fcc 0 2024-02-15T18:27:00.833 INFO:tasks.workunit.client.0.smithi042.stdout:5/609: write d0/d39/d8d/fd3 [297804,68197] 0 2024-02-15T18:27:00.834 INFO:tasks.workunit.client.0.smithi042.stdout:7/609: dread d10/d12/d23/f55 [0,4194304] 0 2024-02-15T18:27:00.835 INFO:tasks.workunit.client.0.smithi042.stdout:2/526: dread d0/f3 [0,4194304] 0 2024-02-15T18:27:00.841 INFO:tasks.workunit.client.1.smithi191.stdout:7/191: mknod d0/d1/d3/c4a 0 2024-02-15T18:27:00.841 INFO:tasks.workunit.client.1.smithi191.stdout:7/192: dread - d0/db/f2d zero size 2024-02-15T18:27:00.841 INFO:tasks.workunit.client.1.smithi191.stdout:7/193: write d0/d1/d3/dc/f39 [516183,118721] 0 2024-02-15T18:27:00.842 INFO:tasks.workunit.client.0.smithi042.stdout:6/647: creat d9/d11/d19/d35/dc4/fe0 x:0 0 0 2024-02-15T18:27:00.846 INFO:tasks.workunit.client.0.smithi042.stdout:0/542: mknod d4/d9/d34/d4e/cbd 0 2024-02-15T18:27:00.847 INFO:tasks.workunit.client.1.smithi191.stdout:3/163: dwrite d6/d3b/f13 [0,4194304] 0 2024-02-15T18:27:00.847 INFO:tasks.workunit.client.1.smithi191.stdout:3/164: write d6/d3b/f2c [1171079,72334] 0 2024-02-15T18:27:00.852 INFO:tasks.workunit.client.0.smithi042.stdout:9/593: rename d6/da/d7b/caa to d6/da/d75/ccb 0 2024-02-15T18:27:00.853 INFO:tasks.workunit.client.1.smithi191.stdout:2/200: creat df/f40 x:0 0 0 2024-02-15T18:27:00.853 INFO:tasks.workunit.client.1.smithi191.stdout:2/201: getdents df/d12/d32 0 2024-02-15T18:27:00.855 INFO:tasks.workunit.client.0.smithi042.stdout:1/577: creat de/d15/d67/d74/d7e/fd5 x:0 0 0 2024-02-15T18:27:00.858 INFO:tasks.workunit.client.0.smithi042.stdout:2/527: dread d0/d8/d10/f17 [0,4194304] 0 2024-02-15T18:27:00.858 INFO:tasks.workunit.client.0.smithi042.stdout:5/610: dwrite d0/d39/d8d/fc6 [0,4194304] 0 2024-02-15T18:27:00.858 INFO:tasks.workunit.client.0.smithi042.stdout:5/611: chown d0/d14/f87 2 1 2024-02-15T18:27:00.858 INFO:tasks.workunit.client.0.smithi042.stdout:6/648: dwrite d9/d11/d19/d1c/dcf/fdd [0,4194304] 0 2024-02-15T18:27:00.858 INFO:tasks.workunit.client.0.smithi042.stdout:6/649: chown d9/d11/d19/d35/f4e 81787 1 2024-02-15T18:27:00.859 INFO:tasks.workunit.client.1.smithi191.stdout:1/253: symlink d0/d8/df/d12/l50 0 2024-02-15T18:27:00.859 INFO:tasks.workunit.client.1.smithi191.stdout:1/254: truncate d0/d8/df/d12/d33/d49/f4a 4227925 0 2024-02-15T18:27:00.859 INFO:tasks.workunit.client.0.smithi042.stdout:7/610: creat d10/d15/d8f/da5/dad/fd0 x:0 0 0 2024-02-15T18:27:00.861 INFO:tasks.workunit.client.1.smithi191.stdout:6/201: link d8/f30 d8/f33 0 2024-02-15T18:27:00.861 INFO:tasks.workunit.client.0.smithi042.stdout:0/543: rmdir d4/d9/d34/d4e/d6c/d70 39 2024-02-15T18:27:00.865 INFO:tasks.workunit.client.1.smithi191.stdout:4/272: truncate d2/f4 4292443 0 2024-02-15T18:27:00.865 INFO:tasks.workunit.client.1.smithi191.stdout:4/273: readlink d2/d5/d14/l2a 0 2024-02-15T18:27:00.865 INFO:tasks.workunit.client.1.smithi191.stdout:4/274: chown d2/d5/d14/l65 10872562 1 2024-02-15T18:27:00.868 INFO:tasks.workunit.client.0.smithi042.stdout:8/513: rename dd/de/d12/f24 to dd/de/d3c/d4b/d67/d5a/f91 0 2024-02-15T18:27:00.868 INFO:tasks.workunit.client.0.smithi042.stdout:9/594: mkdir d6/dd/dcc 0 2024-02-15T18:27:00.868 INFO:tasks.workunit.client.0.smithi042.stdout:9/595: write d6/da/d1d/d2d/d40/f49 [1083235,12378] 0 2024-02-15T18:27:00.868 INFO:tasks.workunit.client.1.smithi191.stdout:7/194: dwrite d0/d1/f21 [0,4194304] 0 2024-02-15T18:27:00.869 INFO:tasks.workunit.client.1.smithi191.stdout:7/195: stat d0/d1/df/f19 0 2024-02-15T18:27:00.869 INFO:tasks.workunit.client.1.smithi191.stdout:7/196: write d0/f3a [8855704,97181] 0 2024-02-15T18:27:00.869 INFO:tasks.workunit.client.1.smithi191.stdout:7/197: dread - d0/d1/d3/d11/f18 zero size 2024-02-15T18:27:00.869 INFO:tasks.workunit.client.1.smithi191.stdout:2/202: creat df/d12/d1e/f41 x:0 0 0 2024-02-15T18:27:00.869 INFO:tasks.workunit.client.0.smithi042.stdout:1/578: mkdir de/d54/dbf/dd0/d7c/dd6 0 2024-02-15T18:27:00.871 INFO:tasks.workunit.client.0.smithi042.stdout:9/596: write d6/da/d2f/d6c/f71 [1979700,80704] 0 2024-02-15T18:27:00.872 INFO:tasks.workunit.client.1.smithi191.stdout:0/118: mknod d7/d1b/c1f 0 2024-02-15T18:27:00.875 INFO:tasks.workunit.client.1.smithi191.stdout:3/165: dwrite d6/f8 [0,4194304] 0 2024-02-15T18:27:00.876 INFO:tasks.workunit.client.0.smithi042.stdout:2/528: link d0/d8/db/de/d1a/d2e/c68 d0/d8/db/de/d1a/c9f 0 2024-02-15T18:27:00.876 INFO:tasks.workunit.client.0.smithi042.stdout:2/529: chown d0/d8/d20/d41 111010461 1 2024-02-15T18:27:00.877 INFO:tasks.workunit.client.1.smithi191.stdout:1/255: rename d0/db to d0/d2f/d51 0 2024-02-15T18:27:00.877 INFO:tasks.workunit.client.1.smithi191.stdout:1/256: stat d0/d8/df/d2e/l46 0 2024-02-15T18:27:00.877 INFO:tasks.workunit.client.1.smithi191.stdout:1/257: chown d0/f9 760578 1 2024-02-15T18:27:00.884 INFO:tasks.workunit.client.1.smithi191.stdout:4/275: symlink d2/d5/d14/l74 0 2024-02-15T18:27:00.884 INFO:tasks.workunit.client.1.smithi191.stdout:4/276: stat d2/d5/d14/d44/d30/f68 0 2024-02-15T18:27:00.889 INFO:tasks.workunit.client.0.smithi042.stdout:6/650: symlink d9/d11/d19/d35/d49/d79/le1 0 2024-02-15T18:27:00.889 INFO:tasks.workunit.client.1.smithi191.stdout:2/203: mkdir df/d12/d1f/d42 0 2024-02-15T18:27:00.892 INFO:tasks.workunit.client.0.smithi042.stdout:8/514: mkdir dd/de/d33/d3b/d92 0 2024-02-15T18:27:00.893 INFO:tasks.workunit.client.1.smithi191.stdout:3/166: creat d6/d3b/d30/f3c x:0 0 0 2024-02-15T18:27:00.893 INFO:tasks.workunit.client.1.smithi191.stdout:3/167: dread - d6/d3b/d30/f3c zero size 2024-02-15T18:27:00.893 INFO:tasks.workunit.client.1.smithi191.stdout:3/168: dread - d6/d1b/f39 zero size 2024-02-15T18:27:00.894 INFO:tasks.workunit.client.0.smithi042.stdout:1/579: dread de/d54/f5e [0,4194304] 0 2024-02-15T18:27:00.896 INFO:tasks.workunit.client.1.smithi191.stdout:1/258: unlink d0/c3c 0 2024-02-15T18:27:00.897 INFO:tasks.workunit.client.0.smithi042.stdout:2/530: symlink d0/d8/d10/d31/d3a/la0 0 2024-02-15T18:27:00.898 INFO:tasks.workunit.client.0.smithi042.stdout:2/531: chown d0/d8 45536638 1 2024-02-15T18:27:00.898 INFO:tasks.workunit.client.0.smithi042.stdout:2/532: chown d0/d8/db/de/f4d 786452027 1 2024-02-15T18:27:00.898 INFO:tasks.workunit.client.1.smithi191.stdout:4/277: creat d2/d5/d7/d2f/f75 x:0 0 0 2024-02-15T18:27:00.898 INFO:tasks.workunit.client.1.smithi191.stdout:4/278: write d2/d5/d14/f58 [1882955,24850] 0 2024-02-15T18:27:00.899 INFO:tasks.workunit.client.0.smithi042.stdout:6/651: creat d9/d11/d19/d35/fe2 x:0 0 0 2024-02-15T18:27:00.902 INFO:tasks.workunit.client.1.smithi191.stdout:1/259: creat d0/d8/df/d12/d2a/f52 x:0 0 0 2024-02-15T18:27:00.902 INFO:tasks.workunit.client.1.smithi191.stdout:3/169: write d6/d3b/ff [3791322,46108] 0 2024-02-15T18:27:00.903 INFO:tasks.workunit.client.0.smithi042.stdout:7/611: dwrite d10/f73 [0,4194304] 0 2024-02-15T18:27:00.903 INFO:tasks.workunit.client.0.smithi042.stdout:7/612: write d10/d12/d23/fa1 [755777,45967] 0 2024-02-15T18:27:00.903 INFO:tasks.workunit.client.0.smithi042.stdout:7/613: dread - d10/d43/d57/f79 zero size 2024-02-15T18:27:00.903 INFO:tasks.workunit.client.0.smithi042.stdout:2/533: rename d0/f49 to d0/d8/d10/d31/fa1 0 2024-02-15T18:27:00.903 INFO:tasks.workunit.client.0.smithi042.stdout:2/534: readlink d0/d8/db/l4f 0 2024-02-15T18:27:00.903 INFO:tasks.workunit.client.0.smithi042.stdout:2/535: write d0/d8/d10/d31/d3a/d53/f5d [4895691,40722] 0 2024-02-15T18:27:00.904 INFO:tasks.workunit.client.0.smithi042.stdout:5/612: dwrite d0/d14/f87 [0,4194304] 0 2024-02-15T18:27:00.905 INFO:tasks.workunit.client.1.smithi191.stdout:6/202: unlink d8/d19/l1a 0 2024-02-15T18:27:00.906 INFO:tasks.workunit.client.0.smithi042.stdout:0/544: dwrite d4/d38/f49 [0,4194304] 0 2024-02-15T18:27:00.907 INFO:tasks.workunit.client.0.smithi042.stdout:8/515: write dd/de/d33/d66/f52 [473732,114216] 0 2024-02-15T18:27:00.908 INFO:tasks.workunit.client.1.smithi191.stdout:1/260: creat d0/d2f/d51/f53 x:0 0 0 2024-02-15T18:27:00.908 INFO:tasks.workunit.client.1.smithi191.stdout:1/261: dread - d0/d8/df/d12/d2a/d2b/f37 zero size 2024-02-15T18:27:00.909 INFO:tasks.workunit.client.1.smithi191.stdout:7/198: dwrite d0/f14 [8388608,4194304] 0 2024-02-15T18:27:00.909 INFO:tasks.workunit.client.0.smithi042.stdout:6/652: creat d9/d11/d19/d35/dc4/fe3 x:0 0 0 2024-02-15T18:27:00.909 INFO:tasks.workunit.client.0.smithi042.stdout:7/614: mknod d10/d15/d39/d6c/d92/cd1 0 2024-02-15T18:27:00.911 INFO:tasks.workunit.client.0.smithi042.stdout:2/536: creat d0/d8/d10/fa2 x:0 0 0 2024-02-15T18:27:00.911 INFO:tasks.workunit.client.1.smithi191.stdout:6/203: symlink d8/d9/d2b/l34 0 2024-02-15T18:27:00.913 INFO:tasks.workunit.client.1.smithi191.stdout:1/262: mkdir d0/d2f/d51/d54 0 2024-02-15T18:27:00.917 INFO:tasks.workunit.client.1.smithi191.stdout:7/199: mkdir d0/d1/d3/dc/d4b 0 2024-02-15T18:27:00.918 INFO:tasks.workunit.client.1.smithi191.stdout:6/204: truncate d8/d9/fc 2896239 0 2024-02-15T18:27:00.919 INFO:tasks.workunit.client.1.smithi191.stdout:1/263: getdents d0/d8/df 0 2024-02-15T18:27:00.920 INFO:tasks.workunit.client.0.smithi042.stdout:1/580: dread de/d15/d95/f99 [0,4194304] 0 2024-02-15T18:27:00.920 INFO:tasks.workunit.client.0.smithi042.stdout:1/581: readlink de/d15/d67/d89/la0 0 2024-02-15T18:27:00.920 INFO:tasks.workunit.client.0.smithi042.stdout:1/582: fdatasync de/d54/dbf/dd0/d7c/fb1 0 2024-02-15T18:27:00.921 INFO:tasks.workunit.client.1.smithi191.stdout:0/119: dwrite d7/dd/f15 [0,4194304] 0 2024-02-15T18:27:00.923 INFO:tasks.workunit.client.0.smithi042.stdout:9/597: dwrite d6/dd/d9/f72 [0,4194304] 0 2024-02-15T18:27:00.923 INFO:tasks.workunit.client.0.smithi042.stdout:9/598: readlink d6/da/d1d/d2d/d40/l53 0 2024-02-15T18:27:00.928 INFO:tasks.workunit.client.0.smithi042.stdout:8/516: creat dd/de/d3c/d4b/d67/d5a/f93 x:0 0 0 2024-02-15T18:27:00.930 INFO:tasks.workunit.client.0.smithi042.stdout:7/615: mkdir d10/d49/dd2 0 2024-02-15T18:27:00.931 INFO:tasks.workunit.client.1.smithi191.stdout:6/205: write f1 [5554179,110757] 0 2024-02-15T18:27:00.932 INFO:tasks.workunit.client.0.smithi042.stdout:6/653: creat d9/d3e/fe4 x:0 0 0 2024-02-15T18:27:00.936 INFO:tasks.workunit.client.0.smithi042.stdout:1/583: link de/d15/d1b/d44/f85 de/d15/d67/d46/fd7 0 2024-02-15T18:27:00.937 INFO:tasks.workunit.client.0.smithi042.stdout:0/545: link d4/l10 d4/d9/d8d/lbe 0 2024-02-15T18:27:00.943 INFO:tasks.workunit.client.0.smithi042.stdout:9/599: truncate d6/da/f25 1439177 0 2024-02-15T18:27:00.943 INFO:tasks.workunit.client.1.smithi191.stdout:4/279: dwrite d2/d5/d7/f6c [0,4194304] 0 2024-02-15T18:27:00.944 INFO:tasks.workunit.client.1.smithi191.stdout:4/280: readlink d2/d5/d7/d2f/d41/l55 0 2024-02-15T18:27:00.944 INFO:tasks.workunit.client.1.smithi191.stdout:4/281: truncate d2/d5/d7/d2f/f6b 384903 0 2024-02-15T18:27:00.944 INFO:tasks.workunit.client.1.smithi191.stdout:2/204: dwrite df/d12/d1f/f26 [0,4194304] 0 2024-02-15T18:27:00.944 INFO:tasks.workunit.client.0.smithi042.stdout:5/613: dwrite d0/d14/f28 [0,4194304] 0 2024-02-15T18:27:00.945 INFO:tasks.workunit.client.0.smithi042.stdout:6/654: truncate d9/d3e/f3b 2947215 0 2024-02-15T18:27:00.945 INFO:tasks.workunit.client.0.smithi042.stdout:8/517: dwrite fc [4194304,4194304] 0 2024-02-15T18:27:00.946 INFO:tasks.workunit.client.1.smithi191.stdout:4/282: read d2/d5/d14/d44/d30/f38 [903696,43188] 0 2024-02-15T18:27:00.946 INFO:tasks.workunit.client.1.smithi191.stdout:4/283: truncate d2/d5/d7/d3e/d42/d52/f62 1368325 0 2024-02-15T18:27:00.947 INFO:tasks.workunit.client.0.smithi042.stdout:7/616: dread d10/d12/f6b [0,4194304] 0 2024-02-15T18:27:00.951 INFO:tasks.workunit.client.0.smithi042.stdout:1/584: link de/d54/dbf/dd0/d7c/fb1 de/d6d/d9e/dcd/fd8 0 2024-02-15T18:27:00.951 INFO:tasks.workunit.client.0.smithi042.stdout:9/600: rmdir d6/da/d1d/d2d/d40 39 2024-02-15T18:27:00.952 INFO:tasks.workunit.client.1.smithi191.stdout:1/264: dwrite d0/d8/df/d12/f23 [0,4194304] 0 2024-02-15T18:27:00.956 INFO:tasks.workunit.client.1.smithi191.stdout:2/205: write df/f16 [1404683,67426] 0 2024-02-15T18:27:00.970 INFO:tasks.workunit.client.0.smithi042.stdout:5/614: creat d0/d12/d2a/fd7 x:0 0 0 2024-02-15T18:27:00.971 INFO:tasks.workunit.client.0.smithi042.stdout:5/615: chown d0/d14/d88/laa 4152564 1 2024-02-15T18:27:00.971 INFO:tasks.workunit.client.1.smithi191.stdout:7/200: truncate d0/d1/f21 628426 0 2024-02-15T18:27:00.973 INFO:tasks.workunit.client.0.smithi042.stdout:4/547: sync 2024-02-15T18:27:00.973 INFO:tasks.workunit.client.0.smithi042.stdout:4/548: truncate da/d11/d29/d4a/d4f/d90/fb2 101727 0 2024-02-15T18:27:00.974 INFO:tasks.workunit.client.1.smithi191.stdout:6/206: dread d8/d19/f1d [0,4194304] 0 2024-02-15T18:27:00.975 INFO:tasks.workunit.client.1.smithi191.stdout:4/284: dwrite d2/d5/f13 [4194304,4194304] 0 2024-02-15T18:27:00.975 INFO:tasks.workunit.client.1.smithi191.stdout:4/285: write d2/f5c [519390,659] 0 2024-02-15T18:27:00.976 INFO:tasks.workunit.client.0.smithi042.stdout:8/518: mkdir dd/de/d94 0 2024-02-15T18:27:00.976 INFO:tasks.workunit.client.0.smithi042.stdout:6/655: dwrite d9/f66 [0,4194304] 0 2024-02-15T18:27:00.976 INFO:tasks.workunit.client.0.smithi042.stdout:6/656: write d9/d11/d19/d1c/d31/d4f/fde [571712,115196] 0 2024-02-15T18:27:00.976 INFO:tasks.workunit.client.0.smithi042.stdout:7/617: mkdir d10/d15/d39/d6c/d92/dd3 0 2024-02-15T18:27:00.976 INFO:tasks.workunit.client.0.smithi042.stdout:8/519: dread dd/d11/f21 [0,4194304] 0 2024-02-15T18:27:00.977 INFO:tasks.workunit.client.0.smithi042.stdout:0/546: dwrite d4/d38/d5c/f61 [0,4194304] 0 2024-02-15T18:27:00.977 INFO:tasks.workunit.client.0.smithi042.stdout:0/547: truncate d4/d9/d5a/d60/f64 278525 0 2024-02-15T18:27:00.979 INFO:tasks.workunit.client.1.smithi191.stdout:1/265: link d0/d8/df/d12/f23 d0/d2f/d51/d54/f55 0 2024-02-15T18:27:00.982 INFO:tasks.workunit.client.1.smithi191.stdout:6/207: dread d8/f12 [0,4194304] 0 2024-02-15T18:27:00.982 INFO:tasks.workunit.client.1.smithi191.stdout:6/208: dread - d8/d9/f29 zero size 2024-02-15T18:27:00.982 INFO:tasks.workunit.client.1.smithi191.stdout:6/209: write d8/f30 [1282061,11119] 0 2024-02-15T18:27:00.984 INFO:tasks.workunit.client.1.smithi191.stdout:9/199: sync 2024-02-15T18:27:00.985 INFO:tasks.workunit.client.1.smithi191.stdout:5/111: sync 2024-02-15T18:27:00.985 INFO:tasks.workunit.client.0.smithi042.stdout:8/520: dread dd/de/d12/f68 [0,4194304] 0 2024-02-15T18:27:00.987 INFO:tasks.workunit.client.1.smithi191.stdout:2/206: creat df/d12/d1f/d42/f43 x:0 0 0 2024-02-15T18:27:00.989 INFO:tasks.workunit.client.0.smithi042.stdout:1/585: creat de/d54/dbf/dd0/d7c/fd9 x:0 0 0 2024-02-15T18:27:00.991 INFO:tasks.workunit.client.1.smithi191.stdout:8/321: sync 2024-02-15T18:27:00.991 INFO:tasks.workunit.client.1.smithi191.stdout:8/322: fdatasync d3/d2c/d1a/d23/d2e/d47/d5f/f50 0 2024-02-15T18:27:00.991 INFO:tasks.workunit.client.0.smithi042.stdout:9/601: link d6/da/d1d/d2d/d40/d48/l4b d6/da/lcd 0 2024-02-15T18:27:00.995 INFO:tasks.workunit.client.0.smithi042.stdout:5/616: rmdir d0/d89 39 2024-02-15T18:27:00.997 INFO:tasks.workunit.client.1.smithi191.stdout:4/286: creat d2/d5/d64/f76 x:0 0 0 2024-02-15T18:27:01.000 INFO:tasks.workunit.client.0.smithi042.stdout:4/549: rename da/d11/d29/d56/f97 to da/d11/d67/d5c/d66/d78/d2f/d4d/fc8 0 2024-02-15T18:27:01.000 INFO:tasks.workunit.client.0.smithi042.stdout:4/550: fdatasync da/f9c 0 2024-02-15T18:27:01.002 INFO:tasks.workunit.client.1.smithi191.stdout:1/266: symlink d0/d8/df/d12/l56 0 2024-02-15T18:27:01.002 INFO:tasks.workunit.client.1.smithi191.stdout:1/267: stat d0/d8/df/d12/d2a/c45 0 2024-02-15T18:27:01.007 INFO:tasks.workunit.client.0.smithi042.stdout:6/657: creat d9/d11/d97/fe5 x:0 0 0 2024-02-15T18:27:01.009 INFO:tasks.workunit.client.1.smithi191.stdout:2/207: creat df/d12/d19/d1a/f44 x:0 0 0 2024-02-15T18:27:01.010 INFO:tasks.workunit.client.0.smithi042.stdout:7/618: symlink d10/d49/d6d/d6e/db3/ld4 0 2024-02-15T18:27:01.010 INFO:tasks.workunit.client.1.smithi191.stdout:7/201: dwrite d0/d1/d3/dc/f39 [0,4194304] 0 2024-02-15T18:27:01.013 INFO:tasks.workunit.client.1.smithi191.stdout:4/287: creat d2/d5/d7/d3e/f77 x:0 0 0 2024-02-15T18:27:01.013 INFO:tasks.workunit.client.1.smithi191.stdout:4/288: write d2/f34 [705791,47516] 0 2024-02-15T18:27:01.014 INFO:tasks.workunit.client.1.smithi191.stdout:0/120: dwrite d7/dd/f15 [4194304,4194304] 0 2024-02-15T18:27:01.014 INFO:tasks.workunit.client.1.smithi191.stdout:1/268: unlink d0/c16 0 2024-02-15T18:27:01.014 INFO:tasks.workunit.client.1.smithi191.stdout:1/269: fsync d0/d8/df/d12/d2a/f52 0 2024-02-15T18:27:01.014 INFO:tasks.workunit.client.0.smithi042.stdout:4/551: dread da/d3b/f81 [0,4194304] 0 2024-02-15T18:27:01.015 INFO:tasks.workunit.client.1.smithi191.stdout:9/200: dwrite d0/d7/dc/d17/f25 [0,4194304] 0 2024-02-15T18:27:01.016 INFO:tasks.workunit.client.0.smithi042.stdout:4/552: dread da/d3b/d41/d8d/f93 [0,4194304] 0 2024-02-15T18:27:01.016 INFO:tasks.workunit.client.0.smithi042.stdout:4/553: read - da/d2b/d7e/fc7 zero size 2024-02-15T18:27:01.017 INFO:tasks.workunit.client.1.smithi191.stdout:6/210: rename d8/d9/fc to d8/d19/f35 0 2024-02-15T18:27:01.020 INFO:tasks.workunit.client.0.smithi042.stdout:4/554: dread da/fa7 [0,4194304] 0 2024-02-15T18:27:01.021 INFO:tasks.workunit.client.1.smithi191.stdout:2/208: dread df/d12/f2b [0,4194304] 0 2024-02-15T18:27:01.022 INFO:tasks.workunit.client.0.smithi042.stdout:0/548: symlink d4/d9/d5a/db5/lbf 0 2024-02-15T18:27:01.022 INFO:tasks.workunit.client.0.smithi042.stdout:0/549: write d4/d38/d77/d2e/f32 [4605442,90563] 0 2024-02-15T18:27:01.022 INFO:tasks.workunit.client.0.smithi042.stdout:0/550: write d4/d9/d62/fa0 [4924726,15453] 0 2024-02-15T18:27:01.022 INFO:tasks.workunit.client.1.smithi191.stdout:2/209: write df/d12/f2b [3988760,70276] 0 2024-02-15T18:27:01.024 INFO:tasks.workunit.client.0.smithi042.stdout:5/617: dwrite d0/d38/d53/d68/f6e [0,4194304] 0 2024-02-15T18:27:01.026 INFO:tasks.workunit.client.0.smithi042.stdout:5/618: write d0/d12/f21 [567457,107306] 0 2024-02-15T18:27:01.028 INFO:tasks.workunit.client.0.smithi042.stdout:1/586: creat de/d54/d9a/dc8/fda x:0 0 0 2024-02-15T18:27:01.028 INFO:tasks.workunit.client.0.smithi042.stdout:1/587: dread - de/d54/dbf/dd0/d7c/f9c zero size 2024-02-15T18:27:01.031 INFO:tasks.workunit.client.1.smithi191.stdout:6/211: dread d8/f15 [0,4194304] 0 2024-02-15T18:27:01.033 INFO:tasks.workunit.client.0.smithi042.stdout:9/602: creat d6/da/d2f/d54/d96/fce x:0 0 0 2024-02-15T18:27:01.035 INFO:tasks.workunit.client.1.smithi191.stdout:5/112: dwrite d0/f1b [0,4194304] 0 2024-02-15T18:27:01.039 INFO:tasks.workunit.client.0.smithi042.stdout:6/658: truncate d9/f40 2216705 0 2024-02-15T18:27:01.040 INFO:tasks.workunit.client.1.smithi191.stdout:0/121: truncate d7/dd/f11 557664 0 2024-02-15T18:27:01.040 INFO:tasks.workunit.client.1.smithi191.stdout:0/122: getdents d7/da/d1a 0 2024-02-15T18:27:01.040 INFO:tasks.workunit.client.0.smithi042.stdout:7/619: truncate d10/f53 1726634 0 2024-02-15T18:27:01.042 INFO:tasks.workunit.client.1.smithi191.stdout:7/202: dwrite d0/d1/d35/f48 [0,4194304] 0 2024-02-15T18:27:01.044 INFO:tasks.workunit.client.0.smithi042.stdout:4/555: mkdir da/d3a/dc9 0 2024-02-15T18:27:01.044 INFO:tasks.workunit.client.1.smithi191.stdout:1/270: link d0/d8/df/d12/f23 d0/d8/f57 0 2024-02-15T18:27:01.044 INFO:tasks.workunit.client.1.smithi191.stdout:1/271: dread - d0/d8/df/d12/d2a/f52 zero size 2024-02-15T18:27:01.044 INFO:tasks.workunit.client.1.smithi191.stdout:1/272: write d0/d8/df/d12/f32 [4563490,57371] 0 2024-02-15T18:27:01.044 INFO:tasks.workunit.client.1.smithi191.stdout:1/273: chown d0/f5 12461 1 2024-02-15T18:27:01.044 INFO:tasks.workunit.client.1.smithi191.stdout:2/210: dwrite df/d12/d19/d1a/f44 [0,4194304] 0 2024-02-15T18:27:01.047 INFO:tasks.workunit.client.1.smithi191.stdout:9/201: creat d0/d7/dc/d17/d33/f4d x:0 0 0 2024-02-15T18:27:01.053 INFO:tasks.workunit.client.1.smithi191.stdout:6/212: dwrite d8/f12 [8388608,4194304] 0 2024-02-15T18:27:01.053 INFO:tasks.workunit.client.1.smithi191.stdout:4/289: rename d2/d5/d14/d44/d30 to d2/d5/d14/d47/d78 0 2024-02-15T18:27:01.053 INFO:tasks.workunit.client.1.smithi191.stdout:4/290: write d2/d5/d7/d3e/d42/d52/f53 [192280,72644] 0 2024-02-15T18:27:01.053 INFO:tasks.workunit.client.1.smithi191.stdout:4/291: write d2/f28 [2370549,36384] 0 2024-02-15T18:27:01.053 INFO:tasks.workunit.client.1.smithi191.stdout:4/292: truncate d2/d5/d7/d2f/f31 1070304 0 2024-02-15T18:27:01.053 INFO:tasks.workunit.client.1.smithi191.stdout:4/293: write d2/d5/d14/f2e [819956,14193] 0 2024-02-15T18:27:01.054 INFO:tasks.workunit.client.1.smithi191.stdout:1/274: dread d0/d2f/d51/d54/f55 [0,4194304] 0 2024-02-15T18:27:01.054 INFO:tasks.workunit.client.1.smithi191.stdout:1/275: truncate d0/d2f/d51/f53 445257 0 2024-02-15T18:27:01.056 INFO:tasks.workunit.client.1.smithi191.stdout:0/123: dwrite d7/fb [4194304,4194304] 0 2024-02-15T18:27:01.057 INFO:tasks.workunit.client.1.smithi191.stdout:4/294: fdatasync d2/f28 0 2024-02-15T18:27:01.057 INFO:tasks.workunit.client.0.smithi042.stdout:0/551: mknod d4/d9/cc0 0 2024-02-15T18:27:01.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:26:59.239034+0000 mgr.smithi042.kfoihb (mgr.24435) 38 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:27:01.062 INFO:tasks.workunit.client.0.smithi042.stdout:5/619: mkdir d0/db/dd8 0 2024-02-15T18:27:01.062 INFO:tasks.workunit.client.1.smithi191.stdout:5/113: chown d0/d23 6190 1 2024-02-15T18:27:01.064 INFO:tasks.workunit.client.0.smithi042.stdout:1/588: truncate de/d15/d2c/d51/d70/f78 6737532 0 2024-02-15T18:27:01.064 INFO:tasks.workunit.client.0.smithi042.stdout:1/589: readlink de/d15/d2c/l3c 0 2024-02-15T18:27:01.065 INFO:tasks.workunit.client.0.smithi042.stdout:9/603: mkdir d6/da/d2f/d55/d73/d8d/dcf 0 2024-02-15T18:27:01.068 INFO:tasks.workunit.client.1.smithi191.stdout:7/203: creat d0/d1/d1d/f4c x:0 0 0 2024-02-15T18:27:01.068 INFO:tasks.workunit.client.1.smithi191.stdout:7/204: stat d0/d1/d35/d42 0 2024-02-15T18:27:01.068 INFO:tasks.workunit.client.1.smithi191.stdout:7/205: chown d0/d1/d35/d42/d46/c34 144820 1 2024-02-15T18:27:01.068 INFO:tasks.workunit.client.1.smithi191.stdout:7/206: chown d0/d1/l2b 2 1 2024-02-15T18:27:01.068 INFO:tasks.workunit.client.1.smithi191.stdout:7/207: readlink d0/d1/l2b 0 2024-02-15T18:27:01.068 INFO:tasks.workunit.client.1.smithi191.stdout:7/208: read - d0/d1/d3/d11/f18 zero size 2024-02-15T18:27:01.069 INFO:tasks.workunit.client.1.smithi191.stdout:7/209: rename d0/d1/d3/dc/d4b to d0/d1/d3/dc/d4b/d4d 22 2024-02-15T18:27:01.069 INFO:tasks.workunit.client.1.smithi191.stdout:7/210: write d0/d1/d1d/f4c [875591,6234] 0 2024-02-15T18:27:01.069 INFO:tasks.workunit.client.0.smithi042.stdout:6/659: rmdir d9/d11/d97/dd8 0 2024-02-15T18:27:01.069 INFO:tasks.workunit.client.0.smithi042.stdout:6/660: write d9/d11/d19/d1c/d31/d53/fc2 [135006,127500] 0 2024-02-15T18:27:01.071 INFO:tasks.workunit.client.0.smithi042.stdout:3/575: sync 2024-02-15T18:27:01.075 INFO:tasks.workunit.client.0.smithi042.stdout:7/620: truncate d10/f7b 3239343 0 2024-02-15T18:27:01.078 INFO:tasks.workunit.client.0.smithi042.stdout:4/556: unlink da/d11/d67/d5c/d66/lc5 0 2024-02-15T18:27:01.078 INFO:tasks.workunit.client.1.smithi191.stdout:2/211: creat df/d12/d1f/f45 x:0 0 0 2024-02-15T18:27:01.084 INFO:tasks.workunit.client.0.smithi042.stdout:0/552: mkdir d4/d9/d34/d57/dc1 0 2024-02-15T18:27:01.084 INFO:tasks.workunit.client.0.smithi042.stdout:9/604: dwrite d6/da/d2f/f9e [4194304,4194304] 0 2024-02-15T18:27:01.091 INFO:tasks.workunit.client.0.smithi042.stdout:2/537: sync 2024-02-15T18:27:01.092 INFO:tasks.workunit.client.0.smithi042.stdout:5/620: rename d0/d12/d65/l8b to d0/d22/d7b/ld9 0 2024-02-15T18:27:01.093 INFO:tasks.workunit.client.0.smithi042.stdout:1/590: creat de/d54/d9a/dc8/fdb x:0 0 0 2024-02-15T18:27:01.094 INFO:tasks.workunit.client.0.smithi042.stdout:6/661: mknod d9/d11/d97/ce6 0 2024-02-15T18:27:01.095 INFO:tasks.workunit.client.0.smithi042.stdout:4/557: mkdir da/d11/d67/d5c/d66/d78/d2f/dca 0 2024-02-15T18:27:01.099 INFO:tasks.workunit.client.1.smithi191.stdout:9/202: creat d0/d7/dc/d30/f4e x:0 0 0 2024-02-15T18:27:01.099 INFO:tasks.workunit.client.0.smithi042.stdout:9/605: mknod d6/cd0 0 2024-02-15T18:27:01.100 INFO:tasks.workunit.client.0.smithi042.stdout:3/576: dwrite d1/d11/d14/d4d/d6c/d7b/fb5 [0,4194304] 0 2024-02-15T18:27:01.101 INFO:tasks.workunit.client.1.smithi191.stdout:6/213: creat d8/d9/f36 x:0 0 0 2024-02-15T18:27:01.103 INFO:tasks.workunit.client.1.smithi191.stdout:1/276: getdents d0/d2f/d51 0 2024-02-15T18:27:01.103 INFO:tasks.workunit.client.1.smithi191.stdout:1/277: chown d0/d8/f17 423798731 1 2024-02-15T18:27:01.108 INFO:tasks.workunit.client.1.smithi191.stdout:4/295: mknod d2/d5/d7/c79 0 2024-02-15T18:27:01.108 INFO:tasks.workunit.client.1.smithi191.stdout:4/296: readlink d2/d5/l10 0 2024-02-15T18:27:01.108 INFO:tasks.workunit.client.1.smithi191.stdout:4/297: stat d2/d5/d14/d47/d78/f68 0 2024-02-15T18:27:01.109 INFO:tasks.workunit.client.1.smithi191.stdout:0/124: rename d7/da/f14 to d7/da/f20 0 2024-02-15T18:27:01.109 INFO:tasks.workunit.client.1.smithi191.stdout:0/125: write d7/da/f18 [684608,114077] 0 2024-02-15T18:27:01.110 INFO:tasks.workunit.client.0.smithi042.stdout:9/606: mkdir d6/dd/dd1 0 2024-02-15T18:27:01.110 INFO:tasks.workunit.client.0.smithi042.stdout:7/621: dwrite d10/d49/fa9 [0,4194304] 0 2024-02-15T18:27:01.110 INFO:tasks.workunit.client.0.smithi042.stdout:5/621: dread d0/d38/d53/d68/f6e [0,4194304] 0 2024-02-15T18:27:01.110 INFO:tasks.workunit.client.0.smithi042.stdout:7/622: write d10/d12/d23/d51/f99 [968686,16098] 0 2024-02-15T18:27:01.112 INFO:tasks.workunit.client.1.smithi191.stdout:1/278: write d0/d8/f35 [1162054,78270] 0 2024-02-15T18:27:01.115 INFO:tasks.workunit.client.0.smithi042.stdout:8/521: sync 2024-02-15T18:27:01.118 INFO:tasks.workunit.client.1.smithi191.stdout:7/211: rmdir d0/d1/d1d 39 2024-02-15T18:27:01.118 INFO:tasks.workunit.client.1.smithi191.stdout:7/212: write d0/db/f2e [1325105,44033] 0 2024-02-15T18:27:01.118 INFO:tasks.workunit.client.1.smithi191.stdout:7/213: dread - d0/d1/d35/d42/d46/f32 zero size 2024-02-15T18:27:01.118 INFO:tasks.workunit.client.1.smithi191.stdout:7/214: chown d0/d1/d35 258 1 2024-02-15T18:27:01.118 INFO:tasks.workunit.client.1.smithi191.stdout:7/215: chown d0/d1/d35/f48 47 1 2024-02-15T18:27:01.120 INFO:tasks.workunit.client.0.smithi042.stdout:6/662: dwrite d9/d3e/f64 [0,4194304] 0 2024-02-15T18:27:01.120 INFO:tasks.workunit.client.0.smithi042.stdout:8/522: write dd/de/ff [433237,50225] 0 2024-02-15T18:27:01.120 INFO:tasks.workunit.client.1.smithi191.stdout:2/212: rename df/d12/d19/d1a/d35/c3c to df/d12/d1e/c46 0 2024-02-15T18:27:01.121 INFO:tasks.workunit.client.0.smithi042.stdout:4/558: dwrite da/d11/d67/fc6 [0,4194304] 0 2024-02-15T18:27:01.121 INFO:tasks.workunit.client.0.smithi042.stdout:4/559: truncate da/d11/d27/f86 1303454 0 2024-02-15T18:27:01.121 INFO:tasks.workunit.client.0.smithi042.stdout:4/560: chown da/d2b/lb3 117685 1 2024-02-15T18:27:01.122 INFO:tasks.workunit.client.1.smithi191.stdout:9/203: mkdir d0/d7/dc/d30/d4f 0 2024-02-15T18:27:01.122 INFO:tasks.workunit.client.0.smithi042.stdout:3/577: symlink d1/de/lc0 0 2024-02-15T18:27:01.123 INFO:tasks.workunit.client.0.smithi042.stdout:6/663: write d9/fb2 [3250447,48662] 0 2024-02-15T18:27:01.124 INFO:tasks.workunit.client.1.smithi191.stdout:4/298: symlink d2/d5/d14/d44/l7a 0 2024-02-15T18:27:01.124 INFO:tasks.workunit.client.1.smithi191.stdout:4/299: chown d2/d5/d7/d2f/f51 303 1 2024-02-15T18:27:01.127 INFO:tasks.workunit.client.0.smithi042.stdout:0/553: rename d4/d9/d5a/db5 to d4/dc2 0 2024-02-15T18:27:01.130 INFO:tasks.workunit.client.1.smithi191.stdout:0/126: symlink d7/d1b/l21 0 2024-02-15T18:27:01.132 INFO:tasks.workunit.client.0.smithi042.stdout:1/591: truncate de/d15/d1b/f25 3154167 0 2024-02-15T18:27:01.132 INFO:tasks.workunit.client.0.smithi042.stdout:1/592: readlink de/d15/d67/d46/d6b/l5d 0 2024-02-15T18:27:01.132 INFO:tasks.workunit.client.0.smithi042.stdout:1/593: write de/d15/d1b/f49 [5190466,123457] 0 2024-02-15T18:27:01.133 INFO:tasks.workunit.client.1.smithi191.stdout:6/214: dwrite f1 [0,4194304] 0 2024-02-15T18:27:01.134 INFO:tasks.workunit.client.1.smithi191.stdout:1/279: mknod d0/d8/c58 0 2024-02-15T18:27:01.136 INFO:tasks.workunit.client.0.smithi042.stdout:9/607: link d6/dd/d9/d59/fbe d6/dd/d9/d59/fd2 0 2024-02-15T18:27:01.137 INFO:tasks.workunit.client.1.smithi191.stdout:5/114: unlink d0/d14/d25/f26 0 2024-02-15T18:27:01.137 INFO:tasks.workunit.client.1.smithi191.stdout:1/280: write d0/d2f/d51/d10/f2d [3780965,14664] 0 2024-02-15T18:27:01.139 INFO:tasks.workunit.client.1.smithi191.stdout:7/216: symlink d0/d1/d3/l4e 0 2024-02-15T18:27:01.139 INFO:tasks.workunit.client.1.smithi191.stdout:6/215: write d8/f1c [1025751,38312] 0 2024-02-15T18:27:01.139 INFO:tasks.workunit.client.1.smithi191.stdout:6/216: chown d8/d9/c21 29 1 2024-02-15T18:27:01.139 INFO:tasks.workunit.client.0.smithi042.stdout:7/623: unlink d10/l9b 0 2024-02-15T18:27:01.140 INFO:tasks.workunit.client.1.smithi191.stdout:2/213: mkdir df/d12/d19/d1a/d35/d47 0 2024-02-15T18:27:01.140 INFO:tasks.workunit.client.1.smithi191.stdout:2/214: fsync df/d12/f2b 0 2024-02-15T18:27:01.140 INFO:tasks.workunit.client.1.smithi191.stdout:2/215: write df/f1b [800059,60169] 0 2024-02-15T18:27:01.142 INFO:tasks.workunit.client.1.smithi191.stdout:9/204: mknod d0/d7/dc/d30/d3b/c50 0 2024-02-15T18:27:01.143 INFO:tasks.workunit.client.1.smithi191.stdout:4/300: rmdir d2/d5/d7/d2f/d41 39 2024-02-15T18:27:01.143 INFO:tasks.workunit.client.0.smithi042.stdout:4/561: rmdir da/d11/d85 39 2024-02-15T18:27:01.145 INFO:tasks.workunit.client.0.smithi042.stdout:3/578: rmdir d1/d11/d14/d4d/d6c/d7b 39 2024-02-15T18:27:01.145 INFO:tasks.workunit.client.0.smithi042.stdout:3/579: write d1/fbe [1032264,45653] 0 2024-02-15T18:27:01.147 INFO:tasks.workunit.client.0.smithi042.stdout:6/664: mknod d9/d11/d19/d1c/d31/d4f/ce7 0 2024-02-15T18:27:01.149 INFO:tasks.workunit.client.0.smithi042.stdout:2/538: dwrite d0/d8/db/f80 [4194304,4194304] 0 2024-02-15T18:27:01.150 INFO:tasks.workunit.client.0.smithi042.stdout:2/539: write d0/d8/d10/d31/d3a/d53/f58 [2518392,52408] 0 2024-02-15T18:27:01.151 INFO:tasks.workunit.client.0.smithi042.stdout:8/523: dwrite dd/de/d12/f65 [0,4194304] 0 2024-02-15T18:27:01.152 INFO:tasks.workunit.client.0.smithi042.stdout:5/622: rename d0/d38/d53/d68/d6a to d0/dda 0 2024-02-15T18:27:01.156 INFO:tasks.workunit.client.1.smithi191.stdout:0/127: dread d7/fb [4194304,4194304] 0 2024-02-15T18:27:01.157 INFO:tasks.workunit.client.0.smithi042.stdout:7/624: dread fa [0,4194304] 0 2024-02-15T18:27:01.158 INFO:tasks.workunit.client.0.smithi042.stdout:0/554: symlink d4/d9/d62/d74/lc3 0 2024-02-15T18:27:01.160 INFO:tasks.workunit.client.0.smithi042.stdout:6/665: write d9/f21 [7844316,54604] 0 2024-02-15T18:27:01.160 INFO:tasks.workunit.client.0.smithi042.stdout:6/666: stat d9/d11/d19/d35/d55/f65 0 2024-02-15T18:27:01.164 INFO:tasks.workunit.client.0.smithi042.stdout:0/555: dread d4/d9/f31 [0,4194304] 0 2024-02-15T18:27:01.166 INFO:tasks.workunit.client.1.smithi191.stdout:5/115: creat d0/d23/f27 x:0 0 0 2024-02-15T18:27:01.166 INFO:tasks.workunit.client.0.smithi042.stdout:1/594: getdents de/d15/d67/d74 0 2024-02-15T18:27:01.167 INFO:tasks.workunit.client.1.smithi191.stdout:1/281: rmdir d0/d8/df/d12 39 2024-02-15T18:27:01.169 INFO:tasks.workunit.client.0.smithi042.stdout:4/562: mknod da/d11/d29/d4a/d4f/ccb 0 2024-02-15T18:27:01.171 INFO:tasks.workunit.client.0.smithi042.stdout:6/667: dread d9/d11/d19/d35/d55/d92/d9e/faa [4194304,4194304] 0 2024-02-15T18:27:01.172 INFO:tasks.workunit.client.1.smithi191.stdout:7/217: truncate d0/db/d26/f3c 2075235 0 2024-02-15T18:27:01.172 INFO:tasks.workunit.client.1.smithi191.stdout:7/218: truncate d0/d1/d3/d11/f18 304314 0 2024-02-15T18:27:01.173 INFO:tasks.workunit.client.0.smithi042.stdout:3/580: dwrite d1/d11/f66 [4194304,4194304] 0 2024-02-15T18:27:01.174 INFO:tasks.workunit.client.1.smithi191.stdout:2/216: mknod df/d12/d19/d1a/d35/d47/c48 0 2024-02-15T18:27:01.175 INFO:tasks.workunit.client.0.smithi042.stdout:9/608: dwrite d6/da/d1d/d2d/d40/f49 [0,4194304] 0 2024-02-15T18:27:01.176 INFO:tasks.workunit.client.1.smithi191.stdout:9/205: creat d0/f51 x:0 0 0 2024-02-15T18:27:01.176 INFO:tasks.workunit.client.1.smithi191.stdout:6/217: dwrite d8/d9/f29 [0,4194304] 0 2024-02-15T18:27:01.177 INFO:tasks.workunit.client.0.smithi042.stdout:2/540: creat d0/d40/d56/fa3 x:0 0 0 2024-02-15T18:27:01.178 INFO:tasks.workunit.client.1.smithi191.stdout:0/128: dwrite d7/dd/f15 [4194304,4194304] 0 2024-02-15T18:27:01.178 INFO:tasks.workunit.client.1.smithi191.stdout:0/129: write d7/da/f18 [862034,77016] 0 2024-02-15T18:27:01.178 INFO:tasks.workunit.client.1.smithi191.stdout:4/301: creat d2/d5/d7/d3e/d42/d52/f7b x:0 0 0 2024-02-15T18:27:01.185 INFO:tasks.workunit.client.0.smithi042.stdout:8/524: creat dd/d11/d75/f95 x:0 0 0 2024-02-15T18:27:01.185 INFO:tasks.workunit.client.0.smithi042.stdout:8/525: write dd/de/d3c/d4b/d67/f43 [420603,111754] 0 2024-02-15T18:27:01.187 INFO:tasks.workunit.client.0.smithi042.stdout:5/623: mkdir d0/d14/ddb 0 2024-02-15T18:27:01.187 INFO:tasks.workunit.client.0.smithi042.stdout:5/624: chown d0/d38/d53/d91/db0 78 1 2024-02-15T18:27:01.190 INFO:tasks.workunit.client.0.smithi042.stdout:7/625: rmdir d10/d12 39 2024-02-15T18:27:01.196 INFO:tasks.workunit.client.0.smithi042.stdout:6/668: rename d9/d11/d19/f93 to d9/d11/d19/d35/d55/d92/fe8 0 2024-02-15T18:27:01.197 INFO:tasks.workunit.client.0.smithi042.stdout:6/669: fsync d9/d11/d1d/fb0 0 2024-02-15T18:27:01.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:26:59.239034+0000 mgr.smithi042.kfoihb (mgr.24435) 38 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:27:01.199 INFO:tasks.workunit.client.0.smithi042.stdout:9/609: creat d6/da/d2f/d55/d73/d8d/fd3 x:0 0 0 2024-02-15T18:27:01.199 INFO:tasks.workunit.client.0.smithi042.stdout:9/610: chown d6/dd 31271 1 2024-02-15T18:27:01.200 INFO:tasks.workunit.client.0.smithi042.stdout:9/611: stat d6/da/d2f/d55/d73/d8d/fd3 0 2024-02-15T18:27:01.201 INFO:tasks.workunit.client.1.smithi191.stdout:5/116: stat d0/cd 0 2024-02-15T18:27:01.201 INFO:tasks.workunit.client.0.smithi042.stdout:2/541: rmdir d0/d8/d10/d7b 39 2024-02-15T18:27:01.202 INFO:tasks.workunit.client.0.smithi042.stdout:0/556: dread d4/f6 [0,4194304] 0 2024-02-15T18:27:01.202 INFO:tasks.workunit.client.0.smithi042.stdout:1/595: dwrite de/d15/f4c [0,4194304] 0 2024-02-15T18:27:01.202 INFO:tasks.workunit.client.0.smithi042.stdout:0/557: fdatasync d4/d9/d15/f8c 0 2024-02-15T18:27:01.202 INFO:tasks.workunit.client.0.smithi042.stdout:1/596: fdatasync de/d15/d2c/f4f 0 2024-02-15T18:27:01.202 INFO:tasks.workunit.client.0.smithi042.stdout:1/597: readlink de/d15/l22 0 2024-02-15T18:27:01.203 INFO:tasks.workunit.client.0.smithi042.stdout:8/526: symlink dd/de/d33/d3b/d92/l96 0 2024-02-15T18:27:01.204 INFO:tasks.workunit.client.0.smithi042.stdout:1/598: write de/d15/d95/f99 [3313685,42293] 0 2024-02-15T18:27:01.205 INFO:tasks.workunit.client.0.smithi042.stdout:1/599: dread de/fa8 [0,4194304] 0 2024-02-15T18:27:01.208 INFO:tasks.workunit.client.1.smithi191.stdout:1/282: link d0/d8/df/d12/d33/d49/f4a d0/d2f/d51/d10/f59 0 2024-02-15T18:27:01.208 INFO:tasks.workunit.client.1.smithi191.stdout:2/217: creat df/d12/d1f/d42/f49 x:0 0 0 2024-02-15T18:27:01.210 INFO:tasks.workunit.client.1.smithi191.stdout:9/206: truncate d0/d7/dc/d17/f25 1508486 0 2024-02-15T18:27:01.210 INFO:tasks.workunit.client.1.smithi191.stdout:9/207: stat d0/d29/f3d 0 2024-02-15T18:27:01.211 INFO:tasks.workunit.client.0.smithi042.stdout:6/670: creat d9/d11/d19/d1c/d31/d53/dbe/fe9 x:0 0 0 2024-02-15T18:27:01.211 INFO:tasks.workunit.client.1.smithi191.stdout:0/130: dwrite d7/dd/f15 [0,4194304] 0 2024-02-15T18:27:01.218 INFO:tasks.workunit.client.0.smithi042.stdout:0/558: truncate d4/d9/f22 3807861 0 2024-02-15T18:27:01.223 INFO:tasks.workunit.client.0.smithi042.stdout:4/563: dwrite da/d11/d29/f32 [0,4194304] 0 2024-02-15T18:27:01.224 INFO:tasks.workunit.client.1.smithi191.stdout:6/218: creat d8/d19/f37 x:0 0 0 2024-02-15T18:27:01.226 INFO:tasks.workunit.client.0.smithi042.stdout:3/581: dwrite d1/fa5 [0,4194304] 0 2024-02-15T18:27:01.226 INFO:tasks.workunit.client.0.smithi042.stdout:3/582: fsync d1/f7 0 2024-02-15T18:27:01.226 INFO:tasks.workunit.client.1.smithi191.stdout:5/117: mkdir d0/d14/d28 0 2024-02-15T18:27:01.226 INFO:tasks.workunit.client.1.smithi191.stdout:6/219: write f1 [1661417,67774] 0 2024-02-15T18:27:01.227 INFO:tasks.workunit.client.0.smithi042.stdout:5/625: mknod d0/d38/d53/d91/db0/cdc 0 2024-02-15T18:27:01.227 INFO:tasks.workunit.client.0.smithi042.stdout:5/626: fsync d0/dda/dc7/fcc 0 2024-02-15T18:27:01.228 INFO:tasks.workunit.client.0.smithi042.stdout:5/627: write d0/d38/d53/d91/d94/fc0 [885012,126788] 0 2024-02-15T18:27:01.228 INFO:tasks.workunit.client.1.smithi191.stdout:2/218: mknod df/d12/d1f/c4a 0 2024-02-15T18:27:01.233 INFO:tasks.workunit.client.0.smithi042.stdout:4/564: dread da/d11/d27/f86 [0,4194304] 0 2024-02-15T18:27:01.233 INFO:tasks.workunit.client.0.smithi042.stdout:4/565: readlink da/d11/d29/l5b 0 2024-02-15T18:27:01.236 INFO:tasks.workunit.client.0.smithi042.stdout:5/628: dread d0/dda/f56 [0,4194304] 0 2024-02-15T18:27:01.236 INFO:tasks.workunit.client.0.smithi042.stdout:5/629: stat d0/d38/d53/d91 0 2024-02-15T18:27:01.240 INFO:tasks.workunit.client.0.smithi042.stdout:6/671: rename d9/d11/d19/d35/f4e to d9/d11/d19/d35/d49/fea 0 2024-02-15T18:27:01.240 INFO:tasks.workunit.client.1.smithi191.stdout:9/208: mknod d0/d7/dc/d30/d3b/c52 0 2024-02-15T18:27:01.243 INFO:tasks.workunit.client.1.smithi191.stdout:0/131: mkdir d7/d22 0 2024-02-15T18:27:01.247 INFO:tasks.workunit.client.0.smithi042.stdout:0/559: dread d4/d38/f9f [0,4194304] 0 2024-02-15T18:27:01.247 INFO:tasks.workunit.client.0.smithi042.stdout:0/560: write d4/d9/d34/d57/f7f [509588,118827] 0 2024-02-15T18:27:01.248 INFO:tasks.workunit.client.1.smithi191.stdout:5/118: read d0/f13 [2569012,19719] 0 2024-02-15T18:27:01.249 INFO:tasks.workunit.client.1.smithi191.stdout:2/219: mkdir df/d12/d19/d1a/d35/d47/d4b 0 2024-02-15T18:27:01.250 INFO:tasks.workunit.client.0.smithi042.stdout:2/542: dwrite d0/d8/d20/d7d/d94/f60 [0,4194304] 0 2024-02-15T18:27:01.250 INFO:tasks.workunit.client.0.smithi042.stdout:2/543: chown d0/d8/db/fd 5894 1 2024-02-15T18:27:01.250 INFO:tasks.workunit.client.0.smithi042.stdout:2/544: truncate d0/d8/f70 924021 0 2024-02-15T18:27:01.250 INFO:tasks.workunit.client.0.smithi042.stdout:2/545: fdatasync d0/d8/db/de/d1a/d2e/f92 0 2024-02-15T18:27:01.250 INFO:tasks.workunit.client.0.smithi042.stdout:2/546: getdents d0/d35/d9e 0 2024-02-15T18:27:01.250 INFO:tasks.workunit.client.0.smithi042.stdout:2/547: chown d0/d8/d10/d5a 0 1 2024-02-15T18:27:01.251 INFO:tasks.workunit.client.0.smithi042.stdout:8/527: dwrite dd/de/d3c/f54 [0,4194304] 0 2024-02-15T18:27:01.251 INFO:tasks.workunit.client.0.smithi042.stdout:8/528: write dd/de/d3c/d4b/d67/f6e [559603,68834] 0 2024-02-15T18:27:01.251 INFO:tasks.workunit.client.0.smithi042.stdout:8/529: write dd/de/d3c/d4b/d81/f88 [984021,106701] 0 2024-02-15T18:27:01.252 INFO:tasks.workunit.client.1.smithi191.stdout:7/219: write d0/db/d26/f3c [1736598,106384] 0 2024-02-15T18:27:01.252 INFO:tasks.workunit.client.1.smithi191.stdout:1/283: dwrite d0/d8/df/f2c [0,4194304] 0 2024-02-15T18:27:01.252 INFO:tasks.workunit.client.0.smithi042.stdout:1/600: dwrite de/d54/fb0 [0,4194304] 0 2024-02-15T18:27:01.252 INFO:tasks.workunit.client.0.smithi042.stdout:9/612: dwrite d6/dd/d9/f9f [0,4194304] 0 2024-02-15T18:27:01.252 INFO:tasks.workunit.client.0.smithi042.stdout:9/613: dread - d6/da/d2f/d55/d73/d8d/fb2 zero size 2024-02-15T18:27:01.253 INFO:tasks.workunit.client.0.smithi042.stdout:9/614: chown d6/dd/d9/da1 6 1 2024-02-15T18:27:01.253 INFO:tasks.workunit.client.1.smithi191.stdout:0/132: unlink d7/c1e 0 2024-02-15T18:27:01.254 INFO:tasks.workunit.client.1.smithi191.stdout:0/133: write d7/fb [4768110,85995] 0 2024-02-15T18:27:01.254 INFO:tasks.workunit.client.1.smithi191.stdout:6/220: rename d8/d19/f22 to d8/d19/f38 0 2024-02-15T18:27:01.258 INFO:tasks.workunit.client.0.smithi042.stdout:4/566: mknod da/d2b/d6e/ccc 0 2024-02-15T18:27:01.259 INFO:tasks.workunit.client.1.smithi191.stdout:5/119: readlink d0/l10 0 2024-02-15T18:27:01.260 INFO:tasks.workunit.client.1.smithi191.stdout:7/220: symlink d0/d1/d35/l4f 0 2024-02-15T18:27:01.262 INFO:tasks.workunit.client.0.smithi042.stdout:3/583: rename d1/d11/d14/lae to d1/d11/d14/dbf/lc1 0 2024-02-15T18:27:01.262 INFO:tasks.workunit.client.0.smithi042.stdout:3/584: dread - d1/d11/d14/d4d/d73/d89/fa7 zero size 2024-02-15T18:27:01.263 INFO:tasks.workunit.client.1.smithi191.stdout:0/134: rename d7/f13 to d7/d1b/f23 0 2024-02-15T18:27:01.266 INFO:tasks.workunit.client.0.smithi042.stdout:6/672: symlink d9/d11/d19/d1c/leb 0 2024-02-15T18:27:01.269 INFO:tasks.workunit.client.0.smithi042.stdout:5/630: dwrite d0/d38/d53/f7a [0,4194304] 0 2024-02-15T18:27:01.271 INFO:tasks.workunit.client.1.smithi191.stdout:7/221: creat d0/d1/d35/f50 x:0 0 0 2024-02-15T18:27:01.271 INFO:tasks.workunit.client.0.smithi042.stdout:5/631: read d0/f4d [564983,5402] 0 2024-02-15T18:27:01.272 INFO:tasks.workunit.client.1.smithi191.stdout:0/135: symlink d7/d22/l24 0 2024-02-15T18:27:01.274 INFO:tasks.workunit.client.1.smithi191.stdout:2/220: dwrite df/d12/f22 [0,4194304] 0 2024-02-15T18:27:01.274 INFO:tasks.workunit.client.1.smithi191.stdout:6/221: symlink d8/d9/l39 0 2024-02-15T18:27:01.275 INFO:tasks.workunit.client.0.smithi042.stdout:5/632: write d0/fe [2598885,108242] 0 2024-02-15T18:27:01.275 INFO:tasks.workunit.client.1.smithi191.stdout:5/120: getdents d0/d18 0 2024-02-15T18:27:01.276 INFO:tasks.workunit.client.1.smithi191.stdout:1/284: dwrite d0/d8/df/d12/f3f [0,4194304] 0 2024-02-15T18:27:01.277 INFO:tasks.workunit.client.1.smithi191.stdout:9/209: dwrite d0/f1 [4194304,4194304] 0 2024-02-15T18:27:01.288 INFO:tasks.workunit.client.1.smithi191.stdout:7/222: unlink d0/d1/f2a 0 2024-02-15T18:27:01.292 INFO:tasks.workunit.client.1.smithi191.stdout:1/285: dwrite d0/d2f/d51/d10/f25 [4194304,4194304] 0 2024-02-15T18:27:01.294 INFO:tasks.workunit.client.0.smithi042.stdout:0/561: creat d4/d38/d5c/fc4 x:0 0 0 2024-02-15T18:27:01.294 INFO:tasks.workunit.client.0.smithi042.stdout:0/562: truncate d4/d9/d5a/d60/f64 1316602 0 2024-02-15T18:27:01.294 INFO:tasks.workunit.client.0.smithi042.stdout:0/563: write d4/d9/d62/fa6 [134418,31084] 0 2024-02-15T18:27:01.295 INFO:tasks.workunit.client.1.smithi191.stdout:0/136: creat d7/da/f25 x:0 0 0 2024-02-15T18:27:01.296 INFO:tasks.workunit.client.1.smithi191.stdout:0/137: dread d7/da/f20 [0,4194304] 0 2024-02-15T18:27:01.297 INFO:tasks.workunit.client.0.smithi042.stdout:2/548: creat d0/d8/d10/d7b/fa4 x:0 0 0 2024-02-15T18:27:01.299 INFO:tasks.workunit.client.1.smithi191.stdout:2/221: mkdir df/d12/d19/d1a/d35/d47/d4b/d4c 0 2024-02-15T18:27:01.299 INFO:tasks.workunit.client.1.smithi191.stdout:0/138: write d7/dd/f15 [5721392,80181] 0 2024-02-15T18:27:01.301 INFO:tasks.workunit.client.0.smithi042.stdout:8/530: mknod dd/de/d33/d66/c97 0 2024-02-15T18:27:01.302 INFO:tasks.workunit.client.1.smithi191.stdout:0/139: write d7/dd/f15 [6834351,126924] 0 2024-02-15T18:27:01.302 INFO:tasks.workunit.client.1.smithi191.stdout:5/121: fdatasync d0/f1a 0 2024-02-15T18:27:01.307 INFO:tasks.workunit.client.1.smithi191.stdout:2/222: dread f7 [0,4194304] 0 2024-02-15T18:27:01.307 INFO:tasks.workunit.client.1.smithi191.stdout:2/223: write df/d12/d1f/d42/f49 [474629,107826] 0 2024-02-15T18:27:01.308 INFO:tasks.workunit.client.1.smithi191.stdout:7/223: dwrite d0/d1/d3/f3e [0,4194304] 0 2024-02-15T18:27:01.311 INFO:tasks.workunit.client.0.smithi042.stdout:1/601: creat de/d15/d95/fdc x:0 0 0 2024-02-15T18:27:01.311 INFO:tasks.workunit.client.0.smithi042.stdout:1/602: write de/d54/d9a/dc8/fda [907383,95949] 0 2024-02-15T18:27:01.311 INFO:tasks.workunit.client.0.smithi042.stdout:1/603: chown de/f58 0 1 2024-02-15T18:27:01.312 INFO:tasks.workunit.client.0.smithi042.stdout:9/615: rmdir d6/dd/dd1 0 2024-02-15T18:27:01.313 INFO:tasks.workunit.client.0.smithi042.stdout:9/616: readlink d6/da/d1d/d2d/d40/d5f/lab 0 2024-02-15T18:27:01.314 INFO:tasks.workunit.client.0.smithi042.stdout:4/567: mknod da/d11/d29/d4a/d4f/ccd 0 2024-02-15T18:27:01.314 INFO:tasks.workunit.client.0.smithi042.stdout:4/568: fdatasync da/d11/d67/d5c/d66/d78/d2f/f34 0 2024-02-15T18:27:01.314 INFO:tasks.workunit.client.0.smithi042.stdout:4/569: fdatasync da/d3b/d41/d8d/fb1 0 2024-02-15T18:27:01.315 INFO:tasks.workunit.client.1.smithi191.stdout:9/210: symlink d0/d1d/d3f/l53 0 2024-02-15T18:27:01.315 INFO:tasks.workunit.client.1.smithi191.stdout:9/211: truncate d0/d7/da/f3e 1039534 0 2024-02-15T18:27:01.315 INFO:tasks.workunit.client.1.smithi191.stdout:9/212: write d0/d7/dc/d17/f49 [373797,81967] 0 2024-02-15T18:27:01.316 INFO:tasks.workunit.client.0.smithi042.stdout:3/585: creat d1/de/d1a/fc2 x:0 0 0 2024-02-15T18:27:01.316 INFO:tasks.workunit.client.0.smithi042.stdout:9/617: dread d6/da/d1d/d2d/d40/f78 [0,4194304] 0 2024-02-15T18:27:01.321 INFO:tasks.workunit.client.0.smithi042.stdout:6/673: creat d9/d11/d46/d89/fec x:0 0 0 2024-02-15T18:27:01.321 INFO:tasks.workunit.client.0.smithi042.stdout:6/674: write d9/d11/d19/d35/d49/d79/f7a [591339,97586] 0 2024-02-15T18:27:01.322 INFO:tasks.workunit.client.0.smithi042.stdout:6/675: write d9/d11/d19/d1c/d31/f6f [3795958,84729] 0 2024-02-15T18:27:01.322 INFO:tasks.workunit.client.0.smithi042.stdout:6/676: write d9/d11/d19/d35/d49/d79/f60 [1533168,34723] 0 2024-02-15T18:27:01.323 INFO:tasks.workunit.client.1.smithi191.stdout:1/286: creat d0/d8/df/d12/d2a/d2b/f5a x:0 0 0 2024-02-15T18:27:01.324 INFO:tasks.workunit.client.1.smithi191.stdout:6/222: mknod d8/d9/c3a 0 2024-02-15T18:27:01.325 INFO:tasks.workunit.client.1.smithi191.stdout:0/140: mkdir d7/d26 0 2024-02-15T18:27:01.328 INFO:tasks.workunit.client.1.smithi191.stdout:5/122: write d0/f1f [680850,59309] 0 2024-02-15T18:27:01.329 INFO:tasks.workunit.client.1.smithi191.stdout:5/123: truncate d0/f1a 4807181 0 2024-02-15T18:27:01.329 INFO:tasks.workunit.client.1.smithi191.stdout:6/223: dread d8/f1c [0,4194304] 0 2024-02-15T18:27:01.331 INFO:tasks.workunit.client.0.smithi042.stdout:5/633: rmdir d0/d39 39 2024-02-15T18:27:01.334 INFO:tasks.workunit.client.1.smithi191.stdout:2/224: rename df/d12/d19/d1a/d35/d47/d4b/d4c to df/d12/d32/d4d 0 2024-02-15T18:27:01.336 INFO:tasks.workunit.client.0.smithi042.stdout:0/564: rename d4/d9/d34/d57/cb7 to d4/d38/d77/d2e/dae/cc5 0 2024-02-15T18:27:01.338 INFO:tasks.workunit.client.1.smithi191.stdout:7/224: link d0/d1/d3/f22 d0/d1/f51 0 2024-02-15T18:27:01.339 INFO:tasks.workunit.client.0.smithi042.stdout:2/549: mknod d0/d8/d20/d7d/ca5 0 2024-02-15T18:27:01.343 INFO:tasks.workunit.client.0.smithi042.stdout:9/618: dwrite d6/dd/d9/d59/d5c/fa6 [0,4194304] 0 2024-02-15T18:27:01.343 INFO:tasks.workunit.client.0.smithi042.stdout:8/531: creat dd/de/d94/f98 x:0 0 0 2024-02-15T18:27:01.343 INFO:tasks.workunit.client.0.smithi042.stdout:7/626: sync 2024-02-15T18:27:01.344 INFO:tasks.workunit.client.0.smithi042.stdout:7/627: stat d10/d43/d57/fb8 0 2024-02-15T18:27:01.344 INFO:tasks.workunit.client.0.smithi042.stdout:7/628: chown d10/d43/d57/l89 1532354791 1 2024-02-15T18:27:01.344 INFO:tasks.workunit.client.0.smithi042.stdout:1/604: unlink de/f4e 0 2024-02-15T18:27:01.344 INFO:tasks.workunit.client.1.smithi191.stdout:7/225: dread d0/d1/d1d/f4c [0,4194304] 0 2024-02-15T18:27:01.344 INFO:tasks.workunit.client.1.smithi191.stdout:7/226: fdatasync d0/d1/d3/d11/f18 0 2024-02-15T18:27:01.345 INFO:tasks.workunit.client.0.smithi042.stdout:4/570: truncate da/d11/f22 770919 0 2024-02-15T18:27:01.347 INFO:tasks.workunit.client.0.smithi042.stdout:3/586: getdents d1/d11/d14/d4d/d73/d89/da8 0 2024-02-15T18:27:01.348 INFO:tasks.workunit.client.0.smithi042.stdout:3/587: write d1/d11/d2e/f3b [3716513,44160] 0 2024-02-15T18:27:01.348 INFO:tasks.workunit.client.1.smithi191.stdout:2/225: symlink df/d12/d19/l4e 0 2024-02-15T18:27:01.349 INFO:tasks.workunit.client.1.smithi191.stdout:7/227: dread d0/db/f2e [0,4194304] 0 2024-02-15T18:27:01.350 INFO:tasks.workunit.client.1.smithi191.stdout:7/228: truncate d0/d1/d3/f3e 4275367 0 2024-02-15T18:27:01.351 INFO:tasks.workunit.client.0.smithi042.stdout:6/677: link d9/f21 d9/d11/d19/fed 0 2024-02-15T18:27:01.351 INFO:tasks.workunit.client.0.smithi042.stdout:6/678: chown d9/d11/d19/d1c/d31/d4f/l6d 19115470 1 2024-02-15T18:27:01.352 INFO:tasks.workunit.client.0.smithi042.stdout:6/679: chown d9/d11/d46/d89/dbc/lda 1359 1 2024-02-15T18:27:01.353 INFO:tasks.workunit.client.0.smithi042.stdout:6/680: read d9/d11/d19/d1c/d31/d53/fc2 [32205,61922] 0 2024-02-15T18:27:01.354 INFO:tasks.workunit.client.0.smithi042.stdout:5/634: truncate d0/f8f 42069 0 2024-02-15T18:27:01.354 INFO:tasks.workunit.client.0.smithi042.stdout:5/635: chown d0/c33 499 1 2024-02-15T18:27:01.354 INFO:tasks.workunit.client.0.smithi042.stdout:0/565: dread d4/d9/d34/d4e/d6c/f71 [0,4194304] 0 2024-02-15T18:27:01.360 INFO:tasks.workunit.client.1.smithi191.stdout:5/124: rename d0/c20 to d0/d14/d28/c29 0 2024-02-15T18:27:01.361 INFO:tasks.workunit.client.0.smithi042.stdout:2/550: mkdir d0/d8/d10/d31/da6 0 2024-02-15T18:27:01.361 INFO:tasks.workunit.client.0.smithi042.stdout:9/619: creat d6/da/d2f/d69/fd4 x:0 0 0 2024-02-15T18:27:01.361 INFO:tasks.workunit.client.1.smithi191.stdout:0/141: dwrite d7/da/f16 [0,4194304] 0 2024-02-15T18:27:01.364 INFO:tasks.workunit.client.0.smithi042.stdout:8/532: rename dd/de/d3c/f73 to dd/f99 0 2024-02-15T18:27:01.366 INFO:tasks.workunit.client.0.smithi042.stdout:4/571: unlink da/d3b/c99 0 2024-02-15T18:27:01.366 INFO:tasks.workunit.client.0.smithi042.stdout:6/681: write d9/f14 [1194049,2264] 0 2024-02-15T18:27:01.366 INFO:tasks.workunit.client.0.smithi042.stdout:4/572: chown da/d11/d29/d4a/d4f/f6c 23069 1 2024-02-15T18:27:01.367 INFO:tasks.workunit.client.1.smithi191.stdout:7/229: dread d0/f8 [0,4194304] 0 2024-02-15T18:27:01.367 INFO:tasks.workunit.client.1.smithi191.stdout:7/230: dread - d0/d1/d3/d11/d1c/f45 zero size 2024-02-15T18:27:01.367 INFO:tasks.workunit.client.1.smithi191.stdout:7/231: rename d0/d1/d3 to d0/d1/d3/dc/d4b/d52 22 2024-02-15T18:27:01.367 INFO:tasks.workunit.client.1.smithi191.stdout:7/232: dread - d0/db/f2f zero size 2024-02-15T18:27:01.367 INFO:tasks.workunit.client.1.smithi191.stdout:7/233: chown d0/d1/d35/d42/d46/d1e 3 1 2024-02-15T18:27:01.367 INFO:tasks.workunit.client.1.smithi191.stdout:5/125: mkdir d0/d14/d25/d2a 0 2024-02-15T18:27:01.368 INFO:tasks.workunit.client.1.smithi191.stdout:6/224: dwrite d8/d19/f1d [0,4194304] 0 2024-02-15T18:27:01.369 INFO:tasks.workunit.client.0.smithi042.stdout:3/588: unlink d1/d9c/l9f 0 2024-02-15T18:27:01.372 INFO:tasks.workunit.client.0.smithi042.stdout:5/636: mkdir d0/d39/ddd 0 2024-02-15T18:27:01.373 INFO:tasks.workunit.client.1.smithi191.stdout:0/142: mkdir d7/dd/d27 0 2024-02-15T18:27:01.374 INFO:tasks.workunit.client.0.smithi042.stdout:3/589: dread d1/d11/d14/d4d/f4f [0,4194304] 0 2024-02-15T18:27:01.375 INFO:tasks.workunit.client.0.smithi042.stdout:3/590: readlink d1/l95 0 2024-02-15T18:27:01.377 INFO:tasks.workunit.client.1.smithi191.stdout:7/234: creat d0/d1/d35/d42/d46/f53 x:0 0 0 2024-02-15T18:27:01.378 INFO:tasks.workunit.client.0.smithi042.stdout:9/620: creat d6/da/d1d/d2d/fd5 x:0 0 0 2024-02-15T18:27:01.380 INFO:tasks.workunit.client.1.smithi191.stdout:5/126: rmdir d0/d14/d25/d2a 0 2024-02-15T18:27:01.380 INFO:tasks.workunit.client.1.smithi191.stdout:5/127: fdatasync d0/f1b 0 2024-02-15T18:27:01.382 INFO:tasks.workunit.client.0.smithi042.stdout:8/533: mkdir dd/de/d3c/d4b/d67/d6f/d78/d9a 0 2024-02-15T18:27:01.384 INFO:tasks.workunit.client.0.smithi042.stdout:4/573: mkdir da/d11/d67/d5c/dce 0 2024-02-15T18:27:01.386 INFO:tasks.workunit.client.0.smithi042.stdout:6/682: symlink d9/d11/d19/d1c/d31/dd2/lee 0 2024-02-15T18:27:01.388 INFO:tasks.workunit.client.0.smithi042.stdout:8/534: creat dd/d11/f9b x:0 0 0 2024-02-15T18:27:01.388 INFO:tasks.workunit.client.1.smithi191.stdout:1/287: dwrite d0/d8/df/d12/d2a/d2b/f37 [0,4194304] 0 2024-02-15T18:27:01.388 INFO:tasks.workunit.client.1.smithi191.stdout:7/235: mkdir d0/d1/d1d/d54 0 2024-02-15T18:27:01.389 INFO:tasks.workunit.client.1.smithi191.stdout:7/236: read d0/db/f2e [337060,5141] 0 2024-02-15T18:27:01.391 INFO:tasks.workunit.client.0.smithi042.stdout:4/574: mkdir da/d11/d29/d4a/d4f/dcf 0 2024-02-15T18:27:01.391 INFO:tasks.workunit.client.1.smithi191.stdout:2/226: dwrite df/f1b [0,4194304] 0 2024-02-15T18:27:01.392 INFO:tasks.workunit.client.0.smithi042.stdout:1/605: dwrite de/d15/d1b/f49 [4194304,4194304] 0 2024-02-15T18:27:01.392 INFO:tasks.workunit.client.0.smithi042.stdout:7/629: dwrite d10/d43/d57/f5e [4194304,4194304] 0 2024-02-15T18:27:01.392 INFO:tasks.workunit.client.0.smithi042.stdout:8/535: symlink dd/de/d3c/l9c 0 2024-02-15T18:27:01.393 INFO:tasks.workunit.client.1.smithi191.stdout:6/225: rename d8/d9/c2f to d8/d9/c3b 0 2024-02-15T18:27:01.393 INFO:tasks.workunit.client.1.smithi191.stdout:6/226: readlink d8/d9/l39 0 2024-02-15T18:27:01.393 INFO:tasks.workunit.client.1.smithi191.stdout:6/227: write d8/d19/f37 [133776,77899] 0 2024-02-15T18:27:01.393 INFO:tasks.workunit.client.1.smithi191.stdout:6/228: fsync d8/f1c 0 2024-02-15T18:27:01.393 INFO:tasks.workunit.client.1.smithi191.stdout:6/229: read d8/f1c [1218527,43277] 0 2024-02-15T18:27:01.397 INFO:tasks.workunit.client.0.smithi042.stdout:0/566: rmdir d4/d38 39 2024-02-15T18:27:01.401 INFO:tasks.workunit.client.1.smithi191.stdout:0/143: dread d7/fb [4194304,4194304] 0 2024-02-15T18:27:01.403 INFO:tasks.workunit.client.0.smithi042.stdout:1/606: creat de/d15/d2c/d51/d70/fdd x:0 0 0 2024-02-15T18:27:01.405 INFO:tasks.workunit.client.0.smithi042.stdout:2/551: dwrite d0/d8/f65 [0,4194304] 0 2024-02-15T18:27:01.407 INFO:tasks.workunit.client.0.smithi042.stdout:1/607: dread de/d15/d67/d46/f50 [0,4194304] 0 2024-02-15T18:27:01.407 INFO:tasks.workunit.client.0.smithi042.stdout:1/608: write de/d15/d67/d46/fc0 [934896,103691] 0 2024-02-15T18:27:01.407 INFO:tasks.workunit.client.0.smithi042.stdout:5/637: dwrite d0/d14/f1c [0,4194304] 0 2024-02-15T18:27:01.410 INFO:tasks.workunit.client.1.smithi191.stdout:1/288: creat d0/d2f/d51/d10/f5b x:0 0 0 2024-02-15T18:27:01.411 INFO:tasks.workunit.client.0.smithi042.stdout:4/575: dread da/d11/d29/f32 [0,4194304] 0 2024-02-15T18:27:01.413 INFO:tasks.workunit.client.0.smithi042.stdout:4/576: read da/d2b/f37 [3926829,22013] 0 2024-02-15T18:27:01.417 INFO:tasks.workunit.client.0.smithi042.stdout:7/630: getdents d10/d49/d6d/d6e 0 2024-02-15T18:27:01.417 INFO:tasks.workunit.client.0.smithi042.stdout:7/631: dread - d10/d15/d8f/f9f zero size 2024-02-15T18:27:01.418 INFO:tasks.workunit.client.0.smithi042.stdout:0/567: truncate d4/d38/f3c 6093115 0 2024-02-15T18:27:01.418 INFO:tasks.workunit.client.0.smithi042.stdout:0/568: fsync d4/d38/d39/d8e/d8f/db2/fb6 0 2024-02-15T18:27:01.420 INFO:tasks.workunit.client.1.smithi191.stdout:5/128: rename d0/f22 to d0/d14/d28/f2b 0 2024-02-15T18:27:01.423 INFO:tasks.workunit.client.0.smithi042.stdout:1/609: creat de/d15/d2c/fde x:0 0 0 2024-02-15T18:27:01.424 INFO:tasks.workunit.client.1.smithi191.stdout:2/227: unlink fd 0 2024-02-15T18:27:01.426 INFO:tasks.workunit.client.1.smithi191.stdout:6/230: link d8/d9/d2b/c31 d8/c3c 0 2024-02-15T18:27:01.426 INFO:tasks.workunit.client.1.smithi191.stdout:6/231: truncate d8/d9/d2b/f32 420757 0 2024-02-15T18:27:01.427 INFO:tasks.workunit.client.0.smithi042.stdout:5/638: mkdir d0/dda/dc7/d52/dad/dba/dbd/dde 0 2024-02-15T18:27:01.427 INFO:tasks.workunit.client.1.smithi191.stdout:0/144: mknod d7/d26/c28 0 2024-02-15T18:27:01.428 INFO:tasks.workunit.client.0.smithi042.stdout:7/632: truncate d10/f7e 3927869 0 2024-02-15T18:27:01.428 INFO:tasks.workunit.client.1.smithi191.stdout:1/289: mkdir d0/d2f/d5c 0 2024-02-15T18:27:01.429 INFO:tasks.workunit.client.0.smithi042.stdout:0/569: creat d4/d38/d5c/fc6 x:0 0 0 2024-02-15T18:27:01.430 INFO:tasks.workunit.client.1.smithi191.stdout:5/129: symlink d0/d23/l2c 0 2024-02-15T18:27:01.432 INFO:tasks.workunit.client.0.smithi042.stdout:1/610: rename de/d15/d67/d46/d6b to de/d54/dbf/dd0/d7c/dd6/ddf 0 2024-02-15T18:27:01.432 INFO:tasks.workunit.client.1.smithi191.stdout:6/232: creat d8/d19/f3d x:0 0 0 2024-02-15T18:27:01.434 INFO:tasks.workunit.client.0.smithi042.stdout:5/639: symlink d0/d38/d53/ldf 0 2024-02-15T18:27:01.434 INFO:tasks.workunit.client.1.smithi191.stdout:0/145: creat d7/dd/f29 x:0 0 0 2024-02-15T18:27:01.436 INFO:tasks.workunit.client.1.smithi191.stdout:1/290: creat d0/d8/df/d12/f5d x:0 0 0 2024-02-15T18:27:01.436 INFO:tasks.workunit.client.1.smithi191.stdout:1/291: chown d0/d2f/d51/f1d 31 1 2024-02-15T18:27:01.438 INFO:tasks.workunit.client.0.smithi042.stdout:9/621: dwrite d6/dd/d9/f32 [0,4194304] 0 2024-02-15T18:27:01.438 INFO:tasks.workunit.client.0.smithi042.stdout:6/683: dwrite d9/d3e/fa3 [0,4194304] 0 2024-02-15T18:27:01.438 INFO:tasks.workunit.client.1.smithi191.stdout:0/146: symlink d7/dd/l2a 0 2024-02-15T18:27:01.439 INFO:tasks.workunit.client.0.smithi042.stdout:4/577: dwrite da/d11/d67/d5c/d66/f8a [0,4194304] 0 2024-02-15T18:27:01.439 INFO:tasks.workunit.client.0.smithi042.stdout:8/536: dwrite dd/de/d3c/d4b/d67/f2b [0,4194304] 0 2024-02-15T18:27:01.441 INFO:tasks.workunit.client.0.smithi042.stdout:3/591: dwrite d1/d11/f45 [4194304,4194304] 0 2024-02-15T18:27:01.443 INFO:tasks.workunit.client.0.smithi042.stdout:2/552: dwrite d0/d40/d56/f7f [0,4194304] 0 2024-02-15T18:27:01.444 INFO:tasks.workunit.client.1.smithi191.stdout:1/292: mkdir d0/d2f/d51/d5e 0 2024-02-15T18:27:01.446 INFO:tasks.workunit.client.0.smithi042.stdout:5/640: mkdir d0/d39/ddd/de0 0 2024-02-15T18:27:01.446 INFO:tasks.workunit.client.0.smithi042.stdout:5/641: getdents d0/dda/dc7/d52/da0 0 2024-02-15T18:27:01.447 INFO:tasks.workunit.client.1.smithi191.stdout:2/228: dwrite df/d12/f36 [0,4194304] 0 2024-02-15T18:27:01.452 INFO:tasks.workunit.client.1.smithi191.stdout:7/237: getdents d0/d1/d1d 0 2024-02-15T18:27:01.456 INFO:tasks.workunit.client.0.smithi042.stdout:7/633: mknod d10/d12/d23/cd5 0 2024-02-15T18:27:01.456 INFO:tasks.workunit.client.0.smithi042.stdout:7/634: fdatasync d10/d43/d57/fb0 0 2024-02-15T18:27:01.457 INFO:tasks.workunit.client.1.smithi191.stdout:5/130: dread d0/f1a [0,4194304] 0 2024-02-15T18:27:01.457 INFO:tasks.workunit.client.1.smithi191.stdout:5/131: fdatasync d0/d23/f27 0 2024-02-15T18:27:01.459 INFO:tasks.workunit.client.1.smithi191.stdout:1/293: mkdir d0/d2f/d51/d54/d5f 0 2024-02-15T18:27:01.463 INFO:tasks.workunit.client.0.smithi042.stdout:1/611: dwrite de/d15/d1b/d44/fb2 [0,4194304] 0 2024-02-15T18:27:01.463 INFO:tasks.workunit.client.0.smithi042.stdout:1/612: read - de/d54/d9a/dc8/fdb zero size 2024-02-15T18:27:01.465 INFO:tasks.workunit.client.0.smithi042.stdout:9/622: mknod d6/da/d2f/d55/d73/d8d/cd6 0 2024-02-15T18:27:01.467 INFO:tasks.workunit.client.1.smithi191.stdout:5/132: write d0/f13 [604723,19670] 0 2024-02-15T18:27:01.467 INFO:tasks.workunit.client.1.smithi191.stdout:7/238: write d0/d1/d35/f48 [1109460,91671] 0 2024-02-15T18:27:01.467 INFO:tasks.workunit.client.1.smithi191.stdout:7/239: chown d0/d1/df/f16 234173617 1 2024-02-15T18:27:01.467 INFO:tasks.workunit.client.0.smithi042.stdout:1/613: write de/f58 [237376,111577] 0 2024-02-15T18:27:01.468 INFO:tasks.workunit.client.1.smithi191.stdout:2/229: mknod df/d12/d19/d1a/d35/c4f 0 2024-02-15T18:27:01.468 INFO:tasks.workunit.client.1.smithi191.stdout:2/230: chown df/d12/d19/d1a/c27 73 1 2024-02-15T18:27:01.470 INFO:tasks.workunit.client.0.smithi042.stdout:7/635: dwrite d10/d15/d39/d6c/dba/fcd [0,4194304] 0 2024-02-15T18:27:01.471 INFO:tasks.workunit.client.1.smithi191.stdout:1/294: mkdir d0/d2f/d51/d54/d5f/d60 0 2024-02-15T18:27:01.472 INFO:tasks.workunit.client.0.smithi042.stdout:8/537: creat dd/d11/d75/f9d x:0 0 0 2024-02-15T18:27:01.477 INFO:tasks.workunit.client.1.smithi191.stdout:1/295: rmdir d0/d2f/d51/d54/d5f 39 2024-02-15T18:27:01.477 INFO:tasks.workunit.client.1.smithi191.stdout:7/240: creat d0/db/f55 x:0 0 0 2024-02-15T18:27:01.477 INFO:tasks.workunit.client.0.smithi042.stdout:2/553: mknod d0/d8/d20/d7d/d7e/ca7 0 2024-02-15T18:27:01.482 INFO:tasks.workunit.client.0.smithi042.stdout:0/570: dwrite d4/f7 [4194304,4194304] 0 2024-02-15T18:27:01.483 INFO:tasks.workunit.client.1.smithi191.stdout:0/147: dwrite d7/dd/f29 [0,4194304] 0 2024-02-15T18:27:01.485 INFO:tasks.workunit.client.1.smithi191.stdout:6/233: dwrite d8/d9/d2b/f2e [0,4194304] 0 2024-02-15T18:27:01.485 INFO:tasks.workunit.client.1.smithi191.stdout:6/234: stat d8/d9/d2b/f2e 0 2024-02-15T18:27:01.489 INFO:tasks.workunit.client.0.smithi042.stdout:5/642: dread d0/fe [4194304,4194304] 0 2024-02-15T18:27:01.490 INFO:tasks.workunit.client.1.smithi191.stdout:1/296: getdents d0/d8/df/d41 0 2024-02-15T18:27:01.494 INFO:tasks.workunit.client.0.smithi042.stdout:9/623: creat d6/da/d2f/d54/d96/fd7 x:0 0 0 2024-02-15T18:27:01.495 INFO:tasks.workunit.client.0.smithi042.stdout:1/614: truncate de/d15/d67/d46/fd7 2854349 0 2024-02-15T18:27:01.499 INFO:tasks.workunit.client.0.smithi042.stdout:6/684: dwrite d9/f76 [0,4194304] 0 2024-02-15T18:27:01.499 INFO:tasks.workunit.client.1.smithi191.stdout:5/133: dwrite d0/d23/f27 [0,4194304] 0 2024-02-15T18:27:01.499 INFO:tasks.workunit.client.1.smithi191.stdout:5/134: fsync d0/f1b 0 2024-02-15T18:27:01.499 INFO:tasks.workunit.client.1.smithi191.stdout:5/135: stat d0/d23 0 2024-02-15T18:27:01.500 INFO:tasks.workunit.client.1.smithi191.stdout:2/231: dwrite df/f1b [4194304,4194304] 0 2024-02-15T18:27:01.500 INFO:tasks.workunit.client.0.smithi042.stdout:7/636: link d10/d12/d23/d51/f99 d10/d49/fd6 0 2024-02-15T18:27:01.501 INFO:tasks.workunit.client.0.smithi042.stdout:7/637: write d10/d49/d6d/d6e/db3/fc7 [364104,37269] 0 2024-02-15T18:27:01.501 INFO:tasks.workunit.client.0.smithi042.stdout:7/638: chown d10/d15/f3d 2616990 1 2024-02-15T18:27:01.501 INFO:tasks.workunit.client.0.smithi042.stdout:7/639: write d10/d49/d6d/d78/f91 [512009,125710] 0 2024-02-15T18:27:01.502 INFO:tasks.workunit.client.1.smithi191.stdout:0/148: creat d7/f2b x:0 0 0 2024-02-15T18:27:01.504 INFO:tasks.workunit.client.0.smithi042.stdout:8/538: creat dd/de/d12/f9e x:0 0 0 2024-02-15T18:27:01.505 INFO:tasks.workunit.client.0.smithi042.stdout:2/554: symlink d0/d8/db/la8 0 2024-02-15T18:27:01.506 INFO:tasks.workunit.client.0.smithi042.stdout:2/555: read d0/d8/f39 [549656,110725] 0 2024-02-15T18:27:01.510 INFO:tasks.workunit.client.0.smithi042.stdout:0/571: unlink d4/d38/d77/d2e/c33 0 2024-02-15T18:27:01.511 INFO:tasks.workunit.client.0.smithi042.stdout:0/572: dread d4/d9/d34/d4e/d6c/d70/faa [0,4194304] 0 2024-02-15T18:27:01.512 INFO:tasks.workunit.client.1.smithi191.stdout:1/297: creat d0/d8/df/d12/d2a/d2b/f61 x:0 0 0 2024-02-15T18:27:01.513 INFO:tasks.workunit.client.0.smithi042.stdout:2/556: dread d0/d40/d56/f7f [0,4194304] 0 2024-02-15T18:27:01.515 INFO:tasks.workunit.client.1.smithi191.stdout:1/298: write d0/d8/df/d12/f32 [3756110,94556] 0 2024-02-15T18:27:01.517 INFO:tasks.workunit.client.0.smithi042.stdout:5/643: rename d0/db to d0/dda/dc7/de1 0 2024-02-15T18:27:01.518 INFO:tasks.workunit.client.0.smithi042.stdout:9/624: mkdir d6/da/d2f/d6c/dd8 0 2024-02-15T18:27:01.524 INFO:tasks.workunit.client.0.smithi042.stdout:4/578: truncate da/d11/d67/d5c/d66/f8a 1035591 0 2024-02-15T18:27:01.529 INFO:tasks.workunit.client.1.smithi191.stdout:7/241: truncate d0/d1/d3/f3e 596297 0 2024-02-15T18:27:01.529 INFO:tasks.workunit.client.1.smithi191.stdout:7/242: write d0/d1/d3/d11/f18 [1183487,75849] 0 2024-02-15T18:27:01.538 INFO:tasks.workunit.client.0.smithi042.stdout:0/573: dwrite d4/d9/d34/f4b [4194304,4194304] 0 2024-02-15T18:27:01.539 INFO:tasks.workunit.client.1.smithi191.stdout:2/232: creat df/d12/d32/d4d/f50 x:0 0 0 2024-02-15T18:27:01.539 INFO:tasks.workunit.client.0.smithi042.stdout:0/574: write d4/d9/d34/d4e/f87 [4317399,20098] 0 2024-02-15T18:27:01.540 INFO:tasks.workunit.client.0.smithi042.stdout:6/685: mknod d9/d11/d19/d35/d55/d92/d9e/cef 0 2024-02-15T18:27:01.541 INFO:tasks.workunit.client.1.smithi191.stdout:1/299: dwrite d0/d8/df/d12/d33/d49/f4a [0,4194304] 0 2024-02-15T18:27:01.542 INFO:tasks.workunit.client.1.smithi191.stdout:0/149: mkdir d7/da/d1a/d2c 0 2024-02-15T18:27:01.548 INFO:tasks.workunit.client.1.smithi191.stdout:6/235: rename d8/d9/d2b/c31 to d8/d9/c3e 0 2024-02-15T18:27:01.548 INFO:tasks.workunit.client.1.smithi191.stdout:6/236: dread - d8/d19/f3d zero size 2024-02-15T18:27:01.552 INFO:tasks.workunit.client.1.smithi191.stdout:5/136: unlink d0/ca 0 2024-02-15T18:27:01.554 INFO:tasks.workunit.client.1.smithi191.stdout:7/243: creat d0/d1/d3/dc/d4b/f56 x:0 0 0 2024-02-15T18:27:01.554 INFO:tasks.workunit.client.1.smithi191.stdout:8/323: sync 2024-02-15T18:27:01.554 INFO:tasks.workunit.client.1.smithi191.stdout:3/170: sync 2024-02-15T18:27:01.556 INFO:tasks.workunit.client.1.smithi191.stdout:2/233: link df/f10 df/d12/d19/d1a/d35/d47/f51 0 2024-02-15T18:27:01.556 INFO:tasks.workunit.client.1.smithi191.stdout:2/234: write df/d12/d1f/f26 [5093985,23172] 0 2024-02-15T18:27:01.561 INFO:tasks.workunit.client.1.smithi191.stdout:1/300: symlink d0/d8/df/d41/l62 0 2024-02-15T18:27:01.562 INFO:tasks.workunit.client.1.smithi191.stdout:1/301: read d0/d8/df/d12/d2a/f3a [217502,94392] 0 2024-02-15T18:27:01.563 INFO:tasks.workunit.client.0.smithi042.stdout:0/575: dwrite d4/d9/d5a/f6b [0,4194304] 0 2024-02-15T18:27:01.565 INFO:tasks.workunit.client.1.smithi191.stdout:0/150: rename d7/dd/lf to d7/da/d1a/l2d 0 2024-02-15T18:27:01.565 INFO:tasks.workunit.client.1.smithi191.stdout:0/151: chown d7 66591592 1 2024-02-15T18:27:01.566 INFO:tasks.workunit.client.1.smithi191.stdout:5/137: dread d0/d23/f27 [0,4194304] 0 2024-02-15T18:27:01.566 INFO:tasks.workunit.client.1.smithi191.stdout:7/244: mkdir d0/d1/d3/d11/d57 0 2024-02-15T18:27:01.567 INFO:tasks.workunit.client.1.smithi191.stdout:7/245: readlink d0/d1/l38 0 2024-02-15T18:27:01.567 INFO:tasks.workunit.client.0.smithi042.stdout:8/539: creat dd/de/d3c/d4b/d67/f9f x:0 0 0 2024-02-15T18:27:01.567 INFO:tasks.workunit.client.0.smithi042.stdout:8/540: readlink dd/de/d33/l58 0 2024-02-15T18:27:01.568 INFO:tasks.workunit.client.0.smithi042.stdout:8/541: read dd/d11/f21 [148240,107676] 0 2024-02-15T18:27:01.568 INFO:tasks.workunit.client.0.smithi042.stdout:8/542: chown dd/f3e 11316 1 2024-02-15T18:27:01.569 INFO:tasks.workunit.client.1.smithi191.stdout:3/171: creat d6/d1b/d1f/f3d x:0 0 0 2024-02-15T18:27:01.569 INFO:tasks.workunit.client.0.smithi042.stdout:2/557: creat d0/d40/d56/fa9 x:0 0 0 2024-02-15T18:27:01.570 INFO:tasks.workunit.client.1.smithi191.stdout:3/172: write d6/fd [3293703,129276] 0 2024-02-15T18:27:01.572 INFO:tasks.workunit.client.1.smithi191.stdout:8/324: mkdir d3/d2c/d1a/d23/d2e/d5b/d77 0 2024-02-15T18:27:01.572 INFO:tasks.workunit.client.1.smithi191.stdout:8/325: dread - d3/d2c/d1a/d23/d2e/d47/f63 zero size 2024-02-15T18:27:01.572 INFO:tasks.workunit.client.1.smithi191.stdout:8/326: write d3/d2c/d1a/d23/d2a/f35 [437961,34457] 0 2024-02-15T18:27:01.572 INFO:tasks.workunit.client.0.smithi042.stdout:0/576: dread d4/d9/d34/d4e/f87 [4194304,4194304] 0 2024-02-15T18:27:01.572 INFO:tasks.workunit.client.1.smithi191.stdout:3/173: dread f4 [0,4194304] 0 2024-02-15T18:27:01.574 INFO:tasks.workunit.client.0.smithi042.stdout:7/640: rename d10/d43/f50 to d10/d49/d6d/fd7 0 2024-02-15T18:27:01.575 INFO:tasks.workunit.client.1.smithi191.stdout:2/235: mkdir df/d12/d19/d1a/d52 0 2024-02-15T18:27:01.576 INFO:tasks.workunit.client.1.smithi191.stdout:2/236: read df/d12/d19/d1a/f44 [332457,124224] 0 2024-02-15T18:27:01.576 INFO:tasks.workunit.client.0.smithi042.stdout:7/641: dread d10/d49/d6d/d78/d83/fa3 [0,4194304] 0 2024-02-15T18:27:01.576 INFO:tasks.workunit.client.0.smithi042.stdout:5/644: creat d0/dda/dc7/d52/da0/fe2 x:0 0 0 2024-02-15T18:27:01.581 INFO:tasks.workunit.client.1.smithi191.stdout:4/302: sync 2024-02-15T18:27:01.582 INFO:tasks.workunit.client.0.smithi042.stdout:8/543: dread dd/de/d3c/d4b/d67/d5a/f91 [0,4194304] 0 2024-02-15T18:27:01.582 INFO:tasks.workunit.client.0.smithi042.stdout:8/544: fdatasync fc 0 2024-02-15T18:27:01.582 INFO:tasks.workunit.client.0.smithi042.stdout:8/545: write dd/de/d3c/d4b/f8d [511832,37889] 0 2024-02-15T18:27:01.582 INFO:tasks.workunit.client.0.smithi042.stdout:8/546: chown dd/de/d12/f1e 12 1 2024-02-15T18:27:01.582 INFO:tasks.workunit.client.1.smithi191.stdout:9/213: sync 2024-02-15T18:27:01.583 INFO:tasks.workunit.client.1.smithi191.stdout:9/214: write d0/d7/dc/d17/f2d [73622,96551] 0 2024-02-15T18:27:01.583 INFO:tasks.workunit.client.1.smithi191.stdout:1/302: creat d0/d8/df/d12/d33/f63 x:0 0 0 2024-02-15T18:27:01.584 INFO:tasks.workunit.client.1.smithi191.stdout:1/303: write d0/d8/df/d12/f28 [913415,94363] 0 2024-02-15T18:27:01.584 INFO:tasks.workunit.client.1.smithi191.stdout:1/304: chown d0/d8/df/d2e/f4f 3246100 1 2024-02-15T18:27:01.584 INFO:tasks.workunit.client.0.smithi042.stdout:9/625: truncate d6/dd/d9/f9f 1858141 0 2024-02-15T18:27:01.584 INFO:tasks.workunit.client.0.smithi042.stdout:9/626: stat d6/da/d1d/d2d/d40/f43 0 2024-02-15T18:27:01.585 INFO:tasks.workunit.client.1.smithi191.stdout:0/152: symlink d7/dd/d27/l2e 0 2024-02-15T18:27:01.587 INFO:tasks.workunit.client.0.smithi042.stdout:1/615: link de/d15/d1b/ca5 de/d15/d95/ce0 0 2024-02-15T18:27:01.587 INFO:tasks.workunit.client.1.smithi191.stdout:1/305: dread d0/f9 [0,4194304] 0 2024-02-15T18:27:01.587 INFO:tasks.workunit.client.1.smithi191.stdout:1/306: chown d0/d8/df/d12/l56 23046984 1 2024-02-15T18:27:01.588 INFO:tasks.workunit.client.1.smithi191.stdout:6/237: rename d8/d9/d2b/l34 to d8/l3f 0 2024-02-15T18:27:01.593 INFO:tasks.workunit.client.1.smithi191.stdout:3/174: creat d6/d1b/f3e x:0 0 0 2024-02-15T18:27:01.594 INFO:tasks.workunit.client.0.smithi042.stdout:6/686: creat d9/d8d/ff0 x:0 0 0 2024-02-15T18:27:01.595 INFO:tasks.workunit.client.0.smithi042.stdout:7/642: dwrite d10/d15/f3d [0,4194304] 0 2024-02-15T18:27:01.595 INFO:tasks.workunit.client.1.smithi191.stdout:2/237: mknod df/d12/d32/c53 0 2024-02-15T18:27:01.596 INFO:tasks.workunit.client.1.smithi191.stdout:1/307: dread d0/d8/df/d12/d2a/d2b/f37 [0,4194304] 0 2024-02-15T18:27:01.597 INFO:tasks.workunit.client.1.smithi191.stdout:1/308: write d0/f1b [19360,34888] 0 2024-02-15T18:27:01.598 INFO:tasks.workunit.client.1.smithi191.stdout:2/238: write df/d12/d19/d1a/f44 [1882308,85314] 0 2024-02-15T18:27:01.598 INFO:tasks.workunit.client.1.smithi191.stdout:2/239: fdatasync df/d12/f22 0 2024-02-15T18:27:01.598 INFO:tasks.workunit.client.0.smithi042.stdout:7/643: fdatasync d10/d49/d6d/d78/f91 0 2024-02-15T18:27:01.600 INFO:tasks.workunit.client.1.smithi191.stdout:2/240: read df/d12/d19/d1a/f21 [578462,96835] 0 2024-02-15T18:27:01.600 INFO:tasks.workunit.client.1.smithi191.stdout:8/327: rmdir d3/d2c/d1a/d23/d2e 39 2024-02-15T18:27:01.605 INFO:tasks.workunit.client.0.smithi042.stdout:3/592: sync 2024-02-15T18:27:01.610 INFO:tasks.workunit.client.1.smithi191.stdout:9/215: rmdir d0/d1d/d3f 39 2024-02-15T18:27:01.610 INFO:tasks.workunit.client.1.smithi191.stdout:9/216: readlink d0/l16 0 2024-02-15T18:27:01.610 INFO:tasks.workunit.client.1.smithi191.stdout:4/303: mknod d2/d5/d14/d47/d78/c7c 0 2024-02-15T18:27:01.611 INFO:tasks.workunit.client.0.smithi042.stdout:2/558: creat d0/d8/d10/d7b/faa x:0 0 0 2024-02-15T18:27:01.612 INFO:tasks.workunit.client.1.smithi191.stdout:4/304: dread d2/d5/d7/d3e/d42/d52/f53 [0,4194304] 0 2024-02-15T18:27:01.612 INFO:tasks.workunit.client.0.smithi042.stdout:3/593: dread d1/de/f42 [0,4194304] 0 2024-02-15T18:27:01.615 INFO:tasks.workunit.client.0.smithi042.stdout:0/577: getdents d4/d9/d34/d4e/d6c 0 2024-02-15T18:27:01.616 INFO:tasks.workunit.client.1.smithi191.stdout:7/246: rename d0/d1/d3/l4e to d0/d1/d3/d11/d1c/l58 0 2024-02-15T18:27:01.617 INFO:tasks.workunit.client.1.smithi191.stdout:6/238: truncate d8/f33 1285492 0 2024-02-15T18:27:01.617 INFO:tasks.workunit.client.1.smithi191.stdout:6/239: stat d8/d19/f3d 0 2024-02-15T18:27:01.617 INFO:tasks.workunit.client.0.smithi042.stdout:0/578: read d4/d9/d34/d4e/f87 [2704414,121114] 0 2024-02-15T18:27:01.618 INFO:tasks.workunit.client.1.smithi191.stdout:8/328: dread d3/d2c/d1a/f1b [4194304,4194304] 0 2024-02-15T18:27:01.619 INFO:tasks.workunit.client.0.smithi042.stdout:4/579: rename da/d3a/l4e to da/d11/d67/d5c/ld0 0 2024-02-15T18:27:01.620 INFO:tasks.workunit.client.0.smithi042.stdout:5/645: unlink d0/d12/d2a/c37 0 2024-02-15T18:27:01.621 INFO:tasks.workunit.client.1.smithi191.stdout:8/329: write d3/d2c/d1a/d23/f6a [4912057,113860] 0 2024-02-15T18:27:01.622 INFO:tasks.workunit.client.1.smithi191.stdout:1/309: truncate d0/d2f/d51/f1d 359538 0 2024-02-15T18:27:01.624 INFO:tasks.workunit.client.0.smithi042.stdout:8/547: unlink dd/de/d3c/d4b/d67/d5a/l61 0 2024-02-15T18:27:01.630 INFO:tasks.workunit.client.0.smithi042.stdout:9/627: mknod d6/cd9 0 2024-02-15T18:27:01.630 INFO:tasks.workunit.client.0.smithi042.stdout:1/616: creat de/d54/dbf/fe1 x:0 0 0 2024-02-15T18:27:01.630 INFO:tasks.workunit.client.0.smithi042.stdout:9/628: fsync f3 0 2024-02-15T18:27:01.630 INFO:tasks.workunit.client.0.smithi042.stdout:6/687: creat d9/d11/ff1 x:0 0 0 2024-02-15T18:27:01.633 INFO:tasks.workunit.client.0.smithi042.stdout:1/617: dread de/d15/d67/d74/fab [0,4194304] 0 2024-02-15T18:27:01.634 INFO:tasks.workunit.client.1.smithi191.stdout:7/247: creat d0/d1/d3/dc/d4b/f59 x:0 0 0 2024-02-15T18:27:01.637 INFO:tasks.workunit.client.1.smithi191.stdout:0/153: rename d7/dd to d7/da/d17/d2f 0 2024-02-15T18:27:01.637 INFO:tasks.workunit.client.0.smithi042.stdout:3/594: symlink d1/d11/d7a/lc3 0 2024-02-15T18:27:01.637 INFO:tasks.workunit.client.0.smithi042.stdout:3/595: write d1/de/d1a/fa4 [672786,92269] 0 2024-02-15T18:27:01.638 INFO:tasks.workunit.client.0.smithi042.stdout:0/579: creat d4/d9/d62/d74/fc7 x:0 0 0 2024-02-15T18:27:01.639 INFO:tasks.workunit.client.1.smithi191.stdout:4/305: dwrite d2/d5/d7/f29 [0,4194304] 0 2024-02-15T18:27:01.639 INFO:tasks.workunit.client.1.smithi191.stdout:6/240: unlink d8/d9/c3a 0 2024-02-15T18:27:01.640 INFO:tasks.workunit.client.1.smithi191.stdout:6/241: write d8/d9/d2b/f2e [1737969,46321] 0 2024-02-15T18:27:01.640 INFO:tasks.workunit.client.1.smithi191.stdout:6/242: fsync d8/d19/f35 0 2024-02-15T18:27:01.641 INFO:tasks.workunit.client.0.smithi042.stdout:2/559: rename d0/f3 to d0/d8/d10/d5a/fab 0 2024-02-15T18:27:01.641 INFO:tasks.workunit.client.1.smithi191.stdout:4/306: dread d2/d5/d7/d3e/d42/d52/f53 [0,4194304] 0 2024-02-15T18:27:01.642 INFO:tasks.workunit.client.0.smithi042.stdout:4/580: rmdir da/d11/d29/d4a/d4f/d90 39 2024-02-15T18:27:01.649 INFO:tasks.workunit.client.1.smithi191.stdout:5/138: rmdir d0/d14/d25 0 2024-02-15T18:27:01.649 INFO:tasks.workunit.client.1.smithi191.stdout:5/139: readlink d0/d23/l2c 0 2024-02-15T18:27:01.652 INFO:tasks.workunit.client.0.smithi042.stdout:6/688: rmdir d9/d3e 39 2024-02-15T18:27:01.652 INFO:tasks.workunit.client.0.smithi042.stdout:9/629: link d6/da/d2f/d6c/f94 d6/dd/d9/fda 0 2024-02-15T18:27:01.652 INFO:tasks.workunit.client.0.smithi042.stdout:9/630: dread - d6/da/d2f/d54/d96/fd7 zero size 2024-02-15T18:27:01.653 INFO:tasks.workunit.client.0.smithi042.stdout:9/631: write d6/dd/d9/da1/fa5 [536372,46892] 0 2024-02-15T18:27:01.653 INFO:tasks.workunit.client.0.smithi042.stdout:5/646: dwrite d0/fce [0,4194304] 0 2024-02-15T18:27:01.653 INFO:tasks.workunit.client.1.smithi191.stdout:8/330: mknod d3/d5e/c78 0 2024-02-15T18:27:01.655 INFO:tasks.workunit.client.0.smithi042.stdout:7/644: symlink d10/d12/d23/ld8 0 2024-02-15T18:27:01.655 INFO:tasks.workunit.client.0.smithi042.stdout:7/645: write d10/d15/d8f/f9f [298254,128846] 0 2024-02-15T18:27:01.655 INFO:tasks.workunit.client.0.smithi042.stdout:7/646: readlink d10/d15/l25 0 2024-02-15T18:27:01.655 INFO:tasks.workunit.client.0.smithi042.stdout:7/647: write d10/d43/d57/f79 [937383,32614] 0 2024-02-15T18:27:01.656 INFO:tasks.workunit.client.1.smithi191.stdout:7/248: mkdir d0/d1/d3/d11/d5a 0 2024-02-15T18:27:01.656 INFO:tasks.workunit.client.1.smithi191.stdout:7/249: write d0/d1/df/f16 [786729,10466] 0 2024-02-15T18:27:01.656 INFO:tasks.workunit.client.1.smithi191.stdout:4/307: write d2/d5/d7/f25 [1524846,65524] 0 2024-02-15T18:27:01.656 INFO:tasks.workunit.client.0.smithi042.stdout:1/618: mknod de/d15/d67/d46/d6a/ce2 0 2024-02-15T18:27:01.659 INFO:tasks.workunit.client.1.smithi191.stdout:3/175: rename d6/d3b/f2c to d6/d27/f3f 0 2024-02-15T18:27:01.659 INFO:tasks.workunit.client.1.smithi191.stdout:3/176: stat d6/d1b/f3e 0 2024-02-15T18:27:01.660 INFO:tasks.workunit.client.0.smithi042.stdout:3/596: symlink d1/de/lc4 0 2024-02-15T18:27:01.661 INFO:tasks.workunit.client.0.smithi042.stdout:0/580: rename d4/d9/d34/d4e to d4/d9/d34/dab/dc8 0 2024-02-15T18:27:01.663 INFO:tasks.workunit.client.0.smithi042.stdout:2/560: creat d0/d8/d10/d31/da6/fac x:0 0 0 2024-02-15T18:27:01.664 INFO:tasks.workunit.client.1.smithi191.stdout:8/331: creat d3/d5e/f79 x:0 0 0 2024-02-15T18:27:01.664 INFO:tasks.workunit.client.1.smithi191.stdout:3/177: write d6/d27/f3f [831330,75111] 0 2024-02-15T18:27:01.664 INFO:tasks.workunit.client.1.smithi191.stdout:9/217: dwrite d0/f3c [0,4194304] 0 2024-02-15T18:27:01.665 INFO:tasks.workunit.client.0.smithi042.stdout:1/619: dread f2 [4194304,4194304] 0 2024-02-15T18:27:01.665 INFO:tasks.workunit.client.0.smithi042.stdout:4/581: creat da/d11/d67/d5c/d66/d78/d2f/fd1 x:0 0 0 2024-02-15T18:27:01.666 INFO:tasks.workunit.client.1.smithi191.stdout:1/310: dwrite d0/d8/df/d12/d33/d49/f4a [0,4194304] 0 2024-02-15T18:27:01.667 INFO:tasks.workunit.client.0.smithi042.stdout:6/689: mkdir d9/d11/d19/d1c/d31/d9b/db4/df2 0 2024-02-15T18:27:01.667 INFO:tasks.workunit.client.0.smithi042.stdout:6/690: readlink d9/d11/l23 0 2024-02-15T18:27:01.667 INFO:tasks.workunit.client.0.smithi042.stdout:9/632: mknod d6/da/d2f/d69/cdb 0 2024-02-15T18:27:01.672 INFO:tasks.workunit.client.0.smithi042.stdout:5/647: unlink d0/d38/d53/d91/d94/fc0 0 2024-02-15T18:27:01.673 INFO:tasks.workunit.client.0.smithi042.stdout:4/582: dread da/d11/d29/d4a/d4f/f6c [0,4194304] 0 2024-02-15T18:27:01.674 INFO:tasks.workunit.client.1.smithi191.stdout:8/332: rmdir d3/d2c/d1a/d23/d2e/d5b/d77 0 2024-02-15T18:27:01.674 INFO:tasks.workunit.client.1.smithi191.stdout:7/250: creat d0/d1/d3/dc/d4b/f5b x:0 0 0 2024-02-15T18:27:01.674 INFO:tasks.workunit.client.1.smithi191.stdout:8/333: write d3/d2c/d1a/d23/d2e/d64/f76 [453140,86901] 0 2024-02-15T18:27:01.674 INFO:tasks.workunit.client.1.smithi191.stdout:8/334: chown d3/d2c/d1a/d23/l66 1548 1 2024-02-15T18:27:01.674 INFO:tasks.workunit.client.1.smithi191.stdout:8/335: write d3/d2c/d1a/d36/f6b [833202,31840] 0 2024-02-15T18:27:01.674 INFO:tasks.workunit.client.1.smithi191.stdout:8/336: readlink d3/d2c/d1a/l1e 0 2024-02-15T18:27:01.675 INFO:tasks.workunit.client.0.smithi042.stdout:3/597: rmdir d1/d11/d14/d4d/d6c/d7b 39 2024-02-15T18:27:01.677 INFO:tasks.workunit.client.0.smithi042.stdout:0/581: mkdir d4/d9/d34/d57/d88/dc9 0 2024-02-15T18:27:01.678 INFO:tasks.workunit.client.1.smithi191.stdout:3/178: mknod d6/d1b/c40 0 2024-02-15T18:27:01.678 INFO:tasks.workunit.client.1.smithi191.stdout:1/311: symlink d0/d8/df/d12/d2a/d2b/l64 0 2024-02-15T18:27:01.678 INFO:tasks.workunit.client.1.smithi191.stdout:0/154: dwrite d7/da/d17/d2f/f29 [4194304,4194304] 0 2024-02-15T18:27:01.678 INFO:tasks.workunit.client.1.smithi191.stdout:8/337: creat d3/d5e/f7a x:0 0 0 2024-02-15T18:27:01.679 INFO:tasks.workunit.client.0.smithi042.stdout:8/548: dwrite dd/de/d33/d66/f7f [0,4194304] 0 2024-02-15T18:27:01.679 INFO:tasks.workunit.client.0.smithi042.stdout:1/620: mkdir de/d54/dbf/dd0/d7c/dd6/de3 0 2024-02-15T18:27:01.680 INFO:tasks.workunit.client.1.smithi191.stdout:1/312: symlink d0/d8/df/d2e/l65 0 2024-02-15T18:27:01.680 INFO:tasks.workunit.client.1.smithi191.stdout:4/308: fsync d2/d5/d7/d2f/f51 0 2024-02-15T18:27:01.681 INFO:tasks.workunit.client.0.smithi042.stdout:6/691: rename d9/d11/d97/ce6 to d9/d11/d19/d35/d49/d79/cf3 0 2024-02-15T18:27:01.682 INFO:tasks.workunit.client.1.smithi191.stdout:3/179: creat d6/d1b/d1f/d21/f41 x:0 0 0 2024-02-15T18:27:01.682 INFO:tasks.workunit.client.1.smithi191.stdout:3/180: readlink d6/d3b/l14 0 2024-02-15T18:27:01.684 INFO:tasks.workunit.client.0.smithi042.stdout:9/633: link d6/dd/fe d6/dd/d9/da1/fdc 0 2024-02-15T18:27:01.684 INFO:tasks.workunit.client.0.smithi042.stdout:9/634: chown d6/da/d2f/d54 2 1 2024-02-15T18:27:01.684 INFO:tasks.workunit.client.0.smithi042.stdout:9/635: stat d6/f8 0 2024-02-15T18:27:01.684 INFO:tasks.workunit.client.0.smithi042.stdout:9/636: fdatasync d6/da/d1d/d2d/d40/f8c 0 2024-02-15T18:27:01.685 INFO:tasks.workunit.client.0.smithi042.stdout:5/648: symlink d0/dda/dc7/le3 0 2024-02-15T18:27:01.685 INFO:tasks.workunit.client.0.smithi042.stdout:5/649: write d0/dda/d42/f49 [425564,32068] 0 2024-02-15T18:27:01.685 INFO:tasks.workunit.client.0.smithi042.stdout:5/650: chown d0/d39/d8d/dd2 112 1 2024-02-15T18:27:01.686 INFO:tasks.workunit.client.1.smithi191.stdout:0/155: mkdir d7/d1b/d30 0 2024-02-15T18:27:01.687 INFO:tasks.workunit.client.1.smithi191.stdout:0/156: write d7/da/f20 [87681,108506] 0 2024-02-15T18:27:01.690 INFO:tasks.workunit.client.1.smithi191.stdout:9/218: dwrite d0/d7/dc/d17/d33/f4d [0,4194304] 0 2024-02-15T18:27:01.693 INFO:tasks.workunit.client.0.smithi042.stdout:4/583: creat da/d3a/dc9/fd2 x:0 0 0 2024-02-15T18:27:01.694 INFO:tasks.workunit.client.0.smithi042.stdout:4/584: truncate da/f63 624895 0 2024-02-15T18:27:01.694 INFO:tasks.workunit.client.1.smithi191.stdout:8/338: mknod d3/d2c/d1a/d23/d3e/c7b 0 2024-02-15T18:27:01.696 INFO:tasks.workunit.client.1.smithi191.stdout:3/181: dread d6/d3b/f13 [0,4194304] 0 2024-02-15T18:27:01.696 INFO:tasks.workunit.client.1.smithi191.stdout:3/182: dread - d6/d1b/f3e zero size 2024-02-15T18:27:01.696 INFO:tasks.workunit.client.1.smithi191.stdout:3/183: write d6/d1b/f39 [131788,16774] 0 2024-02-15T18:27:01.699 INFO:tasks.workunit.client.1.smithi191.stdout:1/313: mkdir d0/d2f/d66 0 2024-02-15T18:27:01.700 INFO:tasks.workunit.client.0.smithi042.stdout:0/582: creat d4/d9/d15/d92/fca x:0 0 0 2024-02-15T18:27:01.700 INFO:tasks.workunit.client.1.smithi191.stdout:4/309: getdents d2/d5/d7/d3e/d42/d52 0 2024-02-15T18:27:01.704 INFO:tasks.workunit.client.1.smithi191.stdout:0/157: rename d7/d22 to d7/da/d17/d31 0 2024-02-15T18:27:01.704 INFO:tasks.workunit.client.0.smithi042.stdout:8/549: mknod dd/de/d3c/ca0 0 2024-02-15T18:27:01.704 INFO:tasks.workunit.client.0.smithi042.stdout:8/550: chown dd/f29 4272347 1 2024-02-15T18:27:01.705 INFO:tasks.workunit.client.0.smithi042.stdout:8/551: truncate dd/de/d12/f9e 180399 0 2024-02-15T18:27:01.706 INFO:tasks.workunit.client.1.smithi191.stdout:6/243: dwrite d8/d9/f36 [0,4194304] 0 2024-02-15T18:27:01.706 INFO:tasks.workunit.client.1.smithi191.stdout:5/140: dwrite d0/d18/f1e [4194304,4194304] 0 2024-02-15T18:27:01.707 INFO:tasks.workunit.client.1.smithi191.stdout:5/141: chown d0/d18/l1c 4134121 1 2024-02-15T18:27:01.707 INFO:tasks.workunit.client.0.smithi042.stdout:1/621: rename de/d15/d67/f3b to de/d6d/d9e/dcd/fe4 0 2024-02-15T18:27:01.710 INFO:tasks.workunit.client.0.smithi042.stdout:7/648: dwrite d10/d15/f1d [0,4194304] 0 2024-02-15T18:27:01.711 INFO:tasks.workunit.client.0.smithi042.stdout:7/649: read d10/d12/d23/f61 [2777579,1821] 0 2024-02-15T18:27:01.711 INFO:tasks.workunit.client.0.smithi042.stdout:7/650: chown d10/d15/d39/l60 2837662 1 2024-02-15T18:27:01.711 INFO:tasks.workunit.client.0.smithi042.stdout:7/651: chown d10/d15/d8f/fb5 93482979 1 2024-02-15T18:27:01.711 INFO:tasks.workunit.client.0.smithi042.stdout:7/652: chown d10/d12/d23/d51/c62 175127 1 2024-02-15T18:27:01.712 INFO:tasks.workunit.client.1.smithi191.stdout:7/251: dwrite d0/d1/df/f19 [4194304,4194304] 0 2024-02-15T18:27:01.712 INFO:tasks.workunit.client.0.smithi042.stdout:6/692: getdents d9/d11/d1d 0 2024-02-15T18:27:01.712 INFO:tasks.workunit.client.0.smithi042.stdout:6/693: read - d9/d8d/ff0 zero size 2024-02-15T18:27:01.712 INFO:tasks.workunit.client.0.smithi042.stdout:6/694: chown d9/d11/d19/d1c/d31/f3a 1732 1 2024-02-15T18:27:01.712 INFO:tasks.workunit.client.0.smithi042.stdout:6/695: stat d9/d11/d1d/l78 0 2024-02-15T18:27:01.712 INFO:tasks.workunit.client.0.smithi042.stdout:6/696: read - d9/d11/d19/d1c/fc5 zero size 2024-02-15T18:27:01.713 INFO:tasks.workunit.client.1.smithi191.stdout:7/252: dread d0/d1/d3/f3e [0,4194304] 0 2024-02-15T18:27:01.713 INFO:tasks.workunit.client.1.smithi191.stdout:9/219: getdents d0/d7/dc 0 2024-02-15T18:27:01.717 INFO:tasks.workunit.client.1.smithi191.stdout:9/220: write d0/d7/dc/d17/d33/f4d [3948261,62790] 0 2024-02-15T18:27:01.717 INFO:tasks.workunit.client.1.smithi191.stdout:9/221: stat d0/d7/dc/d17/d33/f4d 0 2024-02-15T18:27:01.717 INFO:tasks.workunit.client.1.smithi191.stdout:9/222: write d0/d7/da/f1c [957,30902] 0 2024-02-15T18:27:01.719 INFO:tasks.workunit.client.1.smithi191.stdout:8/339: truncate d3/d2c/d1a/f20 4351335 0 2024-02-15T18:27:01.721 INFO:tasks.workunit.client.1.smithi191.stdout:8/340: dread d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/f60 [0,4194304] 0 2024-02-15T18:27:01.732 INFO:tasks.workunit.client.0.smithi042.stdout:6/697: dwrite d9/d11/d19/d35/d49/fea [0,4194304] 0 2024-02-15T18:27:01.733 INFO:tasks.workunit.client.0.smithi042.stdout:6/698: truncate d9/d11/d19/d1c/dcf/fdd 4495192 0 2024-02-15T18:27:01.733 INFO:tasks.workunit.client.0.smithi042.stdout:6/699: write d9/d11/d19/d35/d49/fd6 [791111,17078] 0 2024-02-15T18:27:01.735 INFO:tasks.workunit.client.0.smithi042.stdout:5/651: mkdir d0/dda/dc7/d52/da0/de4 0 2024-02-15T18:27:01.738 INFO:tasks.workunit.client.1.smithi191.stdout:1/314: truncate d0/d8/df/d12/f28 800277 0 2024-02-15T18:27:01.740 INFO:tasks.workunit.client.0.smithi042.stdout:4/585: link da/d11/f6d da/d11/d67/d5c/d66/d78/d2f/dca/fd3 0 2024-02-15T18:27:01.742 INFO:tasks.workunit.client.0.smithi042.stdout:5/652: dread d0/d39/f75 [0,4194304] 0 2024-02-15T18:27:01.742 INFO:tasks.workunit.client.1.smithi191.stdout:3/184: rename d6/d1b/d1c to d6/d3b/d42 0 2024-02-15T18:27:01.742 INFO:tasks.workunit.client.1.smithi191.stdout:3/185: chown d6/fd 103941 1 2024-02-15T18:27:01.744 INFO:tasks.workunit.client.1.smithi191.stdout:0/158: creat d7/da/d1a/d2c/f32 x:0 0 0 2024-02-15T18:27:01.744 INFO:tasks.workunit.client.1.smithi191.stdout:0/159: write d7/da/f18 [731595,31904] 0 2024-02-15T18:27:01.749 INFO:tasks.workunit.client.0.smithi042.stdout:3/598: getdents d1/d11/d14/d4d/d6c/d7b 0 2024-02-15T18:27:01.751 INFO:tasks.workunit.client.0.smithi042.stdout:0/583: symlink d4/d9/d5a/d60/lcb 0 2024-02-15T18:27:01.751 INFO:tasks.workunit.client.0.smithi042.stdout:0/584: chown d4/d38/d39/d8e/c76 576422054 1 2024-02-15T18:27:01.751 INFO:tasks.workunit.client.0.smithi042.stdout:0/585: fsync d4/d9/d62/d74/fc7 0 2024-02-15T18:27:01.752 INFO:tasks.workunit.client.0.smithi042.stdout:8/552: creat dd/d11/d75/fa1 x:0 0 0 2024-02-15T18:27:01.752 INFO:tasks.workunit.client.0.smithi042.stdout:8/553: chown dd/de/d94/f98 466098554 1 2024-02-15T18:27:01.757 INFO:tasks.workunit.client.1.smithi191.stdout:7/253: mknod d0/d1/d3/dc/c5c 0 2024-02-15T18:27:01.757 INFO:tasks.workunit.client.1.smithi191.stdout:7/254: truncate d0/d1/d35/d42/d46/f32 1018924 0 2024-02-15T18:27:01.758 INFO:tasks.workunit.client.0.smithi042.stdout:9/637: rename d6/da/d2f/d55/d73/d8d/fd3 to d6/da/d1d/d2d/d40/d95/fdd 0 2024-02-15T18:27:01.760 INFO:tasks.workunit.client.0.smithi042.stdout:1/622: mknod de/d54/d9a/dc8/ce5 0 2024-02-15T18:27:01.763 INFO:tasks.workunit.client.1.smithi191.stdout:9/223: mknod d0/d7/dc/d30/d4f/c54 0 2024-02-15T18:27:01.763 INFO:tasks.workunit.client.1.smithi191.stdout:9/224: chown d0/d1d/d3f/l53 43 1 2024-02-15T18:27:01.765 INFO:tasks.workunit.client.1.smithi191.stdout:8/341: unlink d3/d2c/d1a/f45 0 2024-02-15T18:27:01.766 INFO:tasks.workunit.client.0.smithi042.stdout:1/623: dread de/d15/d2c/d51/d70/fba [0,4194304] 0 2024-02-15T18:27:01.768 INFO:tasks.workunit.client.0.smithi042.stdout:6/700: mkdir d9/d11/d46/d89/df4 0 2024-02-15T18:27:01.768 INFO:tasks.workunit.client.0.smithi042.stdout:6/701: chown d9/d11/d19/l1f 257694 1 2024-02-15T18:27:01.773 INFO:tasks.workunit.client.0.smithi042.stdout:2/561: sync 2024-02-15T18:27:01.777 INFO:tasks.workunit.client.0.smithi042.stdout:4/586: mkdir da/d11/d29/d4a/d4f/dcf/dd4 0 2024-02-15T18:27:01.777 INFO:tasks.workunit.client.0.smithi042.stdout:4/587: read - da/d11/d29/d4a/fbe zero size 2024-02-15T18:27:01.778 INFO:tasks.workunit.client.1.smithi191.stdout:1/315: creat d0/d8/df/d12/d33/d49/f67 x:0 0 0 2024-02-15T18:27:01.778 INFO:tasks.workunit.client.1.smithi191.stdout:1/316: write d0/d8/df/d12/f5d [178474,113488] 0 2024-02-15T18:27:01.778 INFO:tasks.workunit.client.1.smithi191.stdout:1/317: dread - d0/d8/df/d12/d33/f63 zero size 2024-02-15T18:27:01.779 INFO:tasks.workunit.client.0.smithi042.stdout:3/599: symlink d1/d11/d7a/lc5 0 2024-02-15T18:27:01.779 INFO:tasks.workunit.client.0.smithi042.stdout:3/600: chown d1/d9c/cba 112 1 2024-02-15T18:27:01.780 INFO:tasks.workunit.client.1.smithi191.stdout:4/310: rename d2/d5/d14/f3a to d2/d5/d7/d3e/d63/f7d 0 2024-02-15T18:27:01.780 INFO:tasks.workunit.client.1.smithi191.stdout:4/311: dread - d2/d5/d7/d3e/d63/f7d zero size 2024-02-15T18:27:01.780 INFO:tasks.workunit.client.1.smithi191.stdout:4/312: write d2/d5/d7/d3e/d42/d52/f7b [795435,63898] 0 2024-02-15T18:27:01.781 INFO:tasks.workunit.client.0.smithi042.stdout:0/586: truncate d4/d9/d34/dab/dc8/d6c/d70/faa 87402 0 2024-02-15T18:27:01.781 INFO:tasks.workunit.client.0.smithi042.stdout:0/587: write d4/d9/d15/f8c [1768798,69096] 0 2024-02-15T18:27:01.781 INFO:tasks.workunit.client.1.smithi191.stdout:4/313: write d2/d5/d14/f2e [619703,18761] 0 2024-02-15T18:27:01.781 INFO:tasks.workunit.client.1.smithi191.stdout:4/314: dread - d2/d5/d7/d2f/f75 zero size 2024-02-15T18:27:01.781 INFO:tasks.workunit.client.1.smithi191.stdout:3/186: creat d6/d1b/d1f/f43 x:0 0 0 2024-02-15T18:27:01.782 INFO:tasks.workunit.client.0.smithi042.stdout:1/624: dwrite de/d15/d67/d63/f86 [0,4194304] 0 2024-02-15T18:27:01.784 INFO:tasks.workunit.client.1.smithi191.stdout:5/142: link d0/c15 d0/d23/c2d 0 2024-02-15T18:27:01.784 INFO:tasks.workunit.client.1.smithi191.stdout:5/143: chown d0/f1b 353381 1 2024-02-15T18:27:01.784 INFO:tasks.workunit.client.1.smithi191.stdout:5/144: chown d0/cd 1447885 1 2024-02-15T18:27:01.787 INFO:tasks.workunit.client.0.smithi042.stdout:7/653: rename d10/d15/d39/d6c/dcb to d10/d15/d39/dd9 0 2024-02-15T18:27:01.787 INFO:tasks.workunit.client.0.smithi042.stdout:7/654: chown d10/d49/d6d/ccc 83748707 1 2024-02-15T18:27:01.791 INFO:tasks.workunit.client.1.smithi191.stdout:7/255: mkdir d0/d1/df/d5d 0 2024-02-15T18:27:01.792 INFO:tasks.workunit.client.0.smithi042.stdout:9/638: mkdir d6/da/d1d/dde 0 2024-02-15T18:27:01.793 INFO:tasks.workunit.client.1.smithi191.stdout:9/225: creat d0/d7/dc/d30/d3b/f55 x:0 0 0 2024-02-15T18:27:01.794 INFO:tasks.workunit.client.0.smithi042.stdout:6/702: mknod d9/cf5 0 2024-02-15T18:27:01.798 INFO:tasks.workunit.client.1.smithi191.stdout:8/342: creat d3/d2c/d1a/d23/f7c x:0 0 0 2024-02-15T18:27:01.798 INFO:tasks.workunit.client.0.smithi042.stdout:2/562: creat d0/d8/d10/d31/da6/fad x:0 0 0 2024-02-15T18:27:01.799 INFO:tasks.workunit.client.1.smithi191.stdout:1/318: mkdir d0/d2f/d51/d5e/d68 0 2024-02-15T18:27:01.800 INFO:tasks.workunit.client.0.smithi042.stdout:4/588: link da/d3b/d41/d8d/fb1 da/d11/d67/fd5 0 2024-02-15T18:27:01.801 INFO:tasks.workunit.client.0.smithi042.stdout:3/601: mknod d1/d11/d14/dbf/cc6 0 2024-02-15T18:27:01.802 INFO:tasks.workunit.client.1.smithi191.stdout:4/315: truncate d2/d5/d7/f29 3612864 0 2024-02-15T18:27:01.802 INFO:tasks.workunit.client.1.smithi191.stdout:4/316: dread - d2/d5/d7/d3e/d42/d52/f54 zero size 2024-02-15T18:27:01.802 INFO:tasks.workunit.client.1.smithi191.stdout:4/317: truncate d2/d5/d7/d3e/d42/f6f 235408 0 2024-02-15T18:27:01.803 INFO:tasks.workunit.client.0.smithi042.stdout:0/588: getdents d4/d9/d34/d8b 0 2024-02-15T18:27:01.805 INFO:tasks.workunit.client.1.smithi191.stdout:3/187: rename d6/d3b/d42/l22 to d6/l44 0 2024-02-15T18:27:01.808 INFO:tasks.workunit.client.1.smithi191.stdout:1/319: dread d0/d2f/d51/d10/f25 [4194304,4194304] 0 2024-02-15T18:27:01.815 INFO:tasks.workunit.client.1.smithi191.stdout:5/145: creat d0/d18/f2e x:0 0 0 2024-02-15T18:27:01.816 INFO:tasks.workunit.client.1.smithi191.stdout:7/256: truncate d0/f8 6620436 0 2024-02-15T18:27:01.819 INFO:tasks.workunit.client.0.smithi042.stdout:1/625: mkdir de/d54/de6 0 2024-02-15T18:27:01.821 INFO:tasks.workunit.client.1.smithi191.stdout:9/226: unlink d0/d7/dc/d17/l4a 0 2024-02-15T18:27:01.824 INFO:tasks.workunit.client.0.smithi042.stdout:5/653: rename d0/d12/d65/daf to d0/d38/d53/d64/de5 0 2024-02-15T18:27:01.825 INFO:tasks.workunit.client.1.smithi191.stdout:3/188: unlink d6/fd 0 2024-02-15T18:27:01.826 INFO:tasks.workunit.client.0.smithi042.stdout:7/655: getdents d10/d15/d8f/da5 0 2024-02-15T18:27:01.827 INFO:tasks.workunit.client.1.smithi191.stdout:1/320: unlink d0/l20 0 2024-02-15T18:27:01.827 INFO:tasks.workunit.client.1.smithi191.stdout:1/321: write d0/f9 [1288419,19392] 0 2024-02-15T18:27:01.827 INFO:tasks.workunit.client.0.smithi042.stdout:9/639: stat d6/dd/l27 0 2024-02-15T18:27:01.828 INFO:tasks.workunit.client.0.smithi042.stdout:9/640: chown d6/da/d2f/d6c 86596330 1 2024-02-15T18:27:01.828 INFO:tasks.workunit.client.1.smithi191.stdout:5/146: symlink d0/d18/l2f 0 2024-02-15T18:27:01.829 INFO:tasks.workunit.client.0.smithi042.stdout:6/703: creat d9/d11/d19/d1c/d31/d53/ff6 x:0 0 0 2024-02-15T18:27:01.829 INFO:tasks.workunit.client.0.smithi042.stdout:3/602: dwrite d1/d11/d14/d4d/d73/f87 [0,4194304] 0 2024-02-15T18:27:01.830 INFO:tasks.workunit.client.1.smithi191.stdout:9/227: mkdir d0/d7/dc/d30/d3b/d56 0 2024-02-15T18:27:01.832 INFO:tasks.workunit.client.0.smithi042.stdout:2/563: creat d0/d8/d20/fae x:0 0 0 2024-02-15T18:27:01.836 INFO:tasks.workunit.client.0.smithi042.stdout:4/589: unlink da/d11/d67/d5c/d66/d78/d2f/f34 0 2024-02-15T18:27:01.836 INFO:tasks.workunit.client.1.smithi191.stdout:8/343: symlink d3/l7d 0 2024-02-15T18:27:01.837 INFO:tasks.workunit.client.1.smithi191.stdout:3/189: creat d6/d3b/d30/f45 x:0 0 0 2024-02-15T18:27:01.839 INFO:tasks.workunit.client.0.smithi042.stdout:0/589: creat d4/d9/d5a/fcc x:0 0 0 2024-02-15T18:27:01.839 INFO:tasks.workunit.client.1.smithi191.stdout:4/318: dwrite d2/f28 [0,4194304] 0 2024-02-15T18:27:01.840 INFO:tasks.workunit.client.1.smithi191.stdout:7/257: dwrite d0/d1/df/f1a [0,4194304] 0 2024-02-15T18:27:01.841 INFO:tasks.workunit.client.0.smithi042.stdout:1/626: link de/d15/d67/d46/f68 de/d15/d67/d89/fe7 0 2024-02-15T18:27:01.844 INFO:tasks.workunit.client.1.smithi191.stdout:1/322: creat d0/d8/f69 x:0 0 0 2024-02-15T18:27:01.846 INFO:tasks.workunit.client.1.smithi191.stdout:8/344: link d3/d2c/d1a/d23/d2e/c4e d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/c7e 0 2024-02-15T18:27:01.846 INFO:tasks.workunit.client.1.smithi191.stdout:8/345: truncate d3/d5e/f79 602253 0 2024-02-15T18:27:01.846 INFO:tasks.workunit.client.1.smithi191.stdout:8/346: dread - d3/d2c/f75 zero size 2024-02-15T18:27:01.847 INFO:tasks.workunit.client.1.smithi191.stdout:8/347: truncate d3/d2c/d1a/d23/d2e/f56 805645 0 2024-02-15T18:27:01.847 INFO:tasks.workunit.client.1.smithi191.stdout:8/348: chown d3/d2c/d1a 26220031 1 2024-02-15T18:27:01.847 INFO:tasks.workunit.client.1.smithi191.stdout:8/349: chown d3/d2c/d1a/f1b 123714 1 2024-02-15T18:27:01.848 INFO:tasks.workunit.client.0.smithi042.stdout:7/656: rmdir d10/d15/d39/d6c/d92/dd3 0 2024-02-15T18:27:01.849 INFO:tasks.workunit.client.1.smithi191.stdout:3/190: creat d6/d3b/d30/f46 x:0 0 0 2024-02-15T18:27:01.849 INFO:tasks.workunit.client.1.smithi191.stdout:3/191: fdatasync d6/d1b/d1f/f3d 0 2024-02-15T18:27:01.849 INFO:tasks.workunit.client.1.smithi191.stdout:3/192: truncate d6/d3b/f13 4757502 0 2024-02-15T18:27:01.850 INFO:tasks.workunit.client.0.smithi042.stdout:0/590: dread d4/d38/fb8 [0,4194304] 0 2024-02-15T18:27:01.851 INFO:tasks.workunit.client.1.smithi191.stdout:4/319: symlink d2/d5/d14/d44/d60/l7e 0 2024-02-15T18:27:01.854 INFO:tasks.workunit.client.0.smithi042.stdout:9/641: unlink d6/da/d2f/d69/f70 0 2024-02-15T18:27:01.854 INFO:tasks.workunit.client.0.smithi042.stdout:5/654: dread d0/d38/d53/d91/db0/fb8 [0,4194304] 0 2024-02-15T18:27:01.854 INFO:tasks.workunit.client.0.smithi042.stdout:6/704: mkdir d9/d11/d97/df7 0 2024-02-15T18:27:01.854 INFO:tasks.workunit.client.0.smithi042.stdout:5/655: chown d0/dda/dc7/le3 257171270 1 2024-02-15T18:27:01.854 INFO:tasks.workunit.client.0.smithi042.stdout:5/656: readlink d0/d22/d7b/da9/lac 0 2024-02-15T18:27:01.854 INFO:tasks.workunit.client.0.smithi042.stdout:5/657: rename d0/dda/dc7/d52/da0 to d0/dda/dc7/d52/da0/de4/de6 22 2024-02-15T18:27:01.859 INFO:tasks.workunit.client.1.smithi191.stdout:7/258: rename d0/d1/d3/d11/l1b to d0/l5e 0 2024-02-15T18:27:01.859 INFO:tasks.workunit.client.1.smithi191.stdout:7/259: dread - d0/db/f55 zero size 2024-02-15T18:27:01.860 INFO:tasks.workunit.client.1.smithi191.stdout:7/260: write d0/d1/d35/f50 [408140,71049] 0 2024-02-15T18:27:01.862 INFO:tasks.workunit.client.1.smithi191.stdout:8/350: link d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/c37 d3/d2c/d1a/d23/d2e/d47/c7f 0 2024-02-15T18:27:01.862 INFO:tasks.workunit.client.0.smithi042.stdout:2/564: dwrite d0/d8/d10/d7b/faa [0,4194304] 0 2024-02-15T18:27:01.864 INFO:tasks.workunit.client.0.smithi042.stdout:3/603: creat d1/d11/d14/d4d/d73/d89/da8/fc7 x:0 0 0 2024-02-15T18:27:01.864 INFO:tasks.workunit.client.1.smithi191.stdout:9/228: dwrite d0/d7/dc/d17/d33/f4d [4194304,4194304] 0 2024-02-15T18:27:01.865 INFO:tasks.workunit.client.1.smithi191.stdout:3/193: dread d6/f8 [0,4194304] 0 2024-02-15T18:27:01.865 INFO:tasks.workunit.client.1.smithi191.stdout:5/147: dwrite d0/d23/f27 [0,4194304] 0 2024-02-15T18:27:01.866 INFO:tasks.workunit.client.0.smithi042.stdout:4/590: link da/d11/d67/d5c/d66/fb0 da/d2b/fd6 0 2024-02-15T18:27:01.866 INFO:tasks.workunit.client.0.smithi042.stdout:4/591: chown da/d11/d67/d5c/d66/d78/d2f/d4d/cc1 654253 1 2024-02-15T18:27:01.866 INFO:tasks.workunit.client.0.smithi042.stdout:3/604: write d1/de/f2d [8342429,24598] 0 2024-02-15T18:27:01.867 INFO:tasks.workunit.client.0.smithi042.stdout:4/592: write da/d11/d67/d5c/d66/d78/d2f/d4d/f48 [292773,46115] 0 2024-02-15T18:27:01.868 INFO:tasks.workunit.client.0.smithi042.stdout:4/593: dread da/d11/d29/d4a/d4f/d90/fb2 [0,4194304] 0 2024-02-15T18:27:01.868 INFO:tasks.workunit.client.0.smithi042.stdout:4/594: chown da/d11/d67/d5c/d66/d78 111 1 2024-02-15T18:27:01.872 INFO:tasks.workunit.client.1.smithi191.stdout:3/194: rmdir d6/d1b 39 2024-02-15T18:27:01.872 INFO:tasks.workunit.client.1.smithi191.stdout:3/195: readlink d6/l44 0 2024-02-15T18:27:01.872 INFO:tasks.workunit.client.1.smithi191.stdout:3/196: write d6/fa [2666145,12335] 0 2024-02-15T18:27:01.874 INFO:tasks.workunit.client.1.smithi191.stdout:1/323: dwrite d0/f1b [0,4194304] 0 2024-02-15T18:27:01.874 INFO:tasks.workunit.client.1.smithi191.stdout:1/324: getdents d0/d2f/d51/d54/d5f/d60 0 2024-02-15T18:27:01.876 INFO:tasks.workunit.client.0.smithi042.stdout:1/627: mkdir de/d54/dbf/dd0/d7c/dd6/de8 0 2024-02-15T18:27:01.877 INFO:tasks.workunit.client.1.smithi191.stdout:9/229: mknod d0/d7/c57 0 2024-02-15T18:27:01.877 INFO:tasks.workunit.client.1.smithi191.stdout:9/230: truncate d0/f51 180934 0 2024-02-15T18:27:01.877 INFO:tasks.workunit.client.1.smithi191.stdout:9/231: getdents d0/d7/dc/d32 0 2024-02-15T18:27:01.877 INFO:tasks.workunit.client.1.smithi191.stdout:9/232: chown d0/d1d/d3f 17928 1 2024-02-15T18:27:01.879 INFO:tasks.workunit.client.0.smithi042.stdout:7/657: mkdir d10/d15/d8f/da5/dad/dda 0 2024-02-15T18:27:01.879 INFO:tasks.workunit.client.1.smithi191.stdout:5/148: truncate d0/f1b 1126268 0 2024-02-15T18:27:01.885 INFO:tasks.workunit.client.0.smithi042.stdout:0/591: link d4/d9/d34/d57/c7a d4/d38/d77/d2e/dae/ccd 0 2024-02-15T18:27:01.885 INFO:tasks.workunit.client.0.smithi042.stdout:0/592: chown d4/d38/d77/d2e 3346 1 2024-02-15T18:27:01.887 INFO:tasks.workunit.client.0.smithi042.stdout:5/658: creat d0/d14/d88/fe7 x:0 0 0 2024-02-15T18:27:01.887 INFO:tasks.workunit.client.1.smithi191.stdout:3/197: mknod d6/d1b/d1f/c47 0 2024-02-15T18:27:01.887 INFO:tasks.workunit.client.1.smithi191.stdout:3/198: stat d6/d3b/c19 0 2024-02-15T18:27:01.890 INFO:tasks.workunit.client.1.smithi191.stdout:4/320: dwrite d2/d5/d7/d2f/f31 [0,4194304] 0 2024-02-15T18:27:01.891 INFO:tasks.workunit.client.1.smithi191.stdout:9/233: rename d0/d7/dc/f2b to d0/d1d/f58 0 2024-02-15T18:27:01.896 INFO:tasks.workunit.client.1.smithi191.stdout:8/351: dwrite d3/f12 [0,4194304] 0 2024-02-15T18:27:01.896 INFO:tasks.workunit.client.0.smithi042.stdout:8/554: sync 2024-02-15T18:27:01.897 INFO:tasks.workunit.client.0.smithi042.stdout:8/555: fdatasync dd/de/d3c/d4b/d81/f8c 0 2024-02-15T18:27:01.897 INFO:tasks.workunit.client.0.smithi042.stdout:2/565: getdents d0/d8 0 2024-02-15T18:27:01.899 INFO:tasks.workunit.client.1.smithi191.stdout:5/149: mknod d0/d18/c30 0 2024-02-15T18:27:01.900 INFO:tasks.workunit.client.0.smithi042.stdout:3/605: write d1/de/d1a/f2f [486461,32833] 0 2024-02-15T18:27:01.901 INFO:tasks.workunit.client.0.smithi042.stdout:3/606: stat d1/d11/d2e/d4c/f58 0 2024-02-15T18:27:01.901 INFO:tasks.workunit.client.1.smithi191.stdout:7/261: dwrite d0/d1/d3/dc/d4b/f59 [0,4194304] 0 2024-02-15T18:27:01.901 INFO:tasks.workunit.client.0.smithi042.stdout:4/595: creat da/d3a/dc9/fd7 x:0 0 0 2024-02-15T18:27:01.902 INFO:tasks.workunit.client.0.smithi042.stdout:4/596: write da/d11/d67/d5c/d66/d78/d2f/d4d/f47 [1033891,35653] 0 2024-02-15T18:27:01.903 INFO:tasks.workunit.client.1.smithi191.stdout:7/262: read d0/d1/df/f16 [206066,80731] 0 2024-02-15T18:27:01.903 INFO:tasks.workunit.client.1.smithi191.stdout:7/263: dread - d0/d1/d35/d42/d46/f53 zero size 2024-02-15T18:27:01.903 INFO:tasks.workunit.client.1.smithi191.stdout:7/264: fsync d0/d1/f51 0 2024-02-15T18:27:01.903 INFO:tasks.workunit.client.0.smithi042.stdout:9/642: dwrite d6/dd/f2b [0,4194304] 0 2024-02-15T18:27:01.904 INFO:tasks.workunit.client.1.smithi191.stdout:5/150: write d0/d23/f27 [150405,41662] 0 2024-02-15T18:27:01.907 INFO:tasks.workunit.client.1.smithi191.stdout:3/199: dwrite d6/d3b/d30/f3c [0,4194304] 0 2024-02-15T18:27:01.913 INFO:tasks.workunit.client.0.smithi042.stdout:1/628: dwrite de/d54/fb0 [0,4194304] 0 2024-02-15T18:27:01.913 INFO:tasks.workunit.client.0.smithi042.stdout:1/629: chown de/d15/c30 2180 1 2024-02-15T18:27:01.917 INFO:tasks.workunit.client.1.smithi191.stdout:5/151: dread d0/d18/f1e [4194304,4194304] 0 2024-02-15T18:27:01.918 INFO:tasks.workunit.client.1.smithi191.stdout:4/321: mknod d2/d5/d7/d2f/c7f 0 2024-02-15T18:27:01.918 INFO:tasks.workunit.client.1.smithi191.stdout:4/322: truncate d2/d5/d14/d47/d78/f61 780244 0 2024-02-15T18:27:01.919 INFO:tasks.workunit.client.0.smithi042.stdout:0/593: mknod d4/d9/cce 0 2024-02-15T18:27:01.920 INFO:tasks.workunit.client.1.smithi191.stdout:9/234: mkdir d0/d7/dc/d30/d3b/d56/d59 0 2024-02-15T18:27:01.922 INFO:tasks.workunit.client.0.smithi042.stdout:8/556: rename dd/de/d33/d66/f52 to dd/de/d12/fa2 0 2024-02-15T18:27:01.924 INFO:tasks.workunit.client.0.smithi042.stdout:3/607: mknod d1/de/d19/cc8 0 2024-02-15T18:27:01.926 INFO:tasks.workunit.client.1.smithi191.stdout:8/352: creat d3/d5e/f80 x:0 0 0 2024-02-15T18:27:01.926 INFO:tasks.workunit.client.1.smithi191.stdout:8/353: chown d3/d5e/f7a 1168590 1 2024-02-15T18:27:01.927 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:27:00.681796+0000 mgr.smithi042.kfoihb (mgr.24435) 39 : cluster [DBG] pgmap v29: 65 pgs: 65 active+clean; 1.2 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 66 MiB/s rd, 142 MiB/s wr, 401 op/s 2024-02-15T18:27:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:8/557: dread dd/de/d3c/d4b/f8d [0,4194304] 0 2024-02-15T18:27:01.928 INFO:tasks.workunit.client.0.smithi042.stdout:8/558: chown dd/de/d33/d3b/d92/l96 629487 1 2024-02-15T18:27:01.928 INFO:tasks.workunit.client.0.smithi042.stdout:2/566: dwrite d0/f71 [0,4194304] 0 2024-02-15T18:27:01.928 INFO:tasks.workunit.client.1.smithi191.stdout:8/354: dread d3/d2c/d1a/d23/d2e/d64/f76 [0,4194304] 0 2024-02-15T18:27:01.929 INFO:tasks.workunit.client.1.smithi191.stdout:8/355: fdatasync d3/d5e/f80 0 2024-02-15T18:27:01.931 INFO:tasks.workunit.client.1.smithi191.stdout:3/200: dwrite d6/d3b/ff [4194304,4194304] 0 2024-02-15T18:27:01.932 INFO:tasks.workunit.client.0.smithi042.stdout:7/658: symlink d10/d43/ldb 0 2024-02-15T18:27:01.933 INFO:tasks.workunit.client.0.smithi042.stdout:7/659: chown d10/f73 2466950 1 2024-02-15T18:27:01.933 INFO:tasks.workunit.client.0.smithi042.stdout:0/594: creat d4/d9/d34/d57/d88/fcf x:0 0 0 2024-02-15T18:27:01.935 INFO:tasks.workunit.client.0.smithi042.stdout:4/597: rename da/fad to da/d2b/fd8 0 2024-02-15T18:27:01.935 INFO:tasks.workunit.client.1.smithi191.stdout:7/265: mkdir d0/db/d26/d5f 0 2024-02-15T18:27:01.937 INFO:tasks.workunit.client.0.smithi042.stdout:3/608: creat d1/de/d19/fc9 x:0 0 0 2024-02-15T18:27:01.938 INFO:tasks.workunit.client.0.smithi042.stdout:8/559: creat dd/de/d3c/d4b/fa3 x:0 0 0 2024-02-15T18:27:01.940 INFO:tasks.workunit.client.0.smithi042.stdout:2/567: symlink d0/d8/db/de/d1a/d2e/laf 0 2024-02-15T18:27:01.942 INFO:tasks.workunit.client.0.smithi042.stdout:0/595: symlink d4/d9/d15/ld0 0 2024-02-15T18:27:01.942 INFO:tasks.workunit.client.0.smithi042.stdout:0/596: truncate d4/d38/d5c/fa2 370776 0 2024-02-15T18:27:01.942 INFO:tasks.workunit.client.0.smithi042.stdout:0/597: stat d4/d9/f46 0 2024-02-15T18:27:01.943 INFO:tasks.workunit.client.0.smithi042.stdout:4/598: creat da/d3a/dc9/fd9 x:0 0 0 2024-02-15T18:27:01.943 INFO:tasks.workunit.client.0.smithi042.stdout:4/599: dread - da/d11/d67/d5c/d66/d78/d2f/fd1 zero size 2024-02-15T18:27:01.945 INFO:tasks.workunit.client.0.smithi042.stdout:3/609: mknod d1/d11/cca 0 2024-02-15T18:27:01.945 INFO:tasks.workunit.client.0.smithi042.stdout:7/660: dread d10/f69 [0,4194304] 0 2024-02-15T18:27:01.947 INFO:tasks.workunit.client.0.smithi042.stdout:3/610: symlink d1/d11/d2e/d4c/lcb 0 2024-02-15T18:27:01.947 INFO:tasks.workunit.client.0.smithi042.stdout:3/611: chown d1/de/d19/fc9 10 1 2024-02-15T18:27:01.950 INFO:tasks.workunit.client.0.smithi042.stdout:7/661: creat d10/d49/d6d/fdc x:0 0 0 2024-02-15T18:27:01.950 INFO:tasks.workunit.client.0.smithi042.stdout:4/600: write da/d11/d27/f89 [2855665,5588] 0 2024-02-15T18:27:01.952 INFO:tasks.workunit.client.1.smithi191.stdout:4/323: creat d2/d5/d7/f80 x:0 0 0 2024-02-15T18:27:01.953 INFO:tasks.workunit.client.1.smithi191.stdout:9/235: unlink d0/d29/l3a 0 2024-02-15T18:27:01.955 INFO:tasks.workunit.client.1.smithi191.stdout:9/236: write d0/f3c [2771251,22132] 0 2024-02-15T18:27:01.955 INFO:tasks.workunit.client.0.smithi042.stdout:7/662: creat d10/d12/d23/fdd x:0 0 0 2024-02-15T18:27:01.956 INFO:tasks.workunit.client.1.smithi191.stdout:8/356: creat d3/d2c/d1a/f81 x:0 0 0 2024-02-15T18:27:01.959 INFO:tasks.workunit.client.0.smithi042.stdout:4/601: rename da/d11/d29/d4a/d4f/d90/fb2 to da/d3b/fda 0 2024-02-15T18:27:01.959 INFO:tasks.workunit.client.0.smithi042.stdout:4/602: readlink da/d11/d67/d5c/d66/d78/d2f/l31 0 2024-02-15T18:27:01.959 INFO:tasks.workunit.client.0.smithi042.stdout:4/603: write da/d2b/d7e/fa6 [323466,105147] 0 2024-02-15T18:27:01.959 INFO:tasks.workunit.client.1.smithi191.stdout:3/201: link d6/d3b/c29 d6/d27/c48 0 2024-02-15T18:27:01.960 INFO:tasks.workunit.client.1.smithi191.stdout:7/266: mknod d0/db/d26/d5f/c60 0 2024-02-15T18:27:01.962 INFO:tasks.workunit.client.0.smithi042.stdout:7/663: write fa [1801840,84293] 0 2024-02-15T18:27:01.962 INFO:tasks.workunit.client.0.smithi042.stdout:7/664: chown d10/d49/d6d/fbb 77991 1 2024-02-15T18:27:01.962 INFO:tasks.workunit.client.1.smithi191.stdout:8/357: mkdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82 0 2024-02-15T18:27:01.963 INFO:tasks.workunit.client.1.smithi191.stdout:3/202: rename d6/d1b/d1f to d6/d3b/d42/d49 0 2024-02-15T18:27:01.964 INFO:tasks.workunit.client.1.smithi191.stdout:3/203: write d6/fa [219713,97056] 0 2024-02-15T18:27:01.965 INFO:tasks.workunit.client.0.smithi042.stdout:7/665: rename d10/d15/d39/d6c/f8d to d10/d15/d8f/da5/fde 0 2024-02-15T18:27:01.967 INFO:tasks.workunit.client.1.smithi191.stdout:9/237: mknod d0/c5a 0 2024-02-15T18:27:01.971 INFO:tasks.workunit.client.1.smithi191.stdout:3/204: link d6/d3b/d30/f46 d6/d3b/d42/f4a 0 2024-02-15T18:27:01.971 INFO:tasks.workunit.client.1.smithi191.stdout:7/267: rename d0/f14 to d0/d1/d35/d42/d46/f61 0 2024-02-15T18:27:01.971 INFO:tasks.workunit.client.1.smithi191.stdout:3/205: readlink d6/d3b/d42/l24 0 2024-02-15T18:27:01.971 INFO:tasks.workunit.client.1.smithi191.stdout:3/206: write d6/fa [4850164,39000] 0 2024-02-15T18:27:01.974 INFO:tasks.workunit.client.1.smithi191.stdout:9/238: dread d0/d7/fd [0,4194304] 0 2024-02-15T18:27:01.976 INFO:tasks.workunit.client.1.smithi191.stdout:3/207: mknod d6/d27/c4b 0 2024-02-15T18:27:01.976 INFO:tasks.workunit.client.1.smithi191.stdout:3/208: chown d6/c1e 83735744 1 2024-02-15T18:27:01.982 INFO:tasks.workunit.client.1.smithi191.stdout:9/239: creat d0/d29/d48/f5b x:0 0 0 2024-02-15T18:27:01.982 INFO:tasks.workunit.client.1.smithi191.stdout:9/240: readlink d0/l16 0 2024-02-15T18:27:01.982 INFO:tasks.workunit.client.1.smithi191.stdout:9/241: stat d0/d7/dc 0 2024-02-15T18:27:01.986 INFO:tasks.workunit.client.1.smithi191.stdout:3/209: dread d6/d27/f3f [0,4194304] 0 2024-02-15T18:27:01.987 INFO:tasks.workunit.client.0.smithi042.stdout:9/643: dwrite d6/dd/d9/f18 [0,4194304] 0 2024-02-15T18:27:01.989 INFO:tasks.workunit.client.1.smithi191.stdout:8/358: dread d3/fc [0,4194304] 0 2024-02-15T18:27:01.989 INFO:tasks.workunit.client.1.smithi191.stdout:8/359: fdatasync d3/d2c/d1a/d23/f6a 0 2024-02-15T18:27:01.990 INFO:tasks.workunit.client.1.smithi191.stdout:9/242: dread d0/f51 [0,4194304] 0 2024-02-15T18:27:01.996 INFO:tasks.workunit.client.1.smithi191.stdout:3/210: rename d6/d3b/d30/c36 to d6/c4c 0 2024-02-15T18:27:01.998 INFO:tasks.workunit.client.1.smithi191.stdout:9/243: creat d0/d7/dc/d30/f5c x:0 0 0 2024-02-15T18:27:01.998 INFO:tasks.workunit.client.1.smithi191.stdout:9/244: getdents d0/d7/dc/d17/d44 0 2024-02-15T18:27:01.999 INFO:tasks.workunit.client.0.smithi042.stdout:2/568: dwrite d0/d40/d56/f8d [0,4194304] 0 2024-02-15T18:27:01.999 INFO:tasks.workunit.client.0.smithi042.stdout:8/560: dwrite dd/f29 [0,4194304] 0 2024-02-15T18:27:01.999 INFO:tasks.workunit.client.1.smithi191.stdout:8/360: write d3/fc [2901685,25145] 0 2024-02-15T18:27:02.000 INFO:tasks.workunit.client.1.smithi191.stdout:4/324: dwrite d2/d5/d14/d47/d78/f4f [0,4194304] 0 2024-02-15T18:27:02.001 INFO:tasks.workunit.client.1.smithi191.stdout:3/211: mkdir d6/d27/d4d 0 2024-02-15T18:27:02.003 INFO:tasks.workunit.client.1.smithi191.stdout:4/325: dread d2/f5c [0,4194304] 0 2024-02-15T18:27:02.007 INFO:tasks.workunit.client.1.smithi191.stdout:7/268: dwrite d0/d1/d3/dc/d4b/f59 [0,4194304] 0 2024-02-15T18:27:02.011 INFO:tasks.workunit.client.1.smithi191.stdout:5/152: dwrite d0/d23/f27 [4194304,4194304] 0 2024-02-15T18:27:02.012 INFO:tasks.workunit.client.0.smithi042.stdout:1/630: dwrite de/d15/f31 [0,4194304] 0 2024-02-15T18:27:02.013 INFO:tasks.workunit.client.0.smithi042.stdout:0/598: dwrite d4/d9/d15/f21 [0,4194304] 0 2024-02-15T18:27:02.013 INFO:tasks.workunit.client.0.smithi042.stdout:1/631: readlink de/d15/d67/d74/d7e/l9f 0 2024-02-15T18:27:02.013 INFO:tasks.workunit.client.0.smithi042.stdout:7/666: dwrite d10/d12/f9e [0,4194304] 0 2024-02-15T18:27:02.020 INFO:tasks.workunit.client.1.smithi191.stdout:8/361: dread d3/d2c/d1a/d23/d2e/f3a [0,4194304] 0 2024-02-15T18:27:02.021 INFO:tasks.workunit.client.1.smithi191.stdout:8/362: stat d3/d2c/d1a/d23/f6a 0 2024-02-15T18:27:02.022 INFO:tasks.workunit.client.1.smithi191.stdout:2/241: sync 2024-02-15T18:27:02.040 INFO:tasks.workunit.client.1.smithi191.stdout:9/245: creat d0/d7/da/d39/f5d x:0 0 0 2024-02-15T18:27:02.047 INFO:tasks.workunit.client.0.smithi042.stdout:2/569: rename d0/d8/d10/d31/d3a/d53/c96 to d0/d8/d10/d31/d3a/d53/cb0 0 2024-02-15T18:27:02.051 INFO:tasks.workunit.client.0.smithi042.stdout:6/705: sync 2024-02-15T18:27:02.052 INFO:tasks.workunit.client.0.smithi042.stdout:8/561: unlink dd/de/d33/d3b/c71 0 2024-02-15T18:27:02.053 INFO:tasks.workunit.client.1.smithi191.stdout:4/326: rmdir d2/d5/d7 39 2024-02-15T18:27:02.053 INFO:tasks.workunit.client.1.smithi191.stdout:4/327: read d2/d5/d7/d3e/d42/d52/f53 [44350,57265] 0 2024-02-15T18:27:02.057 INFO:tasks.workunit.client.1.smithi191.stdout:7/269: rmdir d0/d1/d3f 0 2024-02-15T18:27:02.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:00.681796+0000 mgr.smithi042.kfoihb (mgr.24435) 39 : cluster [DBG] pgmap v29: 65 pgs: 65 active+clean; 1.2 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 66 MiB/s rd, 142 MiB/s wr, 401 op/s 2024-02-15T18:27:02.061 INFO:tasks.workunit.client.0.smithi042.stdout:0/599: rename d4/d9/d62/l86 to d4/d9/d34/d57/dc1/ld1 0 2024-02-15T18:27:02.061 INFO:tasks.workunit.client.0.smithi042.stdout:0/600: readlink d4/d9/d8d/lbe 0 2024-02-15T18:27:02.062 INFO:tasks.workunit.client.1.smithi191.stdout:4/328: dread d2/d5/d7/f18 [0,4194304] 0 2024-02-15T18:27:02.062 INFO:tasks.workunit.client.1.smithi191.stdout:7/270: write d0/d1/df/f19 [2645426,6139] 0 2024-02-15T18:27:02.063 INFO:tasks.workunit.client.1.smithi191.stdout:4/329: write d2/d5/d7/f6c [1663783,56841] 0 2024-02-15T18:27:02.063 INFO:tasks.workunit.client.1.smithi191.stdout:5/153: mkdir d0/d14/d28/d31 0 2024-02-15T18:27:02.064 INFO:tasks.workunit.client.1.smithi191.stdout:5/154: write d0/f13 [2853949,82165] 0 2024-02-15T18:27:02.064 INFO:tasks.workunit.client.1.smithi191.stdout:5/155: write d0/f1f [1215247,69101] 0 2024-02-15T18:27:02.064 INFO:tasks.workunit.client.1.smithi191.stdout:5/156: write d0/f1a [4922100,65778] 0 2024-02-15T18:27:02.064 INFO:tasks.workunit.client.1.smithi191.stdout:5/157: truncate d0/d18/f2e 224703 0 2024-02-15T18:27:02.065 INFO:tasks.workunit.client.0.smithi042.stdout:1/632: unlink de/d54/dbf/dd0/d7c/f9c 0 2024-02-15T18:27:02.067 INFO:tasks.workunit.client.1.smithi191.stdout:8/363: unlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/l6f 0 2024-02-15T18:27:02.069 INFO:tasks.workunit.client.0.smithi042.stdout:7/667: unlink d10/d12/f6b 0 2024-02-15T18:27:02.072 INFO:tasks.workunit.client.1.smithi191.stdout:2/242: symlink df/l54 0 2024-02-15T18:27:02.073 INFO:tasks.workunit.client.1.smithi191.stdout:2/243: stat f9 0 2024-02-15T18:27:02.074 INFO:tasks.workunit.client.1.smithi191.stdout:9/246: mknod d0/d7/dc/d30/c5e 0 2024-02-15T18:27:02.074 INFO:tasks.workunit.client.1.smithi191.stdout:9/247: fsync d0/d7/dc/d30/f5c 0 2024-02-15T18:27:02.076 INFO:tasks.workunit.client.0.smithi042.stdout:2/570: symlink d0/d8/d20/d3f/d7c/lb1 0 2024-02-15T18:27:02.076 INFO:tasks.workunit.client.0.smithi042.stdout:2/571: fsync d0/d8/db/de/d1a/f6b 0 2024-02-15T18:27:02.077 INFO:tasks.workunit.client.1.smithi191.stdout:7/271: symlink d0/d1/df/d5d/l62 0 2024-02-15T18:27:02.077 INFO:tasks.workunit.client.1.smithi191.stdout:7/272: chown d0/d1/d35/l4f 44 1 2024-02-15T18:27:02.080 INFO:tasks.workunit.client.0.smithi042.stdout:6/706: mkdir d9/d11/d46/df8 0 2024-02-15T18:27:02.080 INFO:tasks.workunit.client.1.smithi191.stdout:8/364: dread d3/d2c/d1a/d23/f6a [0,4194304] 0 2024-02-15T18:27:02.081 INFO:tasks.workunit.client.0.smithi042.stdout:5/659: sync 2024-02-15T18:27:02.081 INFO:tasks.workunit.client.0.smithi042.stdout:3/612: sync 2024-02-15T18:27:02.081 INFO:tasks.workunit.client.0.smithi042.stdout:9/644: sync 2024-02-15T18:27:02.081 INFO:tasks.workunit.client.0.smithi042.stdout:4/604: sync 2024-02-15T18:27:02.081 INFO:tasks.workunit.client.0.smithi042.stdout:5/660: fdatasync d0/dda/d42/f49 0 2024-02-15T18:27:02.081 INFO:tasks.workunit.client.0.smithi042.stdout:3/613: read - d1/de/d19/fc9 zero size 2024-02-15T18:27:02.081 INFO:tasks.workunit.client.0.smithi042.stdout:4/605: write da/d11/d67/d5c/d66/fa3 [287435,104277] 0 2024-02-15T18:27:02.081 INFO:tasks.workunit.client.0.smithi042.stdout:4/606: read da/d11/d29/f32 [404631,130860] 0 2024-02-15T18:27:02.082 INFO:tasks.workunit.client.0.smithi042.stdout:3/614: write d1/d11/d14/f53 [1800348,54277] 0 2024-02-15T18:27:02.082 INFO:tasks.workunit.client.1.smithi191.stdout:4/330: creat d2/f81 x:0 0 0 2024-02-15T18:27:02.082 INFO:tasks.workunit.client.1.smithi191.stdout:4/331: chown d2/d5/d7/l5d 19624 1 2024-02-15T18:27:02.086 INFO:tasks.workunit.client.0.smithi042.stdout:8/562: rename dd/de/d3c/d4b to dd/de/da4 0 2024-02-15T18:27:02.086 INFO:tasks.workunit.client.1.smithi191.stdout:5/158: mkdir d0/d14/d32 0 2024-02-15T18:27:02.086 INFO:tasks.workunit.client.0.smithi042.stdout:0/601: creat d4/d9/d15/d92/fd2 x:0 0 0 2024-02-15T18:27:02.087 INFO:tasks.workunit.client.1.smithi191.stdout:9/248: creat d0/d7/dc/d30/f5f x:0 0 0 2024-02-15T18:27:02.088 INFO:tasks.workunit.client.0.smithi042.stdout:1/633: creat de/d15/d2c/d51/d70/fe9 x:0 0 0 2024-02-15T18:27:02.088 INFO:tasks.workunit.client.0.smithi042.stdout:1/634: write de/d15/d67/d74/d7e/fd5 [474183,73322] 0 2024-02-15T18:27:02.088 INFO:tasks.workunit.client.0.smithi042.stdout:9/645: dread d6/da/d2f/f56 [0,4194304] 0 2024-02-15T18:27:02.088 INFO:tasks.workunit.client.0.smithi042.stdout:9/646: chown d6/da/d1d/d2d/d40/d5f/lab 0 1 2024-02-15T18:27:02.090 INFO:tasks.workunit.client.1.smithi191.stdout:2/244: dwrite df/d12/d19/d1a/f3d [0,4194304] 0 2024-02-15T18:27:02.095 INFO:tasks.workunit.client.1.smithi191.stdout:7/273: creat d0/d1/d35/d42/d46/d1e/f63 x:0 0 0 2024-02-15T18:27:02.097 INFO:tasks.workunit.client.0.smithi042.stdout:7/668: link d10/d15/d39/d6c/dba/fcd d10/d15/d39/dd9/fdf 0 2024-02-15T18:27:02.100 INFO:tasks.workunit.client.1.smithi191.stdout:8/365: creat d3/d2c/d1a/d23/d2e/d5b/d69/d6d/f83 x:0 0 0 2024-02-15T18:27:02.101 INFO:tasks.workunit.client.0.smithi042.stdout:1/635: dread de/d15/d67/fc4 [0,4194304] 0 2024-02-15T18:27:02.101 INFO:tasks.workunit.client.0.smithi042.stdout:1/636: chown de/d54/l65 112 1 2024-02-15T18:27:02.101 INFO:tasks.workunit.client.0.smithi042.stdout:1/637: chown de/d54/d9a/dc8/ce5 94570 1 2024-02-15T18:27:02.101 INFO:tasks.workunit.client.0.smithi042.stdout:6/707: rmdir d9/d11/d19/d1c/d31/d9b/db4/df2 0 2024-02-15T18:27:02.102 INFO:tasks.workunit.client.0.smithi042.stdout:6/708: dread - d9/d11/d1d/fd1 zero size 2024-02-15T18:27:02.102 INFO:tasks.workunit.client.0.smithi042.stdout:4/607: dwrite da/d2b/f9e [0,4194304] 0 2024-02-15T18:27:02.102 INFO:tasks.workunit.client.1.smithi191.stdout:7/274: fdatasync d0/f8 0 2024-02-15T18:27:02.103 INFO:tasks.workunit.client.1.smithi191.stdout:4/332: mknod d2/c82 0 2024-02-15T18:27:02.103 INFO:tasks.workunit.client.1.smithi191.stdout:4/333: stat d2/d5/d7/d2f/f51 0 2024-02-15T18:27:02.103 INFO:tasks.workunit.client.0.smithi042.stdout:5/661: creat d0/d12/d2a/fe8 x:0 0 0 2024-02-15T18:27:02.105 INFO:tasks.workunit.client.1.smithi191.stdout:5/159: creat d0/d14/d28/d31/f33 x:0 0 0 2024-02-15T18:27:02.107 INFO:tasks.workunit.client.1.smithi191.stdout:9/249: mkdir d0/d7/dc/d30/d3b/d56/d60 0 2024-02-15T18:27:02.109 INFO:tasks.workunit.client.0.smithi042.stdout:8/563: unlink dd/f2f 0 2024-02-15T18:27:02.112 INFO:tasks.workunit.client.0.smithi042.stdout:0/602: creat d4/dc2/fd3 x:0 0 0 2024-02-15T18:27:02.113 INFO:tasks.workunit.client.1.smithi191.stdout:2/245: creat df/d12/f55 x:0 0 0 2024-02-15T18:27:02.115 INFO:tasks.workunit.client.1.smithi191.stdout:8/366: symlink d3/d5e/l84 0 2024-02-15T18:27:02.115 INFO:tasks.workunit.client.1.smithi191.stdout:8/367: chown d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/l4a 107779886 1 2024-02-15T18:27:02.117 INFO:tasks.workunit.client.0.smithi042.stdout:9/647: dwrite d6/da/d2f/d69/f8b [0,4194304] 0 2024-02-15T18:27:02.117 INFO:tasks.workunit.client.0.smithi042.stdout:9/648: write d6/da/d2f/d54/d96/fd7 [555026,27130] 0 2024-02-15T18:27:02.118 INFO:tasks.workunit.client.0.smithi042.stdout:9/649: read d6/dd/d9/d59/f63 [3845895,113344] 0 2024-02-15T18:27:02.118 INFO:tasks.workunit.client.0.smithi042.stdout:6/709: mkdir d9/d11/d19/d1c/d31/dd2/df9 0 2024-02-15T18:27:02.118 INFO:tasks.workunit.client.1.smithi191.stdout:8/368: write d3/d2c/d1a/f1b [1343998,83593] 0 2024-02-15T18:27:02.118 INFO:tasks.workunit.client.1.smithi191.stdout:7/275: rename d0/d1/d35/d42/d46/d1e/f33 to d0/d1/df/f64 0 2024-02-15T18:27:02.119 INFO:tasks.workunit.client.1.smithi191.stdout:7/276: write d0/d1/d35/d42/d46/f53 [740400,80669] 0 2024-02-15T18:27:02.119 INFO:tasks.workunit.client.0.smithi042.stdout:9/650: dread d6/da2/fac [0,4194304] 0 2024-02-15T18:27:02.120 INFO:tasks.workunit.client.1.smithi191.stdout:5/160: rmdir d0/d18 39 2024-02-15T18:27:02.121 INFO:tasks.workunit.client.0.smithi042.stdout:4/608: mkdir da/d11/d67/d5c/d66/d78/ddb 0 2024-02-15T18:27:02.123 INFO:tasks.workunit.client.1.smithi191.stdout:9/250: rmdir d0/d1d 39 2024-02-15T18:27:02.124 INFO:tasks.workunit.client.0.smithi042.stdout:5/662: creat d0/d38/d53/db4/fe9 x:0 0 0 2024-02-15T18:27:02.125 INFO:tasks.workunit.client.1.smithi191.stdout:2/246: symlink df/d12/d1f/l56 0 2024-02-15T18:27:02.126 INFO:tasks.workunit.client.1.smithi191.stdout:2/247: chown df/d12/d1f 127398989 1 2024-02-15T18:27:02.126 INFO:tasks.workunit.client.0.smithi042.stdout:0/603: creat d4/d9/d34/d8b/fd4 x:0 0 0 2024-02-15T18:27:02.128 INFO:tasks.workunit.client.1.smithi191.stdout:8/369: symlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/l85 0 2024-02-15T18:27:02.129 INFO:tasks.workunit.client.1.smithi191.stdout:8/370: write d3/f29 [229844,106006] 0 2024-02-15T18:27:02.129 INFO:tasks.workunit.client.0.smithi042.stdout:8/564: dread dd/de/d12/f1e [0,4194304] 0 2024-02-15T18:27:02.130 INFO:tasks.workunit.client.0.smithi042.stdout:8/565: write dd/f3e [497544,88357] 0 2024-02-15T18:27:02.134 INFO:tasks.workunit.client.1.smithi191.stdout:7/277: rename d0/d1/d3 to d0/db/d65 0 2024-02-15T18:27:02.134 INFO:tasks.workunit.client.1.smithi191.stdout:7/278: write d0/d1/f21 [1140821,80735] 0 2024-02-15T18:27:02.135 INFO:tasks.workunit.client.1.smithi191.stdout:7/279: stat d0/db/d65/d11 0 2024-02-15T18:27:02.136 INFO:tasks.workunit.client.1.smithi191.stdout:7/280: dread d0/db/f2e [0,4194304] 0 2024-02-15T18:27:02.136 INFO:tasks.workunit.client.1.smithi191.stdout:7/281: write d0/db/f2e [1929066,92254] 0 2024-02-15T18:27:02.137 INFO:tasks.workunit.client.1.smithi191.stdout:5/161: link d0/d23/l2c d0/d14/l34 0 2024-02-15T18:27:02.137 INFO:tasks.workunit.client.0.smithi042.stdout:0/604: dread d4/d9/f31 [0,4194304] 0 2024-02-15T18:27:02.139 INFO:tasks.workunit.client.1.smithi191.stdout:9/251: link d0/d7/dc/d30/f5c d0/d7/dc/d17/f61 0 2024-02-15T18:27:02.141 INFO:tasks.workunit.client.0.smithi042.stdout:9/651: dread d6/dd/d9/d59/d5c/fb7 [0,4194304] 0 2024-02-15T18:27:02.141 INFO:tasks.workunit.client.0.smithi042.stdout:4/609: creat da/d11/d27/fdc x:0 0 0 2024-02-15T18:27:02.141 INFO:tasks.workunit.client.0.smithi042.stdout:4/610: dread - da/d11/d67/d5c/d66/d78/fbd zero size 2024-02-15T18:27:02.141 INFO:tasks.workunit.client.0.smithi042.stdout:4/611: write da/d2b/fd8 [4799237,34676] 0 2024-02-15T18:27:02.141 INFO:tasks.workunit.client.0.smithi042.stdout:4/612: write da/d11/d29/f32 [4222467,30931] 0 2024-02-15T18:27:02.142 INFO:tasks.workunit.client.1.smithi191.stdout:8/371: link d3/l71 d3/d2c/d1a/d23/d2e/d47/d5f/l86 0 2024-02-15T18:27:02.143 INFO:tasks.workunit.client.0.smithi042.stdout:8/566: mkdir dd/de/da5 0 2024-02-15T18:27:02.143 INFO:tasks.workunit.client.0.smithi042.stdout:8/567: dread - dd/de/d94/f98 zero size 2024-02-15T18:27:02.143 INFO:tasks.workunit.client.0.smithi042.stdout:8/568: fsync dd/d11/d75/f9d 0 2024-02-15T18:27:02.144 INFO:tasks.workunit.client.1.smithi191.stdout:5/162: rename d0/d18/c24 to d0/d14/d28/d31/c35 0 2024-02-15T18:27:02.145 INFO:tasks.workunit.client.1.smithi191.stdout:8/372: write d3/fa [3803024,23594] 0 2024-02-15T18:27:02.146 INFO:tasks.workunit.client.1.smithi191.stdout:9/252: dread d0/d7/da/f1c [0,4194304] 0 2024-02-15T18:27:02.147 INFO:tasks.workunit.client.0.smithi042.stdout:0/605: rename d4/d9/d15/l1e to d4/d38/d77/ld5 0 2024-02-15T18:27:02.148 INFO:tasks.workunit.client.1.smithi191.stdout:5/163: rename d0/d14/d28/f2b to d0/d14/d28/f36 0 2024-02-15T18:27:02.149 INFO:tasks.workunit.client.1.smithi191.stdout:5/164: rename d0/d14 to d0/d14/d28/d37 22 2024-02-15T18:27:02.151 INFO:tasks.workunit.client.0.smithi042.stdout:0/606: write d4/d9/d34/d57/faf [59535,35101] 0 2024-02-15T18:27:02.151 INFO:tasks.workunit.client.0.smithi042.stdout:9/652: creat d6/da/d1d/d2d/d40/d95/fdf x:0 0 0 2024-02-15T18:27:02.151 INFO:tasks.workunit.client.0.smithi042.stdout:9/653: truncate d6/da/d1d/d2d/d40/d95/fdd 340307 0 2024-02-15T18:27:02.151 INFO:tasks.workunit.client.0.smithi042.stdout:9/654: chown d6/da/d2f/d54/d96/fce 1 1 2024-02-15T18:27:02.152 INFO:tasks.workunit.client.1.smithi191.stdout:5/165: mkdir d0/d14/d28/d31/d38 0 2024-02-15T18:27:02.152 INFO:tasks.workunit.client.1.smithi191.stdout:5/166: rename d0 to d0/d14/d28/d31/d39 22 2024-02-15T18:27:02.152 INFO:tasks.workunit.client.1.smithi191.stdout:4/334: dwrite d2/d5/d7/d3e/f77 [0,4194304] 0 2024-02-15T18:27:02.152 INFO:tasks.workunit.client.1.smithi191.stdout:4/335: write d2/d5/f4d [1145324,71445] 0 2024-02-15T18:27:02.156 INFO:tasks.workunit.client.0.smithi042.stdout:4/613: rmdir da/d11/da4 39 2024-02-15T18:27:02.157 INFO:tasks.workunit.client.0.smithi042.stdout:3/615: dwrite d1/de/d1a/f1e [0,4194304] 0 2024-02-15T18:27:02.158 INFO:tasks.workunit.client.0.smithi042.stdout:0/607: symlink d4/d38/d39/d8e/d8f/db2/ld6 0 2024-02-15T18:27:02.158 INFO:tasks.workunit.client.0.smithi042.stdout:0/608: write d4/d9/d15/f8c [2176066,89966] 0 2024-02-15T18:27:02.158 INFO:tasks.workunit.client.1.smithi191.stdout:4/336: write d2/f4 [2227915,37780] 0 2024-02-15T18:27:02.158 INFO:tasks.workunit.client.0.smithi042.stdout:9/655: truncate d6/dd/d9/f14 7300613 0 2024-02-15T18:27:02.159 INFO:tasks.workunit.client.1.smithi191.stdout:8/373: dread d3/fd [0,4194304] 0 2024-02-15T18:27:02.159 INFO:tasks.workunit.client.0.smithi042.stdout:4/614: dread da/d11/d29/d4a/d4f/f6c [0,4194304] 0 2024-02-15T18:27:02.161 INFO:tasks.workunit.client.0.smithi042.stdout:7/669: dwrite d10/d12/d23/fa1 [0,4194304] 0 2024-02-15T18:27:02.162 INFO:tasks.workunit.client.1.smithi191.stdout:4/337: symlink d2/d5/d14/d44/d60/l83 0 2024-02-15T18:27:02.163 INFO:tasks.workunit.client.0.smithi042.stdout:9/656: mkdir d6/da/d1d/d2d/d40/de0 0 2024-02-15T18:27:02.163 INFO:tasks.workunit.client.0.smithi042.stdout:9/657: stat d6/da/d2f/d6c/l7f 0 2024-02-15T18:27:02.163 INFO:tasks.workunit.client.0.smithi042.stdout:3/616: dread d1/de/d1a/fb3 [0,4194304] 0 2024-02-15T18:27:02.164 INFO:tasks.workunit.client.1.smithi191.stdout:2/248: dwrite df/d12/d1f/f34 [0,4194304] 0 2024-02-15T18:27:02.164 INFO:tasks.workunit.client.0.smithi042.stdout:4/615: creat da/d11/d29/d4a/d4f/d90/fdd x:0 0 0 2024-02-15T18:27:02.165 INFO:tasks.workunit.client.0.smithi042.stdout:5/663: dwrite d0/d38/d53/f7a [0,4194304] 0 2024-02-15T18:27:02.179 INFO:tasks.workunit.client.1.smithi191.stdout:4/338: creat d2/d5/d14/d47/f84 x:0 0 0 2024-02-15T18:27:02.180 INFO:tasks.workunit.client.1.smithi191.stdout:4/339: truncate d2/f2c 1629219 0 2024-02-15T18:27:02.182 INFO:tasks.workunit.client.0.smithi042.stdout:0/609: dread d4/f7 [4194304,4194304] 0 2024-02-15T18:27:02.183 INFO:tasks.workunit.client.1.smithi191.stdout:2/249: symlink df/d12/d1f/l57 0 2024-02-15T18:27:02.183 INFO:tasks.workunit.client.0.smithi042.stdout:9/658: mkdir d6/da/d2f/d55/d73/d8d/dcf/de1 0 2024-02-15T18:27:02.184 INFO:tasks.workunit.client.1.smithi191.stdout:4/340: mknod d2/d5/d7/d3e/d42/d52/c85 0 2024-02-15T18:27:02.186 INFO:tasks.workunit.client.0.smithi042.stdout:7/670: dread d10/d43/d57/f66 [0,4194304] 0 2024-02-15T18:27:02.186 INFO:tasks.workunit.client.0.smithi042.stdout:7/671: write d10/d15/d39/dd9/fdf [4803296,26784] 0 2024-02-15T18:27:02.187 INFO:tasks.workunit.client.0.smithi042.stdout:7/672: dread d10/d49/fd6 [0,4194304] 0 2024-02-15T18:27:02.187 INFO:tasks.workunit.client.1.smithi191.stdout:2/250: mknod df/d12/d19/d1a/c58 0 2024-02-15T18:27:02.187 INFO:tasks.workunit.client.1.smithi191.stdout:2/251: fdatasync df/d12/d32/d4d/f50 0 2024-02-15T18:27:02.188 INFO:tasks.workunit.client.1.smithi191.stdout:2/252: write df/d12/d1f/f34 [1597604,86079] 0 2024-02-15T18:27:02.188 INFO:tasks.workunit.client.1.smithi191.stdout:2/253: write df/f16 [4951234,117527] 0 2024-02-15T18:27:02.188 INFO:tasks.workunit.client.1.smithi191.stdout:2/254: dread - df/d12/d1f/f45 zero size 2024-02-15T18:27:02.188 INFO:tasks.workunit.client.1.smithi191.stdout:2/255: dread - df/d12/d1f/f31 zero size 2024-02-15T18:27:02.192 INFO:tasks.workunit.client.1.smithi191.stdout:4/341: unlink d2/f34 0 2024-02-15T18:27:02.193 INFO:tasks.workunit.client.1.smithi191.stdout:9/253: dwrite d0/d7/da/d39/f5d [0,4194304] 0 2024-02-15T18:27:02.195 INFO:tasks.workunit.client.1.smithi191.stdout:5/167: dwrite d0/d18/f1e [4194304,4194304] 0 2024-02-15T18:27:02.195 INFO:tasks.workunit.client.1.smithi191.stdout:2/256: creat df/d12/d19/d1a/d35/d47/d4b/f59 x:0 0 0 2024-02-15T18:27:02.196 INFO:tasks.workunit.client.0.smithi042.stdout:4/616: symlink da/d11/d85/lde 0 2024-02-15T18:27:02.196 INFO:tasks.workunit.client.0.smithi042.stdout:4/617: truncate da/f9c 532049 0 2024-02-15T18:27:02.198 INFO:tasks.workunit.client.1.smithi191.stdout:8/374: dwrite d3/f15 [0,4194304] 0 2024-02-15T18:27:02.201 INFO:tasks.workunit.client.0.smithi042.stdout:5/664: creat d0/dda/dc7/d52/dad/fea x:0 0 0 2024-02-15T18:27:02.202 INFO:tasks.workunit.client.1.smithi191.stdout:7/282: dwrite d0/db/f2e [0,4194304] 0 2024-02-15T18:27:02.202 INFO:tasks.workunit.client.1.smithi191.stdout:7/283: read - d0/db/d65/dc/d4b/f56 zero size 2024-02-15T18:27:02.203 INFO:tasks.workunit.client.1.smithi191.stdout:7/284: read d0/db/f2d [1886147,113135] 0 2024-02-15T18:27:02.203 INFO:tasks.workunit.client.0.smithi042.stdout:5/665: write d0/d38/f3c [677894,38239] 0 2024-02-15T18:27:02.207 INFO:tasks.workunit.client.0.smithi042.stdout:0/610: rmdir d4/d9/d34/dab/dc8/d6c/d70 39 2024-02-15T18:27:02.207 INFO:tasks.workunit.client.1.smithi191.stdout:9/254: rmdir d0 39 2024-02-15T18:27:02.208 INFO:tasks.workunit.client.1.smithi191.stdout:4/342: mkdir d2/d5/d86 0 2024-02-15T18:27:02.212 INFO:tasks.workunit.client.1.smithi191.stdout:8/375: getdents d3/d2c/d1a/d23/d2e/d5b 0 2024-02-15T18:27:02.213 INFO:tasks.workunit.client.0.smithi042.stdout:9/659: rename d6/dd/d9/f45 to d6/da/d1d/dde/fe2 0 2024-02-15T18:27:02.214 INFO:tasks.workunit.client.0.smithi042.stdout:7/673: mkdir d10/d15/d8f/da5/dad/de0 0 2024-02-15T18:27:02.214 INFO:tasks.workunit.client.0.smithi042.stdout:7/674: getdents d10/d15/d8f/da5/dad/de0 0 2024-02-15T18:27:02.214 INFO:tasks.workunit.client.0.smithi042.stdout:7/675: chown d10/d12/c1f 299 1 2024-02-15T18:27:02.216 INFO:tasks.workunit.client.1.smithi191.stdout:9/255: dread d0/d7/dc/d17/d33/f43 [0,4194304] 0 2024-02-15T18:27:02.223 INFO:tasks.workunit.client.0.smithi042.stdout:2/572: sync 2024-02-15T18:27:02.225 INFO:tasks.workunit.client.0.smithi042.stdout:3/617: symlink d1/de/d1a/lcc 0 2024-02-15T18:27:02.226 INFO:tasks.workunit.client.0.smithi042.stdout:3/618: chown d1/de/c99 3 1 2024-02-15T18:27:02.226 INFO:tasks.workunit.client.0.smithi042.stdout:3/619: chown d1/d11/d14/d4d/d6c/f9b 499 1 2024-02-15T18:27:02.228 INFO:tasks.workunit.client.0.smithi042.stdout:4/618: dwrite da/d11/d67/f95 [0,4194304] 0 2024-02-15T18:27:02.228 INFO:tasks.workunit.client.0.smithi042.stdout:4/619: fsync da/d3a/dc9/fd2 0 2024-02-15T18:27:02.228 INFO:tasks.workunit.client.1.smithi191.stdout:7/285: dwrite d0/d1/df/f64 [0,4194304] 0 2024-02-15T18:27:02.230 INFO:tasks.workunit.client.1.smithi191.stdout:4/343: dwrite d2/d5/f4d [0,4194304] 0 2024-02-15T18:27:02.230 INFO:tasks.workunit.client.1.smithi191.stdout:4/344: dread d2/f5c [0,4194304] 0 2024-02-15T18:27:02.234 INFO:tasks.workunit.client.1.smithi191.stdout:4/345: dread d2/d5/d7/f25 [0,4194304] 0 2024-02-15T18:27:02.238 INFO:tasks.workunit.client.0.smithi042.stdout:4/620: dread da/d11/d67/d5c/f65 [0,4194304] 0 2024-02-15T18:27:02.238 INFO:tasks.workunit.client.0.smithi042.stdout:4/621: chown da/d11/d29/d4a/d4f/d90 398212286 1 2024-02-15T18:27:02.238 INFO:tasks.workunit.client.0.smithi042.stdout:4/622: chown da/d11/d67/d5c/d66/d78/d2f/dca 198339087 1 2024-02-15T18:27:02.275 INFO:tasks.workunit.client.0.smithi042.stdout:5/666: mknod d0/d12/d2a/ceb 0 2024-02-15T18:27:02.276 INFO:tasks.workunit.client.0.smithi042.stdout:7/676: truncate d10/d43/f84 29782 0 2024-02-15T18:27:02.278 INFO:tasks.workunit.client.0.smithi042.stdout:2/573: mknod d0/d8/d10/d7b/cb2 0 2024-02-15T18:27:02.278 INFO:tasks.workunit.client.0.smithi042.stdout:2/574: write d0/d8/d20/fae [826927,107784] 0 2024-02-15T18:27:02.279 INFO:tasks.workunit.client.0.smithi042.stdout:3/620: symlink d1/d11/d14/d4d/d6c/d7b/lcd 0 2024-02-15T18:27:02.280 INFO:tasks.workunit.client.0.smithi042.stdout:3/621: chown d1/d11/d14/dbf 2 1 2024-02-15T18:27:02.280 INFO:tasks.workunit.client.0.smithi042.stdout:4/623: dwrite da/d11/d67/d5c/d66/d78/d2f/dca/fd3 [0,4194304] 0 2024-02-15T18:27:02.285 INFO:tasks.workunit.client.0.smithi042.stdout:1/638: sync 2024-02-15T18:27:02.286 INFO:tasks.workunit.client.0.smithi042.stdout:1/639: write de/d15/d67/f3a [2370336,82550] 0 2024-02-15T18:27:02.287 INFO:tasks.workunit.client.0.smithi042.stdout:1/640: write de/d6d/fa3 [377925,49539] 0 2024-02-15T18:27:02.287 INFO:tasks.workunit.client.0.smithi042.stdout:1/641: write de/d15/d67/d74/f75 [236087,2251] 0 2024-02-15T18:27:02.288 INFO:tasks.workunit.client.0.smithi042.stdout:5/667: rename d0/d50/d98 to d0/d38/d53/d68/dec 0 2024-02-15T18:27:02.290 INFO:tasks.workunit.client.1.smithi191.stdout:8/376: rename d3/l7d to d3/d2c/d1a/d23/d2a/l87 0 2024-02-15T18:27:02.291 INFO:tasks.workunit.client.0.smithi042.stdout:7/677: creat d10/d49/d6d/d78/fe1 x:0 0 0 2024-02-15T18:27:02.292 INFO:tasks.workunit.client.0.smithi042.stdout:7/678: dread d10/d49/d6d/d78/d83/fa3 [0,4194304] 0 2024-02-15T18:27:02.293 INFO:tasks.workunit.client.1.smithi191.stdout:9/256: symlink d0/d7/dc/d32/l62 0 2024-02-15T18:27:02.294 INFO:tasks.workunit.client.0.smithi042.stdout:2/575: rmdir d0/d8/d20/d7d/d7e 39 2024-02-15T18:27:02.294 INFO:tasks.workunit.client.0.smithi042.stdout:2/576: fsync d0/d8/db/de/d1a/f2b 0 2024-02-15T18:27:02.294 INFO:tasks.workunit.client.0.smithi042.stdout:2/577: stat d0/d8/d20/d7d/d94/c59 0 2024-02-15T18:27:02.297 INFO:tasks.workunit.client.1.smithi191.stdout:7/286: rmdir d0/d1/d35/d42/d46 39 2024-02-15T18:27:02.297 INFO:tasks.workunit.client.1.smithi191.stdout:7/287: chown d0/d1/df/f16 6 1 2024-02-15T18:27:02.298 INFO:tasks.workunit.client.0.smithi042.stdout:3/622: getdents d1/de/d1a 0 2024-02-15T18:27:02.303 INFO:tasks.workunit.client.0.smithi042.stdout:6/710: sync 2024-02-15T18:27:02.303 INFO:tasks.workunit.client.0.smithi042.stdout:6/711: truncate d9/d11/d19/d35/dc4/fd5 585515 0 2024-02-15T18:27:02.307 INFO:tasks.workunit.client.0.smithi042.stdout:1/642: mknod de/d54/dbf/dd0/d7c/dd6/de8/cea 0 2024-02-15T18:27:02.309 INFO:tasks.workunit.client.0.smithi042.stdout:5/668: symlink d0/d22/d7b/da9/led 0 2024-02-15T18:27:02.311 INFO:tasks.workunit.client.1.smithi191.stdout:4/346: mknod d2/d5/d64/c87 0 2024-02-15T18:27:02.314 INFO:tasks.workunit.client.1.smithi191.stdout:8/377: creat d3/d2c/d1a/d23/d2e/d47/f88 x:0 0 0 2024-02-15T18:27:02.314 INFO:tasks.workunit.client.1.smithi191.stdout:8/378: stat d3/d2c/d1a/d23/d2e/d5b/d69/d6d 0 2024-02-15T18:27:02.315 INFO:tasks.workunit.client.0.smithi042.stdout:7/679: truncate d10/d15/d39/d6c/f71 1501391 0 2024-02-15T18:27:02.317 INFO:tasks.workunit.client.1.smithi191.stdout:9/257: creat d0/f63 x:0 0 0 2024-02-15T18:27:02.317 INFO:tasks.workunit.client.1.smithi191.stdout:7/288: dwrite d0/d1/d1d/f4c [0,4194304] 0 2024-02-15T18:27:02.317 INFO:tasks.workunit.client.1.smithi191.stdout:7/289: fsync d0/d1/df/f19 0 2024-02-15T18:27:02.319 INFO:tasks.workunit.client.1.smithi191.stdout:8/379: creat d3/d2c/d1a/d23/f89 x:0 0 0 2024-02-15T18:27:02.322 INFO:tasks.workunit.client.0.smithi042.stdout:2/578: mknod d0/d35/d9e/cb3 0 2024-02-15T18:27:02.323 INFO:tasks.workunit.client.0.smithi042.stdout:3/623: getdents d1/d11/d14/d4d 0 2024-02-15T18:27:02.323 INFO:tasks.workunit.client.0.smithi042.stdout:7/680: getdents d10/d12/d23/d51 0 2024-02-15T18:27:02.323 INFO:tasks.workunit.client.0.smithi042.stdout:7/681: chown d10/d12/d23/d51/c90 8192831 1 2024-02-15T18:27:02.323 INFO:tasks.workunit.client.1.smithi191.stdout:4/347: dread d2/d5/d7/f29 [0,4194304] 0 2024-02-15T18:27:02.325 INFO:tasks.workunit.client.0.smithi042.stdout:1/643: rename de/d54/c56 to de/d15/d67/ceb 0 2024-02-15T18:27:02.329 INFO:tasks.workunit.client.0.smithi042.stdout:1/644: read de/d15/d67/d46/f50 [27953,102472] 0 2024-02-15T18:27:02.330 INFO:tasks.workunit.client.0.smithi042.stdout:2/579: creat d0/d8/db/de/fb4 x:0 0 0 2024-02-15T18:27:02.333 INFO:tasks.workunit.client.1.smithi191.stdout:4/348: rename d2/d5/d14/d44/d60 to d2/d5/d7/d3e/d88 0 2024-02-15T18:27:02.334 INFO:tasks.workunit.client.1.smithi191.stdout:4/349: write d2/d5/d14/f58 [2216873,45933] 0 2024-02-15T18:27:02.334 INFO:tasks.workunit.client.1.smithi191.stdout:4/350: fsync d2/d5/d7/d3e/d42/d52/f62 0 2024-02-15T18:27:02.334 INFO:tasks.workunit.client.0.smithi042.stdout:3/624: symlink d1/de/d1a/lce 0 2024-02-15T18:27:02.336 INFO:tasks.workunit.client.0.smithi042.stdout:3/625: write d1/de/f42 [3987725,116264] 0 2024-02-15T18:27:02.336 INFO:tasks.workunit.client.0.smithi042.stdout:4/624: dwrite da/f1a [0,4194304] 0 2024-02-15T18:27:02.336 INFO:tasks.workunit.client.0.smithi042.stdout:4/625: chown da/d11/d29/d4a/d4f/f8e 3 1 2024-02-15T18:27:02.337 INFO:tasks.workunit.client.0.smithi042.stdout:1/645: mkdir de/d54/dbf/dd0/d7c/dd6/de8/dec 0 2024-02-15T18:27:02.337 INFO:tasks.workunit.client.0.smithi042.stdout:7/682: write d10/d43/f4f [8967114,4903] 0 2024-02-15T18:27:02.338 INFO:tasks.workunit.client.0.smithi042.stdout:2/580: getdents d0/d8/db/de/d1a/d2e 0 2024-02-15T18:27:02.338 INFO:tasks.workunit.client.0.smithi042.stdout:7/683: write d10/d15/d8f/da5/fb7 [557639,20635] 0 2024-02-15T18:27:02.339 INFO:tasks.workunit.client.0.smithi042.stdout:6/712: dwrite d9/f2d [0,4194304] 0 2024-02-15T18:27:02.340 INFO:tasks.workunit.client.0.smithi042.stdout:3/626: rmdir d1/de/d19 39 2024-02-15T18:27:02.341 INFO:tasks.workunit.client.0.smithi042.stdout:1/646: chown de/d15/d1b/c39 43 1 2024-02-15T18:27:02.342 INFO:tasks.workunit.client.0.smithi042.stdout:0/611: sync 2024-02-15T18:27:02.342 INFO:tasks.workunit.client.0.smithi042.stdout:9/660: sync 2024-02-15T18:27:02.342 INFO:tasks.workunit.client.0.smithi042.stdout:8/569: sync 2024-02-15T18:27:02.342 INFO:tasks.workunit.client.0.smithi042.stdout:0/612: readlink d4/d9/d34/l94 0 2024-02-15T18:27:02.344 INFO:tasks.workunit.client.0.smithi042.stdout:2/581: truncate d0/d35/f81 732271 0 2024-02-15T18:27:02.345 INFO:tasks.workunit.client.0.smithi042.stdout:7/684: mkdir d10/d15/d8f/da5/dad/de0/de2 0 2024-02-15T18:27:02.345 INFO:tasks.workunit.client.0.smithi042.stdout:7/685: write d10/d12/d23/f61 [5208307,60462] 0 2024-02-15T18:27:02.347 INFO:tasks.workunit.client.0.smithi042.stdout:6/713: link d9/d11/d19/fc1 d9/d3e/dd9/ffa 0 2024-02-15T18:27:02.348 INFO:tasks.workunit.client.0.smithi042.stdout:9/661: symlink d6/dd/le3 0 2024-02-15T18:27:02.351 INFO:tasks.workunit.client.0.smithi042.stdout:0/613: truncate d4/d38/d39/d8e/f67 3227525 0 2024-02-15T18:27:02.352 INFO:tasks.workunit.client.0.smithi042.stdout:2/582: rmdir d0/d8/d20/d7d/d94 39 2024-02-15T18:27:02.354 INFO:tasks.workunit.client.0.smithi042.stdout:7/686: mkdir d10/d15/d39/de3 0 2024-02-15T18:27:02.354 INFO:tasks.workunit.client.0.smithi042.stdout:7/687: chown d10/d15/d39/f5a 33 1 2024-02-15T18:27:02.354 INFO:tasks.workunit.client.0.smithi042.stdout:7/688: readlink d10/d12/d23/d51/l97 0 2024-02-15T18:27:02.355 INFO:tasks.workunit.client.1.smithi191.stdout:8/380: dwrite d3/d2c/f17 [0,4194304] 0 2024-02-15T18:27:02.356 INFO:tasks.workunit.client.0.smithi042.stdout:1/647: write de/d15/f20 [5170154,26111] 0 2024-02-15T18:27:02.358 INFO:tasks.workunit.client.0.smithi042.stdout:6/714: rename d9/l47 to d9/d11/d46/d89/dbc/lfb 0 2024-02-15T18:27:02.358 INFO:tasks.workunit.client.0.smithi042.stdout:2/583: truncate d0/d8/d20/d7d/d94/f99 116177 0 2024-02-15T18:27:02.358 INFO:tasks.workunit.client.0.smithi042.stdout:2/584: chown d0/d8/d10/d7b/cb2 78898340 1 2024-02-15T18:27:02.358 INFO:tasks.workunit.client.1.smithi191.stdout:8/381: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/c8a 0 2024-02-15T18:27:02.362 INFO:tasks.workunit.client.0.smithi042.stdout:2/585: read d0/d8/d10/d7b/faa [1509709,116336] 0 2024-02-15T18:27:02.362 INFO:tasks.workunit.client.0.smithi042.stdout:6/715: mkdir d9/d11/d19/d35/d55/d92/dfc 0 2024-02-15T18:27:02.362 INFO:tasks.workunit.client.0.smithi042.stdout:9/662: rename d6/da/d1d/d2d/f39 to d6/da/d2f/d55/d73/fe4 0 2024-02-15T18:27:02.367 INFO:tasks.workunit.client.0.smithi042.stdout:6/716: getdents d9/d11/d19/d1c/d31/d4f 0 2024-02-15T18:27:02.367 INFO:tasks.workunit.client.0.smithi042.stdout:2/586: creat d0/d8/db/de/d1a/fb5 x:0 0 0 2024-02-15T18:27:02.369 INFO:tasks.workunit.client.0.smithi042.stdout:5/669: dwrite d0/d38/d53/d64/f9f [4194304,4194304] 0 2024-02-15T18:27:02.369 INFO:tasks.workunit.client.0.smithi042.stdout:5/670: write d0/d12/d2a/fe8 [453663,62966] 0 2024-02-15T18:27:02.369 INFO:tasks.workunit.client.0.smithi042.stdout:5/671: readlink d0/dda/lb7 0 2024-02-15T18:27:02.369 INFO:tasks.workunit.client.0.smithi042.stdout:2/587: rmdir d0/d8/d20/d3f 39 2024-02-15T18:27:02.370 INFO:tasks.workunit.client.0.smithi042.stdout:0/614: dread d4/d9/f46 [0,4194304] 0 2024-02-15T18:27:02.371 INFO:tasks.workunit.client.0.smithi042.stdout:0/615: write d4/d9/f31 [777806,2462] 0 2024-02-15T18:27:02.374 INFO:tasks.workunit.client.0.smithi042.stdout:5/672: unlink d0/c2c 0 2024-02-15T18:27:02.374 INFO:tasks.workunit.client.0.smithi042.stdout:5/673: stat d0/d39 0 2024-02-15T18:27:02.374 INFO:tasks.workunit.client.0.smithi042.stdout:1/648: dwrite de/d54/dbf/fe1 [0,4194304] 0 2024-02-15T18:27:02.375 INFO:tasks.workunit.client.0.smithi042.stdout:2/588: creat d0/d40/d56/d8b/fb6 x:0 0 0 2024-02-15T18:27:02.375 INFO:tasks.workunit.client.0.smithi042.stdout:2/589: write d0/d40/d56/d8b/fb6 [629857,17655] 0 2024-02-15T18:27:02.376 INFO:tasks.workunit.client.0.smithi042.stdout:0/616: stat d4/d9/l69 0 2024-02-15T18:27:02.376 INFO:tasks.workunit.client.1.smithi191.stdout:7/290: dwrite d0/d1/d35/f50 [0,4194304] 0 2024-02-15T18:27:02.377 INFO:tasks.workunit.client.0.smithi042.stdout:5/674: creat d0/d22/fee x:0 0 0 2024-02-15T18:27:02.378 INFO:tasks.workunit.client.1.smithi191.stdout:7/291: write d0/d1/df/f64 [684421,107587] 0 2024-02-15T18:27:02.378 INFO:tasks.workunit.client.1.smithi191.stdout:7/292: chown d0/db/d65/d11 89648 1 2024-02-15T18:27:02.378 INFO:tasks.workunit.client.1.smithi191.stdout:7/293: fdatasync d0/db/d65/f3e 0 2024-02-15T18:27:02.378 INFO:tasks.workunit.client.1.smithi191.stdout:6/244: sync 2024-02-15T18:27:02.378 INFO:tasks.workunit.client.1.smithi191.stdout:6/245: chown d8/d9/c3e 26793 1 2024-02-15T18:27:02.379 INFO:tasks.workunit.client.0.smithi042.stdout:4/626: dwrite da/d11/d67/d5c/d66/fa3 [0,4194304] 0 2024-02-15T18:27:02.379 INFO:tasks.workunit.client.0.smithi042.stdout:1/649: creat de/d54/d9a/fed x:0 0 0 2024-02-15T18:27:02.379 INFO:tasks.workunit.client.0.smithi042.stdout:0/617: symlink d4/d9/d34/dab/dc8/d6c/d70/ld7 0 2024-02-15T18:27:02.379 INFO:tasks.workunit.client.0.smithi042.stdout:0/618: chown d4/d9/d62/d74 708 1 2024-02-15T18:27:02.383 INFO:tasks.workunit.client.0.smithi042.stdout:2/590: read d0/d8/db/fd [941877,57063] 0 2024-02-15T18:27:02.386 INFO:tasks.workunit.client.0.smithi042.stdout:4/627: write da/d11/d67/fc6 [1040589,90839] 0 2024-02-15T18:27:02.387 INFO:tasks.workunit.client.1.smithi191.stdout:4/351: dwrite d2/d5/d7/d2f/f31 [0,4194304] 0 2024-02-15T18:27:02.387 INFO:tasks.workunit.client.1.smithi191.stdout:4/352: chown d2/d5/d7/d2f/d41/l5b 48009037 1 2024-02-15T18:27:02.390 INFO:tasks.workunit.client.0.smithi042.stdout:8/570: dwrite dd/de/da4/d67/d5a/f93 [0,4194304] 0 2024-02-15T18:27:02.391 INFO:tasks.workunit.client.0.smithi042.stdout:3/627: dwrite d1/d11/d14/d4d/d73/f87 [0,4194304] 0 2024-02-15T18:27:02.391 INFO:tasks.workunit.client.1.smithi191.stdout:9/258: dwrite d0/f63 [0,4194304] 0 2024-02-15T18:27:02.392 INFO:tasks.workunit.client.1.smithi191.stdout:9/259: write d0/d7/dc/d30/d3b/f55 [951593,46186] 0 2024-02-15T18:27:02.392 INFO:tasks.workunit.client.0.smithi042.stdout:3/628: write d1/d11/d14/f53 [426144,9194] 0 2024-02-15T18:27:02.392 INFO:tasks.workunit.client.0.smithi042.stdout:3/629: truncate d1/d11/d2e/d4c/f5b 1487648 0 2024-02-15T18:27:02.395 INFO:tasks.workunit.client.0.smithi042.stdout:6/717: dwrite d9/d11/d19/d35/f80 [0,4194304] 0 2024-02-15T18:27:02.395 INFO:tasks.workunit.client.0.smithi042.stdout:5/675: dwrite d0/d89/f97 [0,4194304] 0 2024-02-15T18:27:02.396 INFO:tasks.workunit.client.0.smithi042.stdout:0/619: dread d4/d9/f31 [0,4194304] 0 2024-02-15T18:27:02.399 INFO:tasks.workunit.client.0.smithi042.stdout:1/650: getdents de/d54/dbf/dd0 0 2024-02-15T18:27:02.399 INFO:tasks.workunit.client.0.smithi042.stdout:1/651: fdatasync de/d15/d67/d63/f8b 0 2024-02-15T18:27:02.400 INFO:tasks.workunit.client.1.smithi191.stdout:1/325: sync 2024-02-15T18:27:02.400 INFO:tasks.workunit.client.1.smithi191.stdout:0/160: sync 2024-02-15T18:27:02.402 INFO:tasks.workunit.client.0.smithi042.stdout:2/591: symlink d0/d8/d10/lb7 0 2024-02-15T18:27:02.402 INFO:tasks.workunit.client.1.smithi191.stdout:9/260: dread d0/f4 [0,4194304] 0 2024-02-15T18:27:02.403 INFO:tasks.workunit.client.1.smithi191.stdout:9/261: write d0/f1 [6738737,11948] 0 2024-02-15T18:27:02.410 INFO:tasks.workunit.client.0.smithi042.stdout:7/689: dwrite d10/d43/f4f [8388608,4194304] 0 2024-02-15T18:27:02.415 INFO:tasks.workunit.client.1.smithi191.stdout:7/294: getdents d0/db/d65/dc/d4b 0 2024-02-15T18:27:02.417 INFO:tasks.workunit.client.0.smithi042.stdout:4/628: rmdir da/d3b 39 2024-02-15T18:27:02.419 INFO:tasks.workunit.client.1.smithi191.stdout:9/262: dread d0/f1 [4194304,4194304] 0 2024-02-15T18:27:02.419 INFO:tasks.workunit.client.1.smithi191.stdout:9/263: dread - d0/d7/dc/d30/f4e zero size 2024-02-15T18:27:02.419 INFO:tasks.workunit.client.1.smithi191.stdout:9/264: truncate d0/f4 8872977 0 2024-02-15T18:27:02.421 INFO:tasks.workunit.client.1.smithi191.stdout:6/246: getdents d8/d9/d2b 0 2024-02-15T18:27:02.424 INFO:tasks.workunit.client.1.smithi191.stdout:4/353: symlink d2/d5/d7/d3e/d63/l89 0 2024-02-15T18:27:02.424 INFO:tasks.workunit.client.1.smithi191.stdout:4/354: readlink d2/l4b 0 2024-02-15T18:27:02.427 INFO:tasks.workunit.client.0.smithi042.stdout:8/571: rmdir dd 39 2024-02-15T18:27:02.432 INFO:tasks.workunit.client.1.smithi191.stdout:1/326: rmdir d0/d8/df/d12/d33/d49 39 2024-02-15T18:27:02.432 INFO:tasks.workunit.client.0.smithi042.stdout:4/629: dwrite da/d11/d29/d4a/d4f/f9d [8388608,4194304] 0 2024-02-15T18:27:02.433 INFO:tasks.workunit.client.1.smithi191.stdout:0/161: mknod d7/da/d17/c33 0 2024-02-15T18:27:02.433 INFO:tasks.workunit.client.1.smithi191.stdout:1/327: write d0/d8/df/d12/d2a/f3a [569376,73892] 0 2024-02-15T18:27:02.433 INFO:tasks.workunit.client.1.smithi191.stdout:1/328: chown d0/d2f/d51/d5e/d68 21235431 1 2024-02-15T18:27:02.433 INFO:tasks.workunit.client.1.smithi191.stdout:1/329: readlink d0/l1e 0 2024-02-15T18:27:02.434 INFO:tasks.workunit.client.0.smithi042.stdout:3/630: creat d1/de/fcf x:0 0 0 2024-02-15T18:27:02.437 INFO:tasks.workunit.client.0.smithi042.stdout:3/631: dread d1/d11/d14/f60 [0,4194304] 0 2024-02-15T18:27:02.437 INFO:tasks.workunit.client.0.smithi042.stdout:3/632: stat d1/d11/f45 0 2024-02-15T18:27:02.437 INFO:tasks.workunit.client.0.smithi042.stdout:3/633: readlink d1/d11/d2e/d4c/db0/lb2 0 2024-02-15T18:27:02.437 INFO:tasks.workunit.client.0.smithi042.stdout:3/634: fsync d1/d11/d2e/d4c/f6a 0 2024-02-15T18:27:02.440 INFO:tasks.workunit.client.1.smithi191.stdout:7/295: rename d0/d1/d35/d42/d46 to d0/db/d65/d11/d5a/d66 0 2024-02-15T18:27:02.441 INFO:tasks.workunit.client.1.smithi191.stdout:3/212: sync 2024-02-15T18:27:02.443 INFO:tasks.workunit.client.1.smithi191.stdout:9/265: mknod d0/d7/dc/d17/c64 0 2024-02-15T18:27:02.445 INFO:tasks.workunit.client.1.smithi191.stdout:4/355: link d2/d5/d14/d47/d78/c7c d2/d5/d7/d3e/c8a 0 2024-02-15T18:27:02.446 INFO:tasks.workunit.client.1.smithi191.stdout:0/162: getdents d7/da/d17/d31 0 2024-02-15T18:27:02.446 INFO:tasks.workunit.client.1.smithi191.stdout:0/163: write d7/da/f20 [1328893,73222] 0 2024-02-15T18:27:02.447 INFO:tasks.workunit.client.1.smithi191.stdout:4/356: write d2/d5/d7/d2f/f51 [1302353,101622] 0 2024-02-15T18:27:02.448 INFO:tasks.workunit.client.1.smithi191.stdout:1/330: truncate d0/f5 6576028 0 2024-02-15T18:27:02.448 INFO:tasks.workunit.client.1.smithi191.stdout:1/331: chown d0/d2f/d51/d10/c13 1 1 2024-02-15T18:27:02.448 INFO:tasks.workunit.client.1.smithi191.stdout:1/332: truncate d0/d8/fd 1554456 0 2024-02-15T18:27:02.448 INFO:tasks.workunit.client.1.smithi191.stdout:1/333: chown d0/d8/d48/f4d 50350282 1 2024-02-15T18:27:02.450 INFO:tasks.workunit.client.1.smithi191.stdout:7/296: symlink d0/db/d65/d11/d5a/d66/d1e/l67 0 2024-02-15T18:27:02.452 INFO:tasks.workunit.client.1.smithi191.stdout:9/266: unlink d0/d7/dc/d30/f5f 0 2024-02-15T18:27:02.456 INFO:tasks.workunit.client.1.smithi191.stdout:3/213: dwrite d6/d3b/d42/d49/d21/f41 [0,4194304] 0 2024-02-15T18:27:02.456 INFO:tasks.workunit.client.1.smithi191.stdout:1/334: link d0/d8/df/d12/d33/c4b d0/d2f/d5c/c6a 0 2024-02-15T18:27:02.460 INFO:tasks.workunit.client.0.smithi042.stdout:6/718: rmdir d9/d11/d19/d1c/d31/d53 39 2024-02-15T18:27:02.466 INFO:tasks.workunit.client.0.smithi042.stdout:5/676: truncate d0/d14/f77 1276278 0 2024-02-15T18:27:02.470 INFO:tasks.workunit.client.0.smithi042.stdout:0/620: truncate d4/d38/f49 2113968 0 2024-02-15T18:27:02.470 INFO:tasks.workunit.client.0.smithi042.stdout:0/621: getdents d4/d9/d34/dab/dc8/d6c 0 2024-02-15T18:27:02.474 INFO:tasks.workunit.client.0.smithi042.stdout:1/652: rename de/d54/dbf/dd0/cb8 to de/d54/dbf/dd0/d7c/cee 0 2024-02-15T18:27:02.476 INFO:tasks.workunit.client.1.smithi191.stdout:0/164: dwrite d7/da/f20 [0,4194304] 0 2024-02-15T18:27:02.476 INFO:tasks.workunit.client.0.smithi042.stdout:2/592: symlink d0/d8/d10/d31/d3a/lb8 0 2024-02-15T18:27:02.477 INFO:tasks.workunit.client.1.smithi191.stdout:3/214: creat d6/d27/d4d/f4e x:0 0 0 2024-02-15T18:27:02.477 INFO:tasks.workunit.client.1.smithi191.stdout:3/215: chown d6/l12 733 1 2024-02-15T18:27:02.479 INFO:tasks.workunit.client.0.smithi042.stdout:8/572: mknod dd/de/d33/d66/ca6 0 2024-02-15T18:27:02.479 INFO:tasks.workunit.client.0.smithi042.stdout:8/573: write dd/de/da4/fa3 [359580,72297] 0 2024-02-15T18:27:02.482 INFO:tasks.workunit.client.1.smithi191.stdout:1/335: symlink d0/d2f/l6b 0 2024-02-15T18:27:02.483 INFO:tasks.workunit.client.0.smithi042.stdout:3/635: mknod d1/d11/d2e/d4c/cd0 0 2024-02-15T18:27:02.484 INFO:tasks.workunit.client.0.smithi042.stdout:3/636: truncate d1/d11/d14/f39 3849472 0 2024-02-15T18:27:02.484 INFO:tasks.workunit.client.0.smithi042.stdout:3/637: write d1/fbe [813131,89650] 0 2024-02-15T18:27:02.485 INFO:tasks.workunit.client.0.smithi042.stdout:6/719: symlink d9/d11/d46/dc3/lfd 0 2024-02-15T18:27:02.488 INFO:tasks.workunit.client.0.smithi042.stdout:3/638: dread d1/f15 [0,4194304] 0 2024-02-15T18:27:02.489 INFO:tasks.workunit.client.0.smithi042.stdout:5/677: dwrite d0/d38/d53/d64/f93 [0,4194304] 0 2024-02-15T18:27:02.491 INFO:tasks.workunit.client.1.smithi191.stdout:0/165: rename d7/da/d17/d2f to d7/da/d1a/d34 0 2024-02-15T18:27:02.491 INFO:tasks.workunit.client.1.smithi191.stdout:0/166: fsync d7/da/d1a/d34/f29 0 2024-02-15T18:27:02.491 INFO:tasks.workunit.client.1.smithi191.stdout:0/167: chown d7/da/c10 506029687 1 2024-02-15T18:27:02.492 INFO:tasks.workunit.client.1.smithi191.stdout:3/216: symlink d6/d27/l4f 0 2024-02-15T18:27:02.492 INFO:tasks.workunit.client.0.smithi042.stdout:0/622: mknod d4/d9/d34/dab/cd8 0 2024-02-15T18:27:02.494 INFO:tasks.workunit.client.1.smithi191.stdout:9/267: rename d0/c2a to d0/c65 0 2024-02-15T18:27:02.495 INFO:tasks.workunit.client.1.smithi191.stdout:0/168: truncate d7/d1b/f23 10219 0 2024-02-15T18:27:02.498 INFO:tasks.workunit.client.1.smithi191.stdout:3/217: creat d6/d3b/d30/f50 x:0 0 0 2024-02-15T18:27:02.498 INFO:tasks.workunit.client.1.smithi191.stdout:3/218: dread - d6/d3b/d42/d49/f43 zero size 2024-02-15T18:27:02.499 INFO:tasks.workunit.client.0.smithi042.stdout:4/630: rename da/d11/l36 to da/d11/d29/ldf 0 2024-02-15T18:27:02.499 INFO:tasks.workunit.client.0.smithi042.stdout:1/653: creat de/d15/d67/d8e/fef x:0 0 0 2024-02-15T18:27:02.499 INFO:tasks.workunit.client.1.smithi191.stdout:0/169: write d7/da/f16 [965477,46837] 0 2024-02-15T18:27:02.500 INFO:tasks.workunit.client.1.smithi191.stdout:9/268: getdents d0/d7/dc/d17 0 2024-02-15T18:27:02.500 INFO:tasks.workunit.client.1.smithi191.stdout:9/269: write d0/d7/dc/d17/f2d [1113597,64837] 0 2024-02-15T18:27:02.501 INFO:tasks.workunit.client.0.smithi042.stdout:2/593: mknod d0/cb9 0 2024-02-15T18:27:02.503 INFO:tasks.workunit.client.0.smithi042.stdout:6/720: mknod d9/d11/d19/d1c/d31/d9b/cfe 0 2024-02-15T18:27:02.503 INFO:tasks.workunit.client.0.smithi042.stdout:6/721: dread - d9/f72 zero size 2024-02-15T18:27:02.507 INFO:tasks.workunit.client.0.smithi042.stdout:3/639: rmdir d1/d11/d14 39 2024-02-15T18:27:02.507 INFO:tasks.workunit.client.0.smithi042.stdout:5/678: rmdir d0/d14/d88/da1 39 2024-02-15T18:27:02.507 INFO:tasks.workunit.client.0.smithi042.stdout:5/679: write d0/dda/dc7/fc4 [513234,115577] 0 2024-02-15T18:27:02.508 INFO:tasks.workunit.client.0.smithi042.stdout:4/631: truncate da/d11/f23 2345160 0 2024-02-15T18:27:02.508 INFO:tasks.workunit.client.0.smithi042.stdout:4/632: readlink da/d3b/l6a 0 2024-02-15T18:27:02.509 INFO:tasks.workunit.client.0.smithi042.stdout:4/633: dread da/d11/d29/d4a/d4f/d80/fa9 [0,4194304] 0 2024-02-15T18:27:02.510 INFO:tasks.workunit.client.0.smithi042.stdout:4/634: dread da/d11/d29/d4a/d4f/f8e [0,4194304] 0 2024-02-15T18:27:02.512 INFO:tasks.workunit.client.0.smithi042.stdout:2/594: rename d0/d8/d20/d7d/d94/f99 to d0/d8/db/d44/fba 0 2024-02-15T18:27:02.512 INFO:tasks.workunit.client.0.smithi042.stdout:2/595: dread - d0/d8/d10/d31/da6/fad zero size 2024-02-15T18:27:02.513 INFO:tasks.workunit.client.1.smithi191.stdout:0/170: dread d7/da/d1a/d34/f29 [4194304,4194304] 0 2024-02-15T18:27:02.516 INFO:tasks.workunit.client.0.smithi042.stdout:6/722: symlink d9/d3e/dd9/lff 0 2024-02-15T18:27:02.516 INFO:tasks.workunit.client.0.smithi042.stdout:6/723: dread - d9/d11/d19/d35/fe2 zero size 2024-02-15T18:27:02.518 INFO:tasks.workunit.client.1.smithi191.stdout:0/171: mknod d7/da/c35 0 2024-02-15T18:27:02.520 INFO:tasks.workunit.client.0.smithi042.stdout:5/680: link d0/fce d0/d39/fef 0 2024-02-15T18:27:02.520 INFO:tasks.workunit.client.0.smithi042.stdout:5/681: chown d0/d38/d53/db4/fca 475 1 2024-02-15T18:27:02.521 INFO:tasks.workunit.client.0.smithi042.stdout:1/654: link de/d15/d1b/c39 de/d15/d67/cf0 0 2024-02-15T18:27:02.522 INFO:tasks.workunit.client.0.smithi042.stdout:0/623: dwrite d4/d9/d5a/fcc [0,4194304] 0 2024-02-15T18:27:02.522 INFO:tasks.workunit.client.0.smithi042.stdout:1/655: write de/d15/d67/d46/f59 [882427,60723] 0 2024-02-15T18:27:02.522 INFO:tasks.workunit.client.0.smithi042.stdout:8/574: dwrite dd/f17 [0,4194304] 0 2024-02-15T18:27:02.522 INFO:tasks.workunit.client.0.smithi042.stdout:8/575: chown dd/de/da4/l50 25454 1 2024-02-15T18:27:02.523 INFO:tasks.workunit.client.1.smithi191.stdout:1/336: dwrite d0/d8/df/d12/d2a/f42 [0,4194304] 0 2024-02-15T18:27:02.523 INFO:tasks.workunit.client.1.smithi191.stdout:9/270: dwrite d0/d7/dc/d17/d33/f4d [4194304,4194304] 0 2024-02-15T18:27:02.526 INFO:tasks.workunit.client.0.smithi042.stdout:4/635: symlink da/d11/d29/d4a/d4f/le0 0 2024-02-15T18:27:02.526 INFO:tasks.workunit.client.0.smithi042.stdout:4/636: write da/d11/d67/d5c/d66/d78/d2f/d4d/fc8 [2583183,97643] 0 2024-02-15T18:27:02.526 INFO:tasks.workunit.client.0.smithi042.stdout:4/637: getdents da/d11/d85/db5 0 2024-02-15T18:27:02.527 INFO:tasks.workunit.client.0.smithi042.stdout:4/638: write da/d11/d67/d5c/f65 [844735,18009] 0 2024-02-15T18:27:02.530 INFO:tasks.workunit.client.1.smithi191.stdout:3/219: fsync d6/d3b/d30/f50 0 2024-02-15T18:27:02.530 INFO:tasks.workunit.client.1.smithi191.stdout:3/220: read - d6/d3b/d42/f4a zero size 2024-02-15T18:27:02.536 INFO:tasks.workunit.client.0.smithi042.stdout:3/640: write d1/f9 [959749,20437] 0 2024-02-15T18:27:02.537 INFO:tasks.workunit.client.1.smithi191.stdout:1/337: link d0/d8/df/d2e/l46 d0/d8/df/l6c 0 2024-02-15T18:27:02.537 INFO:tasks.workunit.client.0.smithi042.stdout:9/663: sync 2024-02-15T18:27:02.538 INFO:tasks.workunit.client.0.smithi042.stdout:2/596: symlink d0/d8/d20/d41/lbb 0 2024-02-15T18:27:02.538 INFO:tasks.workunit.client.0.smithi042.stdout:2/597: truncate d0/d8/d10/d5a/f88 937828 0 2024-02-15T18:27:02.538 INFO:tasks.workunit.client.0.smithi042.stdout:2/598: truncate d0/d35/f95 598460 0 2024-02-15T18:27:02.539 INFO:tasks.workunit.client.1.smithi191.stdout:9/271: mkdir d0/d29/d48/d66 0 2024-02-15T18:27:02.540 INFO:tasks.workunit.client.0.smithi042.stdout:6/724: creat d9/d11/d46/d89/df4/f100 x:0 0 0 2024-02-15T18:27:02.540 INFO:tasks.workunit.client.0.smithi042.stdout:6/725: dread - d9/d3e/dd9/fdc zero size 2024-02-15T18:27:02.540 INFO:tasks.workunit.client.0.smithi042.stdout:6/726: write d9/f8e [1323654,126124] 0 2024-02-15T18:27:02.542 INFO:tasks.workunit.client.0.smithi042.stdout:5/682: rename d0/d39/f75 to d0/dda/dc7/d52/dad/ff0 0 2024-02-15T18:27:02.543 INFO:tasks.workunit.client.1.smithi191.stdout:3/221: creat d6/d3b/d42/f51 x:0 0 0 2024-02-15T18:27:02.545 INFO:tasks.workunit.client.1.smithi191.stdout:1/338: truncate d0/d2f/d51/fe 809018 0 2024-02-15T18:27:02.545 INFO:tasks.workunit.client.0.smithi042.stdout:9/664: write d6/da/d1d/f24 [2371847,119975] 0 2024-02-15T18:27:02.545 INFO:tasks.workunit.client.0.smithi042.stdout:9/665: chown d6/da/d7b/f92 31998952 1 2024-02-15T18:27:02.545 INFO:tasks.workunit.client.0.smithi042.stdout:9/666: truncate d6/da/d1d/d2d/f83 126512 0 2024-02-15T18:27:02.545 INFO:tasks.workunit.client.0.smithi042.stdout:9/667: chown d6/da/d1d/d2d/d40/d48/l66 1179548420 1 2024-02-15T18:27:02.545 INFO:tasks.workunit.client.0.smithi042.stdout:5/683: write d0/d38/d53/f7a [3792488,9690] 0 2024-02-15T18:27:02.546 INFO:tasks.workunit.client.0.smithi042.stdout:0/624: mknod d4/d38/d5c/cd9 0 2024-02-15T18:27:02.551 INFO:tasks.workunit.client.0.smithi042.stdout:1/656: dwrite de/d15/d67/d46/f59 [4194304,4194304] 0 2024-02-15T18:27:02.551 INFO:tasks.workunit.client.0.smithi042.stdout:8/576: dwrite fc [4194304,4194304] 0 2024-02-15T18:27:02.551 INFO:tasks.workunit.client.1.smithi191.stdout:9/272: mknod d0/d7/da/d39/c67 0 2024-02-15T18:27:02.551 INFO:tasks.workunit.client.1.smithi191.stdout:1/339: creat d0/d8/df/d41/f6d x:0 0 0 2024-02-15T18:27:02.553 INFO:tasks.workunit.client.0.smithi042.stdout:3/641: rmdir d1/d11 39 2024-02-15T18:27:02.553 INFO:tasks.workunit.client.1.smithi191.stdout:1/340: dread d0/d2f/d51/f53 [0,4194304] 0 2024-02-15T18:27:02.555 INFO:tasks.workunit.client.1.smithi191.stdout:9/273: write d0/d29/f3d [605967,36153] 0 2024-02-15T18:27:02.555 INFO:tasks.workunit.client.1.smithi191.stdout:9/274: read - d0/d7/f11 zero size 2024-02-15T18:27:02.555 INFO:tasks.workunit.client.0.smithi042.stdout:6/727: creat d9/d11/d46/df8/f101 x:0 0 0 2024-02-15T18:27:02.556 INFO:tasks.workunit.client.0.smithi042.stdout:3/642: write d1/d11/d14/f43 [187371,94937] 0 2024-02-15T18:27:02.559 INFO:tasks.workunit.client.0.smithi042.stdout:4/639: rename da/d11/d67/d5c/c5f to da/d11/d67/d5c/d66/d78/d53/d82/dc0/ce1 0 2024-02-15T18:27:02.559 INFO:tasks.workunit.client.0.smithi042.stdout:4/640: readlink da/d3b/l6a 0 2024-02-15T18:27:02.560 INFO:tasks.workunit.client.0.smithi042.stdout:4/641: read - da/d3a/dc9/fd2 zero size 2024-02-15T18:27:02.560 INFO:tasks.workunit.client.0.smithi042.stdout:4/642: truncate da/d3a/dc9/fd7 865674 0 2024-02-15T18:27:02.560 INFO:tasks.workunit.client.0.smithi042.stdout:5/684: truncate d0/d38/d53/d91/db0/fb8 2289646 0 2024-02-15T18:27:02.560 INFO:tasks.workunit.client.0.smithi042.stdout:2/599: dread d0/f71 [0,4194304] 0 2024-02-15T18:27:02.562 INFO:tasks.workunit.client.0.smithi042.stdout:6/728: symlink d9/d11/d19/d35/d55/d92/dfc/l102 0 2024-02-15T18:27:02.562 INFO:tasks.workunit.client.0.smithi042.stdout:6/729: readlink d9/d11/d19/d35/d55/lbf 0 2024-02-15T18:27:02.566 INFO:tasks.workunit.client.0.smithi042.stdout:5/685: dread d0/d14/f1c [0,4194304] 0 2024-02-15T18:27:02.566 INFO:tasks.workunit.client.0.smithi042.stdout:5/686: chown d0/d14/fa3 1 1 2024-02-15T18:27:02.575 INFO:tasks.workunit.client.1.smithi191.stdout:3/222: dread d6/d3b/ff [0,4194304] 0 2024-02-15T18:27:02.576 INFO:tasks.workunit.client.1.smithi191.stdout:1/341: write d0/d8/fd [306034,99232] 0 2024-02-15T18:27:02.579 INFO:tasks.workunit.client.0.smithi042.stdout:7/690: sync 2024-02-15T18:27:02.579 INFO:tasks.workunit.client.0.smithi042.stdout:7/691: chown c2 114877 1 2024-02-15T18:27:02.584 INFO:tasks.workunit.client.0.smithi042.stdout:4/643: rmdir da/d11/d67/d5c/d66/d78/ddb 0 2024-02-15T18:27:02.588 INFO:tasks.workunit.client.1.smithi191.stdout:3/223: symlink d6/d3b/d42/d49/d21/l52 0 2024-02-15T18:27:02.588 INFO:tasks.workunit.client.1.smithi191.stdout:3/224: dread - d6/d3b/d42/d49/f43 zero size 2024-02-15T18:27:02.589 INFO:tasks.workunit.client.1.smithi191.stdout:1/342: rename d0/d8/df/d12/l50 to d0/d2f/d51/d54/d5f/d60/l6e 0 2024-02-15T18:27:02.589 INFO:tasks.workunit.client.0.smithi042.stdout:2/600: rmdir d0/d40 39 2024-02-15T18:27:02.589 INFO:tasks.workunit.client.0.smithi042.stdout:1/657: mknod de/d15/d67/cf1 0 2024-02-15T18:27:02.589 INFO:tasks.workunit.client.0.smithi042.stdout:6/730: mknod d9/c103 0 2024-02-15T18:27:02.590 INFO:tasks.workunit.client.1.smithi191.stdout:3/225: rename d6/d3b/d42/d49/c37 to d6/d3b/d42/c53 0 2024-02-15T18:27:02.590 INFO:tasks.workunit.client.0.smithi042.stdout:7/692: symlink d10/d49/d6d/d6e/le4 0 2024-02-15T18:27:02.591 INFO:tasks.workunit.client.0.smithi042.stdout:4/644: symlink da/d3b/le2 0 2024-02-15T18:27:02.592 INFO:tasks.workunit.client.1.smithi191.stdout:3/226: mkdir d6/d3b/d42/d49/d54 0 2024-02-15T18:27:02.592 INFO:tasks.workunit.client.0.smithi042.stdout:1/658: rmdir de/d15/d1b 39 2024-02-15T18:27:02.593 INFO:tasks.workunit.client.1.smithi191.stdout:1/343: dread d0/d8/df/d12/f28 [0,4194304] 0 2024-02-15T18:27:02.593 INFO:tasks.workunit.client.1.smithi191.stdout:1/344: truncate d0/d8/df/d12/d33/d49/f67 660787 0 2024-02-15T18:27:02.593 INFO:tasks.workunit.client.1.smithi191.stdout:1/345: chown d0/d8/df/d2e/l46 48 1 2024-02-15T18:27:02.594 INFO:tasks.workunit.client.0.smithi042.stdout:6/731: truncate d9/d3e/f75 3728813 0 2024-02-15T18:27:02.595 INFO:tasks.workunit.client.0.smithi042.stdout:7/693: unlink d10/l38 0 2024-02-15T18:27:02.596 INFO:tasks.workunit.client.1.smithi191.stdout:1/346: symlink d0/d2f/d51/d10/l6f 0 2024-02-15T18:27:02.596 INFO:tasks.workunit.client.1.smithi191.stdout:9/275: dwrite d0/d7/dc/d30/f4e [0,4194304] 0 2024-02-15T18:27:02.597 INFO:tasks.workunit.client.0.smithi042.stdout:4/645: mkdir da/d2b/de3 0 2024-02-15T18:27:02.602 INFO:tasks.workunit.client.0.smithi042.stdout:8/577: dwrite dd/de/d33/d66/f34 [0,4194304] 0 2024-02-15T18:27:02.602 INFO:tasks.workunit.client.0.smithi042.stdout:8/578: dread - dd/f48 zero size 2024-02-15T18:27:02.602 INFO:tasks.workunit.client.0.smithi042.stdout:8/579: stat dd/de/d12/l27 0 2024-02-15T18:27:02.602 INFO:tasks.workunit.client.0.smithi042.stdout:8/580: readlink dd/de/d12/l26 0 2024-02-15T18:27:02.602 INFO:tasks.workunit.client.0.smithi042.stdout:2/601: dwrite d0/d8/d10/d31/fa1 [4194304,4194304] 0 2024-02-15T18:27:02.602 INFO:tasks.workunit.client.0.smithi042.stdout:4/646: dread da/d11/f22 [0,4194304] 0 2024-02-15T18:27:02.602 INFO:tasks.workunit.client.1.smithi191.stdout:1/347: mknod d0/d2f/d51/d54/c70 0 2024-02-15T18:27:02.603 INFO:tasks.workunit.client.1.smithi191.stdout:9/276: write d0/d29/f3d [358641,14846] 0 2024-02-15T18:27:02.603 INFO:tasks.workunit.client.1.smithi191.stdout:9/277: fsync d0/f1 0 2024-02-15T18:27:02.603 INFO:tasks.workunit.client.1.smithi191.stdout:9/278: readlink d0/d7/dc/d17/d33/l45 0 2024-02-15T18:27:02.603 INFO:tasks.workunit.client.0.smithi042.stdout:1/659: dread de/d54/f5e [0,4194304] 0 2024-02-15T18:27:02.604 INFO:tasks.workunit.client.0.smithi042.stdout:6/732: mknod d9/d3e/c104 0 2024-02-15T18:27:02.605 INFO:tasks.workunit.client.0.smithi042.stdout:0/625: dwrite d4/d9/d5a/f6b [0,4194304] 0 2024-02-15T18:27:02.605 INFO:tasks.workunit.client.0.smithi042.stdout:5/687: dwrite d0/dda/dc7/de1/f35 [0,4194304] 0 2024-02-15T18:27:02.605 INFO:tasks.workunit.client.0.smithi042.stdout:0/626: fsync d4/d9/d15/d92/fca 0 2024-02-15T18:27:02.605 INFO:tasks.workunit.client.0.smithi042.stdout:5/688: dread - d0/d14/d88/f90 zero size 2024-02-15T18:27:02.605 INFO:tasks.workunit.client.0.smithi042.stdout:0/627: chown d4/d38/d39/d8e/d8f 35 1 2024-02-15T18:27:02.605 INFO:tasks.workunit.client.0.smithi042.stdout:3/643: dwrite d1/d11/d2e/d4c/f76 [0,4194304] 0 2024-02-15T18:27:02.605 INFO:tasks.workunit.client.0.smithi042.stdout:3/644: chown d1/de/d1a/l36 207395955 1 2024-02-15T18:27:02.605 INFO:tasks.workunit.client.0.smithi042.stdout:3/645: write d1/d11/f8f [15980,80468] 0 2024-02-15T18:27:02.607 INFO:tasks.workunit.client.1.smithi191.stdout:1/348: dread d0/d2f/d51/f1d [0,4194304] 0 2024-02-15T18:27:02.608 INFO:tasks.workunit.client.0.smithi042.stdout:6/733: dread d9/d11/d19/d35/d49/fd6 [0,4194304] 0 2024-02-15T18:27:02.608 INFO:tasks.workunit.client.0.smithi042.stdout:6/734: fdatasync d9/f72 0 2024-02-15T18:27:02.609 INFO:tasks.workunit.client.0.smithi042.stdout:7/694: symlink d10/d15/d8f/da5/dad/dda/le5 0 2024-02-15T18:27:02.609 INFO:tasks.workunit.client.0.smithi042.stdout:6/735: read d9/d11/d19/d1c/d31/d4f/f98 [798132,103342] 0 2024-02-15T18:27:02.609 INFO:tasks.workunit.client.0.smithi042.stdout:6/736: chown d9/f3c 898024166 1 2024-02-15T18:27:02.609 INFO:tasks.workunit.client.0.smithi042.stdout:6/737: truncate d9/d11/d46/d89/df4/f100 158988 0 2024-02-15T18:27:02.610 INFO:tasks.workunit.client.1.smithi191.stdout:9/279: write d0/d1d/f58 [2526790,34200] 0 2024-02-15T18:27:02.611 INFO:tasks.workunit.client.1.smithi191.stdout:1/349: write d0/d2f/d51/d10/f25 [185806,96700] 0 2024-02-15T18:27:02.612 INFO:tasks.workunit.client.1.smithi191.stdout:3/227: dwrite d6/d3b/ff [4194304,4194304] 0 2024-02-15T18:27:02.614 INFO:tasks.workunit.client.0.smithi042.stdout:8/581: symlink dd/de/da5/la7 0 2024-02-15T18:27:02.614 INFO:tasks.workunit.client.0.smithi042.stdout:8/582: stat dd/de/d12/f9e 0 2024-02-15T18:27:02.618 INFO:tasks.workunit.client.0.smithi042.stdout:2/602: creat d0/d8/d10/d5a/fbc x:0 0 0 2024-02-15T18:27:02.620 INFO:tasks.workunit.client.0.smithi042.stdout:4/647: dwrite da/d11/f3c [0,4194304] 0 2024-02-15T18:27:02.620 INFO:tasks.workunit.client.1.smithi191.stdout:9/280: creat d0/d29/d48/d66/f68 x:0 0 0 2024-02-15T18:27:02.626 INFO:tasks.workunit.client.1.smithi191.stdout:3/228: creat d6/d3b/d42/f55 x:0 0 0 2024-02-15T18:27:02.627 INFO:tasks.workunit.client.1.smithi191.stdout:3/229: read - d6/d27/d4d/f4e zero size 2024-02-15T18:27:02.627 INFO:tasks.workunit.client.1.smithi191.stdout:3/230: truncate d6/d3b/d30/f50 189005 0 2024-02-15T18:27:02.629 INFO:tasks.workunit.client.1.smithi191.stdout:9/281: creat d0/d7/dc/d17/d33/f69 x:0 0 0 2024-02-15T18:27:02.629 INFO:tasks.workunit.client.1.smithi191.stdout:9/282: chown d0/c5a 27252 1 2024-02-15T18:27:02.630 INFO:tasks.workunit.client.1.smithi191.stdout:1/350: dwrite d0/d2f/d51/f14 [4194304,4194304] 0 2024-02-15T18:27:02.630 INFO:tasks.workunit.client.0.smithi042.stdout:5/689: mkdir d0/d39/d8d/d9b/df1 0 2024-02-15T18:27:02.640 INFO:tasks.workunit.client.1.smithi191.stdout:1/351: creat d0/d2f/d5c/f71 x:0 0 0 2024-02-15T18:27:02.641 INFO:tasks.workunit.client.0.smithi042.stdout:3/646: mkdir d1/d11/d2e/dd1 0 2024-02-15T18:27:02.641 INFO:tasks.workunit.client.0.smithi042.stdout:7/695: creat d10/d49/d6d/fe6 x:0 0 0 2024-02-15T18:27:02.641 INFO:tasks.workunit.client.0.smithi042.stdout:7/696: readlink d10/d49/lc0 0 2024-02-15T18:27:02.642 INFO:tasks.workunit.client.0.smithi042.stdout:7/697: chown d10/d15/d39/de3 470879689 1 2024-02-15T18:27:02.642 INFO:tasks.workunit.client.0.smithi042.stdout:7/698: chown d10/d12/f9e 185 1 2024-02-15T18:27:02.642 INFO:tasks.workunit.client.0.smithi042.stdout:8/583: getdents dd/de/d33/d66 0 2024-02-15T18:27:02.643 INFO:tasks.workunit.client.1.smithi191.stdout:1/352: symlink d0/d2f/d51/d54/d5f/d60/l72 0 2024-02-15T18:27:02.643 INFO:tasks.workunit.client.1.smithi191.stdout:1/353: chown d0/d2f/d51/d54/d5f/d60/l6e 105 1 2024-02-15T18:27:02.648 INFO:tasks.workunit.client.1.smithi191.stdout:3/231: dwrite d6/d3b/d42/d49/f43 [0,4194304] 0 2024-02-15T18:27:02.648 INFO:tasks.workunit.client.1.smithi191.stdout:3/232: chown d6/d3b/d42/f2a 146836 1 2024-02-15T18:27:02.652 INFO:tasks.workunit.client.0.smithi042.stdout:2/603: rmdir d0/d8/d20/d7d/d7e 39 2024-02-15T18:27:02.654 INFO:tasks.workunit.client.1.smithi191.stdout:3/233: rename d6/d3b/d42/d49/d21 to d6/d3b/d56 0 2024-02-15T18:27:02.656 INFO:tasks.workunit.client.1.smithi191.stdout:3/234: symlink d6/d1b/l57 0 2024-02-15T18:27:02.658 INFO:tasks.workunit.client.0.smithi042.stdout:4/648: link da/d11/d67/d5c/d66/d78/f3e da/d11/d29/d4a/d4f/d90/fe4 0 2024-02-15T18:27:02.660 INFO:tasks.workunit.client.0.smithi042.stdout:1/660: rename de/d15/d2c/l96 to de/d54/dbf/dd0/d7c/dd6/de8/lf2 0 2024-02-15T18:27:02.662 INFO:tasks.workunit.client.1.smithi191.stdout:9/283: dwrite d0/d7/f19 [0,4194304] 0 2024-02-15T18:27:02.662 INFO:tasks.workunit.client.1.smithi191.stdout:1/354: truncate d0/d2f/d51/d10/f25 1716023 0 2024-02-15T18:27:02.664 INFO:tasks.workunit.client.1.smithi191.stdout:9/284: write d0/d7/da/f3e [460913,89137] 0 2024-02-15T18:27:02.664 INFO:tasks.workunit.client.1.smithi191.stdout:3/235: rename d6/d27/d4d/f4e to d6/d27/d4d/f58 0 2024-02-15T18:27:02.664 INFO:tasks.workunit.client.1.smithi191.stdout:3/236: write d6/fa [270841,27048] 0 2024-02-15T18:27:02.664 INFO:tasks.workunit.client.1.smithi191.stdout:3/237: chown c1 18724685 1 2024-02-15T18:27:02.665 INFO:tasks.workunit.client.0.smithi042.stdout:9/668: sync 2024-02-15T18:27:02.666 INFO:tasks.workunit.client.0.smithi042.stdout:5/690: truncate d0/d39/fef 741290 0 2024-02-15T18:27:02.669 INFO:tasks.workunit.client.0.smithi042.stdout:0/628: mkdir d4/d38/d77/d2e/dda 0 2024-02-15T18:27:02.670 INFO:tasks.workunit.client.0.smithi042.stdout:8/584: creat dd/de/d33/d3b/d92/fa8 x:0 0 0 2024-02-15T18:27:02.671 INFO:tasks.workunit.client.1.smithi191.stdout:3/238: creat d6/d3b/d42/d49/f59 x:0 0 0 2024-02-15T18:27:02.672 INFO:tasks.workunit.client.0.smithi042.stdout:7/699: dwrite d10/d12/d23/f64 [4194304,4194304] 0 2024-02-15T18:27:02.673 INFO:tasks.workunit.client.0.smithi042.stdout:7/700: fsync d10/f21 0 2024-02-15T18:27:02.675 INFO:tasks.workunit.client.0.smithi042.stdout:2/604: truncate d0/d8/d10/d31/d3a/d53/f58 106552 0 2024-02-15T18:27:02.675 INFO:tasks.workunit.client.0.smithi042.stdout:4/649: truncate da/d2b/fd8 3725679 0 2024-02-15T18:27:02.677 INFO:tasks.workunit.client.0.smithi042.stdout:3/647: rename d1/c52 to d1/d11/d7a/cd2 0 2024-02-15T18:27:02.677 INFO:tasks.workunit.client.0.smithi042.stdout:3/648: read - d1/de/d19/fb4 zero size 2024-02-15T18:27:02.678 INFO:tasks.workunit.client.0.smithi042.stdout:5/691: creat d0/d50/ff2 x:0 0 0 2024-02-15T18:27:02.679 INFO:tasks.workunit.client.0.smithi042.stdout:8/585: rmdir dd/de/da4 39 2024-02-15T18:27:02.681 INFO:tasks.workunit.client.0.smithi042.stdout:2/605: unlink d0/d8/d20/d7d/d7e/ca7 0 2024-02-15T18:27:02.681 INFO:tasks.workunit.client.0.smithi042.stdout:2/606: write d0/d8/d10/d31/d3a/d53/f45 [312188,69844] 0 2024-02-15T18:27:02.681 INFO:tasks.workunit.client.1.smithi191.stdout:9/285: dread d0/f63 [0,4194304] 0 2024-02-15T18:27:02.681 INFO:tasks.workunit.client.1.smithi191.stdout:9/286: chown d0/d1d/c2f 0 1 2024-02-15T18:27:02.684 INFO:tasks.workunit.client.0.smithi042.stdout:7/701: dread d10/d15/d39/d6c/f6f [0,4194304] 0 2024-02-15T18:27:02.684 INFO:tasks.workunit.client.0.smithi042.stdout:7/702: fdatasync d10/d49/d6d/f86 0 2024-02-15T18:27:02.684 INFO:tasks.workunit.client.0.smithi042.stdout:0/629: write d4/f68 [2061433,112821] 0 2024-02-15T18:27:02.687 INFO:tasks.workunit.client.0.smithi042.stdout:2/607: write d0/d8/db/d44/f84 [1810355,40750] 0 2024-02-15T18:27:02.688 INFO:tasks.workunit.client.0.smithi042.stdout:1/661: dwrite de/d15/d67/d63/f86 [0,4194304] 0 2024-02-15T18:27:02.690 INFO:tasks.workunit.client.0.smithi042.stdout:6/738: sync 2024-02-15T18:27:02.692 INFO:tasks.workunit.client.0.smithi042.stdout:4/650: rmdir da/d11/d67/d5c/dce 0 2024-02-15T18:27:02.693 INFO:tasks.workunit.client.0.smithi042.stdout:3/649: link d1/c96 d1/de/d19/cd3 0 2024-02-15T18:27:02.695 INFO:tasks.workunit.client.0.smithi042.stdout:4/651: write da/d11/d67/f6f [3418291,37707] 0 2024-02-15T18:27:02.695 INFO:tasks.workunit.client.0.smithi042.stdout:4/652: chown da/d11/d67/fd5 126 1 2024-02-15T18:27:02.695 INFO:tasks.workunit.client.0.smithi042.stdout:4/653: read - da/d11/d29/db8/fb9 zero size 2024-02-15T18:27:02.698 INFO:tasks.workunit.client.1.smithi191.stdout:9/287: mknod d0/d7/dc/d30/d4f/c6a 0 2024-02-15T18:27:02.699 INFO:tasks.workunit.client.0.smithi042.stdout:9/669: rename d6/dd/d9/c51 to d6/da/d1d/d2d/d40/de0/ce5 0 2024-02-15T18:27:02.699 INFO:tasks.workunit.client.0.smithi042.stdout:9/670: chown d6/da/d1d/f38 13348509 1 2024-02-15T18:27:02.702 INFO:tasks.workunit.client.0.smithi042.stdout:4/654: dread da/f24 [0,4194304] 0 2024-02-15T18:27:02.702 INFO:tasks.workunit.client.0.smithi042.stdout:4/655: chown da/d2b/fd6 96589 1 2024-02-15T18:27:02.703 INFO:tasks.workunit.client.0.smithi042.stdout:4/656: readlink da/d11/d29/d4a/d4f/d90/lac 0 2024-02-15T18:27:02.703 INFO:tasks.workunit.client.0.smithi042.stdout:5/692: truncate d0/d89/f97 415405 0 2024-02-15T18:27:02.703 INFO:tasks.workunit.client.0.smithi042.stdout:8/586: dread - dd/d11/d75/f9d zero size 2024-02-15T18:27:02.703 INFO:tasks.workunit.client.0.smithi042.stdout:8/587: write dd/de/d33/d66/f34 [2407896,106582] 0 2024-02-15T18:27:02.703 INFO:tasks.workunit.client.0.smithi042.stdout:7/703: unlink d10/d49/d6d/fe6 0 2024-02-15T18:27:02.703 INFO:tasks.workunit.client.0.smithi042.stdout:7/704: write d10/d12/d65/f88 [603446,91050] 0 2024-02-15T18:27:02.706 INFO:tasks.workunit.client.1.smithi191.stdout:1/355: dwrite d0/f9 [0,4194304] 0 2024-02-15T18:27:02.709 INFO:tasks.workunit.client.0.smithi042.stdout:0/630: rmdir d4 39 2024-02-15T18:27:02.709 INFO:tasks.workunit.client.1.smithi191.stdout:1/356: dread d0/d8/df/d12/f5d [0,4194304] 0 2024-02-15T18:27:02.710 INFO:tasks.workunit.client.1.smithi191.stdout:1/357: stat d0/d8/df/d12/l56 0 2024-02-15T18:27:02.711 INFO:tasks.workunit.client.0.smithi042.stdout:1/662: mkdir de/d15/d1b/d44/df3 0 2024-02-15T18:27:02.711 INFO:tasks.workunit.client.0.smithi042.stdout:1/663: chown f6 7491 1 2024-02-15T18:27:02.712 INFO:tasks.workunit.client.0.smithi042.stdout:1/664: write de/d6d/d9e/dcd/fe4 [3782816,112514] 0 2024-02-15T18:27:02.714 INFO:tasks.workunit.client.0.smithi042.stdout:3/650: dread d1/d11/d2e/f37 [4194304,4194304] 0 2024-02-15T18:27:02.715 INFO:tasks.workunit.client.0.smithi042.stdout:2/608: dwrite d0/d8/f39 [0,4194304] 0 2024-02-15T18:27:02.716 INFO:tasks.workunit.client.0.smithi042.stdout:6/739: creat d9/d11/d46/d89/df4/f105 x:0 0 0 2024-02-15T18:27:02.720 INFO:tasks.workunit.client.1.smithi191.stdout:9/288: dwrite d0/d7/dc/d17/d33/f4d [4194304,4194304] 0 2024-02-15T18:27:02.720 INFO:tasks.workunit.client.1.smithi191.stdout:9/289: write d0/d7/dc/d17/d33/f69 [621871,30000] 0 2024-02-15T18:27:02.720 INFO:tasks.workunit.client.1.smithi191.stdout:9/290: chown d0/d29/d48/d66/f68 51281 1 2024-02-15T18:27:02.720 INFO:tasks.workunit.client.0.smithi042.stdout:9/671: link d6/da/d2f/d69/f7d d6/da/d2f/d69/fe6 0 2024-02-15T18:27:02.720 INFO:tasks.workunit.client.0.smithi042.stdout:9/672: write d6/da/d2f/d6c/f71 [3588004,127403] 0 2024-02-15T18:27:02.721 INFO:tasks.workunit.client.0.smithi042.stdout:3/651: dread d1/d11/d2e/f3b [0,4194304] 0 2024-02-15T18:27:02.726 INFO:tasks.workunit.client.0.smithi042.stdout:4/657: rename da/d11/d29/f2c to da/d11/fe5 0 2024-02-15T18:27:02.728 INFO:tasks.workunit.client.1.smithi191.stdout:1/358: rmdir d0/d8/df/d41 39 2024-02-15T18:27:02.732 INFO:tasks.workunit.client.0.smithi042.stdout:5/693: creat d0/d38/d53/d64/ff3 x:0 0 0 2024-02-15T18:27:02.734 INFO:tasks.workunit.client.1.smithi191.stdout:9/291: link d0/f51 d0/d7/da/f6b 0 2024-02-15T18:27:02.734 INFO:tasks.workunit.client.1.smithi191.stdout:9/292: write d0/d29/d48/f5b [618001,104436] 0 2024-02-15T18:27:02.734 INFO:tasks.workunit.client.1.smithi191.stdout:9/293: chown d0/d7/dc 7 1 2024-02-15T18:27:02.735 INFO:tasks.workunit.client.1.smithi191.stdout:9/294: dread d0/f12 [0,4194304] 0 2024-02-15T18:27:02.736 INFO:tasks.workunit.client.1.smithi191.stdout:1/359: link d0/d8/df/d12/d2a/f52 d0/d2f/f73 0 2024-02-15T18:27:02.737 INFO:tasks.workunit.client.0.smithi042.stdout:8/588: rmdir dd/de/da4/d67/d6f/d78 39 2024-02-15T18:27:02.737 INFO:tasks.workunit.client.0.smithi042.stdout:8/589: dread - dd/f48 zero size 2024-02-15T18:27:02.739 INFO:tasks.workunit.client.0.smithi042.stdout:7/705: unlink d10/d12/d23/d51/c76 0 2024-02-15T18:27:02.740 INFO:tasks.workunit.client.1.smithi191.stdout:1/360: rmdir d0/d8/df/d12/d2a/d2b 39 2024-02-15T18:27:02.741 INFO:tasks.workunit.client.0.smithi042.stdout:0/631: getdents d4/d38/d77/d2e/dae 0 2024-02-15T18:27:02.742 INFO:tasks.workunit.client.0.smithi042.stdout:6/740: dwrite d9/d11/d1d/f90 [0,4194304] 0 2024-02-15T18:27:02.742 INFO:tasks.workunit.client.0.smithi042.stdout:1/665: mkdir de/d6d/d9e/df4 0 2024-02-15T18:27:02.745 INFO:tasks.workunit.client.1.smithi191.stdout:1/361: write d0/f3 [10045274,22295] 0 2024-02-15T18:27:02.745 INFO:tasks.workunit.client.0.smithi042.stdout:9/673: creat d6/da/d75/fe7 x:0 0 0 2024-02-15T18:27:02.745 INFO:tasks.workunit.client.0.smithi042.stdout:9/674: stat d6/da/d1d/d2d/d40/f43 0 2024-02-15T18:27:02.746 INFO:tasks.workunit.client.0.smithi042.stdout:2/609: dwrite d0/d8/d10/d31/d3a/d53/f45 [0,4194304] 0 2024-02-15T18:27:02.746 INFO:tasks.workunit.client.0.smithi042.stdout:2/610: fdatasync d0/d40/d56/fa9 0 2024-02-15T18:27:02.747 INFO:tasks.workunit.client.0.smithi042.stdout:3/652: rename d1/de/d19/cc8 to d1/d11/d14/d4d/d6c/d7b/cd4 0 2024-02-15T18:27:02.747 INFO:tasks.workunit.client.0.smithi042.stdout:3/653: stat d1/d11/d14/d4d/d6c/f75 0 2024-02-15T18:27:02.748 INFO:tasks.workunit.client.0.smithi042.stdout:9/675: write d6/dd/d9/f18 [3363593,99980] 0 2024-02-15T18:27:02.750 INFO:tasks.workunit.client.0.smithi042.stdout:4/658: link da/d11/d29/d4a/d4f/d90/fe4 da/d11/d67/d5c/d66/d78/d53/d82/dc0/fe6 0 2024-02-15T18:27:02.750 INFO:tasks.workunit.client.0.smithi042.stdout:5/694: dwrite d0/d38/d53/d64/f93 [4194304,4194304] 0 2024-02-15T18:27:02.750 INFO:tasks.workunit.client.0.smithi042.stdout:5/695: stat d0/d14/d88/da1/fb1 0 2024-02-15T18:27:02.750 INFO:tasks.workunit.client.1.smithi191.stdout:1/362: truncate d0/d2f/d51/d10/f2d 1895065 0 2024-02-15T18:27:02.751 INFO:tasks.workunit.client.1.smithi191.stdout:9/295: dread d0/d7/dc/d17/d33/f4d [0,4194304] 0 2024-02-15T18:27:02.751 INFO:tasks.workunit.client.1.smithi191.stdout:1/363: write d0/d8/f57 [4308176,67562] 0 2024-02-15T18:27:02.751 INFO:tasks.workunit.client.0.smithi042.stdout:4/659: write da/d11/f23 [1466552,99659] 0 2024-02-15T18:27:02.754 INFO:tasks.workunit.client.0.smithi042.stdout:8/590: mknod dd/de/d12/ca9 0 2024-02-15T18:27:02.754 INFO:tasks.workunit.client.0.smithi042.stdout:8/591: fsync dd/de/d3c/f6d 0 2024-02-15T18:27:02.754 INFO:tasks.workunit.client.0.smithi042.stdout:8/592: dread - dd/d11/d75/fa1 zero size 2024-02-15T18:27:02.754 INFO:tasks.workunit.client.0.smithi042.stdout:8/593: dread - dd/de/d33/f42 zero size 2024-02-15T18:27:02.754 INFO:tasks.workunit.client.0.smithi042.stdout:4/660: dread da/d2b/d7e/fa6 [0,4194304] 0 2024-02-15T18:27:02.754 INFO:tasks.workunit.client.0.smithi042.stdout:4/661: chown da/d11/d67/d5c/d66/d78 16964 1 2024-02-15T18:27:02.755 INFO:tasks.workunit.client.0.smithi042.stdout:4/662: fsync da/d11/d29/d4a/d4f/f9d 0 2024-02-15T18:27:02.758 INFO:tasks.workunit.client.0.smithi042.stdout:7/706: creat d10/d49/d6d/d78/d83/fe7 x:0 0 0 2024-02-15T18:27:02.761 INFO:tasks.workunit.client.0.smithi042.stdout:6/741: dwrite d9/d11/d19/d35/d49/f4c [4194304,4194304] 0 2024-02-15T18:27:02.761 INFO:tasks.workunit.client.0.smithi042.stdout:6/742: write d9/d11/d19/d35/dc4/fe0 [904207,104578] 0 2024-02-15T18:27:02.763 INFO:tasks.workunit.client.1.smithi191.stdout:1/364: mkdir d0/d2f/d51/d54/d5f/d60/d74 0 2024-02-15T18:27:02.763 INFO:tasks.workunit.client.0.smithi042.stdout:0/632: symlink d4/d9/d8d/ldb 0 2024-02-15T18:27:02.764 INFO:tasks.workunit.client.1.smithi191.stdout:9/296: symlink d0/d7/dc/d32/l6c 0 2024-02-15T18:27:02.766 INFO:tasks.workunit.client.1.smithi191.stdout:1/365: mknod d0/d8/c75 0 2024-02-15T18:27:02.768 INFO:tasks.workunit.client.1.smithi191.stdout:9/297: creat d0/d7/dc/d30/d4f/f6d x:0 0 0 2024-02-15T18:27:02.769 INFO:tasks.workunit.client.0.smithi042.stdout:1/666: creat de/d54/dbf/dd0/d7c/dd6/ddf/d88/ff5 x:0 0 0 2024-02-15T18:27:02.770 INFO:tasks.workunit.client.0.smithi042.stdout:1/667: read de/d15/d1b/f25 [691811,12445] 0 2024-02-15T18:27:02.770 INFO:tasks.workunit.client.0.smithi042.stdout:1/668: truncate de/d54/dbf/dd0/d7c/fd9 11269 0 2024-02-15T18:27:02.774 INFO:tasks.workunit.client.1.smithi191.stdout:9/298: unlink d0/d1d/f58 0 2024-02-15T18:27:02.775 INFO:tasks.workunit.client.0.smithi042.stdout:2/611: mkdir d0/d8/d10/d31/d79/dbd 0 2024-02-15T18:27:02.775 INFO:tasks.workunit.client.0.smithi042.stdout:2/612: chown d0/d8/d20/d7d/d94/l3b 1 1 2024-02-15T18:27:02.777 INFO:tasks.workunit.client.0.smithi042.stdout:3/654: getdents d1/d11/d14/d4d/d6c/d7b 0 2024-02-15T18:27:02.779 INFO:tasks.workunit.client.1.smithi191.stdout:9/299: rename d0/f3c to d0/d7/dc/d17/d44/f6e 0 2024-02-15T18:27:02.779 INFO:tasks.workunit.client.0.smithi042.stdout:9/676: mkdir d6/dd/dcc/de8 0 2024-02-15T18:27:02.781 INFO:tasks.workunit.client.1.smithi191.stdout:9/300: mknod d0/d1d/d3f/c6f 0 2024-02-15T18:27:02.781 INFO:tasks.workunit.client.1.smithi191.stdout:9/301: write d0/d7/f28 [318999,104804] 0 2024-02-15T18:27:02.781 INFO:tasks.workunit.client.1.smithi191.stdout:1/366: dwrite d0/d8/fd [0,4194304] 0 2024-02-15T18:27:02.783 INFO:tasks.workunit.client.0.smithi042.stdout:5/696: link d0/d38/d53/db4/fe9 d0/dda/dc7/de1/dd8/ff4 0 2024-02-15T18:27:02.787 INFO:tasks.workunit.client.0.smithi042.stdout:8/594: truncate dd/de/f2a 837522 0 2024-02-15T18:27:02.787 INFO:tasks.workunit.client.0.smithi042.stdout:8/595: stat dd/d11/d75/f95 0 2024-02-15T18:27:02.787 INFO:tasks.workunit.client.0.smithi042.stdout:8/596: chown dd/f7d 134282108 1 2024-02-15T18:27:02.787 INFO:tasks.workunit.client.1.smithi191.stdout:9/302: creat d0/d29/f70 x:0 0 0 2024-02-15T18:27:02.791 INFO:tasks.workunit.client.1.smithi191.stdout:9/303: write d0/d29/d48/f5b [378221,118206] 0 2024-02-15T18:27:02.791 INFO:tasks.workunit.client.0.smithi042.stdout:4/663: creat da/d11/d67/fe7 x:0 0 0 2024-02-15T18:27:02.792 INFO:tasks.workunit.client.0.smithi042.stdout:4/664: chown da/d11/d29/ldf 1788632 1 2024-02-15T18:27:02.792 INFO:tasks.workunit.client.0.smithi042.stdout:8/597: write dd/de/da4/d67/f2e [5889731,38960] 0 2024-02-15T18:27:02.792 INFO:tasks.workunit.client.0.smithi042.stdout:8/598: read dd/de/d33/f53 [1195097,12880] 0 2024-02-15T18:27:02.792 INFO:tasks.workunit.client.0.smithi042.stdout:8/599: chown dd/de/da4/d67/d6f/d78 2 1 2024-02-15T18:27:02.792 INFO:tasks.workunit.client.0.smithi042.stdout:7/707: link d10/d12/d23/f55 d10/d49/d6d/d78/d83/fe8 0 2024-02-15T18:27:02.792 INFO:tasks.workunit.client.1.smithi191.stdout:1/367: truncate d0/d2f/d51/f14 5610769 0 2024-02-15T18:27:02.793 INFO:tasks.workunit.client.1.smithi191.stdout:1/368: write d0/d8/df/d12/d33/f63 [424743,112766] 0 2024-02-15T18:27:02.793 INFO:tasks.workunit.client.1.smithi191.stdout:1/369: write d0/d2f/d51/d54/f55 [2930003,107962] 0 2024-02-15T18:27:02.793 INFO:tasks.workunit.client.1.smithi191.stdout:1/370: fsync d0/d8/df/d12/d2a/f3a 0 2024-02-15T18:27:02.797 INFO:tasks.workunit.client.0.smithi042.stdout:6/743: creat d9/d11/d19/d1c/d31/d53/dbe/f106 x:0 0 0 2024-02-15T18:27:02.803 INFO:tasks.workunit.client.0.smithi042.stdout:0/633: rmdir d4/d38/d39/d8e/d8f/db2 39 2024-02-15T18:27:02.805 INFO:tasks.workunit.client.1.smithi191.stdout:1/371: unlink d0/d8/df/d12/f5d 0 2024-02-15T18:27:02.806 INFO:tasks.workunit.client.0.smithi042.stdout:1/669: getdents de/d54/d9a/dc8 0 2024-02-15T18:27:02.807 INFO:tasks.workunit.client.1.smithi191.stdout:9/304: mknod d0/c71 0 2024-02-15T18:27:02.808 INFO:tasks.workunit.client.0.smithi042.stdout:2/613: mkdir d0/d8/d20/d41/dbe 0 2024-02-15T18:27:02.809 INFO:tasks.workunit.client.0.smithi042.stdout:3/655: creat d1/fd5 x:0 0 0 2024-02-15T18:27:02.811 INFO:tasks.workunit.client.0.smithi042.stdout:9/677: creat d6/da/d2f/d55/d73/d8d/fe9 x:0 0 0 2024-02-15T18:27:02.814 INFO:tasks.workunit.client.1.smithi191.stdout:9/305: dread d0/d29/f3d [0,4194304] 0 2024-02-15T18:27:02.824 INFO:tasks.workunit.client.1.smithi191.stdout:1/372: symlink d0/d8/df/d12/d2a/d2b/l76 0 2024-02-15T18:27:02.824 INFO:tasks.workunit.client.1.smithi191.stdout:9/306: rename d0/f12 to d0/d7/da/f72 0 2024-02-15T18:27:02.825 INFO:tasks.workunit.client.0.smithi042.stdout:4/665: dwrite da/d11/d29/f70 [4194304,4194304] 0 2024-02-15T18:27:02.826 INFO:tasks.workunit.client.0.smithi042.stdout:7/708: mkdir d10/d15/d39/dc2/de9 0 2024-02-15T18:27:02.827 INFO:tasks.workunit.client.1.smithi191.stdout:9/307: symlink d0/d7/dc/d17/d33/l73 0 2024-02-15T18:27:02.828 INFO:tasks.workunit.client.0.smithi042.stdout:0/634: mknod d4/d38/d39/cdc 0 2024-02-15T18:27:02.830 INFO:tasks.workunit.client.0.smithi042.stdout:1/670: truncate de/d54/f5e 3709196 0 2024-02-15T18:27:02.833 INFO:tasks.workunit.client.0.smithi042.stdout:9/678: rename d6/da/d1d/d2d/d40/d5f/fbb to d6/da/d2f/d55/d73/d8d/fea 0 2024-02-15T18:27:02.833 INFO:tasks.workunit.client.0.smithi042.stdout:9/679: chown d6/d9d 44987486 1 2024-02-15T18:27:02.833 INFO:tasks.workunit.client.0.smithi042.stdout:9/680: chown d6/dd/d9/f14 139923 1 2024-02-15T18:27:02.834 INFO:tasks.workunit.client.0.smithi042.stdout:1/671: link de/d15/d67/d63/f8f de/d15/ff6 0 2024-02-15T18:27:02.838 INFO:tasks.workunit.client.0.smithi042.stdout:9/681: symlink d6/da/d75/leb 0 2024-02-15T18:27:02.838 INFO:tasks.workunit.client.0.smithi042.stdout:9/682: fsync d6/f31 0 2024-02-15T18:27:02.839 INFO:tasks.workunit.client.0.smithi042.stdout:1/672: mkdir de/d54/de6/df7 0 2024-02-15T18:27:02.839 INFO:tasks.workunit.client.0.smithi042.stdout:1/673: write de/d54/dbf/dd0/d7c/dd6/ddf/d88/ff5 [414342,32158] 0 2024-02-15T18:27:02.839 INFO:tasks.workunit.client.0.smithi042.stdout:1/674: truncate de/f48 126764 0 2024-02-15T18:27:02.840 INFO:tasks.workunit.client.1.smithi191.stdout:1/373: dread d0/d8/df/d12/d2a/f42 [0,4194304] 0 2024-02-15T18:27:02.841 INFO:tasks.workunit.client.0.smithi042.stdout:8/600: dwrite dd/de/d3c/f7e [0,4194304] 0 2024-02-15T18:27:02.846 INFO:tasks.workunit.client.0.smithi042.stdout:6/744: dwrite d9/d11/d19/d35/d55/d92/f95 [0,4194304] 0 2024-02-15T18:27:02.854 INFO:tasks.workunit.client.0.smithi042.stdout:3/656: dwrite d1/de/f2d [8388608,4194304] 0 2024-02-15T18:27:02.856 INFO:tasks.workunit.client.0.smithi042.stdout:2/614: dwrite d0/d8/d10/d31/da6/fad [0,4194304] 0 2024-02-15T18:27:02.859 INFO:tasks.workunit.client.0.smithi042.stdout:4/666: dwrite da/d11/d67/d5c/d66/d78/d2f/dca/fd3 [0,4194304] 0 2024-02-15T18:27:02.859 INFO:tasks.workunit.client.0.smithi042.stdout:4/667: fdatasync da/d11/d29/d4a/d4f/d80/fa1 0 2024-02-15T18:27:02.859 INFO:tasks.workunit.client.0.smithi042.stdout:4/668: stat da/d11/fe5 0 2024-02-15T18:27:02.866 INFO:tasks.workunit.client.1.smithi191.stdout:5/168: sync 2024-02-15T18:27:02.866 INFO:tasks.workunit.client.1.smithi191.stdout:2/257: sync 2024-02-15T18:27:02.866 INFO:tasks.workunit.client.1.smithi191.stdout:8/382: sync 2024-02-15T18:27:02.874 INFO:tasks.workunit.client.0.smithi042.stdout:0/635: dwrite d4/d9/d62/fa0 [4194304,4194304] 0 2024-02-15T18:27:02.877 INFO:tasks.workunit.client.1.smithi191.stdout:2/258: creat df/d12/d19/d1a/f5a x:0 0 0 2024-02-15T18:27:02.879 INFO:tasks.workunit.client.0.smithi042.stdout:0/636: getdents d4/d9/d34/d57/d88 0 2024-02-15T18:27:02.879 INFO:tasks.workunit.client.0.smithi042.stdout:4/669: creat da/d11/d29/fe8 x:0 0 0 2024-02-15T18:27:02.879 INFO:tasks.workunit.client.0.smithi042.stdout:4/670: chown da/d11/d67/d5c/d66/fbf 255469776 1 2024-02-15T18:27:02.881 INFO:tasks.workunit.client.0.smithi042.stdout:0/637: getdents d4/d38/d77/d2e 0 2024-02-15T18:27:02.883 INFO:tasks.workunit.client.0.smithi042.stdout:0/638: link d4/dc2/fd3 d4/d38/fdd 0 2024-02-15T18:27:02.884 INFO:tasks.workunit.client.1.smithi191.stdout:2/259: dread df/d12/f2b [0,4194304] 0 2024-02-15T18:27:02.885 INFO:tasks.workunit.client.1.smithi191.stdout:1/374: dread d0/f3 [8388608,4194304] 0 2024-02-15T18:27:02.886 INFO:tasks.workunit.client.1.smithi191.stdout:1/375: mkdir d0/d2f/d77 0 2024-02-15T18:27:02.887 INFO:tasks.workunit.client.1.smithi191.stdout:2/260: read df/f10 [2750440,33039] 0 2024-02-15T18:27:02.887 INFO:tasks.workunit.client.1.smithi191.stdout:2/261: write df/d12/f36 [5201767,89649] 0 2024-02-15T18:27:02.888 INFO:tasks.workunit.client.1.smithi191.stdout:1/376: rename d0/d2f/d51/d54/d5f to d0/d8/df/d12/d2a/d2b/d78 0 2024-02-15T18:27:02.888 INFO:tasks.workunit.client.1.smithi191.stdout:1/377: dread - d0/d8/df/d12/d2a/d2b/f61 zero size 2024-02-15T18:27:02.888 INFO:tasks.workunit.client.0.smithi042.stdout:6/745: dread d9/d11/d19/d35/d55/d92/fe8 [4194304,4194304] 0 2024-02-15T18:27:02.889 INFO:tasks.workunit.client.1.smithi191.stdout:2/262: getdents df/d12/d1e 0 2024-02-15T18:27:02.889 INFO:tasks.workunit.client.0.smithi042.stdout:3/657: dread d1/d11/d2e/d4c/f76 [0,4194304] 0 2024-02-15T18:27:02.889 INFO:tasks.workunit.client.0.smithi042.stdout:3/658: getdents d1/d11/d14/d4d 0 2024-02-15T18:27:02.890 INFO:tasks.workunit.client.1.smithi191.stdout:1/378: symlink d0/d8/df/d41/l79 0 2024-02-15T18:27:02.891 INFO:tasks.workunit.client.0.smithi042.stdout:6/746: truncate d9/d11/f2b 787370 0 2024-02-15T18:27:02.892 INFO:tasks.workunit.client.1.smithi191.stdout:2/263: mkdir df/d12/d19/d1a/d35/d5b 0 2024-02-15T18:27:02.893 INFO:tasks.workunit.client.1.smithi191.stdout:2/264: readlink df/d12/d19/l2e 0 2024-02-15T18:27:02.893 INFO:tasks.workunit.client.1.smithi191.stdout:2/265: getdents df/d12/d19/d1a/d52 0 2024-02-15T18:27:02.893 INFO:tasks.workunit.client.1.smithi191.stdout:2/266: write df/d12/d19/d1a/f29 [406973,2226] 0 2024-02-15T18:27:02.894 INFO:tasks.workunit.client.1.smithi191.stdout:1/379: creat d0/d8/df/d12/d2a/d2b/f7a x:0 0 0 2024-02-15T18:27:02.894 INFO:tasks.workunit.client.1.smithi191.stdout:1/380: chown d0/d8/f17 25257 1 2024-02-15T18:27:02.895 INFO:tasks.workunit.client.1.smithi191.stdout:1/381: write d0/d8/df/d12/d2a/d2b/f5a [616987,64607] 0 2024-02-15T18:27:02.896 INFO:tasks.workunit.client.1.smithi191.stdout:1/382: mkdir d0/d2f/d51/d54/d7b 0 2024-02-15T18:27:02.897 INFO:tasks.workunit.client.1.smithi191.stdout:1/383: mknod d0/d8/df/d12/d2a/d2b/d78/c7c 0 2024-02-15T18:27:02.897 INFO:tasks.workunit.client.1.smithi191.stdout:1/384: truncate d0/d8/df/d12/d2a/f42 4436433 0 2024-02-15T18:27:02.897 INFO:tasks.workunit.client.1.smithi191.stdout:1/385: write d0/d8/df/d12/d33/d49/f67 [1077206,57745] 0 2024-02-15T18:27:02.900 INFO:tasks.workunit.client.1.smithi191.stdout:1/386: mknod d0/d2f/d66/c7d 0 2024-02-15T18:27:02.910 INFO:tasks.workunit.client.0.smithi042.stdout:8/601: dwrite dd/de/d12/fa2 [0,4194304] 0 2024-02-15T18:27:02.910 INFO:tasks.workunit.client.1.smithi191.stdout:9/308: dwrite d0/f51 [0,4194304] 0 2024-02-15T18:27:02.910 INFO:tasks.workunit.client.0.smithi042.stdout:1/675: dwrite de/d15/d2c/d51/d70/fba [4194304,4194304] 0 2024-02-15T18:27:02.910 INFO:tasks.workunit.client.0.smithi042.stdout:3/659: dread d1/de/d1a/f1e [4194304,4194304] 0 2024-02-15T18:27:02.911 INFO:tasks.workunit.client.0.smithi042.stdout:8/602: write dd/de/d33/d66/d4e/f7c [1865453,123685] 0 2024-02-15T18:27:02.912 INFO:tasks.workunit.client.0.smithi042.stdout:8/603: fsync dd/de/da4/d67/d5a/f64 0 2024-02-15T18:27:02.915 INFO:tasks.workunit.client.1.smithi191.stdout:1/387: dread d0/d8/df/f2c [0,4194304] 0 2024-02-15T18:27:02.917 INFO:tasks.workunit.client.1.smithi191.stdout:8/383: dwrite d3/d2c/d1a/d23/d2e/d64/f76 [0,4194304] 0 2024-02-15T18:27:02.917 INFO:tasks.workunit.client.1.smithi191.stdout:9/309: link d0/d7/dc/d32/l6c d0/d7/dc/d17/l74 0 2024-02-15T18:27:02.917 INFO:tasks.workunit.client.1.smithi191.stdout:9/310: readlink d0/d7/dc/d17/d33/l73 0 2024-02-15T18:27:02.918 INFO:tasks.workunit.client.0.smithi042.stdout:9/683: dwrite d6/da/d2f/d55/f6d [0,4194304] 0 2024-02-15T18:27:02.918 INFO:tasks.workunit.client.0.smithi042.stdout:9/684: write f2 [946672,16701] 0 2024-02-15T18:27:02.918 INFO:tasks.workunit.client.0.smithi042.stdout:9/685: chown d6/da/d1d/d2d/f83 4 1 2024-02-15T18:27:02.918 INFO:tasks.workunit.client.0.smithi042.stdout:9/686: truncate d6/da/d1d/d2d/d40/d95/fdd 1228494 0 2024-02-15T18:27:02.918 INFO:tasks.workunit.client.0.smithi042.stdout:2/615: dwrite d0/d40/d56/fa3 [0,4194304] 0 2024-02-15T18:27:02.918 INFO:tasks.workunit.client.0.smithi042.stdout:4/671: dwrite da/f25 [0,4194304] 0 2024-02-15T18:27:02.918 INFO:tasks.workunit.client.0.smithi042.stdout:2/616: dread - d0/d8/d10/d5a/f90 zero size 2024-02-15T18:27:02.918 INFO:tasks.workunit.client.0.smithi042.stdout:2/617: chown d0/d8/d20/d7d/d7e 469 1 2024-02-15T18:27:02.918 INFO:tasks.workunit.client.0.smithi042.stdout:4/672: write da/d11/d29/fe8 [471926,32025] 0 2024-02-15T18:27:02.918 INFO:tasks.workunit.client.0.smithi042.stdout:6/747: dwrite d9/d3e/fa3 [4194304,4194304] 0 2024-02-15T18:27:02.919 INFO:tasks.workunit.client.0.smithi042.stdout:1/676: dread de/d15/f31 [0,4194304] 0 2024-02-15T18:27:02.919 INFO:tasks.workunit.client.0.smithi042.stdout:2/618: read d0/d40/d56/f73 [434962,28166] 0 2024-02-15T18:27:02.919 INFO:tasks.workunit.client.0.smithi042.stdout:4/673: write da/f1a [10427933,32289] 0 2024-02-15T18:27:02.919 INFO:tasks.workunit.client.1.smithi191.stdout:9/311: read d0/d7/da/f1c [180334,49039] 0 2024-02-15T18:27:02.921 INFO:tasks.workunit.client.1.smithi191.stdout:1/388: dread d0/d2f/d51/d10/f2d [0,4194304] 0 2024-02-15T18:27:02.924 INFO:tasks.workunit.client.0.smithi042.stdout:4/674: dread da/f7c [0,4194304] 0 2024-02-15T18:27:02.931 INFO:tasks.workunit.client.1.smithi191.stdout:8/384: symlink d3/d2c/d1a/d23/d2e/d5b/l8b 0 2024-02-15T18:27:02.932 INFO:tasks.workunit.client.0.smithi042.stdout:3/660: creat d1/d11/d2e/fd6 x:0 0 0 2024-02-15T18:27:02.932 INFO:tasks.workunit.client.0.smithi042.stdout:3/661: write d1/de/f42 [3546210,69966] 0 2024-02-15T18:27:02.933 INFO:tasks.workunit.client.0.smithi042.stdout:8/604: rmdir dd/de/da4 39 2024-02-15T18:27:02.934 INFO:tasks.workunit.client.1.smithi191.stdout:8/385: mknod d3/d2c/d1a/d23/d2e/d5b/d69/c8c 0 2024-02-15T18:27:02.934 INFO:tasks.workunit.client.1.smithi191.stdout:8/386: stat d3/d2c/d1a/d23/d2e/l32 0 2024-02-15T18:27:02.934 INFO:tasks.workunit.client.1.smithi191.stdout:8/387: chown d3/l2b 48 1 2024-02-15T18:27:02.936 INFO:tasks.workunit.client.0.smithi042.stdout:9/687: rename d6/da/d2f/d54/d96 to d6/da/d1d/dde/dec 0 2024-02-15T18:27:02.937 INFO:tasks.workunit.client.0.smithi042.stdout:9/688: fsync d6/da/d2f/d6c/f94 0 2024-02-15T18:27:02.938 INFO:tasks.workunit.client.1.smithi191.stdout:8/388: dread d3/f29 [0,4194304] 0 2024-02-15T18:27:02.938 INFO:tasks.workunit.client.1.smithi191.stdout:8/389: chown d3/fa 19026 1 2024-02-15T18:27:02.941 INFO:tasks.workunit.client.1.smithi191.stdout:1/389: dwrite d0/d8/df/d2e/f4f [0,4194304] 0 2024-02-15T18:27:02.941 INFO:tasks.workunit.client.1.smithi191.stdout:9/312: dwrite d0/d7/dc/f15 [0,4194304] 0 2024-02-15T18:27:02.941 INFO:tasks.workunit.client.0.smithi042.stdout:6/748: rmdir d9/d11/d46/df8 39 2024-02-15T18:27:02.942 INFO:tasks.workunit.client.0.smithi042.stdout:6/749: chown d9/d11/d19/d1c/d31/d4f/ce7 7 1 2024-02-15T18:27:02.944 INFO:tasks.workunit.client.0.smithi042.stdout:1/677: link de/d54/dbf/dd0/d7c/dd6/ddf/d88/lae de/d54/lf8 0 2024-02-15T18:27:02.944 INFO:tasks.workunit.client.0.smithi042.stdout:1/678: chown de/d54/de6 43627737 1 2024-02-15T18:27:02.946 INFO:tasks.workunit.client.0.smithi042.stdout:2/619: mknod d0/d35/d9e/cbf 0 2024-02-15T18:27:02.950 INFO:tasks.workunit.client.0.smithi042.stdout:4/675: creat da/d11/d29/d56/fe9 x:0 0 0 2024-02-15T18:27:02.954 INFO:tasks.workunit.client.1.smithi191.stdout:1/390: truncate d0/f1b 2207012 0 2024-02-15T18:27:02.958 INFO:tasks.workunit.client.0.smithi042.stdout:5/697: sync 2024-02-15T18:27:02.958 INFO:tasks.workunit.client.0.smithi042.stdout:5/698: chown d0/d22/fee 293865 1 2024-02-15T18:27:02.959 INFO:tasks.workunit.client.1.smithi191.stdout:9/313: link d0/d7/dc/d30/d4f/f6d d0/d7/da/d39/f75 0 2024-02-15T18:27:02.959 INFO:tasks.workunit.client.1.smithi191.stdout:9/314: truncate d0/d7/dc/d17/d33/f69 1476894 0 2024-02-15T18:27:02.959 INFO:tasks.workunit.client.1.smithi191.stdout:8/390: dwrite d3/d5e/f73 [0,4194304] 0 2024-02-15T18:27:02.960 INFO:tasks.workunit.client.0.smithi042.stdout:3/662: rmdir d1/de/d1a 39 2024-02-15T18:27:02.961 INFO:tasks.workunit.client.1.smithi191.stdout:9/315: rename d0/f63 to d0/d7/dc/d30/f76 0 2024-02-15T18:27:02.962 INFO:tasks.workunit.client.0.smithi042.stdout:8/605: truncate dd/f29 2831236 0 2024-02-15T18:27:02.964 INFO:tasks.workunit.client.0.smithi042.stdout:9/689: creat d6/da/d75/fed x:0 0 0 2024-02-15T18:27:02.969 INFO:tasks.workunit.client.0.smithi042.stdout:6/750: mknod d9/d3e/dd9/c107 0 2024-02-15T18:27:02.969 INFO:tasks.workunit.client.0.smithi042.stdout:6/751: write d9/d3e/dd9/ffa [51871,94560] 0 2024-02-15T18:27:02.969 INFO:tasks.workunit.client.0.smithi042.stdout:1/679: creat de/d54/dbf/dd0/ff9 x:0 0 0 2024-02-15T18:27:02.971 INFO:tasks.workunit.client.0.smithi042.stdout:5/699: rename d0/d12/d2a/ca6 to d0/d39/ddd/cf5 0 2024-02-15T18:27:02.971 INFO:tasks.workunit.client.0.smithi042.stdout:5/700: dread - d0/dda/dc7/fcc zero size 2024-02-15T18:27:02.971 INFO:tasks.workunit.client.0.smithi042.stdout:2/620: dwrite d0/f8c [0,4194304] 0 2024-02-15T18:27:02.972 INFO:tasks.workunit.client.0.smithi042.stdout:3/663: truncate d1/d11/d14/f53 1211289 0 2024-02-15T18:27:02.972 INFO:tasks.workunit.client.0.smithi042.stdout:4/676: dwrite da/d11/d29/d4a/fbe [0,4194304] 0 2024-02-15T18:27:02.972 INFO:tasks.workunit.client.0.smithi042.stdout:3/664: read d1/d11/f66 [2901470,98072] 0 2024-02-15T18:27:02.972 INFO:tasks.workunit.client.0.smithi042.stdout:6/752: dread d9/d11/d19/d1c/d31/d53/fc2 [0,4194304] 0 2024-02-15T18:27:02.974 INFO:tasks.workunit.client.0.smithi042.stdout:1/680: creat de/d6d/ffa x:0 0 0 2024-02-15T18:27:02.974 INFO:tasks.workunit.client.0.smithi042.stdout:5/701: symlink d0/d38/d53/d64/lf6 0 2024-02-15T18:27:02.976 INFO:tasks.workunit.client.0.smithi042.stdout:2/621: creat d0/d8/d10/d31/fc0 x:0 0 0 2024-02-15T18:27:02.983 INFO:tasks.workunit.client.0.smithi042.stdout:0/639: sync 2024-02-15T18:27:02.983 INFO:tasks.workunit.client.0.smithi042.stdout:7/709: sync 2024-02-15T18:27:02.983 INFO:tasks.workunit.client.0.smithi042.stdout:7/710: stat d10/d15/d8f/f9f 0 2024-02-15T18:27:02.983 INFO:tasks.workunit.client.0.smithi042.stdout:0/640: truncate d4/d9/d34/f4b 8402309 0 2024-02-15T18:27:02.983 INFO:tasks.workunit.client.0.smithi042.stdout:0/641: write d4/d9/d5a/fcc [4598776,121203] 0 2024-02-15T18:27:02.983 INFO:tasks.workunit.client.0.smithi042.stdout:0/642: write d4/d38/fdd [641572,100156] 0 2024-02-15T18:27:02.990 INFO:tasks.workunit.client.0.smithi042.stdout:8/606: unlink dd/d11/c20 0 2024-02-15T18:27:02.990 INFO:tasks.workunit.client.0.smithi042.stdout:6/753: dwrite d9/d11/d19/d1c/d31/d4f/f98 [0,4194304] 0 2024-02-15T18:27:02.992 INFO:tasks.workunit.client.0.smithi042.stdout:4/677: creat da/d11/d29/d4a/d4f/fea x:0 0 0 2024-02-15T18:27:02.992 INFO:tasks.workunit.client.0.smithi042.stdout:7/711: dread d10/d15/d39/dd9/fdf [0,4194304] 0 2024-02-15T18:27:02.995 INFO:tasks.workunit.client.0.smithi042.stdout:3/665: rename d1/de/f10 to d1/d11/d2e/fd7 0 2024-02-15T18:27:02.999 INFO:tasks.workunit.client.0.smithi042.stdout:5/702: getdents d0/dda/dc7/d52 0 2024-02-15T18:27:03.001 INFO:tasks.workunit.client.0.smithi042.stdout:2/622: symlink d0/d8/db/d44/lc1 0 2024-02-15T18:27:03.001 INFO:tasks.workunit.client.0.smithi042.stdout:2/623: chown d0/d8/db/de/d1a/d2e/c68 647493 1 2024-02-15T18:27:03.002 INFO:tasks.workunit.client.0.smithi042.stdout:0/643: dwrite d4/d9/f31 [0,4194304] 0 2024-02-15T18:27:03.003 INFO:tasks.workunit.client.0.smithi042.stdout:6/754: mknod d9/d11/d46/d89/dbc/c108 0 2024-02-15T18:27:03.006 INFO:tasks.workunit.client.0.smithi042.stdout:5/703: write d0/d14/f63 [3361334,18279] 0 2024-02-15T18:27:03.006 INFO:tasks.workunit.client.0.smithi042.stdout:5/704: dread - d0/d14/d88/f90 zero size 2024-02-15T18:27:03.011 INFO:tasks.workunit.client.0.smithi042.stdout:4/678: rename da/d2b/cb6 to da/d11/d29/d4a/d4f/ceb 0 2024-02-15T18:27:03.011 INFO:tasks.workunit.client.0.smithi042.stdout:4/679: getdents da/d2b/de3 0 2024-02-15T18:27:03.011 INFO:tasks.workunit.client.0.smithi042.stdout:1/681: symlink de/d54/dbf/dd0/lfb 0 2024-02-15T18:27:03.012 INFO:tasks.workunit.client.0.smithi042.stdout:3/666: dwrite d1/d11/f45 [0,4194304] 0 2024-02-15T18:27:03.013 INFO:tasks.workunit.client.0.smithi042.stdout:2/624: truncate d0/d8/db/de/d1a/f6b 4104744 0 2024-02-15T18:27:03.016 INFO:tasks.workunit.client.0.smithi042.stdout:4/680: symlink da/d11/d27/lec 0 2024-02-15T18:27:03.017 INFO:tasks.workunit.client.0.smithi042.stdout:6/755: dread d9/f14 [0,4194304] 0 2024-02-15T18:27:03.018 INFO:tasks.workunit.client.0.smithi042.stdout:1/682: rename de/c12 to de/d54/dbf/dd0/d7c/dd6/ddf/dca/cfc 0 2024-02-15T18:27:03.019 INFO:tasks.workunit.client.0.smithi042.stdout:3/667: write d1/d11/d14/d4d/d73/f87 [1242944,8948] 0 2024-02-15T18:27:03.020 INFO:tasks.workunit.client.0.smithi042.stdout:3/668: fdatasync d1/d11/d14/d4d/f7e 0 2024-02-15T18:27:03.020 INFO:tasks.workunit.client.0.smithi042.stdout:6/756: mknod d9/d11/d19/d1c/d31/d53/c109 0 2024-02-15T18:27:03.020 INFO:tasks.workunit.client.0.smithi042.stdout:1/683: link de/d15/d2c/d51/d70/fe9 de/d54/dbf/dd0/d7c/dd6/ddf/dca/ffd 0 2024-02-15T18:27:03.020 INFO:tasks.workunit.client.0.smithi042.stdout:8/607: dread dd/d11/f39 [0,4194304] 0 2024-02-15T18:27:03.021 INFO:tasks.workunit.client.0.smithi042.stdout:3/669: creat d1/d11/d14/fd8 x:0 0 0 2024-02-15T18:27:03.022 INFO:tasks.workunit.client.0.smithi042.stdout:6/757: rmdir d9/d11/d1d 39 2024-02-15T18:27:03.022 INFO:tasks.workunit.client.0.smithi042.stdout:0/644: dwrite d4/f41 [0,4194304] 0 2024-02-15T18:27:03.022 INFO:tasks.workunit.client.0.smithi042.stdout:0/645: chown d4/d38/d39/d8e/d8f/db2 2250 1 2024-02-15T18:27:03.022 INFO:tasks.workunit.client.0.smithi042.stdout:1/684: truncate de/d15/d67/d46/fc0 817186 0 2024-02-15T18:27:03.023 INFO:tasks.workunit.client.0.smithi042.stdout:5/705: dwrite d0/d12/d2a/fe8 [0,4194304] 0 2024-02-15T18:27:03.023 INFO:tasks.workunit.client.0.smithi042.stdout:5/706: fsync d0/d38/d53/d64/de5/fc9 0 2024-02-15T18:27:03.024 INFO:tasks.workunit.client.0.smithi042.stdout:8/608: mkdir dd/de/da4/d67/daa 0 2024-02-15T18:27:03.024 INFO:tasks.workunit.client.0.smithi042.stdout:3/670: mkdir d1/d11/d14/d4d/d73/d89/dd9 0 2024-02-15T18:27:03.025 INFO:tasks.workunit.client.0.smithi042.stdout:6/758: mkdir d9/d11/d19/d35/d49/d10a 0 2024-02-15T18:27:03.025 INFO:tasks.workunit.client.0.smithi042.stdout:6/759: chown d9/d11/d19/d35/d49/f52 2 1 2024-02-15T18:27:03.025 INFO:tasks.workunit.client.0.smithi042.stdout:6/760: chown d9/f14 2985924 1 2024-02-15T18:27:03.026 INFO:tasks.workunit.client.0.smithi042.stdout:6/761: truncate d9/d11/d19/d1c/fdb 512255 0 2024-02-15T18:27:03.028 INFO:tasks.workunit.client.0.smithi042.stdout:4/681: dread da/d2b/f37 [0,4194304] 0 2024-02-15T18:27:03.033 INFO:tasks.workunit.client.0.smithi042.stdout:1/685: dread de/d54/fb0 [0,4194304] 0 2024-02-15T18:27:03.058 INFO:tasks.workunit.client.1.smithi191.stdout:4/357: sync 2024-02-15T18:27:03.058 INFO:tasks.workunit.client.1.smithi191.stdout:7/297: sync 2024-02-15T18:27:03.058 INFO:tasks.workunit.client.1.smithi191.stdout:6/247: sync 2024-02-15T18:27:03.058 INFO:tasks.workunit.client.1.smithi191.stdout:4/358: chown d2/d5/d7/l27 92841717 1 2024-02-15T18:27:03.058 INFO:tasks.workunit.client.1.smithi191.stdout:7/298: truncate d0/db/d65/d11/d1c/f45 946 0 2024-02-15T18:27:03.058 INFO:tasks.workunit.client.1.smithi191.stdout:4/359: truncate d2/d5/d14/d47/d78/f68 384965 0 2024-02-15T18:27:03.063 INFO:tasks.workunit.client.0.smithi042.stdout:0/646: creat d4/d9/d34/dab/fde x:0 0 0 2024-02-15T18:27:03.064 INFO:tasks.workunit.client.1.smithi191.stdout:6/248: truncate d8/f30 221339 0 2024-02-15T18:27:03.067 INFO:tasks.workunit.client.0.smithi042.stdout:4/682: write da/d3a/dbb/fc3 [352795,5364] 0 2024-02-15T18:27:03.069 INFO:tasks.workunit.client.1.smithi191.stdout:6/249: dread d8/d19/f37 [0,4194304] 0 2024-02-15T18:27:03.069 INFO:tasks.workunit.client.1.smithi191.stdout:6/250: chown d8/d9/d2b/l2d 5861337 1 2024-02-15T18:27:03.072 INFO:tasks.workunit.client.1.smithi191.stdout:7/299: creat d0/db/d26/d5f/f68 x:0 0 0 2024-02-15T18:27:03.073 INFO:tasks.workunit.client.0.smithi042.stdout:6/762: dwrite d9/d11/d19/d35/d55/d92/d9e/faa [4194304,4194304] 0 2024-02-15T18:27:03.075 INFO:tasks.workunit.client.0.smithi042.stdout:5/707: mkdir d0/d50/df7 0 2024-02-15T18:27:03.076 INFO:tasks.workunit.client.0.smithi042.stdout:3/671: creat d1/d11/d14/d4d/fda x:0 0 0 2024-02-15T18:27:03.076 INFO:tasks.workunit.client.0.smithi042.stdout:3/672: chown d1/de/d1a/f9e 107541804 1 2024-02-15T18:27:03.076 INFO:tasks.workunit.client.0.smithi042.stdout:3/673: write d1/d11/d2e/d4c/f5b [402251,53592] 0 2024-02-15T18:27:03.079 INFO:tasks.workunit.client.0.smithi042.stdout:1/686: mknod de/d15/d95/cfe 0 2024-02-15T18:27:03.080 INFO:tasks.workunit.client.1.smithi191.stdout:4/360: mkdir d2/d5/d8b 0 2024-02-15T18:27:03.080 INFO:tasks.workunit.client.1.smithi191.stdout:4/361: dread - d2/d5/d7/f80 zero size 2024-02-15T18:27:03.080 INFO:tasks.workunit.client.1.smithi191.stdout:4/362: fsync d2/d5/f4d 0 2024-02-15T18:27:03.082 INFO:tasks.workunit.client.1.smithi191.stdout:7/300: stat d0/db/d65/d11/d1c/l58 0 2024-02-15T18:27:03.084 INFO:tasks.workunit.client.0.smithi042.stdout:6/763: creat d9/d11/d19/d35/d55/f10b x:0 0 0 2024-02-15T18:27:03.085 INFO:tasks.workunit.client.1.smithi191.stdout:4/363: mkdir d2/d5/d7/d3e/d63/d8c 0 2024-02-15T18:27:03.085 INFO:tasks.workunit.client.1.smithi191.stdout:4/364: dread - d2/d5/d7/f80 zero size 2024-02-15T18:27:03.088 INFO:tasks.workunit.client.0.smithi042.stdout:0/647: dwrite d4/d9/d34/dab/dc8/f87 [4194304,4194304] 0 2024-02-15T18:27:03.088 INFO:tasks.workunit.client.1.smithi191.stdout:6/251: rename d8/d9/d2b/f32 to d8/d19/f40 0 2024-02-15T18:27:03.090 INFO:tasks.workunit.client.0.smithi042.stdout:8/609: link dd/d11/c1f dd/de/d12/cab 0 2024-02-15T18:27:03.090 INFO:tasks.workunit.client.0.smithi042.stdout:8/610: write dd/de/da4/d67/d5a/f8f [697957,107794] 0 2024-02-15T18:27:03.091 INFO:tasks.workunit.client.0.smithi042.stdout:4/683: dwrite da/d11/d67/d5c/d66/d78/d2f/d4d/f48 [0,4194304] 0 2024-02-15T18:27:03.091 INFO:tasks.workunit.client.0.smithi042.stdout:6/764: dread d9/d11/d19/d35/d49/f4c [4194304,4194304] 0 2024-02-15T18:27:03.091 INFO:tasks.workunit.client.1.smithi191.stdout:4/365: rename d2/d5/f1a to d2/d5/d8b/f8d 0 2024-02-15T18:27:03.094 INFO:tasks.workunit.client.1.smithi191.stdout:6/252: symlink d8/l41 0 2024-02-15T18:27:03.096 INFO:tasks.workunit.client.0.smithi042.stdout:5/708: getdents d0/dda/dc7/de1/dd8 0 2024-02-15T18:27:03.096 INFO:tasks.workunit.client.0.smithi042.stdout:5/709: dread - d0/d14/d88/f90 zero size 2024-02-15T18:27:03.098 INFO:tasks.workunit.client.0.smithi042.stdout:9/690: sync 2024-02-15T18:27:03.098 INFO:tasks.workunit.client.0.smithi042.stdout:3/674: symlink d1/d11/d2e/dd1/ldb 0 2024-02-15T18:27:03.100 INFO:tasks.workunit.client.1.smithi191.stdout:7/301: getdents d0/db/d26/d5f 0 2024-02-15T18:27:03.101 INFO:tasks.workunit.client.0.smithi042.stdout:1/687: creat de/d54/dbf/dd0/d7c/dd6/ddf/d88/fff x:0 0 0 2024-02-15T18:27:03.105 INFO:tasks.workunit.client.0.smithi042.stdout:0/648: dwrite d4/f3b [0,4194304] 0 2024-02-15T18:27:03.105 INFO:tasks.workunit.client.0.smithi042.stdout:3/675: fsync d1/d11/d2e/f54 0 2024-02-15T18:27:03.111 INFO:tasks.workunit.client.1.smithi191.stdout:4/366: creat d2/d5/d14/d47/f8e x:0 0 0 2024-02-15T18:27:03.112 INFO:tasks.workunit.client.1.smithi191.stdout:6/253: getdents d8/d19 0 2024-02-15T18:27:03.112 INFO:tasks.workunit.client.0.smithi042.stdout:8/611: rmdir dd/de/d12 39 2024-02-15T18:27:03.113 INFO:tasks.workunit.client.0.smithi042.stdout:6/765: symlink d9/d11/d19/d1c/d31/d53/dbe/l10c 0 2024-02-15T18:27:03.114 INFO:tasks.workunit.client.1.smithi191.stdout:7/302: dread d0/d1/d35/f48 [0,4194304] 0 2024-02-15T18:27:03.114 INFO:tasks.workunit.client.1.smithi191.stdout:6/254: creat d8/d9/d2b/f42 x:0 0 0 2024-02-15T18:27:03.114 INFO:tasks.workunit.client.1.smithi191.stdout:6/255: chown d8/d19/c24 75728 1 2024-02-15T18:27:03.114 INFO:tasks.workunit.client.1.smithi191.stdout:6/256: stat d8/d9/d2b 0 2024-02-15T18:27:03.115 INFO:tasks.workunit.client.0.smithi042.stdout:5/710: truncate d0/fe 3914139 0 2024-02-15T18:27:03.116 INFO:tasks.workunit.client.0.smithi042.stdout:1/688: mkdir de/d54/dbf/dd0/d7c/dd6/de8/dec/d100 0 2024-02-15T18:27:03.116 INFO:tasks.workunit.client.1.smithi191.stdout:6/257: write d8/f33 [73531,74095] 0 2024-02-15T18:27:03.119 INFO:tasks.workunit.client.1.smithi191.stdout:6/258: rename d8/d9/l20 to d8/d9/d2b/l43 0 2024-02-15T18:27:03.119 INFO:tasks.workunit.client.1.smithi191.stdout:6/259: chown d8/d9/d2b/l43 331129448 1 2024-02-15T18:27:03.120 INFO:tasks.workunit.client.1.smithi191.stdout:6/260: read f1 [7042586,82606] 0 2024-02-15T18:27:03.120 INFO:tasks.workunit.client.1.smithi191.stdout:6/261: chown d8/d9/c3b 13973 1 2024-02-15T18:27:03.120 INFO:tasks.workunit.client.0.smithi042.stdout:4/684: dwrite da/d11/fe5 [0,4194304] 0 2024-02-15T18:27:03.121 INFO:tasks.workunit.client.1.smithi191.stdout:6/262: dread d8/d19/f26 [0,4194304] 0 2024-02-15T18:27:03.122 INFO:tasks.workunit.client.0.smithi042.stdout:0/649: rename d4/d38/d39/d8e/d8f/db2 to d4/d38/d77/d2e/dae/ddf 0 2024-02-15T18:27:03.123 INFO:tasks.workunit.client.1.smithi191.stdout:4/367: dread d2/d5/d7/f25 [0,4194304] 0 2024-02-15T18:27:03.123 INFO:tasks.workunit.client.0.smithi042.stdout:8/612: mknod dd/d11/d75/cac 0 2024-02-15T18:27:03.124 INFO:tasks.workunit.client.1.smithi191.stdout:6/263: creat d8/d9/f44 x:0 0 0 2024-02-15T18:27:03.126 INFO:tasks.workunit.client.0.smithi042.stdout:6/766: creat d9/d11/d46/d89/f10d x:0 0 0 2024-02-15T18:27:03.126 INFO:tasks.workunit.client.0.smithi042.stdout:7/712: sync 2024-02-15T18:27:03.129 INFO:tasks.workunit.client.0.smithi042.stdout:5/711: mkdir d0/d14/d88/df8 0 2024-02-15T18:27:03.129 INFO:tasks.workunit.client.0.smithi042.stdout:5/712: fsync d0/d14/d88/f90 0 2024-02-15T18:27:03.132 INFO:tasks.workunit.client.0.smithi042.stdout:9/691: dwrite d6/da/d1d/d2d/d40/d48/fba [0,4194304] 0 2024-02-15T18:27:03.132 INFO:tasks.workunit.client.0.smithi042.stdout:2/625: sync 2024-02-15T18:27:03.133 INFO:tasks.workunit.client.1.smithi191.stdout:6/264: creat d8/f45 x:0 0 0 2024-02-15T18:27:03.134 INFO:tasks.workunit.client.0.smithi042.stdout:1/689: creat de/d6d/f101 x:0 0 0 2024-02-15T18:27:03.137 INFO:tasks.workunit.client.0.smithi042.stdout:4/685: creat da/d11/d85/db5/fed x:0 0 0 2024-02-15T18:27:03.137 INFO:tasks.workunit.client.0.smithi042.stdout:4/686: fdatasync da/d11/d67/d5c/d66/d78/d2f/d4d/f47 0 2024-02-15T18:27:03.137 INFO:tasks.workunit.client.1.smithi191.stdout:7/303: dwrite d0/d1/df/f1a [4194304,4194304] 0 2024-02-15T18:27:03.138 INFO:tasks.workunit.client.0.smithi042.stdout:3/676: rename d1/f9 to d1/de/d19/fdc 0 2024-02-15T18:27:03.151 INFO:tasks.workunit.client.0.smithi042.stdout:8/613: dwrite dd/de/d33/f40 [0,4194304] 0 2024-02-15T18:27:03.154 INFO:tasks.workunit.client.0.smithi042.stdout:1/690: dwrite de/d15/d2c/f38 [0,4194304] 0 2024-02-15T18:27:03.154 INFO:tasks.workunit.client.0.smithi042.stdout:1/691: write de/d15/d2c/fde [170244,120243] 0 2024-02-15T18:27:03.154 INFO:tasks.workunit.client.1.smithi191.stdout:4/368: dwrite d2/d5/d7/d2f/f75 [0,4194304] 0 2024-02-15T18:27:03.155 INFO:tasks.workunit.client.1.smithi191.stdout:4/369: write d2/d5/d7/f80 [823059,7288] 0 2024-02-15T18:27:03.206 INFO:tasks.workunit.client.0.smithi042.stdout:0/650: creat d4/d9/d34/d57/dc1/fe0 x:0 0 0 2024-02-15T18:27:03.206 INFO:tasks.workunit.client.0.smithi042.stdout:0/651: write d4/f6 [642377,69616] 0 2024-02-15T18:27:03.342 INFO:tasks.workunit.client.1.smithi191.stdout:6/265: getdents d8 0 2024-02-15T18:27:03.345 INFO:tasks.workunit.client.1.smithi191.stdout:6/266: dread d8/d19/f38 [0,4194304] 0 2024-02-15T18:27:03.345 INFO:tasks.workunit.client.0.smithi042.stdout:6/767: getdents d9/d11/d46/d89 0 2024-02-15T18:27:03.346 INFO:tasks.workunit.client.0.smithi042.stdout:6/768: fdatasync d9/d11/d19/d1c/d31/f56 0 2024-02-15T18:27:03.347 INFO:tasks.workunit.client.1.smithi191.stdout:7/304: rename d0/db/d65/dc/d4b/f56 to d0/d1/df/d5d/f69 0 2024-02-15T18:27:03.347 INFO:tasks.workunit.client.0.smithi042.stdout:6/769: dread d9/d11/d97/fb8 [0,4194304] 0 2024-02-15T18:27:03.347 INFO:tasks.workunit.client.0.smithi042.stdout:6/770: getdents d9/d11/d97/df7 0 2024-02-15T18:27:03.368 INFO:tasks.workunit.client.1.smithi191.stdout:6/267: dwrite d8/f33 [0,4194304] 0 2024-02-15T18:27:03.375 INFO:tasks.workunit.client.1.smithi191.stdout:7/305: dwrite d0/db/f2d [4194304,4194304] 0 2024-02-15T18:27:03.376 INFO:tasks.workunit.client.0.smithi042.stdout:7/713: symlink d10/d12/d65/da7/lea 0 2024-02-15T18:27:03.382 INFO:tasks.workunit.client.1.smithi191.stdout:6/268: dwrite f6 [12582912,4194304] 0 2024-02-15T18:27:03.389 INFO:tasks.workunit.client.1.smithi191.stdout:4/370: creat d2/d5/d7/d2f/d41/f8f x:0 0 0 2024-02-15T18:27:03.391 INFO:tasks.workunit.client.0.smithi042.stdout:9/692: creat d6/d9d/fee x:0 0 0 2024-02-15T18:27:03.392 INFO:tasks.workunit.client.0.smithi042.stdout:2/626: mknod d0/d8/d20/cc2 0 2024-02-15T18:27:03.392 INFO:tasks.workunit.client.0.smithi042.stdout:9/693: dread d6/da/d2f/d55/d73/d8d/faf [0,4194304] 0 2024-02-15T18:27:03.393 INFO:tasks.workunit.client.0.smithi042.stdout:4/687: symlink da/d2b/lee 0 2024-02-15T18:27:03.396 INFO:tasks.workunit.client.0.smithi042.stdout:5/713: rename d0/dda/dc7/d52/fcd to d0/d38/d53/d91/d94/ff9 0 2024-02-15T18:27:03.397 INFO:tasks.workunit.client.0.smithi042.stdout:3/677: creat d1/d11/d14/d4d/d73/fdd x:0 0 0 2024-02-15T18:27:03.397 INFO:tasks.workunit.client.0.smithi042.stdout:3/678: chown d1/d11/c72 0 1 2024-02-15T18:27:03.400 INFO:tasks.workunit.client.1.smithi191.stdout:7/306: creat d0/d1/f6a x:0 0 0 2024-02-15T18:27:03.400 INFO:tasks.workunit.client.1.smithi191.stdout:7/307: truncate d0/db/d65/d11/d5a/d66/f32 2026125 0 2024-02-15T18:27:03.402 INFO:tasks.workunit.client.1.smithi191.stdout:6/269: creat d8/d9/f46 x:0 0 0 2024-02-15T18:27:03.402 INFO:tasks.workunit.client.1.smithi191.stdout:6/270: fdatasync d8/d19/f38 0 2024-02-15T18:27:03.402 INFO:tasks.workunit.client.1.smithi191.stdout:6/271: write d8/d9/f44 [566397,90763] 0 2024-02-15T18:27:03.402 INFO:tasks.workunit.client.1.smithi191.stdout:7/308: write d0/d1/d35/f48 [1569885,24550] 0 2024-02-15T18:27:03.404 INFO:tasks.workunit.client.0.smithi042.stdout:8/614: unlink dd/de/da4/d67/f6e 0 2024-02-15T18:27:03.406 INFO:tasks.workunit.client.0.smithi042.stdout:8/615: write dd/de/d33/d66/f5f [124618,43205] 0 2024-02-15T18:27:03.407 INFO:tasks.workunit.client.0.smithi042.stdout:1/692: creat de/d54/dbf/dd0/f102 x:0 0 0 2024-02-15T18:27:03.408 INFO:tasks.workunit.client.1.smithi191.stdout:6/272: write d8/f33 [3204012,106688] 0 2024-02-15T18:27:03.410 INFO:tasks.workunit.client.1.smithi191.stdout:7/309: symlink d0/d1/df/d5d/l6b 0 2024-02-15T18:27:03.410 INFO:tasks.workunit.client.0.smithi042.stdout:6/771: mknod d9/d11/d19/d1c/d31/d53/c10e 0 2024-02-15T18:27:03.411 INFO:tasks.workunit.client.1.smithi191.stdout:4/371: dwrite d2/d5/d64/f76 [0,4194304] 0 2024-02-15T18:27:03.415 INFO:tasks.workunit.client.0.smithi042.stdout:2/627: stat d0/c9 0 2024-02-15T18:27:03.415 INFO:tasks.workunit.client.0.smithi042.stdout:2/628: readlink d0/d8/db/de/d1a/l1f 0 2024-02-15T18:27:03.415 INFO:tasks.workunit.client.0.smithi042.stdout:2/629: fdatasync d0/d8/d10/d31/da6/fac 0 2024-02-15T18:27:03.417 INFO:tasks.workunit.client.0.smithi042.stdout:8/616: dread dd/de/da4/d67/d5a/f8f [0,4194304] 0 2024-02-15T18:27:03.417 INFO:tasks.workunit.client.1.smithi191.stdout:7/310: creat d0/d1/d35/d42/f6c x:0 0 0 2024-02-15T18:27:03.418 INFO:tasks.workunit.client.0.smithi042.stdout:5/714: dwrite d0/d89/f97 [0,4194304] 0 2024-02-15T18:27:03.420 INFO:tasks.workunit.client.1.smithi191.stdout:7/311: read d0/d1/df/f1a [527808,93520] 0 2024-02-15T18:27:03.420 INFO:tasks.workunit.client.1.smithi191.stdout:7/312: write d0/db/d65/d11/f18 [1286087,69105] 0 2024-02-15T18:27:03.420 INFO:tasks.workunit.client.1.smithi191.stdout:7/313: chown d0/db/f2f 57 1 2024-02-15T18:27:03.420 INFO:tasks.workunit.client.1.smithi191.stdout:7/314: rename d0 to d0/db/d65/dc/d6d 22 2024-02-15T18:27:03.421 INFO:tasks.workunit.client.0.smithi042.stdout:9/694: unlink d6/da/d75/ccb 0 2024-02-15T18:27:03.426 INFO:tasks.workunit.client.0.smithi042.stdout:4/688: creat da/d11/d29/d4a/d4f/fef x:0 0 0 2024-02-15T18:27:03.426 INFO:tasks.workunit.client.1.smithi191.stdout:4/372: dwrite d2/d5/d7/d2f/d41/f73 [0,4194304] 0 2024-02-15T18:27:03.426 INFO:tasks.workunit.client.0.smithi042.stdout:4/689: write da/d11/d27/f86 [1227921,118988] 0 2024-02-15T18:27:03.427 INFO:tasks.workunit.client.1.smithi191.stdout:3/239: sync 2024-02-15T18:27:03.427 INFO:tasks.workunit.client.1.smithi191.stdout:0/172: sync 2024-02-15T18:27:03.427 INFO:tasks.workunit.client.1.smithi191.stdout:2/267: sync 2024-02-15T18:27:03.427 INFO:tasks.workunit.client.1.smithi191.stdout:5/169: sync 2024-02-15T18:27:03.427 INFO:tasks.workunit.client.1.smithi191.stdout:5/170: readlink d0/l10 0 2024-02-15T18:27:03.427 INFO:tasks.workunit.client.1.smithi191.stdout:8/391: sync 2024-02-15T18:27:03.427 INFO:tasks.workunit.client.1.smithi191.stdout:1/391: sync 2024-02-15T18:27:03.428 INFO:tasks.workunit.client.1.smithi191.stdout:9/316: sync 2024-02-15T18:27:03.428 INFO:tasks.workunit.client.1.smithi191.stdout:9/317: fsync d0/d7/dc/d30/d4f/f6d 0 2024-02-15T18:27:03.428 INFO:tasks.workunit.client.1.smithi191.stdout:9/318: chown d0/d7/dc/d30/d3b/d56 20124389 1 2024-02-15T18:27:03.428 INFO:tasks.workunit.client.1.smithi191.stdout:9/319: write d0/d7/dc/d17/d33/f4d [9236433,25121] 0 2024-02-15T18:27:03.428 INFO:tasks.workunit.client.1.smithi191.stdout:4/373: write d2/d5/d7/d3e/f77 [2301093,15943] 0 2024-02-15T18:27:03.428 INFO:tasks.workunit.client.1.smithi191.stdout:1/392: write d0/d8/df/d12/d2a/f42 [3200955,24654] 0 2024-02-15T18:27:03.428 INFO:tasks.workunit.client.1.smithi191.stdout:1/393: chown d0/d2f/d77 1393117 1 2024-02-15T18:27:03.430 INFO:tasks.workunit.client.1.smithi191.stdout:9/320: write d0/d7/da/f35 [416954,75436] 0 2024-02-15T18:27:03.431 INFO:tasks.workunit.client.0.smithi042.stdout:7/714: rename d10/d49/d6d/d6e/le4 to d10/d12/d23/d51/leb 0 2024-02-15T18:27:03.433 INFO:tasks.workunit.client.0.smithi042.stdout:7/715: read d10/d49/fd6 [164899,88659] 0 2024-02-15T18:27:03.434 INFO:tasks.workunit.client.0.smithi042.stdout:7/716: fdatasync d10/d15/d8f/fb5 0 2024-02-15T18:27:03.434 INFO:tasks.workunit.client.1.smithi191.stdout:6/273: mknod d8/d9/c47 0 2024-02-15T18:27:03.434 INFO:tasks.workunit.client.1.smithi191.stdout:6/274: getdents d8/d19 0 2024-02-15T18:27:03.436 INFO:tasks.workunit.client.1.smithi191.stdout:4/374: dread d2/d5/d64/f76 [0,4194304] 0 2024-02-15T18:27:03.436 INFO:tasks.workunit.client.1.smithi191.stdout:6/275: write d8/d9/f29 [3147924,117074] 0 2024-02-15T18:27:03.532 INFO:tasks.workunit.client.1.smithi191.stdout:7/315: mkdir d0/db/d65/dc/d6e 0 2024-02-15T18:27:03.534 INFO:tasks.workunit.client.0.smithi042.stdout:3/679: mknod d1/d11/d14/d4d/cde 0 2024-02-15T18:27:03.535 INFO:tasks.workunit.client.0.smithi042.stdout:1/693: mkdir de/d54/dbf/dd0/d7c/dd6/de8/d103 0 2024-02-15T18:27:03.538 INFO:tasks.workunit.client.0.smithi042.stdout:0/652: link d4/f20 d4/d9/d5a/d60/fe1 0 2024-02-15T18:27:03.538 INFO:tasks.workunit.client.0.smithi042.stdout:0/653: write d4/d38/d5c/fc4 [555320,74537] 0 2024-02-15T18:27:03.538 INFO:tasks.workunit.client.0.smithi042.stdout:0/654: dread - d4/d38/d5c/f6a zero size 2024-02-15T18:27:03.538 INFO:tasks.workunit.client.0.smithi042.stdout:0/655: truncate d4/d9/d34/d8b/f90 747560 0 2024-02-15T18:27:03.541 INFO:tasks.workunit.client.1.smithi191.stdout:3/240: unlink d6/d3b/d56/l52 0 2024-02-15T18:27:03.543 INFO:tasks.workunit.client.0.smithi042.stdout:6/772: creat d9/d11/d46/d89/dbc/f10f x:0 0 0 2024-02-15T18:27:03.545 INFO:tasks.workunit.client.0.smithi042.stdout:2/630: creat d0/d40/fc3 x:0 0 0 2024-02-15T18:27:03.548 INFO:tasks.workunit.client.1.smithi191.stdout:0/173: unlink d7/c8 0 2024-02-15T18:27:03.548 INFO:tasks.workunit.client.1.smithi191.stdout:2/268: symlink df/d12/d19/d1a/d52/l5c 0 2024-02-15T18:27:03.552 INFO:tasks.workunit.client.0.smithi042.stdout:8/617: creat dd/de/da4/fad x:0 0 0 2024-02-15T18:27:03.553 INFO:tasks.workunit.client.0.smithi042.stdout:8/618: truncate dd/de/da4/d67/f43 1376502 0 2024-02-15T18:27:03.553 INFO:tasks.workunit.client.1.smithi191.stdout:2/269: dread f7 [0,4194304] 0 2024-02-15T18:27:03.553 INFO:tasks.workunit.client.1.smithi191.stdout:2/270: truncate df/d12/d19/d1a/d35/d47/f51 4347123 0 2024-02-15T18:27:03.553 INFO:tasks.workunit.client.1.smithi191.stdout:2/271: readlink df/d12/d1f/l56 0 2024-02-15T18:27:03.564 INFO:tasks.workunit.client.1.smithi191.stdout:8/392: creat d3/d2c/d1a/d23/d2e/d64/f8d x:0 0 0 2024-02-15T18:27:03.566 INFO:tasks.workunit.client.0.smithi042.stdout:5/715: symlink d0/d39/d8d/d9b/lfa 0 2024-02-15T18:27:03.566 INFO:tasks.workunit.client.0.smithi042.stdout:5/716: fdatasync d0/fce 0 2024-02-15T18:27:03.567 INFO:tasks.workunit.client.0.smithi042.stdout:9/695: creat d6/da/d7b/fef x:0 0 0 2024-02-15T18:27:03.567 INFO:tasks.workunit.client.0.smithi042.stdout:9/696: stat d6/da2/db1 0 2024-02-15T18:27:03.567 INFO:tasks.workunit.client.0.smithi042.stdout:9/697: stat d6/dd/d9/d59/d5c 0 2024-02-15T18:27:03.568 INFO:tasks.workunit.client.1.smithi191.stdout:1/394: mknod d0/d2f/d51/d54/d7b/c7e 0 2024-02-15T18:27:03.569 INFO:tasks.workunit.client.0.smithi042.stdout:4/690: rename da/d11/d67/d5c/d66/d78/d2f/d4d/fc8 to da/d3b/d41/ff0 0 2024-02-15T18:27:03.570 INFO:tasks.workunit.client.0.smithi042.stdout:7/717: getdents d10/d43 0 2024-02-15T18:27:03.571 INFO:tasks.workunit.client.0.smithi042.stdout:3/680: dwrite d1/d11/d14/f39 [0,4194304] 0 2024-02-15T18:27:03.571 INFO:tasks.workunit.client.0.smithi042.stdout:3/681: fdatasync d1/f34 0 2024-02-15T18:27:03.571 INFO:tasks.workunit.client.0.smithi042.stdout:3/682: chown d1/d11/d2e/d4c/cb1 1 1 2024-02-15T18:27:03.571 INFO:tasks.workunit.client.0.smithi042.stdout:3/683: chown d1/d11/c91 12 1 2024-02-15T18:27:03.571 INFO:tasks.workunit.client.0.smithi042.stdout:3/684: fsync d1/d11/f65 0 2024-02-15T18:27:03.571 INFO:tasks.workunit.client.0.smithi042.stdout:3/685: read d1/d11/f8f [61144,50969] 0 2024-02-15T18:27:03.572 INFO:tasks.workunit.client.0.smithi042.stdout:7/718: write d10/f14 [2919425,8778] 0 2024-02-15T18:27:03.572 INFO:tasks.workunit.client.1.smithi191.stdout:3/241: dwrite d6/d3b/d30/f46 [0,4194304] 0 2024-02-15T18:27:03.572 INFO:tasks.workunit.client.1.smithi191.stdout:3/242: dread - d6/d1b/f3e zero size 2024-02-15T18:27:03.572 INFO:tasks.workunit.client.1.smithi191.stdout:3/243: stat d6/d3b/d42/d49/f43 0 2024-02-15T18:27:03.573 INFO:tasks.workunit.client.1.smithi191.stdout:9/321: creat d0/d7/dc/d17/d33/f77 x:0 0 0 2024-02-15T18:27:03.573 INFO:tasks.workunit.client.1.smithi191.stdout:9/322: stat d0/d29/d48/f5b 0 2024-02-15T18:27:03.573 INFO:tasks.workunit.client.1.smithi191.stdout:9/323: truncate d0/d7/f11 291757 0 2024-02-15T18:27:03.573 INFO:tasks.workunit.client.1.smithi191.stdout:2/272: dwrite df/d12/f22 [0,4194304] 0 2024-02-15T18:27:03.574 INFO:tasks.workunit.client.0.smithi042.stdout:0/656: creat d4/d9/d34/dab/db0/fe2 x:0 0 0 2024-02-15T18:27:03.574 INFO:tasks.workunit.client.0.smithi042.stdout:0/657: write d4/f6 [928142,104806] 0 2024-02-15T18:27:03.576 INFO:tasks.workunit.client.1.smithi191.stdout:2/273: write fb [1073493,94426] 0 2024-02-15T18:27:03.576 INFO:tasks.workunit.client.1.smithi191.stdout:4/375: rename d2/d5/d7/d3e/d42 to d2/d5/d7/d90 0 2024-02-15T18:27:03.577 INFO:tasks.workunit.client.0.smithi042.stdout:6/773: getdents d9/d11/d19/d35/d55/d92/dfc 0 2024-02-15T18:27:03.578 INFO:tasks.workunit.client.0.smithi042.stdout:2/631: mknod d0/d8/d10/d31/cc4 0 2024-02-15T18:27:03.578 INFO:tasks.workunit.client.0.smithi042.stdout:2/632: chown d0/d8/d20/d7d/d94/c59 2458 1 2024-02-15T18:27:03.578 INFO:tasks.workunit.client.0.smithi042.stdout:2/633: chown d0/d8/d10/fa2 13372127 1 2024-02-15T18:27:03.579 INFO:tasks.workunit.client.0.smithi042.stdout:8/619: mkdir dd/de/da4/d81/dae 0 2024-02-15T18:27:03.580 INFO:tasks.workunit.client.0.smithi042.stdout:7/719: dread d10/f7b [0,4194304] 0 2024-02-15T18:27:03.581 INFO:tasks.workunit.client.0.smithi042.stdout:7/720: write d10/d12/f9e [3592198,61311] 0 2024-02-15T18:27:03.581 INFO:tasks.workunit.client.0.smithi042.stdout:5/717: creat d0/d50/df7/ffb x:0 0 0 2024-02-15T18:27:03.581 INFO:tasks.workunit.client.0.smithi042.stdout:5/718: chown d0/d38/f3c 7501 1 2024-02-15T18:27:03.585 INFO:tasks.workunit.client.1.smithi191.stdout:2/274: dread df/d12/f39 [0,4194304] 0 2024-02-15T18:27:03.586 INFO:tasks.workunit.client.0.smithi042.stdout:2/634: write d0/d8/d10/d5a/fab [6861555,112557] 0 2024-02-15T18:27:03.586 INFO:tasks.workunit.client.0.smithi042.stdout:1/694: rename de/d15/d67/d46/d6a/lc3 to de/d15/d2c/d51/d70/l104 0 2024-02-15T18:27:03.588 INFO:tasks.workunit.client.0.smithi042.stdout:4/691: creat da/d11/d29/d4a/ff1 x:0 0 0 2024-02-15T18:27:03.588 INFO:tasks.workunit.client.0.smithi042.stdout:3/686: mknod d1/d11/d2e/d4c/db0/cdf 0 2024-02-15T18:27:03.589 INFO:tasks.workunit.client.0.smithi042.stdout:2/635: write d0/d40/d56/f7f [878319,65019] 0 2024-02-15T18:27:03.589 INFO:tasks.workunit.client.0.smithi042.stdout:2/636: chown d0/d8/l5c 728 1 2024-02-15T18:27:03.589 INFO:tasks.workunit.client.1.smithi191.stdout:0/174: creat d7/da/d17/d31/f36 x:0 0 0 2024-02-15T18:27:03.591 INFO:tasks.workunit.client.1.smithi191.stdout:0/175: read d7/da/f18 [746966,110100] 0 2024-02-15T18:27:03.592 INFO:tasks.workunit.client.1.smithi191.stdout:8/393: mkdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d8e 0 2024-02-15T18:27:03.592 INFO:tasks.workunit.client.0.smithi042.stdout:0/658: dread d4/d9/d5a/f6b [0,4194304] 0 2024-02-15T18:27:03.593 INFO:tasks.workunit.client.0.smithi042.stdout:8/620: dread dd/de/d3c/f54 [0,4194304] 0 2024-02-15T18:27:03.593 INFO:tasks.workunit.client.0.smithi042.stdout:8/621: getdents dd/de/d33/d66/d4e 0 2024-02-15T18:27:03.593 INFO:tasks.workunit.client.0.smithi042.stdout:8/622: truncate dd/de/f2a 1618491 0 2024-02-15T18:27:03.593 INFO:tasks.workunit.client.1.smithi191.stdout:1/395: creat d0/d8/df/d2e/f7f x:0 0 0 2024-02-15T18:27:03.594 INFO:tasks.workunit.client.1.smithi191.stdout:1/396: write d0/d8/df/d41/f6d [909368,33079] 0 2024-02-15T18:27:03.594 INFO:tasks.workunit.client.1.smithi191.stdout:1/397: write d0/d8/df/d12/d2a/d2b/f61 [194826,125895] 0 2024-02-15T18:27:03.594 INFO:tasks.workunit.client.1.smithi191.stdout:1/398: dread - d0/d8/df/d2e/f7f zero size 2024-02-15T18:27:03.594 INFO:tasks.workunit.client.0.smithi042.stdout:6/774: creat d9/d11/d46/d89/dbc/f110 x:0 0 0 2024-02-15T18:27:03.595 INFO:tasks.workunit.client.0.smithi042.stdout:8/623: write dd/f29 [1540824,42765] 0 2024-02-15T18:27:03.595 INFO:tasks.workunit.client.0.smithi042.stdout:8/624: truncate dd/de/d33/f42 92632 0 2024-02-15T18:27:03.595 INFO:tasks.workunit.client.0.smithi042.stdout:8/625: truncate dd/d11/d75/f9d 686278 0 2024-02-15T18:27:03.595 INFO:tasks.workunit.client.0.smithi042.stdout:8/626: stat dd/de/da4/f76 0 2024-02-15T18:27:03.596 INFO:tasks.workunit.client.0.smithi042.stdout:5/719: creat d0/ffc x:0 0 0 2024-02-15T18:27:03.596 INFO:tasks.workunit.client.0.smithi042.stdout:5/720: write d0/d50/f5f [4555343,9544] 0 2024-02-15T18:27:03.596 INFO:tasks.workunit.client.0.smithi042.stdout:5/721: fsync d0/d38/d53/d91/db0/fb8 0 2024-02-15T18:27:03.600 INFO:tasks.workunit.client.0.smithi042.stdout:9/698: rename d6/da/d2f/l33 to d6/da/d2f/d55/d73/d8d/dcf/lf0 0 2024-02-15T18:27:03.601 INFO:tasks.workunit.client.1.smithi191.stdout:3/244: mkdir d6/d27/d4d/d5a 0 2024-02-15T18:27:03.602 INFO:tasks.workunit.client.0.smithi042.stdout:5/722: write d0/d39/d8d/fc6 [139312,78280] 0 2024-02-15T18:27:03.603 INFO:tasks.workunit.client.0.smithi042.stdout:1/695: unlink c3 0 2024-02-15T18:27:03.604 INFO:tasks.workunit.client.1.smithi191.stdout:9/324: creat d0/d7/da/f78 x:0 0 0 2024-02-15T18:27:03.607 INFO:tasks.workunit.client.0.smithi042.stdout:4/692: mknod da/d11/d67/d5c/d66/d78/d53/d82/cf2 0 2024-02-15T18:27:03.608 INFO:tasks.workunit.client.0.smithi042.stdout:4/693: chown da/d11/d67/d5c/d66/fb0 0 1 2024-02-15T18:27:03.608 INFO:tasks.workunit.client.0.smithi042.stdout:3/687: getdents d1/d11 0 2024-02-15T18:27:03.611 INFO:tasks.workunit.client.0.smithi042.stdout:8/627: dread dd/de/d33/d66/d4e/f28 [0,4194304] 0 2024-02-15T18:27:03.611 INFO:tasks.workunit.client.0.smithi042.stdout:8/628: chown dd/de/d94 17531785 1 2024-02-15T18:27:03.611 INFO:tasks.workunit.client.0.smithi042.stdout:8/629: readlink dd/de/d33/d3b/d92/l96 0 2024-02-15T18:27:03.613 INFO:tasks.workunit.client.1.smithi191.stdout:6/276: rename d8/d9/c13 to d8/c48 0 2024-02-15T18:27:03.614 INFO:tasks.workunit.client.1.smithi191.stdout:2/275: link df/f16 df/d12/f5d 0 2024-02-15T18:27:03.614 INFO:tasks.workunit.client.1.smithi191.stdout:2/276: dread - df/d12/d19/d1a/d35/d47/d4b/f59 zero size 2024-02-15T18:27:03.614 INFO:tasks.workunit.client.1.smithi191.stdout:3/245: dread d6/d3b/f13 [0,4194304] 0 2024-02-15T18:27:03.614 INFO:tasks.workunit.client.1.smithi191.stdout:3/246: dread - d6/d3b/d42/d49/f59 zero size 2024-02-15T18:27:03.614 INFO:tasks.workunit.client.1.smithi191.stdout:3/247: fdatasync d6/d1b/f3e 0 2024-02-15T18:27:03.619 INFO:tasks.workunit.client.1.smithi191.stdout:2/277: read df/d12/d19/d1a/f3d [2498051,109814] 0 2024-02-15T18:27:03.622 INFO:tasks.workunit.client.0.smithi042.stdout:9/699: dread d6/da/d2f/d55/fa3 [0,4194304] 0 2024-02-15T18:27:03.622 INFO:tasks.workunit.client.0.smithi042.stdout:9/700: write d6/dd/d9/fc3 [687716,73769] 0 2024-02-15T18:27:03.625 INFO:tasks.workunit.client.0.smithi042.stdout:2/637: rmdir d0/d8/d10/d7b 39 2024-02-15T18:27:03.625 INFO:tasks.workunit.client.0.smithi042.stdout:2/638: truncate d0/d40/d56/fa9 891209 0 2024-02-15T18:27:03.631 INFO:tasks.workunit.client.1.smithi191.stdout:2/278: dread df/d12/f5d [0,4194304] 0 2024-02-15T18:27:03.631 INFO:tasks.workunit.client.1.smithi191.stdout:0/176: getdents d7/da/d1a/d2c 0 2024-02-15T18:27:03.633 INFO:tasks.workunit.client.0.smithi042.stdout:0/659: creat d4/d9/d34/d57/d88/dc9/fe3 x:0 0 0 2024-02-15T18:27:03.633 INFO:tasks.workunit.client.0.smithi042.stdout:0/660: chown d4/lb1 1839 1 2024-02-15T18:27:03.633 INFO:tasks.workunit.client.0.smithi042.stdout:0/661: write d4/d9/d34/f40 [1541234,96109] 0 2024-02-15T18:27:03.633 INFO:tasks.workunit.client.0.smithi042.stdout:0/662: fdatasync d4/d9/d34/d57/d88/dc9/fe3 0 2024-02-15T18:27:03.633 INFO:tasks.workunit.client.0.smithi042.stdout:0/663: chown d4/d9/d62/d74 1820290272 1 2024-02-15T18:27:03.633 INFO:tasks.workunit.client.1.smithi191.stdout:8/394: creat d3/d2c/d1a/f8f x:0 0 0 2024-02-15T18:27:03.634 INFO:tasks.workunit.client.1.smithi191.stdout:1/399: symlink d0/d8/df/d12/d2a/d2b/d78/l80 0 2024-02-15T18:27:03.635 INFO:tasks.workunit.client.1.smithi191.stdout:9/325: dwrite d0/d7/dc/d30/f4e [0,4194304] 0 2024-02-15T18:27:03.635 INFO:tasks.workunit.client.0.smithi042.stdout:6/775: symlink d9/d11/d46/df8/l111 0 2024-02-15T18:27:03.637 INFO:tasks.workunit.client.1.smithi191.stdout:6/277: rename d8/d9/c3e to d8/c49 0 2024-02-15T18:27:03.637 INFO:tasks.workunit.client.0.smithi042.stdout:4/694: dwrite da/d11/f6d [4194304,4194304] 0 2024-02-15T18:27:03.638 INFO:tasks.workunit.client.1.smithi191.stdout:6/278: read d8/d9/f44 [556090,19330] 0 2024-02-15T18:27:03.638 INFO:tasks.workunit.client.0.smithi042.stdout:4/695: write da/d11/f23 [910541,43054] 0 2024-02-15T18:27:03.640 INFO:tasks.workunit.client.1.smithi191.stdout:3/248: link d6/d3b/c19 d6/d3b/d42/d49/d54/c5b 0 2024-02-15T18:27:03.640 INFO:tasks.workunit.client.1.smithi191.stdout:3/249: write d6/d1b/f3e [586322,118472] 0 2024-02-15T18:27:03.644 INFO:tasks.workunit.client.1.smithi191.stdout:0/177: rmdir d7/d1b 39 2024-02-15T18:27:03.646 INFO:tasks.workunit.client.1.smithi191.stdout:8/395: unlink d3/d2c/d1a/d23/f89 0 2024-02-15T18:27:03.647 INFO:tasks.workunit.client.1.smithi191.stdout:1/400: creat d0/d2f/d77/f81 x:0 0 0 2024-02-15T18:27:03.648 INFO:tasks.workunit.client.1.smithi191.stdout:1/401: write d0/d8/d48/f4d [1503883,86366] 0 2024-02-15T18:27:03.649 INFO:tasks.workunit.client.1.smithi191.stdout:7/316: rename d0/db/d26/d5f/f68 to d0/db/d65/d11/d1c/f6f 0 2024-02-15T18:27:03.652 INFO:tasks.workunit.client.0.smithi042.stdout:5/723: truncate d0/dda/dc7/de1/f15 2908755 0 2024-02-15T18:27:03.653 INFO:tasks.workunit.client.1.smithi191.stdout:9/326: dwrite d0/d7/dc/d30/f5c [0,4194304] 0 2024-02-15T18:27:03.653 INFO:tasks.workunit.client.1.smithi191.stdout:3/250: creat d6/d3b/d30/f5c x:0 0 0 2024-02-15T18:27:03.653 INFO:tasks.workunit.client.1.smithi191.stdout:3/251: fdatasync f4 0 2024-02-15T18:27:03.653 INFO:tasks.workunit.client.1.smithi191.stdout:3/252: chown d6/d3b/d42/f51 186 1 2024-02-15T18:27:03.654 INFO:tasks.workunit.client.0.smithi042.stdout:1/696: link de/d6d/d9e/cbe de/d15/d67/d8e/c105 0 2024-02-15T18:27:03.656 INFO:tasks.workunit.client.1.smithi191.stdout:0/178: link d7/da/le d7/da/d17/l37 0 2024-02-15T18:27:03.656 INFO:tasks.workunit.client.1.smithi191.stdout:0/179: write d7/d1b/f23 [847629,37497] 0 2024-02-15T18:27:03.657 INFO:tasks.workunit.client.1.smithi191.stdout:8/396: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d8e/f90 x:0 0 0 2024-02-15T18:27:03.658 INFO:tasks.workunit.client.0.smithi042.stdout:3/688: mknod d1/d11/d14/d4d/d73/d89/dd9/ce0 0 2024-02-15T18:27:03.663 INFO:tasks.workunit.client.1.smithi191.stdout:1/402: dwrite d0/d2f/d51/fe [0,4194304] 0 2024-02-15T18:27:03.663 INFO:tasks.workunit.client.1.smithi191.stdout:1/403: write d0/d8/df/d12/d2a/d2b/f37 [4969396,85354] 0 2024-02-15T18:27:03.663 INFO:tasks.workunit.client.1.smithi191.stdout:0/180: fdatasync d7/d1b/f23 0 2024-02-15T18:27:03.669 INFO:tasks.workunit.client.1.smithi191.stdout:2/279: rename df/d12/d19/d1a/d35/d47/d4b/f59 to df/d12/d19/d1a/f5e 0 2024-02-15T18:27:03.670 INFO:tasks.workunit.client.0.smithi042.stdout:8/630: unlink dd/f48 0 2024-02-15T18:27:03.671 INFO:tasks.workunit.client.1.smithi191.stdout:0/181: dread d7/da/f16 [0,4194304] 0 2024-02-15T18:27:03.671 INFO:tasks.workunit.client.1.smithi191.stdout:7/317: mknod d0/db/d65/dc/c70 0 2024-02-15T18:27:03.673 INFO:tasks.workunit.client.1.smithi191.stdout:6/279: mkdir d8/d9/d4a 0 2024-02-15T18:27:03.673 INFO:tasks.workunit.client.1.smithi191.stdout:6/280: chown d8/d9/c27 96067 1 2024-02-15T18:27:03.674 INFO:tasks.workunit.client.0.smithi042.stdout:9/701: symlink d6/da/d2f/d6c/dd8/lf1 0 2024-02-15T18:27:03.674 INFO:tasks.workunit.client.0.smithi042.stdout:9/702: readlink d6/da/d2f/d54/lc2 0 2024-02-15T18:27:03.678 INFO:tasks.workunit.client.1.smithi191.stdout:9/327: creat d0/d1d/f79 x:0 0 0 2024-02-15T18:27:03.680 INFO:tasks.workunit.client.1.smithi191.stdout:3/253: mkdir d6/d5d 0 2024-02-15T18:27:03.681 INFO:tasks.workunit.client.1.smithi191.stdout:1/404: mknod d0/d8/df/d12/d2a/c82 0 2024-02-15T18:27:03.683 INFO:tasks.workunit.client.1.smithi191.stdout:0/182: link d7/da/le d7/da/d1a/d34/d27/l38 0 2024-02-15T18:27:03.684 INFO:tasks.workunit.client.1.smithi191.stdout:0/183: chown d7/da/d17/c1d 6 1 2024-02-15T18:27:03.684 INFO:tasks.workunit.client.1.smithi191.stdout:0/184: write d7/da/d1a/d34/f11 [884215,37694] 0 2024-02-15T18:27:03.684 INFO:tasks.workunit.client.1.smithi191.stdout:0/185: stat d7 0 2024-02-15T18:27:03.684 INFO:tasks.workunit.client.1.smithi191.stdout:2/280: dwrite df/d12/d1f/f26 [0,4194304] 0 2024-02-15T18:27:03.685 INFO:tasks.workunit.client.1.smithi191.stdout:7/318: mknod d0/db/d65/dc/d6e/c71 0 2024-02-15T18:27:03.686 INFO:tasks.workunit.client.0.smithi042.stdout:2/639: symlink d0/d8/d10/d5a/lc5 0 2024-02-15T18:27:03.687 INFO:tasks.workunit.client.1.smithi191.stdout:6/281: creat d8/d9/d2b/f4b x:0 0 0 2024-02-15T18:27:03.687 INFO:tasks.workunit.client.1.smithi191.stdout:6/282: write d8/d9/f46 [1032563,108791] 0 2024-02-15T18:27:03.688 INFO:tasks.workunit.client.1.smithi191.stdout:6/283: chown d8/d19/f3d 33 1 2024-02-15T18:27:03.688 INFO:tasks.workunit.client.0.smithi042.stdout:0/664: symlink d4/d9/d8d/le4 0 2024-02-15T18:27:03.696 INFO:tasks.workunit.client.1.smithi191.stdout:3/254: mkdir d6/d3b/d56/d5e 0 2024-02-15T18:27:03.696 INFO:tasks.workunit.client.1.smithi191.stdout:3/255: dread - d6/d3b/d30/f45 zero size 2024-02-15T18:27:03.696 INFO:tasks.workunit.client.1.smithi191.stdout:3/256: write d6/d3b/d42/f31 [5151177,37745] 0 2024-02-15T18:27:03.697 INFO:tasks.workunit.client.0.smithi042.stdout:6/776: creat d9/d3e/f112 x:0 0 0 2024-02-15T18:27:03.700 INFO:tasks.workunit.client.1.smithi191.stdout:8/397: link d3/d2c/d1a/d23/d3e/l59 d3/l91 0 2024-02-15T18:27:03.700 INFO:tasks.workunit.client.1.smithi191.stdout:8/398: dread - d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d8e/f90 zero size 2024-02-15T18:27:03.700 INFO:tasks.workunit.client.1.smithi191.stdout:8/399: readlink d3/l2b 0 2024-02-15T18:27:03.700 INFO:tasks.workunit.client.1.smithi191.stdout:8/400: read - d3/d2c/d1a/d23/f7c zero size 2024-02-15T18:27:03.700 INFO:tasks.workunit.client.1.smithi191.stdout:8/401: write d3/d2c/d1a/d23/d2e/d47/f88 [311039,12745] 0 2024-02-15T18:27:03.701 INFO:tasks.workunit.client.1.smithi191.stdout:7/319: dwrite d0/db/d65/d11/d5a/d66/f61 [0,4194304] 0 2024-02-15T18:27:03.703 INFO:tasks.workunit.client.1.smithi191.stdout:1/405: mknod d0/d2f/d51/d5e/d68/c83 0 2024-02-15T18:27:03.703 INFO:tasks.workunit.client.1.smithi191.stdout:1/406: dread - d0/d2f/f73 zero size 2024-02-15T18:27:03.704 INFO:tasks.workunit.client.1.smithi191.stdout:1/407: fdatasync d0/d8/df/d12/d33/f63 0 2024-02-15T18:27:03.705 INFO:tasks.workunit.client.0.smithi042.stdout:4/696: getdents da/d11/d29 0 2024-02-15T18:27:03.705 INFO:tasks.workunit.client.0.smithi042.stdout:4/697: read da/d11/d29/fe8 [199288,10289] 0 2024-02-15T18:27:03.705 INFO:tasks.workunit.client.0.smithi042.stdout:4/698: chown da/f25 252 1 2024-02-15T18:27:03.706 INFO:tasks.workunit.client.0.smithi042.stdout:4/699: write da/d11/fe5 [92028,86541] 0 2024-02-15T18:27:03.707 INFO:tasks.workunit.client.1.smithi191.stdout:0/186: creat d7/f39 x:0 0 0 2024-02-15T18:27:03.710 INFO:tasks.workunit.client.0.smithi042.stdout:5/724: truncate d0/d38/d53/d91/db0/fb8 2190368 0 2024-02-15T18:27:03.710 INFO:tasks.workunit.client.0.smithi042.stdout:5/725: truncate d0/d38/f5a 4271904 0 2024-02-15T18:27:03.711 INFO:tasks.workunit.client.0.smithi042.stdout:1/697: creat de/d15/d1b/d44/f106 x:0 0 0 2024-02-15T18:27:03.713 INFO:tasks.workunit.client.1.smithi191.stdout:2/281: rename df/c25 to df/c5f 0 2024-02-15T18:27:03.713 INFO:tasks.workunit.client.1.smithi191.stdout:2/282: readlink df/l38 0 2024-02-15T18:27:03.720 INFO:tasks.workunit.client.0.smithi042.stdout:8/631: creat dd/de/d12/faf x:0 0 0 2024-02-15T18:27:03.722 INFO:tasks.workunit.client.0.smithi042.stdout:9/703: link d6/da/d2f/f56 d6/da/d75/ff2 0 2024-02-15T18:27:03.724 INFO:tasks.workunit.client.1.smithi191.stdout:6/284: getdents d8/d9/d2b 0 2024-02-15T18:27:03.724 INFO:tasks.workunit.client.1.smithi191.stdout:6/285: chown d8/l16 606 1 2024-02-15T18:27:03.726 INFO:tasks.workunit.client.0.smithi042.stdout:2/640: mkdir d0/d35/dc6 0 2024-02-15T18:27:03.727 INFO:tasks.workunit.client.1.smithi191.stdout:3/257: creat d6/d3b/d56/f5f x:0 0 0 2024-02-15T18:27:03.728 INFO:tasks.workunit.client.0.smithi042.stdout:2/641: dread d0/d8/db/de/f14 [0,4194304] 0 2024-02-15T18:27:03.729 INFO:tasks.workunit.client.0.smithi042.stdout:0/665: unlink d4/d9/d34/dab/dc8/d6c/d70/ld7 0 2024-02-15T18:27:03.730 INFO:tasks.workunit.client.1.smithi191.stdout:8/402: symlink d3/d2c/d1a/d23/d2a/l92 0 2024-02-15T18:27:03.730 INFO:tasks.workunit.client.1.smithi191.stdout:8/403: rename d3/d2c/d1a/d23 to d3/d2c/d1a/d23/d93 22 2024-02-15T18:27:03.730 INFO:tasks.workunit.client.1.smithi191.stdout:8/404: truncate d3/d2c/d1a/d23/d2a/f35 626858 0 2024-02-15T18:27:03.731 INFO:tasks.workunit.client.0.smithi042.stdout:6/777: symlink d9/d11/d19/d1c/d31/d53/l113 0 2024-02-15T18:27:03.731 INFO:tasks.workunit.client.0.smithi042.stdout:6/778: chown d9/d11/d19/d35/d55/fac 40921797 1 2024-02-15T18:27:03.732 INFO:tasks.workunit.client.1.smithi191.stdout:7/320: symlink d0/db/d65/d11/d57/l72 0 2024-02-15T18:27:03.732 INFO:tasks.workunit.client.0.smithi042.stdout:4/700: symlink da/d11/d67/d5c/lf3 0 2024-02-15T18:27:03.735 INFO:tasks.workunit.client.1.smithi191.stdout:1/408: getdents d0/d2f/d77 0 2024-02-15T18:27:03.735 INFO:tasks.workunit.client.0.smithi042.stdout:2/642: dread d0/d8/d10/d7b/faa [0,4194304] 0 2024-02-15T18:27:03.735 INFO:tasks.workunit.client.0.smithi042.stdout:2/643: chown d0/d8/db/d44/l76 968 1 2024-02-15T18:27:03.735 INFO:tasks.workunit.client.0.smithi042.stdout:5/726: rename d0/dda/d42/d7d/fc2 to d0/dda/dc7/ffd 0 2024-02-15T18:27:03.738 INFO:tasks.workunit.client.1.smithi191.stdout:0/187: creat d7/da/d1a/d34/d27/f3a x:0 0 0 2024-02-15T18:27:03.738 INFO:tasks.workunit.client.1.smithi191.stdout:0/188: read - d7/da/d17/d31/f36 zero size 2024-02-15T18:27:03.740 INFO:tasks.workunit.client.0.smithi042.stdout:3/689: sync 2024-02-15T18:27:03.740 INFO:tasks.workunit.client.0.smithi042.stdout:7/721: sync 2024-02-15T18:27:03.741 INFO:tasks.workunit.client.1.smithi191.stdout:7/321: dread d0/db/d65/d11/d5a/d66/f61 [0,4194304] 0 2024-02-15T18:27:03.741 INFO:tasks.workunit.client.1.smithi191.stdout:7/322: rename d0 to d0/db/d65/dc/d73 22 2024-02-15T18:27:03.741 INFO:tasks.workunit.client.0.smithi042.stdout:3/690: read d1/d11/d14/d4d/d6c/f75 [1031723,44055] 0 2024-02-15T18:27:03.742 INFO:tasks.workunit.client.0.smithi042.stdout:3/691: truncate d1/d11/d14/d4d/d73/d89/da8/fc7 983936 0 2024-02-15T18:27:03.743 INFO:tasks.workunit.client.0.smithi042.stdout:1/698: creat de/d15/d67/d74/f107 x:0 0 0 2024-02-15T18:27:03.746 INFO:tasks.workunit.client.0.smithi042.stdout:8/632: creat dd/de/da4/d67/d6f/fb0 x:0 0 0 2024-02-15T18:27:03.746 INFO:tasks.workunit.client.1.smithi191.stdout:2/283: creat df/d12/d19/d1a/d35/f60 x:0 0 0 2024-02-15T18:27:03.748 INFO:tasks.workunit.client.1.smithi191.stdout:6/286: mkdir d8/d9/d4a/d4c 0 2024-02-15T18:27:03.749 INFO:tasks.workunit.client.0.smithi042.stdout:9/704: mkdir d6/da/d2f/d55/d73/d8d/df3 0 2024-02-15T18:27:03.752 INFO:tasks.workunit.client.0.smithi042.stdout:0/666: mknod d4/d38/d77/d2e/dae/ddf/ce5 0 2024-02-15T18:27:03.752 INFO:tasks.workunit.client.0.smithi042.stdout:0/667: write d4/d38/fdd [883328,71430] 0 2024-02-15T18:27:03.754 INFO:tasks.workunit.client.0.smithi042.stdout:4/701: rmdir da/d11/d29/d4a/d4f/dcf/dd4 0 2024-02-15T18:27:03.758 INFO:tasks.workunit.client.1.smithi191.stdout:3/258: mkdir d6/d1b/d60 0 2024-02-15T18:27:03.758 INFO:tasks.workunit.client.1.smithi191.stdout:1/409: symlink d0/d8/df/d12/d33/d49/l84 0 2024-02-15T18:27:03.758 INFO:tasks.workunit.client.1.smithi191.stdout:1/410: fsync d0/d8/df/f2c 0 2024-02-15T18:27:03.759 INFO:tasks.workunit.client.0.smithi042.stdout:7/722: dread d10/d43/f4f [8388608,4194304] 0 2024-02-15T18:27:03.759 INFO:tasks.workunit.client.1.smithi191.stdout:0/189: rename d7/d26 to d7/d3b 0 2024-02-15T18:27:03.762 INFO:tasks.workunit.client.0.smithi042.stdout:5/727: dwrite d0/d50/df7/ffb [0,4194304] 0 2024-02-15T18:27:03.762 INFO:tasks.workunit.client.0.smithi042.stdout:5/728: dread - d0/dda/dc7/fcc zero size 2024-02-15T18:27:03.766 INFO:tasks.workunit.client.0.smithi042.stdout:2/644: rename d0/d40/d56 to d0/d8/d20/d7d/d7e/dc7 0 2024-02-15T18:27:03.766 INFO:tasks.workunit.client.0.smithi042.stdout:2/645: fsync d0/d8/db/de/f4d 0 2024-02-15T18:27:03.767 INFO:tasks.workunit.client.1.smithi191.stdout:7/323: unlink d0/db/d65/dc/f39 0 2024-02-15T18:27:03.768 INFO:tasks.workunit.client.1.smithi191.stdout:6/287: mknod d8/d9/d4a/c4d 0 2024-02-15T18:27:03.768 INFO:tasks.workunit.client.1.smithi191.stdout:6/288: getdents d8/d9/d4a/d4c 0 2024-02-15T18:27:03.768 INFO:tasks.workunit.client.1.smithi191.stdout:8/405: dwrite d3/d2c/d1a/f20 [0,4194304] 0 2024-02-15T18:27:03.769 INFO:tasks.workunit.client.1.smithi191.stdout:8/406: chown d3/d2c/d1a/d23/f4c 24801 1 2024-02-15T18:27:03.769 INFO:tasks.workunit.client.0.smithi042.stdout:3/692: creat d1/d11/d7a/fe1 x:0 0 0 2024-02-15T18:27:03.770 INFO:tasks.workunit.client.1.smithi191.stdout:3/259: creat d6/d3b/d56/f61 x:0 0 0 2024-02-15T18:27:03.770 INFO:tasks.workunit.client.1.smithi191.stdout:3/260: write d6/d1b/f39 [578997,43021] 0 2024-02-15T18:27:03.770 INFO:tasks.workunit.client.1.smithi191.stdout:3/261: chown d6/d3b/d42/d49/f3d 22 1 2024-02-15T18:27:03.770 INFO:tasks.workunit.client.0.smithi042.stdout:1/699: symlink de/d15/d95/l108 0 2024-02-15T18:27:03.770 INFO:tasks.workunit.client.0.smithi042.stdout:1/700: readlink de/d15/d67/d8e/ld2 0 2024-02-15T18:27:03.771 INFO:tasks.workunit.client.0.smithi042.stdout:1/701: write de/d15/d2c/f41 [553448,48639] 0 2024-02-15T18:27:03.771 INFO:tasks.workunit.client.1.smithi191.stdout:1/411: creat d0/d8/df/f85 x:0 0 0 2024-02-15T18:27:03.772 INFO:tasks.workunit.client.1.smithi191.stdout:3/262: write d6/d3b/d42/d49/f43 [2680287,99622] 0 2024-02-15T18:27:03.772 INFO:tasks.workunit.client.1.smithi191.stdout:3/263: fsync d6/d3b/d56/f34 0 2024-02-15T18:27:03.772 INFO:tasks.workunit.client.0.smithi042.stdout:8/633: rmdir dd/d11/d75 39 2024-02-15T18:27:03.773 INFO:tasks.workunit.client.1.smithi191.stdout:0/190: rename d7/d3b/c28 to d7/da/d1a/d34/c3c 0 2024-02-15T18:27:03.776 INFO:tasks.workunit.client.0.smithi042.stdout:9/705: link d6/da/f25 d6/da/d2f/d7c/ff4 0 2024-02-15T18:27:03.777 INFO:tasks.workunit.client.0.smithi042.stdout:3/693: dread d1/d11/d14/d4d/f4f [0,4194304] 0 2024-02-15T18:27:03.780 INFO:tasks.workunit.client.0.smithi042.stdout:0/668: truncate d4/d38/d5c/f61 1387635 0 2024-02-15T18:27:03.781 INFO:tasks.workunit.client.1.smithi191.stdout:7/324: unlink d0/db/f55 0 2024-02-15T18:27:03.781 INFO:tasks.workunit.client.1.smithi191.stdout:6/289: creat d8/d9/d2b/f4e x:0 0 0 2024-02-15T18:27:03.783 INFO:tasks.workunit.client.0.smithi042.stdout:7/723: mkdir d10/d15/d39/dd9/dec 0 2024-02-15T18:27:03.784 INFO:tasks.workunit.client.1.smithi191.stdout:1/412: unlink d0/d8/df/l6c 0 2024-02-15T18:27:03.784 INFO:tasks.workunit.client.1.smithi191.stdout:1/413: write d0/d2f/d51/d10/f2d [2234105,59786] 0 2024-02-15T18:27:03.786 INFO:tasks.workunit.client.1.smithi191.stdout:0/191: mknod d7/da/d1a/d34/c3d 0 2024-02-15T18:27:03.786 INFO:tasks.workunit.client.1.smithi191.stdout:0/192: write d7/da/d1a/d34/f29 [8591175,35520] 0 2024-02-15T18:27:03.788 INFO:tasks.workunit.client.1.smithi191.stdout:2/284: dwrite f9 [0,4194304] 0 2024-02-15T18:27:03.789 INFO:tasks.workunit.client.1.smithi191.stdout:2/285: chown df/d12/d1f/f45 27 1 2024-02-15T18:27:03.789 INFO:tasks.workunit.client.0.smithi042.stdout:5/729: mkdir d0/d14/d88/dfe 0 2024-02-15T18:27:03.789 INFO:tasks.workunit.client.0.smithi042.stdout:5/730: readlink d0/d12/d2a/l51 0 2024-02-15T18:27:03.792 INFO:tasks.workunit.client.0.smithi042.stdout:2/646: creat d0/fc8 x:0 0 0 2024-02-15T18:27:03.793 INFO:tasks.workunit.client.1.smithi191.stdout:1/414: creat d0/d2f/d51/d5e/d68/f86 x:0 0 0 2024-02-15T18:27:03.793 INFO:tasks.workunit.client.1.smithi191.stdout:7/325: write d0/db/f2d [6686689,30702] 0 2024-02-15T18:27:03.793 INFO:tasks.workunit.client.1.smithi191.stdout:7/326: chown d0/db/d65/d11/d5a/d66/f53 354309 1 2024-02-15T18:27:03.794 INFO:tasks.workunit.client.1.smithi191.stdout:6/290: write d8/f30 [3362664,85852] 0 2024-02-15T18:27:03.794 INFO:tasks.workunit.client.1.smithi191.stdout:6/291: stat d8/f1c 0 2024-02-15T18:27:03.795 INFO:tasks.workunit.client.1.smithi191.stdout:0/193: rename d7/da/c10 to d7/d1b/d30/c3e 0 2024-02-15T18:27:03.795 INFO:tasks.workunit.client.1.smithi191.stdout:0/194: stat d7/da/d1a/d34/c3d 0 2024-02-15T18:27:03.796 INFO:tasks.workunit.client.0.smithi042.stdout:3/694: dread d1/d11/d2e/f37 [0,4194304] 0 2024-02-15T18:27:03.797 INFO:tasks.workunit.client.0.smithi042.stdout:4/702: dwrite da/d11/d67/d5c/d66/d78/d2f/d4d/f48 [4194304,4194304] 0 2024-02-15T18:27:03.797 INFO:tasks.workunit.client.0.smithi042.stdout:0/669: rename d4/d38/f49 to d4/d38/d77/fe6 0 2024-02-15T18:27:03.798 INFO:tasks.workunit.client.0.smithi042.stdout:0/670: write d4/d38/d5c/fc4 [119116,26485] 0 2024-02-15T18:27:03.799 INFO:tasks.workunit.client.0.smithi042.stdout:4/703: dread da/d3a/dc9/fd7 [0,4194304] 0 2024-02-15T18:27:03.804 INFO:tasks.workunit.client.1.smithi191.stdout:2/286: mkdir df/d61 0 2024-02-15T18:27:03.804 INFO:tasks.workunit.client.1.smithi191.stdout:2/287: fsync df/d12/f22 0 2024-02-15T18:27:03.805 INFO:tasks.workunit.client.0.smithi042.stdout:7/724: creat d10/d12/fed x:0 0 0 2024-02-15T18:27:03.806 INFO:tasks.workunit.client.0.smithi042.stdout:8/634: dwrite dd/d11/d75/f9d [0,4194304] 0 2024-02-15T18:27:03.807 INFO:tasks.workunit.client.0.smithi042.stdout:5/731: creat d0/d89/fff x:0 0 0 2024-02-15T18:27:03.807 INFO:tasks.workunit.client.0.smithi042.stdout:5/732: fsync d0/d38/d53/d68/dec/fae 0 2024-02-15T18:27:03.808 INFO:tasks.workunit.client.1.smithi191.stdout:7/327: mknod d0/d1/df/d5d/c74 0 2024-02-15T18:27:03.808 INFO:tasks.workunit.client.1.smithi191.stdout:7/328: write d0/db/f2f [904642,73638] 0 2024-02-15T18:27:03.811 INFO:tasks.workunit.client.0.smithi042.stdout:3/695: getdents d1/de/d19 0 2024-02-15T18:27:03.811 INFO:tasks.workunit.client.0.smithi042.stdout:3/696: truncate d1/de/d19/f44 5648143 0 2024-02-15T18:27:03.812 INFO:tasks.workunit.client.1.smithi191.stdout:0/195: chown l4 1321 1 2024-02-15T18:27:03.812 INFO:tasks.workunit.client.1.smithi191.stdout:3/264: dwrite d6/d27/d4d/f58 [0,4194304] 0 2024-02-15T18:27:03.812 INFO:tasks.workunit.client.1.smithi191.stdout:3/265: readlink d6/d27/l4f 0 2024-02-15T18:27:03.813 INFO:tasks.workunit.client.0.smithi042.stdout:9/706: dwrite d6/f8 [4194304,4194304] 0 2024-02-15T18:27:03.813 INFO:tasks.workunit.client.0.smithi042.stdout:1/702: dwrite de/d15/d1b/d44/fbd [0,4194304] 0 2024-02-15T18:27:03.813 INFO:tasks.workunit.client.0.smithi042.stdout:1/703: truncate de/d15/d2c/f4f 861583 0 2024-02-15T18:27:03.813 INFO:tasks.workunit.client.0.smithi042.stdout:2/647: rename d0/d8/d10/d31/d3a/d53/l26 to d0/d35/lc9 0 2024-02-15T18:27:03.813 INFO:tasks.workunit.client.0.smithi042.stdout:2/648: fsync d0/d8/d10/d31/da6/fac 0 2024-02-15T18:27:03.813 INFO:tasks.workunit.client.1.smithi191.stdout:8/407: dwrite d3/d2c/d1a/d23/d2e/d47/d5f/f50 [0,4194304] 0 2024-02-15T18:27:03.813 INFO:tasks.workunit.client.1.smithi191.stdout:1/415: dread d0/d2f/d51/f1d [0,4194304] 0 2024-02-15T18:27:03.813 INFO:tasks.workunit.client.1.smithi191.stdout:1/416: fdatasync d0/d8/f69 0 2024-02-15T18:27:03.815 INFO:tasks.workunit.client.0.smithi042.stdout:0/671: getdents d4/d9/d5a 0 2024-02-15T18:27:03.815 INFO:tasks.workunit.client.0.smithi042.stdout:0/672: dread - d4/d38/d77/d2e/dae/ddf/fb6 zero size 2024-02-15T18:27:03.816 INFO:tasks.workunit.client.1.smithi191.stdout:8/408: write d3/d2c/d1a/f2d [2704278,35940] 0 2024-02-15T18:27:03.818 INFO:tasks.workunit.client.1.smithi191.stdout:2/288: link df/d12/d19/d1a/c23 df/d12/d19/d1a/d35/c62 0 2024-02-15T18:27:03.822 INFO:tasks.workunit.client.0.smithi042.stdout:4/704: mknod da/d11/d27/cf4 0 2024-02-15T18:27:03.832 INFO:tasks.workunit.client.0.smithi042.stdout:7/725: dwrite d10/d15/d8f/da5/fb7 [0,4194304] 0 2024-02-15T18:27:03.836 INFO:tasks.workunit.client.1.smithi191.stdout:0/196: dwrite d7/da/d1a/d34/f11 [0,4194304] 0 2024-02-15T18:27:03.836 INFO:tasks.workunit.client.1.smithi191.stdout:0/197: truncate d7/f39 173897 0 2024-02-15T18:27:03.840 INFO:tasks.workunit.client.1.smithi191.stdout:7/329: creat d0/db/d65/f75 x:0 0 0 2024-02-15T18:27:03.841 INFO:tasks.workunit.client.0.smithi042.stdout:8/635: rmdir dd/de/da4/d67/d6f 39 2024-02-15T18:27:03.843 INFO:tasks.workunit.client.1.smithi191.stdout:6/292: symlink d8/d9/l4f 0 2024-02-15T18:27:03.843 INFO:tasks.workunit.client.1.smithi191.stdout:7/330: write d0/d1/df/f64 [3254663,70149] 0 2024-02-15T18:27:03.844 INFO:tasks.workunit.client.0.smithi042.stdout:5/733: symlink d0/d39/d8d/d9b/df1/l100 0 2024-02-15T18:27:03.844 INFO:tasks.workunit.client.0.smithi042.stdout:5/734: chown d0/dda/c43 3 1 2024-02-15T18:27:03.844 INFO:tasks.workunit.client.0.smithi042.stdout:5/735: fdatasync d0/dda/dc7/de1/f1f 0 2024-02-15T18:27:03.844 INFO:tasks.workunit.client.0.smithi042.stdout:5/736: chown d0/c5 1893420 1 2024-02-15T18:27:03.845 INFO:tasks.workunit.client.0.smithi042.stdout:3/697: creat d1/de/fe2 x:0 0 0 2024-02-15T18:27:03.845 INFO:tasks.workunit.client.0.smithi042.stdout:3/698: write d1/d11/d14/f47 [2216788,273] 0 2024-02-15T18:27:03.848 INFO:tasks.workunit.client.0.smithi042.stdout:9/707: symlink d6/d9d/lf5 0 2024-02-15T18:27:03.850 INFO:tasks.workunit.client.1.smithi191.stdout:3/266: mkdir d6/d27/d4d/d5a/d62 0 2024-02-15T18:27:03.855 INFO:tasks.workunit.client.0.smithi042.stdout:5/737: write d0/d38/f5a [1692562,121580] 0 2024-02-15T18:27:03.856 INFO:tasks.workunit.client.1.smithi191.stdout:0/198: dread d7/da/f20 [0,4194304] 0 2024-02-15T18:27:03.860 INFO:tasks.workunit.client.1.smithi191.stdout:1/417: getdents d0/d8/df 0 2024-02-15T18:27:03.860 INFO:tasks.workunit.client.1.smithi191.stdout:1/418: chown d0/d2f/d51/f1d 124 1 2024-02-15T18:27:03.866 INFO:tasks.workunit.client.0.smithi042.stdout:1/704: mkdir de/d6d/d9e/d109 0 2024-02-15T18:27:03.866 INFO:tasks.workunit.client.0.smithi042.stdout:2/649: getdents d0/d8/db/de 0 2024-02-15T18:27:03.868 INFO:tasks.workunit.client.0.smithi042.stdout:0/673: rename d4/d9/d15/f18 to d4/d9/d34/fe7 0 2024-02-15T18:27:03.870 INFO:tasks.workunit.client.0.smithi042.stdout:4/705: creat da/d11/d29/d56/ff5 x:0 0 0 2024-02-15T18:27:03.870 INFO:tasks.workunit.client.0.smithi042.stdout:7/726: dwrite d10/d49/f8e [0,4194304] 0 2024-02-15T18:27:03.871 INFO:tasks.workunit.client.1.smithi191.stdout:6/293: dwrite d8/d19/f37 [0,4194304] 0 2024-02-15T18:27:03.872 INFO:tasks.workunit.client.0.smithi042.stdout:3/699: dwrite d1/de/d1a/f5a [0,4194304] 0 2024-02-15T18:27:03.872 INFO:tasks.workunit.client.0.smithi042.stdout:1/705: dread de/d54/f5e [0,4194304] 0 2024-02-15T18:27:03.875 INFO:tasks.workunit.client.0.smithi042.stdout:2/650: dread d0/d8/d10/d31/d3a/d53/f5d [0,4194304] 0 2024-02-15T18:27:03.876 INFO:tasks.workunit.client.1.smithi191.stdout:6/294: dread d8/d9/f44 [0,4194304] 0 2024-02-15T18:27:03.876 INFO:tasks.workunit.client.1.smithi191.stdout:6/295: truncate f6 17011054 0 2024-02-15T18:27:03.879 INFO:tasks.workunit.client.0.smithi042.stdout:6/779: sync 2024-02-15T18:27:03.879 INFO:tasks.workunit.client.0.smithi042.stdout:6/780: truncate d9/d11/d19/d1c/d31/d53/dbe/fe9 497473 0 2024-02-15T18:27:03.881 INFO:tasks.workunit.client.0.smithi042.stdout:6/781: write d9/f2f [1461787,89965] 0 2024-02-15T18:27:03.883 INFO:tasks.workunit.client.0.smithi042.stdout:4/706: dread da/d3a/f44 [0,4194304] 0 2024-02-15T18:27:03.883 INFO:tasks.workunit.client.0.smithi042.stdout:4/707: chown da/d11/d85/db5/fed 146 1 2024-02-15T18:27:03.885 INFO:tasks.workunit.client.0.smithi042.stdout:8/636: mkdir dd/de/d94/db1 0 2024-02-15T18:27:03.887 INFO:tasks.workunit.client.0.smithi042.stdout:9/708: symlink d6/da/d1d/d2d/d40/d95/lf6 0 2024-02-15T18:27:03.892 INFO:tasks.workunit.client.0.smithi042.stdout:5/738: rename d0/d22/c70 to d0/d14/ddb/c101 0 2024-02-15T18:27:03.892 INFO:tasks.workunit.client.0.smithi042.stdout:5/739: readlink d0/d39/l7e 0 2024-02-15T18:27:03.893 INFO:tasks.workunit.client.1.smithi191.stdout:8/409: mknod d3/d5e/c94 0 2024-02-15T18:27:03.895 INFO:tasks.workunit.client.0.smithi042.stdout:1/706: dwrite de/d54/fb0 [0,4194304] 0 2024-02-15T18:27:03.895 INFO:tasks.workunit.client.1.smithi191.stdout:2/289: truncate df/d12/d19/d1a/f44 3130389 0 2024-02-15T18:27:03.896 INFO:tasks.workunit.client.1.smithi191.stdout:2/290: fdatasync df/d12/d19/d1a/d35/d47/f51 0 2024-02-15T18:27:03.897 INFO:tasks.workunit.client.0.smithi042.stdout:0/674: creat d4/d38/d77/d2e/dae/ddf/fe8 x:0 0 0 2024-02-15T18:27:03.901 INFO:tasks.workunit.client.1.smithi191.stdout:2/291: dread df/d12/f39 [0,4194304] 0 2024-02-15T18:27:03.901 INFO:tasks.workunit.client.1.smithi191.stdout:2/292: truncate df/d12/d19/d1a/d35/f60 724517 0 2024-02-15T18:27:03.902 INFO:tasks.workunit.client.0.smithi042.stdout:0/675: write d4/d9/d15/f21 [2273750,8484] 0 2024-02-15T18:27:03.903 INFO:tasks.workunit.client.0.smithi042.stdout:0/676: dread d4/d38/d5c/fa2 [0,4194304] 0 2024-02-15T18:27:03.905 INFO:tasks.workunit.client.1.smithi191.stdout:7/331: mknod d0/db/d26/d5f/c76 0 2024-02-15T18:27:03.905 INFO:tasks.workunit.client.1.smithi191.stdout:7/332: write d0/db/d65/f22 [5103533,116461] 0 2024-02-15T18:27:03.905 INFO:tasks.workunit.client.1.smithi191.stdout:8/410: dread d3/d2c/f43 [0,4194304] 0 2024-02-15T18:27:03.905 INFO:tasks.workunit.client.0.smithi042.stdout:7/727: rmdir d10/d49 39 2024-02-15T18:27:03.906 INFO:tasks.workunit.client.1.smithi191.stdout:7/333: write d0/db/d65/d11/d5a/d66/f32 [1301533,69233] 0 2024-02-15T18:27:03.906 INFO:tasks.workunit.client.1.smithi191.stdout:3/267: symlink d6/d27/d4d/l63 0 2024-02-15T18:27:03.906 INFO:tasks.workunit.client.0.smithi042.stdout:3/700: symlink d1/d11/d14/d4d/le3 0 2024-02-15T18:27:03.907 INFO:tasks.workunit.client.0.smithi042.stdout:9/709: dwrite d6/f17 [0,4194304] 0 2024-02-15T18:27:03.907 INFO:tasks.workunit.client.0.smithi042.stdout:1/707: dread de/d15/d1b/f36 [0,4194304] 0 2024-02-15T18:27:03.907 INFO:tasks.workunit.client.0.smithi042.stdout:1/708: write de/d15/d67/d63/f8f [590496,122778] 0 2024-02-15T18:27:03.907 INFO:tasks.workunit.client.1.smithi191.stdout:0/199: symlink d7/d3b/l3f 0 2024-02-15T18:27:03.908 INFO:tasks.workunit.client.1.smithi191.stdout:0/200: fsync d7/da/f16 0 2024-02-15T18:27:03.908 INFO:tasks.workunit.client.1.smithi191.stdout:0/201: chown d7/da/d17/c1d 28156 1 2024-02-15T18:27:03.908 INFO:tasks.workunit.client.0.smithi042.stdout:2/651: creat d0/d8/d20/d7d/d94/fca x:0 0 0 2024-02-15T18:27:03.909 INFO:tasks.workunit.client.1.smithi191.stdout:1/419: creat d0/d2f/d51/f87 x:0 0 0 2024-02-15T18:27:03.909 INFO:tasks.workunit.client.1.smithi191.stdout:1/420: write d0/d8/df/d2e/f3d [843071,48663] 0 2024-02-15T18:27:03.910 INFO:tasks.workunit.client.1.smithi191.stdout:2/293: dread df/f1b [4194304,4194304] 0 2024-02-15T18:27:03.911 INFO:tasks.workunit.client.0.smithi042.stdout:4/708: mknod da/cf6 0 2024-02-15T18:27:03.912 INFO:tasks.workunit.client.0.smithi042.stdout:8/637: mkdir dd/de/da4/d67/d5a/db2 0 2024-02-15T18:27:03.913 INFO:tasks.workunit.client.1.smithi191.stdout:6/296: mknod d8/d9/d4a/c50 0 2024-02-15T18:27:03.916 INFO:tasks.workunit.client.0.smithi042.stdout:6/782: dwrite d9/fb2 [0,4194304] 0 2024-02-15T18:27:03.916 INFO:tasks.workunit.client.0.smithi042.stdout:6/783: dread - d9/d11/d46/df8/f101 zero size 2024-02-15T18:27:03.916 INFO:tasks.workunit.client.0.smithi042.stdout:6/784: truncate d9/d11/ff1 609303 0 2024-02-15T18:27:03.916 INFO:tasks.workunit.client.0.smithi042.stdout:5/740: dwrite d0/d38/d53/d64/f93 [0,4194304] 0 2024-02-15T18:27:03.916 INFO:tasks.workunit.client.0.smithi042.stdout:0/677: unlink d4/d9/c43 0 2024-02-15T18:27:03.916 INFO:tasks.workunit.client.0.smithi042.stdout:0/678: chown d4/d38/d5c/cd9 4693 1 2024-02-15T18:27:03.920 INFO:tasks.workunit.client.0.smithi042.stdout:7/728: link d10/d12/d23/d51/l81 d10/d15/d39/de3/lee 0 2024-02-15T18:27:03.920 INFO:tasks.workunit.client.0.smithi042.stdout:7/729: fdatasync d10/d49/d6d/d78/d83/fe8 0 2024-02-15T18:27:03.928 INFO:tasks.workunit.client.0.smithi042.stdout:3/701: rmdir d1/d11/d2e/dd1 39 2024-02-15T18:27:03.928 INFO:tasks.workunit.client.0.smithi042.stdout:3/702: write d1/de/d19/fc9 [385887,129851] 0 2024-02-15T18:27:03.931 INFO:tasks.workunit.client.0.smithi042.stdout:1/709: creat de/d54/dbf/dd0/d7c/dd6/ddf/dca/f10a x:0 0 0 2024-02-15T18:27:03.931 INFO:tasks.workunit.client.0.smithi042.stdout:1/710: chown de/d6d/d9e/ca4 465 1 2024-02-15T18:27:03.933 INFO:tasks.workunit.client.1.smithi191.stdout:8/411: mkdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95 0 2024-02-15T18:27:03.934 INFO:tasks.workunit.client.0.smithi042.stdout:6/785: symlink d9/d11/d19/l114 0 2024-02-15T18:27:03.936 INFO:tasks.workunit.client.1.smithi191.stdout:7/334: rename d0/d1 to d0/db/d26/d5f/d77 0 2024-02-15T18:27:03.937 INFO:tasks.workunit.client.0.smithi042.stdout:5/741: mkdir d0/d12/d2a/d102 0 2024-02-15T18:27:03.938 INFO:tasks.workunit.client.1.smithi191.stdout:3/268: mknod d6/d3b/c64 0 2024-02-15T18:27:03.938 INFO:tasks.workunit.client.0.smithi042.stdout:0/679: mknod d4/d9/d34/dab/db0/ce9 0 2024-02-15T18:27:03.939 INFO:tasks.workunit.client.1.smithi191.stdout:0/202: creat d7/da/d17/f40 x:0 0 0 2024-02-15T18:27:03.940 INFO:tasks.workunit.client.1.smithi191.stdout:1/421: mknod d0/d2f/c88 0 2024-02-15T18:27:03.940 INFO:tasks.workunit.client.1.smithi191.stdout:2/294: mknod df/d12/d19/d1a/d35/d47/c63 0 2024-02-15T18:27:03.941 INFO:tasks.workunit.client.0.smithi042.stdout:3/703: getdents d1/d11/d14/d4d/d73/d89/dd9 0 2024-02-15T18:27:03.943 INFO:tasks.workunit.client.0.smithi042.stdout:1/711: rename de/d15/f31 to de/d54/dbf/dd0/d7c/dd6/de8/dec/d100/f10b 0 2024-02-15T18:27:03.943 INFO:tasks.workunit.client.0.smithi042.stdout:9/710: dwrite d6/da/d1d/f38 [0,4194304] 0 2024-02-15T18:27:03.944 INFO:tasks.workunit.client.0.smithi042.stdout:9/711: write f2 [1209571,117890] 0 2024-02-15T18:27:03.945 INFO:tasks.workunit.client.0.smithi042.stdout:2/652: dwrite d0/d8/db/d44/f69 [0,4194304] 0 2024-02-15T18:27:03.947 INFO:tasks.workunit.client.1.smithi191.stdout:8/412: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/f96 x:0 0 0 2024-02-15T18:27:03.948 INFO:tasks.workunit.client.0.smithi042.stdout:8/638: link dd/d11/f21 dd/de/d33/d3b/d92/fb3 0 2024-02-15T18:27:03.952 INFO:tasks.workunit.client.1.smithi191.stdout:7/335: creat d0/db/d26/d5f/d77/d35/d42/f78 x:0 0 0 2024-02-15T18:27:03.952 INFO:tasks.workunit.client.1.smithi191.stdout:7/336: fsync d0/db/d26/d5f/d77/df/f1a 0 2024-02-15T18:27:03.954 INFO:tasks.workunit.client.0.smithi042.stdout:6/786: dread d9/f21 [4194304,4194304] 0 2024-02-15T18:27:03.954 INFO:tasks.workunit.client.0.smithi042.stdout:4/709: dwrite da/f25 [4194304,4194304] 0 2024-02-15T18:27:03.956 INFO:tasks.workunit.client.0.smithi042.stdout:4/710: dread da/f9c [0,4194304] 0 2024-02-15T18:27:03.960 INFO:tasks.workunit.client.0.smithi042.stdout:0/680: truncate d4/d9/d15/f8c 2047849 0 2024-02-15T18:27:03.962 INFO:tasks.workunit.client.1.smithi191.stdout:3/269: unlink d6/d3b/d42/d49/f43 0 2024-02-15T18:27:03.962 INFO:tasks.workunit.client.1.smithi191.stdout:3/270: chown d6/d3b/d42/d49/f3d 473037471 1 2024-02-15T18:27:03.963 INFO:tasks.workunit.client.1.smithi191.stdout:0/203: creat d7/da/d17/d31/f41 x:0 0 0 2024-02-15T18:27:03.964 INFO:tasks.workunit.client.0.smithi042.stdout:1/712: creat de/d15/d67/d74/d7e/f10c x:0 0 0 2024-02-15T18:27:03.965 INFO:tasks.workunit.client.1.smithi191.stdout:1/422: link d0/d2f/d51/l43 d0/d8/df/l89 0 2024-02-15T18:27:03.966 INFO:tasks.workunit.client.1.smithi191.stdout:2/295: getdents df/d12/d19/d1a/d35 0 2024-02-15T18:27:03.968 INFO:tasks.workunit.client.0.smithi042.stdout:9/712: dwrite d6/f31 [0,4194304] 0 2024-02-15T18:27:03.970 INFO:tasks.workunit.client.1.smithi191.stdout:6/297: unlink d8/d19/f37 0 2024-02-15T18:27:03.970 INFO:tasks.workunit.client.0.smithi042.stdout:1/713: write de/d15/d1b/d44/f85 [1070307,32935] 0 2024-02-15T18:27:03.970 INFO:tasks.workunit.client.0.smithi042.stdout:8/639: dwrite dd/de/d12/f68 [0,4194304] 0 2024-02-15T18:27:03.970 INFO:tasks.workunit.client.0.smithi042.stdout:8/640: chown dd/d11/f39 1 1 2024-02-15T18:27:03.971 INFO:tasks.workunit.client.1.smithi191.stdout:7/337: truncate d0/db/d26/d5f/d77/d35/f50 4111133 0 2024-02-15T18:27:03.976 INFO:tasks.workunit.client.1.smithi191.stdout:3/271: truncate d6/d27/f3f 299669 0 2024-02-15T18:27:03.977 INFO:tasks.workunit.client.0.smithi042.stdout:0/681: dwrite d4/d9/d15/f83 [0,4194304] 0 2024-02-15T18:27:03.977 INFO:tasks.workunit.client.0.smithi042.stdout:0/682: stat d4/dc2/lbf 0 2024-02-15T18:27:03.977 INFO:tasks.workunit.client.0.smithi042.stdout:0/683: stat d4/d38/d77/d2e/dae 0 2024-02-15T18:27:03.977 INFO:tasks.workunit.client.0.smithi042.stdout:2/653: mknod d0/d8/d20/d7d/d7e/ccb 0 2024-02-15T18:27:03.980 INFO:tasks.workunit.client.1.smithi191.stdout:0/204: mkdir d7/d42 0 2024-02-15T18:27:03.986 INFO:tasks.workunit.client.1.smithi191.stdout:1/423: symlink d0/d8/df/d12/d33/d49/l8a 0 2024-02-15T18:27:03.986 INFO:tasks.workunit.client.1.smithi191.stdout:1/424: chown d0/d2f/d51/d54/d7b 3663 1 2024-02-15T18:27:03.986 INFO:tasks.workunit.client.1.smithi191.stdout:1/425: readlink d0/d2f/d51/lc 0 2024-02-15T18:27:03.986 INFO:tasks.workunit.client.1.smithi191.stdout:6/298: dread d8/f15 [0,4194304] 0 2024-02-15T18:27:03.987 INFO:tasks.workunit.client.1.smithi191.stdout:2/296: dwrite df/d12/d1f/d42/f43 [0,4194304] 0 2024-02-15T18:27:03.988 INFO:tasks.workunit.client.1.smithi191.stdout:8/413: link d3/d2c/d1a/d23/d2e/d47/c7f d3/c97 0 2024-02-15T18:27:03.990 INFO:tasks.workunit.client.1.smithi191.stdout:0/205: rename d7/da/d1a/d34/d27/f3a to d7/d1b/d30/f43 0 2024-02-15T18:27:03.991 INFO:tasks.workunit.client.1.smithi191.stdout:0/206: write d7/da/d17/d31/f41 [840344,90730] 0 2024-02-15T18:27:03.994 INFO:tasks.workunit.client.1.smithi191.stdout:1/426: creat d0/d2f/d51/d54/f8b x:0 0 0 2024-02-15T18:27:03.997 INFO:tasks.workunit.client.1.smithi191.stdout:4/376: sync 2024-02-15T18:27:03.997 INFO:tasks.workunit.client.1.smithi191.stdout:5/171: sync 2024-02-15T18:27:03.997 INFO:tasks.workunit.client.1.smithi191.stdout:9/328: sync 2024-02-15T18:27:03.997 INFO:tasks.workunit.client.1.smithi191.stdout:4/377: stat d2/f28 0 2024-02-15T18:27:03.997 INFO:tasks.workunit.client.1.smithi191.stdout:9/329: dread - d0/d1d/f79 zero size 2024-02-15T18:27:04.000 INFO:tasks.workunit.client.1.smithi191.stdout:2/297: symlink df/d12/d1f/d42/l64 0 2024-02-15T18:27:04.001 INFO:tasks.workunit.client.1.smithi191.stdout:8/414: chown d3/c10 0 1 2024-02-15T18:27:04.001 INFO:tasks.workunit.client.1.smithi191.stdout:3/272: dwrite d6/fa [0,4194304] 0 2024-02-15T18:27:04.001 INFO:tasks.workunit.client.1.smithi191.stdout:3/273: chown d6/d27/d4d/l63 2466 1 2024-02-15T18:27:04.003 INFO:tasks.workunit.client.1.smithi191.stdout:1/427: creat d0/d8/df/d12/d33/f8c x:0 0 0 2024-02-15T18:27:04.003 INFO:tasks.workunit.client.1.smithi191.stdout:1/428: write d0/d2f/d51/d10/f59 [5460018,130143] 0 2024-02-15T18:27:04.003 INFO:tasks.workunit.client.1.smithi191.stdout:3/274: read d6/d3b/ff [944090,123547] 0 2024-02-15T18:27:04.003 INFO:tasks.workunit.client.1.smithi191.stdout:1/429: truncate d0/d2f/d51/f53 578060 0 2024-02-15T18:27:04.006 INFO:tasks.workunit.client.1.smithi191.stdout:4/378: symlink d2/d5/d7/d90/d52/l91 0 2024-02-15T18:27:04.006 INFO:tasks.workunit.client.1.smithi191.stdout:9/330: getdents d0/d1d 0 2024-02-15T18:27:04.008 INFO:tasks.workunit.client.0.smithi042.stdout:6/787: mkdir d9/d11/d19/d1c/d31/d115 0 2024-02-15T18:27:04.009 INFO:tasks.workunit.client.0.smithi042.stdout:6/788: write d9/d11/d46/d89/df4/f100 [14935,111757] 0 2024-02-15T18:27:04.010 INFO:tasks.workunit.client.1.smithi191.stdout:2/298: unlink f9 0 2024-02-15T18:27:04.014 INFO:tasks.workunit.client.0.smithi042.stdout:2/654: dwrite d0/d8/d10/d31/d3a/d53/f5d [4194304,4194304] 0 2024-02-15T18:27:04.014 INFO:tasks.workunit.client.0.smithi042.stdout:4/711: creat da/d11/d67/d5c/d66/d78/d2f/dca/ff7 x:0 0 0 2024-02-15T18:27:04.015 INFO:tasks.workunit.client.1.smithi191.stdout:0/207: dwrite d7/da/d17/d31/f36 [0,4194304] 0 2024-02-15T18:27:04.015 INFO:tasks.workunit.client.1.smithi191.stdout:0/208: chown d7/da/d1a/d34/l19 330590637 1 2024-02-15T18:27:04.015 INFO:tasks.workunit.client.1.smithi191.stdout:8/415: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/f98 x:0 0 0 2024-02-15T18:27:04.024 INFO:tasks.workunit.client.1.smithi191.stdout:7/338: dread d0/db/d26/d5f/d77/d35/f50 [0,4194304] 0 2024-02-15T18:27:04.030 INFO:tasks.workunit.client.0.smithi042.stdout:9/713: mkdir d6/da/d2f/d55/d73/d8d/dcf/de1/df7 0 2024-02-15T18:27:04.031 INFO:tasks.workunit.client.0.smithi042.stdout:1/714: rmdir de/d15/d67/d74 39 2024-02-15T18:27:04.034 INFO:tasks.workunit.client.1.smithi191.stdout:2/299: link df/c5f df/d12/d19/d1a/d35/c65 0 2024-02-15T18:27:04.034 INFO:tasks.workunit.client.1.smithi191.stdout:3/275: dwrite d6/d1b/f39 [0,4194304] 0 2024-02-15T18:27:04.034 INFO:tasks.workunit.client.1.smithi191.stdout:9/331: dwrite d0/d29/f70 [0,4194304] 0 2024-02-15T18:27:04.038 INFO:tasks.workunit.client.1.smithi191.stdout:4/379: dwrite d2/d5/d7/f3b [0,4194304] 0 2024-02-15T18:27:04.039 INFO:tasks.workunit.client.0.smithi042.stdout:8/641: mkdir dd/de/da4/db4 0 2024-02-15T18:27:04.039 INFO:tasks.workunit.client.0.smithi042.stdout:8/642: read - dd/d11/d75/fa1 zero size 2024-02-15T18:27:04.039 INFO:tasks.workunit.client.0.smithi042.stdout:8/643: fdatasync dd/f17 0 2024-02-15T18:27:04.042 INFO:tasks.workunit.client.0.smithi042.stdout:0/684: symlink d4/d9/d34/dab/dc8/lea 0 2024-02-15T18:27:04.043 INFO:tasks.workunit.client.0.smithi042.stdout:0/685: write d4/d9/d34/d57/faf [3466392,24380] 0 2024-02-15T18:27:04.043 INFO:tasks.workunit.client.0.smithi042.stdout:0/686: chown d4/d9/d34/dab/dc8/d6c/d70/c8a 397092840 1 2024-02-15T18:27:04.044 INFO:tasks.workunit.client.0.smithi042.stdout:6/789: creat d9/d11/d19/d1c/d31/d53/f116 x:0 0 0 2024-02-15T18:27:04.044 INFO:tasks.workunit.client.1.smithi191.stdout:1/430: dwrite d0/f5 [0,4194304] 0 2024-02-15T18:27:04.044 INFO:tasks.workunit.client.1.smithi191.stdout:0/209: symlink d7/da/d1a/l44 0 2024-02-15T18:27:04.046 INFO:tasks.workunit.client.1.smithi191.stdout:1/431: write d0/d2f/d51/d10/f59 [3718165,105193] 0 2024-02-15T18:27:04.046 INFO:tasks.workunit.client.1.smithi191.stdout:7/339: rename d0/db/d26/d5f/d77/d35/d42/f78 to d0/db/d65/d11/d5a/d66/f79 0 2024-02-15T18:27:04.046 INFO:tasks.workunit.client.1.smithi191.stdout:7/340: dread - d0/db/d65/f75 zero size 2024-02-15T18:27:04.048 INFO:tasks.workunit.client.1.smithi191.stdout:2/300: getdents df/d12/d1f/d42 0 2024-02-15T18:27:04.048 INFO:tasks.workunit.client.1.smithi191.stdout:2/301: chown df/f40 75998944 1 2024-02-15T18:27:04.048 INFO:tasks.workunit.client.1.smithi191.stdout:2/302: stat df/d12/d1e/c46 0 2024-02-15T18:27:04.049 INFO:tasks.workunit.client.1.smithi191.stdout:8/416: dwrite d3/d5e/f73 [0,4194304] 0 2024-02-15T18:27:04.053 INFO:tasks.workunit.client.1.smithi191.stdout:3/276: unlink d6/d27/c48 0 2024-02-15T18:27:04.053 INFO:tasks.workunit.client.1.smithi191.stdout:3/277: chown d6/d3b/d42/l33 3101911 1 2024-02-15T18:27:04.054 INFO:tasks.workunit.client.1.smithi191.stdout:3/278: chown d6 15 1 2024-02-15T18:27:04.055 INFO:tasks.workunit.client.0.smithi042.stdout:2/655: creat d0/d8/d10/d31/d3a/d53/fcc x:0 0 0 2024-02-15T18:27:04.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:02.682353+0000 mgr.smithi042.kfoihb (mgr.24435) 40 : cluster [DBG] pgmap v30: 65 pgs: 65 active+clean; 2.1 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 124 MiB/s rd, 278 MiB/s wr, 609 op/s 2024-02-15T18:27:04.140 INFO:tasks.workunit.client.0.smithi042.stdout:4/712: truncate da/d11/d67/d5c/d66/d78/d2f/d4d/f48 2216183 0 2024-02-15T18:27:04.142 INFO:tasks.workunit.client.0.smithi042.stdout:7/730: sync 2024-02-15T18:27:04.142 INFO:tasks.workunit.client.0.smithi042.stdout:5/742: sync 2024-02-15T18:27:04.142 INFO:tasks.workunit.client.0.smithi042.stdout:7/731: chown d10/f5b 30 1 2024-02-15T18:27:04.142 INFO:tasks.workunit.client.0.smithi042.stdout:5/743: chown d0/dda/d42/f95 54693 1 2024-02-15T18:27:04.142 INFO:tasks.workunit.client.0.smithi042.stdout:7/732: write d10/d43/d57/fb8 [11137,93042] 0 2024-02-15T18:27:04.143 INFO:tasks.workunit.client.0.smithi042.stdout:1/715: getdents de/d54/dbf/dd0/d7c/dd6/ddf/d88 0 2024-02-15T18:27:04.143 INFO:tasks.workunit.client.0.smithi042.stdout:1/716: stat de/d15/d2c/d51/c5c 0 2024-02-15T18:27:04.144 INFO:tasks.workunit.client.0.smithi042.stdout:1/717: dread de/d15/d67/d74/f75 [0,4194304] 0 2024-02-15T18:27:04.144 INFO:tasks.workunit.client.1.smithi191.stdout:9/332: symlink d0/d29/l7a 0 2024-02-15T18:27:04.145 INFO:tasks.workunit.client.1.smithi191.stdout:9/333: write d0/d7/f19 [1064062,59697] 0 2024-02-15T18:27:04.145 INFO:tasks.workunit.client.1.smithi191.stdout:9/334: getdents d0/d7/dc/d30/d3b/d56/d60 0 2024-02-15T18:27:04.145 INFO:tasks.workunit.client.1.smithi191.stdout:9/335: write d0/d7/dc/d17/f2d [627951,106443] 0 2024-02-15T18:27:04.146 INFO:tasks.workunit.client.1.smithi191.stdout:9/336: fdatasync d0/f1 0 2024-02-15T18:27:04.147 INFO:tasks.workunit.client.0.smithi042.stdout:8/644: creat dd/de/da4/d67/d6f/d78/d9a/fb5 x:0 0 0 2024-02-15T18:27:04.147 INFO:tasks.workunit.client.0.smithi042.stdout:8/645: fdatasync dd/f99 0 2024-02-15T18:27:04.147 INFO:tasks.workunit.client.0.smithi042.stdout:3/704: sync 2024-02-15T18:27:04.152 INFO:tasks.workunit.client.0.smithi042.stdout:0/687: creat d4/d9/feb x:0 0 0 2024-02-15T18:27:04.152 INFO:tasks.workunit.client.0.smithi042.stdout:0/688: truncate d4/d38/d77/fb4 1004006 0 2024-02-15T18:27:04.152 INFO:tasks.workunit.client.0.smithi042.stdout:0/689: chown d4/d38/d77/d2e/c59 523450 1 2024-02-15T18:27:04.152 INFO:tasks.workunit.client.1.smithi191.stdout:4/380: truncate d2/d5/d7/f67 2500312 0 2024-02-15T18:27:04.156 INFO:tasks.workunit.client.0.smithi042.stdout:2/656: link d0/d35/f81 d0/d8/d10/d31/d3a/d53/fcd 0 2024-02-15T18:27:04.157 INFO:tasks.workunit.client.1.smithi191.stdout:0/210: mkdir d7/da/d1a/d34/d45 0 2024-02-15T18:27:04.158 INFO:tasks.workunit.client.1.smithi191.stdout:9/337: dread d0/d7/da/d39/f5d [0,4194304] 0 2024-02-15T18:27:04.158 INFO:tasks.workunit.client.1.smithi191.stdout:9/338: write d0/d7/dc/f15 [5028928,89454] 0 2024-02-15T18:27:04.158 INFO:tasks.workunit.client.1.smithi191.stdout:0/211: write d7/da/d1a/d34/f15 [1406364,58780] 0 2024-02-15T18:27:04.161 INFO:tasks.workunit.client.1.smithi191.stdout:1/432: creat d0/d8/df/d2e/f8d x:0 0 0 2024-02-15T18:27:04.165 INFO:tasks.workunit.client.0.smithi042.stdout:5/744: symlink d0/d12/d65/l103 0 2024-02-15T18:27:04.166 INFO:tasks.workunit.client.1.smithi191.stdout:7/341: mknod d0/db/d65/dc/d4b/c7a 0 2024-02-15T18:27:04.167 INFO:tasks.workunit.client.0.smithi042.stdout:7/733: unlink d10/d15/d8f/da5/fb7 0 2024-02-15T18:27:04.169 INFO:tasks.workunit.client.1.smithi191.stdout:2/303: creat df/d12/d19/d1a/d35/d5b/f66 x:0 0 0 2024-02-15T18:27:04.169 INFO:tasks.workunit.client.1.smithi191.stdout:2/304: write df/d12/d19/d1a/f5a [765168,118644] 0 2024-02-15T18:27:04.170 INFO:tasks.workunit.client.0.smithi042.stdout:1/718: mkdir de/d15/d67/d89/d10d 0 2024-02-15T18:27:04.172 INFO:tasks.workunit.client.0.smithi042.stdout:8/646: creat dd/de/da4/d81/dae/fb6 x:0 0 0 2024-02-15T18:27:04.172 INFO:tasks.workunit.client.0.smithi042.stdout:8/647: write dd/de/da4/d67/d5a/f8f [1639797,21217] 0 2024-02-15T18:27:04.173 INFO:tasks.workunit.client.1.smithi191.stdout:8/417: creat d3/d2c/d1a/d23/d3e/f99 x:0 0 0 2024-02-15T18:27:04.174 INFO:tasks.workunit.client.0.smithi042.stdout:1/719: write de/d15/d95/f99 [1610089,63539] 0 2024-02-15T18:27:04.175 INFO:tasks.workunit.client.1.smithi191.stdout:8/418: write d3/d2c/d1a/d23/d3e/f68 [2288512,98358] 0 2024-02-15T18:27:04.175 INFO:tasks.workunit.client.1.smithi191.stdout:8/419: fsync d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d8e/f90 0 2024-02-15T18:27:04.175 INFO:tasks.workunit.client.1.smithi191.stdout:8/420: read - d3/d2c/d1a/d23/d3e/f99 zero size 2024-02-15T18:27:04.176 INFO:tasks.workunit.client.0.smithi042.stdout:3/705: rmdir d1 39 2024-02-15T18:27:04.177 INFO:tasks.workunit.client.1.smithi191.stdout:3/279: rename d6/d3b/d42/f55 to d6/f65 0 2024-02-15T18:27:04.178 INFO:tasks.workunit.client.0.smithi042.stdout:0/690: mkdir d4/d9/d34/dec 0 2024-02-15T18:27:04.178 INFO:tasks.workunit.client.1.smithi191.stdout:9/339: dwrite d0/d7/dc/f15 [4194304,4194304] 0 2024-02-15T18:27:04.181 INFO:tasks.workunit.client.1.smithi191.stdout:9/340: read d0/f1 [1775716,131061] 0 2024-02-15T18:27:04.181 INFO:tasks.workunit.client.1.smithi191.stdout:9/341: chown d0/d7/da/f27 1154045 1 2024-02-15T18:27:04.181 INFO:tasks.workunit.client.1.smithi191.stdout:9/342: chown d0/d7/dc/f15 7 1 2024-02-15T18:27:04.181 INFO:tasks.workunit.client.0.smithi042.stdout:2/657: creat d0/d8/d20/d41/dbe/fce x:0 0 0 2024-02-15T18:27:04.181 INFO:tasks.workunit.client.0.smithi042.stdout:2/658: fsync d0/d8/d10/d31/d79/f8e 0 2024-02-15T18:27:04.181 INFO:tasks.workunit.client.0.smithi042.stdout:9/714: sync 2024-02-15T18:27:04.181 INFO:tasks.workunit.client.0.smithi042.stdout:6/790: sync 2024-02-15T18:27:04.183 INFO:tasks.workunit.client.1.smithi191.stdout:0/212: creat d7/d1b/f46 x:0 0 0 2024-02-15T18:27:04.188 INFO:tasks.workunit.client.0.smithi042.stdout:4/713: link da/d11/d67/d5c/d66/d78/d2f/d4d/f48 da/d11/d85/ff8 0 2024-02-15T18:27:04.189 INFO:tasks.workunit.client.0.smithi042.stdout:5/745: creat d0/d12/f104 x:0 0 0 2024-02-15T18:27:04.193 INFO:tasks.workunit.client.1.smithi191.stdout:0/213: write d7/fb [5867810,95900] 0 2024-02-15T18:27:04.194 INFO:tasks.workunit.client.0.smithi042.stdout:7/734: creat d10/d15/d8f/da5/dad/de0/de2/fef x:0 0 0 2024-02-15T18:27:04.194 INFO:tasks.workunit.client.1.smithi191.stdout:1/433: creat d0/d8/df/d12/d33/f8e x:0 0 0 2024-02-15T18:27:04.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:27:02.682353+0000 mgr.smithi042.kfoihb (mgr.24435) 40 : cluster [DBG] pgmap v30: 65 pgs: 65 active+clean; 2.1 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 124 MiB/s rd, 278 MiB/s wr, 609 op/s 2024-02-15T18:27:04.197 INFO:tasks.workunit.client.1.smithi191.stdout:7/342: link d0/db/d65/f75 d0/db/d26/d5f/d77/d35/d42/f7b 0 2024-02-15T18:27:04.198 INFO:tasks.workunit.client.1.smithi191.stdout:0/214: dread d7/d1b/f23 [0,4194304] 0 2024-02-15T18:27:04.199 INFO:tasks.workunit.client.0.smithi042.stdout:0/691: dwrite d4/d9/d34/dab/dc8/f87 [8388608,4194304] 0 2024-02-15T18:27:04.199 INFO:tasks.workunit.client.0.smithi042.stdout:1/720: mknod de/d54/de6/c10e 0 2024-02-15T18:27:04.201 INFO:tasks.workunit.client.1.smithi191.stdout:2/305: link df/d12/d1f/d42/f49 df/d61/f67 0 2024-02-15T18:27:04.202 INFO:tasks.workunit.client.1.smithi191.stdout:1/434: write d0/f3 [502032,12484] 0 2024-02-15T18:27:04.202 INFO:tasks.workunit.client.1.smithi191.stdout:1/435: readlink d0/d2f/l6b 0 2024-02-15T18:27:04.203 INFO:tasks.workunit.client.0.smithi042.stdout:3/706: creat d1/de/d1a/fe4 x:0 0 0 2024-02-15T18:27:04.206 INFO:tasks.workunit.client.1.smithi191.stdout:3/280: rename d6/d3b/d56 to d6/d27/d4d/d66 0 2024-02-15T18:27:04.207 INFO:tasks.workunit.client.0.smithi042.stdout:2/659: rename d0/d8/d20/d7d/d94/f60 to d0/d8/db/de/d1a/d2e/fcf 0 2024-02-15T18:27:04.209 INFO:tasks.workunit.client.0.smithi042.stdout:8/648: dwrite dd/de/d12/f1e [0,4194304] 0 2024-02-15T18:27:04.209 INFO:tasks.workunit.client.0.smithi042.stdout:8/649: write dd/f13 [1218371,71668] 0 2024-02-15T18:27:04.209 INFO:tasks.workunit.client.0.smithi042.stdout:5/746: mkdir d0/d12/d2a/d105 0 2024-02-15T18:27:04.209 INFO:tasks.workunit.client.0.smithi042.stdout:5/747: write d0/dda/d42/fb9 [800792,36826] 0 2024-02-15T18:27:04.209 INFO:tasks.workunit.client.0.smithi042.stdout:9/715: dwrite d6/da/d75/ff2 [0,4194304] 0 2024-02-15T18:27:04.211 INFO:tasks.workunit.client.0.smithi042.stdout:7/735: creat d10/d49/d6d/db6/ff0 x:0 0 0 2024-02-15T18:27:04.212 INFO:tasks.workunit.client.1.smithi191.stdout:8/421: dwrite d3/d2c/f75 [0,4194304] 0 2024-02-15T18:27:04.213 INFO:tasks.workunit.client.0.smithi042.stdout:1/721: mkdir de/d54/dbf/dd0/d7c/dd6/de8/d103/d10f 0 2024-02-15T18:27:04.214 INFO:tasks.workunit.client.1.smithi191.stdout:0/215: getdents d7/da/d1a/d34/d27 0 2024-02-15T18:27:04.214 INFO:tasks.workunit.client.1.smithi191.stdout:9/343: mknod d0/d7/dc/d30/d3b/d56/c7b 0 2024-02-15T18:27:04.215 INFO:tasks.workunit.client.0.smithi042.stdout:3/707: link d1/d11/d14/d4d/d73/f9d d1/d11/d2e/dd1/fe5 0 2024-02-15T18:27:04.219 INFO:tasks.workunit.client.0.smithi042.stdout:6/791: rename d9/d11/d46/df8/f101 to d9/d11/d19/d1c/d31/d9b/db4/f117 0 2024-02-15T18:27:04.219 INFO:tasks.workunit.client.0.smithi042.stdout:6/792: dread d9/d11/d19/d1c/fdb [0,4194304] 0 2024-02-15T18:27:04.220 INFO:tasks.workunit.client.1.smithi191.stdout:2/306: symlink df/d12/d19/d1a/d35/d47/l68 0 2024-02-15T18:27:04.220 INFO:tasks.workunit.client.1.smithi191.stdout:2/307: fsync df/d12/d1f/f45 0 2024-02-15T18:27:04.220 INFO:tasks.workunit.client.1.smithi191.stdout:8/422: write d3/fa [5441903,130757] 0 2024-02-15T18:27:04.220 INFO:tasks.workunit.client.0.smithi042.stdout:6/793: dread d9/d11/d19/d35/dc4/fd5 [0,4194304] 0 2024-02-15T18:27:04.220 INFO:tasks.workunit.client.0.smithi042.stdout:6/794: write d9/d11/d97/fe5 [577307,57622] 0 2024-02-15T18:27:04.221 INFO:tasks.workunit.client.0.smithi042.stdout:4/714: dwrite da/d3b/fc2 [0,4194304] 0 2024-02-15T18:27:04.222 INFO:tasks.workunit.client.0.smithi042.stdout:0/692: dwrite d4/f7 [0,4194304] 0 2024-02-15T18:27:04.224 INFO:tasks.workunit.client.0.smithi042.stdout:2/660: mknod d0/d8/d10/d31/da6/cd0 0 2024-02-15T18:27:04.224 INFO:tasks.workunit.client.0.smithi042.stdout:2/661: read - d0/d8/d20/d41/f85 zero size 2024-02-15T18:27:04.227 INFO:tasks.workunit.client.0.smithi042.stdout:4/715: dread da/d11/d27/f69 [0,4194304] 0 2024-02-15T18:27:04.228 INFO:tasks.workunit.client.1.smithi191.stdout:7/343: dwrite d0/db/f2d [8388608,4194304] 0 2024-02-15T18:27:04.229 INFO:tasks.workunit.client.1.smithi191.stdout:2/308: dread df/d12/d19/d1a/f21 [0,4194304] 0 2024-02-15T18:27:04.229 INFO:tasks.workunit.client.1.smithi191.stdout:2/309: write df/d61/f67 [1338176,92534] 0 2024-02-15T18:27:04.229 INFO:tasks.workunit.client.1.smithi191.stdout:2/310: write df/d12/d19/d1a/f29 [748651,52067] 0 2024-02-15T18:27:04.229 INFO:tasks.workunit.client.1.smithi191.stdout:2/311: write df/d12/d19/d1a/f44 [428464,17728] 0 2024-02-15T18:27:04.230 INFO:tasks.workunit.client.0.smithi042.stdout:7/736: dwrite d10/d49/d6d/d78/d83/f87 [0,4194304] 0 2024-02-15T18:27:04.230 INFO:tasks.workunit.client.0.smithi042.stdout:7/737: readlink d10/d49/d6d/d6e/lca 0 2024-02-15T18:27:04.230 INFO:tasks.workunit.client.1.smithi191.stdout:2/312: write df/d12/d1f/f34 [3647087,101439] 0 2024-02-15T18:27:04.233 INFO:tasks.workunit.client.1.smithi191.stdout:1/436: unlink d0/d8/df/d12/d2a/d2b/d78/c7c 0 2024-02-15T18:27:04.235 INFO:tasks.workunit.client.0.smithi042.stdout:5/748: link d0/d38/d53/l6b d0/d39/d8d/dd2/l106 0 2024-02-15T18:27:04.235 INFO:tasks.workunit.client.0.smithi042.stdout:5/749: write d0/f23 [4791226,92147] 0 2024-02-15T18:27:04.236 INFO:tasks.workunit.client.1.smithi191.stdout:3/281: link d6/d27/l2b d6/d3b/d42/l67 0 2024-02-15T18:27:04.237 INFO:tasks.workunit.client.1.smithi191.stdout:3/282: write d6/fa [2162188,100601] 0 2024-02-15T18:27:04.246 INFO:tasks.workunit.client.1.smithi191.stdout:3/283: dread d6/d3b/d42/f31 [0,4194304] 0 2024-02-15T18:27:04.246 INFO:tasks.workunit.client.1.smithi191.stdout:3/284: fsync d6/f8 0 2024-02-15T18:27:04.247 INFO:tasks.workunit.client.1.smithi191.stdout:3/285: write d6/d3b/f13 [4248387,118556] 0 2024-02-15T18:27:04.251 INFO:tasks.workunit.client.1.smithi191.stdout:7/344: dwrite d0/db/d26/d5f/d77/df/f16 [0,4194304] 0 2024-02-15T18:27:04.251 INFO:tasks.workunit.client.1.smithi191.stdout:7/345: stat d0/db/d26/d5f/d77 0 2024-02-15T18:27:04.252 INFO:tasks.workunit.client.0.smithi042.stdout:9/716: truncate d6/da/f65 254510 0 2024-02-15T18:27:04.253 INFO:tasks.workunit.client.1.smithi191.stdout:7/346: read d0/db/d26/d5f/d77/df/f17 [2083697,114006] 0 2024-02-15T18:27:04.254 INFO:tasks.workunit.client.0.smithi042.stdout:3/708: creat d1/d11/d14/dbf/fe6 x:0 0 0 2024-02-15T18:27:04.255 INFO:tasks.workunit.client.0.smithi042.stdout:3/709: chown d1/fa5 50911573 1 2024-02-15T18:27:04.257 INFO:tasks.workunit.client.0.smithi042.stdout:1/722: rename de/d15/d95/fdc to de/d15/d1b/d44/f110 0 2024-02-15T18:27:04.257 INFO:tasks.workunit.client.0.smithi042.stdout:1/723: chown de/d15/d1b/d44/f110 239 1 2024-02-15T18:27:04.258 INFO:tasks.workunit.client.0.smithi042.stdout:1/724: write de/d15/d67/f34 [1644885,59932] 0 2024-02-15T18:27:04.265 INFO:tasks.workunit.client.1.smithi191.stdout:0/216: creat d7/da/d17/f47 x:0 0 0 2024-02-15T18:27:04.270 INFO:tasks.workunit.client.0.smithi042.stdout:6/795: creat d9/d11/d19/d1c/d31/dd2/df9/f118 x:0 0 0 2024-02-15T18:27:04.278 INFO:tasks.workunit.client.1.smithi191.stdout:8/423: mkdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a 0 2024-02-15T18:27:04.279 INFO:tasks.workunit.client.0.smithi042.stdout:2/662: symlink d0/d8/d20/d7d/d94/ld1 0 2024-02-15T18:27:04.279 INFO:tasks.workunit.client.1.smithi191.stdout:2/313: symlink df/d12/d1f/l69 0 2024-02-15T18:27:04.281 INFO:tasks.workunit.client.1.smithi191.stdout:1/437: rename d0/d8/df/d41/f6d to d0/d8/df/d12/f8f 0 2024-02-15T18:27:04.282 INFO:tasks.workunit.client.1.smithi191.stdout:3/286: getdents d6/d3b/d42 0 2024-02-15T18:27:04.286 INFO:tasks.workunit.client.1.smithi191.stdout:7/347: mknod d0/db/d65/d11/d5a/d66/d1e/c7c 0 2024-02-15T18:27:04.288 INFO:tasks.workunit.client.1.smithi191.stdout:8/424: truncate d3/f15 2371155 0 2024-02-15T18:27:04.289 INFO:tasks.workunit.client.0.smithi042.stdout:4/716: getdents da/d11/d85/db5 0 2024-02-15T18:27:04.290 INFO:tasks.workunit.client.1.smithi191.stdout:1/438: mkdir d0/d8/df/d90 0 2024-02-15T18:27:04.292 INFO:tasks.workunit.client.1.smithi191.stdout:8/425: mkdir d3/d2c/d1a/d36/d9b 0 2024-02-15T18:27:04.292 INFO:tasks.workunit.client.1.smithi191.stdout:2/314: dread df/d12/d19/d1a/d35/d47/f51 [0,4194304] 0 2024-02-15T18:27:04.295 INFO:tasks.workunit.client.1.smithi191.stdout:1/439: write d0/d2f/d51/d54/f55 [1915746,83534] 0 2024-02-15T18:27:04.300 INFO:tasks.workunit.client.0.smithi042.stdout:5/750: mkdir d0/d38/d53/d91/d94/d107 0 2024-02-15T18:27:04.301 INFO:tasks.workunit.client.0.smithi042.stdout:9/717: symlink d6/da/d75/lf8 0 2024-02-15T18:27:04.302 INFO:tasks.workunit.client.1.smithi191.stdout:8/426: unlink d3/d2c/d1a/d23/d2e/d47/f52 0 2024-02-15T18:27:04.302 INFO:tasks.workunit.client.1.smithi191.stdout:8/427: readlink d3/d2c/d1a/d23/d2e/l32 0 2024-02-15T18:27:04.302 INFO:tasks.workunit.client.1.smithi191.stdout:8/428: chown d3/d2c/d1a/d23/d2a/l87 57078464 1 2024-02-15T18:27:04.302 INFO:tasks.workunit.client.1.smithi191.stdout:0/217: dwrite d7/da/d17/f47 [0,4194304] 0 2024-02-15T18:27:04.303 INFO:tasks.workunit.client.1.smithi191.stdout:8/429: read d3/d2c/f48 [626596,26931] 0 2024-02-15T18:27:04.303 INFO:tasks.workunit.client.1.smithi191.stdout:7/348: dread d0/f3a [0,4194304] 0 2024-02-15T18:27:04.303 INFO:tasks.workunit.client.1.smithi191.stdout:1/440: dread d0/d8/df/d12/d2a/d2b/f5a [0,4194304] 0 2024-02-15T18:27:04.304 INFO:tasks.workunit.client.1.smithi191.stdout:3/287: dwrite d6/d27/d4d/d66/f61 [0,4194304] 0 2024-02-15T18:27:04.304 INFO:tasks.workunit.client.1.smithi191.stdout:3/288: chown d6/d3b/d42/f51 2 1 2024-02-15T18:27:04.304 INFO:tasks.workunit.client.0.smithi042.stdout:3/710: truncate d1/d11/d2e/d4c/f5b 568431 0 2024-02-15T18:27:04.304 INFO:tasks.workunit.client.0.smithi042.stdout:3/711: fdatasync d1/d11/f65 0 2024-02-15T18:27:04.309 INFO:tasks.workunit.client.1.smithi191.stdout:0/218: symlink d7/d42/l48 0 2024-02-15T18:27:04.310 INFO:tasks.workunit.client.1.smithi191.stdout:0/219: write d7/da/d1a/d34/f15 [3112860,11611] 0 2024-02-15T18:27:04.310 INFO:tasks.workunit.client.1.smithi191.stdout:0/220: write d7/da/d17/d31/f41 [1933316,68911] 0 2024-02-15T18:27:04.310 INFO:tasks.workunit.client.1.smithi191.stdout:0/221: truncate d7/da/f25 927490 0 2024-02-15T18:27:04.311 INFO:tasks.workunit.client.1.smithi191.stdout:7/349: dread d0/f8 [0,4194304] 0 2024-02-15T18:27:04.316 INFO:tasks.workunit.client.0.smithi042.stdout:1/725: mkdir de/d54/dbf/dd0/d7c/dd6/ddf/d88/d111 0 2024-02-15T18:27:04.317 INFO:tasks.workunit.client.1.smithi191.stdout:8/430: rename d3/d2c/d1a/d23/f54 to d3/d2c/d1a/d23/d2e/d5b/f9c 0 2024-02-15T18:27:04.319 INFO:tasks.workunit.client.1.smithi191.stdout:1/441: link d0/d8/df/d12/d2a/f3a d0/d8/df/d12/d2a/d2b/f91 0 2024-02-15T18:27:04.320 INFO:tasks.workunit.client.0.smithi042.stdout:6/796: creat d9/d11/d97/df7/f119 x:0 0 0 2024-02-15T18:27:04.322 INFO:tasks.workunit.client.0.smithi042.stdout:2/663: truncate d0/d8/db/de/f4d 2872905 0 2024-02-15T18:27:04.323 INFO:tasks.workunit.client.1.smithi191.stdout:8/431: link d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/f98 d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/f9d 0 2024-02-15T18:27:04.323 INFO:tasks.workunit.client.1.smithi191.stdout:8/432: dread - d3/d2c/d1a/d23/d2a/f6e zero size 2024-02-15T18:27:04.323 INFO:tasks.workunit.client.1.smithi191.stdout:8/433: dread - d3/d2c/d1a/d23/d2e/d47/f63 zero size 2024-02-15T18:27:04.324 INFO:tasks.workunit.client.0.smithi042.stdout:4/717: mknod da/d11/d67/cf9 0 2024-02-15T18:27:04.324 INFO:tasks.workunit.client.1.smithi191.stdout:0/222: dread d7/da/d17/f47 [0,4194304] 0 2024-02-15T18:27:04.325 INFO:tasks.workunit.client.1.smithi191.stdout:8/434: write d3/d2c/f17 [2510850,83673] 0 2024-02-15T18:27:04.325 INFO:tasks.workunit.client.1.smithi191.stdout:8/435: chown d3/d2c/d1a/d23/d2e/d47/d5f/d4f 42022 1 2024-02-15T18:27:04.326 INFO:tasks.workunit.client.0.smithi042.stdout:5/751: rename d0/dda/dc7/d52/fa7 to d0/d39/f108 0 2024-02-15T18:27:04.327 INFO:tasks.workunit.client.1.smithi191.stdout:2/315: dwrite fb [4194304,4194304] 0 2024-02-15T18:27:04.327 INFO:tasks.workunit.client.0.smithi042.stdout:9/718: creat d6/da/d1d/d2d/d40/d5f/ff9 x:0 0 0 2024-02-15T18:27:04.331 INFO:tasks.workunit.client.0.smithi042.stdout:0/693: dwrite d4/d9/d34/d8b/f90 [0,4194304] 0 2024-02-15T18:27:04.333 INFO:tasks.workunit.client.1.smithi191.stdout:0/223: dread d7/f39 [0,4194304] 0 2024-02-15T18:27:04.333 INFO:tasks.workunit.client.1.smithi191.stdout:0/224: fdatasync d7/d1b/f23 0 2024-02-15T18:27:04.333 INFO:tasks.workunit.client.1.smithi191.stdout:1/442: read d0/d2f/d51/d54/f55 [1328718,39070] 0 2024-02-15T18:27:04.334 INFO:tasks.workunit.client.0.smithi042.stdout:3/712: mkdir d1/d11/d2e/de7 0 2024-02-15T18:27:04.334 INFO:tasks.workunit.client.0.smithi042.stdout:3/713: getdents d1/de/d19 0 2024-02-15T18:27:04.337 INFO:tasks.workunit.client.1.smithi191.stdout:8/436: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/c9e 0 2024-02-15T18:27:04.337 INFO:tasks.workunit.client.0.smithi042.stdout:4/718: dread da/d11/d67/d5c/d66/d78/d2f/d4d/f96 [0,4194304] 0 2024-02-15T18:27:04.338 INFO:tasks.workunit.client.0.smithi042.stdout:4/719: write da/f9c [1466698,39432] 0 2024-02-15T18:27:04.338 INFO:tasks.workunit.client.0.smithi042.stdout:4/720: stat da/d2b/lb3 0 2024-02-15T18:27:04.343 INFO:tasks.workunit.client.0.smithi042.stdout:1/726: mknod de/d15/d2c/d51/d70/c112 0 2024-02-15T18:27:04.344 INFO:tasks.workunit.client.0.smithi042.stdout:8/650: sync 2024-02-15T18:27:04.344 INFO:tasks.workunit.client.0.smithi042.stdout:8/651: readlink dd/de/da5/la7 0 2024-02-15T18:27:04.344 INFO:tasks.workunit.client.0.smithi042.stdout:8/652: write dd/de/d12/f62 [364080,55735] 0 2024-02-15T18:27:04.344 INFO:tasks.workunit.client.0.smithi042.stdout:8/653: dread - dd/de/da4/d67/d5a/f64 zero size 2024-02-15T18:27:04.346 INFO:tasks.workunit.client.0.smithi042.stdout:6/797: mkdir d9/d11/d19/d1c/d11a 0 2024-02-15T18:27:04.347 INFO:tasks.workunit.client.0.smithi042.stdout:7/738: sync 2024-02-15T18:27:04.348 INFO:tasks.workunit.client.0.smithi042.stdout:2/664: rmdir d0/d8/db/de 39 2024-02-15T18:27:04.349 INFO:tasks.workunit.client.0.smithi042.stdout:2/665: write d0/d8/d10/d5a/f88 [639990,70708] 0 2024-02-15T18:27:04.350 INFO:tasks.workunit.client.1.smithi191.stdout:2/316: unlink df/d12/l1d 0 2024-02-15T18:27:04.351 INFO:tasks.workunit.client.1.smithi191.stdout:2/317: chown df/d12/d19/d1a/d35/c4f 27347371 1 2024-02-15T18:27:04.351 INFO:tasks.workunit.client.1.smithi191.stdout:2/318: chown df/d12/f2b 1833482444 1 2024-02-15T18:27:04.351 INFO:tasks.workunit.client.0.smithi042.stdout:5/752: getdents d0/dda 0 2024-02-15T18:27:04.352 INFO:tasks.workunit.client.1.smithi191.stdout:2/319: dread df/d12/d19/d1a/f2c [0,4194304] 0 2024-02-15T18:27:04.352 INFO:tasks.workunit.client.1.smithi191.stdout:2/320: dread - df/d12/d1f/f31 zero size 2024-02-15T18:27:04.353 INFO:tasks.workunit.client.0.smithi042.stdout:9/719: unlink d6/da/d2f/d6c/l7f 0 2024-02-15T18:27:04.358 INFO:tasks.workunit.client.0.smithi042.stdout:3/714: creat d1/de/fe8 x:0 0 0 2024-02-15T18:27:04.358 INFO:tasks.workunit.client.0.smithi042.stdout:3/715: write d1/de/d19/fdc [1411649,43600] 0 2024-02-15T18:27:04.361 INFO:tasks.workunit.client.1.smithi191.stdout:8/437: creat d3/d2c/d1a/d23/d2e/f9f x:0 0 0 2024-02-15T18:27:04.361 INFO:tasks.workunit.client.0.smithi042.stdout:4/721: rmdir da/d11/d67/d5c/d66/d78/d53/d82/dc0 39 2024-02-15T18:27:04.362 INFO:tasks.workunit.client.1.smithi191.stdout:2/321: mknod df/d12/d1f/c6a 0 2024-02-15T18:27:04.362 INFO:tasks.workunit.client.1.smithi191.stdout:2/322: fsync df/f10 0 2024-02-15T18:27:04.362 INFO:tasks.workunit.client.1.smithi191.stdout:2/323: stat df/d12/d1f/d42 0 2024-02-15T18:27:04.363 INFO:tasks.workunit.client.0.smithi042.stdout:1/727: creat de/d54/dbf/dd0/d7c/dd6/de8/d103/f113 x:0 0 0 2024-02-15T18:27:04.365 INFO:tasks.workunit.client.1.smithi191.stdout:8/438: mkdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/da0 0 2024-02-15T18:27:04.366 INFO:tasks.workunit.client.0.smithi042.stdout:0/694: dwrite d4/f41 [0,4194304] 0 2024-02-15T18:27:04.366 INFO:tasks.workunit.client.0.smithi042.stdout:0/695: truncate d4/d9/d34/dab/fde 112186 0 2024-02-15T18:27:04.366 INFO:tasks.workunit.client.0.smithi042.stdout:8/654: link dd/l31 dd/de/d33/d66/lb7 0 2024-02-15T18:27:04.367 INFO:tasks.workunit.client.1.smithi191.stdout:2/324: mknod df/d12/d1e/c6b 0 2024-02-15T18:27:04.368 INFO:tasks.workunit.client.1.smithi191.stdout:2/325: chown df/d12/d19/d1a/f3d 16095411 1 2024-02-15T18:27:04.368 INFO:tasks.workunit.client.1.smithi191.stdout:2/326: read df/d12/f36 [4657971,25734] 0 2024-02-15T18:27:04.368 INFO:tasks.workunit.client.1.smithi191.stdout:2/327: truncate df/d12/d1f/f31 668330 0 2024-02-15T18:27:04.369 INFO:tasks.workunit.client.0.smithi042.stdout:8/655: read dd/de/d3c/f70 [150490,76828] 0 2024-02-15T18:27:04.369 INFO:tasks.workunit.client.0.smithi042.stdout:8/656: write dd/de/da4/d67/d5a/f91 [5018850,44839] 0 2024-02-15T18:27:04.369 INFO:tasks.workunit.client.0.smithi042.stdout:0/696: dread d4/d38/d77/fe6 [0,4194304] 0 2024-02-15T18:27:04.370 INFO:tasks.workunit.client.1.smithi191.stdout:0/225: dwrite d7/da/d1a/d34/f29 [4194304,4194304] 0 2024-02-15T18:27:04.370 INFO:tasks.workunit.client.1.smithi191.stdout:0/226: truncate d7/da/f25 1169682 0 2024-02-15T18:27:04.370 INFO:tasks.workunit.client.0.smithi042.stdout:6/798: rename d9/d11/d19/d35/dc4/fd5 to d9/d11/d19/d35/d49/d10a/f11b 0 2024-02-15T18:27:04.371 INFO:tasks.workunit.client.1.smithi191.stdout:0/227: write d7/da/f16 [2492493,28344] 0 2024-02-15T18:27:04.371 INFO:tasks.workunit.client.0.smithi042.stdout:0/697: write d4/f6 [754233,118489] 0 2024-02-15T18:27:04.371 INFO:tasks.workunit.client.0.smithi042.stdout:1/728: dread de/d15/d95/fb9 [4194304,4194304] 0 2024-02-15T18:27:04.371 INFO:tasks.workunit.client.0.smithi042.stdout:1/729: truncate de/d15/d95/fa6 729375 0 2024-02-15T18:27:04.372 INFO:tasks.workunit.client.0.smithi042.stdout:1/730: write de/d15/d1b/d44/f106 [772443,103871] 0 2024-02-15T18:27:04.372 INFO:tasks.workunit.client.0.smithi042.stdout:1/731: write de/d15/d1b/d44/f110 [1010659,109273] 0 2024-02-15T18:27:04.372 INFO:tasks.workunit.client.0.smithi042.stdout:7/739: creat d10/d15/d8f/da5/dad/de0/ff1 x:0 0 0 2024-02-15T18:27:04.373 INFO:tasks.workunit.client.1.smithi191.stdout:1/443: dwrite d0/d2f/d51/d54/f8b [0,4194304] 0 2024-02-15T18:27:04.373 INFO:tasks.workunit.client.1.smithi191.stdout:1/444: dread - d0/d2f/d51/d5e/d68/f86 zero size 2024-02-15T18:27:04.373 INFO:tasks.workunit.client.1.smithi191.stdout:8/439: creat d3/d2c/d1a/d23/d2e/d5b/fa1 x:0 0 0 2024-02-15T18:27:04.374 INFO:tasks.workunit.client.0.smithi042.stdout:2/666: dwrite d0/d8/d20/d7d/d7e/dc7/f73 [0,4194304] 0 2024-02-15T18:27:04.377 INFO:tasks.workunit.client.0.smithi042.stdout:2/667: write d0/d8/d20/d7d/d7e/dc7/f8d [3922363,16490] 0 2024-02-15T18:27:04.379 INFO:tasks.workunit.client.1.smithi191.stdout:2/328: getdents df/d12/d19/d1a/d35/d47 0 2024-02-15T18:27:04.380 INFO:tasks.workunit.client.1.smithi191.stdout:2/329: write df/d12/d19/d1a/f29 [1773684,86426] 0 2024-02-15T18:27:04.380 INFO:tasks.workunit.client.0.smithi042.stdout:5/753: mkdir d0/d12/d2a/d105/d109 0 2024-02-15T18:27:04.381 INFO:tasks.workunit.client.1.smithi191.stdout:0/228: creat d7/da/d1a/d34/d45/f49 x:0 0 0 2024-02-15T18:27:04.390 INFO:tasks.workunit.client.0.smithi042.stdout:9/720: creat d6/da/d2f/d55/d73/d8d/dcf/de1/ffa x:0 0 0 2024-02-15T18:27:04.392 INFO:tasks.workunit.client.0.smithi042.stdout:3/716: mkdir d1/de/d1a/de9 0 2024-02-15T18:27:04.392 INFO:tasks.workunit.client.0.smithi042.stdout:3/717: stat d1/de 0 2024-02-15T18:27:04.392 INFO:tasks.workunit.client.0.smithi042.stdout:3/718: write d1/d11/d14/d4d/d6c/f83 [821510,49390] 0 2024-02-15T18:27:04.394 INFO:tasks.workunit.client.1.smithi191.stdout:1/445: mknod d0/d8/df/d12/d2a/d2b/d78/d60/c92 0 2024-02-15T18:27:04.394 INFO:tasks.workunit.client.1.smithi191.stdout:1/446: fdatasync d0/d8/df/d2e/f8d 0 2024-02-15T18:27:04.394 INFO:tasks.workunit.client.1.smithi191.stdout:8/440: mknod d3/d2c/d1a/ca2 0 2024-02-15T18:27:04.394 INFO:tasks.workunit.client.1.smithi191.stdout:8/441: chown d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/f60 18 1 2024-02-15T18:27:04.395 INFO:tasks.workunit.client.1.smithi191.stdout:2/330: creat df/d12/d19/d1a/d35/d47/d4b/f6c x:0 0 0 2024-02-15T18:27:04.397 INFO:tasks.workunit.client.0.smithi042.stdout:4/722: rename da/d11/d67/d5c/d66/fbf to da/d11/d67/ffa 0 2024-02-15T18:27:04.397 INFO:tasks.workunit.client.0.smithi042.stdout:5/754: dwrite d0/d39/d8d/fd3 [0,4194304] 0 2024-02-15T18:27:04.398 INFO:tasks.workunit.client.1.smithi191.stdout:0/229: unlink d7/da/d1a/d34/l1c 0 2024-02-15T18:27:04.402 INFO:tasks.workunit.client.0.smithi042.stdout:6/799: truncate d9/d11/d19/d35/d55/d92/fe8 3206076 0 2024-02-15T18:27:04.404 INFO:tasks.workunit.client.0.smithi042.stdout:0/698: rmdir d4/d38/d77/d2e/dae 39 2024-02-15T18:27:04.404 INFO:tasks.workunit.client.0.smithi042.stdout:0/699: fdatasync d4/d9/d34/f4b 0 2024-02-15T18:27:04.406 INFO:tasks.workunit.client.1.smithi191.stdout:1/447: unlink d0/d2f/f73 0 2024-02-15T18:27:04.407 INFO:tasks.workunit.client.0.smithi042.stdout:1/732: creat de/d15/f114 x:0 0 0 2024-02-15T18:27:04.409 INFO:tasks.workunit.client.1.smithi191.stdout:8/442: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/ca3 0 2024-02-15T18:27:04.410 INFO:tasks.workunit.client.0.smithi042.stdout:7/740: symlink d10/d12/d65/da7/lf2 0 2024-02-15T18:27:04.419 INFO:tasks.workunit.client.1.smithi191.stdout:0/230: dwrite d7/da/d17/f47 [0,4194304] 0 2024-02-15T18:27:04.420 INFO:tasks.workunit.client.1.smithi191.stdout:1/448: creat d0/d2f/d51/f93 x:0 0 0 2024-02-15T18:27:04.437 INFO:tasks.workunit.client.0.smithi042.stdout:2/668: mkdir d0/d35/d9e/dd2 0 2024-02-15T18:27:04.438 INFO:tasks.workunit.client.0.smithi042.stdout:7/741: dwrite d10/d12/d23/f55 [0,4194304] 0 2024-02-15T18:27:04.438 INFO:tasks.workunit.client.0.smithi042.stdout:7/742: readlink d10/d43/ldb 0 2024-02-15T18:27:04.439 INFO:tasks.workunit.client.1.smithi191.stdout:0/231: mkdir d7/d4a 0 2024-02-15T18:27:04.439 INFO:tasks.workunit.client.1.smithi191.stdout:0/232: readlink d7/lc 0 2024-02-15T18:27:04.442 INFO:tasks.workunit.client.1.smithi191.stdout:1/449: mkdir d0/d8/df/d90/d94 0 2024-02-15T18:27:04.444 INFO:tasks.workunit.client.1.smithi191.stdout:2/331: rename df/l2f to df/l6d 0 2024-02-15T18:27:04.448 INFO:tasks.workunit.client.1.smithi191.stdout:8/443: dwrite d3/d2c/d1a/f51 [0,4194304] 0 2024-02-15T18:27:04.448 INFO:tasks.workunit.client.1.smithi191.stdout:0/233: rmdir d7/da/d1a/d34 39 2024-02-15T18:27:04.448 INFO:tasks.workunit.client.1.smithi191.stdout:0/234: fdatasync d7/da/d1a/d34/f11 0 2024-02-15T18:27:04.451 INFO:tasks.workunit.client.0.smithi042.stdout:9/721: rmdir d6/da/d2f/d69 39 2024-02-15T18:27:04.451 INFO:tasks.workunit.client.0.smithi042.stdout:3/719: link d1/de/lc4 d1/d11/d14/d4d/d73/lea 0 2024-02-15T18:27:04.451 INFO:tasks.workunit.client.0.smithi042.stdout:3/720: write d1/f15 [1386396,57812] 0 2024-02-15T18:27:04.454 INFO:tasks.workunit.client.0.smithi042.stdout:4/723: rename da/f24 to da/d11/d67/d5c/d66/d78/d53/ffb 0 2024-02-15T18:27:04.464 INFO:tasks.workunit.client.0.smithi042.stdout:2/669: dwrite d0/d35/f81 [0,4194304] 0 2024-02-15T18:27:04.464 INFO:tasks.workunit.client.0.smithi042.stdout:2/670: read - d0/d40/fc3 zero size 2024-02-15T18:27:04.464 INFO:tasks.workunit.client.0.smithi042.stdout:2/671: getdents d0/d8/d10/d31/d79/dbd 0 2024-02-15T18:27:04.468 INFO:tasks.workunit.client.1.smithi191.stdout:8/444: link d3/d2c/d1a/d23/d2e/f56 d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/fa4 0 2024-02-15T18:27:04.471 INFO:tasks.workunit.client.1.smithi191.stdout:1/450: dwrite d0/d8/df/d12/d33/f8e [0,4194304] 0 2024-02-15T18:27:04.471 INFO:tasks.workunit.client.1.smithi191.stdout:1/451: truncate d0/d8/df/d12/d33/d49/f4a 6494576 0 2024-02-15T18:27:04.472 INFO:tasks.workunit.client.0.smithi042.stdout:2/672: write d0/d8/db/de/d1a/d2e/fcf [2414319,96927] 0 2024-02-15T18:27:04.476 INFO:tasks.workunit.client.1.smithi191.stdout:0/235: truncate d7/da/f20 2898473 0 2024-02-15T18:27:04.476 INFO:tasks.workunit.client.1.smithi191.stdout:0/236: write d7/da/f16 [4209204,43061] 0 2024-02-15T18:27:04.476 INFO:tasks.workunit.client.1.smithi191.stdout:0/237: readlink d7/da/d1a/l2d 0 2024-02-15T18:27:04.478 INFO:tasks.workunit.client.1.smithi191.stdout:2/332: creat df/d12/d19/d1a/d35/d5b/f6e x:0 0 0 2024-02-15T18:27:04.478 INFO:tasks.workunit.client.0.smithi042.stdout:5/755: getdents d0 0 2024-02-15T18:27:04.479 INFO:tasks.workunit.client.0.smithi042.stdout:5/756: chown d0/dda/d42/d7d/cc1 296681 1 2024-02-15T18:27:04.480 INFO:tasks.workunit.client.1.smithi191.stdout:1/452: dread d0/d8/df/d12/d2a/d2b/f5a [0,4194304] 0 2024-02-15T18:27:04.482 INFO:tasks.workunit.client.1.smithi191.stdout:1/453: read d0/d8/df/d41/f36 [11057873,87388] 0 2024-02-15T18:27:04.482 INFO:tasks.workunit.client.1.smithi191.stdout:1/454: readlink d0/d8/df/d41/l31 0 2024-02-15T18:27:04.488 INFO:tasks.workunit.client.0.smithi042.stdout:6/800: getdents d9/d11/d46/dc3 0 2024-02-15T18:27:04.488 INFO:tasks.workunit.client.0.smithi042.stdout:6/801: stat d9/d11/d19/d35/d49/d79/f60 0 2024-02-15T18:27:04.489 INFO:tasks.workunit.client.0.smithi042.stdout:6/802: read d9/d11/d19/d35/d49/f4c [3518334,122495] 0 2024-02-15T18:27:04.489 INFO:tasks.workunit.client.0.smithi042.stdout:0/700: truncate d4/d38/f3a 6167649 0 2024-02-15T18:27:04.490 INFO:tasks.workunit.client.0.smithi042.stdout:1/733: rmdir de/d54/d9a/dc8 39 2024-02-15T18:27:04.493 INFO:tasks.workunit.client.1.smithi191.stdout:0/238: unlink d7/da/d17/c1d 0 2024-02-15T18:27:04.493 INFO:tasks.workunit.client.1.smithi191.stdout:0/239: truncate d7/f2b 950561 0 2024-02-15T18:27:04.494 INFO:tasks.workunit.client.0.smithi042.stdout:1/734: read de/d6d/fa3 [34596,98726] 0 2024-02-15T18:27:04.494 INFO:tasks.workunit.client.0.smithi042.stdout:1/735: readlink de/d54/lf8 0 2024-02-15T18:27:04.494 INFO:tasks.workunit.client.0.smithi042.stdout:1/736: write de/d54/dbf/dd0/d7c/dd6/ddf/fa7 [852092,10871] 0 2024-02-15T18:27:04.494 INFO:tasks.workunit.client.0.smithi042.stdout:0/701: write d4/f3b [2223359,4703] 0 2024-02-15T18:27:04.494 INFO:tasks.workunit.client.0.smithi042.stdout:0/702: readlink d4/d9/l1f 0 2024-02-15T18:27:04.498 INFO:tasks.workunit.client.1.smithi191.stdout:8/445: dwrite d3/f18 [0,4194304] 0 2024-02-15T18:27:04.498 INFO:tasks.workunit.client.1.smithi191.stdout:8/446: write d3/d2c/d1a/d23/d2e/d47/f63 [384916,23283] 0 2024-02-15T18:27:04.500 INFO:tasks.workunit.client.0.smithi042.stdout:1/737: dread de/d15/f43 [0,4194304] 0 2024-02-15T18:27:04.500 INFO:tasks.workunit.client.0.smithi042.stdout:1/738: dread - de/d15/d67/f93 zero size 2024-02-15T18:27:04.504 INFO:tasks.workunit.client.0.smithi042.stdout:6/803: dwrite d9/d11/d19/d1c/dcf/fdd [0,4194304] 0 2024-02-15T18:27:04.509 INFO:tasks.workunit.client.1.smithi191.stdout:8/447: dread d3/d2c/d1a/f1b [0,4194304] 0 2024-02-15T18:27:04.518 INFO:tasks.workunit.client.0.smithi042.stdout:7/743: creat d10/d15/d39/d6c/ff3 x:0 0 0 2024-02-15T18:27:04.518 INFO:tasks.workunit.client.1.smithi191.stdout:2/333: mkdir df/d12/d1e/d6f 0 2024-02-15T18:27:04.521 INFO:tasks.workunit.client.0.smithi042.stdout:3/721: symlink d1/d11/d14/d4d/d73/d89/da8/leb 0 2024-02-15T18:27:04.521 INFO:tasks.workunit.client.0.smithi042.stdout:3/722: write d1/d11/d2e/dd1/fe5 [1018976,81459] 0 2024-02-15T18:27:04.524 INFO:tasks.workunit.client.1.smithi191.stdout:1/455: rename d0/d2f/d51/f53 to d0/d2f/d66/f95 0 2024-02-15T18:27:04.524 INFO:tasks.workunit.client.0.smithi042.stdout:8/657: sync 2024-02-15T18:27:04.525 INFO:tasks.workunit.client.1.smithi191.stdout:8/448: mknod d3/d2c/d1a/ca5 0 2024-02-15T18:27:04.525 INFO:tasks.workunit.client.1.smithi191.stdout:8/449: read - d3/d2c/d1a/f81 zero size 2024-02-15T18:27:04.525 INFO:tasks.workunit.client.1.smithi191.stdout:8/450: chown d3/d2c/d1a/d23/d2a/f6e 207 1 2024-02-15T18:27:04.527 INFO:tasks.workunit.client.0.smithi042.stdout:4/724: creat da/d11/d29/db8/ffc x:0 0 0 2024-02-15T18:27:04.529 INFO:tasks.workunit.client.0.smithi042.stdout:9/722: rename d6/da/d1d/dde/dec/fd7 to d6/dd/dcc/ffb 0 2024-02-15T18:27:04.537 INFO:tasks.workunit.client.0.smithi042.stdout:2/673: rmdir d0/d8/d20/d41/dbe 39 2024-02-15T18:27:04.539 INFO:tasks.workunit.client.0.smithi042.stdout:2/674: write d0/d8/d10/d31/d3a/d53/f66 [2080785,55532] 0 2024-02-15T18:27:04.539 INFO:tasks.workunit.client.0.smithi042.stdout:2/675: truncate d0/d8/d10/d5a/fbc 714961 0 2024-02-15T18:27:04.541 INFO:tasks.workunit.client.1.smithi191.stdout:0/240: rename d7/da/d1a/d34/d45/f49 to d7/da/d1a/d2c/f4b 0 2024-02-15T18:27:04.543 INFO:tasks.workunit.client.0.smithi042.stdout:5/757: creat d0/d38/d53/d68/f10a x:0 0 0 2024-02-15T18:27:04.543 INFO:tasks.workunit.client.0.smithi042.stdout:0/703: creat d4/d9/d34/dab/db0/fed x:0 0 0 2024-02-15T18:27:04.543 INFO:tasks.workunit.client.0.smithi042.stdout:0/704: stat d4/d38/c47 0 2024-02-15T18:27:04.544 INFO:tasks.workunit.client.1.smithi191.stdout:1/456: rename d0/d8/fd to d0/d8/f96 0 2024-02-15T18:27:04.544 INFO:tasks.workunit.client.0.smithi042.stdout:1/739: mknod de/d54/de6/df7/c115 0 2024-02-15T18:27:04.549 INFO:tasks.workunit.client.1.smithi191.stdout:8/451: symlink d3/d2c/d1a/la6 0 2024-02-15T18:27:04.551 INFO:tasks.workunit.client.0.smithi042.stdout:6/804: link d9/d11/d19/d1c/d31/d53/ff6 d9/d11/d19/d35/d55/d92/dfc/f11c 0 2024-02-15T18:27:04.551 INFO:tasks.workunit.client.0.smithi042.stdout:6/805: chown d9/lb 267026275 1 2024-02-15T18:27:04.551 INFO:tasks.workunit.client.0.smithi042.stdout:6/806: read d9/d11/d19/f2a [916134,5575] 0 2024-02-15T18:27:04.552 INFO:tasks.workunit.client.1.smithi191.stdout:2/334: rename df/d12/d1f/f45 to df/d12/d19/d1a/d35/f70 0 2024-02-15T18:27:04.553 INFO:tasks.workunit.client.1.smithi191.stdout:2/335: write df/d12/d19/d1a/f29 [727505,83070] 0 2024-02-15T18:27:04.553 INFO:tasks.workunit.client.0.smithi042.stdout:7/744: link d10/d12/d23/d51/f99 d10/d49/ff4 0 2024-02-15T18:27:04.554 INFO:tasks.workunit.client.1.smithi191.stdout:0/241: mkdir d7/da/d1a/d2c/d4c 0 2024-02-15T18:27:04.555 INFO:tasks.workunit.client.0.smithi042.stdout:8/658: link dd/de/d3c/ca0 dd/de/d94/db1/cb8 0 2024-02-15T18:27:04.556 INFO:tasks.workunit.client.1.smithi191.stdout:1/457: mknod d0/d2f/d66/c97 0 2024-02-15T18:27:04.556 INFO:tasks.workunit.client.1.smithi191.stdout:1/458: truncate d0/d8/df/d12/d33/d49/f4a 7468670 0 2024-02-15T18:27:04.557 INFO:tasks.workunit.client.0.smithi042.stdout:3/723: rename d1/d11/cca to d1/d11/d2e/cec 0 2024-02-15T18:27:04.558 INFO:tasks.workunit.client.1.smithi191.stdout:8/452: symlink d3/d2c/la7 0 2024-02-15T18:27:04.559 INFO:tasks.workunit.client.1.smithi191.stdout:8/453: stat d3/d2c/la7 0 2024-02-15T18:27:04.559 INFO:tasks.workunit.client.1.smithi191.stdout:8/454: dread - d3/d2c/d1a/d23/d2a/f49 zero size 2024-02-15T18:27:04.559 INFO:tasks.workunit.client.1.smithi191.stdout:8/455: truncate d3/d5e/f73 4969011 0 2024-02-15T18:27:04.559 INFO:tasks.workunit.client.0.smithi042.stdout:2/676: mknod d0/d8/d20/d41/dbe/cd3 0 2024-02-15T18:27:04.560 INFO:tasks.workunit.client.1.smithi191.stdout:2/336: symlink df/d12/d1e/l71 0 2024-02-15T18:27:04.561 INFO:tasks.workunit.client.0.smithi042.stdout:4/725: dwrite da/d2b/fd8 [0,4194304] 0 2024-02-15T18:27:04.561 INFO:tasks.workunit.client.0.smithi042.stdout:5/758: creat d0/d39/ddd/de0/f10b x:0 0 0 2024-02-15T18:27:04.562 INFO:tasks.workunit.client.0.smithi042.stdout:5/759: chown d0/dda/dc7/de1/dd8 0 1 2024-02-15T18:27:04.562 INFO:tasks.workunit.client.1.smithi191.stdout:2/337: read df/d12/d1f/d42/f43 [1165526,78174] 0 2024-02-15T18:27:04.562 INFO:tasks.workunit.client.1.smithi191.stdout:8/456: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/fa8 x:0 0 0 2024-02-15T18:27:04.563 INFO:tasks.workunit.client.0.smithi042.stdout:0/705: rmdir d4/d38/d77 39 2024-02-15T18:27:04.565 INFO:tasks.workunit.client.0.smithi042.stdout:1/740: mknod de/d54/dbf/dd0/d7c/dd6/de8/d103/d10f/c116 0 2024-02-15T18:27:04.566 INFO:tasks.workunit.client.0.smithi042.stdout:6/807: link d9/d11/d19/d1c/d31/d4f/f68 d9/d11/d19/d35/d49/f11d 0 2024-02-15T18:27:04.568 INFO:tasks.workunit.client.0.smithi042.stdout:9/723: dwrite f2 [4194304,4194304] 0 2024-02-15T18:27:04.568 INFO:tasks.workunit.client.0.smithi042.stdout:8/659: dread dd/de/d3c/f70 [0,4194304] 0 2024-02-15T18:27:04.568 INFO:tasks.workunit.client.0.smithi042.stdout:8/660: dread - dd/d11/d75/fa1 zero size 2024-02-15T18:27:04.568 INFO:tasks.workunit.client.1.smithi191.stdout:8/457: rename d3/d2c/d1a/d23/d2e/d5b/d69/d6d/f83 to d3/d2c/d1a/d23/d3e/fa9 0 2024-02-15T18:27:04.569 INFO:tasks.workunit.client.0.smithi042.stdout:7/745: creat d10/d12/d65/da7/ff5 x:0 0 0 2024-02-15T18:27:04.569 INFO:tasks.workunit.client.0.smithi042.stdout:7/746: chown d10/f14 5 1 2024-02-15T18:27:04.569 INFO:tasks.workunit.client.0.smithi042.stdout:7/747: readlink d10/d15/d39/l60 0 2024-02-15T18:27:04.569 INFO:tasks.workunit.client.1.smithi191.stdout:2/338: link df/d12/c14 df/d61/c72 0 2024-02-15T18:27:04.571 INFO:tasks.workunit.client.1.smithi191.stdout:8/458: creat d3/faa x:0 0 0 2024-02-15T18:27:04.571 INFO:tasks.workunit.client.1.smithi191.stdout:0/242: dwrite d7/da/d17/d31/f36 [0,4194304] 0 2024-02-15T18:27:04.571 INFO:tasks.workunit.client.0.smithi042.stdout:3/724: mknod d1/d11/d14/d4d/d73/d89/dd9/ced 0 2024-02-15T18:27:04.572 INFO:tasks.workunit.client.0.smithi042.stdout:2/677: symlink d0/d8/db/ld4 0 2024-02-15T18:27:04.574 INFO:tasks.workunit.client.1.smithi191.stdout:8/459: rename d3/d2c/d1a/d23/f6a to d3/d2c/d1a/d23/d2e/d5b/d69/d6d/fab 0 2024-02-15T18:27:04.574 INFO:tasks.workunit.client.1.smithi191.stdout:8/460: write d3/d5e/f79 [946259,90742] 0 2024-02-15T18:27:04.574 INFO:tasks.workunit.client.1.smithi191.stdout:8/461: truncate d3/d5e/f80 626762 0 2024-02-15T18:27:04.575 INFO:tasks.workunit.client.1.smithi191.stdout:2/339: dread df/f10 [0,4194304] 0 2024-02-15T18:27:04.576 INFO:tasks.workunit.client.1.smithi191.stdout:0/243: creat d7/d1b/f4d x:0 0 0 2024-02-15T18:27:04.579 INFO:tasks.workunit.client.0.smithi042.stdout:4/726: rename da/d11/d67/d5c to da/d11/d29/dfd 0 2024-02-15T18:27:04.579 INFO:tasks.workunit.client.0.smithi042.stdout:4/727: write da/d11/d29/dfd/d66/d78/d53/ffb [4455072,70051] 0 2024-02-15T18:27:04.579 INFO:tasks.workunit.client.0.smithi042.stdout:4/728: dread - da/d2b/d7e/fc7 zero size 2024-02-15T18:27:04.580 INFO:tasks.workunit.client.0.smithi042.stdout:5/760: creat d0/d39/d8d/d9b/f10c x:0 0 0 2024-02-15T18:27:04.580 INFO:tasks.workunit.client.0.smithi042.stdout:5/761: dread - d0/dda/dc7/d52/dad/fea zero size 2024-02-15T18:27:04.581 INFO:tasks.workunit.client.1.smithi191.stdout:2/340: mknod df/d12/d19/d1a/c73 0 2024-02-15T18:27:04.582 INFO:tasks.workunit.client.0.smithi042.stdout:0/706: symlink d4/d38/d77/d2e/dae/ddf/lee 0 2024-02-15T18:27:04.582 INFO:tasks.workunit.client.0.smithi042.stdout:0/707: write d4/d38/d39/d8e/f67 [3723588,57054] 0 2024-02-15T18:27:04.583 INFO:tasks.workunit.client.1.smithi191.stdout:0/244: creat d7/da/d1a/d34/f4e x:0 0 0 2024-02-15T18:27:04.583 INFO:tasks.workunit.client.0.smithi042.stdout:1/741: creat de/d6d/d9e/dcd/f117 x:0 0 0 2024-02-15T18:27:04.584 INFO:tasks.workunit.client.1.smithi191.stdout:2/341: creat df/d12/d19/d1a/d35/f74 x:0 0 0 2024-02-15T18:27:04.585 INFO:tasks.workunit.client.1.smithi191.stdout:0/245: creat d7/da/d1a/d2c/f4f x:0 0 0 2024-02-15T18:27:04.586 INFO:tasks.workunit.client.0.smithi042.stdout:6/808: creat d9/d11/d19/d1c/d31/d115/f11e x:0 0 0 2024-02-15T18:27:04.588 INFO:tasks.workunit.client.1.smithi191.stdout:2/342: getdents df/d12 0 2024-02-15T18:27:04.588 INFO:tasks.workunit.client.1.smithi191.stdout:2/343: getdents df/d12/d19/d1a/d35/d47 0 2024-02-15T18:27:04.588 INFO:tasks.workunit.client.0.smithi042.stdout:7/748: dwrite d10/d49/d6d/f86 [0,4194304] 0 2024-02-15T18:27:04.590 INFO:tasks.workunit.client.0.smithi042.stdout:9/724: truncate d6/da/d7b/f92 825958 0 2024-02-15T18:27:04.591 INFO:tasks.workunit.client.1.smithi191.stdout:0/246: unlink d7/da/d1a/d34/f15 0 2024-02-15T18:27:04.591 INFO:tasks.workunit.client.1.smithi191.stdout:0/247: readlink d7/d3b/l3f 0 2024-02-15T18:27:04.592 INFO:tasks.workunit.client.0.smithi042.stdout:8/661: creat dd/de/fb9 x:0 0 0 2024-02-15T18:27:04.593 INFO:tasks.workunit.client.1.smithi191.stdout:2/344: mknod df/d12/d19/d1a/d35/d5b/c75 0 2024-02-15T18:27:04.594 INFO:tasks.workunit.client.0.smithi042.stdout:3/725: creat d1/d11/d2e/d4c/fee x:0 0 0 2024-02-15T18:27:04.686 INFO:tasks.workunit.client.1.smithi191.stdout:0/248: rename d7/da/d1a/d34/l2a to d7/d1b/d30/l50 0 2024-02-15T18:27:04.686 INFO:tasks.workunit.client.1.smithi191.stdout:0/249: chown d7/f39 1713012 1 2024-02-15T18:27:04.686 INFO:tasks.workunit.client.1.smithi191.stdout:8/462: getdents d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67 0 2024-02-15T18:27:04.688 INFO:tasks.workunit.client.0.smithi042.stdout:2/678: rename d0/d8/d10/lb7 to d0/d35/ld5 0 2024-02-15T18:27:04.689 INFO:tasks.workunit.client.1.smithi191.stdout:2/345: symlink df/d12/d1f/l76 0 2024-02-15T18:27:04.692 INFO:tasks.workunit.client.0.smithi042.stdout:4/729: mknod da/d3a/dc9/cfe 0 2024-02-15T18:27:04.693 INFO:tasks.workunit.client.0.smithi042.stdout:5/762: rmdir d0/d12/d2a/d105 39 2024-02-15T18:27:04.694 INFO:tasks.workunit.client.0.smithi042.stdout:0/708: mknod d4/d38/d5c/cef 0 2024-02-15T18:27:04.694 INFO:tasks.workunit.client.0.smithi042.stdout:0/709: readlink l3 0 2024-02-15T18:27:04.697 INFO:tasks.workunit.client.0.smithi042.stdout:9/725: mkdir d6/da2/dfc 0 2024-02-15T18:27:04.698 INFO:tasks.workunit.client.1.smithi191.stdout:0/250: mknod d7/d1b/d30/c51 0 2024-02-15T18:27:04.698 INFO:tasks.workunit.client.1.smithi191.stdout:0/251: fsync d7/d1b/f4d 0 2024-02-15T18:27:04.703 INFO:tasks.workunit.client.1.smithi191.stdout:8/463: mkdir d3/d2c/d1a/d23/d2e/d5b/d69/d6d/dac 0 2024-02-15T18:27:04.705 INFO:tasks.workunit.client.0.smithi042.stdout:8/662: creat dd/de/da4/d81/dae/fba x:0 0 0 2024-02-15T18:27:04.705 INFO:tasks.workunit.client.0.smithi042.stdout:8/663: fsync dd/de/d33/d3b/f46 0 2024-02-15T18:27:04.705 INFO:tasks.workunit.client.0.smithi042.stdout:8/664: truncate dd/de/d33/d3b/d92/fb3 503608 0 2024-02-15T18:27:04.705 INFO:tasks.workunit.client.0.smithi042.stdout:8/665: dread - dd/de/d33/d66/d4e/f87 zero size 2024-02-15T18:27:04.706 INFO:tasks.workunit.client.1.smithi191.stdout:2/346: creat df/d12/d19/f77 x:0 0 0 2024-02-15T18:27:04.707 INFO:tasks.workunit.client.1.smithi191.stdout:0/252: getdents d7/da/d1a/d2c 0 2024-02-15T18:27:04.707 INFO:tasks.workunit.client.1.smithi191.stdout:0/253: readlink d7/da/d1a/d34/d27/l2e 0 2024-02-15T18:27:04.707 INFO:tasks.workunit.client.1.smithi191.stdout:0/254: chown d7/d4a 22 1 2024-02-15T18:27:04.709 INFO:tasks.workunit.client.1.smithi191.stdout:8/464: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/fad x:0 0 0 2024-02-15T18:27:04.714 INFO:tasks.workunit.client.1.smithi191.stdout:2/347: link df/d12/d19/d1a/d35/d47/c63 df/d12/d1e/c78 0 2024-02-15T18:27:04.714 INFO:tasks.workunit.client.1.smithi191.stdout:2/348: chown df/d12/d19/d1a/f5e 558916356 1 2024-02-15T18:27:04.716 INFO:tasks.workunit.client.1.smithi191.stdout:0/255: rename d7/d1b/f4d to d7/da/d1a/d34/f52 0 2024-02-15T18:27:04.716 INFO:tasks.workunit.client.1.smithi191.stdout:0/256: write d7/d1b/f23 [1300515,96569] 0 2024-02-15T18:27:04.717 INFO:tasks.workunit.client.1.smithi191.stdout:8/465: mknod d3/d2c/d1a/d23/d2e/d47/d5f/cae 0 2024-02-15T18:27:04.718 INFO:tasks.workunit.client.1.smithi191.stdout:2/349: creat df/d61/f79 x:0 0 0 2024-02-15T18:27:04.720 INFO:tasks.workunit.client.1.smithi191.stdout:0/257: mknod d7/da/d1a/d34/c53 0 2024-02-15T18:27:04.721 INFO:tasks.workunit.client.0.smithi042.stdout:8/666: dwrite dd/de/d12/f9e [0,4194304] 0 2024-02-15T18:27:04.723 INFO:tasks.workunit.client.1.smithi191.stdout:2/350: truncate df/d12/d1f/d42/f43 2277701 0 2024-02-15T18:27:04.724 INFO:tasks.workunit.client.1.smithi191.stdout:2/351: write df/d12/d19/d1a/f29 [1110845,99684] 0 2024-02-15T18:27:04.725 INFO:tasks.workunit.client.1.smithi191.stdout:0/258: link l4 d7/da/d1a/d2c/l54 0 2024-02-15T18:27:04.728 INFO:tasks.workunit.client.1.smithi191.stdout:8/466: rmdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82 39 2024-02-15T18:27:04.731 INFO:tasks.workunit.client.1.smithi191.stdout:2/352: rmdir df/d12/d19 39 2024-02-15T18:27:04.736 INFO:tasks.workunit.client.0.smithi042.stdout:3/726: creat d1/de/fef x:0 0 0 2024-02-15T18:27:04.741 INFO:tasks.workunit.client.1.smithi191.stdout:0/259: link d7/da/d1a/d34/f11 d7/da/d17/d31/f55 0 2024-02-15T18:27:04.742 INFO:tasks.workunit.client.0.smithi042.stdout:8/667: dwrite dd/de/d12/f45 [0,4194304] 0 2024-02-15T18:27:04.742 INFO:tasks.workunit.client.1.smithi191.stdout:5/172: sync 2024-02-15T18:27:04.742 INFO:tasks.workunit.client.1.smithi191.stdout:6/299: sync 2024-02-15T18:27:04.743 INFO:tasks.workunit.client.0.smithi042.stdout:2/679: mkdir d0/d8/d20/d3f/dd6 0 2024-02-15T18:27:04.743 INFO:tasks.workunit.client.0.smithi042.stdout:2/680: chown d0/d8/f5f 20921 1 2024-02-15T18:27:04.744 INFO:tasks.workunit.client.1.smithi191.stdout:8/467: creat d3/d2c/d1a/d36/d9b/faf x:0 0 0 2024-02-15T18:27:04.744 INFO:tasks.workunit.client.1.smithi191.stdout:8/468: read d3/d2c/d1a/d23/d2e/d47/f63 [373921,12802] 0 2024-02-15T18:27:04.746 INFO:tasks.workunit.client.0.smithi042.stdout:1/742: rename de/d15/d67/d46/f50 to de/d54/dbf/dd0/d7c/f118 0 2024-02-15T18:27:04.746 INFO:tasks.workunit.client.0.smithi042.stdout:6/809: sync 2024-02-15T18:27:04.748 INFO:tasks.workunit.client.1.smithi191.stdout:2/353: mkdir df/d12/d19/d1a/d35/d47/d4b/d7a 0 2024-02-15T18:27:04.748 INFO:tasks.workunit.client.1.smithi191.stdout:2/354: chown df/d12/d1e/d6f 873732 1 2024-02-15T18:27:04.748 INFO:tasks.workunit.client.0.smithi042.stdout:4/730: truncate da/d11/d27/f69 2540291 0 2024-02-15T18:27:04.750 INFO:tasks.workunit.client.1.smithi191.stdout:0/260: creat d7/d4a/f56 x:0 0 0 2024-02-15T18:27:04.750 INFO:tasks.workunit.client.0.smithi042.stdout:9/726: symlink d6/da2/db1/lfd 0 2024-02-15T18:27:04.750 INFO:tasks.workunit.client.0.smithi042.stdout:9/727: write d6/da/d2f/f6a [1128721,103362] 0 2024-02-15T18:27:04.751 INFO:tasks.workunit.client.0.smithi042.stdout:9/728: write d6/da/f23 [465722,100416] 0 2024-02-15T18:27:04.751 INFO:tasks.workunit.client.1.smithi191.stdout:5/173: unlink d0/cd 0 2024-02-15T18:27:04.751 INFO:tasks.workunit.client.1.smithi191.stdout:5/174: rename d0 to d0/d14/d3a 22 2024-02-15T18:27:04.772 INFO:tasks.workunit.client.1.smithi191.stdout:5/175: dwrite d0/d14/d28/f36 [0,4194304] 0 2024-02-15T18:27:04.811 INFO:tasks.workunit.client.1.smithi191.stdout:6/300: creat d8/d9/d2b/f51 x:0 0 0 2024-02-15T18:27:04.812 INFO:tasks.workunit.client.1.smithi191.stdout:8/469: rmdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67 39 2024-02-15T18:27:04.812 INFO:tasks.workunit.client.1.smithi191.stdout:8/470: chown d3/d2c/d1a/d23/d2a/f35 1454 1 2024-02-15T18:27:04.812 INFO:tasks.workunit.client.1.smithi191.stdout:8/471: truncate d3/d2c/d1a/d23/f5c 380426 0 2024-02-15T18:27:04.812 INFO:tasks.workunit.client.1.smithi191.stdout:8/472: fdatasync d3/d5e/f7a 0 2024-02-15T18:27:04.813 INFO:tasks.workunit.client.0.smithi042.stdout:3/727: getdents d1/d11/d14/d4d/d6c 0 2024-02-15T18:27:04.817 INFO:tasks.workunit.client.0.smithi042.stdout:2/681: mkdir d0/d8/d20/d7d/d7e/dc7/d8b/dd7 0 2024-02-15T18:27:04.819 INFO:tasks.workunit.client.1.smithi191.stdout:0/261: mknod d7/da/d1a/d34/d27/c57 0 2024-02-15T18:27:04.820 INFO:tasks.workunit.client.1.smithi191.stdout:5/176: unlink d0/f13 0 2024-02-15T18:27:04.821 INFO:tasks.workunit.client.1.smithi191.stdout:5/177: write d0/f1f [1073374,92584] 0 2024-02-15T18:27:04.821 INFO:tasks.workunit.client.1.smithi191.stdout:5/178: chown d0/f1b 1158328 1 2024-02-15T18:27:04.822 INFO:tasks.workunit.client.0.smithi042.stdout:7/749: rename d10/d12/c54 to d10/d15/d8f/da5/dad/dda/cf6 0 2024-02-15T18:27:04.824 INFO:tasks.workunit.client.0.smithi042.stdout:1/743: creat de/d54/dbf/dd0/d7c/dd6/de8/dec/d100/f119 x:0 0 0 2024-02-15T18:27:04.824 INFO:tasks.workunit.client.1.smithi191.stdout:6/301: symlink d8/l52 0 2024-02-15T18:27:04.825 INFO:tasks.workunit.client.0.smithi042.stdout:6/810: creat d9/d11/d46/d89/dbc/f11f x:0 0 0 2024-02-15T18:27:04.829 INFO:tasks.workunit.client.1.smithi191.stdout:8/473: symlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/da0/lb0 0 2024-02-15T18:27:04.830 INFO:tasks.workunit.client.0.smithi042.stdout:4/731: mknod da/d11/d29/d4a/d4f/dcf/cff 0 2024-02-15T18:27:04.831 INFO:tasks.workunit.client.0.smithi042.stdout:1/744: dread de/d54/dbf/fe1 [0,4194304] 0 2024-02-15T18:27:04.831 INFO:tasks.workunit.client.1.smithi191.stdout:0/262: creat d7/da/d1a/f58 x:0 0 0 2024-02-15T18:27:04.833 INFO:tasks.workunit.client.0.smithi042.stdout:9/729: rmdir d6/da/d2f/d6c 39 2024-02-15T18:27:04.833 INFO:tasks.workunit.client.0.smithi042.stdout:9/730: write d6/da/d2f/d7c/f98 [163076,97689] 0 2024-02-15T18:27:04.835 INFO:tasks.workunit.client.1.smithi191.stdout:5/179: readlink d0/l1d 0 2024-02-15T18:27:04.840 INFO:tasks.workunit.client.0.smithi042.stdout:3/728: symlink d1/de/d1a/de9/lf0 0 2024-02-15T18:27:04.841 INFO:tasks.workunit.client.1.smithi191.stdout:8/474: creat d3/d2c/d1a/fb1 x:0 0 0 2024-02-15T18:27:04.842 INFO:tasks.workunit.client.1.smithi191.stdout:0/263: link d7/da/d1a/d34/f29 d7/d4a/f59 0 2024-02-15T18:27:04.843 INFO:tasks.workunit.client.0.smithi042.stdout:2/682: mknod d0/d8/d10/d7b/cd8 0 2024-02-15T18:27:04.844 INFO:tasks.workunit.client.0.smithi042.stdout:2/683: write d0/d8/f65 [3939834,47973] 0 2024-02-15T18:27:04.846 INFO:tasks.workunit.client.0.smithi042.stdout:5/763: rename d0/d12/c9a to d0/dda/dc7/de1/c10d 0 2024-02-15T18:27:04.848 INFO:tasks.workunit.client.0.smithi042.stdout:6/811: mknod d9/d3e/dd9/c120 0 2024-02-15T18:27:04.848 INFO:tasks.workunit.client.0.smithi042.stdout:7/750: creat d10/d12/d65/ff7 x:0 0 0 2024-02-15T18:27:04.848 INFO:tasks.workunit.client.0.smithi042.stdout:6/812: getdents d9/d11/d46/dc3 0 2024-02-15T18:27:04.848 INFO:tasks.workunit.client.0.smithi042.stdout:6/813: fsync d9/d11/d19/d35/d49/f11d 0 2024-02-15T18:27:04.848 INFO:tasks.workunit.client.0.smithi042.stdout:7/751: write d10/d12/d65/da7/ff5 [377739,123198] 0 2024-02-15T18:27:04.849 INFO:tasks.workunit.client.1.smithi191.stdout:5/180: mkdir d0/d14/d28/d31/d38/d3b 0 2024-02-15T18:27:04.849 INFO:tasks.workunit.client.0.smithi042.stdout:4/732: getdents da/d3b/d41/d8d 0 2024-02-15T18:27:04.849 INFO:tasks.workunit.client.0.smithi042.stdout:4/733: write da/d11/d85/ff8 [3238976,28958] 0 2024-02-15T18:27:04.850 INFO:tasks.workunit.client.1.smithi191.stdout:6/302: dwrite d8/f1c [0,4194304] 0 2024-02-15T18:27:04.852 INFO:tasks.workunit.client.0.smithi042.stdout:9/731: creat d6/da/d1d/d2d/d40/ffe x:0 0 0 2024-02-15T18:27:04.852 INFO:tasks.workunit.client.0.smithi042.stdout:1/745: dwrite de/d15/d67/f2a [0,4194304] 0 2024-02-15T18:27:04.852 INFO:tasks.workunit.client.0.smithi042.stdout:1/746: write de/d15/d2c/d51/fcf [1047207,66762] 0 2024-02-15T18:27:04.856 INFO:tasks.workunit.client.0.smithi042.stdout:6/814: dread d9/d11/d19/fed [0,4194304] 0 2024-02-15T18:27:04.858 INFO:tasks.workunit.client.0.smithi042.stdout:3/729: symlink d1/d11/lf1 0 2024-02-15T18:27:04.859 INFO:tasks.workunit.client.1.smithi191.stdout:5/181: mkdir d0/d14/d3c 0 2024-02-15T18:27:04.860 INFO:tasks.workunit.client.0.smithi042.stdout:0/710: sync 2024-02-15T18:27:04.864 INFO:tasks.workunit.client.0.smithi042.stdout:4/734: dread da/d11/d29/f70 [0,4194304] 0 2024-02-15T18:27:04.865 INFO:tasks.workunit.client.0.smithi042.stdout:2/684: creat d0/d8/d10/fd9 x:0 0 0 2024-02-15T18:27:04.866 INFO:tasks.workunit.client.1.smithi191.stdout:5/182: creat d0/d23/f3d x:0 0 0 2024-02-15T18:27:04.869 INFO:tasks.workunit.client.1.smithi191.stdout:5/183: rmdir d0/d14/d28/d31/d38 39 2024-02-15T18:27:04.870 INFO:tasks.workunit.client.0.smithi042.stdout:8/668: rename dd/de/d12/l27 to dd/de/d33/d66/lbb 0 2024-02-15T18:27:04.871 INFO:tasks.workunit.client.0.smithi042.stdout:7/752: rmdir d10/d15 39 2024-02-15T18:27:04.872 INFO:tasks.workunit.client.1.smithi191.stdout:8/475: dwrite d3/d2c/d1a/d23/d3e/fa9 [0,4194304] 0 2024-02-15T18:27:04.872 INFO:tasks.workunit.client.1.smithi191.stdout:8/476: fsync d3/d2c/d1a/d23/d2e/f3a 0 2024-02-15T18:27:04.872 INFO:tasks.workunit.client.1.smithi191.stdout:8/477: getdents d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a 0 2024-02-15T18:27:04.873 INFO:tasks.workunit.client.1.smithi191.stdout:5/184: creat d0/d14/d28/f3e x:0 0 0 2024-02-15T18:27:04.876 INFO:tasks.workunit.client.0.smithi042.stdout:6/815: unlink d9/d11/d19/lbb 0 2024-02-15T18:27:04.876 INFO:tasks.workunit.client.0.smithi042.stdout:6/816: chown d9/d11/d46/d89/dbc/f110 544 1 2024-02-15T18:27:04.878 INFO:tasks.workunit.client.1.smithi191.stdout:6/303: dwrite d8/d19/f1f [0,4194304] 0 2024-02-15T18:27:04.878 INFO:tasks.workunit.client.0.smithi042.stdout:5/764: rename d0/f3 to d0/d38/d53/d91/d94/f10e 0 2024-02-15T18:27:04.880 INFO:tasks.workunit.client.1.smithi191.stdout:8/478: rename d3/d2c/d1a/d23/d2e/d5b/d69/d6d/dac to d3/d2c/d1a/d23/d3e/db2 0 2024-02-15T18:27:04.880 INFO:tasks.workunit.client.0.smithi042.stdout:8/669: mknod dd/de/da4/cbc 0 2024-02-15T18:27:04.881 INFO:tasks.workunit.client.1.smithi191.stdout:6/304: write d8/f15 [273920,34369] 0 2024-02-15T18:27:04.881 INFO:tasks.workunit.client.1.smithi191.stdout:6/305: write d8/d19/f3d [826125,47784] 0 2024-02-15T18:27:04.881 INFO:tasks.workunit.client.0.smithi042.stdout:7/753: creat d10/ff8 x:0 0 0 2024-02-15T18:27:04.882 INFO:tasks.workunit.client.1.smithi191.stdout:8/479: mkdir d3/d2c/d1a/db3 0 2024-02-15T18:27:04.883 INFO:tasks.workunit.client.1.smithi191.stdout:6/306: unlink l2 0 2024-02-15T18:27:04.884 INFO:tasks.workunit.client.0.smithi042.stdout:1/747: dwrite de/d15/d67/d74/fab [0,4194304] 0 2024-02-15T18:27:04.886 INFO:tasks.workunit.client.0.smithi042.stdout:3/730: dwrite d1/d11/d14/d4d/d73/f82 [0,4194304] 0 2024-02-15T18:27:04.886 INFO:tasks.workunit.client.0.smithi042.stdout:3/731: chown d1/d11/d14/d4d/d73/d89/da8/lbd 0 1 2024-02-15T18:27:04.886 INFO:tasks.workunit.client.0.smithi042.stdout:9/732: rename d6/da/d1d/d2d/d40/d48/l4b to d6/dd/lff 0 2024-02-15T18:27:04.887 INFO:tasks.workunit.client.1.smithi191.stdout:8/480: truncate d3/fc 976654 0 2024-02-15T18:27:04.887 INFO:tasks.workunit.client.1.smithi191.stdout:8/481: write d3/d2c/d1a/d23/d2e/d64/f8d [377331,7018] 0 2024-02-15T18:27:04.887 INFO:tasks.workunit.client.1.smithi191.stdout:8/482: fsync d3/d2c/d1a/d36/d9b/faf 0 2024-02-15T18:27:04.889 INFO:tasks.workunit.client.1.smithi191.stdout:6/307: unlink d8/d9/d4a/c50 0 2024-02-15T18:27:04.891 INFO:tasks.workunit.client.1.smithi191.stdout:6/308: creat d8/d9/d4a/d4c/f53 x:0 0 0 2024-02-15T18:27:04.892 INFO:tasks.workunit.client.0.smithi042.stdout:6/817: dwrite d9/d11/d19/d35/d49/d79/fa8 [0,4194304] 0 2024-02-15T18:27:04.894 INFO:tasks.workunit.client.0.smithi042.stdout:7/754: mknod d10/d15/d39/cf9 0 2024-02-15T18:27:04.895 INFO:tasks.workunit.client.0.smithi042.stdout:7/755: write d10/d12/d23/f61 [1911490,81683] 0 2024-02-15T18:27:04.904 INFO:tasks.workunit.client.1.smithi191.stdout:4/381: sync 2024-02-15T18:27:04.906 INFO:tasks.workunit.client.1.smithi191.stdout:6/309: rename d8/f12 to d8/f54 0 2024-02-15T18:27:04.908 INFO:tasks.workunit.client.1.smithi191.stdout:6/310: dread d8/f15 [4194304,4194304] 0 2024-02-15T18:27:04.914 INFO:tasks.workunit.client.0.smithi042.stdout:1/748: creat de/d15/d67/d63/f11a x:0 0 0 2024-02-15T18:27:04.914 INFO:tasks.workunit.client.0.smithi042.stdout:6/818: dwrite d9/d11/d19/d1c/d31/f5b [0,4194304] 0 2024-02-15T18:27:04.918 INFO:tasks.workunit.client.1.smithi191.stdout:4/382: creat d2/d5/d7/d2f/d41/f92 x:0 0 0 2024-02-15T18:27:04.920 INFO:tasks.workunit.client.0.smithi042.stdout:3/732: mkdir d1/d11/d14/d4d/d73/df2 0 2024-02-15T18:27:04.920 INFO:tasks.workunit.client.0.smithi042.stdout:3/733: chown d1/d11/d14/f60 240227238 1 2024-02-15T18:27:04.921 INFO:tasks.workunit.client.1.smithi191.stdout:4/383: write d2/d5/d14/d47/d78/f38 [336174,101301] 0 2024-02-15T18:27:04.923 INFO:tasks.workunit.client.0.smithi042.stdout:0/711: rename d4/d9/d5a/f6b to d4/d9/d34/d57/dc1/ff0 0 2024-02-15T18:27:04.924 INFO:tasks.workunit.client.0.smithi042.stdout:9/733: mkdir d6/dd/d9/d59/dbf/d100 0 2024-02-15T18:27:04.935 INFO:tasks.workunit.client.0.smithi042.stdout:7/756: symlink d10/d15/d39/de3/lfa 0 2024-02-15T18:27:04.936 INFO:tasks.workunit.client.0.smithi042.stdout:6/819: rmdir d9/d11/d46/df8 39 2024-02-15T18:27:04.936 INFO:tasks.workunit.client.0.smithi042.stdout:6/820: chown d9/d11/c29 16685094 1 2024-02-15T18:27:04.937 INFO:tasks.workunit.client.0.smithi042.stdout:6/821: chown d9/f40 11 1 2024-02-15T18:27:04.937 INFO:tasks.workunit.client.0.smithi042.stdout:4/735: rename da/d11/d29/d4a/d4f/f8e to da/d3a/f100 0 2024-02-15T18:27:04.938 INFO:tasks.workunit.client.0.smithi042.stdout:7/757: creat d10/d15/d39/dd9/ffb x:0 0 0 2024-02-15T18:27:04.938 INFO:tasks.workunit.client.0.smithi042.stdout:7/758: truncate d10/d12/d23/d51/f99 1227304 0 2024-02-15T18:27:04.939 INFO:tasks.workunit.client.0.smithi042.stdout:6/822: mkdir d9/d11/d19/d35/d55/d92/dfc/d121 0 2024-02-15T18:27:04.939 INFO:tasks.workunit.client.0.smithi042.stdout:6/823: dread - d9/d11/d46/d89/dbc/f10f zero size 2024-02-15T18:27:04.940 INFO:tasks.workunit.client.0.smithi042.stdout:5/765: rename d0/d38/c73 to d0/d12/d2a/d102/c10f 0 2024-02-15T18:27:04.942 INFO:tasks.workunit.client.1.smithi191.stdout:4/384: dread d2/d5/f4d [0,4194304] 0 2024-02-15T18:27:04.946 INFO:tasks.workunit.client.0.smithi042.stdout:0/712: dread d4/d9/f31 [0,4194304] 0 2024-02-15T18:27:04.949 INFO:tasks.workunit.client.1.smithi191.stdout:6/311: dwrite d8/d19/f40 [0,4194304] 0 2024-02-15T18:27:04.949 INFO:tasks.workunit.client.1.smithi191.stdout:6/312: dread - d8/d9/d2b/f4b zero size 2024-02-15T18:27:04.949 INFO:tasks.workunit.client.1.smithi191.stdout:6/313: readlink d8/d19/l2c 0 2024-02-15T18:27:04.949 INFO:tasks.workunit.client.1.smithi191.stdout:6/314: write d8/d19/f35 [3010455,70288] 0 2024-02-15T18:27:04.950 INFO:tasks.workunit.client.0.smithi042.stdout:9/734: dread d6/dd/f2b [0,4194304] 0 2024-02-15T18:27:04.950 INFO:tasks.workunit.client.0.smithi042.stdout:1/749: dwrite de/d15/d67/d46/f71 [0,4194304] 0 2024-02-15T18:27:04.950 INFO:tasks.workunit.client.0.smithi042.stdout:3/734: dwrite d1/de/d19/f1f [4194304,4194304] 0 2024-02-15T18:27:04.957 INFO:tasks.workunit.client.0.smithi042.stdout:7/759: dwrite d10/ff8 [0,4194304] 0 2024-02-15T18:27:05.008 INFO:tasks.workunit.client.0.smithi042.stdout:4/736: mknod da/d11/d29/dfd/d66/d78/c101 0 2024-02-15T18:27:05.013 INFO:tasks.workunit.client.0.smithi042.stdout:6/824: link d9/d11/d19/d1c/f6c d9/d11/d46/f122 0 2024-02-15T18:27:05.014 INFO:tasks.workunit.client.1.smithi191.stdout:4/385: link d2/d5/f13 d2/d5/d7/f93 0 2024-02-15T18:27:05.014 INFO:tasks.workunit.client.1.smithi191.stdout:4/386: write d2/f2c [2647920,19911] 0 2024-02-15T18:27:05.014 INFO:tasks.workunit.client.1.smithi191.stdout:4/387: readlink d2/d5/d7/l1b 0 2024-02-15T18:27:05.016 INFO:tasks.workunit.client.0.smithi042.stdout:5/766: link d0/l17 d0/dda/dc7/d52/dad/dba/dbd/dde/l110 0 2024-02-15T18:27:05.017 INFO:tasks.workunit.client.0.smithi042.stdout:0/713: creat d4/d9/d5a/d60/ff1 x:0 0 0 2024-02-15T18:27:05.018 INFO:tasks.workunit.client.0.smithi042.stdout:5/767: dread d0/d39/fef [0,4194304] 0 2024-02-15T18:27:05.023 INFO:tasks.workunit.client.1.smithi191.stdout:6/315: mknod d8/d19/c55 0 2024-02-15T18:27:05.024 INFO:tasks.workunit.client.0.smithi042.stdout:9/735: creat d6/da/d1d/d2d/f101 x:0 0 0 2024-02-15T18:27:05.024 INFO:tasks.workunit.client.0.smithi042.stdout:9/736: chown d6/da/d2f/d69/cdb 121100 1 2024-02-15T18:27:05.026 INFO:tasks.workunit.client.0.smithi042.stdout:1/750: mknod de/d15/d1b/d44/df3/c11b 0 2024-02-15T18:27:05.026 INFO:tasks.workunit.client.0.smithi042.stdout:1/751: truncate de/d54/dbf/dd0/d7c/dd6/ddf/d88/fff 880699 0 2024-02-15T18:27:05.028 INFO:tasks.workunit.client.1.smithi191.stdout:4/388: creat d2/d5/d7/d3e/f94 x:0 0 0 2024-02-15T18:27:05.030 INFO:tasks.workunit.client.1.smithi191.stdout:6/316: rename d8/d9/l4f to d8/d9/d4a/l56 0 2024-02-15T18:27:05.031 INFO:tasks.workunit.client.1.smithi191.stdout:4/389: rmdir d2/d5/d7 39 2024-02-15T18:27:05.031 INFO:tasks.workunit.client.1.smithi191.stdout:4/390: write d2/d5/d7/d90/f6f [912535,127085] 0 2024-02-15T18:27:05.032 INFO:tasks.workunit.client.0.smithi042.stdout:3/735: symlink d1/d11/d14/d4d/d6c/d7b/lf3 0 2024-02-15T18:27:05.032 INFO:tasks.workunit.client.0.smithi042.stdout:3/736: dread - d1/fd5 zero size 2024-02-15T18:27:05.033 INFO:tasks.workunit.client.1.smithi191.stdout:6/317: mkdir d8/d9/d2b/d57 0 2024-02-15T18:27:05.034 INFO:tasks.workunit.client.1.smithi191.stdout:4/391: symlink d2/d5/d7/d3e/d63/d8c/l95 0 2024-02-15T18:27:05.036 INFO:tasks.workunit.client.1.smithi191.stdout:6/318: creat d8/d9/f58 x:0 0 0 2024-02-15T18:27:05.036 INFO:tasks.workunit.client.1.smithi191.stdout:6/319: fdatasync d8/d19/f40 0 2024-02-15T18:27:05.039 INFO:tasks.workunit.client.0.smithi042.stdout:7/760: unlink d10/d12/d65/da7/fc8 0 2024-02-15T18:27:05.040 INFO:tasks.workunit.client.1.smithi191.stdout:4/392: creat d2/d5/d86/f96 x:0 0 0 2024-02-15T18:27:05.040 INFO:tasks.workunit.client.0.smithi042.stdout:0/714: dwrite d4/d9/d34/d57/dc1/ff0 [0,4194304] 0 2024-02-15T18:27:05.040 INFO:tasks.workunit.client.0.smithi042.stdout:4/737: symlink da/d2b/de3/l102 0 2024-02-15T18:27:05.041 INFO:tasks.workunit.client.1.smithi191.stdout:4/393: write d2/d5/d7/d2f/d41/f73 [2117739,111020] 0 2024-02-15T18:27:05.042 INFO:tasks.workunit.client.1.smithi191.stdout:6/320: creat d8/d9/d2b/d57/f59 x:0 0 0 2024-02-15T18:27:05.042 INFO:tasks.workunit.client.1.smithi191.stdout:6/321: stat d8/f15 0 2024-02-15T18:27:05.042 INFO:tasks.workunit.client.0.smithi042.stdout:6/825: mkdir d9/d11/d19/d35/dc4/d123 0 2024-02-15T18:27:05.046 INFO:tasks.workunit.client.0.smithi042.stdout:5/768: truncate d0/dda/f85 2093645 0 2024-02-15T18:27:05.048 INFO:tasks.workunit.client.1.smithi191.stdout:1/459: sync 2024-02-15T18:27:05.048 INFO:tasks.workunit.client.1.smithi191.stdout:7/350: sync 2024-02-15T18:27:05.048 INFO:tasks.workunit.client.1.smithi191.stdout:3/289: sync 2024-02-15T18:27:05.048 INFO:tasks.workunit.client.1.smithi191.stdout:9/344: sync 2024-02-15T18:27:05.051 INFO:tasks.workunit.client.0.smithi042.stdout:8/670: sync 2024-02-15T18:27:05.052 INFO:tasks.workunit.client.0.smithi042.stdout:2/685: sync 2024-02-15T18:27:05.053 INFO:tasks.workunit.client.0.smithi042.stdout:9/737: symlink d6/da/d1d/d2d/d40/d95/l102 0 2024-02-15T18:27:05.054 INFO:tasks.workunit.client.1.smithi191.stdout:3/290: rmdir d6/d27/d4d/d5a/d62 0 2024-02-15T18:27:05.055 INFO:tasks.workunit.client.0.smithi042.stdout:1/752: mknod de/c11c 0 2024-02-15T18:27:05.056 INFO:tasks.workunit.client.1.smithi191.stdout:7/351: mknod d0/db/d65/d11/d1c/c7d 0 2024-02-15T18:27:05.057 INFO:tasks.workunit.client.0.smithi042.stdout:3/737: dwrite d1/d11/d14/d4d/d6c/f9b [0,4194304] 0 2024-02-15T18:27:05.058 INFO:tasks.workunit.client.0.smithi042.stdout:7/761: getdents d10/d49 0 2024-02-15T18:27:05.059 INFO:tasks.workunit.client.0.smithi042.stdout:4/738: dwrite da/d2b/d6e/f71 [0,4194304] 0 2024-02-15T18:27:05.059 INFO:tasks.workunit.client.0.smithi042.stdout:4/739: readlink da/d11/d27/lec 0 2024-02-15T18:27:05.059 INFO:tasks.workunit.client.0.smithi042.stdout:4/740: write da/f63 [972717,72239] 0 2024-02-15T18:27:05.059 INFO:tasks.workunit.client.0.smithi042.stdout:4/741: readlink da/d11/d29/dfd/ld0 0 2024-02-15T18:27:05.059 INFO:tasks.workunit.client.0.smithi042.stdout:4/742: write da/d11/f22 [1456193,70230] 0 2024-02-15T18:27:05.059 INFO:tasks.workunit.client.1.smithi191.stdout:1/460: mknod d0/d8/df/d90/d94/c98 0 2024-02-15T18:27:05.060 INFO:tasks.workunit.client.0.smithi042.stdout:0/715: truncate d4/d38/f9f 2217111 0 2024-02-15T18:27:05.061 INFO:tasks.workunit.client.0.smithi042.stdout:0/716: chown d4/d9/cc0 35916847 1 2024-02-15T18:27:05.061 INFO:tasks.workunit.client.0.smithi042.stdout:1/753: write de/d15/d67/f34 [2222310,98728] 0 2024-02-15T18:27:05.061 INFO:tasks.workunit.client.1.smithi191.stdout:9/345: symlink d0/d7/dc/d30/d3b/d56/d60/l7c 0 2024-02-15T18:27:05.062 INFO:tasks.workunit.client.0.smithi042.stdout:6/826: creat d9/d8d/f124 x:0 0 0 2024-02-15T18:27:05.064 INFO:tasks.workunit.client.1.smithi191.stdout:6/322: link d8/d19/c24 d8/d9/c5a 0 2024-02-15T18:27:05.065 INFO:tasks.workunit.client.1.smithi191.stdout:9/346: read d0/d7/da/d39/f5d [1291403,47623] 0 2024-02-15T18:27:05.066 INFO:tasks.workunit.client.1.smithi191.stdout:3/291: creat d6/d1b/d60/f68 x:0 0 0 2024-02-15T18:27:05.068 INFO:tasks.workunit.client.0.smithi042.stdout:5/769: symlink d0/d39/ddd/de0/l111 0 2024-02-15T18:27:05.068 INFO:tasks.workunit.client.0.smithi042.stdout:5/770: stat d0/d89/f97 0 2024-02-15T18:27:05.068 INFO:tasks.workunit.client.0.smithi042.stdout:5/771: chown d0/dda/d42/l44 2559498 1 2024-02-15T18:27:05.068 INFO:tasks.workunit.client.0.smithi042.stdout:5/772: dread - d0/dda/dc7/fcc zero size 2024-02-15T18:27:05.068 INFO:tasks.workunit.client.0.smithi042.stdout:5/773: read - d0/d38/d53/d64/ff3 zero size 2024-02-15T18:27:05.068 INFO:tasks.workunit.client.0.smithi042.stdout:5/774: truncate d0/ffc 292943 0 2024-02-15T18:27:05.068 INFO:tasks.workunit.client.0.smithi042.stdout:5/775: read - d0/dda/dc7/de1/dd8/ff4 zero size 2024-02-15T18:27:05.068 INFO:tasks.workunit.client.0.smithi042.stdout:5/776: stat d0/d39/d8d/d9b/lfa 0 2024-02-15T18:27:05.069 INFO:tasks.workunit.client.0.smithi042.stdout:5/777: chown d0/c8 263 1 2024-02-15T18:27:05.069 INFO:tasks.workunit.client.1.smithi191.stdout:1/461: rename d0/d8/df/d12/d33/f8c to d0/d8/df/d41/f99 0 2024-02-15T18:27:05.069 INFO:tasks.workunit.client.1.smithi191.stdout:1/462: write d0/d8/f30 [87494,48565] 0 2024-02-15T18:27:05.070 INFO:tasks.workunit.client.0.smithi042.stdout:8/671: mknod dd/de/da4/d67/d5a/cbd 0 2024-02-15T18:27:05.073 INFO:tasks.workunit.client.1.smithi191.stdout:9/347: link d0/d7/dc/d30/d3b/c52 d0/d29/c7d 0 2024-02-15T18:27:05.075 INFO:tasks.workunit.client.1.smithi191.stdout:6/323: dread d8/d19/f1d [0,4194304] 0 2024-02-15T18:27:05.076 INFO:tasks.workunit.client.0.smithi042.stdout:2/686: rename d0/d8/db to d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda 0 2024-02-15T18:27:05.077 INFO:tasks.workunit.client.1.smithi191.stdout:3/292: rmdir d6/d3b/d30 39 2024-02-15T18:27:05.077 INFO:tasks.workunit.client.1.smithi191.stdout:3/293: write d6/d3b/d42/d49/f59 [90774,2540] 0 2024-02-15T18:27:05.078 INFO:tasks.workunit.client.1.smithi191.stdout:9/348: read d0/d7/dc/d17/f25 [582204,30646] 0 2024-02-15T18:27:05.078 INFO:tasks.workunit.client.1.smithi191.stdout:9/349: write d0/d7/dc/d30/d4f/f6d [546275,19111] 0 2024-02-15T18:27:05.078 INFO:tasks.workunit.client.0.smithi042.stdout:7/762: symlink d10/d15/d39/d6c/d92/lfc 0 2024-02-15T18:27:05.079 INFO:tasks.workunit.client.0.smithi042.stdout:7/763: stat d10/d49/d6d/d78/fe1 0 2024-02-15T18:27:05.080 INFO:tasks.workunit.client.1.smithi191.stdout:1/463: truncate d0/d2f/d51/d10/f59 409312 0 2024-02-15T18:27:05.080 INFO:tasks.workunit.client.1.smithi191.stdout:1/464: readlink d0/d8/df/l89 0 2024-02-15T18:27:05.081 INFO:tasks.workunit.client.0.smithi042.stdout:0/717: dread d4/d38/f3a [0,4194304] 0 2024-02-15T18:27:05.081 INFO:tasks.workunit.client.0.smithi042.stdout:0/718: chown d4/d9/d34/dab/dc8/d6c 406 1 2024-02-15T18:27:05.081 INFO:tasks.workunit.client.0.smithi042.stdout:0/719: readlink d4/d9/d34/l94 0 2024-02-15T18:27:05.082 INFO:tasks.workunit.client.1.smithi191.stdout:3/294: rename d6/d3b/l15 to d6/d5d/l69 0 2024-02-15T18:27:05.082 INFO:tasks.workunit.client.1.smithi191.stdout:3/295: dread - d6/d27/d4d/d66/f5f zero size 2024-02-15T18:27:05.083 INFO:tasks.workunit.client.1.smithi191.stdout:9/350: symlink d0/d1d/d3f/l7e 0 2024-02-15T18:27:05.084 INFO:tasks.workunit.client.1.smithi191.stdout:1/465: mkdir d0/d8/df/d90/d94/d9a 0 2024-02-15T18:27:05.085 INFO:tasks.workunit.client.0.smithi042.stdout:1/754: link de/d15/l1f de/d54/dbf/dd0/d7c/dd6/de8/l11d 0 2024-02-15T18:27:05.086 INFO:tasks.workunit.client.1.smithi191.stdout:4/394: dwrite d2/f28 [0,4194304] 0 2024-02-15T18:27:05.087 INFO:tasks.workunit.client.0.smithi042.stdout:6/827: mkdir d9/d11/d46/df8/d125 0 2024-02-15T18:27:05.087 INFO:tasks.workunit.client.0.smithi042.stdout:6/828: chown d9/f72 320066 1 2024-02-15T18:27:05.087 INFO:tasks.workunit.client.0.smithi042.stdout:6/829: chown d9/d11/d19/d1c/d31/d9b/db4/f117 10 1 2024-02-15T18:27:05.087 INFO:tasks.workunit.client.0.smithi042.stdout:6/830: read d9/d11/d19/d1c/d31/d4f/f68 [198272,63353] 0 2024-02-15T18:27:05.087 INFO:tasks.workunit.client.1.smithi191.stdout:3/296: mknod d6/d3b/d30/c6a 0 2024-02-15T18:27:05.088 INFO:tasks.workunit.client.0.smithi042.stdout:5/778: mkdir d0/dda/dc7/d52/dad/d112 0 2024-02-15T18:27:05.088 INFO:tasks.workunit.client.1.smithi191.stdout:9/351: mknod d0/d7/dc/d30/d4f/c7f 0 2024-02-15T18:27:05.089 INFO:tasks.workunit.client.0.smithi042.stdout:5/779: read d0/dda/f56 [1059628,118144] 0 2024-02-15T18:27:05.089 INFO:tasks.workunit.client.0.smithi042.stdout:8/672: unlink dd/de/da4/d67/l77 0 2024-02-15T18:27:05.091 INFO:tasks.workunit.client.1.smithi191.stdout:1/466: symlink d0/d8/df/d12/d33/d49/l9b 0 2024-02-15T18:27:05.091 INFO:tasks.workunit.client.1.smithi191.stdout:1/467: readlink d0/d8/df/d12/d2a/d2b/d78/l80 0 2024-02-15T18:27:05.093 INFO:tasks.workunit.client.1.smithi191.stdout:4/395: mknod d2/d5/d7/d90/c97 0 2024-02-15T18:27:05.093 INFO:tasks.workunit.client.0.smithi042.stdout:9/738: dwrite d6/da/d2f/d54/f64 [0,4194304] 0 2024-02-15T18:27:05.094 INFO:tasks.workunit.client.0.smithi042.stdout:2/687: creat d0/d8/d20/d3f/fdb x:0 0 0 2024-02-15T18:27:05.094 INFO:tasks.workunit.client.0.smithi042.stdout:2/688: dread - d0/d8/d10/d7b/fa4 zero size 2024-02-15T18:27:05.094 INFO:tasks.workunit.client.1.smithi191.stdout:1/468: mkdir d0/d2f/d51/d10/d9c 0 2024-02-15T18:27:05.095 INFO:tasks.workunit.client.0.smithi042.stdout:3/738: dwrite d1/de/d19/f1f [4194304,4194304] 0 2024-02-15T18:27:05.095 INFO:tasks.workunit.client.1.smithi191.stdout:1/469: read d0/d2f/d51/f14 [529677,77796] 0 2024-02-15T18:27:05.095 INFO:tasks.workunit.client.1.smithi191.stdout:1/470: stat d0/d2f/d51/d10/f25 0 2024-02-15T18:27:05.096 INFO:tasks.workunit.client.1.smithi191.stdout:7/352: dwrite d0/db/d26/d5f/d77/d35/d42/f6c [0,4194304] 0 2024-02-15T18:27:05.096 INFO:tasks.workunit.client.0.smithi042.stdout:7/764: creat d10/d15/d8f/da5/dad/de0/de2/ffd x:0 0 0 2024-02-15T18:27:05.097 INFO:tasks.workunit.client.1.smithi191.stdout:6/324: dwrite d8/d19/f1f [0,4194304] 0 2024-02-15T18:27:05.097 INFO:tasks.workunit.client.1.smithi191.stdout:9/352: fdatasync d0/d7/da/d39/f75 0 2024-02-15T18:27:05.097 INFO:tasks.workunit.client.1.smithi191.stdout:4/396: truncate d2/d5/d7/d2f/f75 2488708 0 2024-02-15T18:27:05.097 INFO:tasks.workunit.client.1.smithi191.stdout:7/353: write d0/db/d26/d5f/d77/d35/f50 [300820,27756] 0 2024-02-15T18:27:05.097 INFO:tasks.workunit.client.1.smithi191.stdout:7/354: chown d0/db/d26/d5f/d77/df/d5d 47701 1 2024-02-15T18:27:05.104 INFO:tasks.workunit.client.0.smithi042.stdout:0/720: rename d4/d38/d77/d2e/c2b to d4/d9/d62/d74/cf2 0 2024-02-15T18:27:05.107 INFO:tasks.workunit.client.0.smithi042.stdout:1/755: mkdir de/d15/d67/d63/d11e 0 2024-02-15T18:27:05.109 INFO:tasks.workunit.client.0.smithi042.stdout:6/831: creat d9/d11/d19/d35/d55/d92/dfc/d121/f126 x:0 0 0 2024-02-15T18:27:05.110 INFO:tasks.workunit.client.0.smithi042.stdout:6/832: fdatasync d9/f76 0 2024-02-15T18:27:05.110 INFO:tasks.workunit.client.1.smithi191.stdout:3/297: dwrite d6/d3b/d30/f45 [0,4194304] 0 2024-02-15T18:27:05.117 INFO:tasks.workunit.client.0.smithi042.stdout:6/833: dread d9/d11/d19/d1c/d31/f3a [0,4194304] 0 2024-02-15T18:27:05.119 INFO:tasks.workunit.client.1.smithi191.stdout:6/325: rmdir d8/d9 39 2024-02-15T18:27:05.119 INFO:tasks.workunit.client.1.smithi191.stdout:6/326: truncate d8/d9/d2b/f42 59273 0 2024-02-15T18:27:05.121 INFO:tasks.workunit.client.1.smithi191.stdout:1/471: dwrite d0/d8/df/f2c [0,4194304] 0 2024-02-15T18:27:05.122 INFO:tasks.workunit.client.1.smithi191.stdout:1/472: write d0/d8/df/d12/d2a/d2b/f5a [171813,18614] 0 2024-02-15T18:27:05.124 INFO:tasks.workunit.client.0.smithi042.stdout:7/765: dwrite d10/f21 [0,4194304] 0 2024-02-15T18:27:05.127 INFO:tasks.workunit.client.0.smithi042.stdout:5/780: rmdir d0/d14/ddb 39 2024-02-15T18:27:05.139 INFO:tasks.workunit.client.1.smithi191.stdout:9/353: rename d0/d7/da/d39 to d0/d1d/d3f/d80 0 2024-02-15T18:27:05.139 INFO:tasks.workunit.client.1.smithi191.stdout:9/354: fdatasync d0/d7/dc/d30/d3b/f55 0 2024-02-15T18:27:05.139 INFO:tasks.workunit.client.1.smithi191.stdout:9/355: fdatasync d0/d7/dc/d17/d33/f43 0 2024-02-15T18:27:05.140 INFO:tasks.workunit.client.1.smithi191.stdout:9/356: fsync d0/d7/dc/d17/f25 0 2024-02-15T18:27:05.141 INFO:tasks.workunit.client.1.smithi191.stdout:4/397: creat d2/d5/d7/d2f/d41/f98 x:0 0 0 2024-02-15T18:27:05.141 INFO:tasks.workunit.client.1.smithi191.stdout:4/398: readlink d2/d5/d7/d3e/d88/l7e 0 2024-02-15T18:27:05.142 INFO:tasks.workunit.client.0.smithi042.stdout:7/766: dread d10/f21 [0,4194304] 0 2024-02-15T18:27:05.142 INFO:tasks.workunit.client.0.smithi042.stdout:6/834: dwrite d9/d3e/f3b [0,4194304] 0 2024-02-15T18:27:05.144 INFO:tasks.workunit.client.0.smithi042.stdout:9/739: symlink d6/dd/d9/d59/dbf/l103 0 2024-02-15T18:27:05.144 INFO:tasks.workunit.client.0.smithi042.stdout:7/767: dread d10/d49/ff4 [0,4194304] 0 2024-02-15T18:27:05.144 INFO:tasks.workunit.client.0.smithi042.stdout:9/740: chown d6/da/d7b/f92 5195 1 2024-02-15T18:27:05.144 INFO:tasks.workunit.client.0.smithi042.stdout:9/741: chown d6/f8 18 1 2024-02-15T18:27:05.144 INFO:tasks.workunit.client.1.smithi191.stdout:7/355: dwrite d0/db/d65/dc/d4b/f59 [0,4194304] 0 2024-02-15T18:27:05.144 INFO:tasks.workunit.client.1.smithi191.stdout:7/356: dread - d0/db/d26/d5f/d77/d35/d42/f7b zero size 2024-02-15T18:27:05.148 INFO:tasks.workunit.client.0.smithi042.stdout:5/781: dread d0/dda/dc7/de1/f35 [0,4194304] 0 2024-02-15T18:27:05.148 INFO:tasks.workunit.client.1.smithi191.stdout:3/298: creat d6/d3b/d42/d49/d54/f6b x:0 0 0 2024-02-15T18:27:05.149 INFO:tasks.workunit.client.0.smithi042.stdout:2/689: symlink d0/d8/d10/d31/d79/ldc 0 2024-02-15T18:27:05.149 INFO:tasks.workunit.client.0.smithi042.stdout:2/690: chown d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a 364 1 2024-02-15T18:27:05.151 INFO:tasks.workunit.client.1.smithi191.stdout:1/473: symlink d0/l9d 0 2024-02-15T18:27:05.152 INFO:tasks.workunit.client.0.smithi042.stdout:9/742: read d6/da/d1d/d2d/d40/d48/fba [1800577,74549] 0 2024-02-15T18:27:05.153 INFO:tasks.workunit.client.1.smithi191.stdout:4/399: creat d2/d5/d7/d2f/d41/f99 x:0 0 0 2024-02-15T18:27:05.153 INFO:tasks.workunit.client.1.smithi191.stdout:4/400: write d2/d5/d7/d2f/d41/f8f [665141,48279] 0 2024-02-15T18:27:05.160 INFO:tasks.workunit.client.1.smithi191.stdout:7/357: mknod d0/db/d65/dc/c7e 0 2024-02-15T18:27:05.166 INFO:tasks.workunit.client.1.smithi191.stdout:9/357: dwrite d0/d7/dc/d17/d33/f4d [0,4194304] 0 2024-02-15T18:27:05.166 INFO:tasks.workunit.client.1.smithi191.stdout:1/474: creat d0/d8/df/d41/f9e x:0 0 0 2024-02-15T18:27:05.167 INFO:tasks.workunit.client.0.smithi042.stdout:1/756: chown de/d54/dbf/dd0/fd4 3 1 2024-02-15T18:27:05.170 INFO:tasks.workunit.client.1.smithi191.stdout:4/401: creat d2/d5/d7/d3e/d88/f9a x:0 0 0 2024-02-15T18:27:05.174 INFO:tasks.workunit.client.0.smithi042.stdout:6/835: link d9/d3e/f64 d9/d11/d19/d35/dc4/f127 0 2024-02-15T18:27:05.174 INFO:tasks.workunit.client.0.smithi042.stdout:6/836: write d9/d3e/f96 [775022,109836] 0 2024-02-15T18:27:05.175 INFO:tasks.workunit.client.0.smithi042.stdout:5/782: symlink d0/dda/dc7/de1/l113 0 2024-02-15T18:27:05.176 INFO:tasks.workunit.client.1.smithi191.stdout:7/358: creat d0/db/d65/dc/d4b/f7f x:0 0 0 2024-02-15T18:27:05.177 INFO:tasks.workunit.client.0.smithi042.stdout:2/691: getdents d0/d8/d20/d3f 0 2024-02-15T18:27:05.180 INFO:tasks.workunit.client.1.smithi191.stdout:3/299: dwrite d6/d3b/d42/f4a [4194304,4194304] 0 2024-02-15T18:27:05.184 INFO:tasks.workunit.client.0.smithi042.stdout:9/743: link d6/da/d1d/d2d/d40/d5f/lab d6/da/d2f/d6c/dd8/l104 0 2024-02-15T18:27:05.184 INFO:tasks.workunit.client.0.smithi042.stdout:9/744: dread - d6/da/d2f/d55/d73/d8d/dcf/de1/ffa zero size 2024-02-15T18:27:05.184 INFO:tasks.workunit.client.0.smithi042.stdout:6/837: creat d9/d11/d19/d1c/d31/d53/dbe/f128 x:0 0 0 2024-02-15T18:27:05.184 INFO:tasks.workunit.client.0.smithi042.stdout:1/757: creat de/d15/d1b/d44/f11f x:0 0 0 2024-02-15T18:27:05.184 INFO:tasks.workunit.client.0.smithi042.stdout:2/692: symlink d0/d8/d20/d7d/d7e/dc7/d8b/ldd 0 2024-02-15T18:27:05.184 INFO:tasks.workunit.client.0.smithi042.stdout:2/693: chown d0/d8/d20/d7d/d94/c59 970 1 2024-02-15T18:27:05.184 INFO:tasks.workunit.client.0.smithi042.stdout:2/694: dread - d0/d8/d20/d3f/fdb zero size 2024-02-15T18:27:05.187 INFO:tasks.workunit.client.1.smithi191.stdout:3/300: write d6/fa [4696892,24597] 0 2024-02-15T18:27:05.187 INFO:tasks.workunit.client.1.smithi191.stdout:3/301: stat d6/d27/d4d/l63 0 2024-02-15T18:27:05.188 INFO:tasks.workunit.client.0.smithi042.stdout:6/838: creat d9/d11/d19/d1c/f129 x:0 0 0 2024-02-15T18:27:05.188 INFO:tasks.workunit.client.0.smithi042.stdout:1/758: symlink de/d15/l120 0 2024-02-15T18:27:05.188 INFO:tasks.workunit.client.0.smithi042.stdout:1/759: truncate de/d15/d2c/d51/d70/faf 382866 0 2024-02-15T18:27:05.189 INFO:tasks.workunit.client.0.smithi042.stdout:1/760: write de/d15/d67/d46/f59 [6140723,45492] 0 2024-02-15T18:27:05.190 INFO:tasks.workunit.client.0.smithi042.stdout:2/695: symlink d0/d8/d20/d7d/lde 0 2024-02-15T18:27:05.190 INFO:tasks.workunit.client.0.smithi042.stdout:2/696: dread - d0/f63 zero size 2024-02-15T18:27:05.190 INFO:tasks.workunit.client.0.smithi042.stdout:2/697: rename d0/d8/d20/d7d to d0/d8/d20/d7d/d7e/dc7/d8b/dd7/ddf 22 2024-02-15T18:27:05.190 INFO:tasks.workunit.client.0.smithi042.stdout:2/698: readlink d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/l1f 0 2024-02-15T18:27:05.192 INFO:tasks.workunit.client.0.smithi042.stdout:6/839: write d9/d11/d19/d35/d55/f65 [1122953,96257] 0 2024-02-15T18:27:05.193 INFO:tasks.workunit.client.0.smithi042.stdout:7/768: dwrite d10/d15/d8f/da5/dad/fd0 [0,4194304] 0 2024-02-15T18:27:05.193 INFO:tasks.workunit.client.0.smithi042.stdout:7/769: dread - d10/d49/d6d/db6/ff0 zero size 2024-02-15T18:27:05.195 INFO:tasks.workunit.client.1.smithi191.stdout:6/327: dwrite d8/d19/f26 [0,4194304] 0 2024-02-15T18:27:05.196 INFO:tasks.workunit.client.0.smithi042.stdout:1/761: creat de/d6d/d9e/d109/f121 x:0 0 0 2024-02-15T18:27:05.199 INFO:tasks.workunit.client.0.smithi042.stdout:3/739: sync 2024-02-15T18:27:05.199 INFO:tasks.workunit.client.0.smithi042.stdout:0/721: sync 2024-02-15T18:27:05.199 INFO:tasks.workunit.client.0.smithi042.stdout:5/783: dwrite d0/d12/d2a/fd7 [0,4194304] 0 2024-02-15T18:27:05.199 INFO:tasks.workunit.client.0.smithi042.stdout:4/743: sync 2024-02-15T18:27:05.199 INFO:tasks.workunit.client.0.smithi042.stdout:8/673: sync 2024-02-15T18:27:05.199 INFO:tasks.workunit.client.0.smithi042.stdout:4/744: stat da/d11/d29/dfd/d66/d78/d53/d82/dc0 0 2024-02-15T18:27:05.200 INFO:tasks.workunit.client.0.smithi042.stdout:0/722: write d4/d38/d77/fb4 [140737,90619] 0 2024-02-15T18:27:05.200 INFO:tasks.workunit.client.0.smithi042.stdout:2/699: creat d0/d8/d20/d7d/d7e/fe0 x:0 0 0 2024-02-15T18:27:05.200 INFO:tasks.workunit.client.0.smithi042.stdout:2/700: fdatasync d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/f43 0 2024-02-15T18:27:05.201 INFO:tasks.workunit.client.1.smithi191.stdout:6/328: read d8/d19/f26 [1497686,118093] 0 2024-02-15T18:27:05.201 INFO:tasks.workunit.client.1.smithi191.stdout:6/329: chown d8/d9/d2b/l43 233 1 2024-02-15T18:27:05.203 INFO:tasks.workunit.client.0.smithi042.stdout:6/840: truncate d9/d3e/f64 2029291 0 2024-02-15T18:27:05.205 INFO:tasks.workunit.client.1.smithi191.stdout:1/475: dwrite d0/d8/df/d12/f3f [0,4194304] 0 2024-02-15T18:27:05.205 INFO:tasks.workunit.client.1.smithi191.stdout:1/476: chown d0/d8/df/d12/d2a/d2b/l76 604 1 2024-02-15T18:27:05.210 INFO:tasks.workunit.client.1.smithi191.stdout:3/302: getdents d6/d3b/d30 0 2024-02-15T18:27:05.210 INFO:tasks.workunit.client.1.smithi191.stdout:3/303: stat d6/l12 0 2024-02-15T18:27:05.212 INFO:tasks.workunit.client.0.smithi042.stdout:7/770: creat d10/d12/d23/d51/dc9/ffe x:0 0 0 2024-02-15T18:27:05.214 INFO:tasks.workunit.client.1.smithi191.stdout:7/359: dwrite d0/db/f2d [8388608,4194304] 0 2024-02-15T18:27:05.216 INFO:tasks.workunit.client.0.smithi042.stdout:9/745: dwrite d6/da/d2f/d55/d73/d8d/fb2 [0,4194304] 0 2024-02-15T18:27:05.216 INFO:tasks.workunit.client.0.smithi042.stdout:9/746: write d6/d9d/fee [403236,87622] 0 2024-02-15T18:27:05.216 INFO:tasks.workunit.client.0.smithi042.stdout:9/747: readlink d6/l84 0 2024-02-15T18:27:05.216 INFO:tasks.workunit.client.0.smithi042.stdout:9/748: write d6/da/d2f/d69/f5b [944539,122867] 0 2024-02-15T18:27:05.217 INFO:tasks.workunit.client.1.smithi191.stdout:6/330: dwrite d8/d9/d4a/d4c/f53 [0,4194304] 0 2024-02-15T18:27:05.267 INFO:tasks.workunit.client.0.smithi042.stdout:3/740: mkdir d1/d11/d14/d4d/d6c/df4 0 2024-02-15T18:27:05.267 INFO:tasks.workunit.client.0.smithi042.stdout:3/741: chown d1/de/f2d 5 1 2024-02-15T18:27:05.267 INFO:tasks.workunit.client.0.smithi042.stdout:3/742: chown d1/de/d1a/lcc 1583269 1 2024-02-15T18:27:05.268 INFO:tasks.workunit.client.1.smithi191.stdout:9/358: rename d0/c5 to d0/d7/dc/d32/c81 0 2024-02-15T18:27:05.269 INFO:tasks.workunit.client.1.smithi191.stdout:4/402: mknod d2/d5/c9b 0 2024-02-15T18:27:05.271 INFO:tasks.workunit.client.1.smithi191.stdout:6/331: dwrite d8/d9/f36 [4194304,4194304] 0 2024-02-15T18:27:05.273 INFO:tasks.workunit.client.0.smithi042.stdout:5/784: mknod d0/d14/d88/da1/c114 0 2024-02-15T18:27:05.273 INFO:tasks.workunit.client.0.smithi042.stdout:5/785: write d0/dda/dc7/d52/da0/fe2 [729017,39053] 0 2024-02-15T18:27:05.276 INFO:tasks.workunit.client.0.smithi042.stdout:8/674: creat dd/de/da4/d67/d6f/fbe x:0 0 0 2024-02-15T18:27:05.280 INFO:tasks.workunit.client.1.smithi191.stdout:2/355: sync 2024-02-15T18:27:05.282 INFO:tasks.workunit.client.0.smithi042.stdout:4/745: rename da/d11/d29/dfd to da/d11/d85/db5/d103 0 2024-02-15T18:27:05.282 INFO:tasks.workunit.client.0.smithi042.stdout:4/746: dread - da/d11/d27/fdc zero size 2024-02-15T18:27:05.283 INFO:tasks.workunit.client.0.smithi042.stdout:4/747: dread da/d3a/dc9/fd7 [0,4194304] 0 2024-02-15T18:27:05.283 INFO:tasks.workunit.client.0.smithi042.stdout:4/748: write da/d11/f22 [2352296,124371] 0 2024-02-15T18:27:05.283 INFO:tasks.workunit.client.0.smithi042.stdout:4/749: write da/d3a/dc9/fd2 [274504,44912] 0 2024-02-15T18:27:05.283 INFO:tasks.workunit.client.1.smithi191.stdout:7/360: creat d0/f80 x:0 0 0 2024-02-15T18:27:05.284 INFO:tasks.workunit.client.0.smithi042.stdout:0/723: mknod d4/d38/d39/d8e/cf3 0 2024-02-15T18:27:05.284 INFO:tasks.workunit.client.0.smithi042.stdout:0/724: dread - d4/d38/d77/d2e/dae/ddf/fe8 zero size 2024-02-15T18:27:05.284 INFO:tasks.workunit.client.0.smithi042.stdout:0/725: fdatasync d4/d38/f3a 0 2024-02-15T18:27:05.284 INFO:tasks.workunit.client.0.smithi042.stdout:0/726: write d4/d9/feb [67656,121072] 0 2024-02-15T18:27:05.286 INFO:tasks.workunit.client.1.smithi191.stdout:0/264: sync 2024-02-15T18:27:05.286 INFO:tasks.workunit.client.1.smithi191.stdout:5/185: sync 2024-02-15T18:27:05.287 INFO:tasks.workunit.client.0.smithi042.stdout:2/701: stat d0/d8/d20/d41/c9c 0 2024-02-15T18:27:05.287 INFO:tasks.workunit.client.0.smithi042.stdout:2/702: readlink d0/l9b 0 2024-02-15T18:27:05.290 INFO:tasks.workunit.client.1.smithi191.stdout:4/403: dwrite d2/d5/d7/d3e/f94 [0,4194304] 0 2024-02-15T18:27:05.290 INFO:tasks.workunit.client.1.smithi191.stdout:3/304: rename d6/d3b/d42/c53 to d6/d1b/d60/c6c 0 2024-02-15T18:27:05.291 INFO:tasks.workunit.client.0.smithi042.stdout:7/771: creat d10/d15/d39/d6c/dba/fff x:0 0 0 2024-02-15T18:27:05.291 INFO:tasks.workunit.client.0.smithi042.stdout:5/786: dwrite d0/ffc [0,4194304] 0 2024-02-15T18:27:05.292 INFO:tasks.workunit.client.1.smithi191.stdout:9/359: mknod d0/d7/dc/d30/d3b/d56/c82 0 2024-02-15T18:27:05.292 INFO:tasks.workunit.client.0.smithi042.stdout:7/772: write d10/d12/d23/f55 [356824,89190] 0 2024-02-15T18:27:05.292 INFO:tasks.workunit.client.0.smithi042.stdout:7/773: read - d10/d15/d8f/da5/dad/de0/de2/fef zero size 2024-02-15T18:27:05.295 INFO:tasks.workunit.client.1.smithi191.stdout:8/483: sync 2024-02-15T18:27:05.295 INFO:tasks.workunit.client.1.smithi191.stdout:8/484: readlink d3/l30 0 2024-02-15T18:27:05.297 INFO:tasks.workunit.client.1.smithi191.stdout:6/332: getdents d8/d9/d2b 0 2024-02-15T18:27:05.301 INFO:tasks.workunit.client.1.smithi191.stdout:7/361: mknod d0/db/d26/d5f/d77/d1d/d54/c81 0 2024-02-15T18:27:05.303 INFO:tasks.workunit.client.0.smithi042.stdout:9/749: creat d6/dd/d9/da1/f105 x:0 0 0 2024-02-15T18:27:05.304 INFO:tasks.workunit.client.1.smithi191.stdout:4/404: unlink d2/d5/d7/f25 0 2024-02-15T18:27:05.306 INFO:tasks.workunit.client.0.smithi042.stdout:9/750: read d6/da/d2f/d6c/f71 [1861733,59790] 0 2024-02-15T18:27:05.306 INFO:tasks.workunit.client.0.smithi042.stdout:9/751: fdatasync d6/f2e 0 2024-02-15T18:27:05.307 INFO:tasks.workunit.client.0.smithi042.stdout:3/743: truncate d1/de/d1a/f6f 96524 0 2024-02-15T18:27:05.309 INFO:tasks.workunit.client.1.smithi191.stdout:3/305: unlink d6/d27/d4d/d66/f5f 0 2024-02-15T18:27:05.309 INFO:tasks.workunit.client.1.smithi191.stdout:3/306: dread - d6/d3b/d42/d49/f3d zero size 2024-02-15T18:27:05.311 INFO:tasks.workunit.client.0.smithi042.stdout:8/675: mkdir dd/de/d33/dbf 0 2024-02-15T18:27:05.311 INFO:tasks.workunit.client.1.smithi191.stdout:2/356: dread df/f1b [0,4194304] 0 2024-02-15T18:27:05.312 INFO:tasks.workunit.client.1.smithi191.stdout:0/265: dwrite d7/da/d17/f40 [0,4194304] 0 2024-02-15T18:27:05.312 INFO:tasks.workunit.client.1.smithi191.stdout:6/333: mkdir d8/d9/d2b/d5b 0 2024-02-15T18:27:05.312 INFO:tasks.workunit.client.1.smithi191.stdout:0/266: write d7/f39 [1215300,53652] 0 2024-02-15T18:27:05.315 INFO:tasks.workunit.client.0.smithi042.stdout:6/841: rename d9/d3e/f75 to d9/d11/d19/d1c/f12a 0 2024-02-15T18:27:05.319 INFO:tasks.workunit.client.1.smithi191.stdout:8/485: dwrite d3/d2c/d1a/d36/f6b [0,4194304] 0 2024-02-15T18:27:05.320 INFO:tasks.workunit.client.0.smithi042.stdout:1/762: sync 2024-02-15T18:27:05.320 INFO:tasks.workunit.client.0.smithi042.stdout:1/763: read de/d15/d67/f2a [1008959,30846] 0 2024-02-15T18:27:05.321 INFO:tasks.workunit.client.1.smithi191.stdout:7/362: creat d0/db/d65/f82 x:0 0 0 2024-02-15T18:27:05.322 INFO:tasks.workunit.client.1.smithi191.stdout:9/360: dwrite d0/f51 [0,4194304] 0 2024-02-15T18:27:05.322 INFO:tasks.workunit.client.1.smithi191.stdout:4/405: mknod d2/d5/d86/c9c 0 2024-02-15T18:27:05.323 INFO:tasks.workunit.client.0.smithi042.stdout:4/750: rmdir da/d11/da4 39 2024-02-15T18:27:05.325 INFO:tasks.workunit.client.1.smithi191.stdout:5/186: rename d0/d18/f1e to d0/d14/d28/f3f 0 2024-02-15T18:27:05.327 INFO:tasks.workunit.client.0.smithi042.stdout:0/727: mkdir d4/d38/d39/df4 0 2024-02-15T18:27:05.329 INFO:tasks.workunit.client.1.smithi191.stdout:3/307: creat d6/d3b/d42/d49/f6d x:0 0 0 2024-02-15T18:27:05.331 INFO:tasks.workunit.client.0.smithi042.stdout:2/703: creat d0/d8/d20/d7d/d7e/dc7/fe1 x:0 0 0 2024-02-15T18:27:05.331 INFO:tasks.workunit.client.0.smithi042.stdout:2/704: read - d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/fb4 zero size 2024-02-15T18:27:05.331 INFO:tasks.workunit.client.0.smithi042.stdout:2/705: read - d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/fb4 zero size 2024-02-15T18:27:05.331 INFO:tasks.workunit.client.0.smithi042.stdout:2/706: chown d0/d8/d10/d5a/f88 9439 1 2024-02-15T18:27:05.331 INFO:tasks.workunit.client.0.smithi042.stdout:2/707: chown d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/c6d 58998 1 2024-02-15T18:27:05.332 INFO:tasks.workunit.client.1.smithi191.stdout:2/357: symlink df/d12/d1e/l7b 0 2024-02-15T18:27:05.332 INFO:tasks.workunit.client.1.smithi191.stdout:2/358: fdatasync df/d12/d19/d1a/d35/d47/f51 0 2024-02-15T18:27:05.332 INFO:tasks.workunit.client.1.smithi191.stdout:2/359: truncate df/d12/f55 169430 0 2024-02-15T18:27:05.334 INFO:tasks.workunit.client.0.smithi042.stdout:5/787: getdents d0/d38/d53/d91 0 2024-02-15T18:27:05.335 INFO:tasks.workunit.client.1.smithi191.stdout:6/334: truncate d8/d19/f26 1403593 0 2024-02-15T18:27:05.335 INFO:tasks.workunit.client.1.smithi191.stdout:6/335: truncate d8/f45 152624 0 2024-02-15T18:27:05.336 INFO:tasks.workunit.client.0.smithi042.stdout:7/774: mknod d10/c100 0 2024-02-15T18:27:05.343 INFO:tasks.workunit.client.0.smithi042.stdout:9/752: unlink d6/da/d75/f93 0 2024-02-15T18:27:05.344 INFO:tasks.workunit.client.1.smithi191.stdout:7/363: rmdir d0/db/d26/d5f 39 2024-02-15T18:27:05.344 INFO:tasks.workunit.client.1.smithi191.stdout:7/364: write d0/db/d65/d11/d1c/f45 [737053,121822] 0 2024-02-15T18:27:05.346 INFO:tasks.workunit.client.0.smithi042.stdout:3/744: creat d1/d11/d14/d4d/d73/ff5 x:0 0 0 2024-02-15T18:27:05.346 INFO:tasks.workunit.client.1.smithi191.stdout:9/361: unlink d0/f1 0 2024-02-15T18:27:05.347 INFO:tasks.workunit.client.1.smithi191.stdout:9/362: chown d0/f51 39 1 2024-02-15T18:27:05.347 INFO:tasks.workunit.client.1.smithi191.stdout:9/363: chown d0/d7/dc/d30/d3b/c4b 0 1 2024-02-15T18:27:05.347 INFO:tasks.workunit.client.0.smithi042.stdout:8/676: mkdir dd/de/dc0 0 2024-02-15T18:27:05.348 INFO:tasks.workunit.client.0.smithi042.stdout:8/677: write dd/d11/d75/f9d [430648,77759] 0 2024-02-15T18:27:05.349 INFO:tasks.workunit.client.0.smithi042.stdout:1/764: mknod de/d54/dbf/dd0/d7c/dd6/de8/c122 0 2024-02-15T18:27:05.351 INFO:tasks.workunit.client.1.smithi191.stdout:8/486: rename d3/f29 to d3/d2c/d1a/d23/d3e/fb4 0 2024-02-15T18:27:05.352 INFO:tasks.workunit.client.1.smithi191.stdout:5/187: creat d0/d14/d28/d31/d38/f40 x:0 0 0 2024-02-15T18:27:05.354 INFO:tasks.workunit.client.0.smithi042.stdout:5/788: dwrite d0/d14/f9c [0,4194304] 0 2024-02-15T18:27:05.359 INFO:tasks.workunit.client.1.smithi191.stdout:3/308: link d6/d3b/d30/c6a d6/d3b/d42/d49/c6e 0 2024-02-15T18:27:05.360 INFO:tasks.workunit.client.1.smithi191.stdout:2/360: creat df/d12/f7c x:0 0 0 2024-02-15T18:27:05.362 INFO:tasks.workunit.client.0.smithi042.stdout:4/751: mknod da/d11/d85/db5/d103/d66/d78/d53/d82/c104 0 2024-02-15T18:27:05.362 INFO:tasks.workunit.client.1.smithi191.stdout:7/365: mknod d0/db/d26/d5f/d77/c83 0 2024-02-15T18:27:05.363 INFO:tasks.workunit.client.1.smithi191.stdout:7/366: dread - d0/db/d65/dc/d4b/f5b zero size 2024-02-15T18:27:05.365 INFO:tasks.workunit.client.1.smithi191.stdout:4/406: rename d2/d5/d7/d90/d52/f62 to d2/d5/d86/f9d 0 2024-02-15T18:27:05.367 INFO:tasks.workunit.client.0.smithi042.stdout:0/728: unlink d4/d9/d34/d8b/fd4 0 2024-02-15T18:27:05.368 INFO:tasks.workunit.client.0.smithi042.stdout:0/729: write d4/d9/d34/d57/dc1/fe0 [846155,46344] 0 2024-02-15T18:27:05.371 INFO:tasks.workunit.client.1.smithi191.stdout:7/367: dread d0/db/d26/d5f/d77/d1d/f4c [0,4194304] 0 2024-02-15T18:27:05.372 INFO:tasks.workunit.client.1.smithi191.stdout:4/407: dread d2/f2c [0,4194304] 0 2024-02-15T18:27:05.376 INFO:tasks.workunit.client.1.smithi191.stdout:2/361: mknod df/d12/d19/d1a/d52/c7d 0 2024-02-15T18:27:05.376 INFO:tasks.workunit.client.0.smithi042.stdout:2/708: creat d0/d8/d20/d7d/d7e/dc7/d8b/dd7/fe2 x:0 0 0 2024-02-15T18:27:05.376 INFO:tasks.workunit.client.0.smithi042.stdout:2/709: chown d0/d8/f70 14832 1 2024-02-15T18:27:05.376 INFO:tasks.workunit.client.0.smithi042.stdout:0/730: dread d4/d9/d34/dab/dc8/f87 [4194304,4194304] 0 2024-02-15T18:27:05.376 INFO:tasks.workunit.client.0.smithi042.stdout:7/775: creat d10/d43/d57/f101 x:0 0 0 2024-02-15T18:27:05.377 INFO:tasks.workunit.client.1.smithi191.stdout:6/336: symlink d8/d9/l5c 0 2024-02-15T18:27:05.378 INFO:tasks.workunit.client.0.smithi042.stdout:9/753: symlink d6/da/d1d/l106 0 2024-02-15T18:27:05.380 INFO:tasks.workunit.client.1.smithi191.stdout:9/364: rename d0/d7/dc/le to d0/d1d/l83 0 2024-02-15T18:27:05.381 INFO:tasks.workunit.client.1.smithi191.stdout:9/365: fdatasync d0/d29/d48/f5b 0 2024-02-15T18:27:05.381 INFO:tasks.workunit.client.1.smithi191.stdout:4/408: chown d2/d5/d7/d2f/c33 57 1 2024-02-15T18:27:05.382 INFO:tasks.workunit.client.1.smithi191.stdout:4/409: read d2/f3 [371290,70841] 0 2024-02-15T18:27:05.382 INFO:tasks.workunit.client.1.smithi191.stdout:4/410: truncate d2/d5/d14/d47/f84 543100 0 2024-02-15T18:27:05.383 INFO:tasks.workunit.client.0.smithi042.stdout:4/752: dwrite da/d11/d29/d4a/d4f/d90/fae [0,4194304] 0 2024-02-15T18:27:05.384 INFO:tasks.workunit.client.0.smithi042.stdout:4/753: read da/d11/d67/f6f [3587970,11719] 0 2024-02-15T18:27:05.384 INFO:tasks.workunit.client.0.smithi042.stdout:4/754: truncate da/d11/d27/fdc 331293 0 2024-02-15T18:27:05.384 INFO:tasks.workunit.client.0.smithi042.stdout:8/678: rename dd/de/d33/d66/lbb to dd/de/da4/d67/d6f/d78/lc1 0 2024-02-15T18:27:05.384 INFO:tasks.workunit.client.1.smithi191.stdout:7/368: symlink d0/db/d65/dc/l84 0 2024-02-15T18:27:05.385 INFO:tasks.workunit.client.1.smithi191.stdout:7/369: write d0/db/d65/d11/d5a/d66/f79 [425297,32699] 0 2024-02-15T18:27:05.387 INFO:tasks.workunit.client.0.smithi042.stdout:1/765: creat de/d6d/d9e/df4/f123 x:0 0 0 2024-02-15T18:27:05.388 INFO:tasks.workunit.client.1.smithi191.stdout:6/337: creat d8/d9/d4a/d4c/f5d x:0 0 0 2024-02-15T18:27:05.392 INFO:tasks.workunit.client.1.smithi191.stdout:8/487: rename d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/f9d to d3/d2c/fb5 0 2024-02-15T18:27:05.393 INFO:tasks.workunit.client.0.smithi042.stdout:2/710: dread d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d2e/fcf [0,4194304] 0 2024-02-15T18:27:05.394 INFO:tasks.workunit.client.0.smithi042.stdout:5/789: getdents d0/d50 0 2024-02-15T18:27:05.395 INFO:tasks.workunit.client.0.smithi042.stdout:0/731: truncate d4/d9/f31 2547954 0 2024-02-15T18:27:05.398 INFO:tasks.workunit.client.0.smithi042.stdout:7/776: symlink d10/d49/d6d/db6/l102 0 2024-02-15T18:27:05.398 INFO:tasks.workunit.client.0.smithi042.stdout:9/754: truncate f3 2767848 0 2024-02-15T18:27:05.399 INFO:tasks.workunit.client.0.smithi042.stdout:7/777: write d10/d15/d8f/da5/dad/de0/de2/ffd [2842,34358] 0 2024-02-15T18:27:05.399 INFO:tasks.workunit.client.0.smithi042.stdout:3/745: mkdir d1/d11/d14/df6 0 2024-02-15T18:27:05.399 INFO:tasks.workunit.client.1.smithi191.stdout:7/370: dread d0/db/d26/d5f/d77/df/f17 [4194304,4194304] 0 2024-02-15T18:27:05.408 INFO:tasks.workunit.client.0.smithi042.stdout:1/766: truncate de/d15/d67/d74/d7e/f94 54585 0 2024-02-15T18:27:05.408 INFO:tasks.workunit.client.0.smithi042.stdout:2/711: mknod d0/d8/d20/d7d/d7e/dc7/d8b/ce3 0 2024-02-15T18:27:05.408 INFO:tasks.workunit.client.0.smithi042.stdout:5/790: creat d0/d39/d8d/d9b/f115 x:0 0 0 2024-02-15T18:27:05.408 INFO:tasks.workunit.client.0.smithi042.stdout:2/712: getdents d0/d8/d20/d3f 0 2024-02-15T18:27:05.408 INFO:tasks.workunit.client.0.smithi042.stdout:0/732: symlink d4/dc2/lf5 0 2024-02-15T18:27:05.408 INFO:tasks.workunit.client.0.smithi042.stdout:8/679: mknod dd/de/d33/d3b/cc2 0 2024-02-15T18:27:05.409 INFO:tasks.workunit.client.1.smithi191.stdout:7/371: creat d0/db/d26/f85 x:0 0 0 2024-02-15T18:27:05.410 INFO:tasks.workunit.client.0.smithi042.stdout:0/733: read d4/d38/fdd [355844,90498] 0 2024-02-15T18:27:05.410 INFO:tasks.workunit.client.0.smithi042.stdout:9/755: mkdir d6/da/d2f/d55/d107 0 2024-02-15T18:27:05.410 INFO:tasks.workunit.client.1.smithi191.stdout:7/372: symlink d0/db/d26/l86 0 2024-02-15T18:27:05.412 INFO:tasks.workunit.client.0.smithi042.stdout:2/713: mknod d0/d8/d10/d31/d79/ce4 0 2024-02-15T18:27:05.412 INFO:tasks.workunit.client.0.smithi042.stdout:2/714: truncate d0/d8/f70 1058625 0 2024-02-15T18:27:05.413 INFO:tasks.workunit.client.0.smithi042.stdout:8/680: creat dd/de/da4/d67/d5a/db2/fc3 x:0 0 0 2024-02-15T18:27:05.413 INFO:tasks.workunit.client.1.smithi191.stdout:7/373: read d0/db/d65/d11/d5a/d66/f61 [4388454,117193] 0 2024-02-15T18:27:05.414 INFO:tasks.workunit.client.0.smithi042.stdout:9/756: getdents d6/da/d1d/d2d/d40/de0 0 2024-02-15T18:27:05.414 INFO:tasks.workunit.client.0.smithi042.stdout:2/715: rmdir d0/d8/d20/d41 39 2024-02-15T18:27:05.415 INFO:tasks.workunit.client.0.smithi042.stdout:9/757: write d6/da/d1d/f38 [22194,95886] 0 2024-02-15T18:27:05.415 INFO:tasks.workunit.client.0.smithi042.stdout:9/758: readlink d6/da/d1d/d2d/d40/d95/lf6 0 2024-02-15T18:27:05.416 INFO:tasks.workunit.client.1.smithi191.stdout:7/374: rmdir d0/db/d26/d5f/d77/d1d/d54 39 2024-02-15T18:27:05.418 INFO:tasks.workunit.client.1.smithi191.stdout:7/375: creat d0/db/d26/d5f/d77/d1d/f87 x:0 0 0 2024-02-15T18:27:05.425 INFO:tasks.workunit.client.1.smithi191.stdout:7/376: dread d0/db/d65/d11/d5a/d66/f32 [0,4194304] 0 2024-02-15T18:27:05.425 INFO:tasks.workunit.client.1.smithi191.stdout:7/377: chown d0/db/d65/dc/l84 628245 1 2024-02-15T18:27:05.428 INFO:tasks.workunit.client.1.smithi191.stdout:4/411: dwrite d2/d5/d14/d47/d78/f6a [0,4194304] 0 2024-02-15T18:27:05.431 INFO:tasks.workunit.client.1.smithi191.stdout:4/412: link d2/d5/d7/d2f/l32 d2/d5/d14/d71/l9e 0 2024-02-15T18:27:05.442 INFO:tasks.workunit.client.1.smithi191.stdout:6/338: dwrite d8/f30 [0,4194304] 0 2024-02-15T18:27:05.445 INFO:tasks.workunit.client.0.smithi042.stdout:9/759: dread d6/dd/d9/f18 [0,4194304] 0 2024-02-15T18:27:05.445 INFO:tasks.workunit.client.0.smithi042.stdout:9/760: chown d6/da/d1d/d2d/d40/d95/fb3 1477 1 2024-02-15T18:27:05.447 INFO:tasks.workunit.client.0.smithi042.stdout:9/761: write d6/da/d1d/d2d/d40/f49 [335840,90069] 0 2024-02-15T18:27:05.454 INFO:tasks.workunit.client.1.smithi191.stdout:8/488: dwrite d3/d2c/d1a/d23/d3e/fb4 [0,4194304] 0 2024-02-15T18:27:05.454 INFO:tasks.workunit.client.1.smithi191.stdout:8/489: write d3/faa [359532,51137] 0 2024-02-15T18:27:05.454 INFO:tasks.workunit.client.1.smithi191.stdout:8/490: fsync d3/d2c/d1a/d23/d2e/d47/f63 0 2024-02-15T18:27:05.454 INFO:tasks.workunit.client.1.smithi191.stdout:8/491: chown d3/faa 126908 1 2024-02-15T18:27:05.454 INFO:tasks.workunit.client.0.smithi042.stdout:4/755: dwrite da/d11/d29/fe8 [0,4194304] 0 2024-02-15T18:27:05.456 INFO:tasks.workunit.client.0.smithi042.stdout:3/746: dwrite d1/d11/d14/f60 [0,4194304] 0 2024-02-15T18:27:05.457 INFO:tasks.workunit.client.0.smithi042.stdout:5/791: dwrite d0/d50/df7/ffb [0,4194304] 0 2024-02-15T18:27:05.457 INFO:tasks.workunit.client.0.smithi042.stdout:5/792: fdatasync d0/d14/f63 0 2024-02-15T18:27:05.457 INFO:tasks.workunit.client.0.smithi042.stdout:5/793: stat d0/dda/d42/d7d/db5 0 2024-02-15T18:27:05.457 INFO:tasks.workunit.client.0.smithi042.stdout:8/681: dwrite dd/de/da4/d81/dae/fb6 [0,4194304] 0 2024-02-15T18:27:05.457 INFO:tasks.workunit.client.0.smithi042.stdout:4/756: write da/d11/d29/f32 [4192898,90747] 0 2024-02-15T18:27:05.457 INFO:tasks.workunit.client.0.smithi042.stdout:2/716: dwrite d0/d8/d20/d41/f85 [0,4194304] 0 2024-02-15T18:27:05.457 INFO:tasks.workunit.client.1.smithi191.stdout:9/366: dwrite d0/d7/dc/d30/d4f/f6d [0,4194304] 0 2024-02-15T18:27:05.457 INFO:tasks.workunit.client.1.smithi191.stdout:7/378: dwrite d0/db/d26/d5f/d77/f51 [4194304,4194304] 0 2024-02-15T18:27:05.460 INFO:tasks.workunit.client.1.smithi191.stdout:9/367: write d0/d7/dc/d30/f5c [27428,16136] 0 2024-02-15T18:27:05.463 INFO:tasks.workunit.client.1.smithi191.stdout:8/492: dread d3/d2c/d1a/d23/d2e/f3a [0,4194304] 0 2024-02-15T18:27:05.486 INFO:tasks.workunit.client.0.smithi042.stdout:2/717: dwrite d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d2e/f74 [0,4194304] 0 2024-02-15T18:27:05.486 INFO:tasks.workunit.client.0.smithi042.stdout:5/794: dwrite d0/d50/f5f [0,4194304] 0 2024-02-15T18:27:05.494 INFO:tasks.workunit.client.0.smithi042.stdout:2/718: dread d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/f4d [0,4194304] 0 2024-02-15T18:27:05.495 INFO:tasks.workunit.client.1.smithi191.stdout:4/413: symlink d2/d5/d7/d3e/d88/l9f 0 2024-02-15T18:27:05.497 INFO:tasks.workunit.client.1.smithi191.stdout:6/339: creat d8/d9/d2b/d5b/f5e x:0 0 0 2024-02-15T18:27:05.505 INFO:tasks.workunit.client.0.smithi042.stdout:5/795: dwrite d0/d12/f104 [0,4194304] 0 2024-02-15T18:27:05.506 INFO:tasks.workunit.client.0.smithi042.stdout:6/842: sync 2024-02-15T18:27:05.515 INFO:tasks.workunit.client.0.smithi042.stdout:9/762: creat d6/da/d2f/d55/d107/f108 x:0 0 0 2024-02-15T18:27:05.520 INFO:tasks.workunit.client.0.smithi042.stdout:1/767: truncate de/d15/d67/d74/d7e/f94 611596 0 2024-02-15T18:27:05.523 INFO:tasks.workunit.client.0.smithi042.stdout:1/768: write de/d15/f20 [7399709,57456] 0 2024-02-15T18:27:05.523 INFO:tasks.workunit.client.0.smithi042.stdout:1/769: truncate de/d6d/d9e/dcd/fd8 979293 0 2024-02-15T18:27:05.523 INFO:tasks.workunit.client.0.smithi042.stdout:1/770: write de/f5b [4600360,88306] 0 2024-02-15T18:27:05.524 INFO:tasks.workunit.client.0.smithi042.stdout:1/771: write de/d15/d2c/d51/d70/fba [4716193,20374] 0 2024-02-15T18:27:05.526 INFO:tasks.workunit.client.1.smithi191.stdout:7/379: mkdir d0/db/d65/d11/d88 0 2024-02-15T18:27:05.527 INFO:tasks.workunit.client.1.smithi191.stdout:7/380: write d0/db/f2d [9612177,124594] 0 2024-02-15T18:27:05.527 INFO:tasks.workunit.client.1.smithi191.stdout:7/381: write d0/f8 [7529854,48258] 0 2024-02-15T18:27:05.530 INFO:tasks.workunit.client.1.smithi191.stdout:9/368: creat d0/d7/dc/d17/f84 x:0 0 0 2024-02-15T18:27:05.532 INFO:tasks.workunit.client.1.smithi191.stdout:8/493: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/cb6 0 2024-02-15T18:27:05.533 INFO:tasks.workunit.client.1.smithi191.stdout:4/414: mknod d2/d5/d7/d3e/d63/d8c/ca0 0 2024-02-15T18:27:05.534 INFO:tasks.workunit.client.0.smithi042.stdout:1/772: dread de/d15/d2c/d51/f55 [0,4194304] 0 2024-02-15T18:27:05.539 INFO:tasks.workunit.client.0.smithi042.stdout:3/747: creat d1/d11/d14/df6/ff7 x:0 0 0 2024-02-15T18:27:05.539 INFO:tasks.workunit.client.0.smithi042.stdout:3/748: write d1/d11/d14/d4d/d6c/d7b/fb5 [3316052,54531] 0 2024-02-15T18:27:05.540 INFO:tasks.workunit.client.0.smithi042.stdout:3/749: read - d1/d11/d14/df6/ff7 zero size 2024-02-15T18:27:05.542 INFO:tasks.workunit.client.0.smithi042.stdout:8/682: creat dd/de/fc4 x:0 0 0 2024-02-15T18:27:05.544 INFO:tasks.workunit.client.0.smithi042.stdout:4/757: link da/d11/d29/d4a/d4f/dcf/cff da/d11/d29/d4a/d4f/c105 0 2024-02-15T18:27:05.550 INFO:tasks.workunit.client.1.smithi191.stdout:8/494: dwrite d3/d2c/d1a/f51 [0,4194304] 0 2024-02-15T18:27:05.551 INFO:tasks.workunit.client.1.smithi191.stdout:8/495: truncate d3/d2c/d1a/d23/d2a/f49 224526 0 2024-02-15T18:27:05.551 INFO:tasks.workunit.client.1.smithi191.stdout:8/496: chown d3/f18 1045634 1 2024-02-15T18:27:05.552 INFO:tasks.workunit.client.1.smithi191.stdout:7/382: symlink d0/l89 0 2024-02-15T18:27:05.552 INFO:tasks.workunit.client.1.smithi191.stdout:7/383: write d0/db/d26/d5f/d77/df/d5d/f69 [961653,22309] 0 2024-02-15T18:27:05.552 INFO:tasks.workunit.client.1.smithi191.stdout:7/384: chown d0/db/d65/d11/d5a/d66/f79 8 1 2024-02-15T18:27:05.552 INFO:tasks.workunit.client.1.smithi191.stdout:7/385: stat d0/db/d26/d5f/c60 0 2024-02-15T18:27:05.552 INFO:tasks.workunit.client.1.smithi191.stdout:7/386: chown d0/db/d65 4420909 1 2024-02-15T18:27:05.552 INFO:tasks.workunit.client.1.smithi191.stdout:7/387: write d0/db/d26/d5f/d77/d35/d42/f6c [2101825,62742] 0 2024-02-15T18:27:05.553 INFO:tasks.workunit.client.0.smithi042.stdout:2/719: symlink d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/le5 0 2024-02-15T18:27:05.555 INFO:tasks.workunit.client.1.smithi191.stdout:9/369: creat d0/d7/dc/d30/f85 x:0 0 0 2024-02-15T18:27:05.555 INFO:tasks.workunit.client.1.smithi191.stdout:9/370: dread - d0/d7/da/f78 zero size 2024-02-15T18:27:05.558 INFO:tasks.workunit.client.1.smithi191.stdout:4/415: symlink d2/d5/d14/d47/la1 0 2024-02-15T18:27:05.559 INFO:tasks.workunit.client.1.smithi191.stdout:6/340: symlink d8/d9/d2b/l5f 0 2024-02-15T18:27:05.561 INFO:tasks.workunit.client.1.smithi191.stdout:8/497: getdents d3/d2c/d1a/d23/d2e/d5b/d69/d6d 0 2024-02-15T18:27:05.563 INFO:tasks.workunit.client.1.smithi191.stdout:7/388: creat d0/db/d65/d11/d5a/d66/d1e/f8a x:0 0 0 2024-02-15T18:27:05.565 INFO:tasks.workunit.client.0.smithi042.stdout:6/843: creat d9/d11/d19/d35/d55/d92/f12b x:0 0 0 2024-02-15T18:27:05.566 INFO:tasks.workunit.client.0.smithi042.stdout:7/778: sync 2024-02-15T18:27:05.567 INFO:tasks.workunit.client.0.smithi042.stdout:0/734: sync 2024-02-15T18:27:05.567 INFO:tasks.workunit.client.0.smithi042.stdout:7/779: readlink d10/d49/d6d/lae 0 2024-02-15T18:27:05.567 INFO:tasks.workunit.client.0.smithi042.stdout:0/735: chown d4/d38/d5c/la9 0 1 2024-02-15T18:27:05.567 INFO:tasks.workunit.client.0.smithi042.stdout:0/736: chown d4/d38 168 1 2024-02-15T18:27:05.567 INFO:tasks.workunit.client.1.smithi191.stdout:9/371: creat d0/d29/f86 x:0 0 0 2024-02-15T18:27:05.568 INFO:tasks.workunit.client.0.smithi042.stdout:9/763: getdents d6/da/d2f/d6c 0 2024-02-15T18:27:05.569 INFO:tasks.workunit.client.1.smithi191.stdout:4/416: write d2/d5/d7/d2f/f6b [553220,92360] 0 2024-02-15T18:27:05.569 INFO:tasks.workunit.client.0.smithi042.stdout:9/764: write d6/dd/d9/f18 [2415554,110036] 0 2024-02-15T18:27:05.570 INFO:tasks.workunit.client.1.smithi191.stdout:6/341: fsync d8/d9/d2b/f4e 0 2024-02-15T18:27:05.571 INFO:tasks.workunit.client.1.smithi191.stdout:6/342: write d8/d9/f46 [634683,79653] 0 2024-02-15T18:27:05.571 INFO:tasks.workunit.client.0.smithi042.stdout:1/773: symlink de/d54/dbf/dd0/d7c/dd6/de8/d103/d10f/l124 0 2024-02-15T18:27:05.572 INFO:tasks.workunit.client.1.smithi191.stdout:8/498: link d3/d2c/d1a/d23/d2a/f49 d3/d2c/d1a/d23/d2e/d5b/d69/fb7 0 2024-02-15T18:27:05.573 INFO:tasks.workunit.client.1.smithi191.stdout:8/499: write d3/d2c/f48 [364702,72086] 0 2024-02-15T18:27:05.575 INFO:tasks.workunit.client.1.smithi191.stdout:7/389: unlink d0/db/d65/d11/d5a/d66/f61 0 2024-02-15T18:27:05.576 INFO:tasks.workunit.client.1.smithi191.stdout:9/372: mknod d0/d7/dc/d30/d3b/d56/c87 0 2024-02-15T18:27:05.577 INFO:tasks.workunit.client.0.smithi042.stdout:3/750: rmdir d1/d11/d14/df6 39 2024-02-15T18:27:05.577 INFO:tasks.workunit.client.0.smithi042.stdout:3/751: write d1/d11/d2e/d4c/f5b [1164406,62278] 0 2024-02-15T18:27:05.577 INFO:tasks.workunit.client.0.smithi042.stdout:8/683: mknod dd/de/da4/d67/daa/cc5 0 2024-02-15T18:27:05.577 INFO:tasks.workunit.client.0.smithi042.stdout:8/684: write dd/de/d33/d66/f5f [741879,66061] 0 2024-02-15T18:27:05.579 INFO:tasks.workunit.client.0.smithi042.stdout:8/685: dread dd/de/d12/f4c [0,4194304] 0 2024-02-15T18:27:05.580 INFO:tasks.workunit.client.1.smithi191.stdout:9/373: write d0/f4 [2505190,68097] 0 2024-02-15T18:27:05.582 INFO:tasks.workunit.client.1.smithi191.stdout:4/417: rmdir d2/d5/d7/d2f 39 2024-02-15T18:27:05.582 INFO:tasks.workunit.client.0.smithi042.stdout:4/758: creat da/d2b/d6e/f106 x:0 0 0 2024-02-15T18:27:05.583 INFO:tasks.workunit.client.0.smithi042.stdout:4/759: write da/d2b/d7e/fc7 [1038435,8283] 0 2024-02-15T18:27:05.583 INFO:tasks.workunit.client.0.smithi042.stdout:4/760: write da/d11/d85/db5/d103/f65 [5074349,91263] 0 2024-02-15T18:27:05.586 INFO:tasks.workunit.client.1.smithi191.stdout:8/500: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/cb8 0 2024-02-15T18:27:05.587 INFO:tasks.workunit.client.0.smithi042.stdout:1/774: dwrite de/d15/d2c/f41 [0,4194304] 0 2024-02-15T18:27:05.587 INFO:tasks.workunit.client.0.smithi042.stdout:1/775: write de/d54/dbf/dd0/d7c/f118 [89365,70334] 0 2024-02-15T18:27:05.587 INFO:tasks.workunit.client.0.smithi042.stdout:1/776: truncate de/d54/dbf/dd0/d7c/fd9 756888 0 2024-02-15T18:27:05.587 INFO:tasks.workunit.client.0.smithi042.stdout:1/777: chown de/d15/d67/d63 375 1 2024-02-15T18:27:05.588 INFO:tasks.workunit.client.1.smithi191.stdout:7/390: creat d0/db/f8b x:0 0 0 2024-02-15T18:27:05.591 INFO:tasks.workunit.client.0.smithi042.stdout:2/720: link d0/d8/d10/d31/cc4 d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/d44/ce6 0 2024-02-15T18:27:05.591 INFO:tasks.workunit.client.0.smithi042.stdout:2/721: write d0/d8/d10/d5a/fbc [1636946,72659] 0 2024-02-15T18:27:05.594 INFO:tasks.workunit.client.1.smithi191.stdout:9/374: mknod d0/d7/dc/d17/d33/c88 0 2024-02-15T18:27:05.597 INFO:tasks.workunit.client.0.smithi042.stdout:5/796: sync 2024-02-15T18:27:05.598 INFO:tasks.workunit.client.0.smithi042.stdout:6/844: creat d9/d11/d46/d89/f12c x:0 0 0 2024-02-15T18:27:05.599 INFO:tasks.workunit.client.1.smithi191.stdout:4/418: creat d2/d5/d64/fa2 x:0 0 0 2024-02-15T18:27:05.599 INFO:tasks.workunit.client.1.smithi191.stdout:6/343: dwrite d8/d9/d2b/d57/f59 [0,4194304] 0 2024-02-15T18:27:05.599 INFO:tasks.workunit.client.0.smithi042.stdout:0/737: truncate d4/d9/d34/dab/dc8/d6c/d70/faa 2707194 0 2024-02-15T18:27:05.599 INFO:tasks.workunit.client.0.smithi042.stdout:6/845: read d9/d11/d19/d35/d55/f65 [6730305,49745] 0 2024-02-15T18:27:05.601 INFO:tasks.workunit.client.0.smithi042.stdout:9/765: unlink d6/da/d7b/fef 0 2024-02-15T18:27:05.601 INFO:tasks.workunit.client.0.smithi042.stdout:9/766: chown d6/da/d7b/f92 394878950 1 2024-02-15T18:27:05.608 INFO:tasks.workunit.client.0.smithi042.stdout:8/686: getdents dd/de/da5 0 2024-02-15T18:27:05.610 INFO:tasks.workunit.client.1.smithi191.stdout:8/501: creat d3/d2c/d1a/d23/d3e/db2/fb9 x:0 0 0 2024-02-15T18:27:05.612 INFO:tasks.workunit.client.0.smithi042.stdout:4/761: symlink da/d2b/l107 0 2024-02-15T18:27:05.614 INFO:tasks.workunit.client.1.smithi191.stdout:7/391: mkdir d0/db/d26/d5f/d8c 0 2024-02-15T18:27:05.615 INFO:tasks.workunit.client.1.smithi191.stdout:9/375: dwrite d0/d7/f19 [0,4194304] 0 2024-02-15T18:27:05.615 INFO:tasks.workunit.client.1.smithi191.stdout:9/376: fdatasync d0/d7/da/f78 0 2024-02-15T18:27:05.623 INFO:tasks.workunit.client.0.smithi042.stdout:1/778: rename de/d15/d1b/d44/f106 to de/d54/d9a/f125 0 2024-02-15T18:27:05.623 INFO:tasks.workunit.client.0.smithi042.stdout:1/779: fdatasync de/d15/d67/d63/f8b 0 2024-02-15T18:27:05.623 INFO:tasks.workunit.client.0.smithi042.stdout:1/780: chown de/d15/d67/d63/f69 186225745 1 2024-02-15T18:27:05.625 INFO:tasks.workunit.client.0.smithi042.stdout:0/738: mknod d4/d9/d34/dab/cf6 0 2024-02-15T18:27:05.625 INFO:tasks.workunit.client.0.smithi042.stdout:0/739: chown d4/d9/d34/d8b/f90 9 1 2024-02-15T18:27:05.625 INFO:tasks.workunit.client.0.smithi042.stdout:0/740: stat d4/d38/d77/f7c 0 2024-02-15T18:27:05.626 INFO:tasks.workunit.client.0.smithi042.stdout:6/846: truncate d9/d11/d19/d1c/d31/f6f 3076935 0 2024-02-15T18:27:05.628 INFO:tasks.workunit.client.0.smithi042.stdout:6/847: write d9/d11/ff1 [371022,118990] 0 2024-02-15T18:27:05.628 INFO:tasks.workunit.client.0.smithi042.stdout:6/848: write d9/d11/d19/d35/d55/d92/dfc/f11c [331012,122023] 0 2024-02-15T18:27:05.628 INFO:tasks.workunit.client.1.smithi191.stdout:4/419: mknod d2/d5/d14/d44/ca3 0 2024-02-15T18:27:05.629 INFO:tasks.workunit.client.1.smithi191.stdout:6/344: creat d8/d9/d2b/d57/f60 x:0 0 0 2024-02-15T18:27:05.630 INFO:tasks.workunit.client.1.smithi191.stdout:7/392: truncate d0/db/f2f 574878 0 2024-02-15T18:27:05.630 INFO:tasks.workunit.client.1.smithi191.stdout:7/393: truncate d0/db/d65/d11/d5a/d66/d1e/f8a 999410 0 2024-02-15T18:27:05.632 INFO:tasks.workunit.client.0.smithi042.stdout:9/767: unlink d6/dd/d9/f18 0 2024-02-15T18:27:05.632 INFO:tasks.workunit.client.1.smithi191.stdout:7/394: write d0/db/d26/d5f/d77/df/f17 [7892370,56246] 0 2024-02-15T18:27:05.634 INFO:tasks.workunit.client.1.smithi191.stdout:8/502: dwrite d3/d5e/f73 [0,4194304] 0 2024-02-15T18:27:05.636 INFO:tasks.workunit.client.1.smithi191.stdout:9/377: getdents d0/d29/d48 0 2024-02-15T18:27:05.636 INFO:tasks.workunit.client.1.smithi191.stdout:9/378: readlink d0/d7/dc/d32/l62 0 2024-02-15T18:27:05.637 INFO:tasks.workunit.client.1.smithi191.stdout:4/420: rename d2/d5/d64 to d2/d5/d7/d3e/d63/d8c/da4 0 2024-02-15T18:27:05.640 INFO:tasks.workunit.client.0.smithi042.stdout:8/687: unlink dd/de/l80 0 2024-02-15T18:27:05.640 INFO:tasks.workunit.client.0.smithi042.stdout:0/741: dread d4/d9/d5a/d60/fe1 [0,4194304] 0 2024-02-15T18:27:05.641 INFO:tasks.workunit.client.0.smithi042.stdout:4/762: unlink da/d11/c13 0 2024-02-15T18:27:05.648 INFO:tasks.workunit.client.0.smithi042.stdout:2/722: rename d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/d44/fba to d0/d8/d10/d7b/fe7 0 2024-02-15T18:27:05.648 INFO:tasks.workunit.client.0.smithi042.stdout:2/723: chown d0/l9b 100 1 2024-02-15T18:27:05.654 INFO:tasks.workunit.client.1.smithi191.stdout:9/379: getdents d0/d7/dc 0 2024-02-15T18:27:05.655 INFO:tasks.workunit.client.0.smithi042.stdout:1/781: mknod de/d6d/c126 0 2024-02-15T18:27:05.657 INFO:tasks.workunit.client.1.smithi191.stdout:7/395: rename d0/db/d26/d5f/d77/df/f16 to d0/db/f8d 0 2024-02-15T18:27:05.659 INFO:tasks.workunit.client.0.smithi042.stdout:6/849: creat d9/d11/d19/d35/dc4/d123/f12d x:0 0 0 2024-02-15T18:27:05.660 INFO:tasks.workunit.client.1.smithi191.stdout:6/345: mkdir d8/d9/d61 0 2024-02-15T18:27:05.660 INFO:tasks.workunit.client.0.smithi042.stdout:9/768: mknod d6/da/d2f/d55/d107/c109 0 2024-02-15T18:27:05.661 INFO:tasks.workunit.client.0.smithi042.stdout:9/769: chown d6/dd/d9 7199 1 2024-02-15T18:27:05.661 INFO:tasks.workunit.client.0.smithi042.stdout:9/770: readlink d6/da/d7b/lca 0 2024-02-15T18:27:05.662 INFO:tasks.workunit.client.1.smithi191.stdout:9/380: unlink d0/d7/dc/d30/d4f/c6a 0 2024-02-15T18:27:05.665 INFO:tasks.workunit.client.1.smithi191.stdout:6/346: creat d8/d19/f62 x:0 0 0 2024-02-15T18:27:05.665 INFO:tasks.workunit.client.0.smithi042.stdout:2/724: dwrite d0/d8/d10/d31/fa1 [4194304,4194304] 0 2024-02-15T18:27:05.666 INFO:tasks.workunit.client.0.smithi042.stdout:2/725: chown d0/c91 1712 1 2024-02-15T18:27:05.666 INFO:tasks.workunit.client.0.smithi042.stdout:2/726: readlink d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/l48 0 2024-02-15T18:27:05.666 INFO:tasks.workunit.client.0.smithi042.stdout:8/688: mkdir dd/de/da5/dc6 0 2024-02-15T18:27:05.666 INFO:tasks.workunit.client.1.smithi191.stdout:9/381: write d0/d7/dc/d17/d44/f6e [194430,130575] 0 2024-02-15T18:27:05.666 INFO:tasks.workunit.client.0.smithi042.stdout:2/727: write d0/d8/f65 [2872393,117555] 0 2024-02-15T18:27:05.667 INFO:tasks.workunit.client.0.smithi042.stdout:4/763: dwrite da/d11/d29/db8/fb9 [0,4194304] 0 2024-02-15T18:27:05.667 INFO:tasks.workunit.client.0.smithi042.stdout:4/764: truncate da/d3b/fda 614630 0 2024-02-15T18:27:05.667 INFO:tasks.workunit.client.0.smithi042.stdout:2/728: read d0/f8c [4062741,103166] 0 2024-02-15T18:27:05.667 INFO:tasks.workunit.client.0.smithi042.stdout:2/729: dread - d0/fc8 zero size 2024-02-15T18:27:05.668 INFO:tasks.workunit.client.0.smithi042.stdout:0/742: unlink d4/d38/f3a 0 2024-02-15T18:27:05.668 INFO:tasks.workunit.client.0.smithi042.stdout:9/771: dread d6/dd/d9/f41 [0,4194304] 0 2024-02-15T18:27:05.669 INFO:tasks.workunit.client.1.smithi191.stdout:8/503: dwrite d3/d2c/d1a/d23/d3e/fa9 [0,4194304] 0 2024-02-15T18:27:05.669 INFO:tasks.workunit.client.1.smithi191.stdout:8/504: truncate d3/d2c/d1a/d36/f65 503059 0 2024-02-15T18:27:05.676 INFO:tasks.workunit.client.0.smithi042.stdout:1/782: rename de/d54/dbf/dd0/d7c/dd6/de8/dec/d100/f10b to de/d6d/d9e/dcd/f127 0 2024-02-15T18:27:05.676 INFO:tasks.workunit.client.0.smithi042.stdout:1/783: write de/d15/d67/d63/f8b [1176926,69331] 0 2024-02-15T18:27:05.678 INFO:tasks.workunit.client.0.smithi042.stdout:6/850: mkdir d9/d11/d19/d35/d55/d92/d9e/d12e 0 2024-02-15T18:27:05.678 INFO:tasks.workunit.client.0.smithi042.stdout:6/851: fsync d9/d11/d19/d35/dc4/fe3 0 2024-02-15T18:27:05.678 INFO:tasks.workunit.client.0.smithi042.stdout:6/852: chown d9/d11/d46/dc3/lfd 64 1 2024-02-15T18:27:05.680 INFO:tasks.workunit.client.1.smithi191.stdout:7/396: truncate d0/db/d26/d5f/d77/df/f17 7462347 0 2024-02-15T18:27:05.680 INFO:tasks.workunit.client.1.smithi191.stdout:7/397: truncate d0/db/d65/d11/d1c/f45 1736953 0 2024-02-15T18:27:05.682 INFO:tasks.workunit.client.0.smithi042.stdout:7/780: sync 2024-02-15T18:27:05.682 INFO:tasks.workunit.client.0.smithi042.stdout:7/781: fsync d10/fbf 0 2024-02-15T18:27:05.683 INFO:tasks.workunit.client.0.smithi042.stdout:7/782: dread d10/d12/d23/d51/f99 [0,4194304] 0 2024-02-15T18:27:05.684 INFO:tasks.workunit.client.1.smithi191.stdout:8/505: rename d3/f12 to d3/d2c/d1a/d23/d2e/d47/d5f/fba 0 2024-02-15T18:27:05.685 INFO:tasks.workunit.client.1.smithi191.stdout:7/398: creat d0/db/d26/d5f/d77/df/f8e x:0 0 0 2024-02-15T18:27:05.687 INFO:tasks.workunit.client.1.smithi191.stdout:7/399: symlink d0/db/d26/d5f/d77/l8f 0 2024-02-15T18:27:05.689 INFO:tasks.workunit.client.0.smithi042.stdout:4/765: link da/d11/d85/lde da/l108 0 2024-02-15T18:27:05.690 INFO:tasks.workunit.client.0.smithi042.stdout:2/730: getdents d0/d8/d10/d5a 0 2024-02-15T18:27:05.692 INFO:tasks.workunit.client.0.smithi042.stdout:0/743: mkdir d4/d9/d34/dab/dc8/d6c/df7 0 2024-02-15T18:27:05.693 INFO:tasks.workunit.client.0.smithi042.stdout:1/784: creat de/d54/d9a/f128 x:0 0 0 2024-02-15T18:27:05.695 INFO:tasks.workunit.client.0.smithi042.stdout:6/853: mkdir d9/d11/d97/df7/d12f 0 2024-02-15T18:27:05.695 INFO:tasks.workunit.client.0.smithi042.stdout:0/744: symlink d4/d38/d77/d2e/dda/lf8 0 2024-02-15T18:27:05.697 INFO:tasks.workunit.client.0.smithi042.stdout:0/745: write d4/d9/f46 [3144617,8279] 0 2024-02-15T18:27:05.697 INFO:tasks.workunit.client.0.smithi042.stdout:0/746: write d4/d9/feb [428683,57101] 0 2024-02-15T18:27:05.697 INFO:tasks.workunit.client.0.smithi042.stdout:0/747: stat d4/d38/d39/d8e 0 2024-02-15T18:27:05.697 INFO:tasks.workunit.client.0.smithi042.stdout:0/748: chown d4/d38/d77/d2e/dae/ddf/fe8 1116 1 2024-02-15T18:27:05.697 INFO:tasks.workunit.client.0.smithi042.stdout:0/749: chown d4/d9/d62/d74/cf2 905983 1 2024-02-15T18:27:05.697 INFO:tasks.workunit.client.0.smithi042.stdout:6/854: symlink d9/d11/d19/d35/d55/l130 0 2024-02-15T18:27:05.697 INFO:tasks.workunit.client.0.smithi042.stdout:6/855: chown d9/d11/d19/d35/d55/d92/d9e/cef 26725 1 2024-02-15T18:27:05.698 INFO:tasks.workunit.client.0.smithi042.stdout:8/689: dread dd/d11/f15 [0,4194304] 0 2024-02-15T18:27:05.698 INFO:tasks.workunit.client.0.smithi042.stdout:0/750: mkdir d4/d9/d34/dab/dc8/d6c/df9 0 2024-02-15T18:27:05.699 INFO:tasks.workunit.client.0.smithi042.stdout:6/856: creat d9/d11/d46/d89/dbc/f131 x:0 0 0 2024-02-15T18:27:05.699 INFO:tasks.workunit.client.0.smithi042.stdout:6/857: fdatasync d9/d11/d19/f99 0 2024-02-15T18:27:05.700 INFO:tasks.workunit.client.0.smithi042.stdout:6/858: write d9/d3e/f8b [4097325,34841] 0 2024-02-15T18:27:05.700 INFO:tasks.workunit.client.0.smithi042.stdout:8/690: creat dd/de/d33/dbf/fc7 x:0 0 0 2024-02-15T18:27:05.701 INFO:tasks.workunit.client.0.smithi042.stdout:0/751: creat d4/d9/d34/ffa x:0 0 0 2024-02-15T18:27:05.701 INFO:tasks.workunit.client.0.smithi042.stdout:0/752: write d4/d9/d5a/d60/ff1 [116439,128356] 0 2024-02-15T18:27:05.702 INFO:tasks.workunit.client.1.smithi191.stdout:9/382: dwrite d0/f51 [0,4194304] 0 2024-02-15T18:27:05.706 INFO:tasks.workunit.client.1.smithi191.stdout:7/400: dwrite d0/db/d26/d5f/d77/f51 [8388608,4194304] 0 2024-02-15T18:27:05.707 INFO:tasks.workunit.client.0.smithi042.stdout:9/772: dwrite d6/da/d2f/d55/f6d [4194304,4194304] 0 2024-02-15T18:27:05.707 INFO:tasks.workunit.client.0.smithi042.stdout:4/766: dread da/d11/d29/f32 [0,4194304] 0 2024-02-15T18:27:05.707 INFO:tasks.workunit.client.0.smithi042.stdout:4/767: chown da/d11/d85/db5/d103/d66/d78/d2f/d4d/cc1 584208 1 2024-02-15T18:27:05.707 INFO:tasks.workunit.client.0.smithi042.stdout:2/731: write d0/d8/d10/d5a/fbc [882938,121549] 0 2024-02-15T18:27:05.707 INFO:tasks.workunit.client.0.smithi042.stdout:2/732: fdatasync d0/f63 0 2024-02-15T18:27:05.710 INFO:tasks.workunit.client.0.smithi042.stdout:2/733: write d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/f2b [4145379,40399] 0 2024-02-15T18:27:05.712 INFO:tasks.workunit.client.0.smithi042.stdout:6/859: unlink d9/d11/d19/fbd 0 2024-02-15T18:27:05.715 INFO:tasks.workunit.client.0.smithi042.stdout:8/691: link dd/de/d3c/f7e dd/de/d12/fc8 0 2024-02-15T18:27:05.715 INFO:tasks.workunit.client.1.smithi191.stdout:9/383: mkdir d0/d7/dc/d32/d89 0 2024-02-15T18:27:05.717 INFO:tasks.workunit.client.0.smithi042.stdout:0/753: symlink d4/d9/d62/d74/lfb 0 2024-02-15T18:27:05.718 INFO:tasks.workunit.client.0.smithi042.stdout:6/860: dread d9/d11/f2b [0,4194304] 0 2024-02-15T18:27:05.718 INFO:tasks.workunit.client.0.smithi042.stdout:6/861: chown d9/d11/d19/d1c/d31/d4f/fc8 2136497124 1 2024-02-15T18:27:05.718 INFO:tasks.workunit.client.0.smithi042.stdout:6/862: write d9/d11/d19/d1c/d31/d53/fc2 [389058,66358] 0 2024-02-15T18:27:05.718 INFO:tasks.workunit.client.0.smithi042.stdout:0/754: write d4/d38/d5c/fc4 [254267,129411] 0 2024-02-15T18:27:05.719 INFO:tasks.workunit.client.0.smithi042.stdout:1/785: dwrite de/d15/d2c/d51/fcc [0,4194304] 0 2024-02-15T18:27:05.719 INFO:tasks.workunit.client.0.smithi042.stdout:1/786: write de/d15/d2c/fde [215827,96313] 0 2024-02-15T18:27:05.723 INFO:tasks.workunit.client.1.smithi191.stdout:7/401: truncate d0/db/d26/d5f/d77/df/f1a 3342435 0 2024-02-15T18:27:05.724 INFO:tasks.workunit.client.1.smithi191.stdout:9/384: mknod d0/d7/dc/d30/d3b/d56/d60/c8a 0 2024-02-15T18:27:05.726 INFO:tasks.workunit.client.1.smithi191.stdout:7/402: creat d0/db/d65/d11/f90 x:0 0 0 2024-02-15T18:27:05.729 INFO:tasks.workunit.client.1.smithi191.stdout:9/385: mkdir d0/d7/dc/d30/d3b/d8b 0 2024-02-15T18:27:05.732 INFO:tasks.workunit.client.1.smithi191.stdout:9/386: rename d0/d7/da to d0/d7/dc/d8c 0 2024-02-15T18:27:05.733 INFO:tasks.workunit.client.1.smithi191.stdout:7/403: dread d0/db/d26/d5f/d77/df/f17 [0,4194304] 0 2024-02-15T18:27:05.735 INFO:tasks.workunit.client.1.smithi191.stdout:9/387: link d0/c5a d0/d7/dc/d32/d89/c8d 0 2024-02-15T18:27:05.736 INFO:tasks.workunit.client.0.smithi042.stdout:9/773: creat d6/da/d1d/d2d/d40/d95/f10a x:0 0 0 2024-02-15T18:27:05.738 INFO:tasks.workunit.client.1.smithi191.stdout:7/404: symlink d0/db/d26/d5f/d77/l91 0 2024-02-15T18:27:05.740 INFO:tasks.workunit.client.0.smithi042.stdout:5/797: sync 2024-02-15T18:27:05.741 INFO:tasks.workunit.client.0.smithi042.stdout:3/752: sync 2024-02-15T18:27:05.741 INFO:tasks.workunit.client.0.smithi042.stdout:4/768: creat da/d11/d29/d4a/f109 x:0 0 0 2024-02-15T18:27:05.742 INFO:tasks.workunit.client.0.smithi042.stdout:5/798: read d0/d14/f63 [2297300,88182] 0 2024-02-15T18:27:05.742 INFO:tasks.workunit.client.1.smithi191.stdout:9/388: creat d0/d7/dc/f8e x:0 0 0 2024-02-15T18:27:05.742 INFO:tasks.workunit.client.1.smithi191.stdout:9/389: chown d0/d7/dc/d17/c34 78598153 1 2024-02-15T18:27:05.742 INFO:tasks.workunit.client.0.smithi042.stdout:2/734: symlink d0/d8/d20/d7d/d7e/dc7/d8b/dd7/le8 0 2024-02-15T18:27:05.743 INFO:tasks.workunit.client.1.smithi191.stdout:7/405: symlink d0/db/d26/d5f/d77/d1d/d54/l92 0 2024-02-15T18:27:05.743 INFO:tasks.workunit.client.1.smithi191.stdout:7/406: read d0/db/d65/d11/d1c/f45 [1620325,116034] 0 2024-02-15T18:27:05.746 INFO:tasks.workunit.client.0.smithi042.stdout:8/692: link dd/de/d33/d66/lb7 dd/de/da4/d67/d6f/d78/lc9 0 2024-02-15T18:27:05.746 INFO:tasks.workunit.client.0.smithi042.stdout:8/693: chown dd/de/da4/d67/f43 39492 1 2024-02-15T18:27:05.746 INFO:tasks.workunit.client.0.smithi042.stdout:6/863: creat d9/d11/d97/df7/f132 x:0 0 0 2024-02-15T18:27:05.750 INFO:tasks.workunit.client.0.smithi042.stdout:1/787: creat de/d15/d67/d74/d7e/f129 x:0 0 0 2024-02-15T18:27:05.754 INFO:tasks.workunit.client.0.smithi042.stdout:6/864: dread d9/d11/d19/d1c/dcf/fdd [0,4194304] 0 2024-02-15T18:27:05.759 INFO:tasks.workunit.client.1.smithi191.stdout:9/390: getdents d0/d7/dc 0 2024-02-15T18:27:05.759 INFO:tasks.workunit.client.1.smithi191.stdout:9/391: fsync d0/d7/dc/d17/d33/f69 0 2024-02-15T18:27:05.763 INFO:tasks.workunit.client.1.smithi191.stdout:9/392: link d0/d7/dc/c1e d0/d7/dc/d17/d33/c8f 0 2024-02-15T18:27:05.766 INFO:tasks.workunit.client.1.smithi191.stdout:7/407: dwrite d0/f3a [4194304,4194304] 0 2024-02-15T18:27:05.767 INFO:tasks.workunit.client.0.smithi042.stdout:9/774: rmdir d6/da/d1d/d2d/d40/d95 39 2024-02-15T18:27:05.768 INFO:tasks.workunit.client.0.smithi042.stdout:3/753: link d1/d11/d14/d4d/f4f d1/d11/d7a/ff8 0 2024-02-15T18:27:05.768 INFO:tasks.workunit.client.1.smithi191.stdout:9/393: rename d0/d7/dc/d30/f85 to d0/d29/d48/d66/f90 0 2024-02-15T18:27:05.768 INFO:tasks.workunit.client.1.smithi191.stdout:9/394: fsync d0/d7/dc/d17/f84 0 2024-02-15T18:27:05.771 INFO:tasks.workunit.client.1.smithi191.stdout:7/408: rmdir d0/db/d26/d5f/d77 39 2024-02-15T18:27:05.771 INFO:tasks.workunit.client.1.smithi191.stdout:7/409: fsync d0/db/d26/f3c 0 2024-02-15T18:27:05.771 INFO:tasks.workunit.client.0.smithi042.stdout:4/769: link da/d11/d85/db5/d103/d66/f8a da/d3b/d41/d8d/f10a 0 2024-02-15T18:27:05.772 INFO:tasks.workunit.client.0.smithi042.stdout:5/799: creat d0/d14/d88/da1/f116 x:0 0 0 2024-02-15T18:27:05.773 INFO:tasks.workunit.client.0.smithi042.stdout:5/800: fsync d0/d39/d8d/d9b/f115 0 2024-02-15T18:27:05.773 INFO:tasks.workunit.client.0.smithi042.stdout:5/801: chown d0/d12/ca2 137165 1 2024-02-15T18:27:05.774 INFO:tasks.workunit.client.0.smithi042.stdout:2/735: creat d0/d35/d9e/fe9 x:0 0 0 2024-02-15T18:27:05.775 INFO:tasks.workunit.client.1.smithi191.stdout:7/410: creat d0/db/d26/d5f/d77/d35/d42/f93 x:0 0 0 2024-02-15T18:27:05.777 INFO:tasks.workunit.client.0.smithi042.stdout:8/694: creat dd/fca x:0 0 0 2024-02-15T18:27:05.778 INFO:tasks.workunit.client.0.smithi042.stdout:1/788: link de/d15/d67/d74/f90 de/d6d/d9e/df4/f12a 0 2024-02-15T18:27:05.780 INFO:tasks.workunit.client.0.smithi042.stdout:2/736: read d0/d8/d10/d31/d3a/d53/f45 [2423838,77999] 0 2024-02-15T18:27:05.780 INFO:tasks.workunit.client.0.smithi042.stdout:5/802: unlink d0/dda/dc7/f72 0 2024-02-15T18:27:05.781 INFO:tasks.workunit.client.0.smithi042.stdout:8/695: creat dd/de/d94/fcb x:0 0 0 2024-02-15T18:27:05.783 INFO:tasks.workunit.client.0.smithi042.stdout:1/789: link de/d15/d67/d46/f68 de/d54/dbf/dd0/d7c/f12b 0 2024-02-15T18:27:05.783 INFO:tasks.workunit.client.0.smithi042.stdout:1/790: chown de/d54/dbf/dd0/d7c/fb1 0 1 2024-02-15T18:27:05.784 INFO:tasks.workunit.client.0.smithi042.stdout:5/803: creat d0/d38/d53/d64/f117 x:0 0 0 2024-02-15T18:27:05.785 INFO:tasks.workunit.client.0.smithi042.stdout:1/791: symlink de/d15/d67/d46/l12c 0 2024-02-15T18:27:05.785 INFO:tasks.workunit.client.0.smithi042.stdout:1/792: readlink de/d15/d2c/d51/l80 0 2024-02-15T18:27:05.787 INFO:tasks.workunit.client.0.smithi042.stdout:5/804: truncate d0/d14/f63 577143 0 2024-02-15T18:27:05.788 INFO:tasks.workunit.client.0.smithi042.stdout:4/770: dread da/d2b/f37 [0,4194304] 0 2024-02-15T18:27:05.788 INFO:tasks.workunit.client.0.smithi042.stdout:4/771: readlink da/d3b/d41/l46 0 2024-02-15T18:27:05.788 INFO:tasks.workunit.client.0.smithi042.stdout:5/805: stat d0/d22/l2f 0 2024-02-15T18:27:05.789 INFO:tasks.workunit.client.0.smithi042.stdout:8/696: dread dd/de/d33/d66/f5f [0,4194304] 0 2024-02-15T18:27:05.789 INFO:tasks.workunit.client.0.smithi042.stdout:4/772: rmdir da/d11/d85/db5/d103/d66/d78/d2f/d4d 39 2024-02-15T18:27:05.790 INFO:tasks.workunit.client.0.smithi042.stdout:5/806: creat d0/dda/dc7/d52/dad/dba/dbd/f118 x:0 0 0 2024-02-15T18:27:05.790 INFO:tasks.workunit.client.0.smithi042.stdout:8/697: truncate dd/de/da4/d67/f2e 6145206 0 2024-02-15T18:27:05.792 INFO:tasks.workunit.client.0.smithi042.stdout:4/773: link da/d11/d29/d4a/d4f/c105 da/d11/d67/c10b 0 2024-02-15T18:27:05.793 INFO:tasks.workunit.client.0.smithi042.stdout:5/807: symlink d0/dda/dc7/d52/da0/de4/l119 0 2024-02-15T18:27:05.793 INFO:tasks.workunit.client.0.smithi042.stdout:5/808: fsync d0/d38/f5a 0 2024-02-15T18:27:05.793 INFO:tasks.workunit.client.0.smithi042.stdout:5/809: creat d0/d14/d88/df8/f11a x:0 0 0 2024-02-15T18:27:05.800 INFO:tasks.workunit.client.0.smithi042.stdout:4/774: dread da/d11/d29/d4a/d4f/d90/fae [0,4194304] 0 2024-02-15T18:27:05.801 INFO:tasks.workunit.client.0.smithi042.stdout:4/775: mknod da/d3a/dc9/c10c 0 2024-02-15T18:27:05.802 INFO:tasks.workunit.client.0.smithi042.stdout:4/776: getdents da/d11/d85/db5/d103/d66/d78/d2f/dca 0 2024-02-15T18:27:05.804 INFO:tasks.workunit.client.0.smithi042.stdout:4/777: symlink da/d11/d67/l10d 0 2024-02-15T18:27:05.804 INFO:tasks.workunit.client.0.smithi042.stdout:4/778: readlink da/d3b/d41/l46 0 2024-02-15T18:27:05.808 INFO:tasks.workunit.client.0.smithi042.stdout:2/737: dread d0/d8/d10/d31/da6/fad [0,4194304] 0 2024-02-15T18:27:05.811 INFO:tasks.workunit.client.0.smithi042.stdout:6/865: dwrite d9/f14 [0,4194304] 0 2024-02-15T18:27:05.812 INFO:tasks.workunit.client.0.smithi042.stdout:0/755: dwrite d4/d9/d34/dab/dc8/d6c/d70/faa [0,4194304] 0 2024-02-15T18:27:05.814 INFO:tasks.workunit.client.0.smithi042.stdout:0/756: rename d4/d9/d15 to d4/d9/d5a/dfc 0 2024-02-15T18:27:05.818 INFO:tasks.workunit.client.0.smithi042.stdout:8/698: dread dd/de/d12/f45 [0,4194304] 0 2024-02-15T18:27:05.818 INFO:tasks.workunit.client.0.smithi042.stdout:8/699: stat dd/de/da4/cbc 0 2024-02-15T18:27:05.818 INFO:tasks.workunit.client.0.smithi042.stdout:4/779: dread da/f7c [0,4194304] 0 2024-02-15T18:27:05.818 INFO:tasks.workunit.client.0.smithi042.stdout:4/780: creat da/d11/d67/f10e x:0 0 0 2024-02-15T18:27:05.818 INFO:tasks.workunit.client.0.smithi042.stdout:2/738: write d0/d8/d10/d31/d3a/d53/f45 [2976463,57652] 0 2024-02-15T18:27:05.822 INFO:tasks.workunit.client.0.smithi042.stdout:4/781: truncate da/d11/d85/db5/d103/d66/d78/f3e 3580804 0 2024-02-15T18:27:05.822 INFO:tasks.workunit.client.0.smithi042.stdout:4/782: fsync da/d11/f22 0 2024-02-15T18:27:05.822 INFO:tasks.workunit.client.0.smithi042.stdout:4/783: read - da/d11/d85/db5/d103/d66/d78/fbd zero size 2024-02-15T18:27:05.823 INFO:tasks.workunit.client.0.smithi042.stdout:2/739: write d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/d44/f84 [2703928,81294] 0 2024-02-15T18:27:05.823 INFO:tasks.workunit.client.0.smithi042.stdout:9/775: dwrite f3 [0,4194304] 0 2024-02-15T18:27:05.824 INFO:tasks.workunit.client.0.smithi042.stdout:3/754: dwrite d1/d11/d14/d4d/d6c/d7b/f78 [0,4194304] 0 2024-02-15T18:27:05.825 INFO:tasks.workunit.client.0.smithi042.stdout:4/784: mknod da/d11/d29/d4a/c10f 0 2024-02-15T18:27:05.825 INFO:tasks.workunit.client.0.smithi042.stdout:4/785: truncate da/d11/d85/db5/d103/d66/d78/fbd 889483 0 2024-02-15T18:27:05.825 INFO:tasks.workunit.client.0.smithi042.stdout:4/786: write da/d11/d67/fd5 [2769883,109284] 0 2024-02-15T18:27:05.825 INFO:tasks.workunit.client.0.smithi042.stdout:5/810: dwrite d0/f9d [0,4194304] 0 2024-02-15T18:27:05.826 INFO:tasks.workunit.client.1.smithi191.stdout:9/395: dwrite d0/d7/dc/d30/d4f/f6d [0,4194304] 0 2024-02-15T18:27:05.826 INFO:tasks.workunit.client.0.smithi042.stdout:2/740: creat d0/d8/d10/d31/d79/fea x:0 0 0 2024-02-15T18:27:05.829 INFO:tasks.workunit.client.0.smithi042.stdout:3/755: creat d1/d9c/ff9 x:0 0 0 2024-02-15T18:27:05.834 INFO:tasks.workunit.client.0.smithi042.stdout:7/783: sync 2024-02-15T18:27:05.845 INFO:tasks.workunit.client.0.smithi042.stdout:1/793: dwrite de/d15/d2c/d51/d70/fe9 [0,4194304] 0 2024-02-15T18:27:05.845 INFO:tasks.workunit.client.0.smithi042.stdout:9/776: dread d6/dd/d9/d59/d5c/fb7 [0,4194304] 0 2024-02-15T18:27:05.849 INFO:tasks.workunit.client.0.smithi042.stdout:5/811: mknod d0/d14/d88/dfe/c11b 0 2024-02-15T18:27:05.850 INFO:tasks.workunit.client.0.smithi042.stdout:2/741: creat d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/feb x:0 0 0 2024-02-15T18:27:05.851 INFO:tasks.workunit.client.0.smithi042.stdout:7/784: creat d10/f103 x:0 0 0 2024-02-15T18:27:05.851 INFO:tasks.workunit.client.0.smithi042.stdout:7/785: write d10/d49/d6d/fd7 [442901,105652] 0 2024-02-15T18:27:05.852 INFO:tasks.workunit.client.0.smithi042.stdout:9/777: mkdir d6/da/d2f/d55/d73/d10b 0 2024-02-15T18:27:05.853 INFO:tasks.workunit.client.0.smithi042.stdout:1/794: creat de/d15/d67/d74/d7e/f12d x:0 0 0 2024-02-15T18:27:05.856 INFO:tasks.workunit.client.0.smithi042.stdout:5/812: symlink d0/dda/dc7/de1/l11c 0 2024-02-15T18:27:05.858 INFO:tasks.workunit.client.0.smithi042.stdout:2/742: rename d0/d8/d10/d31/d79/ce4 to d0/d8/d20/d7d/d94/cec 0 2024-02-15T18:27:05.858 INFO:tasks.workunit.client.0.smithi042.stdout:2/743: truncate d0/d8/d20/d7d/d7e/dc7/f73 4632882 0 2024-02-15T18:27:05.858 INFO:tasks.workunit.client.0.smithi042.stdout:9/778: creat d6/da/d1d/d2d/d40/d5f/f10c x:0 0 0 2024-02-15T18:27:05.859 INFO:tasks.workunit.client.0.smithi042.stdout:1/795: symlink de/d54/dbf/dd0/d7c/dd6/l12e 0 2024-02-15T18:27:05.860 INFO:tasks.workunit.client.0.smithi042.stdout:6/866: dwrite d9/d11/d19/d1c/d31/f6f [0,4194304] 0 2024-02-15T18:27:05.860 INFO:tasks.workunit.client.0.smithi042.stdout:8/700: dwrite dd/de/da4/fa3 [0,4194304] 0 2024-02-15T18:27:05.860 INFO:tasks.workunit.client.0.smithi042.stdout:5/813: read d0/dda/dc7/d52/f9e [24449,81448] 0 2024-02-15T18:27:05.861 INFO:tasks.workunit.client.1.smithi191.stdout:9/396: dwrite d0/d7/f11 [0,4194304] 0 2024-02-15T18:27:05.865 INFO:tasks.workunit.client.0.smithi042.stdout:7/786: rename d10/d12 to d10/d15/d8f/da5/dad/d104 0 2024-02-15T18:27:05.865 INFO:tasks.workunit.client.0.smithi042.stdout:3/756: dwrite d1/d11/d14/d4d/d73/f9d [0,4194304] 0 2024-02-15T18:27:05.865 INFO:tasks.workunit.client.0.smithi042.stdout:9/779: dread d6/da/d2f/d69/f8b [0,4194304] 0 2024-02-15T18:27:05.865 INFO:tasks.workunit.client.0.smithi042.stdout:5/814: write d0/d38/d53/d68/f6e [1524536,37871] 0 2024-02-15T18:27:05.865 INFO:tasks.workunit.client.0.smithi042.stdout:5/815: fsync d0/d38/d53/d64/f81 0 2024-02-15T18:27:05.866 INFO:tasks.workunit.client.0.smithi042.stdout:4/787: rmdir da/d3a/dc9 39 2024-02-15T18:27:05.868 INFO:tasks.workunit.client.0.smithi042.stdout:2/744: symlink d0/d8/d10/d31/led 0 2024-02-15T18:27:05.871 INFO:tasks.workunit.client.0.smithi042.stdout:4/788: write da/d2b/fd8 [69856,109687] 0 2024-02-15T18:27:05.873 INFO:tasks.workunit.client.1.smithi191.stdout:0/267: sync 2024-02-15T18:27:05.873 INFO:tasks.workunit.client.1.smithi191.stdout:1/477: sync 2024-02-15T18:27:05.876 INFO:tasks.workunit.client.0.smithi042.stdout:1/796: truncate de/d15/d67/f2a 2320933 0 2024-02-15T18:27:05.884 INFO:tasks.workunit.client.0.smithi042.stdout:6/867: getdents d9/d11/d19/d35/d49/d79 0 2024-02-15T18:27:05.885 INFO:tasks.workunit.client.1.smithi191.stdout:9/397: symlink d0/d29/d48/d66/l91 0 2024-02-15T18:27:05.885 INFO:tasks.workunit.client.1.smithi191.stdout:9/398: write d0/d7/dc/d30/d3b/f55 [1839526,130306] 0 2024-02-15T18:27:05.885 INFO:tasks.workunit.client.1.smithi191.stdout:9/399: readlink d0/d29/l7a 0 2024-02-15T18:27:05.887 INFO:tasks.workunit.client.1.smithi191.stdout:0/268: mknod d7/d1b/c5a 0 2024-02-15T18:27:05.888 INFO:tasks.workunit.client.1.smithi191.stdout:1/478: mknod d0/d8/c9f 0 2024-02-15T18:27:05.889 INFO:tasks.workunit.client.0.smithi042.stdout:8/701: symlink dd/de/da4/lcc 0 2024-02-15T18:27:05.890 INFO:tasks.workunit.client.1.smithi191.stdout:9/400: dread d0/d7/dc/d8c/f35 [0,4194304] 0 2024-02-15T18:27:05.890 INFO:tasks.workunit.client.1.smithi191.stdout:9/401: chown d0/d7/dc/d30/d3b 52 1 2024-02-15T18:27:05.890 INFO:tasks.workunit.client.1.smithi191.stdout:9/402: write d0/d7/dc/d30/f76 [3690226,29398] 0 2024-02-15T18:27:05.891 INFO:tasks.workunit.client.1.smithi191.stdout:0/269: creat d7/d42/f5b x:0 0 0 2024-02-15T18:27:05.891 INFO:tasks.workunit.client.1.smithi191.stdout:0/270: chown d7/da/d1a/d34/f29 10 1 2024-02-15T18:27:05.895 INFO:tasks.workunit.client.1.smithi191.stdout:1/479: truncate d0/d8/df/d12/f23 2550974 0 2024-02-15T18:27:05.896 INFO:tasks.workunit.client.0.smithi042.stdout:7/787: mknod d10/d15/d8f/c105 0 2024-02-15T18:27:05.896 INFO:tasks.workunit.client.0.smithi042.stdout:7/788: chown d10/d15/d8f/da5/dad/d104/d23/ld8 1732 1 2024-02-15T18:27:05.897 INFO:tasks.workunit.client.1.smithi191.stdout:9/403: rename d0/d7/dc/d30/d3b/d56/d60 to d0/d29/d48/d66/d92 0 2024-02-15T18:27:05.900 INFO:tasks.workunit.client.1.smithi191.stdout:9/404: fdatasync d0/d7/dc/d30/d3b/f55 0 2024-02-15T18:27:05.902 INFO:tasks.workunit.client.0.smithi042.stdout:3/757: unlink d1/d11/l38 0 2024-02-15T18:27:05.902 INFO:tasks.workunit.client.0.smithi042.stdout:3/758: stat d1/de/fef 0 2024-02-15T18:27:05.902 INFO:tasks.workunit.client.0.smithi042.stdout:3/759: dread - d1/d11/d2e/f79 zero size 2024-02-15T18:27:05.902 INFO:tasks.workunit.client.0.smithi042.stdout:3/760: fdatasync d1/d11/d14/d4d/d73/fdd 0 2024-02-15T18:27:05.902 INFO:tasks.workunit.client.0.smithi042.stdout:3/761: write d1/fa5 [4951392,50552] 0 2024-02-15T18:27:05.902 INFO:tasks.workunit.client.1.smithi191.stdout:0/271: getdents d7/da/d1a/d2c 0 2024-02-15T18:27:05.902 INFO:tasks.workunit.client.1.smithi191.stdout:0/272: write d7/d4a/f56 [1046616,28153] 0 2024-02-15T18:27:05.903 INFO:tasks.workunit.client.1.smithi191.stdout:1/480: creat d0/d8/df/d12/fa0 x:0 0 0 2024-02-15T18:27:05.904 INFO:tasks.workunit.client.1.smithi191.stdout:1/481: fsync d0/d8/df/d41/f9e 0 2024-02-15T18:27:05.907 INFO:tasks.workunit.client.0.smithi042.stdout:9/780: rename d6/dd/d9/fda to d6/da2/dfc/f10d 0 2024-02-15T18:27:05.907 INFO:tasks.workunit.client.0.smithi042.stdout:9/781: write d6/da/fbc [872879,115748] 0 2024-02-15T18:27:05.907 INFO:tasks.workunit.client.0.smithi042.stdout:9/782: write d6/da/d2f/d69/fd4 [830386,26189] 0 2024-02-15T18:27:05.908 INFO:tasks.workunit.client.1.smithi191.stdout:0/273: rename d7/da/d1a/d34/c53 to d7/d42/c5c 0 2024-02-15T18:27:05.908 INFO:tasks.workunit.client.1.smithi191.stdout:0/274: read d7/da/d17/f40 [2100665,30895] 0 2024-02-15T18:27:05.912 INFO:tasks.workunit.client.0.smithi042.stdout:5/816: creat d0/dda/d42/d7d/db5/f11d x:0 0 0 2024-02-15T18:27:05.912 INFO:tasks.workunit.client.0.smithi042.stdout:5/817: write d0/ffc [4882284,50687] 0 2024-02-15T18:27:05.915 INFO:tasks.workunit.client.1.smithi191.stdout:1/482: creat d0/d8/d48/fa1 x:0 0 0 2024-02-15T18:27:05.919 INFO:tasks.workunit.client.1.smithi191.stdout:9/405: dwrite d0/d7/f19 [0,4194304] 0 2024-02-15T18:27:05.919 INFO:tasks.workunit.client.1.smithi191.stdout:9/406: write d0/d7/dc/d30/d3b/f55 [2486513,130324] 0 2024-02-15T18:27:05.919 INFO:tasks.workunit.client.1.smithi191.stdout:9/407: chown d0/d29/d48/d66/d92/l7c 202204939 1 2024-02-15T18:27:05.919 INFO:tasks.workunit.client.1.smithi191.stdout:1/483: symlink d0/d8/d48/la2 0 2024-02-15T18:27:05.920 INFO:tasks.workunit.client.0.smithi042.stdout:2/745: link d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/c37 d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/d44/cee 0 2024-02-15T18:27:05.920 INFO:tasks.workunit.client.1.smithi191.stdout:9/408: write d0/d7/dc/d30/d3b/f55 [1736232,44094] 0 2024-02-15T18:27:05.920 INFO:tasks.workunit.client.1.smithi191.stdout:9/409: chown d0/d7/dc/d17/c34 63396 1 2024-02-15T18:27:05.920 INFO:tasks.workunit.client.1.smithi191.stdout:9/410: chown d0 2280 1 2024-02-15T18:27:05.923 INFO:tasks.workunit.client.1.smithi191.stdout:9/411: getdents d0/d7/dc/d32/d89 0 2024-02-15T18:27:05.924 INFO:tasks.workunit.client.1.smithi191.stdout:0/275: dwrite d7/da/d1a/d2c/f4b [0,4194304] 0 2024-02-15T18:27:05.928 INFO:tasks.workunit.client.0.smithi042.stdout:4/789: mkdir da/d3b/d41/d110 0 2024-02-15T18:27:05.928 INFO:tasks.workunit.client.0.smithi042.stdout:1/797: creat de/d15/d67/d74/d7e/f12f x:0 0 0 2024-02-15T18:27:05.929 INFO:tasks.workunit.client.0.smithi042.stdout:1/798: stat de/d54/dbf/dd0/lfb 0 2024-02-15T18:27:05.932 INFO:tasks.workunit.client.0.smithi042.stdout:6/868: mkdir d9/d11/d19/d35/d55/d133 0 2024-02-15T18:27:05.934 INFO:tasks.workunit.client.0.smithi042.stdout:8/702: chown dd/d11/f74 20 1 2024-02-15T18:27:05.934 INFO:tasks.workunit.client.0.smithi042.stdout:8/703: write dd/f99 [751512,4122] 0 2024-02-15T18:27:05.934 INFO:tasks.workunit.client.0.smithi042.stdout:8/704: fdatasync dd/de/d3c/f54 0 2024-02-15T18:27:05.934 INFO:tasks.workunit.client.1.smithi191.stdout:9/412: symlink d0/d7/dc/l93 0 2024-02-15T18:27:05.938 INFO:tasks.workunit.client.0.smithi042.stdout:5/818: truncate d0/d38/f5a 756549 0 2024-02-15T18:27:05.938 INFO:tasks.workunit.client.0.smithi042.stdout:2/746: creat d0/fef x:0 0 0 2024-02-15T18:27:05.941 INFO:tasks.workunit.client.0.smithi042.stdout:7/789: read d10/d43/f58 [1238677,56897] 0 2024-02-15T18:27:05.941 INFO:tasks.workunit.client.0.smithi042.stdout:9/783: dread d6/da/d1d/d2d/f83 [0,4194304] 0 2024-02-15T18:27:05.942 INFO:tasks.workunit.client.0.smithi042.stdout:4/790: mknod da/d11/d85/db5/d103/d66/d78/d2f/c111 0 2024-02-15T18:27:05.942 INFO:tasks.workunit.client.0.smithi042.stdout:6/869: creat d9/d11/d19/d35/dc4/f134 x:0 0 0 2024-02-15T18:27:05.942 INFO:tasks.workunit.client.0.smithi042.stdout:8/705: creat dd/de/da4/d67/fcd x:0 0 0 2024-02-15T18:27:05.942 INFO:tasks.workunit.client.0.smithi042.stdout:6/870: fsync d9/d3e/dd9/fdc 0 2024-02-15T18:27:05.942 INFO:tasks.workunit.client.0.smithi042.stdout:6/871: chown d9/d11/d46/d89/df4/f105 855085954 1 2024-02-15T18:27:05.947 INFO:tasks.workunit.client.0.smithi042.stdout:3/762: mknod d1/d11/d14/cfa 0 2024-02-15T18:27:05.948 INFO:tasks.workunit.client.0.smithi042.stdout:3/763: fsync d1/d11/d2e/d4c/f5b 0 2024-02-15T18:27:05.949 INFO:tasks.workunit.client.0.smithi042.stdout:5/819: creat d0/d12/d2a/d105/f11e x:0 0 0 2024-02-15T18:27:05.949 INFO:tasks.workunit.client.0.smithi042.stdout:5/820: getdents d0/dda/dc7/de1/dd8 0 2024-02-15T18:27:05.950 INFO:tasks.workunit.client.0.smithi042.stdout:0/757: sync 2024-02-15T18:27:05.952 INFO:tasks.workunit.client.0.smithi042.stdout:9/784: rename d6/da/d1d/l106 to d6/dd/d9/da1/l10e 0 2024-02-15T18:27:05.954 INFO:tasks.workunit.client.0.smithi042.stdout:3/764: symlink d1/de/d1a/de9/lfb 0 2024-02-15T18:27:05.954 INFO:tasks.workunit.client.0.smithi042.stdout:8/706: mkdir dd/de/da4/d67/d6f/d78/d9a/dce 0 2024-02-15T18:27:05.954 INFO:tasks.workunit.client.0.smithi042.stdout:3/765: truncate d1/d11/d14/f60 5129278 0 2024-02-15T18:27:05.956 INFO:tasks.workunit.client.0.smithi042.stdout:4/791: rename da/d3b/c58 to da/d11/d29/d4a/d4f/c112 0 2024-02-15T18:27:05.956 INFO:tasks.workunit.client.0.smithi042.stdout:4/792: write da/d11/d67/fe7 [955934,120619] 0 2024-02-15T18:27:05.956 INFO:tasks.workunit.client.0.smithi042.stdout:8/707: mknod dd/de/d33/ccf 0 2024-02-15T18:27:05.957 INFO:tasks.workunit.client.1.smithi191.stdout:0/276: dwrite d7/d4a/f56 [0,4194304] 0 2024-02-15T18:27:05.957 INFO:tasks.workunit.client.1.smithi191.stdout:0/277: chown d7/d42/c5c 43 1 2024-02-15T18:27:05.957 INFO:tasks.workunit.client.1.smithi191.stdout:0/278: dread - d7/da/d1a/d34/f52 zero size 2024-02-15T18:27:05.958 INFO:tasks.workunit.client.0.smithi042.stdout:5/821: write d0/d14/f1c [3653484,76422] 0 2024-02-15T18:27:05.958 INFO:tasks.workunit.client.0.smithi042.stdout:1/799: dwrite de/d15/d67/d63/f86 [4194304,4194304] 0 2024-02-15T18:27:05.958 INFO:tasks.workunit.client.0.smithi042.stdout:9/785: rename d6/dd/d9/d59/fd2 to d6/da/d2f/d6c/f10f 0 2024-02-15T18:27:05.959 INFO:tasks.workunit.client.0.smithi042.stdout:8/708: mkdir dd/de/da4/dd0 0 2024-02-15T18:27:05.964 INFO:tasks.workunit.client.0.smithi042.stdout:2/747: dwrite d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/f93 [0,4194304] 0 2024-02-15T18:27:05.965 INFO:tasks.workunit.client.0.smithi042.stdout:1/800: dread de/d6d/d9e/dcd/fe4 [0,4194304] 0 2024-02-15T18:27:05.965 INFO:tasks.workunit.client.1.smithi191.stdout:0/279: link d7/da/f18 d7/da/d17/f5d 0 2024-02-15T18:27:05.968 INFO:tasks.workunit.client.0.smithi042.stdout:5/822: mknod d0/c11f 0 2024-02-15T18:27:05.968 INFO:tasks.workunit.client.0.smithi042.stdout:9/786: symlink d6/da/d2f/l110 0 2024-02-15T18:27:05.968 INFO:tasks.workunit.client.1.smithi191.stdout:9/413: dwrite d0/f18 [0,4194304] 0 2024-02-15T18:27:05.968 INFO:tasks.workunit.client.1.smithi191.stdout:9/414: write d0/d7/dc/d8c/f72 [608686,90265] 0 2024-02-15T18:27:05.972 INFO:tasks.workunit.client.1.smithi191.stdout:1/484: dwrite d0/d2f/d51/f1d [0,4194304] 0 2024-02-15T18:27:05.973 INFO:tasks.workunit.client.1.smithi191.stdout:1/485: dread d0/d8/df/d12/d2a/f42 [0,4194304] 0 2024-02-15T18:27:05.974 INFO:tasks.workunit.client.0.smithi042.stdout:2/748: mknod d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/d44/cf0 0 2024-02-15T18:27:05.976 INFO:tasks.workunit.client.0.smithi042.stdout:1/801: mkdir de/d54/d130 0 2024-02-15T18:27:05.980 INFO:tasks.workunit.client.0.smithi042.stdout:5/823: creat d0/dda/dc7/de1/dd8/f120 x:0 0 0 2024-02-15T18:27:05.981 INFO:tasks.workunit.client.1.smithi191.stdout:9/415: write d0/d7/dc/d30/f4e [3209192,44082] 0 2024-02-15T18:27:05.982 INFO:tasks.workunit.client.1.smithi191.stdout:1/486: mknod d0/d2f/d51/d5e/d68/ca3 0 2024-02-15T18:27:05.982 INFO:tasks.workunit.client.1.smithi191.stdout:1/487: chown d0/d8/df/d12/fa0 1 1 2024-02-15T18:27:05.982 INFO:tasks.workunit.client.0.smithi042.stdout:4/793: dread da/f25 [0,4194304] 0 2024-02-15T18:27:05.983 INFO:tasks.workunit.client.1.smithi191.stdout:9/416: dread d0/d7/dc/d8c/f35 [0,4194304] 0 2024-02-15T18:27:05.984 INFO:tasks.workunit.client.0.smithi042.stdout:6/872: dwrite d9/d11/d19/d35/d55/d92/f95 [0,4194304] 0 2024-02-15T18:27:05.984 INFO:tasks.workunit.client.0.smithi042.stdout:9/787: mkdir d6/da/d111 0 2024-02-15T18:27:05.984 INFO:tasks.workunit.client.0.smithi042.stdout:2/749: mkdir d0/d8/d20/d41/df1 0 2024-02-15T18:27:05.984 INFO:tasks.workunit.client.0.smithi042.stdout:2/750: chown d0/d8/d10/d31/l72 92749 1 2024-02-15T18:27:05.985 INFO:tasks.workunit.client.1.smithi191.stdout:1/488: rename d0/d2f/d51/d5e/d68/ca3 to d0/d8/df/d12/ca4 0 2024-02-15T18:27:05.988 INFO:tasks.workunit.client.0.smithi042.stdout:4/794: dread da/d11/d85/db5/d103/d66/d78/d2f/d4d/f48 [0,4194304] 0 2024-02-15T18:27:05.989 INFO:tasks.workunit.client.0.smithi042.stdout:4/795: chown da/d11/d67/fe7 1 1 2024-02-15T18:27:05.989 INFO:tasks.workunit.client.1.smithi191.stdout:9/417: write d0/d7/dc/d17/f61 [2856616,121155] 0 2024-02-15T18:27:05.990 INFO:tasks.workunit.client.1.smithi191.stdout:9/418: write d0/d7/dc/d17/f84 [375963,22759] 0 2024-02-15T18:27:05.992 INFO:tasks.workunit.client.0.smithi042.stdout:5/824: mknod d0/dda/dc7/de1/dd8/c121 0 2024-02-15T18:27:05.993 INFO:tasks.workunit.client.0.smithi042.stdout:1/802: dread de/d15/d2c/d51/d70/fe9 [0,4194304] 0 2024-02-15T18:27:05.999 INFO:tasks.workunit.client.0.smithi042.stdout:6/873: symlink d9/d11/d19/d35/dc4/d123/l135 0 2024-02-15T18:27:06.001 INFO:tasks.workunit.client.0.smithi042.stdout:9/788: getdents d6/da/d2f/d55/d73/d8d 0 2024-02-15T18:27:06.001 INFO:tasks.workunit.client.0.smithi042.stdout:9/789: write d6/dd/d9/d59/f63 [4335304,58394] 0 2024-02-15T18:27:06.001 INFO:tasks.workunit.client.0.smithi042.stdout:9/790: chown d6/dd/d9/d59/f5e 10300 1 2024-02-15T18:27:06.001 INFO:tasks.workunit.client.0.smithi042.stdout:7/790: dwrite d10/d15/d8f/da5/dad/de0/ff1 [0,4194304] 0 2024-02-15T18:27:06.001 INFO:tasks.workunit.client.0.smithi042.stdout:7/791: chown d10/d15/d39/d6c/d92/cd1 14 1 2024-02-15T18:27:06.001 INFO:tasks.workunit.client.0.smithi042.stdout:7/792: readlink d10/d49/d6d/d6e/lb4 0 2024-02-15T18:27:06.004 INFO:tasks.workunit.client.0.smithi042.stdout:3/766: dwrite d1/d11/d14/d4d/d73/fdd [0,4194304] 0 2024-02-15T18:27:06.004 INFO:tasks.workunit.client.0.smithi042.stdout:7/793: dread d10/d15/d8f/da5/dad/d104/f96 [0,4194304] 0 2024-02-15T18:27:06.007 INFO:tasks.workunit.client.0.smithi042.stdout:8/709: dwrite dd/de/d12/f1e [0,4194304] 0 2024-02-15T18:27:06.007 INFO:tasks.workunit.client.0.smithi042.stdout:8/710: write dd/de/d33/d66/d4e/f6c [902707,61367] 0 2024-02-15T18:27:06.008 INFO:tasks.workunit.client.0.smithi042.stdout:4/796: mkdir da/d11/d85/db5/d103/d66/d78/d53/d82/d113 0 2024-02-15T18:27:06.009 INFO:tasks.workunit.client.0.smithi042.stdout:5/825: getdents d0/d50/df7 0 2024-02-15T18:27:06.009 INFO:tasks.workunit.client.0.smithi042.stdout:5/826: read - d0/dda/dc7/de1/dd8/f120 zero size 2024-02-15T18:27:06.009 INFO:tasks.workunit.client.0.smithi042.stdout:5/827: fdatasync d0/d39/d8d/fc6 0 2024-02-15T18:27:06.012 INFO:tasks.workunit.client.0.smithi042.stdout:3/767: dread d1/d11/d14/f47 [0,4194304] 0 2024-02-15T18:27:06.012 INFO:tasks.workunit.client.0.smithi042.stdout:3/768: fsync d1/de/fe2 0 2024-02-15T18:27:06.012 INFO:tasks.workunit.client.0.smithi042.stdout:3/769: fdatasync d1/de/fe8 0 2024-02-15T18:27:06.012 INFO:tasks.workunit.client.0.smithi042.stdout:3/770: fdatasync d1/fa5 0 2024-02-15T18:27:06.012 INFO:tasks.workunit.client.0.smithi042.stdout:3/771: truncate d1/de/d1a/f7f 5497791 0 2024-02-15T18:27:06.012 INFO:tasks.workunit.client.0.smithi042.stdout:3/772: readlink d1/d11/d2e/d4c/lcb 0 2024-02-15T18:27:06.012 INFO:tasks.workunit.client.0.smithi042.stdout:3/773: write d1/d11/d14/d4d/d6c/f83 [4805959,107650] 0 2024-02-15T18:27:06.018 INFO:tasks.workunit.client.1.smithi191.stdout:0/280: dwrite d7/da/d17/f40 [0,4194304] 0 2024-02-15T18:27:06.018 INFO:tasks.workunit.client.0.smithi042.stdout:1/803: getdents de/d54/dbf/dd0/d7c/dd6/ddf/dca 0 2024-02-15T18:27:06.020 INFO:tasks.workunit.client.0.smithi042.stdout:6/874: mknod d9/d11/d19/d35/d55/d92/d9e/d12e/c136 0 2024-02-15T18:27:06.022 INFO:tasks.workunit.client.1.smithi191.stdout:0/281: creat d7/da/d1a/d34/f5e x:0 0 0 2024-02-15T18:27:06.022 INFO:tasks.workunit.client.1.smithi191.stdout:0/282: write d7/da/d17/f5d [922592,58561] 0 2024-02-15T18:27:06.022 INFO:tasks.workunit.client.0.smithi042.stdout:9/791: truncate d6/dd/d9/f72 648153 0 2024-02-15T18:27:06.023 INFO:tasks.workunit.client.1.smithi191.stdout:3/309: sync 2024-02-15T18:27:06.025 INFO:tasks.workunit.client.1.smithi191.stdout:3/310: creat d6/d3b/f6f x:0 0 0 2024-02-15T18:27:06.026 INFO:tasks.workunit.client.0.smithi042.stdout:2/751: dwrite d0/d8/d10/d5a/fbc [0,4194304] 0 2024-02-15T18:27:06.027 INFO:tasks.workunit.client.1.smithi191.stdout:1/489: dwrite d0/d2f/d77/f81 [0,4194304] 0 2024-02-15T18:27:06.028 INFO:tasks.workunit.client.1.smithi191.stdout:3/311: creat d6/d3b/d42/f70 x:0 0 0 2024-02-15T18:27:06.028 INFO:tasks.workunit.client.0.smithi042.stdout:7/794: symlink d10/d15/d8f/da5/dad/de0/l106 0 2024-02-15T18:27:06.029 INFO:tasks.workunit.client.1.smithi191.stdout:1/490: write d0/d8/df/d12/d2a/f3a [2701656,3970] 0 2024-02-15T18:27:06.029 INFO:tasks.workunit.client.0.smithi042.stdout:8/711: dwrite dd/de/d33/f42 [0,4194304] 0 2024-02-15T18:27:06.031 INFO:tasks.workunit.client.1.smithi191.stdout:9/419: dwrite d0/d7/fd [0,4194304] 0 2024-02-15T18:27:06.031 INFO:tasks.workunit.client.1.smithi191.stdout:0/283: dread d7/fb [4194304,4194304] 0 2024-02-15T18:27:06.037 INFO:tasks.workunit.client.0.smithi042.stdout:8/712: dread dd/de/d12/f68 [0,4194304] 0 2024-02-15T18:27:06.039 INFO:tasks.workunit.client.1.smithi191.stdout:3/312: dread d6/d27/d4d/d66/f61 [0,4194304] 0 2024-02-15T18:27:06.039 INFO:tasks.workunit.client.1.smithi191.stdout:3/313: truncate d6/d3b/d42/f2a 818190 0 2024-02-15T18:27:06.040 INFO:tasks.workunit.client.1.smithi191.stdout:1/491: mknod d0/d8/df/d12/d33/ca5 0 2024-02-15T18:27:06.040 INFO:tasks.workunit.client.1.smithi191.stdout:1/492: chown d0/d2f/d51 855756 1 2024-02-15T18:27:06.041 INFO:tasks.workunit.client.0.smithi042.stdout:4/797: creat da/d3a/dc9/f114 x:0 0 0 2024-02-15T18:27:06.043 INFO:tasks.workunit.client.0.smithi042.stdout:5/828: rename d0/l4 to d0/dda/d42/d7d/db5/l122 0 2024-02-15T18:27:06.045 INFO:tasks.workunit.client.0.smithi042.stdout:3/774: link d1/d11/d14/d4d/f4f d1/d11/d2e/de7/ffc 0 2024-02-15T18:27:06.045 INFO:tasks.workunit.client.0.smithi042.stdout:3/775: fdatasync d1/d11/d14/f60 0 2024-02-15T18:27:06.046 INFO:tasks.workunit.client.1.smithi191.stdout:0/284: truncate d7/da/d17/d31/f36 1685235 0 2024-02-15T18:27:06.046 INFO:tasks.workunit.client.1.smithi191.stdout:0/285: chown d7/da/d17/f40 430 1 2024-02-15T18:27:06.046 INFO:tasks.workunit.client.0.smithi042.stdout:1/804: symlink de/d54/dbf/dd0/d7c/db3/l131 0 2024-02-15T18:27:06.047 INFO:tasks.workunit.client.1.smithi191.stdout:0/286: write d7/da/f25 [1048714,121487] 0 2024-02-15T18:27:06.047 INFO:tasks.workunit.client.1.smithi191.stdout:0/287: truncate d7/da/d1a/d34/f5e 1018691 0 2024-02-15T18:27:06.047 INFO:tasks.workunit.client.0.smithi042.stdout:1/805: write de/d54/dbf/dd0/d7c/dd6/ddf/dca/ffd [2750956,53304] 0 2024-02-15T18:27:06.047 INFO:tasks.workunit.client.0.smithi042.stdout:1/806: read - de/d15/d67/d63/f11a zero size 2024-02-15T18:27:06.048 INFO:tasks.workunit.client.1.smithi191.stdout:3/314: mknod d6/c71 0 2024-02-15T18:27:06.048 INFO:tasks.workunit.client.1.smithi191.stdout:3/315: chown d6/d3b/d42 21909805 1 2024-02-15T18:27:06.048 INFO:tasks.workunit.client.1.smithi191.stdout:3/316: chown d6/d3b/d42/f70 906 1 2024-02-15T18:27:06.048 INFO:tasks.workunit.client.1.smithi191.stdout:3/317: write d6/d27/d4d/d66/f61 [2747201,11883] 0 2024-02-15T18:27:06.049 INFO:tasks.workunit.client.0.smithi042.stdout:6/875: unlink d9/d11/d19/d1c/d31/d53/c10e 0 2024-02-15T18:27:06.049 INFO:tasks.workunit.client.0.smithi042.stdout:6/876: getdents d9/d11/d46/df8/d125 0 2024-02-15T18:27:06.054 INFO:tasks.workunit.client.1.smithi191.stdout:1/493: creat d0/fa6 x:0 0 0 2024-02-15T18:27:06.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:04.682736+0000 mgr.smithi042.kfoihb (mgr.24435) 41 : cluster [DBG] pgmap v31: 65 pgs: 65 active+clean; 2.1 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 118 MiB/s rd, 261 MiB/s wr, 382 op/s 2024-02-15T18:27:06.060 INFO:tasks.workunit.client.0.smithi042.stdout:9/792: getdents d6/dd/d9/d59/dbf 0 2024-02-15T18:27:06.065 INFO:tasks.workunit.client.0.smithi042.stdout:2/752: creat d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d2e/ff2 x:0 0 0 2024-02-15T18:27:06.067 INFO:tasks.workunit.client.1.smithi191.stdout:0/288: rename d7/d42 to d7/da/d17/d31/d5f 0 2024-02-15T18:27:06.070 INFO:tasks.workunit.client.0.smithi042.stdout:7/795: mknod d10/d15/d39/c107 0 2024-02-15T18:27:06.075 INFO:tasks.workunit.client.0.smithi042.stdout:2/753: dread d0/d8/d10/d5a/fbc [0,4194304] 0 2024-02-15T18:27:06.075 INFO:tasks.workunit.client.0.smithi042.stdout:2/754: dread - d0/d8/d10/d7b/fa4 zero size 2024-02-15T18:27:06.075 INFO:tasks.workunit.client.0.smithi042.stdout:2/755: fdatasync d0/d8/d10/d5a/fbc 0 2024-02-15T18:27:06.076 INFO:tasks.workunit.client.0.smithi042.stdout:2/756: write d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d2e/ff2 [215643,25620] 0 2024-02-15T18:27:06.076 INFO:tasks.workunit.client.0.smithi042.stdout:2/757: readlink d0/d8/d20/d7d/d7e/dc7/d8b/ldd 0 2024-02-15T18:27:06.076 INFO:tasks.workunit.client.0.smithi042.stdout:2/758: write d0/d8/d10/d31/fa1 [1838138,87943] 0 2024-02-15T18:27:06.077 INFO:tasks.workunit.client.1.smithi191.stdout:3/318: mkdir d6/d5d/d72 0 2024-02-15T18:27:06.077 INFO:tasks.workunit.client.1.smithi191.stdout:3/319: write d6/d3b/d42/f51 [999254,69286] 0 2024-02-15T18:27:06.077 INFO:tasks.workunit.client.1.smithi191.stdout:1/494: symlink d0/d2f/d51/d54/d7b/la7 0 2024-02-15T18:27:06.080 INFO:tasks.workunit.client.1.smithi191.stdout:0/289: link d7/da/f18 d7/da/d1a/d34/d27/f60 0 2024-02-15T18:27:06.080 INFO:tasks.workunit.client.1.smithi191.stdout:0/290: fdatasync d7/da/d17/f5d 0 2024-02-15T18:27:06.080 INFO:tasks.workunit.client.1.smithi191.stdout:0/291: chown d7/da/d17/f40 915977 1 2024-02-15T18:27:06.080 INFO:tasks.workunit.client.1.smithi191.stdout:0/292: chown d7/da/d17/d31/d5f/l48 1019681 1 2024-02-15T18:27:06.084 INFO:tasks.workunit.client.1.smithi191.stdout:3/320: rename d6/d27/d4d/d66 to d6/d1b/d60/d73 0 2024-02-15T18:27:06.085 INFO:tasks.workunit.client.1.smithi191.stdout:1/495: creat d0/fa8 x:0 0 0 2024-02-15T18:27:06.087 INFO:tasks.workunit.client.1.smithi191.stdout:3/321: read d6/d3b/d42/f4a [2418729,93662] 0 2024-02-15T18:27:06.088 INFO:tasks.workunit.client.0.smithi042.stdout:8/713: creat dd/de/d33/d66/d4e/fd1 x:0 0 0 2024-02-15T18:27:06.088 INFO:tasks.workunit.client.0.smithi042.stdout:8/714: truncate dd/de/d12/fc8 5011968 0 2024-02-15T18:27:06.089 INFO:tasks.workunit.client.0.smithi042.stdout:4/798: creat da/d11/d85/db5/f115 x:0 0 0 2024-02-15T18:27:06.089 INFO:tasks.workunit.client.0.smithi042.stdout:8/715: dread dd/de/d33/d3b/f46 [0,4194304] 0 2024-02-15T18:27:06.090 INFO:tasks.workunit.client.0.smithi042.stdout:8/716: readlink dd/de/da4/l50 0 2024-02-15T18:27:06.090 INFO:tasks.workunit.client.1.smithi191.stdout:3/322: read d6/fa [2399443,62068] 0 2024-02-15T18:27:06.090 INFO:tasks.workunit.client.1.smithi191.stdout:3/323: truncate d6/d3b/f13 5032352 0 2024-02-15T18:27:06.092 INFO:tasks.workunit.client.0.smithi042.stdout:0/758: sync 2024-02-15T18:27:06.092 INFO:tasks.workunit.client.0.smithi042.stdout:8/717: write dd/de/d12/f45 [1435369,26392] 0 2024-02-15T18:27:06.096 INFO:tasks.workunit.client.1.smithi191.stdout:1/496: creat d0/d8/df/d12/d2a/d2b/d78/d60/d74/fa9 x:0 0 0 2024-02-15T18:27:06.096 INFO:tasks.workunit.client.0.smithi042.stdout:5/829: creat d0/d14/d88/f123 x:0 0 0 2024-02-15T18:27:06.098 INFO:tasks.workunit.client.1.smithi191.stdout:1/497: dread d0/d8/f30 [0,4194304] 0 2024-02-15T18:27:06.099 INFO:tasks.workunit.client.0.smithi042.stdout:2/759: dwrite d0/d35/d9e/fe9 [0,4194304] 0 2024-02-15T18:27:06.099 INFO:tasks.workunit.client.0.smithi042.stdout:2/760: fdatasync d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/f43 0 2024-02-15T18:27:06.099 INFO:tasks.workunit.client.0.smithi042.stdout:2/761: getdents d0/d8/d20/d3f 0 2024-02-15T18:27:06.099 INFO:tasks.workunit.client.0.smithi042.stdout:2/762: write d0/d8/d20/d7d/d7e/dc7/fa9 [1767376,98010] 0 2024-02-15T18:27:06.099 INFO:tasks.workunit.client.0.smithi042.stdout:3/776: truncate d1/d11/d14/d4d/d6c/f9b 1352169 0 2024-02-15T18:27:06.100 INFO:tasks.workunit.client.0.smithi042.stdout:3/777: fdatasync d1/d11/f45 0 2024-02-15T18:27:06.100 INFO:tasks.workunit.client.1.smithi191.stdout:3/324: rename d6/d3b/d30/f3c to d6/d1b/d60/d73/f74 0 2024-02-15T18:27:06.102 INFO:tasks.workunit.client.1.smithi191.stdout:4/421: sync 2024-02-15T18:27:06.102 INFO:tasks.workunit.client.1.smithi191.stdout:1/498: mkdir d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa 0 2024-02-15T18:27:06.103 INFO:tasks.workunit.client.1.smithi191.stdout:8/506: sync 2024-02-15T18:27:06.103 INFO:tasks.workunit.client.1.smithi191.stdout:5/188: sync 2024-02-15T18:27:06.103 INFO:tasks.workunit.client.1.smithi191.stdout:5/189: fdatasync d0/d18/f2e 0 2024-02-15T18:27:06.103 INFO:tasks.workunit.client.1.smithi191.stdout:7/411: sync 2024-02-15T18:27:06.103 INFO:tasks.workunit.client.1.smithi191.stdout:2/362: sync 2024-02-15T18:27:06.103 INFO:tasks.workunit.client.0.smithi042.stdout:1/807: link de/d15/d67/cb4 de/d54/dbf/dd0/d7c/dd6/ddf/d88/d111/c132 0 2024-02-15T18:27:06.104 INFO:tasks.workunit.client.1.smithi191.stdout:6/347: sync 2024-02-15T18:27:06.104 INFO:tasks.workunit.client.1.smithi191.stdout:3/325: write d6/d3b/d42/f4a [2016851,60623] 0 2024-02-15T18:27:06.104 INFO:tasks.workunit.client.1.smithi191.stdout:3/326: dread - d6/d1b/d60/d73/f34 zero size 2024-02-15T18:27:06.105 INFO:tasks.workunit.client.1.smithi191.stdout:4/422: truncate d2/d5/d7/d2f/d41/f92 253109 0 2024-02-15T18:27:06.106 INFO:tasks.workunit.client.0.smithi042.stdout:6/877: symlink d9/d11/d46/d89/df4/l137 0 2024-02-15T18:27:06.108 INFO:tasks.workunit.client.1.smithi191.stdout:1/499: symlink d0/d8/d48/lab 0 2024-02-15T18:27:06.108 INFO:tasks.workunit.client.1.smithi191.stdout:1/500: truncate d0/d8/df/f85 487482 0 2024-02-15T18:27:06.110 INFO:tasks.workunit.client.0.smithi042.stdout:9/793: rename d6/dd/d9/d59/d5c/f7e to d6/da/d111/f112 0 2024-02-15T18:27:06.110 INFO:tasks.workunit.client.0.smithi042.stdout:9/794: chown d6/da/d2f/d54 189198 1 2024-02-15T18:27:06.111 INFO:tasks.workunit.client.1.smithi191.stdout:7/412: symlink d0/db/d65/l94 0 2024-02-15T18:27:06.111 INFO:tasks.workunit.client.1.smithi191.stdout:7/413: chown d0/db/d65/d11/d57/l72 164217852 1 2024-02-15T18:27:06.111 INFO:tasks.workunit.client.1.smithi191.stdout:7/414: stat d0/db/d26/d5f/d77/d1d 0 2024-02-15T18:27:06.113 INFO:tasks.workunit.client.0.smithi042.stdout:0/759: stat d4/d38/d5c/la9 0 2024-02-15T18:27:06.116 INFO:tasks.workunit.client.1.smithi191.stdout:6/348: truncate d8/d19/f1d 3345246 0 2024-02-15T18:27:06.116 INFO:tasks.workunit.client.1.smithi191.stdout:4/423: truncate d2/d5/d14/d47/d78/f4f 987451 0 2024-02-15T18:27:06.116 INFO:tasks.workunit.client.1.smithi191.stdout:4/424: chown d2/d5/d86/f96 1 1 2024-02-15T18:27:06.116 INFO:tasks.workunit.client.0.smithi042.stdout:5/830: unlink d0/dda/dc7/d52/l6f 0 2024-02-15T18:27:06.116 INFO:tasks.workunit.client.0.smithi042.stdout:5/831: chown d0/d38/d53/d68/dec 134939444 1 2024-02-15T18:27:06.121 INFO:tasks.workunit.client.1.smithi191.stdout:2/363: dread df/d12/d19/d1a/d35/d47/f51 [0,4194304] 0 2024-02-15T18:27:06.126 INFO:tasks.workunit.client.1.smithi191.stdout:4/425: mkdir d2/d5/d7/d90/da5 0 2024-02-15T18:27:06.126 INFO:tasks.workunit.client.1.smithi191.stdout:4/426: readlink d2/d5/d7/l1b 0 2024-02-15T18:27:06.127 INFO:tasks.workunit.client.1.smithi191.stdout:2/364: creat df/d12/d19/d1a/d35/d47/d4b/f7e x:0 0 0 2024-02-15T18:27:06.128 INFO:tasks.workunit.client.0.smithi042.stdout:2/763: getdents d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/d44 0 2024-02-15T18:27:06.133 INFO:tasks.workunit.client.0.smithi042.stdout:4/799: dwrite da/d3a/dbb/fc3 [0,4194304] 0 2024-02-15T18:27:06.133 INFO:tasks.workunit.client.0.smithi042.stdout:4/800: write da/d3a/dc9/fd9 [286988,6828] 0 2024-02-15T18:27:06.133 INFO:tasks.workunit.client.0.smithi042.stdout:3/778: dwrite d1/d11/d14/d4d/d73/ff5 [0,4194304] 0 2024-02-15T18:27:06.134 INFO:tasks.workunit.client.1.smithi191.stdout:2/365: unlink df/d12/d19/l4e 0 2024-02-15T18:27:06.136 INFO:tasks.workunit.client.0.smithi042.stdout:1/808: creat de/d54/d9a/f133 x:0 0 0 2024-02-15T18:27:06.136 INFO:tasks.workunit.client.0.smithi042.stdout:1/809: write de/d15/d2c/d51/d70/fe9 [4837036,28222] 0 2024-02-15T18:27:06.145 INFO:tasks.workunit.client.0.smithi042.stdout:6/878: rmdir d9/d11/d19/d35/d55/d133 0 2024-02-15T18:27:06.146 INFO:tasks.workunit.client.1.smithi191.stdout:1/501: rmdir d0/d8/df/d12/d2a/d2b/d78 39 2024-02-15T18:27:06.148 INFO:tasks.workunit.client.1.smithi191.stdout:2/366: mknod df/d12/d19/d1a/d52/c7f 0 2024-02-15T18:27:06.148 INFO:tasks.workunit.client.1.smithi191.stdout:2/367: fdatasync df/d12/d19/d1a/d35/d5b/f6e 0 2024-02-15T18:27:06.149 INFO:tasks.workunit.client.1.smithi191.stdout:2/368: write df/d12/f39 [893317,35261] 0 2024-02-15T18:27:06.150 INFO:tasks.workunit.client.1.smithi191.stdout:1/502: creat d0/d8/d48/fac x:0 0 0 2024-02-15T18:27:06.151 INFO:tasks.workunit.client.0.smithi042.stdout:9/795: creat d6/da/d2f/d55/d73/d8d/dcf/f113 x:0 0 0 2024-02-15T18:27:06.152 INFO:tasks.workunit.client.0.smithi042.stdout:9/796: fsync d6/dd/d9/f9f 0 2024-02-15T18:27:06.153 INFO:tasks.workunit.client.1.smithi191.stdout:7/415: dwrite d0/db/d65/d11/f90 [0,4194304] 0 2024-02-15T18:27:06.153 INFO:tasks.workunit.client.1.smithi191.stdout:7/416: truncate d0/db/d65/f3e 980880 0 2024-02-15T18:27:06.153 INFO:tasks.workunit.client.1.smithi191.stdout:1/503: rmdir d0/d8/df/d12/d33/d49 39 2024-02-15T18:27:06.153 INFO:tasks.workunit.client.1.smithi191.stdout:3/327: dwrite d6/d3b/d30/f46 [0,4194304] 0 2024-02-15T18:27:06.154 INFO:tasks.workunit.client.1.smithi191.stdout:2/369: dread df/d12/d19/d1a/f21 [0,4194304] 0 2024-02-15T18:27:06.154 INFO:tasks.workunit.client.1.smithi191.stdout:2/370: read - df/d12/d19/d1a/d35/d5b/f66 zero size 2024-02-15T18:27:06.156 INFO:tasks.workunit.client.0.smithi042.stdout:0/760: rename d4/d9/d34/f36 to d4/d9/d5a/dfc/ffd 0 2024-02-15T18:27:06.158 INFO:tasks.workunit.client.1.smithi191.stdout:5/190: dwrite d0/d14/d28/d31/d38/f40 [0,4194304] 0 2024-02-15T18:27:06.158 INFO:tasks.workunit.client.1.smithi191.stdout:5/191: truncate d0/d14/d28/d31/d38/f40 4667736 0 2024-02-15T18:27:06.158 INFO:tasks.workunit.client.1.smithi191.stdout:6/349: dwrite d8/f54 [8388608,4194304] 0 2024-02-15T18:27:06.158 INFO:tasks.workunit.client.1.smithi191.stdout:4/427: dwrite d2/d5/d7/d2f/f51 [0,4194304] 0 2024-02-15T18:27:06.159 INFO:tasks.workunit.client.1.smithi191.stdout:4/428: chown d2/f81 0 1 2024-02-15T18:27:06.161 INFO:tasks.workunit.client.1.smithi191.stdout:4/429: dread d2/d5/d7/d90/f6f [0,4194304] 0 2024-02-15T18:27:06.162 INFO:tasks.workunit.client.0.smithi042.stdout:5/832: creat d0/dda/dc7/f124 x:0 0 0 2024-02-15T18:27:06.164 INFO:tasks.workunit.client.1.smithi191.stdout:7/417: unlink d0/db/d26/d5f/d77/d1d/f87 0 2024-02-15T18:27:06.164 INFO:tasks.workunit.client.0.smithi042.stdout:9/797: dread d6/da/d2f/d55/f6d [0,4194304] 0 2024-02-15T18:27:06.165 INFO:tasks.workunit.client.0.smithi042.stdout:2/764: mkdir d0/d8/d20/d3f/dd6/df3 0 2024-02-15T18:27:06.165 INFO:tasks.workunit.client.0.smithi042.stdout:2/765: readlink d0/d8/d10/d31/d3a/d53/l22 0 2024-02-15T18:27:06.170 INFO:tasks.workunit.client.1.smithi191.stdout:1/504: rename d0/d8/df/d12/d2a/d2b/d78/d60/l6e to d0/d8/d48/lad 0 2024-02-15T18:27:06.172 INFO:tasks.workunit.client.0.smithi042.stdout:4/801: unlink da/d11/d29/d4a/fbe 0 2024-02-15T18:27:06.174 INFO:tasks.workunit.client.1.smithi191.stdout:3/328: link d6/d27/f32 d6/d1b/d60/d73/f75 0 2024-02-15T18:27:06.174 INFO:tasks.workunit.client.1.smithi191.stdout:3/329: read d6/d3b/d30/f45 [1621537,12285] 0 2024-02-15T18:27:06.176 INFO:tasks.workunit.client.1.smithi191.stdout:2/371: creat df/d12/d19/d1a/d35/f80 x:0 0 0 2024-02-15T18:27:06.176 INFO:tasks.workunit.client.1.smithi191.stdout:2/372: chown df/d12/d19/d1a/d35/d47/l68 51735 1 2024-02-15T18:27:06.179 INFO:tasks.workunit.client.0.smithi042.stdout:3/779: mknod d1/d11/d2e/d4c/cfd 0 2024-02-15T18:27:06.180 INFO:tasks.workunit.client.0.smithi042.stdout:1/810: mkdir de/d15/d67/d8e/d134 0 2024-02-15T18:27:06.180 INFO:tasks.workunit.client.0.smithi042.stdout:1/811: fdatasync de/d15/d2c/fde 0 2024-02-15T18:27:06.180 INFO:tasks.workunit.client.1.smithi191.stdout:6/350: fdatasync d8/d9/d2b/f2e 0 2024-02-15T18:27:06.183 INFO:tasks.workunit.client.1.smithi191.stdout:2/373: dread df/d12/d19/d1a/f44 [0,4194304] 0 2024-02-15T18:27:06.184 INFO:tasks.workunit.client.1.smithi191.stdout:7/418: link d0/db/d26/d5f/d77/d1d/f4c d0/db/d26/d5f/d77/d1d/d54/f95 0 2024-02-15T18:27:06.189 INFO:tasks.workunit.client.1.smithi191.stdout:5/192: rename d0/d14 to d0/d23/d41 0 2024-02-15T18:27:06.194 INFO:tasks.workunit.client.1.smithi191.stdout:1/505: truncate d0/d8/df/d12/d2a/f52 770911 0 2024-02-15T18:27:06.194 INFO:tasks.workunit.client.0.smithi042.stdout:1/812: dread de/d15/d67/d74/f90 [0,4194304] 0 2024-02-15T18:27:06.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:27:04.682736+0000 mgr.smithi042.kfoihb (mgr.24435) 41 : cluster [DBG] pgmap v31: 65 pgs: 65 active+clean; 2.1 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 118 MiB/s rd, 261 MiB/s wr, 382 op/s 2024-02-15T18:27:06.196 INFO:tasks.workunit.client.1.smithi191.stdout:7/419: dread d0/db/d26/d5f/d77/df/f64 [0,4194304] 0 2024-02-15T18:27:06.198 INFO:tasks.workunit.client.1.smithi191.stdout:3/330: dwrite f4 [0,4194304] 0 2024-02-15T18:27:06.198 INFO:tasks.workunit.client.1.smithi191.stdout:3/331: write d6/d3b/d42/f4a [9325354,15852] 0 2024-02-15T18:27:06.199 INFO:tasks.workunit.client.0.smithi042.stdout:4/802: dwrite da/d11/d29/f32 [0,4194304] 0 2024-02-15T18:27:06.199 INFO:tasks.workunit.client.0.smithi042.stdout:6/879: creat d9/d11/d19/d1c/d31/d9b/db4/f138 x:0 0 0 2024-02-15T18:27:06.200 INFO:tasks.workunit.client.0.smithi042.stdout:6/880: write d9/d11/d19/d1c/d31/dd2/df9/f118 [1044933,83503] 0 2024-02-15T18:27:06.200 INFO:tasks.workunit.client.1.smithi191.stdout:2/374: symlink df/d12/d19/d1a/d35/l81 0 2024-02-15T18:27:06.200 INFO:tasks.workunit.client.1.smithi191.stdout:2/375: write df/d12/d1e/f41 [378887,99079] 0 2024-02-15T18:27:06.201 INFO:tasks.workunit.client.1.smithi191.stdout:5/193: creat d0/d18/f42 x:0 0 0 2024-02-15T18:27:06.201 INFO:tasks.workunit.client.1.smithi191.stdout:5/194: fsync d0/f1b 0 2024-02-15T18:27:06.201 INFO:tasks.workunit.client.1.smithi191.stdout:5/195: chown d0/d23/d41/d28/f3f 1722 1 2024-02-15T18:27:06.204 INFO:tasks.workunit.client.1.smithi191.stdout:4/430: rename d2/d5/l45 to d2/d5/d14/d44/la6 0 2024-02-15T18:27:06.205 INFO:tasks.workunit.client.0.smithi042.stdout:5/833: mkdir d0/dda/dc7/d52/da0/d125 0 2024-02-15T18:27:06.206 INFO:tasks.workunit.client.1.smithi191.stdout:1/506: link d0/d8/df/d12/d33/ca5 d0/d8/df/d12/d33/d49/cae 0 2024-02-15T18:27:06.208 INFO:tasks.workunit.client.0.smithi042.stdout:9/798: creat d6/dd/d9/f114 x:0 0 0 2024-02-15T18:27:06.212 INFO:tasks.workunit.client.0.smithi042.stdout:2/766: creat d0/d8/d20/ff4 x:0 0 0 2024-02-15T18:27:06.212 INFO:tasks.workunit.client.0.smithi042.stdout:2/767: readlink d0/d8/d20/d7d/d94/l86 0 2024-02-15T18:27:06.213 INFO:tasks.workunit.client.0.smithi042.stdout:2/768: chown d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d2e 29 1 2024-02-15T18:27:06.213 INFO:tasks.workunit.client.0.smithi042.stdout:2/769: write d0/d8/d10/d7b/faa [4970823,90015] 0 2024-02-15T18:27:06.213 INFO:tasks.workunit.client.0.smithi042.stdout:3/780: rmdir d1/d11/d14/d4d/d73/df2 0 2024-02-15T18:27:06.214 INFO:tasks.workunit.client.0.smithi042.stdout:1/813: mknod de/d15/d67/d89/c135 0 2024-02-15T18:27:06.214 INFO:tasks.workunit.client.1.smithi191.stdout:5/196: creat d0/d23/f43 x:0 0 0 2024-02-15T18:27:06.215 INFO:tasks.workunit.client.1.smithi191.stdout:5/197: getdents d0/d23/d41/d3c 0 2024-02-15T18:27:06.217 INFO:tasks.workunit.client.1.smithi191.stdout:6/351: rename d8/f54 to d8/d9/d2b/d57/f63 0 2024-02-15T18:27:06.219 INFO:tasks.workunit.client.0.smithi042.stdout:0/761: rename d4/d9/d5a/d60 to d4/d9/d34/dab/dc8/dfe 0 2024-02-15T18:27:06.221 INFO:tasks.workunit.client.0.smithi042.stdout:1/814: dread de/d15/ff6 [0,4194304] 0 2024-02-15T18:27:06.221 INFO:tasks.workunit.client.0.smithi042.stdout:5/834: creat d0/dda/d42/d7d/db5/f126 x:0 0 0 2024-02-15T18:27:06.221 INFO:tasks.workunit.client.0.smithi042.stdout:5/835: dread - d0/d38/d53/db4/fe9 zero size 2024-02-15T18:27:06.222 INFO:tasks.workunit.client.1.smithi191.stdout:7/420: rename d0/db/d26/d5f/d77/c2 to d0/db/d26/d5f/d77/d1d/c96 0 2024-02-15T18:27:06.223 INFO:tasks.workunit.client.0.smithi042.stdout:9/799: link f3 d6/da/d2f/d55/f115 0 2024-02-15T18:27:06.225 INFO:tasks.workunit.client.1.smithi191.stdout:6/352: rmdir d8/d19 39 2024-02-15T18:27:06.228 INFO:tasks.workunit.client.1.smithi191.stdout:3/332: rename d6/d3b/d42/l67 to d6/d5d/d72/l76 0 2024-02-15T18:27:06.228 INFO:tasks.workunit.client.1.smithi191.stdout:3/333: chown f4 273517 1 2024-02-15T18:27:06.228 INFO:tasks.workunit.client.1.smithi191.stdout:3/334: write d6/d3b/d30/f50 [193468,39330] 0 2024-02-15T18:27:06.229 INFO:tasks.workunit.client.0.smithi042.stdout:2/770: truncate d0/d8/d10/d31/d79/f8e 89888 0 2024-02-15T18:27:06.231 INFO:tasks.workunit.client.1.smithi191.stdout:7/421: creat d0/db/d26/d5f/d77/f97 x:0 0 0 2024-02-15T18:27:06.232 INFO:tasks.workunit.client.0.smithi042.stdout:4/803: dwrite da/d11/d85/db5/d103/d66/fb0 [0,4194304] 0 2024-02-15T18:27:06.232 INFO:tasks.workunit.client.0.smithi042.stdout:0/762: dread d4/d9/f31 [0,4194304] 0 2024-02-15T18:27:06.232 INFO:tasks.workunit.client.0.smithi042.stdout:7/796: sync 2024-02-15T18:27:06.232 INFO:tasks.workunit.client.0.smithi042.stdout:0/763: fsync d4/d38/d77/fe6 0 2024-02-15T18:27:06.233 INFO:tasks.workunit.client.0.smithi042.stdout:7/797: truncate d10/f21 5163073 0 2024-02-15T18:27:06.233 INFO:tasks.workunit.client.1.smithi191.stdout:2/376: getdents df/d12/d19/d1a/d35 0 2024-02-15T18:27:06.233 INFO:tasks.workunit.client.1.smithi191.stdout:1/507: dwrite d0/f1b [0,4194304] 0 2024-02-15T18:27:06.233 INFO:tasks.workunit.client.1.smithi191.stdout:1/508: readlink d0/d2f/l6b 0 2024-02-15T18:27:06.236 INFO:tasks.workunit.client.0.smithi042.stdout:3/781: creat d1/d11/d2e/ffe x:0 0 0 2024-02-15T18:27:06.237 INFO:tasks.workunit.client.0.smithi042.stdout:6/881: dwrite d9/d11/d19/d1c/d31/f5b [4194304,4194304] 0 2024-02-15T18:27:06.239 INFO:tasks.workunit.client.0.smithi042.stdout:8/718: sync 2024-02-15T18:27:06.241 INFO:tasks.workunit.client.0.smithi042.stdout:1/815: dwrite de/d6d/d9e/dcd/f127 [0,4194304] 0 2024-02-15T18:27:06.242 INFO:tasks.workunit.client.0.smithi042.stdout:5/836: symlink d0/d50/l127 0 2024-02-15T18:27:06.243 INFO:tasks.workunit.client.0.smithi042.stdout:8/719: write dd/de/da4/d81/dae/fb6 [863624,70203] 0 2024-02-15T18:27:06.243 INFO:tasks.workunit.client.1.smithi191.stdout:5/198: rmdir d0/d23 39 2024-02-15T18:27:06.243 INFO:tasks.workunit.client.1.smithi191.stdout:5/199: truncate d0/d23/d41/d28/d31/f33 358663 0 2024-02-15T18:27:06.243 INFO:tasks.workunit.client.1.smithi191.stdout:5/200: write d0/d23/f43 [184044,128947] 0 2024-02-15T18:27:06.244 INFO:tasks.workunit.client.1.smithi191.stdout:5/201: getdents d0/d23/d41/d3c 0 2024-02-15T18:27:06.246 INFO:tasks.workunit.client.1.smithi191.stdout:4/431: rename d2/d5/d7/d2f/d41/l55 to d2/d5/d7/d90/la7 0 2024-02-15T18:27:06.251 INFO:tasks.workunit.client.0.smithi042.stdout:9/800: mknod d6/da/d1d/d2d/d40/c116 0 2024-02-15T18:27:06.251 INFO:tasks.workunit.client.0.smithi042.stdout:9/801: write d6/dd/f2b [5049843,119445] 0 2024-02-15T18:27:06.251 INFO:tasks.workunit.client.0.smithi042.stdout:9/802: chown d6/da/d2f/d54/lc2 7317919 1 2024-02-15T18:27:06.252 INFO:tasks.workunit.client.0.smithi042.stdout:9/803: dread - d6/dd/d9/f114 zero size 2024-02-15T18:27:06.252 INFO:tasks.workunit.client.0.smithi042.stdout:9/804: stat d6/da/d2f/d55/d73/d8d/dcf/de1/ffa 0 2024-02-15T18:27:06.252 INFO:tasks.workunit.client.0.smithi042.stdout:9/805: dread - d6/da/d1d/d2d/d40/d95/fdf zero size 2024-02-15T18:27:06.253 INFO:tasks.workunit.client.1.smithi191.stdout:3/335: rename d6/d3b/d42/d49/l2d to d6/d3b/d42/d49/l77 0 2024-02-15T18:27:06.253 INFO:tasks.workunit.client.1.smithi191.stdout:3/336: chown d6/c1e 2 1 2024-02-15T18:27:06.254 INFO:tasks.workunit.client.0.smithi042.stdout:2/771: creat d0/d8/d20/d7d/d7e/dc7/ff5 x:0 0 0 2024-02-15T18:27:06.256 INFO:tasks.workunit.client.1.smithi191.stdout:2/377: creat df/d12/d19/d1a/f82 x:0 0 0 2024-02-15T18:27:06.256 INFO:tasks.workunit.client.1.smithi191.stdout:7/422: dwrite d0/db/d65/dc/d4b/f7f [0,4194304] 0 2024-02-15T18:27:06.256 INFO:tasks.workunit.client.1.smithi191.stdout:2/378: truncate df/d12/f22 5136921 0 2024-02-15T18:27:06.256 INFO:tasks.workunit.client.1.smithi191.stdout:7/423: rename d0/db to d0/db/d65/d11/d1c/d98 22 2024-02-15T18:27:06.256 INFO:tasks.workunit.client.1.smithi191.stdout:2/379: write df/d61/f67 [2408751,22365] 0 2024-02-15T18:27:06.257 INFO:tasks.workunit.client.1.smithi191.stdout:6/353: dwrite d8/d9/d2b/d57/f63 [8388608,4194304] 0 2024-02-15T18:27:06.259 INFO:tasks.workunit.client.0.smithi042.stdout:4/804: creat da/d11/da4/f116 x:0 0 0 2024-02-15T18:27:06.259 INFO:tasks.workunit.client.0.smithi042.stdout:6/882: dwrite d9/d3e/f96 [0,4194304] 0 2024-02-15T18:27:06.261 INFO:tasks.workunit.client.1.smithi191.stdout:1/509: unlink d0/d2f/d51/d54/f55 0 2024-02-15T18:27:06.261 INFO:tasks.workunit.client.1.smithi191.stdout:1/510: stat d0/d2f/d51/d10/f2d 0 2024-02-15T18:27:06.263 INFO:tasks.workunit.client.0.smithi042.stdout:0/764: rmdir d4/d38/d39/d8e/d8f 39 2024-02-15T18:27:06.263 INFO:tasks.workunit.client.1.smithi191.stdout:1/511: write d0/d8/df/d12/d2a/f42 [306865,114228] 0 2024-02-15T18:27:06.264 INFO:tasks.workunit.client.1.smithi191.stdout:5/202: symlink d0/d23/d41/d28/d31/d38/l44 0 2024-02-15T18:27:06.264 INFO:tasks.workunit.client.1.smithi191.stdout:5/203: fdatasync d0/d23/d41/d28/d31/f33 0 2024-02-15T18:27:06.265 INFO:tasks.workunit.client.0.smithi042.stdout:7/798: mkdir d10/d15/d108 0 2024-02-15T18:27:06.266 INFO:tasks.workunit.client.1.smithi191.stdout:4/432: creat d2/d5/d14/fa8 x:0 0 0 2024-02-15T18:27:06.268 INFO:tasks.workunit.client.0.smithi042.stdout:9/806: dread d6/da/d1d/d2d/d40/f8c [0,4194304] 0 2024-02-15T18:27:06.268 INFO:tasks.workunit.client.0.smithi042.stdout:9/807: truncate d6/da/d75/fe7 897754 0 2024-02-15T18:27:06.271 INFO:tasks.workunit.client.0.smithi042.stdout:3/782: rename d1/de/c99 to d1/d9c/cff 0 2024-02-15T18:27:06.271 INFO:tasks.workunit.client.0.smithi042.stdout:1/816: mkdir de/d15/d2c/d136 0 2024-02-15T18:27:06.273 INFO:tasks.workunit.client.0.smithi042.stdout:5/837: mkdir d0/d128 0 2024-02-15T18:27:06.274 INFO:tasks.workunit.client.0.smithi042.stdout:5/838: write d0/dda/dc7/d52/dad/ff0 [5010993,99352] 0 2024-02-15T18:27:06.274 INFO:tasks.workunit.client.1.smithi191.stdout:4/433: dread d2/d5/d7/f29 [0,4194304] 0 2024-02-15T18:27:06.274 INFO:tasks.workunit.client.1.smithi191.stdout:7/424: creat d0/f99 x:0 0 0 2024-02-15T18:27:06.277 INFO:tasks.workunit.client.1.smithi191.stdout:1/512: creat d0/d2f/d77/faf x:0 0 0 2024-02-15T18:27:06.278 INFO:tasks.workunit.client.1.smithi191.stdout:7/425: symlink d0/db/d26/d5f/d77/d1d/d54/l9a 0 2024-02-15T18:27:06.280 INFO:tasks.workunit.client.0.smithi042.stdout:2/772: mkdir d0/d8/d20/d3f/dd6/df3/df6 0 2024-02-15T18:27:06.280 INFO:tasks.workunit.client.0.smithi042.stdout:2/773: chown d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d2e/laf 139245 1 2024-02-15T18:27:06.282 INFO:tasks.workunit.client.0.smithi042.stdout:4/805: symlink da/d11/da4/l117 0 2024-02-15T18:27:06.283 INFO:tasks.workunit.client.0.smithi042.stdout:4/806: read da/d11/da4/fb4 [564654,60991] 0 2024-02-15T18:27:06.285 INFO:tasks.workunit.client.1.smithi191.stdout:2/380: dwrite df/d12/d19/d1a/d35/f70 [0,4194304] 0 2024-02-15T18:27:06.285 INFO:tasks.workunit.client.1.smithi191.stdout:5/204: dwrite d0/d18/f42 [0,4194304] 0 2024-02-15T18:27:06.285 INFO:tasks.workunit.client.0.smithi042.stdout:6/883: unlink d9/f67 0 2024-02-15T18:27:06.288 INFO:tasks.workunit.client.0.smithi042.stdout:0/765: rmdir d4/d38/d77/d2e/dda 39 2024-02-15T18:27:06.288 INFO:tasks.workunit.client.0.smithi042.stdout:0/766: fdatasync d4/d38/d77/d2e/dae/ddf/fb6 0 2024-02-15T18:27:06.289 INFO:tasks.workunit.client.1.smithi191.stdout:6/354: dwrite d8/d19/f40 [0,4194304] 0 2024-02-15T18:27:06.289 INFO:tasks.workunit.client.1.smithi191.stdout:6/355: write d8/d9/d2b/d57/f63 [12865583,111965] 0 2024-02-15T18:27:06.289 INFO:tasks.workunit.client.1.smithi191.stdout:6/356: fdatasync d8/f15 0 2024-02-15T18:27:06.289 INFO:tasks.workunit.client.1.smithi191.stdout:2/381: mknod df/d12/d19/d1a/d35/d47/c83 0 2024-02-15T18:27:06.289 INFO:tasks.workunit.client.1.smithi191.stdout:2/382: readlink df/d12/d1e/l71 0 2024-02-15T18:27:06.292 INFO:tasks.workunit.client.0.smithi042.stdout:7/799: rename d10/d15/d8f/da5/dad/d104/d65/ca2 to d10/d49/d6d/d78/c109 0 2024-02-15T18:27:06.294 INFO:tasks.workunit.client.0.smithi042.stdout:3/783: mknod d1/d11/d14/d4d/d73/d89/dd9/c100 0 2024-02-15T18:27:06.295 INFO:tasks.workunit.client.1.smithi191.stdout:6/357: dread d8/d19/f3d [0,4194304] 0 2024-02-15T18:27:06.296 INFO:tasks.workunit.client.0.smithi042.stdout:2/774: creat d0/d8/d10/d31/d3a/ff7 x:0 0 0 2024-02-15T18:27:06.297 INFO:tasks.workunit.client.0.smithi042.stdout:3/784: write d1/d11/d14/f53 [810881,116960] 0 2024-02-15T18:27:06.297 INFO:tasks.workunit.client.0.smithi042.stdout:3/785: write d1/d11/d14/d4d/d6c/d7b/fbb [881465,35998] 0 2024-02-15T18:27:06.298 INFO:tasks.workunit.client.0.smithi042.stdout:4/807: mknod da/d11/d85/db5/c118 0 2024-02-15T18:27:06.299 INFO:tasks.workunit.client.1.smithi191.stdout:2/383: mkdir df/d61/d84 0 2024-02-15T18:27:06.300 INFO:tasks.workunit.client.0.smithi042.stdout:6/884: creat d9/d11/d19/d1c/f139 x:0 0 0 2024-02-15T18:27:06.300 INFO:tasks.workunit.client.1.smithi191.stdout:2/384: write df/d12/d1f/d42/f49 [1768523,116205] 0 2024-02-15T18:27:06.301 INFO:tasks.workunit.client.0.smithi042.stdout:0/767: getdents d4/d9/d62/d74 0 2024-02-15T18:27:06.301 INFO:tasks.workunit.client.0.smithi042.stdout:0/768: write d4/d38/d77/fb4 [498569,17522] 0 2024-02-15T18:27:06.302 INFO:tasks.workunit.client.1.smithi191.stdout:1/513: dwrite d0/d8/f57 [0,4194304] 0 2024-02-15T18:27:06.303 INFO:tasks.workunit.client.0.smithi042.stdout:1/817: rename de/d54/dbf/dd0/d7c/dd6/ddf/d88/l8c to de/d15/d1b/d44/df3/l137 0 2024-02-15T18:27:06.303 INFO:tasks.workunit.client.0.smithi042.stdout:1/818: fdatasync f6 0 2024-02-15T18:27:06.304 INFO:tasks.workunit.client.1.smithi191.stdout:1/514: write d0/f5 [3005464,57900] 0 2024-02-15T18:27:06.304 INFO:tasks.workunit.client.0.smithi042.stdout:7/800: mknod d10/d15/d8f/da5/dad/de0/c10a 0 2024-02-15T18:27:06.304 INFO:tasks.workunit.client.0.smithi042.stdout:7/801: chown d10/d15/d8f/da5/dad/d104/d23/d51/leb 1185 1 2024-02-15T18:27:06.307 INFO:tasks.workunit.client.0.smithi042.stdout:3/786: rename d1/d11/d2e/d4c/l93 to d1/d11/d2e/d4c/db0/l101 0 2024-02-15T18:27:06.307 INFO:tasks.workunit.client.0.smithi042.stdout:3/787: chown d1/de/fe2 3887 1 2024-02-15T18:27:06.307 INFO:tasks.workunit.client.0.smithi042.stdout:6/885: dread d9/d11/d19/d1c/d31/f3a [0,4194304] 0 2024-02-15T18:27:06.307 INFO:tasks.workunit.client.0.smithi042.stdout:1/819: creat de/d15/d67/d89/f138 x:0 0 0 2024-02-15T18:27:06.307 INFO:tasks.workunit.client.0.smithi042.stdout:1/820: dread - de/d54/dbf/dd0/f102 zero size 2024-02-15T18:27:06.307 INFO:tasks.workunit.client.0.smithi042.stdout:3/788: write d1/d11/d14/d4d/d6c/d7b/f98 [775206,74084] 0 2024-02-15T18:27:06.308 INFO:tasks.workunit.client.0.smithi042.stdout:1/821: rename de/d15/d67/d74/f107 to de/d15/d2c/d51/d70/f139 0 2024-02-15T18:27:06.310 INFO:tasks.workunit.client.0.smithi042.stdout:3/789: rename d1/d11/d14/d4d/d6c/d7b/f78 to d1/d11/d14/d4d/d6c/df4/f102 0 2024-02-15T18:27:06.311 INFO:tasks.workunit.client.0.smithi042.stdout:3/790: creat d1/d11/f103 x:0 0 0 2024-02-15T18:27:06.312 INFO:tasks.workunit.client.0.smithi042.stdout:0/769: dread d4/d38/d5c/f61 [0,4194304] 0 2024-02-15T18:27:06.313 INFO:tasks.workunit.client.0.smithi042.stdout:0/770: fdatasync d4/f6 0 2024-02-15T18:27:06.313 INFO:tasks.workunit.client.0.smithi042.stdout:1/822: rename de/d54/dbf/dd0/cd1 to de/d54/d9a/dc8/dd3/c13a 0 2024-02-15T18:27:06.313 INFO:tasks.workunit.client.0.smithi042.stdout:1/823: fsync de/d15/d67/d46/f59 0 2024-02-15T18:27:06.315 INFO:tasks.workunit.client.0.smithi042.stdout:0/771: truncate d4/d9/d5a/dfc/f83 446565 0 2024-02-15T18:27:06.315 INFO:tasks.workunit.client.0.smithi042.stdout:0/772: write d4/d9/d34/d57/d88/fcf [176311,42812] 0 2024-02-15T18:27:06.316 INFO:tasks.workunit.client.1.smithi191.stdout:5/205: dread d0/d23/d41/d28/f36 [0,4194304] 0 2024-02-15T18:27:06.316 INFO:tasks.workunit.client.1.smithi191.stdout:7/426: dwrite d0/db/d65/d11/d1c/f6f [0,4194304] 0 2024-02-15T18:27:06.316 INFO:tasks.workunit.client.1.smithi191.stdout:5/206: fsync d0/f1b 0 2024-02-15T18:27:06.316 INFO:tasks.workunit.client.0.smithi042.stdout:0/773: dread - d4/d38/d77/d2e/dae/ddf/fe8 zero size 2024-02-15T18:27:06.318 INFO:tasks.workunit.client.1.smithi191.stdout:7/427: getdents d0/db/d65/dc 0 2024-02-15T18:27:06.319 INFO:tasks.workunit.client.1.smithi191.stdout:1/515: dread d0/d2f/d51/d10/f25 [0,4194304] 0 2024-02-15T18:27:06.319 INFO:tasks.workunit.client.1.smithi191.stdout:1/516: fsync d0/fa6 0 2024-02-15T18:27:06.320 INFO:tasks.workunit.client.1.smithi191.stdout:5/207: link d0/d23/d41/d28/f3f d0/d18/f45 0 2024-02-15T18:27:06.320 INFO:tasks.workunit.client.1.smithi191.stdout:5/208: chown d0/d18/f45 108739338 1 2024-02-15T18:27:06.320 INFO:tasks.workunit.client.1.smithi191.stdout:5/209: write d0/d18/f2e [250244,37230] 0 2024-02-15T18:27:06.321 INFO:tasks.workunit.client.0.smithi042.stdout:0/774: mknod d4/d9/d34/d8b/cff 0 2024-02-15T18:27:06.321 INFO:tasks.workunit.client.1.smithi191.stdout:7/428: creat d0/db/d65/d11/d5a/d66/d1e/f9b x:0 0 0 2024-02-15T18:27:06.321 INFO:tasks.workunit.client.1.smithi191.stdout:1/517: creat d0/d8/df/d12/d2a/fb0 x:0 0 0 2024-02-15T18:27:06.321 INFO:tasks.workunit.client.1.smithi191.stdout:1/518: chown d0/d2f/d51/d10/c13 1536013 1 2024-02-15T18:27:06.323 INFO:tasks.workunit.client.0.smithi042.stdout:0/775: rename d4/d9/f31 to d4/d9/d34/dab/f100 0 2024-02-15T18:27:06.324 INFO:tasks.workunit.client.0.smithi042.stdout:0/776: write d4/d9/d34/d57/f7f [960204,63368] 0 2024-02-15T18:27:06.324 INFO:tasks.workunit.client.1.smithi191.stdout:5/210: creat d0/f46 x:0 0 0 2024-02-15T18:27:06.325 INFO:tasks.workunit.client.1.smithi191.stdout:7/429: mknod d0/db/d26/d5f/d8c/c9c 0 2024-02-15T18:27:06.325 INFO:tasks.workunit.client.1.smithi191.stdout:7/430: fdatasync d0/db/d65/d11/f18 0 2024-02-15T18:27:06.325 INFO:tasks.workunit.client.0.smithi042.stdout:0/777: truncate d4/f68 761642 0 2024-02-15T18:27:06.326 INFO:tasks.workunit.client.1.smithi191.stdout:5/211: truncate d0/d18/f45 5148223 0 2024-02-15T18:27:06.327 INFO:tasks.workunit.client.1.smithi191.stdout:7/431: mkdir d0/db/d65/d11/d88/d9d 0 2024-02-15T18:27:06.327 INFO:tasks.workunit.client.0.smithi042.stdout:7/802: dread d10/d15/f3d [0,4194304] 0 2024-02-15T18:27:06.328 INFO:tasks.workunit.client.0.smithi042.stdout:0/778: unlink d4/d38/d77/ca7 0 2024-02-15T18:27:06.329 INFO:tasks.workunit.client.1.smithi191.stdout:5/212: creat d0/d23/d41/d28/d31/f47 x:0 0 0 2024-02-15T18:27:06.330 INFO:tasks.workunit.client.0.smithi042.stdout:1/824: dwrite de/d54/dbf/fe1 [0,4194304] 0 2024-02-15T18:27:06.331 INFO:tasks.workunit.client.0.smithi042.stdout:0/779: mkdir d4/d38/d77/d2e/dda/d101 0 2024-02-15T18:27:06.332 INFO:tasks.workunit.client.1.smithi191.stdout:6/358: dwrite d8/d9/f29 [4194304,4194304] 0 2024-02-15T18:27:06.332 INFO:tasks.workunit.client.1.smithi191.stdout:6/359: chown d8/d9/d4a/d4c 838614 1 2024-02-15T18:27:06.332 INFO:tasks.workunit.client.1.smithi191.stdout:6/360: write d8/d9/d2b/f4e [291117,104404] 0 2024-02-15T18:27:06.332 INFO:tasks.workunit.client.1.smithi191.stdout:6/361: chown d8/d9/d4a/d4c 460326176 1 2024-02-15T18:27:06.333 INFO:tasks.workunit.client.1.smithi191.stdout:7/432: link d0/db/d65/d11/f90 d0/f9e 0 2024-02-15T18:27:06.333 INFO:tasks.workunit.client.0.smithi042.stdout:7/803: dread d10/d49/f67 [0,4194304] 0 2024-02-15T18:27:06.333 INFO:tasks.workunit.client.0.smithi042.stdout:7/804: stat d10/d15/l25 0 2024-02-15T18:27:06.334 INFO:tasks.workunit.client.0.smithi042.stdout:7/805: read - d10/d43/d57/f101 zero size 2024-02-15T18:27:06.334 INFO:tasks.workunit.client.0.smithi042.stdout:7/806: stat d10/d15/d39/d6c/d92/cd1 0 2024-02-15T18:27:06.334 INFO:tasks.workunit.client.1.smithi191.stdout:5/213: mkdir d0/d23/d48 0 2024-02-15T18:27:06.334 INFO:tasks.workunit.client.0.smithi042.stdout:7/807: read d10/d15/f34 [4152553,99824] 0 2024-02-15T18:27:06.334 INFO:tasks.workunit.client.0.smithi042.stdout:1/825: rmdir de/d54/dbf/dd0/d7c/dd6/de8/dec 39 2024-02-15T18:27:06.335 INFO:tasks.workunit.client.0.smithi042.stdout:7/808: read d10/d49/d6d/f85 [292154,78885] 0 2024-02-15T18:27:06.336 INFO:tasks.workunit.client.0.smithi042.stdout:1/826: symlink de/d15/d2c/d51/d70/l13b 0 2024-02-15T18:27:06.336 INFO:tasks.workunit.client.1.smithi191.stdout:6/362: creat d8/d9/d4a/d4c/f64 x:0 0 0 2024-02-15T18:27:06.337 INFO:tasks.workunit.client.0.smithi042.stdout:7/809: rename d10/d15/d39/c107 to d10/d49/d6d/d6e/c10b 0 2024-02-15T18:27:06.338 INFO:tasks.workunit.client.1.smithi191.stdout:7/433: mknod d0/db/d65/d11/d88/c9f 0 2024-02-15T18:27:06.339 INFO:tasks.workunit.client.0.smithi042.stdout:1/827: creat de/d6d/d9e/df4/f13c x:0 0 0 2024-02-15T18:27:06.339 INFO:tasks.workunit.client.0.smithi042.stdout:1/828: chown de/l6e 83 1 2024-02-15T18:27:06.339 INFO:tasks.workunit.client.1.smithi191.stdout:5/214: mkdir d0/d23/d48/d49 0 2024-02-15T18:27:06.341 INFO:tasks.workunit.client.0.smithi042.stdout:7/810: truncate d10/d15/d39/d6c/f6f 63663 0 2024-02-15T18:27:06.341 INFO:tasks.workunit.client.0.smithi042.stdout:7/811: truncate d10/d15/d8f/da5/dad/de0/de2/ffd 274403 0 2024-02-15T18:27:06.342 INFO:tasks.workunit.client.1.smithi191.stdout:6/363: creat d8/d9/f65 x:0 0 0 2024-02-15T18:27:06.343 INFO:tasks.workunit.client.1.smithi191.stdout:6/364: creat d8/d9/d4a/d4c/f66 x:0 0 0 2024-02-15T18:27:06.343 INFO:tasks.workunit.client.0.smithi042.stdout:7/812: write d10/d49/d6d/d78/d83/fe8 [3688375,27161] 0 2024-02-15T18:27:06.344 INFO:tasks.workunit.client.1.smithi191.stdout:6/365: truncate d8/d9/f36 4460250 0 2024-02-15T18:27:06.345 INFO:tasks.workunit.client.0.smithi042.stdout:7/813: symlink d10/d15/d39/dc2/l10c 0 2024-02-15T18:27:06.345 INFO:tasks.workunit.client.0.smithi042.stdout:7/814: dread - d10/d15/fa6 zero size 2024-02-15T18:27:06.345 INFO:tasks.workunit.client.0.smithi042.stdout:7/815: write d10/d43/d57/fb8 [1024307,114547] 0 2024-02-15T18:27:06.345 INFO:tasks.workunit.client.0.smithi042.stdout:7/816: dread - d10/d49/d6d/db6/ff0 zero size 2024-02-15T18:27:06.346 INFO:tasks.workunit.client.1.smithi191.stdout:6/366: getdents d8/d19 0 2024-02-15T18:27:06.346 INFO:tasks.workunit.client.1.smithi191.stdout:6/367: unlink f6 0 2024-02-15T18:27:06.348 INFO:tasks.workunit.client.0.smithi042.stdout:7/817: mknod d10/d49/d6d/d78/c10d 0 2024-02-15T18:27:06.360 INFO:tasks.workunit.client.1.smithi191.stdout:6/368: dwrite d8/d19/f26 [0,4194304] 0 2024-02-15T18:27:06.361 INFO:tasks.workunit.client.1.smithi191.stdout:6/369: mknod d8/d9/d4a/d4c/c67 0 2024-02-15T18:27:06.362 INFO:tasks.workunit.client.1.smithi191.stdout:6/370: symlink d8/d9/l68 0 2024-02-15T18:27:06.363 INFO:tasks.workunit.client.1.smithi191.stdout:6/371: mkdir d8/d9/d4a/d4c/d69 0 2024-02-15T18:27:06.373 INFO:tasks.workunit.client.0.smithi042.stdout:9/808: write d6/da/d1d/f24 [3788063,84902] 0 2024-02-15T18:27:06.376 INFO:tasks.workunit.client.0.smithi042.stdout:9/809: truncate d6/da/d2f/f56 136153 0 2024-02-15T18:27:06.376 INFO:tasks.workunit.client.0.smithi042.stdout:9/810: write d6/da/d1d/d2d/fb4 [664285,120902] 0 2024-02-15T18:27:06.381 INFO:tasks.workunit.client.0.smithi042.stdout:2/775: dread d0/d8/f70 [0,4194304] 0 2024-02-15T18:27:06.381 INFO:tasks.workunit.client.0.smithi042.stdout:2/776: fsync d0/d8/d20/d7d/d7e/dc7/f8d 0 2024-02-15T18:27:06.381 INFO:tasks.workunit.client.0.smithi042.stdout:2/777: write d0/d8/d20/d7d/d7e/dc7/d8b/dd7/fe2 [181341,50766] 0 2024-02-15T18:27:06.382 INFO:tasks.workunit.client.0.smithi042.stdout:9/811: mkdir d6/da/d2f/d6c/dd8/d117 0 2024-02-15T18:27:06.395 INFO:tasks.workunit.client.0.smithi042.stdout:4/808: dread da/d3a/dbb/fc3 [0,4194304] 0 2024-02-15T18:27:06.396 INFO:tasks.workunit.client.0.smithi042.stdout:4/809: chown da/d3b/d41/d8d/f93 557494553 1 2024-02-15T18:27:06.414 INFO:tasks.workunit.client.0.smithi042.stdout:5/839: dwrite d0/d22/d7b/f66 [0,4194304] 0 2024-02-15T18:27:06.414 INFO:tasks.workunit.client.0.smithi042.stdout:6/886: dwrite d9/d11/ff1 [0,4194304] 0 2024-02-15T18:27:06.415 INFO:tasks.workunit.client.0.smithi042.stdout:0/780: dwrite d4/d9/d5a/dfc/f8c [0,4194304] 0 2024-02-15T18:27:06.422 INFO:tasks.workunit.client.0.smithi042.stdout:2/778: dwrite d0/d8/d10/fd9 [0,4194304] 0 2024-02-15T18:27:06.422 INFO:tasks.workunit.client.0.smithi042.stdout:2/779: write d0/d8/d10/d31/d3a/ff7 [666051,69945] 0 2024-02-15T18:27:06.423 INFO:tasks.workunit.client.1.smithi191.stdout:2/385: dwrite df/d12/d19/d1a/d35/d5b/f6e [0,4194304] 0 2024-02-15T18:27:06.423 INFO:tasks.workunit.client.1.smithi191.stdout:2/386: dread - df/d12/d19/d1a/d35/f80 zero size 2024-02-15T18:27:06.423 INFO:tasks.workunit.client.0.smithi042.stdout:3/791: dwrite d1/d11/d14/d4d/d73/ff5 [0,4194304] 0 2024-02-15T18:27:06.423 INFO:tasks.workunit.client.0.smithi042.stdout:3/792: chown d1/d11/d2e/d4c 113 1 2024-02-15T18:27:06.435 INFO:tasks.workunit.client.0.smithi042.stdout:5/840: dwrite d0/dda/dc7/de1/f35 [4194304,4194304] 0 2024-02-15T18:27:06.462 INFO:tasks.workunit.client.1.smithi191.stdout:3/337: getdents d6/d3b/d42/d49 0 2024-02-15T18:27:06.468 INFO:tasks.workunit.client.1.smithi191.stdout:7/434: truncate d0/db/d65/dc/d4b/f7f 1444479 0 2024-02-15T18:27:06.494 INFO:tasks.workunit.client.1.smithi191.stdout:5/215: getdents d0/d23/d48 0 2024-02-15T18:27:06.497 INFO:tasks.workunit.client.1.smithi191.stdout:5/216: dread d0/d18/f45 [4194304,4194304] 0 2024-02-15T18:27:06.509 INFO:tasks.workunit.client.0.smithi042.stdout:9/812: link d6/da/d75/lf8 d6/da/d2f/d55/d73/d8d/dcf/de1/l118 0 2024-02-15T18:27:06.509 INFO:tasks.workunit.client.0.smithi042.stdout:9/813: truncate d6/fa8 856580 0 2024-02-15T18:27:06.512 INFO:tasks.workunit.client.1.smithi191.stdout:2/387: creat df/d12/d19/d1a/d35/f85 x:0 0 0 2024-02-15T18:27:06.512 INFO:tasks.workunit.client.1.smithi191.stdout:2/388: fsync fb 0 2024-02-15T18:27:06.513 INFO:tasks.workunit.client.1.smithi191.stdout:2/389: write df/d12/d19/d1a/f5e [736389,12901] 0 2024-02-15T18:27:06.514 INFO:tasks.workunit.client.1.smithi191.stdout:5/217: dwrite d0/f1f [0,4194304] 0 2024-02-15T18:27:06.514 INFO:tasks.workunit.client.1.smithi191.stdout:5/218: chown d0/d23/d41/d28/d31/c35 239 1 2024-02-15T18:27:06.515 INFO:tasks.workunit.client.1.smithi191.stdout:3/338: creat d6/d27/f78 x:0 0 0 2024-02-15T18:27:06.522 INFO:tasks.workunit.client.0.smithi042.stdout:6/887: symlink d9/d11/d19/d1c/d11a/l13a 0 2024-02-15T18:27:06.524 INFO:tasks.workunit.client.1.smithi191.stdout:3/339: dread d6/d27/d4d/f58 [0,4194304] 0 2024-02-15T18:27:06.526 INFO:tasks.workunit.client.0.smithi042.stdout:0/781: rename d4/d38/d5c/cd9 to d4/d9/d34/dab/db0/c102 0 2024-02-15T18:27:06.529 INFO:tasks.workunit.client.0.smithi042.stdout:2/780: symlink d0/d35/dc6/lf8 0 2024-02-15T18:27:06.537 INFO:tasks.workunit.client.1.smithi191.stdout:2/390: rmdir df/d12/d19/d1a/d35/d5b 39 2024-02-15T18:27:06.537 INFO:tasks.workunit.client.1.smithi191.stdout:2/391: write df/f1b [3583657,37700] 0 2024-02-15T18:27:06.545 INFO:tasks.workunit.client.0.smithi042.stdout:3/793: link d1/de/d1a/l8d d1/d11/d14/d4d/d73/d89/da8/l104 0 2024-02-15T18:27:06.547 INFO:tasks.workunit.client.1.smithi191.stdout:3/340: rmdir d6/d3b 39 2024-02-15T18:27:06.547 INFO:tasks.workunit.client.1.smithi191.stdout:3/341: fsync d6/d1b/d60/d73/f75 0 2024-02-15T18:27:06.547 INFO:tasks.workunit.client.1.smithi191.stdout:3/342: write d6/d1b/d60/f68 [729198,98984] 0 2024-02-15T18:27:06.549 INFO:tasks.workunit.client.1.smithi191.stdout:2/392: getdents df/d12/d19/d1a/d35/d47 0 2024-02-15T18:27:06.550 INFO:tasks.workunit.client.0.smithi042.stdout:8/720: sync 2024-02-15T18:27:06.552 INFO:tasks.workunit.client.0.smithi042.stdout:8/721: dread dd/de/da4/d67/f2d [0,4194304] 0 2024-02-15T18:27:06.553 INFO:tasks.workunit.client.0.smithi042.stdout:5/841: symlink d0/dda/l129 0 2024-02-15T18:27:06.553 INFO:tasks.workunit.client.0.smithi042.stdout:5/842: readlink d0/d38/d53/d64/lf6 0 2024-02-15T18:27:06.553 INFO:tasks.workunit.client.1.smithi191.stdout:2/393: creat df/d12/d19/d1a/d35/d47/d4b/d7a/f86 x:0 0 0 2024-02-15T18:27:06.557 INFO:tasks.workunit.client.0.smithi042.stdout:9/814: mkdir d6/da/d2f/d55/d73/d119 0 2024-02-15T18:27:06.557 INFO:tasks.workunit.client.0.smithi042.stdout:9/815: dread - d6/da/d1d/dde/dec/fce zero size 2024-02-15T18:27:06.559 INFO:tasks.workunit.client.0.smithi042.stdout:6/888: mknod d9/d11/d19/d35/d55/c13b 0 2024-02-15T18:27:06.560 INFO:tasks.workunit.client.0.smithi042.stdout:2/781: mknod d0/d8/d20/d3f/dd6/cf9 0 2024-02-15T18:27:06.560 INFO:tasks.workunit.client.1.smithi191.stdout:2/394: rmdir df 39 2024-02-15T18:27:06.562 INFO:tasks.workunit.client.0.smithi042.stdout:3/794: rename d1/d11/d14/d4d/d6c/d7b/lcd to d1/d11/d2e/dd1/l105 0 2024-02-15T18:27:06.562 INFO:tasks.workunit.client.0.smithi042.stdout:3/795: readlink d1/d11/d14/d4d/d73/d89/da8/lbd 0 2024-02-15T18:27:06.562 INFO:tasks.workunit.client.0.smithi042.stdout:3/796: fdatasync d1/d11/f8f 0 2024-02-15T18:27:06.563 INFO:tasks.workunit.client.0.smithi042.stdout:9/816: rmdir d6/dd/d9/d59/dbf/d100 0 2024-02-15T18:27:06.564 INFO:tasks.workunit.client.0.smithi042.stdout:3/797: stat d1/d11/d2e/dd1/l105 0 2024-02-15T18:27:06.565 INFO:tasks.workunit.client.0.smithi042.stdout:9/817: symlink d6/da/d2f/d55/d73/d119/l11a 0 2024-02-15T18:27:06.565 INFO:tasks.workunit.client.0.smithi042.stdout:9/818: dread - d6/dd/d9/d59/fbe zero size 2024-02-15T18:27:06.566 INFO:tasks.workunit.client.0.smithi042.stdout:9/819: fsync d6/dd/d9/d59/d5c/fb7 0 2024-02-15T18:27:06.566 INFO:tasks.workunit.client.0.smithi042.stdout:9/820: write d6/dd/dcc/ffb [971360,57612] 0 2024-02-15T18:27:06.566 INFO:tasks.workunit.client.1.smithi191.stdout:2/395: symlink df/l87 0 2024-02-15T18:27:06.568 INFO:tasks.workunit.client.0.smithi042.stdout:6/889: write d9/f32 [3264385,80188] 0 2024-02-15T18:27:06.568 INFO:tasks.workunit.client.0.smithi042.stdout:2/782: dread d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/f6b [0,4194304] 0 2024-02-15T18:27:06.568 INFO:tasks.workunit.client.0.smithi042.stdout:3/798: creat d1/d11/d14/d4d/d73/d89/dd9/f106 x:0 0 0 2024-02-15T18:27:06.568 INFO:tasks.workunit.client.0.smithi042.stdout:3/799: dread - d1/de/fe2 zero size 2024-02-15T18:27:06.569 INFO:tasks.workunit.client.1.smithi191.stdout:2/396: mknod df/d12/d32/d4d/c88 0 2024-02-15T18:27:06.569 INFO:tasks.workunit.client.0.smithi042.stdout:3/800: creat d1/d11/d14/d4d/d73/f107 x:0 0 0 2024-02-15T18:27:06.570 INFO:tasks.workunit.client.1.smithi191.stdout:2/397: mkdir df/d12/d32/d4d/d89 0 2024-02-15T18:27:06.572 INFO:tasks.workunit.client.1.smithi191.stdout:3/343: dwrite d6/d1b/d60/d73/f61 [4194304,4194304] 0 2024-02-15T18:27:06.573 INFO:tasks.workunit.client.0.smithi042.stdout:2/783: read d0/d8/d10/d31/d3a/ff7 [48595,534] 0 2024-02-15T18:27:06.575 INFO:tasks.workunit.client.1.smithi191.stdout:3/344: mknod d6/d3b/d30/c79 0 2024-02-15T18:27:06.576 INFO:tasks.workunit.client.0.smithi042.stdout:5/843: dread d0/d14/f9c [0,4194304] 0 2024-02-15T18:27:06.577 INFO:tasks.workunit.client.0.smithi042.stdout:7/818: sync 2024-02-15T18:27:06.578 INFO:tasks.workunit.client.0.smithi042.stdout:4/810: sync 2024-02-15T18:27:06.578 INFO:tasks.workunit.client.0.smithi042.stdout:1/829: sync 2024-02-15T18:27:06.578 INFO:tasks.workunit.client.0.smithi042.stdout:7/819: chown d10/f14 1 1 2024-02-15T18:27:06.578 INFO:tasks.workunit.client.0.smithi042.stdout:2/784: creat d0/d8/d10/d31/d79/dbd/ffa x:0 0 0 2024-02-15T18:27:06.579 INFO:tasks.workunit.client.1.smithi191.stdout:3/345: creat d6/d3b/d30/f7a x:0 0 0 2024-02-15T18:27:06.579 INFO:tasks.workunit.client.0.smithi042.stdout:3/801: dread d1/d11/d2e/f54 [0,4194304] 0 2024-02-15T18:27:06.579 INFO:tasks.workunit.client.0.smithi042.stdout:8/722: dwrite dd/f17 [0,4194304] 0 2024-02-15T18:27:06.580 INFO:tasks.workunit.client.0.smithi042.stdout:3/802: readlink d1/d11/d14/d4d/l68 0 2024-02-15T18:27:06.580 INFO:tasks.workunit.client.0.smithi042.stdout:8/723: chown dd/de/d33/d3b/d92/fa8 21558 1 2024-02-15T18:27:06.581 INFO:tasks.workunit.client.0.smithi042.stdout:5/844: creat d0/dda/dc7/d52/dad/dba/dbd/f12a x:0 0 0 2024-02-15T18:27:06.581 INFO:tasks.workunit.client.0.smithi042.stdout:5/845: dread - d0/d38/d53/d68/fa8 zero size 2024-02-15T18:27:06.581 INFO:tasks.workunit.client.0.smithi042.stdout:5/846: fdatasync d0/d14/f9c 0 2024-02-15T18:27:06.581 INFO:tasks.workunit.client.0.smithi042.stdout:5/847: dread - d0/dda/d42/d7d/db5/f126 zero size 2024-02-15T18:27:06.585 INFO:tasks.workunit.client.0.smithi042.stdout:4/811: getdents da/d11/d67 0 2024-02-15T18:27:06.588 INFO:tasks.workunit.client.0.smithi042.stdout:1/830: link de/d54/dbf/dd0/ff9 de/d15/d1b/d44/f13d 0 2024-02-15T18:27:06.588 INFO:tasks.workunit.client.0.smithi042.stdout:1/831: fsync de/d54/dbf/dd0/d7c/fd9 0 2024-02-15T18:27:06.588 INFO:tasks.workunit.client.0.smithi042.stdout:1/832: stat de/d54/dbf/dd0/d7c/f12b 0 2024-02-15T18:27:06.588 INFO:tasks.workunit.client.0.smithi042.stdout:1/833: truncate de/d15/d67/d46/f68 673781 0 2024-02-15T18:27:06.589 INFO:tasks.workunit.client.0.smithi042.stdout:7/820: creat d10/d49/d6d/d6e/db3/f10e x:0 0 0 2024-02-15T18:27:06.592 INFO:tasks.workunit.client.0.smithi042.stdout:8/724: mknod dd/de/da4/d81/cd2 0 2024-02-15T18:27:06.593 INFO:tasks.workunit.client.0.smithi042.stdout:4/812: mknod da/d11/d29/db8/c119 0 2024-02-15T18:27:06.595 INFO:tasks.workunit.client.0.smithi042.stdout:1/834: mkdir de/d6d/d9e/df4/d13e 0 2024-02-15T18:27:06.596 INFO:tasks.workunit.client.0.smithi042.stdout:7/821: symlink d10/d43/d57/l10f 0 2024-02-15T18:27:06.606 INFO:tasks.workunit.client.0.smithi042.stdout:9/821: dwrite d6/da/d2f/d55/d107/f108 [0,4194304] 0 2024-02-15T18:27:06.607 INFO:tasks.workunit.client.0.smithi042.stdout:5/848: dread d0/dda/dc7/ffd [0,4194304] 0 2024-02-15T18:27:06.607 INFO:tasks.workunit.client.0.smithi042.stdout:5/849: fsync d0/dda/d42/fb9 0 2024-02-15T18:27:06.607 INFO:tasks.workunit.client.0.smithi042.stdout:9/822: fdatasync d6/dd/f4a 0 2024-02-15T18:27:06.607 INFO:tasks.workunit.client.0.smithi042.stdout:5/850: chown d0/dda/c8a 78246414 1 2024-02-15T18:27:06.607 INFO:tasks.workunit.client.0.smithi042.stdout:9/823: chown d6/dd/d9/d59/fbe 76 1 2024-02-15T18:27:06.617 INFO:tasks.workunit.client.0.smithi042.stdout:2/785: dwrite d0/d8/d10/f17 [0,4194304] 0 2024-02-15T18:27:06.617 INFO:tasks.workunit.client.0.smithi042.stdout:3/803: dwrite d1/d11/f66 [4194304,4194304] 0 2024-02-15T18:27:06.617 INFO:tasks.workunit.client.0.smithi042.stdout:3/804: stat d1/d11/d14/d4d/f7e 0 2024-02-15T18:27:06.617 INFO:tasks.workunit.client.0.smithi042.stdout:3/805: fsync d1/de/d19/fdc 0 2024-02-15T18:27:06.618 INFO:tasks.workunit.client.0.smithi042.stdout:6/890: dwrite d9/d11/d46/f77 [0,4194304] 0 2024-02-15T18:27:06.618 INFO:tasks.workunit.client.1.smithi191.stdout:2/398: dwrite df/d12/d19/d1a/f82 [0,4194304] 0 2024-02-15T18:27:06.618 INFO:tasks.workunit.client.1.smithi191.stdout:3/346: dwrite d6/d3b/d42/d49/f3d [0,4194304] 0 2024-02-15T18:27:06.621 INFO:tasks.workunit.client.0.smithi042.stdout:6/891: read d9/f2f [122814,60193] 0 2024-02-15T18:27:06.623 INFO:tasks.workunit.client.0.smithi042.stdout:8/725: symlink dd/de/dc0/ld3 0 2024-02-15T18:27:06.627 INFO:tasks.workunit.client.0.smithi042.stdout:4/813: symlink da/d11/d29/d4a/d4f/d80/l11a 0 2024-02-15T18:27:06.627 INFO:tasks.workunit.client.0.smithi042.stdout:4/814: stat da/d11/d85/db5/d103/d66/d78/d53/d82 0 2024-02-15T18:27:06.627 INFO:tasks.workunit.client.0.smithi042.stdout:4/815: write da/d11/d85/ff8 [3261575,123551] 0 2024-02-15T18:27:06.632 INFO:tasks.workunit.client.0.smithi042.stdout:1/835: mknod de/d54/dbf/dd0/d7c/dd6/ddf/d88/d111/c13f 0 2024-02-15T18:27:06.636 INFO:tasks.workunit.client.0.smithi042.stdout:0/782: sync 2024-02-15T18:27:06.638 INFO:tasks.workunit.client.0.smithi042.stdout:7/822: mkdir d10/d15/d110 0 2024-02-15T18:27:06.641 INFO:tasks.workunit.client.0.smithi042.stdout:9/824: unlink d6/da/d2f/d54/f64 0 2024-02-15T18:27:06.643 INFO:tasks.workunit.client.0.smithi042.stdout:5/851: creat d0/d38/d53/f12b x:0 0 0 2024-02-15T18:27:06.646 INFO:tasks.workunit.client.0.smithi042.stdout:2/786: symlink d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/lfb 0 2024-02-15T18:27:06.648 INFO:tasks.workunit.client.1.smithi191.stdout:3/347: rename d6/l12 to d6/d1b/l7b 0 2024-02-15T18:27:06.649 INFO:tasks.workunit.client.1.smithi191.stdout:3/348: write d6/d3b/d42/f4a [1800945,80189] 0 2024-02-15T18:27:06.649 INFO:tasks.workunit.client.0.smithi042.stdout:3/806: getdents d1/d11/d14/d4d/d73 0 2024-02-15T18:27:06.649 INFO:tasks.workunit.client.0.smithi042.stdout:4/816: dwrite da/f9c [0,4194304] 0 2024-02-15T18:27:06.649 INFO:tasks.workunit.client.0.smithi042.stdout:4/817: dread - da/d11/d29/d4a/d4f/fea zero size 2024-02-15T18:27:06.653 INFO:tasks.workunit.client.1.smithi191.stdout:2/399: truncate df/f1b 6372387 0 2024-02-15T18:27:06.654 INFO:tasks.workunit.client.1.smithi191.stdout:9/420: sync 2024-02-15T18:27:06.658 INFO:tasks.workunit.client.0.smithi042.stdout:6/892: rename d9/d11/d19/d1c/d31/d53/l113 to d9/d11/d97/df7/d12f/l13c 0 2024-02-15T18:27:06.659 INFO:tasks.workunit.client.0.smithi042.stdout:6/893: dread - d9/d11/d19/d35/dc4/fe3 zero size 2024-02-15T18:27:06.659 INFO:tasks.workunit.client.0.smithi042.stdout:6/894: chown d9/d8d/ff0 836 1 2024-02-15T18:27:06.660 INFO:tasks.workunit.client.0.smithi042.stdout:8/726: truncate dd/f13 335262 0 2024-02-15T18:27:06.661 INFO:tasks.workunit.client.0.smithi042.stdout:1/836: creat de/d15/d1b/d44/f140 x:0 0 0 2024-02-15T18:27:06.664 INFO:tasks.workunit.client.1.smithi191.stdout:9/421: mknod d0/c94 0 2024-02-15T18:27:06.665 INFO:tasks.workunit.client.1.smithi191.stdout:9/422: write d0/d7/dc/d8c/f35 [841198,83179] 0 2024-02-15T18:27:06.667 INFO:tasks.workunit.client.0.smithi042.stdout:0/783: chown d4/l11 116149 1 2024-02-15T18:27:06.667 INFO:tasks.workunit.client.0.smithi042.stdout:9/825: creat d6/dd/dcc/f11b x:0 0 0 2024-02-15T18:27:06.667 INFO:tasks.workunit.client.0.smithi042.stdout:5/852: truncate d0/dda/dc7/de1/f15 1300370 0 2024-02-15T18:27:06.671 INFO:tasks.workunit.client.1.smithi191.stdout:3/349: dwrite d6/d27/d4d/f58 [4194304,4194304] 0 2024-02-15T18:27:06.671 INFO:tasks.workunit.client.0.smithi042.stdout:3/807: symlink d1/d11/d2e/l108 0 2024-02-15T18:27:06.673 INFO:tasks.workunit.client.1.smithi191.stdout:9/423: rename d0/d7/dc/f8e to d0/d29/f95 0 2024-02-15T18:27:06.673 INFO:tasks.workunit.client.1.smithi191.stdout:9/424: write d0/d1d/f79 [742812,46188] 0 2024-02-15T18:27:06.677 INFO:tasks.workunit.client.0.smithi042.stdout:4/818: mkdir da/d2b/de3/d11b 0 2024-02-15T18:27:06.677 INFO:tasks.workunit.client.0.smithi042.stdout:4/819: fsync da/d11/d29/d4a/d4f/d90/fae 0 2024-02-15T18:27:06.678 INFO:tasks.workunit.client.0.smithi042.stdout:4/820: stat da/d11/d85/db5/d103/d66/f8a 0 2024-02-15T18:27:06.678 INFO:tasks.workunit.client.1.smithi191.stdout:3/350: truncate d6/d1b/d60/d73/f61 5908664 0 2024-02-15T18:27:06.680 INFO:tasks.workunit.client.0.smithi042.stdout:4/821: read da/d11/d29/d4a/d4f/d80/fa1 [4157948,78178] 0 2024-02-15T18:27:06.681 INFO:tasks.workunit.client.1.smithi191.stdout:9/425: link d0/d7/f9 d0/d29/d48/f96 0 2024-02-15T18:27:06.681 INFO:tasks.workunit.client.0.smithi042.stdout:6/895: dwrite d9/d11/d19/d35/f51 [0,4194304] 0 2024-02-15T18:27:06.681 INFO:tasks.workunit.client.0.smithi042.stdout:6/896: stat d9/d11/d46/df8 0 2024-02-15T18:27:06.681 INFO:tasks.workunit.client.0.smithi042.stdout:4/822: dread da/d11/d29/d4a/d4f/d80/fa9 [0,4194304] 0 2024-02-15T18:27:06.682 INFO:tasks.workunit.client.0.smithi042.stdout:4/823: write da/d11/d27/f89 [4051024,127914] 0 2024-02-15T18:27:06.683 INFO:tasks.workunit.client.0.smithi042.stdout:4/824: fsync da/d11/d29/d56/fe9 0 2024-02-15T18:27:06.683 INFO:tasks.workunit.client.0.smithi042.stdout:7/823: rename d10/ff8 to d10/d49/d6d/db6/f111 0 2024-02-15T18:27:06.683 INFO:tasks.workunit.client.0.smithi042.stdout:7/824: dread d10/d15/d39/d6c/f6f [0,4194304] 0 2024-02-15T18:27:06.684 INFO:tasks.workunit.client.1.smithi191.stdout:3/351: creat d6/d3b/d30/f7c x:0 0 0 2024-02-15T18:27:06.685 INFO:tasks.workunit.client.1.smithi191.stdout:3/352: truncate d6/f65 622355 0 2024-02-15T18:27:06.685 INFO:tasks.workunit.client.1.smithi191.stdout:3/353: chown d6/d3b/d30/f5c 65662 1 2024-02-15T18:27:06.686 INFO:tasks.workunit.client.0.smithi042.stdout:8/727: rename dd/de/d12/f9e to dd/de/d12/fd4 0 2024-02-15T18:27:06.691 INFO:tasks.workunit.client.0.smithi042.stdout:1/837: symlink de/d15/d1b/d44/df3/l141 0 2024-02-15T18:27:06.692 INFO:tasks.workunit.client.1.smithi191.stdout:3/354: dread d6/d1b/d60/d73/f41 [0,4194304] 0 2024-02-15T18:27:06.693 INFO:tasks.workunit.client.1.smithi191.stdout:9/426: unlink d0/d1d/d3f/c6f 0 2024-02-15T18:27:06.695 INFO:tasks.workunit.client.0.smithi042.stdout:0/784: truncate d4/d38/d5c/f61 951046 0 2024-02-15T18:27:06.695 INFO:tasks.workunit.client.0.smithi042.stdout:0/785: dread - d4/d38/d77/d2e/dae/ddf/fe8 zero size 2024-02-15T18:27:06.695 INFO:tasks.workunit.client.0.smithi042.stdout:9/826: mknod d6/da2/db1/c11c 0 2024-02-15T18:27:06.695 INFO:tasks.workunit.client.0.smithi042.stdout:9/827: stat d6/da/d1d 0 2024-02-15T18:27:06.696 INFO:tasks.workunit.client.1.smithi191.stdout:9/427: truncate d0/d7/dc/d30/d4f/f6d 543404 0 2024-02-15T18:27:06.697 INFO:tasks.workunit.client.1.smithi191.stdout:9/428: fsync d0/d1d/d3f/d80/f5d 0 2024-02-15T18:27:06.697 INFO:tasks.workunit.client.1.smithi191.stdout:9/429: fsync d0/d7/dc/d17/d33/f40 0 2024-02-15T18:27:06.699 INFO:tasks.workunit.client.0.smithi042.stdout:5/853: link d0/d38/d53/d64/f117 d0/dda/dc7/d52/da0/f12c 0 2024-02-15T18:27:06.699 INFO:tasks.workunit.client.0.smithi042.stdout:3/808: creat d1/d11/d14/d4d/f109 x:0 0 0 2024-02-15T18:27:06.700 INFO:tasks.workunit.client.0.smithi042.stdout:1/838: dread de/d15/d1b/fc7 [0,4194304] 0 2024-02-15T18:27:06.700 INFO:tasks.workunit.client.0.smithi042.stdout:3/809: readlink d1/l95 0 2024-02-15T18:27:06.700 INFO:tasks.workunit.client.0.smithi042.stdout:3/810: chown d1/d11/d2e/d4c 2 1 2024-02-15T18:27:06.700 INFO:tasks.workunit.client.0.smithi042.stdout:5/854: write d0/d39/fef [1540791,117528] 0 2024-02-15T18:27:06.700 INFO:tasks.workunit.client.0.smithi042.stdout:5/855: fdatasync d0/d89/fbe 0 2024-02-15T18:27:06.700 INFO:tasks.workunit.client.0.smithi042.stdout:5/856: fsync d0/d14/d88/da1/f116 0 2024-02-15T18:27:06.700 INFO:tasks.workunit.client.0.smithi042.stdout:5/857: chown d0/d38/d53/d64/f93 149141715 1 2024-02-15T18:27:06.700 INFO:tasks.workunit.client.0.smithi042.stdout:3/811: write d1/d11/d14/df6/ff7 [170865,97560] 0 2024-02-15T18:27:06.700 INFO:tasks.workunit.client.0.smithi042.stdout:6/897: symlink d9/d11/d46/l13d 0 2024-02-15T18:27:06.701 INFO:tasks.workunit.client.0.smithi042.stdout:4/825: symlink da/d11/da4/l11c 0 2024-02-15T18:27:06.702 INFO:tasks.workunit.client.0.smithi042.stdout:2/787: rename d0/d8/d10/d5a/fbc to d0/d8/d20/d3f/d7c/ffc 0 2024-02-15T18:27:06.702 INFO:tasks.workunit.client.0.smithi042.stdout:2/788: fsync d0/d8/d10/d31/d3a/d53/fcc 0 2024-02-15T18:27:06.705 INFO:tasks.workunit.client.0.smithi042.stdout:7/825: mkdir d10/d49/dc4/d112 0 2024-02-15T18:27:06.712 INFO:tasks.workunit.client.0.smithi042.stdout:0/786: dread d4/d9/d34/d8b/f90 [0,4194304] 0 2024-02-15T18:27:06.712 INFO:tasks.workunit.client.0.smithi042.stdout:0/787: write d4/d9/d5a/dfc/f83 [301397,99463] 0 2024-02-15T18:27:06.713 INFO:tasks.workunit.client.0.smithi042.stdout:0/788: chown l3 30947434 1 2024-02-15T18:27:06.714 INFO:tasks.workunit.client.0.smithi042.stdout:1/839: creat de/d54/dbf/dd0/d7c/dd6/ddf/d88/d111/f142 x:0 0 0 2024-02-15T18:27:06.715 INFO:tasks.workunit.client.0.smithi042.stdout:1/840: write de/d15/d2c/f38 [3572847,20803] 0 2024-02-15T18:27:06.715 INFO:tasks.workunit.client.0.smithi042.stdout:1/841: chown de/d54/dbf/dd0/d7c 15 1 2024-02-15T18:27:06.716 INFO:tasks.workunit.client.0.smithi042.stdout:5/858: rmdir d0/d39/d8d 39 2024-02-15T18:27:06.716 INFO:tasks.workunit.client.0.smithi042.stdout:5/859: write d0/dda/dc7/de1/dd8/ff4 [259614,55734] 0 2024-02-15T18:27:06.717 INFO:tasks.workunit.client.0.smithi042.stdout:3/812: symlink d1/de/d1a/l10a 0 2024-02-15T18:27:06.717 INFO:tasks.workunit.client.0.smithi042.stdout:3/813: chown d1/d11/d7a/lc5 7 1 2024-02-15T18:27:06.717 INFO:tasks.workunit.client.0.smithi042.stdout:2/789: creat d0/d8/d20/d41/ffd x:0 0 0 2024-02-15T18:27:06.718 INFO:tasks.workunit.client.0.smithi042.stdout:0/789: getdents d4/d38/d77/d2e 0 2024-02-15T18:27:06.719 INFO:tasks.workunit.client.0.smithi042.stdout:5/860: read d0/d14/fa3 [3464905,7343] 0 2024-02-15T18:27:06.719 INFO:tasks.workunit.client.0.smithi042.stdout:5/861: write d0/d12/d2a/d105/f11e [290692,113963] 0 2024-02-15T18:27:06.719 INFO:tasks.workunit.client.0.smithi042.stdout:1/842: creat de/d54/dbf/dd0/f143 x:0 0 0 2024-02-15T18:27:06.720 INFO:tasks.workunit.client.0.smithi042.stdout:9/828: dread d6/da/d2f/d55/d107/f108 [0,4194304] 0 2024-02-15T18:27:06.720 INFO:tasks.workunit.client.0.smithi042.stdout:3/814: mknod d1/d11/d14/d4d/d73/d89/dd9/c10b 0 2024-02-15T18:27:06.720 INFO:tasks.workunit.client.0.smithi042.stdout:6/898: dwrite d9/d3e/f8b [4194304,4194304] 0 2024-02-15T18:27:06.721 INFO:tasks.workunit.client.0.smithi042.stdout:0/790: creat d4/d9/d34/f103 x:0 0 0 2024-02-15T18:27:06.721 INFO:tasks.workunit.client.0.smithi042.stdout:0/791: readlink d4/d38/d77/d2e/l52 0 2024-02-15T18:27:06.722 INFO:tasks.workunit.client.0.smithi042.stdout:5/862: creat d0/dda/dc7/de1/f12d x:0 0 0 2024-02-15T18:27:06.726 INFO:tasks.workunit.client.0.smithi042.stdout:9/829: link d6/da/d2f/d55/d73/d8d/fe9 d6/da/d1d/d2d/d40/f11d 0 2024-02-15T18:27:06.726 INFO:tasks.workunit.client.0.smithi042.stdout:9/830: truncate d6/da/d1d/d2d/d40/d95/fdf 740914 0 2024-02-15T18:27:06.726 INFO:tasks.workunit.client.0.smithi042.stdout:7/826: dread d10/d43/d57/f66 [0,4194304] 0 2024-02-15T18:27:06.727 INFO:tasks.workunit.client.0.smithi042.stdout:4/826: dread da/d11/d29/f32 [0,4194304] 0 2024-02-15T18:27:06.727 INFO:tasks.workunit.client.1.smithi191.stdout:9/430: dread d0/d7/dc/f15 [0,4194304] 0 2024-02-15T18:27:06.732 INFO:tasks.workunit.client.0.smithi042.stdout:3/815: mkdir d1/d11/d10c 0 2024-02-15T18:27:06.733 INFO:tasks.workunit.client.1.smithi191.stdout:9/431: creat d0/d7/dc/d30/d3b/d56/d59/f97 x:0 0 0 2024-02-15T18:27:06.733 INFO:tasks.workunit.client.0.smithi042.stdout:1/843: dread de/d15/d2c/f5a [0,4194304] 0 2024-02-15T18:27:06.734 INFO:tasks.workunit.client.0.smithi042.stdout:1/844: chown ca 6920 1 2024-02-15T18:27:06.735 INFO:tasks.workunit.client.0.smithi042.stdout:6/899: getdents d9/d11/d19/d35/d55/d92/dfc/d121 0 2024-02-15T18:27:06.736 INFO:tasks.workunit.client.0.smithi042.stdout:0/792: mknod d4/d9/d34/dab/dc8/d6c/df7/c104 0 2024-02-15T18:27:06.738 INFO:tasks.workunit.client.1.smithi191.stdout:9/432: unlink d0/d29/c7d 0 2024-02-15T18:27:06.739 INFO:tasks.workunit.client.0.smithi042.stdout:6/900: write d9/d11/d1d/f90 [4168881,20950] 0 2024-02-15T18:27:06.739 INFO:tasks.workunit.client.0.smithi042.stdout:5/863: symlink d0/d39/d8d/l12e 0 2024-02-15T18:27:06.739 INFO:tasks.workunit.client.0.smithi042.stdout:5/864: chown d0/d38/d53/d64/f9f 361 1 2024-02-15T18:27:06.739 INFO:tasks.workunit.client.1.smithi191.stdout:9/433: read d0/d7/dc/d30/f4e [888447,86368] 0 2024-02-15T18:27:06.740 INFO:tasks.workunit.client.0.smithi042.stdout:9/831: creat d6/da/d2f/d55/d107/f11e x:0 0 0 2024-02-15T18:27:06.743 INFO:tasks.workunit.client.0.smithi042.stdout:4/827: symlink da/d11/d85/db5/d103/d66/d78/d53/d82/d113/l11d 0 2024-02-15T18:27:06.743 INFO:tasks.workunit.client.0.smithi042.stdout:2/790: dwrite d0/d8/d20/d41/ffd [0,4194304] 0 2024-02-15T18:27:06.746 INFO:tasks.workunit.client.1.smithi191.stdout:9/434: symlink d0/d7/dc/d8c/l98 0 2024-02-15T18:27:06.747 INFO:tasks.workunit.client.1.smithi191.stdout:9/435: readlink d0/d7/l47 0 2024-02-15T18:27:06.750 INFO:tasks.workunit.client.0.smithi042.stdout:3/816: link d1/d9c/ff9 d1/d11/d2e/de7/f10d 0 2024-02-15T18:27:06.750 INFO:tasks.workunit.client.0.smithi042.stdout:4/828: dread da/d11/d67/f6f [0,4194304] 0 2024-02-15T18:27:06.750 INFO:tasks.workunit.client.0.smithi042.stdout:7/827: dwrite d10/d15/d8f/da5/dad/d104/d23/d51/f99 [0,4194304] 0 2024-02-15T18:27:06.751 INFO:tasks.workunit.client.0.smithi042.stdout:1/845: link de/d15/d67/d74/fab de/d15/d2c/f144 0 2024-02-15T18:27:06.751 INFO:tasks.workunit.client.0.smithi042.stdout:1/846: chown de/d15/d2c/d51/d70/l83 2652532 1 2024-02-15T18:27:06.751 INFO:tasks.workunit.client.0.smithi042.stdout:3/817: write d1/d11/d7a/fe1 [613494,64797] 0 2024-02-15T18:27:06.751 INFO:tasks.workunit.client.0.smithi042.stdout:3/818: dread - d1/d11/d14/dbf/fe6 zero size 2024-02-15T18:27:06.751 INFO:tasks.workunit.client.1.smithi191.stdout:9/436: creat d0/d7/dc/d17/f99 x:0 0 0 2024-02-15T18:27:06.752 INFO:tasks.workunit.client.0.smithi042.stdout:0/793: getdents d4/d9/d62 0 2024-02-15T18:27:06.752 INFO:tasks.workunit.client.0.smithi042.stdout:6/901: mknod d9/d11/d19/d35/d55/d92/dfc/d121/c13e 0 2024-02-15T18:27:06.752 INFO:tasks.workunit.client.0.smithi042.stdout:6/902: read d9/d11/d19/d1c/d31/d4f/f68 [374182,3935] 0 2024-02-15T18:27:06.753 INFO:tasks.workunit.client.1.smithi191.stdout:9/437: mkdir d0/d29/d9a 0 2024-02-15T18:27:06.753 INFO:tasks.workunit.client.1.smithi191.stdout:9/438: write d0/d29/f3d [1788611,86223] 0 2024-02-15T18:27:06.753 INFO:tasks.workunit.client.1.smithi191.stdout:9/439: chown d0/l16 9 1 2024-02-15T18:27:06.753 INFO:tasks.workunit.client.0.smithi042.stdout:9/832: creat d6/da/d1d/d2d/d40/de0/f11f x:0 0 0 2024-02-15T18:27:06.753 INFO:tasks.workunit.client.0.smithi042.stdout:9/833: read - d6/da/d2f/d55/d73/d8d/fea zero size 2024-02-15T18:27:06.754 INFO:tasks.workunit.client.0.smithi042.stdout:9/834: fdatasync d6/da/d2f/d55/d73/d8d/fea 0 2024-02-15T18:27:06.754 INFO:tasks.workunit.client.1.smithi191.stdout:9/440: truncate d0/d7/dc/d17/d33/f4d 8712813 0 2024-02-15T18:27:06.756 INFO:tasks.workunit.client.0.smithi042.stdout:2/791: mkdir d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/dfe 0 2024-02-15T18:27:06.756 INFO:tasks.workunit.client.0.smithi042.stdout:2/792: write d0/d8/d10/d5a/f90 [881953,18449] 0 2024-02-15T18:27:06.757 INFO:tasks.workunit.client.1.smithi191.stdout:9/441: creat d0/d7/dc/f9b x:0 0 0 2024-02-15T18:27:06.758 INFO:tasks.workunit.client.0.smithi042.stdout:2/793: dread d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d2e/ff2 [0,4194304] 0 2024-02-15T18:27:06.758 INFO:tasks.workunit.client.0.smithi042.stdout:5/865: dwrite d0/d14/fa3 [4194304,4194304] 0 2024-02-15T18:27:06.758 INFO:tasks.workunit.client.0.smithi042.stdout:2/794: fdatasync d0/d8/d10/fd9 0 2024-02-15T18:27:06.758 INFO:tasks.workunit.client.0.smithi042.stdout:5/866: chown d0/d14/d88/df8/f11a 113 1 2024-02-15T18:27:06.770 INFO:tasks.workunit.client.1.smithi191.stdout:9/442: mknod d0/d29/d48/d66/c9c 0 2024-02-15T18:27:06.770 INFO:tasks.workunit.client.1.smithi191.stdout:9/443: stat d0/d7/dc/d30/d4f 0 2024-02-15T18:27:06.770 INFO:tasks.workunit.client.1.smithi191.stdout:9/444: chown d0/d7/dc/d30/d4f/f6d 314212 1 2024-02-15T18:27:06.770 INFO:tasks.workunit.client.1.smithi191.stdout:9/445: write d0/d7/dc/d30/d4f/f6d [1366172,111694] 0 2024-02-15T18:27:06.770 INFO:tasks.workunit.client.1.smithi191.stdout:9/446: write d0/d7/f28 [1153476,115886] 0 2024-02-15T18:27:06.770 INFO:tasks.workunit.client.1.smithi191.stdout:9/447: chown d0/d29/d48/f96 11 1 2024-02-15T18:27:06.770 INFO:tasks.workunit.client.0.smithi042.stdout:4/829: mkdir da/d11/d29/d11e 0 2024-02-15T18:27:06.771 INFO:tasks.workunit.client.0.smithi042.stdout:7/828: mknod d10/d43/d57/c113 0 2024-02-15T18:27:06.772 INFO:tasks.workunit.client.0.smithi042.stdout:1/847: rmdir de/d15/d2c/d51/d70 39 2024-02-15T18:27:06.774 INFO:tasks.workunit.client.1.smithi191.stdout:9/448: write d0/d7/dc/f15 [5278336,22271] 0 2024-02-15T18:27:06.775 INFO:tasks.workunit.client.0.smithi042.stdout:3/819: creat d1/d11/d14/df6/f10e x:0 0 0 2024-02-15T18:27:06.775 INFO:tasks.workunit.client.0.smithi042.stdout:3/820: chown d1/d11/d2e/de7 119 1 2024-02-15T18:27:06.775 INFO:tasks.workunit.client.0.smithi042.stdout:3/821: fdatasync d1/de/f2d 0 2024-02-15T18:27:06.777 INFO:tasks.workunit.client.0.smithi042.stdout:0/794: getdents d4/d9/d34/dab/db0 0 2024-02-15T18:27:06.777 INFO:tasks.workunit.client.0.smithi042.stdout:0/795: chown d4/d9/d34/dab/dc8/d6c/df9 346525 1 2024-02-15T18:27:06.777 INFO:tasks.workunit.client.0.smithi042.stdout:0/796: fdatasync d4/d9/d5a/dfc/ffd 0 2024-02-15T18:27:06.779 INFO:tasks.workunit.client.0.smithi042.stdout:5/867: dwrite d0/d38/d53/d64/f93 [0,4194304] 0 2024-02-15T18:27:06.779 INFO:tasks.workunit.client.0.smithi042.stdout:5/868: chown d0/dda/d42/fb9 2710 1 2024-02-15T18:27:06.780 INFO:tasks.workunit.client.0.smithi042.stdout:6/903: getdents d9/d11/d19/d1c/d31/d9b/db4 0 2024-02-15T18:27:06.781 INFO:tasks.workunit.client.0.smithi042.stdout:3/822: dread d1/d11/d7a/fe1 [0,4194304] 0 2024-02-15T18:27:06.782 INFO:tasks.workunit.client.1.smithi191.stdout:9/449: mknod d0/d7/dc/d32/c9d 0 2024-02-15T18:27:06.784 INFO:tasks.workunit.client.0.smithi042.stdout:9/835: mkdir d6/da/d75/d120 0 2024-02-15T18:27:06.784 INFO:tasks.workunit.client.0.smithi042.stdout:9/836: dread - d6/da/f9a zero size 2024-02-15T18:27:06.784 INFO:tasks.workunit.client.0.smithi042.stdout:9/837: chown d6/dd/d9/c52 5 1 2024-02-15T18:27:06.789 INFO:tasks.workunit.client.0.smithi042.stdout:1/848: dwrite de/d15/d95/fb9 [0,4194304] 0 2024-02-15T18:27:06.790 INFO:tasks.workunit.client.0.smithi042.stdout:1/849: read de/d15/d1b/d44/fb2 [2094646,14509] 0 2024-02-15T18:27:06.791 INFO:tasks.workunit.client.0.smithi042.stdout:2/795: mkdir d0/d8/d10/d31/d3a/dff 0 2024-02-15T18:27:06.791 INFO:tasks.workunit.client.0.smithi042.stdout:2/796: fdatasync d0/d8/d20/d7d/d94/fca 0 2024-02-15T18:27:06.794 INFO:tasks.workunit.client.0.smithi042.stdout:4/830: creat da/d11/d29/f11f x:0 0 0 2024-02-15T18:27:06.795 INFO:tasks.workunit.client.0.smithi042.stdout:7/829: creat d10/d15/d39/d6c/dba/f114 x:0 0 0 2024-02-15T18:27:06.795 INFO:tasks.workunit.client.0.smithi042.stdout:7/830: getdents d10/d49 0 2024-02-15T18:27:06.795 INFO:tasks.workunit.client.0.smithi042.stdout:7/831: write d10/d49/d6d/d6e/db3/fc7 [1429611,17083] 0 2024-02-15T18:27:06.796 INFO:tasks.workunit.client.0.smithi042.stdout:7/832: chown d10/d15/d8f/da5/dad/fd0 316571488 1 2024-02-15T18:27:06.796 INFO:tasks.workunit.client.1.smithi191.stdout:9/450: dread d0/d7/dc/d8c/f6b [0,4194304] 0 2024-02-15T18:27:06.796 INFO:tasks.workunit.client.1.smithi191.stdout:9/451: write d0/d7/f21 [4072965,46873] 0 2024-02-15T18:27:06.797 INFO:tasks.workunit.client.0.smithi042.stdout:6/904: dwrite d9/d11/d19/fed [8388608,4194304] 0 2024-02-15T18:27:06.799 INFO:tasks.workunit.client.0.smithi042.stdout:8/728: sync 2024-02-15T18:27:06.801 INFO:tasks.workunit.client.0.smithi042.stdout:0/797: creat d4/d38/d77/d2e/dda/d101/f105 x:0 0 0 2024-02-15T18:27:06.801 INFO:tasks.workunit.client.0.smithi042.stdout:0/798: truncate d4/d9/f46 5829304 0 2024-02-15T18:27:06.801 INFO:tasks.workunit.client.0.smithi042.stdout:0/799: dread - d4/d38/d77/d2e/dae/ddf/fb6 zero size 2024-02-15T18:27:06.806 INFO:tasks.workunit.client.0.smithi042.stdout:3/823: symlink d1/d11/l10f 0 2024-02-15T18:27:06.806 INFO:tasks.workunit.client.0.smithi042.stdout:5/869: rename d0/d12/d2a/d102/c10f to d0/d38/d53/c12f 0 2024-02-15T18:27:06.806 INFO:tasks.workunit.client.0.smithi042.stdout:9/838: truncate d6/dd/f19 1372111 0 2024-02-15T18:27:06.808 INFO:tasks.workunit.client.0.smithi042.stdout:2/797: creat d0/d8/d20/d7d/d7e/f100 x:0 0 0 2024-02-15T18:27:06.808 INFO:tasks.workunit.client.0.smithi042.stdout:4/831: symlink da/d11/d29/d4a/d4f/dcf/l120 0 2024-02-15T18:27:06.809 INFO:tasks.workunit.client.0.smithi042.stdout:7/833: mkdir d10/d43/d57/d115 0 2024-02-15T18:27:06.812 INFO:tasks.workunit.client.0.smithi042.stdout:8/729: creat dd/de/d3c/fd5 x:0 0 0 2024-02-15T18:27:06.812 INFO:tasks.workunit.client.0.smithi042.stdout:8/730: write dd/de/da4/d67/d5a/f91 [5995315,49331] 0 2024-02-15T18:27:06.812 INFO:tasks.workunit.client.0.smithi042.stdout:8/731: chown dd/de/da4/d67/d6f/d78/d9a/fb5 22313786 1 2024-02-15T18:27:06.812 INFO:tasks.workunit.client.0.smithi042.stdout:8/732: stat dd/de/da4/dd0 0 2024-02-15T18:27:06.812 INFO:tasks.workunit.client.0.smithi042.stdout:5/870: write d0/dda/dc7/de1/f10 [1767397,47053] 0 2024-02-15T18:27:06.813 INFO:tasks.workunit.client.0.smithi042.stdout:0/800: rename d4/d9/d34/dab/cba to d4/d9/c106 0 2024-02-15T18:27:06.813 INFO:tasks.workunit.client.0.smithi042.stdout:6/905: dwrite d9/d11/d19/d35/d49/f4c [0,4194304] 0 2024-02-15T18:27:06.813 INFO:tasks.workunit.client.0.smithi042.stdout:0/801: truncate d4/d9/d62/d74/fc7 41948 0 2024-02-15T18:27:06.814 INFO:tasks.workunit.client.0.smithi042.stdout:4/832: creat da/d3b/d41/d110/f121 x:0 0 0 2024-02-15T18:27:06.818 INFO:tasks.workunit.client.1.smithi191.stdout:9/452: dwrite d0/d7/dc/d8c/f1c [0,4194304] 0 2024-02-15T18:27:06.818 INFO:tasks.workunit.client.0.smithi042.stdout:7/834: truncate d10/d49/d6d/d78/d83/f87 940590 0 2024-02-15T18:27:06.819 INFO:tasks.workunit.client.0.smithi042.stdout:8/733: truncate dd/de/da4/d81/f88 66752 0 2024-02-15T18:27:06.821 INFO:tasks.workunit.client.0.smithi042.stdout:5/871: creat d0/dda/dc7/d52/dad/dba/f130 x:0 0 0 2024-02-15T18:27:06.822 INFO:tasks.workunit.client.1.smithi191.stdout:8/507: sync 2024-02-15T18:27:06.822 INFO:tasks.workunit.client.1.smithi191.stdout:0/293: sync 2024-02-15T18:27:06.822 INFO:tasks.workunit.client.1.smithi191.stdout:0/294: fsync d7/d4a/f59 0 2024-02-15T18:27:06.822 INFO:tasks.workunit.client.1.smithi191.stdout:4/434: sync 2024-02-15T18:27:06.824 INFO:tasks.workunit.client.1.smithi191.stdout:8/508: write d3/fd [3245621,122415] 0 2024-02-15T18:27:06.825 INFO:tasks.workunit.client.0.smithi042.stdout:2/798: rename d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/la8 to d0/d8/d10/d31/d3a/l101 0 2024-02-15T18:27:06.825 INFO:tasks.workunit.client.0.smithi042.stdout:2/799: readlink d0/d8/d20/d7d/d7e/dc7/l5b 0 2024-02-15T18:27:06.825 INFO:tasks.workunit.client.0.smithi042.stdout:2/800: truncate d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d2e/f74 4386354 0 2024-02-15T18:27:06.826 INFO:tasks.workunit.client.1.smithi191.stdout:0/295: rename d7/da/d1a/d34/f4e to d7/da/d17/d31/d5f/f61 0 2024-02-15T18:27:06.827 INFO:tasks.workunit.client.0.smithi042.stdout:7/835: write d10/d49/d6d/fd7 [269831,29289] 0 2024-02-15T18:27:06.827 INFO:tasks.workunit.client.0.smithi042.stdout:7/836: fsync d10/d43/f45 0 2024-02-15T18:27:06.828 INFO:tasks.workunit.client.0.smithi042.stdout:6/906: mknod d9/d11/d19/d1c/d31/d9b/db4/c13f 0 2024-02-15T18:27:06.829 INFO:tasks.workunit.client.1.smithi191.stdout:4/435: creat d2/d5/d7/fa9 x:0 0 0 2024-02-15T18:27:06.829 INFO:tasks.workunit.client.1.smithi191.stdout:4/436: fsync d2/f28 0 2024-02-15T18:27:06.829 INFO:tasks.workunit.client.1.smithi191.stdout:4/437: chown d2/d5/d7/d90 857 1 2024-02-15T18:27:06.829 INFO:tasks.workunit.client.1.smithi191.stdout:4/438: write d2/f2c [2682746,62649] 0 2024-02-15T18:27:06.830 INFO:tasks.workunit.client.1.smithi191.stdout:4/439: write d2/f5c [1935,32239] 0 2024-02-15T18:27:06.830 INFO:tasks.workunit.client.1.smithi191.stdout:4/440: dread - d2/d5/d86/f96 zero size 2024-02-15T18:27:06.831 INFO:tasks.workunit.client.0.smithi042.stdout:4/833: unlink da/d3b/c54 0 2024-02-15T18:27:06.832 INFO:tasks.workunit.client.1.smithi191.stdout:8/509: creat d3/d2c/d1a/d23/d3e/db2/fbb x:0 0 0 2024-02-15T18:27:06.834 INFO:tasks.workunit.client.1.smithi191.stdout:9/453: dwrite d0/d29/d48/d66/f68 [0,4194304] 0 2024-02-15T18:27:06.835 INFO:tasks.workunit.client.0.smithi042.stdout:8/734: mknod dd/de/d12/cd6 0 2024-02-15T18:27:06.835 INFO:tasks.workunit.client.0.smithi042.stdout:8/735: dread - dd/de/da4/d67/d5a/db2/fc3 zero size 2024-02-15T18:27:06.837 INFO:tasks.workunit.client.1.smithi191.stdout:9/454: write d0/d7/fd [1566345,9721] 0 2024-02-15T18:27:06.842 INFO:tasks.workunit.client.0.smithi042.stdout:5/872: mkdir d0/dda/dc7/d52/da0/d125/d131 0 2024-02-15T18:27:06.842 INFO:tasks.workunit.client.0.smithi042.stdout:5/873: write d0/d22/fee [64777,90957] 0 2024-02-15T18:27:06.844 INFO:tasks.workunit.client.0.smithi042.stdout:2/801: unlink d0/d8/d20/d7d/d7e/dc7/d8b/ldd 0 2024-02-15T18:27:06.849 INFO:tasks.workunit.client.0.smithi042.stdout:0/802: mknod d4/c107 0 2024-02-15T18:27:06.849 INFO:tasks.workunit.client.0.smithi042.stdout:0/803: readlink d4/d9/l1f 0 2024-02-15T18:27:06.850 INFO:tasks.workunit.client.0.smithi042.stdout:7/837: mkdir d10/d15/d39/de3/d116 0 2024-02-15T18:27:06.850 INFO:tasks.workunit.client.0.smithi042.stdout:7/838: fsync d10/d15/d8f/da5/dad/d104/f9e 0 2024-02-15T18:27:06.850 INFO:tasks.workunit.client.0.smithi042.stdout:6/907: creat d9/d11/d46/f140 x:0 0 0 2024-02-15T18:27:06.850 INFO:tasks.workunit.client.0.smithi042.stdout:6/908: chown d9/d3e/f96 215 1 2024-02-15T18:27:06.850 INFO:tasks.workunit.client.0.smithi042.stdout:6/909: chown d9/d11/d19/d35/d49/l4a 26 1 2024-02-15T18:27:06.850 INFO:tasks.workunit.client.0.smithi042.stdout:6/910: readlink d9/d11/d19/d1c/d31/dd2/lee 0 2024-02-15T18:27:06.851 INFO:tasks.workunit.client.1.smithi191.stdout:0/296: dwrite d7/da/f25 [0,4194304] 0 2024-02-15T18:27:06.851 INFO:tasks.workunit.client.0.smithi042.stdout:4/834: creat da/d3a/dc9/f122 x:0 0 0 2024-02-15T18:27:06.859 INFO:tasks.workunit.client.1.smithi191.stdout:0/297: write d7/da/f16 [2491209,128116] 0 2024-02-15T18:27:06.859 INFO:tasks.workunit.client.1.smithi191.stdout:0/298: chown d7/da/d1a/d34/c3c 82087546 1 2024-02-15T18:27:06.859 INFO:tasks.workunit.client.1.smithi191.stdout:9/455: link d0/d7/dc/d30/d3b/d56/c87 d0/d7/dc/d17/c9e 0 2024-02-15T18:27:06.860 INFO:tasks.workunit.client.1.smithi191.stdout:0/299: write d7/da/d1a/d34/f5e [120050,53893] 0 2024-02-15T18:27:06.864 INFO:tasks.workunit.client.0.smithi042.stdout:0/804: write d4/d38/d39/d8e/f67 [3109955,44369] 0 2024-02-15T18:27:06.865 INFO:tasks.workunit.client.0.smithi042.stdout:0/805: chown d4/d38/c47 13471 1 2024-02-15T18:27:06.865 INFO:tasks.workunit.client.0.smithi042.stdout:8/736: dwrite dd/de/da4/d67/d5a/f64 [0,4194304] 0 2024-02-15T18:27:06.870 INFO:tasks.workunit.client.1.smithi191.stdout:9/456: mknod d0/d7/dc/d32/d89/c9f 0 2024-02-15T18:27:06.871 INFO:tasks.workunit.client.1.smithi191.stdout:0/300: rmdir d7/d4a 39 2024-02-15T18:27:06.872 INFO:tasks.workunit.client.1.smithi191.stdout:0/301: chown d7/da/d1a/d2c/d4c 0 1 2024-02-15T18:27:06.872 INFO:tasks.workunit.client.0.smithi042.stdout:8/737: dread fc [4194304,4194304] 0 2024-02-15T18:27:06.876 INFO:tasks.workunit.client.1.smithi191.stdout:0/302: write d7/da/d1a/d2c/f4b [245395,25796] 0 2024-02-15T18:27:06.876 INFO:tasks.workunit.client.1.smithi191.stdout:0/303: chown d7/da/d1a/d2c/f4b 648537 1 2024-02-15T18:27:06.876 INFO:tasks.workunit.client.1.smithi191.stdout:9/457: link d0/d7/dc/d30/c46 d0/d7/dc/d17/ca0 0 2024-02-15T18:27:06.876 INFO:tasks.workunit.client.1.smithi191.stdout:9/458: write d0/d7/dc/d30/f76 [604698,88259] 0 2024-02-15T18:27:06.876 INFO:tasks.workunit.client.1.smithi191.stdout:0/304: mknod d7/da/d1a/d34/d27/c62 0 2024-02-15T18:27:06.877 INFO:tasks.workunit.client.0.smithi042.stdout:5/874: link d0/dda/dc7/d52/dad/dba/dbd/f12a d0/dda/dc7/d52/dad/f132 0 2024-02-15T18:27:06.878 INFO:tasks.workunit.client.0.smithi042.stdout:5/875: write d0/d38/d53/d68/f10a [124014,32121] 0 2024-02-15T18:27:06.878 INFO:tasks.workunit.client.0.smithi042.stdout:5/876: write d0/dda/d42/d7d/db5/f126 [95635,56557] 0 2024-02-15T18:27:06.878 INFO:tasks.workunit.client.1.smithi191.stdout:8/510: dwrite d3/d5e/f73 [0,4194304] 0 2024-02-15T18:27:06.878 INFO:tasks.workunit.client.1.smithi191.stdout:8/511: dread - d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/f98 zero size 2024-02-15T18:27:06.879 INFO:tasks.workunit.client.0.smithi042.stdout:2/802: dwrite d0/d35/d9e/fe9 [0,4194304] 0 2024-02-15T18:27:06.879 INFO:tasks.workunit.client.0.smithi042.stdout:2/803: dread - d0/d8/d20/d7d/d7e/dc7/fe1 zero size 2024-02-15T18:27:06.880 INFO:tasks.workunit.client.1.smithi191.stdout:9/459: creat d0/d7/dc/d30/d3b/fa1 x:0 0 0 2024-02-15T18:27:06.880 INFO:tasks.workunit.client.0.smithi042.stdout:7/839: mkdir d10/d15/d39/d6c/dba/d117 0 2024-02-15T18:27:06.880 INFO:tasks.workunit.client.0.smithi042.stdout:4/835: dwrite da/d11/d29/d56/fe9 [0,4194304] 0 2024-02-15T18:27:06.881 INFO:tasks.workunit.client.1.smithi191.stdout:9/460: dread d0/d7/dc/d17/d33/f43 [0,4194304] 0 2024-02-15T18:27:06.883 INFO:tasks.workunit.client.0.smithi042.stdout:6/911: getdents d9 0 2024-02-15T18:27:06.890 INFO:tasks.workunit.client.0.smithi042.stdout:0/806: link d4/d9/d5a/dfc/f8c d4/d38/d39/f108 0 2024-02-15T18:27:06.891 INFO:tasks.workunit.client.0.smithi042.stdout:4/836: dread da/d11/d67/f95 [0,4194304] 0 2024-02-15T18:27:06.894 INFO:tasks.workunit.client.0.smithi042.stdout:8/738: symlink dd/de/da4/d67/d6f/ld7 0 2024-02-15T18:27:06.897 INFO:tasks.workunit.client.0.smithi042.stdout:5/877: mknod d0/dda/dc7/d52/dad/c133 0 2024-02-15T18:27:06.898 INFO:tasks.workunit.client.1.smithi191.stdout:9/461: creat d0/d1d/d3f/d80/fa2 x:0 0 0 2024-02-15T18:27:06.901 INFO:tasks.workunit.client.0.smithi042.stdout:2/804: mkdir d0/d8/d20/d3f/d7c/d102 0 2024-02-15T18:27:06.902 INFO:tasks.workunit.client.1.smithi191.stdout:8/512: mkdir d3/d2c/d1a/d23/d2e/d64/dbc 0 2024-02-15T18:27:06.904 INFO:tasks.workunit.client.0.smithi042.stdout:7/840: dwrite d10/d15/f3d [0,4194304] 0 2024-02-15T18:27:06.904 INFO:tasks.workunit.client.0.smithi042.stdout:7/841: dread - d10/d49/d6d/d6e/db3/f10e zero size 2024-02-15T18:27:06.905 INFO:tasks.workunit.client.1.smithi191.stdout:8/513: dread d3/d2c/d1a/d23/f5c [0,4194304] 0 2024-02-15T18:27:06.905 INFO:tasks.workunit.client.1.smithi191.stdout:8/514: chown d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/c9e 0 1 2024-02-15T18:27:06.910 INFO:tasks.workunit.client.0.smithi042.stdout:6/912: rename d9/d11/d19/d35/d55/lbf to d9/d11/d19/d1c/d31/d53/dbe/l141 0 2024-02-15T18:27:06.910 INFO:tasks.workunit.client.0.smithi042.stdout:6/913: truncate d9/d11/d46/d89/dbc/f131 675962 0 2024-02-15T18:27:06.913 INFO:tasks.workunit.client.0.smithi042.stdout:4/837: symlink da/d11/d29/d4a/d4f/d80/l123 0 2024-02-15T18:27:06.913 INFO:tasks.workunit.client.0.smithi042.stdout:4/838: stat da/d11/d29/d4a/fa0 0 2024-02-15T18:27:06.913 INFO:tasks.workunit.client.0.smithi042.stdout:4/839: read da/d11/d85/db5/d103/d66/d78/d2f/d4d/f47 [261431,120382] 0 2024-02-15T18:27:06.915 INFO:tasks.workunit.client.0.smithi042.stdout:5/878: creat d0/d50/df7/f134 x:0 0 0 2024-02-15T18:27:06.915 INFO:tasks.workunit.client.0.smithi042.stdout:5/879: truncate d0/d14/f2e 1742325 0 2024-02-15T18:27:06.916 INFO:tasks.workunit.client.0.smithi042.stdout:1/850: sync 2024-02-15T18:27:06.916 INFO:tasks.workunit.client.0.smithi042.stdout:9/839: sync 2024-02-15T18:27:06.916 INFO:tasks.workunit.client.0.smithi042.stdout:3/824: sync 2024-02-15T18:27:06.916 INFO:tasks.workunit.client.0.smithi042.stdout:1/851: chown de/d15/d2c 50221899 1 2024-02-15T18:27:06.917 INFO:tasks.workunit.client.0.smithi042.stdout:9/840: dread d6/da/d75/ff2 [0,4194304] 0 2024-02-15T18:27:06.917 INFO:tasks.workunit.client.0.smithi042.stdout:9/841: stat d6/d9d 0 2024-02-15T18:27:06.917 INFO:tasks.workunit.client.0.smithi042.stdout:0/807: dwrite d4/d9/d34/f40 [0,4194304] 0 2024-02-15T18:27:06.917 INFO:tasks.workunit.client.0.smithi042.stdout:0/808: truncate d4/d38/d77/fb4 1524275 0 2024-02-15T18:27:06.917 INFO:tasks.workunit.client.0.smithi042.stdout:2/805: creat d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/d44/f103 x:0 0 0 2024-02-15T18:27:06.918 INFO:tasks.workunit.client.0.smithi042.stdout:2/806: read - d0/d8/d10/d31/da6/fac zero size 2024-02-15T18:27:06.918 INFO:tasks.workunit.client.0.smithi042.stdout:7/842: symlink d10/d43/d57/l118 0 2024-02-15T18:27:06.918 INFO:tasks.workunit.client.1.smithi191.stdout:8/515: dread d3/d2c/d1a/d23/d3e/f68 [0,4194304] 0 2024-02-15T18:27:06.920 INFO:tasks.workunit.client.0.smithi042.stdout:6/914: link d9/d11/d46/d89/dbc/lda d9/d11/d19/d35/d55/d92/d9e/d12e/l142 0 2024-02-15T18:27:06.920 INFO:tasks.workunit.client.0.smithi042.stdout:4/840: symlink da/d3a/dbb/l124 0 2024-02-15T18:27:06.925 INFO:tasks.workunit.client.1.smithi191.stdout:8/516: link d3/d2c/d1a/d23/d2e/d5b/f9c d3/d2c/d1a/d23/d2e/d5b/d69/d6d/fbd 0 2024-02-15T18:27:06.925 INFO:tasks.workunit.client.1.smithi191.stdout:8/517: truncate d3/d2c/d1a/d23/d2e/f9f 317844 0 2024-02-15T18:27:06.926 INFO:tasks.workunit.client.0.smithi042.stdout:1/852: mknod de/d15/c145 0 2024-02-15T18:27:06.927 INFO:tasks.workunit.client.0.smithi042.stdout:5/880: dread d0/ffc [0,4194304] 0 2024-02-15T18:27:06.927 INFO:tasks.workunit.client.0.smithi042.stdout:5/881: write d0/d22/fee [804498,116752] 0 2024-02-15T18:27:06.927 INFO:tasks.workunit.client.0.smithi042.stdout:5/882: stat d0/dda/dc7/f6d 0 2024-02-15T18:27:06.928 INFO:tasks.workunit.client.0.smithi042.stdout:9/842: mknod d6/dd/d9/da1/c121 0 2024-02-15T18:27:06.928 INFO:tasks.workunit.client.0.smithi042.stdout:9/843: getdents d6/da/d2f/d6c/dd8/d117 0 2024-02-15T18:27:06.930 INFO:tasks.workunit.client.0.smithi042.stdout:2/807: creat d0/d8/d20/d3f/dd6/f104 x:0 0 0 2024-02-15T18:27:06.930 INFO:tasks.workunit.client.0.smithi042.stdout:7/843: symlink d10/d49/d6d/db6/l119 0 2024-02-15T18:27:06.935 INFO:tasks.workunit.client.0.smithi042.stdout:6/915: truncate d9/d11/d19/d35/d55/d92/f95 2022427 0 2024-02-15T18:27:06.937 INFO:tasks.workunit.client.0.smithi042.stdout:6/916: write d9/d11/d19/f2a [1749757,69528] 0 2024-02-15T18:27:06.937 INFO:tasks.workunit.client.0.smithi042.stdout:6/917: read d9/d11/d19/d1c/d31/d4f/f68 [9334,20138] 0 2024-02-15T18:27:06.937 INFO:tasks.workunit.client.0.smithi042.stdout:4/841: rename da/d11/d29/d4a/d4f/d80 to da/d11/d85/db5/d103/d66/d125 0 2024-02-15T18:27:06.939 INFO:tasks.workunit.client.0.smithi042.stdout:3/825: mknod d1/d11/d14/c110 0 2024-02-15T18:27:06.939 INFO:tasks.workunit.client.0.smithi042.stdout:3/826: chown d1/d11/f65 2849919 1 2024-02-15T18:27:06.939 INFO:tasks.workunit.client.0.smithi042.stdout:3/827: fsync d1/de/d1a/f2f 0 2024-02-15T18:27:06.939 INFO:tasks.workunit.client.0.smithi042.stdout:0/809: dwrite d4/d9/d34/dab/dc8/dfe/ff1 [0,4194304] 0 2024-02-15T18:27:06.940 INFO:tasks.workunit.client.0.smithi042.stdout:1/853: mknod de/d15/d2c/d136/c146 0 2024-02-15T18:27:06.941 INFO:tasks.workunit.client.0.smithi042.stdout:5/883: unlink d0/d14/d88/da1/f116 0 2024-02-15T18:27:06.943 INFO:tasks.workunit.client.1.smithi191.stdout:8/518: dwrite d3/d2c/d1a/d23/f4c [0,4194304] 0 2024-02-15T18:27:06.943 INFO:tasks.workunit.client.0.smithi042.stdout:9/844: symlink d6/da/d2f/d55/d73/d8d/dcf/l122 0 2024-02-15T18:27:06.945 INFO:tasks.workunit.client.0.smithi042.stdout:2/808: getdents d0/d8/d10/d5a 0 2024-02-15T18:27:06.945 INFO:tasks.workunit.client.0.smithi042.stdout:2/809: fdatasync d0/d8/d20/d7d/d7e/fe0 0 2024-02-15T18:27:06.945 INFO:tasks.workunit.client.0.smithi042.stdout:2/810: write d0/d8/d10/fa2 [187745,46284] 0 2024-02-15T18:27:06.945 INFO:tasks.workunit.client.0.smithi042.stdout:2/811: chown d0/d8/d20/d41 2 1 2024-02-15T18:27:06.946 INFO:tasks.workunit.client.0.smithi042.stdout:7/844: symlink d10/d49/d6d/d78/d83/l11a 0 2024-02-15T18:27:06.946 INFO:tasks.workunit.client.0.smithi042.stdout:7/845: chown d10/d43/d57/f66 28522 1 2024-02-15T18:27:06.948 INFO:tasks.workunit.client.0.smithi042.stdout:6/918: mkdir d9/d11/d46/df8/d125/d143 0 2024-02-15T18:27:06.950 INFO:tasks.workunit.client.1.smithi191.stdout:8/519: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/cbe 0 2024-02-15T18:27:06.950 INFO:tasks.workunit.client.1.smithi191.stdout:8/520: chown d3/fa 509411515 1 2024-02-15T18:27:06.952 INFO:tasks.workunit.client.0.smithi042.stdout:4/842: unlink da/d11/da4/fb4 0 2024-02-15T18:27:06.960 INFO:tasks.workunit.client.1.smithi191.stdout:8/521: truncate d3/d2c/d1a/d23/d3e/fa9 2765523 0 2024-02-15T18:27:06.960 INFO:tasks.workunit.client.1.smithi191.stdout:8/522: readlink d3/d2c/d1a/d23/l66 0 2024-02-15T18:27:06.963 INFO:tasks.workunit.client.0.smithi042.stdout:0/810: symlink d4/d9/d34/dab/db0/l109 0 2024-02-15T18:27:06.963 INFO:tasks.workunit.client.0.smithi042.stdout:0/811: fdatasync d4/d38/d5c/fc6 0 2024-02-15T18:27:06.963 INFO:tasks.workunit.client.1.smithi191.stdout:8/523: rmdir d3/d2c/d1a/d23/d2a 39 2024-02-15T18:27:06.964 INFO:tasks.workunit.client.0.smithi042.stdout:1/854: creat de/d54/dbf/f147 x:0 0 0 2024-02-15T18:27:06.964 INFO:tasks.workunit.client.0.smithi042.stdout:1/855: truncate de/f58 1613743 0 2024-02-15T18:27:06.966 INFO:tasks.workunit.client.1.smithi191.stdout:8/524: creat d3/d2c/d1a/d23/d2e/d47/fbf x:0 0 0 2024-02-15T18:27:06.966 INFO:tasks.workunit.client.1.smithi191.stdout:8/525: write d3/faa [395770,122713] 0 2024-02-15T18:27:06.967 INFO:tasks.workunit.client.0.smithi042.stdout:2/812: symlink d0/d8/d20/d3f/d7c/d102/l105 0 2024-02-15T18:27:06.967 INFO:tasks.workunit.client.0.smithi042.stdout:2/813: chown d0/d8/d20/d7d/l89 53950 1 2024-02-15T18:27:06.968 INFO:tasks.workunit.client.0.smithi042.stdout:4/843: mknod da/d11/d85/db5/d103/d66/c126 0 2024-02-15T18:27:06.968 INFO:tasks.workunit.client.0.smithi042.stdout:4/844: chown da/d11/d27/ca5 481 1 2024-02-15T18:27:06.968 INFO:tasks.workunit.client.0.smithi042.stdout:2/814: write d0/d8/d10/d7b/faa [3965987,50985] 0 2024-02-15T18:27:06.968 INFO:tasks.workunit.client.0.smithi042.stdout:2/815: write d0/d8/d20/d41/dbe/fce [922224,112141] 0 2024-02-15T18:27:06.968 INFO:tasks.workunit.client.0.smithi042.stdout:2/816: write d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/fb4 [838866,14858] 0 2024-02-15T18:27:06.969 INFO:tasks.workunit.client.1.smithi191.stdout:8/526: creat d3/d2c/d1a/d23/d2e/d64/dbc/fc0 x:0 0 0 2024-02-15T18:27:06.972 INFO:tasks.workunit.client.0.smithi042.stdout:0/812: mknod d4/d9/d5a/dfc/d92/c10a 0 2024-02-15T18:27:06.973 INFO:tasks.workunit.client.0.smithi042.stdout:1/856: truncate de/d15/d1b/fc7 1092912 0 2024-02-15T18:27:06.975 INFO:tasks.workunit.client.1.smithi191.stdout:8/527: dread d3/d2c/d1a/f1b [0,4194304] 0 2024-02-15T18:27:06.975 INFO:tasks.workunit.client.1.smithi191.stdout:8/528: chown d3/d2c/d1a/d23/d2e/d5b/fa1 13 1 2024-02-15T18:27:06.975 INFO:tasks.workunit.client.0.smithi042.stdout:0/813: rmdir d4/d9/d34/d8b 39 2024-02-15T18:27:06.980 INFO:tasks.workunit.client.0.smithi042.stdout:1/857: creat de/d15/d67/d89/d10d/f148 x:0 0 0 2024-02-15T18:27:06.982 INFO:tasks.workunit.client.0.smithi042.stdout:1/858: mknod de/d54/dbf/dd0/d7c/dd6/de8/d103/d10f/c149 0 2024-02-15T18:27:06.982 INFO:tasks.workunit.client.0.smithi042.stdout:1/859: write de/d15/d67/d89/d10d/f148 [526869,128069] 0 2024-02-15T18:27:06.984 INFO:tasks.workunit.client.0.smithi042.stdout:9/845: dwrite f3 [0,4194304] 0 2024-02-15T18:27:06.985 INFO:tasks.workunit.client.0.smithi042.stdout:6/919: dwrite d9/d11/d19/d1c/d31/f5c [0,4194304] 0 2024-02-15T18:27:06.985 INFO:tasks.workunit.client.0.smithi042.stdout:6/920: fsync d9/f2d 0 2024-02-15T18:27:06.985 INFO:tasks.workunit.client.0.smithi042.stdout:6/921: fdatasync d9/d11/d19/d35/dc4/fe3 0 2024-02-15T18:27:06.986 INFO:tasks.workunit.client.0.smithi042.stdout:7/846: dwrite d10/d49/d6d/f93 [4194304,4194304] 0 2024-02-15T18:27:06.987 INFO:tasks.workunit.client.0.smithi042.stdout:4/845: getdents da/d11/da4 0 2024-02-15T18:27:06.988 INFO:tasks.workunit.client.0.smithi042.stdout:4/846: write da/d11/d67/f6f [597018,80980] 0 2024-02-15T18:27:06.988 INFO:tasks.workunit.client.0.smithi042.stdout:4/847: fdatasync da/d3a/dc9/fd2 0 2024-02-15T18:27:06.989 INFO:tasks.workunit.client.0.smithi042.stdout:6/922: mknod d9/d11/d19/d35/d55/d92/dfc/c144 0 2024-02-15T18:27:06.992 INFO:tasks.workunit.client.0.smithi042.stdout:9/846: write d6/dd/f2b [2634694,38293] 0 2024-02-15T18:27:06.993 INFO:tasks.workunit.client.0.smithi042.stdout:9/847: dread - d6/da/d2f/d55/d73/d8d/dcf/f113 zero size 2024-02-15T18:27:06.993 INFO:tasks.workunit.client.0.smithi042.stdout:9/848: readlink d6/da/d2f/d6c/l6f 0 2024-02-15T18:27:06.996 INFO:tasks.workunit.client.0.smithi042.stdout:0/814: dread d4/d9/d34/f40 [0,4194304] 0 2024-02-15T18:27:06.997 INFO:tasks.workunit.client.0.smithi042.stdout:0/815: read d4/d9/d62/fa0 [806393,124611] 0 2024-02-15T18:27:06.997 INFO:tasks.workunit.client.0.smithi042.stdout:0/816: write d4/d9/d34/dab/db0/fed [539282,7156] 0 2024-02-15T18:27:07.011 INFO:tasks.workunit.client.0.smithi042.stdout:1/860: dwrite de/d54/dbf/dd0/d7c/dd6/ddf/d88/fff [0,4194304] 0 2024-02-15T18:27:07.012 INFO:tasks.workunit.client.0.smithi042.stdout:0/817: dwrite d4/d38/d77/d2e/f32 [0,4194304] 0 2024-02-15T18:27:07.016 INFO:tasks.workunit.client.0.smithi042.stdout:0/818: write d4/d9/d5a/dfc/f21 [2236882,126604] 0 2024-02-15T18:27:07.016 INFO:tasks.workunit.client.0.smithi042.stdout:0/819: chown d4/d38/d39/d8e/cf3 73112 1 2024-02-15T18:27:07.016 INFO:tasks.workunit.client.0.smithi042.stdout:0/820: write d4/d9/d34/dab/dc8/d6c/d70/faa [5182997,45815] 0 2024-02-15T18:27:07.016 INFO:tasks.workunit.client.0.smithi042.stdout:0/821: fsync d4/d9/d5a/dfc/ffd 0 2024-02-15T18:27:07.018 INFO:tasks.workunit.client.0.smithi042.stdout:9/849: dwrite d6/dd/d9/fc3 [0,4194304] 0 2024-02-15T18:27:07.018 INFO:tasks.workunit.client.0.smithi042.stdout:9/850: fsync d6/da/d2f/d55/f6d 0 2024-02-15T18:27:07.018 INFO:tasks.workunit.client.0.smithi042.stdout:9/851: chown d6/dd/d9/c4d 344 1 2024-02-15T18:27:07.018 INFO:tasks.workunit.client.0.smithi042.stdout:9/852: stat d6/da/d1d/d2d/d40/d95/fdd 0 2024-02-15T18:27:07.018 INFO:tasks.workunit.client.0.smithi042.stdout:9/853: chown d6/fa8 17 1 2024-02-15T18:27:07.025 INFO:tasks.workunit.client.1.smithi191.stdout:8/529: rmdir d3/d2c/d1a 39 2024-02-15T18:27:07.034 INFO:tasks.workunit.client.0.smithi042.stdout:7/847: creat d10/d15/d39/dd9/dec/f11b x:0 0 0 2024-02-15T18:27:07.034 INFO:tasks.workunit.client.0.smithi042.stdout:7/848: readlink d10/d15/d39/l8a 0 2024-02-15T18:27:07.038 INFO:tasks.workunit.client.0.smithi042.stdout:4/848: creat da/d2b/f127 x:0 0 0 2024-02-15T18:27:07.039 INFO:tasks.workunit.client.0.smithi042.stdout:6/923: rename d9/d11/d46/d89/dbc/f131 to d9/d11/d19/d1c/d31/f145 0 2024-02-15T18:27:07.041 INFO:tasks.workunit.client.0.smithi042.stdout:1/861: creat de/d54/dbf/f14a x:0 0 0 2024-02-15T18:27:07.041 INFO:tasks.workunit.client.0.smithi042.stdout:1/862: write de/d15/d67/d74/d7e/fd5 [603450,6272] 0 2024-02-15T18:27:07.041 INFO:tasks.workunit.client.0.smithi042.stdout:1/863: chown de/d15/lc1 0 1 2024-02-15T18:27:07.045 INFO:tasks.workunit.client.0.smithi042.stdout:0/822: mknod d4/d38/d77/c10b 0 2024-02-15T18:27:07.047 INFO:tasks.workunit.client.0.smithi042.stdout:9/854: mknod d6/da/d1d/d2d/d40/d5f/c123 0 2024-02-15T18:27:07.048 INFO:tasks.workunit.client.0.smithi042.stdout:7/849: creat d10/d49/dd2/f11c x:0 0 0 2024-02-15T18:27:07.050 INFO:tasks.workunit.client.0.smithi042.stdout:6/924: mkdir d9/d11/d46/d89/df4/d146 0 2024-02-15T18:27:07.050 INFO:tasks.workunit.client.0.smithi042.stdout:7/850: write d10/d15/d39/dd9/fdf [3837811,60311] 0 2024-02-15T18:27:07.050 INFO:tasks.workunit.client.0.smithi042.stdout:7/851: chown d10/d49/d6d/fd7 0 1 2024-02-15T18:27:07.050 INFO:tasks.workunit.client.0.smithi042.stdout:7/852: fdatasync d10/d15/d39/d6c/f9a 0 2024-02-15T18:27:07.050 INFO:tasks.workunit.client.1.smithi191.stdout:8/530: dwrite d3/d2c/d1a/f20 [0,4194304] 0 2024-02-15T18:27:07.051 INFO:tasks.workunit.client.1.smithi191.stdout:8/531: stat d3/d2c/d1a/d36/f6b 0 2024-02-15T18:27:07.052 INFO:tasks.workunit.client.0.smithi042.stdout:9/855: rmdir d6/da/d2f/d7c 39 2024-02-15T18:27:07.054 INFO:tasks.workunit.client.1.smithi191.stdout:8/532: rename d3/d2c/d1a/d36/f65 to d3/d2c/d1a/d23/d2e/d5b/d69/d6d/fc1 0 2024-02-15T18:27:07.054 INFO:tasks.workunit.client.1.smithi191.stdout:8/533: fdatasync d3/d2c/d1a/d36/d9b/faf 0 2024-02-15T18:27:07.060 INFO:tasks.workunit.client.0.smithi042.stdout:7/853: link d10/d49/d6d/d78/d83/fe8 d10/d15/d108/f11d 0 2024-02-15T18:27:07.060 INFO:tasks.workunit.client.0.smithi042.stdout:7/854: chown c9 771237 1 2024-02-15T18:27:07.061 INFO:tasks.workunit.client.0.smithi042.stdout:1/864: symlink de/d15/d1b/l14b 0 2024-02-15T18:27:07.061 INFO:tasks.workunit.client.0.smithi042.stdout:1/865: dread - de/d6d/d9e/dcd/f117 zero size 2024-02-15T18:27:07.061 INFO:tasks.workunit.client.0.smithi042.stdout:1/866: read de/d15/d67/d89/d10d/f148 [355378,86183] 0 2024-02-15T18:27:07.063 INFO:tasks.workunit.client.0.smithi042.stdout:9/856: unlink d6/dd/d9/d59/d5c/fb7 0 2024-02-15T18:27:07.064 INFO:tasks.workunit.client.0.smithi042.stdout:7/855: mkdir d10/d15/d39/de3/d116/d11e 0 2024-02-15T18:27:07.065 INFO:tasks.workunit.client.0.smithi042.stdout:1/867: mknod de/d54/c14c 0 2024-02-15T18:27:07.066 INFO:tasks.workunit.client.0.smithi042.stdout:6/925: dread d9/d11/d1d/f90 [0,4194304] 0 2024-02-15T18:27:07.066 INFO:tasks.workunit.client.0.smithi042.stdout:9/857: symlink d6/da/d1d/d2d/d40/d48/l124 0 2024-02-15T18:27:07.067 INFO:tasks.workunit.client.0.smithi042.stdout:0/823: dwrite d4/d38/d77/f7c [0,4194304] 0 2024-02-15T18:27:07.067 INFO:tasks.workunit.client.0.smithi042.stdout:7/856: mkdir d10/d15/d39/d6c/dba/d117/d11f 0 2024-02-15T18:27:07.067 INFO:tasks.workunit.client.0.smithi042.stdout:4/849: dwrite da/d3b/d41/d8d/f10a [0,4194304] 0 2024-02-15T18:27:07.069 INFO:tasks.workunit.client.0.smithi042.stdout:1/868: getdents de/d15/d2c/d136 0 2024-02-15T18:27:07.071 INFO:tasks.workunit.client.1.smithi191.stdout:6/372: sync 2024-02-15T18:27:07.071 INFO:tasks.workunit.client.1.smithi191.stdout:1/519: sync 2024-02-15T18:27:07.071 INFO:tasks.workunit.client.1.smithi191.stdout:1/520: write d0/fa6 [74561,58488] 0 2024-02-15T18:27:07.072 INFO:tasks.workunit.client.0.smithi042.stdout:6/926: rename d9/d11/d19/d1c/d31/f5c to d9/d11/d19/d35/d55/d92/d9e/d12e/f147 0 2024-02-15T18:27:07.073 INFO:tasks.workunit.client.1.smithi191.stdout:1/521: dread d0/d8/f30 [0,4194304] 0 2024-02-15T18:27:07.073 INFO:tasks.workunit.client.1.smithi191.stdout:1/522: dread - d0/d8/df/d12/d2a/fb0 zero size 2024-02-15T18:27:07.073 INFO:tasks.workunit.client.1.smithi191.stdout:1/523: write d0/d8/df/d2e/f3d [1234352,64799] 0 2024-02-15T18:27:07.073 INFO:tasks.workunit.client.1.smithi191.stdout:1/524: truncate d0/d2f/d51/d54/f8b 4758328 0 2024-02-15T18:27:07.073 INFO:tasks.workunit.client.0.smithi042.stdout:9/858: mknod d6/dd/d9/c125 0 2024-02-15T18:27:07.076 INFO:tasks.workunit.client.1.smithi191.stdout:1/525: mknod d0/d8/cb1 0 2024-02-15T18:27:07.079 INFO:tasks.workunit.client.0.smithi042.stdout:0/824: getdents d4/d9/d34/dab/dc8/d6c/d70 0 2024-02-15T18:27:07.079 INFO:tasks.workunit.client.1.smithi191.stdout:8/534: dwrite d3/d2c/d1a/f8f [0,4194304] 0 2024-02-15T18:27:07.079 INFO:tasks.workunit.client.1.smithi191.stdout:6/373: dread d8/d19/f1d [0,4194304] 0 2024-02-15T18:27:07.080 INFO:tasks.workunit.client.0.smithi042.stdout:7/857: creat d10/d15/d8f/da5/dad/dda/f120 x:0 0 0 2024-02-15T18:27:07.080 INFO:tasks.workunit.client.0.smithi042.stdout:7/858: write d10/d49/d6d/fbb [32074,26310] 0 2024-02-15T18:27:07.081 INFO:tasks.workunit.client.0.smithi042.stdout:1/869: creat de/d54/d9a/dc8/dd3/f14d x:0 0 0 2024-02-15T18:27:07.083 INFO:tasks.workunit.client.0.smithi042.stdout:6/927: write d9/d11/d19/d35/d55/fac [3240325,115803] 0 2024-02-15T18:27:07.084 INFO:tasks.workunit.client.0.smithi042.stdout:6/928: write d9/f40 [2154538,46428] 0 2024-02-15T18:27:07.084 INFO:tasks.workunit.client.0.smithi042.stdout:6/929: readlink d9/d11/d46/dc3/lfd 0 2024-02-15T18:27:07.084 INFO:tasks.workunit.client.0.smithi042.stdout:6/930: chown d9/d3e/f3b 415356 1 2024-02-15T18:27:07.085 INFO:tasks.workunit.client.0.smithi042.stdout:9/859: unlink d6/da/d2f/d7c/ff4 0 2024-02-15T18:27:07.087 INFO:tasks.workunit.client.0.smithi042.stdout:0/825: mkdir d4/d9/d62/d10c 0 2024-02-15T18:27:07.087 INFO:tasks.workunit.client.0.smithi042.stdout:0/826: write d4/d9/d34/dab/dc8/d6c/d70/fad [375069,60985] 0 2024-02-15T18:27:07.088 INFO:tasks.workunit.client.0.smithi042.stdout:7/859: creat d10/d49/f121 x:0 0 0 2024-02-15T18:27:07.090 INFO:tasks.workunit.client.0.smithi042.stdout:1/870: unlink de/d15/d67/d8e/fef 0 2024-02-15T18:27:07.091 INFO:tasks.workunit.client.1.smithi191.stdout:1/526: dread d0/d2f/d51/fe [0,4194304] 0 2024-02-15T18:27:07.094 INFO:tasks.workunit.client.1.smithi191.stdout:6/374: unlink d8/d9/l5c 0 2024-02-15T18:27:07.094 INFO:tasks.workunit.client.1.smithi191.stdout:6/375: fsync d8/d9/d4a/d4c/f5d 0 2024-02-15T18:27:07.096 INFO:tasks.workunit.client.0.smithi042.stdout:6/931: creat d9/d11/d19/d35/d49/d10a/f148 x:0 0 0 2024-02-15T18:27:07.097 INFO:tasks.workunit.client.1.smithi191.stdout:1/527: unlink d0/d8/df/d12/d2a/c82 0 2024-02-15T18:27:07.098 INFO:tasks.workunit.client.0.smithi042.stdout:0/827: creat d4/d38/d77/d2e/dda/d101/f10d x:0 0 0 2024-02-15T18:27:07.098 INFO:tasks.workunit.client.1.smithi191.stdout:6/376: fdatasync d8/d9/d2b/f42 0 2024-02-15T18:27:07.100 INFO:tasks.workunit.client.0.smithi042.stdout:1/871: mkdir de/d15/d67/d46/d14e 0 2024-02-15T18:27:07.100 INFO:tasks.workunit.client.0.smithi042.stdout:0/828: dread d4/d38/d5c/f61 [0,4194304] 0 2024-02-15T18:27:07.100 INFO:tasks.workunit.client.0.smithi042.stdout:0/829: fdatasync d4/d9/d62/d74/fc7 0 2024-02-15T18:27:07.101 INFO:tasks.workunit.client.1.smithi191.stdout:1/528: unlink d0/d8/df/d2e/f3d 0 2024-02-15T18:27:07.103 INFO:tasks.workunit.client.0.smithi042.stdout:6/932: rename d9/d11/d19/d35/d49/f52 to d9/d11/d19/d35/d55/f149 0 2024-02-15T18:27:07.103 INFO:tasks.workunit.client.0.smithi042.stdout:6/933: truncate d9/d11/d19/d35/f73 1196074 0 2024-02-15T18:27:07.103 INFO:tasks.workunit.client.0.smithi042.stdout:6/934: chown d9/d3e/dd9/c120 36 1 2024-02-15T18:27:07.103 INFO:tasks.workunit.client.0.smithi042.stdout:7/860: dread d10/d49/d6d/f86 [0,4194304] 0 2024-02-15T18:27:07.104 INFO:tasks.workunit.client.0.smithi042.stdout:7/861: dread - d10/d15/d39/d6c/dba/f114 zero size 2024-02-15T18:27:07.105 INFO:tasks.workunit.client.1.smithi191.stdout:6/377: rename f1 to d8/d9/d61/f6a 0 2024-02-15T18:27:07.105 INFO:tasks.workunit.client.1.smithi191.stdout:1/529: dread d0/d2f/d51/d10/f25 [0,4194304] 0 2024-02-15T18:27:07.108 INFO:tasks.workunit.client.0.smithi042.stdout:6/935: rename d9/d11/d19/d35/d49/fea to d9/d11/d97/f14a 0 2024-02-15T18:27:07.110 INFO:tasks.workunit.client.0.smithi042.stdout:4/850: dwrite da/f63 [0,4194304] 0 2024-02-15T18:27:07.111 INFO:tasks.workunit.client.1.smithi191.stdout:8/535: dwrite d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/f60 [0,4194304] 0 2024-02-15T18:27:07.111 INFO:tasks.workunit.client.1.smithi191.stdout:1/530: rename d0/d8/d48/fac to d0/d8/df/fb2 0 2024-02-15T18:27:07.111 INFO:tasks.workunit.client.1.smithi191.stdout:1/531: chown d0/d2f/d66 29082329 1 2024-02-15T18:27:07.111 INFO:tasks.workunit.client.0.smithi042.stdout:9/860: dwrite d6/dd/d9/f41 [4194304,4194304] 0 2024-02-15T18:27:07.112 INFO:tasks.workunit.client.1.smithi191.stdout:1/532: read d0/d2f/d51/d10/f59 [195031,115558] 0 2024-02-15T18:27:07.115 INFO:tasks.workunit.client.0.smithi042.stdout:7/862: mknod d10/d15/c122 0 2024-02-15T18:27:07.116 INFO:tasks.workunit.client.0.smithi042.stdout:1/872: dread de/d15/d2c/d51/f55 [0,4194304] 0 2024-02-15T18:27:07.119 INFO:tasks.workunit.client.1.smithi191.stdout:1/533: dread d0/d8/df/d12/f3f [0,4194304] 0 2024-02-15T18:27:07.120 INFO:tasks.workunit.client.1.smithi191.stdout:1/534: stat d0/d2f/d51/d5e 0 2024-02-15T18:27:07.122 INFO:tasks.workunit.client.0.smithi042.stdout:0/830: dread d4/d9/d34/f4b [4194304,4194304] 0 2024-02-15T18:27:07.123 INFO:tasks.workunit.client.0.smithi042.stdout:8/739: sync 2024-02-15T18:27:07.123 INFO:tasks.workunit.client.0.smithi042.stdout:3/828: sync 2024-02-15T18:27:07.123 INFO:tasks.workunit.client.0.smithi042.stdout:3/829: dread - d1/d11/f103 zero size 2024-02-15T18:27:07.123 INFO:tasks.workunit.client.0.smithi042.stdout:8/740: fsync dd/de/da4/d67/d5a/f93 0 2024-02-15T18:27:07.129 INFO:tasks.workunit.client.1.smithi191.stdout:6/378: dwrite d8/d9/f29 [0,4194304] 0 2024-02-15T18:27:07.129 INFO:tasks.workunit.client.1.smithi191.stdout:6/379: dread - d8/d9/d2b/d57/f60 zero size 2024-02-15T18:27:07.129 INFO:tasks.workunit.client.0.smithi042.stdout:6/936: rename d9/d11/d19/d1c/lce to d9/d11/d46/df8/d125/l14b 0 2024-02-15T18:27:07.131 INFO:tasks.workunit.client.0.smithi042.stdout:9/861: dwrite f2 [0,4194304] 0 2024-02-15T18:27:07.132 INFO:tasks.workunit.client.1.smithi191.stdout:8/536: mknod d3/d2c/d1a/d23/d2a/cc2 0 2024-02-15T18:27:07.135 INFO:tasks.workunit.client.1.smithi191.stdout:1/535: unlink d0/d8/df/d12/d2a/d2b/d78/d60/d74/fa9 0 2024-02-15T18:27:07.136 INFO:tasks.workunit.client.1.smithi191.stdout:1/536: write d0/d8/df/d41/f36 [2107291,11030] 0 2024-02-15T18:27:07.136 INFO:tasks.workunit.client.1.smithi191.stdout:6/380: mkdir d8/d19/d6b 0 2024-02-15T18:27:07.136 INFO:tasks.workunit.client.1.smithi191.stdout:6/381: stat d8/d9/f46 0 2024-02-15T18:27:07.136 INFO:tasks.workunit.client.1.smithi191.stdout:6/382: fdatasync d8/d9/d4a/d4c/f64 0 2024-02-15T18:27:07.137 INFO:tasks.workunit.client.1.smithi191.stdout:8/537: unlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/fa4 0 2024-02-15T18:27:07.141 INFO:tasks.workunit.client.1.smithi191.stdout:6/383: link d8/d19/f35 d8/d9/d2b/d57/f6c 0 2024-02-15T18:27:07.141 INFO:tasks.workunit.client.1.smithi191.stdout:6/384: stat d8/d9/c47 0 2024-02-15T18:27:07.143 INFO:tasks.workunit.client.1.smithi191.stdout:8/538: symlink d3/d5e/lc3 0 2024-02-15T18:27:07.143 INFO:tasks.workunit.client.1.smithi191.stdout:1/537: dread d0/d8/df/d2e/f4f [0,4194304] 0 2024-02-15T18:27:07.144 INFO:tasks.workunit.client.0.smithi042.stdout:4/851: creat da/d3a/dc9/f128 x:0 0 0 2024-02-15T18:27:07.144 INFO:tasks.workunit.client.0.smithi042.stdout:4/852: chown da/d2b/lb3 97100 1 2024-02-15T18:27:07.147 INFO:tasks.workunit.client.0.smithi042.stdout:7/863: mknod d10/d15/d39/d6c/d92/c123 0 2024-02-15T18:27:07.153 INFO:tasks.workunit.client.1.smithi191.stdout:6/385: dread d8/d9/d61/f6a [0,4194304] 0 2024-02-15T18:27:07.153 INFO:tasks.workunit.client.1.smithi191.stdout:8/539: dread d3/d2c/f17 [0,4194304] 0 2024-02-15T18:27:07.153 INFO:tasks.workunit.client.1.smithi191.stdout:1/538: creat d0/d8/df/d41/fb3 x:0 0 0 2024-02-15T18:27:07.154 INFO:tasks.workunit.client.0.smithi042.stdout:1/873: mkdir de/d54/dbf/dd0/d7c/dd6/de8/d103/d14f 0 2024-02-15T18:27:07.154 INFO:tasks.workunit.client.0.smithi042.stdout:2/817: sync 2024-02-15T18:27:07.154 INFO:tasks.workunit.client.0.smithi042.stdout:5/884: sync 2024-02-15T18:27:07.154 INFO:tasks.workunit.client.0.smithi042.stdout:5/885: fdatasync d0/dda/d42/fb9 0 2024-02-15T18:27:07.154 INFO:tasks.workunit.client.0.smithi042.stdout:5/886: getdents d0/d38/d53/d91/d94/d107 0 2024-02-15T18:27:07.156 INFO:tasks.workunit.client.1.smithi191.stdout:6/386: mknod d8/c6d 0 2024-02-15T18:27:07.160 INFO:tasks.workunit.client.0.smithi042.stdout:0/831: mknod d4/d9/d34/d57/d88/c10e 0 2024-02-15T18:27:07.161 INFO:tasks.workunit.client.1.smithi191.stdout:1/539: creat d0/d2f/d51/d10/fb4 x:0 0 0 2024-02-15T18:27:07.161 INFO:tasks.workunit.client.1.smithi191.stdout:1/540: dread - d0/d8/df/d12/d2a/d2b/f7a zero size 2024-02-15T18:27:07.163 INFO:tasks.workunit.client.1.smithi191.stdout:1/541: write d0/d8/f35 [335780,1516] 0 2024-02-15T18:27:07.163 INFO:tasks.workunit.client.1.smithi191.stdout:1/542: readlink d0/d8/df/d41/l3e 0 2024-02-15T18:27:07.163 INFO:tasks.workunit.client.1.smithi191.stdout:1/543: chown d0/d2f/d77 1653 1 2024-02-15T18:27:07.163 INFO:tasks.workunit.client.0.smithi042.stdout:8/741: creat dd/de/da5/fd8 x:0 0 0 2024-02-15T18:27:07.163 INFO:tasks.workunit.client.0.smithi042.stdout:8/742: write dd/d11/d75/f85 [4759729,62081] 0 2024-02-15T18:27:07.165 INFO:tasks.workunit.client.1.smithi191.stdout:6/387: rmdir d8/d9/d2b/d57 39 2024-02-15T18:27:07.167 INFO:tasks.workunit.client.1.smithi191.stdout:8/540: dread d3/d2c/d1a/d23/f4c [0,4194304] 0 2024-02-15T18:27:07.167 INFO:tasks.workunit.client.1.smithi191.stdout:8/541: chown d3/d5e/c94 1756420073 1 2024-02-15T18:27:07.167 INFO:tasks.workunit.client.1.smithi191.stdout:8/542: stat d3/d2c/d1a/la6 0 2024-02-15T18:27:07.167 INFO:tasks.workunit.client.1.smithi191.stdout:8/543: write d3/d5e/f79 [1797278,108686] 0 2024-02-15T18:27:07.167 INFO:tasks.workunit.client.0.smithi042.stdout:3/830: creat d1/d11/d14/dbf/f111 x:0 0 0 2024-02-15T18:27:07.169 INFO:tasks.workunit.client.0.smithi042.stdout:6/937: symlink d9/d3e/dd9/l14c 0 2024-02-15T18:27:07.172 INFO:tasks.workunit.client.0.smithi042.stdout:9/862: mkdir d6/da/d2f/d6c/dd8/d117/d126 0 2024-02-15T18:27:07.172 INFO:tasks.workunit.client.0.smithi042.stdout:6/938: write d9/d11/d19/d35/d55/d92/d9e/d12e/f147 [1790118,96575] 0 2024-02-15T18:27:07.172 INFO:tasks.workunit.client.1.smithi191.stdout:1/544: mknod d0/d8/df/cb5 0 2024-02-15T18:27:07.175 INFO:tasks.workunit.client.0.smithi042.stdout:4/853: creat da/d11/d85/db5/d103/d66/d78/d53/f129 x:0 0 0 2024-02-15T18:27:07.176 INFO:tasks.workunit.client.1.smithi191.stdout:6/388: rename d8/d9/d2b/l5f to d8/d19/d6b/l6e 0 2024-02-15T18:27:07.178 INFO:tasks.workunit.client.0.smithi042.stdout:2/818: dwrite d0/d35/d9e/fe9 [0,4194304] 0 2024-02-15T18:27:07.178 INFO:tasks.workunit.client.0.smithi042.stdout:7/864: link d10/d15/fa6 d10/d49/d6d/d6e/f124 0 2024-02-15T18:27:07.181 INFO:tasks.workunit.client.0.smithi042.stdout:1/874: truncate de/d15/d67/d46/f71 2762721 0 2024-02-15T18:27:07.183 INFO:tasks.workunit.client.1.smithi191.stdout:1/545: creat d0/d2f/d51/d54/d7b/fb6 x:0 0 0 2024-02-15T18:27:07.185 INFO:tasks.workunit.client.0.smithi042.stdout:0/832: symlink d4/d9/d34/dec/l10f 0 2024-02-15T18:27:07.185 INFO:tasks.workunit.client.0.smithi042.stdout:5/887: mkdir d0/d39/d135 0 2024-02-15T18:27:07.185 INFO:tasks.workunit.client.0.smithi042.stdout:5/888: write d0/d38/d53/db4/fc5 [1016902,90961] 0 2024-02-15T18:27:07.188 INFO:tasks.workunit.client.0.smithi042.stdout:9/863: dwrite d6/da/d2f/f56 [0,4194304] 0 2024-02-15T18:27:07.189 INFO:tasks.workunit.client.0.smithi042.stdout:3/831: creat d1/d11/d14/dbf/f112 x:0 0 0 2024-02-15T18:27:07.191 INFO:tasks.workunit.client.1.smithi191.stdout:1/546: mkdir d0/d2f/d51/d54/db7 0 2024-02-15T18:27:07.192 INFO:tasks.workunit.client.0.smithi042.stdout:3/832: dread d1/d11/f8f [0,4194304] 0 2024-02-15T18:27:07.192 INFO:tasks.workunit.client.1.smithi191.stdout:6/389: mknod d8/d9/d2b/c6f 0 2024-02-15T18:27:07.193 INFO:tasks.workunit.client.0.smithi042.stdout:8/743: dwrite dd/d11/d75/f85 [0,4194304] 0 2024-02-15T18:27:07.193 INFO:tasks.workunit.client.0.smithi042.stdout:8/744: dread - dd/de/d33/d66/d4e/f87 zero size 2024-02-15T18:27:07.193 INFO:tasks.workunit.client.0.smithi042.stdout:8/745: dread - dd/de/fc4 zero size 2024-02-15T18:27:07.194 INFO:tasks.workunit.client.0.smithi042.stdout:6/939: rename d9/d11/d19/d35/d49/d79/le1 to d9/d11/d19/d1c/d31/dd2/df9/l14d 0 2024-02-15T18:27:07.195 INFO:tasks.workunit.client.1.smithi191.stdout:8/544: dwrite d3/d2c/d1a/d23/d2e/d64/f76 [0,4194304] 0 2024-02-15T18:27:07.195 INFO:tasks.workunit.client.1.smithi191.stdout:6/390: write d8/d19/f1f [771271,112237] 0 2024-02-15T18:27:07.195 INFO:tasks.workunit.client.1.smithi191.stdout:8/545: readlink d3/d2c/d1a/d23/l41 0 2024-02-15T18:27:07.196 INFO:tasks.workunit.client.0.smithi042.stdout:4/854: creat da/d11/d85/db5/d103/d66/d78/d2f/dca/f12a x:0 0 0 2024-02-15T18:27:07.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:27:06.683026+0000 mgr.smithi042.kfoihb (mgr.24435) 42 : cluster [DBG] pgmap v32: 65 pgs: 65 active+clean; 2.1 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 118 MiB/s rd, 261 MiB/s wr, 382 op/s 2024-02-15T18:27:07.201 INFO:tasks.workunit.client.0.smithi042.stdout:7/865: link d10/d49/d6d/f86 d10/d43/f125 0 2024-02-15T18:27:07.203 INFO:tasks.workunit.client.0.smithi042.stdout:5/889: creat d0/dda/d42/d7d/f136 x:0 0 0 2024-02-15T18:27:07.203 INFO:tasks.workunit.client.0.smithi042.stdout:5/890: chown d0/dda/lcb 46 1 2024-02-15T18:27:07.205 INFO:tasks.workunit.client.1.smithi191.stdout:1/547: write d0/f9 [818934,18372] 0 2024-02-15T18:27:07.205 INFO:tasks.workunit.client.1.smithi191.stdout:8/546: write d3/d2c/d1a/d23/d2e/d5b/d69/d6d/fab [2340497,44183] 0 2024-02-15T18:27:07.205 INFO:tasks.workunit.client.0.smithi042.stdout:9/864: mknod d6/da/d2f/d6c/c127 0 2024-02-15T18:27:07.210 INFO:tasks.workunit.client.0.smithi042.stdout:8/746: dread dd/de/da4/d67/f2e [0,4194304] 0 2024-02-15T18:27:07.210 INFO:tasks.workunit.client.0.smithi042.stdout:8/747: chown dd/de/da4/cbc 6585323 1 2024-02-15T18:27:07.211 INFO:tasks.workunit.client.0.smithi042.stdout:3/833: rmdir d1/de 39 2024-02-15T18:27:07.214 INFO:tasks.workunit.client.0.smithi042.stdout:9/865: dread d6/da/d1d/d2d/d40/f49 [0,4194304] 0 2024-02-15T18:27:07.215 INFO:tasks.workunit.client.0.smithi042.stdout:6/940: rename d9/d11/d46/df8/d125 to d9/d11/d19/d1c/d31/d9b/db4/d14e 0 2024-02-15T18:27:07.215 INFO:tasks.workunit.client.0.smithi042.stdout:6/941: chown d9/d11/d19/fed 120 1 2024-02-15T18:27:07.216 INFO:tasks.workunit.client.1.smithi191.stdout:1/548: creat d0/d8/df/d12/d2a/fb8 x:0 0 0 2024-02-15T18:27:07.217 INFO:tasks.workunit.client.1.smithi191.stdout:1/549: fsync d0/d8/df/d12/fa0 0 2024-02-15T18:27:07.217 INFO:tasks.workunit.client.0.smithi042.stdout:4/855: mkdir da/d11/d29/d11e/d12b 0 2024-02-15T18:27:07.218 INFO:tasks.workunit.client.0.smithi042.stdout:1/875: dwrite de/d15/d2c/d51/d70/fe9 [0,4194304] 0 2024-02-15T18:27:07.218 INFO:tasks.workunit.client.0.smithi042.stdout:1/876: chown de/d15/d67/d89/la0 122956442 1 2024-02-15T18:27:07.219 INFO:tasks.workunit.client.1.smithi191.stdout:8/547: creat d3/d2c/d1a/d23/d2e/d64/fc4 x:0 0 0 2024-02-15T18:27:07.219 INFO:tasks.workunit.client.0.smithi042.stdout:6/942: write d9/d11/d19/fed [2014850,49716] 0 2024-02-15T18:27:07.219 INFO:tasks.workunit.client.0.smithi042.stdout:5/891: dwrite d0/dda/dc7/de1/f10 [0,4194304] 0 2024-02-15T18:27:07.220 INFO:tasks.workunit.client.1.smithi191.stdout:7/435: sync 2024-02-15T18:27:07.220 INFO:tasks.workunit.client.1.smithi191.stdout:7/436: fdatasync d0/db/d65/d11/d5a/d66/f79 0 2024-02-15T18:27:07.221 INFO:tasks.workunit.client.0.smithi042.stdout:8/748: creat dd/de/da4/d67/d6f/d78/d9a/dce/fd9 x:0 0 0 2024-02-15T18:27:07.221 INFO:tasks.workunit.client.0.smithi042.stdout:8/749: chown dd/de/da4/d67/d5a 6089 1 2024-02-15T18:27:07.221 INFO:tasks.workunit.client.0.smithi042.stdout:8/750: dread - dd/de/d94/f98 zero size 2024-02-15T18:27:07.222 INFO:tasks.workunit.client.1.smithi191.stdout:1/550: dread d0/d2f/d51/d10/f59 [0,4194304] 0 2024-02-15T18:27:07.222 INFO:tasks.workunit.client.1.smithi191.stdout:1/551: readlink d0/d8/df/d41/l3e 0 2024-02-15T18:27:07.222 INFO:tasks.workunit.client.1.smithi191.stdout:1/552: stat d0/d8/df/d12/d2a/d2b/f91 0 2024-02-15T18:27:07.228 INFO:tasks.workunit.client.0.smithi042.stdout:0/833: dwrite d4/d9/d5a/dfc/f78 [0,4194304] 0 2024-02-15T18:27:07.228 INFO:tasks.workunit.client.1.smithi191.stdout:6/391: dread d8/d19/f1f [0,4194304] 0 2024-02-15T18:27:07.229 INFO:tasks.workunit.client.0.smithi042.stdout:2/819: write d0/d8/f70 [1920678,2396] 0 2024-02-15T18:27:07.232 INFO:tasks.workunit.client.0.smithi042.stdout:3/834: getdents d1/de/d1a/de9 0 2024-02-15T18:27:07.234 INFO:tasks.workunit.client.0.smithi042.stdout:4/856: dread da/d11/d85/db5/d103/d66/d125/fa1 [0,4194304] 0 2024-02-15T18:27:07.234 INFO:tasks.workunit.client.1.smithi191.stdout:8/548: creat d3/d2c/d1a/d23/d2e/d47/d5f/fc5 x:0 0 0 2024-02-15T18:27:07.235 INFO:tasks.workunit.client.1.smithi191.stdout:8/549: dread d3/d2c/f1f [0,4194304] 0 2024-02-15T18:27:07.235 INFO:tasks.workunit.client.1.smithi191.stdout:8/550: write d3/d2c/d1a/d23/d2a/f6e [1006411,100469] 0 2024-02-15T18:27:07.236 INFO:tasks.workunit.client.0.smithi042.stdout:9/866: rmdir d6/da/d7b 39 2024-02-15T18:27:07.239 INFO:tasks.workunit.client.1.smithi191.stdout:7/437: creat d0/db/d65/dc/fa0 x:0 0 0 2024-02-15T18:27:07.239 INFO:tasks.workunit.client.1.smithi191.stdout:6/392: symlink d8/d9/d4a/d4c/l70 0 2024-02-15T18:27:07.240 INFO:tasks.workunit.client.0.smithi042.stdout:1/877: creat de/f150 x:0 0 0 2024-02-15T18:27:07.242 INFO:tasks.workunit.client.1.smithi191.stdout:8/551: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/fc6 x:0 0 0 2024-02-15T18:27:07.243 INFO:tasks.workunit.client.1.smithi191.stdout:1/553: dwrite d0/d8/df/d12/f3f [0,4194304] 0 2024-02-15T18:27:07.243 INFO:tasks.workunit.client.1.smithi191.stdout:1/554: dread - d0/d2f/d77/faf zero size 2024-02-15T18:27:07.244 INFO:tasks.workunit.client.1.smithi191.stdout:7/438: link d0/db/d26/d5f/d77/df/f19 d0/db/d26/d5f/d77/df/d5d/fa1 0 2024-02-15T18:27:07.244 INFO:tasks.workunit.client.0.smithi042.stdout:8/751: creat dd/de/fda x:0 0 0 2024-02-15T18:27:07.245 INFO:tasks.workunit.client.1.smithi191.stdout:8/552: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/fc7 x:0 0 0 2024-02-15T18:27:07.251 INFO:tasks.workunit.client.0.smithi042.stdout:0/834: dread d4/d9/d34/d57/dc1/ff0 [0,4194304] 0 2024-02-15T18:27:07.254 INFO:tasks.workunit.client.0.smithi042.stdout:2/820: creat d0/d8/d20/d7d/d7e/dc7/d8b/f106 x:0 0 0 2024-02-15T18:27:07.258 INFO:tasks.workunit.client.1.smithi191.stdout:6/393: dwrite d8/d9/d4a/d4c/f66 [0,4194304] 0 2024-02-15T18:27:07.258 INFO:tasks.workunit.client.1.smithi191.stdout:6/394: stat d8/d9/d2b/d57/f59 0 2024-02-15T18:27:07.258 INFO:tasks.workunit.client.1.smithi191.stdout:1/555: creat d0/d8/df/d12/d33/d49/fb9 x:0 0 0 2024-02-15T18:27:07.258 INFO:tasks.workunit.client.0.smithi042.stdout:7/866: rename d10/d15/d8f/da5/dad/d104/d23/d51/l97 to d10/d15/d8f/da5/dad/d104/d23/l126 0 2024-02-15T18:27:07.258 INFO:tasks.workunit.client.0.smithi042.stdout:7/867: truncate d10/d49/d6d/d78/fe1 107975 0 2024-02-15T18:27:07.259 INFO:tasks.workunit.client.1.smithi191.stdout:7/439: getdents d0/db/d26 0 2024-02-15T18:27:07.259 INFO:tasks.workunit.client.1.smithi191.stdout:7/440: fdatasync d0/db/d65/d11/d5a/d66/f53 0 2024-02-15T18:27:07.260 INFO:tasks.workunit.client.0.smithi042.stdout:1/878: getdents de/d54/de6 0 2024-02-15T18:27:07.263 INFO:tasks.workunit.client.1.smithi191.stdout:8/553: link d3/d2c/f17 d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/fc8 0 2024-02-15T18:27:07.263 INFO:tasks.workunit.client.1.smithi191.stdout:8/554: dread - d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/fa8 zero size 2024-02-15T18:27:07.263 INFO:tasks.workunit.client.1.smithi191.stdout:8/555: chown d3/d2c/d1a/f20 0 1 2024-02-15T18:27:07.276 INFO:tasks.workunit.client.1.smithi191.stdout:6/395: rename d8/d9/d4a/d4c/c67 to d8/d9/d61/c71 0 2024-02-15T18:27:07.276 INFO:tasks.workunit.client.1.smithi191.stdout:6/396: readlink d8/l3f 0 2024-02-15T18:27:07.276 INFO:tasks.workunit.client.1.smithi191.stdout:6/397: stat d8/d9/d2b/d57/f6c 0 2024-02-15T18:27:07.277 INFO:tasks.workunit.client.1.smithi191.stdout:6/398: write d8/d19/f62 [763505,108400] 0 2024-02-15T18:27:07.278 INFO:tasks.workunit.client.1.smithi191.stdout:7/441: link d0/db/d65/dc/d4b/c7a d0/db/d65/d11/d88/ca2 0 2024-02-15T18:27:07.278 INFO:tasks.workunit.client.1.smithi191.stdout:7/442: stat d0/db/d26/d5f/d77/df/f1a 0 2024-02-15T18:27:07.278 INFO:tasks.workunit.client.1.smithi191.stdout:7/443: chown d0/f99 218815781 1 2024-02-15T18:27:07.279 INFO:tasks.workunit.client.1.smithi191.stdout:6/399: mkdir d8/d19/d6b/d72 0 2024-02-15T18:27:07.280 INFO:tasks.workunit.client.1.smithi191.stdout:6/400: read d8/f15 [1780729,30003] 0 2024-02-15T18:27:07.280 INFO:tasks.workunit.client.1.smithi191.stdout:7/444: getdents d0/db/d26/d5f/d77/d35 0 2024-02-15T18:27:07.280 INFO:tasks.workunit.client.1.smithi191.stdout:6/401: write d8/d9/d2b/d57/f60 [813635,51730] 0 2024-02-15T18:27:07.280 INFO:tasks.workunit.client.1.smithi191.stdout:7/445: write d0/f99 [672735,110155] 0 2024-02-15T18:27:07.283 INFO:tasks.workunit.client.0.smithi042.stdout:0/835: creat d4/d38/d77/d2e/dae/f110 x:0 0 0 2024-02-15T18:27:07.283 INFO:tasks.workunit.client.1.smithi191.stdout:5/219: sync 2024-02-15T18:27:07.284 INFO:tasks.workunit.client.1.smithi191.stdout:6/402: mknod d8/c73 0 2024-02-15T18:27:07.284 INFO:tasks.workunit.client.1.smithi191.stdout:6/403: truncate d8/d9/f58 721427 0 2024-02-15T18:27:07.284 INFO:tasks.workunit.client.1.smithi191.stdout:6/404: write d8/d9/d4a/d4c/f5d [777982,18191] 0 2024-02-15T18:27:07.284 INFO:tasks.workunit.client.1.smithi191.stdout:7/446: mknod d0/db/d65/ca3 0 2024-02-15T18:27:07.285 INFO:tasks.workunit.client.0.smithi042.stdout:6/943: rename d9/d11/d19/d1c/d31/d53/fd7 to d9/d11/d19/d1c/d31/d9b/db4/f14f 0 2024-02-15T18:27:07.285 INFO:tasks.workunit.client.1.smithi191.stdout:6/405: write d8/d19/f35 [83492,40038] 0 2024-02-15T18:27:07.286 INFO:tasks.workunit.client.1.smithi191.stdout:5/220: truncate d0/f1b 10967 0 2024-02-15T18:27:07.286 INFO:tasks.workunit.client.1.smithi191.stdout:5/221: write d0/d23/d41/d28/f3e [358441,1442] 0 2024-02-15T18:27:07.287 INFO:tasks.workunit.client.0.smithi042.stdout:1/879: unlink de/f48 0 2024-02-15T18:27:07.287 INFO:tasks.workunit.client.0.smithi042.stdout:1/880: chown de/d15/d1b/d44/df3 93 1 2024-02-15T18:27:07.287 INFO:tasks.workunit.client.1.smithi191.stdout:7/447: truncate d0/db/d26/d5f/d77/df/d5d/fa1 1188719 0 2024-02-15T18:27:07.290 INFO:tasks.workunit.client.0.smithi042.stdout:3/835: rename d1/de/lc0 to d1/l113 0 2024-02-15T18:27:07.290 INFO:tasks.workunit.client.0.smithi042.stdout:3/836: write d1/d11/d14/d4d/d6c/d7b/f98 [2854003,106512] 0 2024-02-15T18:27:07.290 INFO:tasks.workunit.client.0.smithi042.stdout:3/837: write d1/d11/d14/d4d/d6c/d7b/f98 [3366093,18298] 0 2024-02-15T18:27:07.290 INFO:tasks.workunit.client.1.smithi191.stdout:6/406: truncate d8/d9/f29 8154345 0 2024-02-15T18:27:07.291 INFO:tasks.workunit.client.0.smithi042.stdout:0/836: dread d4/f6 [0,4194304] 0 2024-02-15T18:27:07.291 INFO:tasks.workunit.client.0.smithi042.stdout:1/881: creat de/d54/dbf/dd0/d7c/dd6/f151 x:0 0 0 2024-02-15T18:27:07.292 INFO:tasks.workunit.client.1.smithi191.stdout:5/222: creat d0/d23/d41/d28/f4a x:0 0 0 2024-02-15T18:27:07.294 INFO:tasks.workunit.client.0.smithi042.stdout:4/857: rename da/d11/d29/d56 to da/d11/d29/d11e/d12c 0 2024-02-15T18:27:07.294 INFO:tasks.workunit.client.1.smithi191.stdout:7/448: rmdir d0/db/d26/d5f 39 2024-02-15T18:27:07.295 INFO:tasks.workunit.client.0.smithi042.stdout:3/838: creat d1/d11/d14/df6/f114 x:0 0 0 2024-02-15T18:27:07.296 INFO:tasks.workunit.client.1.smithi191.stdout:6/407: symlink d8/d19/d6b/d72/l74 0 2024-02-15T18:27:07.298 INFO:tasks.workunit.client.0.smithi042.stdout:0/837: mkdir d4/d38/d39/d8e/d111 0 2024-02-15T18:27:07.298 INFO:tasks.workunit.client.1.smithi191.stdout:5/223: chown d0/c3 1 1 2024-02-15T18:27:07.298 INFO:tasks.workunit.client.1.smithi191.stdout:5/224: stat d0/d18/f42 0 2024-02-15T18:27:07.301 INFO:tasks.workunit.client.0.smithi042.stdout:9/867: rename d6/da/d1d/d2d/d40/d95 to d6/dd/d9/d59/dbf/d128 0 2024-02-15T18:27:07.302 INFO:tasks.workunit.client.1.smithi191.stdout:1/556: dwrite d0/d8/df/d12/f28 [0,4194304] 0 2024-02-15T18:27:07.302 INFO:tasks.workunit.client.0.smithi042.stdout:2/821: dwrite d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/d44/f69 [4194304,4194304] 0 2024-02-15T18:27:07.303 INFO:tasks.workunit.client.0.smithi042.stdout:2/822: fsync d0/d35/f95 0 2024-02-15T18:27:07.303 INFO:tasks.workunit.client.0.smithi042.stdout:1/882: dread de/d6d/d9e/df4/f12a [0,4194304] 0 2024-02-15T18:27:07.303 INFO:tasks.workunit.client.0.smithi042.stdout:1/883: fsync de/d15/d2c/f4f 0 2024-02-15T18:27:07.303 INFO:tasks.workunit.client.0.smithi042.stdout:4/858: mkdir da/d11/d29/d4a/d4f/d12d 0 2024-02-15T18:27:07.303 INFO:tasks.workunit.client.1.smithi191.stdout:7/449: symlink d0/db/d26/d5f/d77/la4 0 2024-02-15T18:27:07.305 INFO:tasks.workunit.client.0.smithi042.stdout:0/838: mknod d4/d9/d34/d57/d88/dc9/c112 0 2024-02-15T18:27:07.305 INFO:tasks.workunit.client.0.smithi042.stdout:9/868: dread d6/dd/d9/d59/dbf/d128/fdf [0,4194304] 0 2024-02-15T18:27:07.305 INFO:tasks.workunit.client.0.smithi042.stdout:9/869: chown d6/da/f9a 39554 1 2024-02-15T18:27:07.305 INFO:tasks.workunit.client.0.smithi042.stdout:9/870: fsync d6/da/d2f/d6c/f94 0 2024-02-15T18:27:07.305 INFO:tasks.workunit.client.0.smithi042.stdout:9/871: readlink d6/da/d2f/d55/d73/d8d/la7 0 2024-02-15T18:27:07.305 INFO:tasks.workunit.client.0.smithi042.stdout:7/868: dwrite d10/f14 [0,4194304] 0 2024-02-15T18:27:07.305 INFO:tasks.workunit.client.0.smithi042.stdout:7/869: chown d10/d15/d39/d6c/ff3 437687 1 2024-02-15T18:27:07.305 INFO:tasks.workunit.client.1.smithi191.stdout:8/556: dwrite d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/fc8 [0,4194304] 0 2024-02-15T18:27:07.306 INFO:tasks.workunit.client.1.smithi191.stdout:8/557: write d3/d2c/d1a/fb1 [156210,4924] 0 2024-02-15T18:27:07.306 INFO:tasks.workunit.client.1.smithi191.stdout:5/225: creat d0/d18/f4b x:0 0 0 2024-02-15T18:27:07.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:06.683026+0000 mgr.smithi042.kfoihb (mgr.24435) 42 : cluster [DBG] pgmap v32: 65 pgs: 65 active+clean; 2.1 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 118 MiB/s rd, 261 MiB/s wr, 382 op/s 2024-02-15T18:27:07.313 INFO:tasks.workunit.client.0.smithi042.stdout:2/823: dread d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/f93 [0,4194304] 0 2024-02-15T18:27:07.313 INFO:tasks.workunit.client.0.smithi042.stdout:2/824: write d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/d44/f84 [2892065,86455] 0 2024-02-15T18:27:07.313 INFO:tasks.workunit.client.0.smithi042.stdout:2/825: chown d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/l4f 1984 1 2024-02-15T18:27:07.313 INFO:tasks.workunit.client.0.smithi042.stdout:2/826: write d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/fb5 [684218,89035] 0 2024-02-15T18:27:07.313 INFO:tasks.workunit.client.0.smithi042.stdout:8/752: rename dd/de/da4/f76 to dd/de/da4/dd0/fdb 0 2024-02-15T18:27:07.314 INFO:tasks.workunit.client.0.smithi042.stdout:2/827: write d0/d8/f65 [1888626,44056] 0 2024-02-15T18:27:07.314 INFO:tasks.workunit.client.0.smithi042.stdout:2/828: chown d0/d8/d20/c47 23632 1 2024-02-15T18:27:07.318 INFO:tasks.workunit.client.1.smithi191.stdout:7/450: dread d0/db/d65/d11/f90 [0,4194304] 0 2024-02-15T18:27:07.320 INFO:tasks.workunit.client.1.smithi191.stdout:5/226: dread d0/d18/f42 [0,4194304] 0 2024-02-15T18:27:07.322 INFO:tasks.workunit.client.0.smithi042.stdout:4/859: dread da/d2b/fd8 [0,4194304] 0 2024-02-15T18:27:07.325 INFO:tasks.workunit.client.1.smithi191.stdout:3/355: sync 2024-02-15T18:27:07.325 INFO:tasks.workunit.client.1.smithi191.stdout:3/356: read - d6/d1b/d60/d73/f34 zero size 2024-02-15T18:27:07.327 INFO:tasks.workunit.client.1.smithi191.stdout:4/441: sync 2024-02-15T18:27:07.327 INFO:tasks.workunit.client.1.smithi191.stdout:2/400: sync 2024-02-15T18:27:07.328 INFO:tasks.workunit.client.1.smithi191.stdout:1/557: dwrite d0/d8/df/d12/d2a/d2b/f61 [0,4194304] 0 2024-02-15T18:27:07.328 INFO:tasks.workunit.client.1.smithi191.stdout:9/462: sync 2024-02-15T18:27:07.328 INFO:tasks.workunit.client.1.smithi191.stdout:9/463: readlink d0/d7/dc/d17/d33/l73 0 2024-02-15T18:27:07.328 INFO:tasks.workunit.client.1.smithi191.stdout:0/305: sync 2024-02-15T18:27:07.329 INFO:tasks.workunit.client.0.smithi042.stdout:3/839: dwrite d1/de/d1a/fc2 [0,4194304] 0 2024-02-15T18:27:07.330 INFO:tasks.workunit.client.0.smithi042.stdout:1/884: creat de/d54/dbf/dd0/d7c/dd6/de3/f152 x:0 0 0 2024-02-15T18:27:07.331 INFO:tasks.workunit.client.1.smithi191.stdout:1/558: write d0/d8/df/d12/d2a/f52 [105387,109008] 0 2024-02-15T18:27:07.331 INFO:tasks.workunit.client.1.smithi191.stdout:9/464: write d0/d29/f3d [432361,9097] 0 2024-02-15T18:27:07.331 INFO:tasks.workunit.client.1.smithi191.stdout:9/465: write d0/d29/f95 [703422,108770] 0 2024-02-15T18:27:07.331 INFO:tasks.workunit.client.1.smithi191.stdout:8/558: link d3/d2c/d1a/d23/d2e/d5b/fa1 d3/d2c/d1a/d23/d2e/d5b/fc9 0 2024-02-15T18:27:07.331 INFO:tasks.workunit.client.1.smithi191.stdout:8/559: fdatasync d3/d2c/d1a/d23/d3e/db2/fbb 0 2024-02-15T18:27:07.333 INFO:tasks.workunit.client.0.smithi042.stdout:9/872: creat d6/dd/f129 x:0 0 0 2024-02-15T18:27:07.334 INFO:tasks.workunit.client.0.smithi042.stdout:0/839: dwrite d4/d38/d77/d2e/f32 [0,4194304] 0 2024-02-15T18:27:07.339 INFO:tasks.workunit.client.0.smithi042.stdout:7/870: dwrite d10/d49/d6d/fd7 [0,4194304] 0 2024-02-15T18:27:07.341 INFO:tasks.workunit.client.0.smithi042.stdout:7/871: read d10/d15/d8f/da5/dad/d104/f96 [489839,68840] 0 2024-02-15T18:27:07.343 INFO:tasks.workunit.client.0.smithi042.stdout:7/872: write d10/d49/d6d/fd7 [3581148,7942] 0 2024-02-15T18:27:07.347 INFO:tasks.workunit.client.1.smithi191.stdout:7/451: link d0/db/d26/d5f/d77/df/d5d/f69 d0/db/d65/d11/d5a/d66/d1e/fa5 0 2024-02-15T18:27:07.350 INFO:tasks.workunit.client.0.smithi042.stdout:6/944: rename d9/d8d/f124 to d9/d11/d19/d1c/dcf/f150 0 2024-02-15T18:27:07.351 INFO:tasks.workunit.client.0.smithi042.stdout:6/945: readlink d9/d11/d19/d35/d55/d92/d9e/lca 0 2024-02-15T18:27:07.352 INFO:tasks.workunit.client.1.smithi191.stdout:3/357: mknod d6/d5d/d72/c7d 0 2024-02-15T18:27:07.352 INFO:tasks.workunit.client.1.smithi191.stdout:3/358: write d6/d3b/d42/d49/d54/f6b [941465,29148] 0 2024-02-15T18:27:07.354 INFO:tasks.workunit.client.0.smithi042.stdout:8/753: rmdir dd/de/da4/db4 0 2024-02-15T18:27:07.359 INFO:tasks.workunit.client.1.smithi191.stdout:4/442: creat d2/d5/d86/faa x:0 0 0 2024-02-15T18:27:07.360 INFO:tasks.workunit.client.0.smithi042.stdout:9/873: dwrite d6/da/d2f/d55/f115 [0,4194304] 0 2024-02-15T18:27:07.360 INFO:tasks.workunit.client.0.smithi042.stdout:7/873: dread d10/d49/d6d/f93 [4194304,4194304] 0 2024-02-15T18:27:07.361 INFO:tasks.workunit.client.0.smithi042.stdout:2/829: creat d0/d8/d20/d7d/d7e/f107 x:0 0 0 2024-02-15T18:27:07.363 INFO:tasks.workunit.client.1.smithi191.stdout:4/443: dread d2/d5/d7/d90/d52/f7b [0,4194304] 0 2024-02-15T18:27:07.364 INFO:tasks.workunit.client.0.smithi042.stdout:5/892: sync 2024-02-15T18:27:07.366 INFO:tasks.workunit.client.0.smithi042.stdout:4/860: creat da/d2b/f12e x:0 0 0 2024-02-15T18:27:07.368 INFO:tasks.workunit.client.1.smithi191.stdout:1/559: creat d0/d8/df/d90/d94/fba x:0 0 0 2024-02-15T18:27:07.369 INFO:tasks.workunit.client.0.smithi042.stdout:3/840: unlink d1/d11/d14/dbf/lc1 0 2024-02-15T18:27:07.370 INFO:tasks.workunit.client.1.smithi191.stdout:9/466: mkdir d0/d7/dc/d32/da3 0 2024-02-15T18:27:07.371 INFO:tasks.workunit.client.0.smithi042.stdout:1/885: rmdir de/d6d/d9e/dcd 39 2024-02-15T18:27:07.371 INFO:tasks.workunit.client.0.smithi042.stdout:1/886: fsync de/d54/dbf/dd0/d7c/dd6/ddf/dca/ffd 0 2024-02-15T18:27:07.374 INFO:tasks.workunit.client.0.smithi042.stdout:0/840: creat d4/d9/d34/dec/f113 x:0 0 0 2024-02-15T18:27:07.375 INFO:tasks.workunit.client.1.smithi191.stdout:7/452: mknod d0/db/d26/ca6 0 2024-02-15T18:27:07.375 INFO:tasks.workunit.client.1.smithi191.stdout:4/444: dread d2/d5/f13 [0,4194304] 0 2024-02-15T18:27:07.378 INFO:tasks.workunit.client.1.smithi191.stdout:5/227: rename d0/d18/f4b to d0/f4c 0 2024-02-15T18:27:07.380 INFO:tasks.workunit.client.1.smithi191.stdout:3/359: mknod d6/d5d/c7e 0 2024-02-15T18:27:07.380 INFO:tasks.workunit.client.1.smithi191.stdout:3/360: truncate f4 4799005 0 2024-02-15T18:27:07.383 INFO:tasks.workunit.client.1.smithi191.stdout:2/401: link df/c5f df/d12/d32/c8a 0 2024-02-15T18:27:07.384 INFO:tasks.workunit.client.0.smithi042.stdout:9/874: mkdir d6/da/d2f/d6c/dd8/d12a 0 2024-02-15T18:27:07.385 INFO:tasks.workunit.client.1.smithi191.stdout:1/560: link d0/d8/df/d41/c4e d0/d8/df/d90/d94/cbb 0 2024-02-15T18:27:07.385 INFO:tasks.workunit.client.1.smithi191.stdout:1/561: write d0/d8/d48/fa1 [571061,70232] 0 2024-02-15T18:27:07.385 INFO:tasks.workunit.client.0.smithi042.stdout:6/946: dwrite d9/d11/d19/fed [4194304,4194304] 0 2024-02-15T18:27:07.385 INFO:tasks.workunit.client.0.smithi042.stdout:6/947: chown d9/d11/d19/l1f 259 1 2024-02-15T18:27:07.386 INFO:tasks.workunit.client.0.smithi042.stdout:7/874: symlink d10/d15/d39/de3/l127 0 2024-02-15T18:27:07.386 INFO:tasks.workunit.client.0.smithi042.stdout:9/875: write d6/f21 [693201,52875] 0 2024-02-15T18:27:07.386 INFO:tasks.workunit.client.0.smithi042.stdout:9/876: stat d6/da/d2f/d54/lbd 0 2024-02-15T18:27:07.387 INFO:tasks.workunit.client.1.smithi191.stdout:9/467: creat d0/d7/dc/d17/d44/fa4 x:0 0 0 2024-02-15T18:27:07.388 INFO:tasks.workunit.client.0.smithi042.stdout:5/893: mkdir d0/d12/d2a/d105/d109/d137 0 2024-02-15T18:27:07.389 INFO:tasks.workunit.client.1.smithi191.stdout:9/468: write d0/d7/dc/d17/d44/f6e [3633072,129350] 0 2024-02-15T18:27:07.389 INFO:tasks.workunit.client.1.smithi191.stdout:8/560: mkdir d3/d2c/d1a/d23/d2e/d64/dca 0 2024-02-15T18:27:07.389 INFO:tasks.workunit.client.1.smithi191.stdout:9/469: write d0/d7/dc/d8c/f78 [17685,120984] 0 2024-02-15T18:27:07.389 INFO:tasks.workunit.client.1.smithi191.stdout:9/470: write d0/d29/f95 [856333,91719] 0 2024-02-15T18:27:07.389 INFO:tasks.workunit.client.1.smithi191.stdout:9/471: chown d0/d7/dc/d30/d4f/c54 2372 1 2024-02-15T18:27:07.389 INFO:tasks.workunit.client.1.smithi191.stdout:9/472: write d0/d1d/d3f/d80/fa2 [497945,11336] 0 2024-02-15T18:27:07.389 INFO:tasks.workunit.client.1.smithi191.stdout:9/473: chown d0/d7/dc/d30/d3b/d56/c82 513320 1 2024-02-15T18:27:07.389 INFO:tasks.workunit.client.1.smithi191.stdout:9/474: dread - d0/d7/dc/d17/d33/f77 zero size 2024-02-15T18:27:07.391 INFO:tasks.workunit.client.1.smithi191.stdout:7/453: creat d0/db/d26/d5f/d77/d35/d42/fa7 x:0 0 0 2024-02-15T18:27:07.392 INFO:tasks.workunit.client.0.smithi042.stdout:4/861: mkdir da/d11/d85/db5/d103/d66/d78/d53/d82/d12f 0 2024-02-15T18:27:07.392 INFO:tasks.workunit.client.0.smithi042.stdout:4/862: dread - da/d11/d67/f10e zero size 2024-02-15T18:27:07.392 INFO:tasks.workunit.client.0.smithi042.stdout:3/841: creat d1/de/f115 x:0 0 0 2024-02-15T18:27:07.392 INFO:tasks.workunit.client.0.smithi042.stdout:4/863: write da/d3a/f44 [553068,105831] 0 2024-02-15T18:27:07.395 INFO:tasks.workunit.client.1.smithi191.stdout:4/445: mknod d2/d5/d7/d3e/cab 0 2024-02-15T18:27:07.395 INFO:tasks.workunit.client.1.smithi191.stdout:4/446: fsync d2/d5/d7/d3e/d63/d8c/da4/fa2 0 2024-02-15T18:27:07.396 INFO:tasks.workunit.client.0.smithi042.stdout:8/754: dwrite dd/de/da4/d81/f8c [0,4194304] 0 2024-02-15T18:27:07.396 INFO:tasks.workunit.client.0.smithi042.stdout:0/841: rename d4/d9/d34/dab/dc8 to d4/d38/d39/df4/d114 0 2024-02-15T18:27:07.396 INFO:tasks.workunit.client.0.smithi042.stdout:2/830: dwrite d0/d8/d20/d7d/d7e/dc7/f7f [0,4194304] 0 2024-02-15T18:27:07.396 INFO:tasks.workunit.client.0.smithi042.stdout:1/887: dwrite de/d6d/d9e/dcd/fe4 [4194304,4194304] 0 2024-02-15T18:27:07.396 INFO:tasks.workunit.client.0.smithi042.stdout:1/888: stat de/d54/dbf/dd0/d7c/dd6/de8/d103 0 2024-02-15T18:27:07.399 INFO:tasks.workunit.client.0.smithi042.stdout:8/755: dread dd/de/da4/d67/d5a/f8f [0,4194304] 0 2024-02-15T18:27:07.401 INFO:tasks.workunit.client.1.smithi191.stdout:5/228: mknod d0/d23/d41/d28/d31/d38/c4d 0 2024-02-15T18:27:07.401 INFO:tasks.workunit.client.1.smithi191.stdout:5/229: truncate d0/f4c 32867 0 2024-02-15T18:27:07.401 INFO:tasks.workunit.client.1.smithi191.stdout:5/230: fdatasync d0/d23/f3d 0 2024-02-15T18:27:07.402 INFO:tasks.workunit.client.1.smithi191.stdout:5/231: dread d0/d18/f2e [0,4194304] 0 2024-02-15T18:27:07.402 INFO:tasks.workunit.client.1.smithi191.stdout:5/232: chown d0/d18/f2e 49724 1 2024-02-15T18:27:07.407 INFO:tasks.workunit.client.1.smithi191.stdout:0/306: rename d7/da/d17/f40 to d7/f63 0 2024-02-15T18:27:07.411 INFO:tasks.workunit.client.0.smithi042.stdout:7/875: creat d10/d15/d8f/da5/dad/d104/d65/f128 x:0 0 0 2024-02-15T18:27:07.412 INFO:tasks.workunit.client.1.smithi191.stdout:2/402: unlink df/d12/d1e/l71 0 2024-02-15T18:27:07.413 INFO:tasks.workunit.client.0.smithi042.stdout:9/877: creat d6/da/d75/f12b x:0 0 0 2024-02-15T18:27:07.414 INFO:tasks.workunit.client.1.smithi191.stdout:1/562: mknod d0/d8/df/d12/d2a/d2b/d78/cbc 0 2024-02-15T18:27:07.415 INFO:tasks.workunit.client.1.smithi191.stdout:2/403: dread df/d12/d1e/f41 [0,4194304] 0 2024-02-15T18:27:07.415 INFO:tasks.workunit.client.0.smithi042.stdout:5/894: creat d0/dda/dc7/d52/dad/dba/dbd/dde/f138 x:0 0 0 2024-02-15T18:27:07.416 INFO:tasks.workunit.client.0.smithi042.stdout:5/895: read d0/dda/d42/f49 [396353,59395] 0 2024-02-15T18:27:07.416 INFO:tasks.workunit.client.1.smithi191.stdout:8/561: getdents d3/d5e 0 2024-02-15T18:27:07.418 INFO:tasks.workunit.client.0.smithi042.stdout:3/842: creat d1/d11/d14/d4d/d73/d89/f116 x:0 0 0 2024-02-15T18:27:07.418 INFO:tasks.workunit.client.1.smithi191.stdout:5/233: dwrite d0/d23/d41/d28/f3e [0,4194304] 0 2024-02-15T18:27:07.418 INFO:tasks.workunit.client.1.smithi191.stdout:5/234: chown d0/d23/d41/c16 88 1 2024-02-15T18:27:07.420 INFO:tasks.workunit.client.0.smithi042.stdout:4/864: mknod da/d11/d85/db5/d103/d66/d78/d53/c130 0 2024-02-15T18:27:07.420 INFO:tasks.workunit.client.1.smithi191.stdout:9/475: stat d0/d1d/l83 0 2024-02-15T18:27:07.431 INFO:tasks.workunit.client.1.smithi191.stdout:0/307: dwrite d7/da/f20 [0,4194304] 0 2024-02-15T18:27:07.432 INFO:tasks.workunit.client.1.smithi191.stdout:0/308: truncate d7/da/d1a/f58 562664 0 2024-02-15T18:27:07.432 INFO:tasks.workunit.client.1.smithi191.stdout:7/454: creat d0/db/d65/dc/fa8 x:0 0 0 2024-02-15T18:27:07.433 INFO:tasks.workunit.client.0.smithi042.stdout:5/896: dwrite d0/d39/ddd/de0/f10b [0,4194304] 0 2024-02-15T18:27:07.436 INFO:tasks.workunit.client.1.smithi191.stdout:4/447: creat d2/d5/d14/d71/fac x:0 0 0 2024-02-15T18:27:07.440 INFO:tasks.workunit.client.1.smithi191.stdout:1/563: link d0/d8/df/d12/d2a/f42 d0/d2f/d77/fbd 0 2024-02-15T18:27:07.441 INFO:tasks.workunit.client.1.smithi191.stdout:1/564: dread d0/d2f/d66/f95 [0,4194304] 0 2024-02-15T18:27:07.441 INFO:tasks.workunit.client.0.smithi042.stdout:1/889: creat de/d15/d67/d46/d6a/f153 x:0 0 0 2024-02-15T18:27:07.442 INFO:tasks.workunit.client.0.smithi042.stdout:4/865: dwrite da/d2b/f37 [0,4194304] 0 2024-02-15T18:27:07.443 INFO:tasks.workunit.client.1.smithi191.stdout:8/562: rename d3/d5e/c94 to d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/ccb 0 2024-02-15T18:27:07.450 INFO:tasks.workunit.client.1.smithi191.stdout:9/476: creat d0/d7/dc/fa5 x:0 0 0 2024-02-15T18:27:07.450 INFO:tasks.workunit.client.1.smithi191.stdout:9/477: chown d0/d7/dc/d17/f25 30987 1 2024-02-15T18:27:07.450 INFO:tasks.workunit.client.1.smithi191.stdout:9/478: write d0/d7/dc/d17/f49 [144242,116846] 0 2024-02-15T18:27:07.453 INFO:tasks.workunit.client.1.smithi191.stdout:0/309: symlink d7/da/d1a/d34/d27/l64 0 2024-02-15T18:27:07.454 INFO:tasks.workunit.client.1.smithi191.stdout:4/448: getdents d2/d5/d14/d47 0 2024-02-15T18:27:07.454 INFO:tasks.workunit.client.1.smithi191.stdout:4/449: chown d2/d5/d7/d2f/d41/f73 484873876 1 2024-02-15T18:27:07.454 INFO:tasks.workunit.client.1.smithi191.stdout:4/450: fsync d2/d5/d7/d90/d52/f7b 0 2024-02-15T18:27:07.454 INFO:tasks.workunit.client.1.smithi191.stdout:4/451: readlink d2/d5/d7/l27 0 2024-02-15T18:27:07.455 INFO:tasks.workunit.client.0.smithi042.stdout:8/756: unlink dd/de/d33/d66/d4e/c30 0 2024-02-15T18:27:07.456 INFO:tasks.workunit.client.1.smithi191.stdout:2/404: mknod df/c8b 0 2024-02-15T18:27:07.533 INFO:tasks.workunit.client.0.smithi042.stdout:7/876: creat d10/d15/d8f/da5/f129 x:0 0 0 2024-02-15T18:27:07.535 INFO:tasks.workunit.client.1.smithi191.stdout:5/235: rename d0/d23/c2d to d0/d23/d41/c4e 0 2024-02-15T18:27:07.536 INFO:tasks.workunit.client.0.smithi042.stdout:9/878: mkdir d6/da/d2f/d55/d12c 0 2024-02-15T18:27:07.536 INFO:tasks.workunit.client.0.smithi042.stdout:9/879: dread - d6/da/d2f/d69/f8a zero size 2024-02-15T18:27:07.536 INFO:tasks.workunit.client.1.smithi191.stdout:5/236: dread d0/f4c [0,4194304] 0 2024-02-15T18:27:07.537 INFO:tasks.workunit.client.1.smithi191.stdout:8/563: unlink d3/d2c/d1a/d23/d3e/l59 0 2024-02-15T18:27:07.537 INFO:tasks.workunit.client.1.smithi191.stdout:8/564: chown d3/d2c/d1a/d36/d9b/faf 46 1 2024-02-15T18:27:07.538 INFO:tasks.workunit.client.0.smithi042.stdout:3/843: rmdir d1/d11/d14/d4d/d73/d89/da8 39 2024-02-15T18:27:07.541 INFO:tasks.workunit.client.1.smithi191.stdout:0/310: truncate d7/da/f18 703416 0 2024-02-15T18:27:07.543 INFO:tasks.workunit.client.1.smithi191.stdout:4/452: unlink d2/d5/l40 0 2024-02-15T18:27:07.545 INFO:tasks.workunit.client.1.smithi191.stdout:2/405: creat df/d12/d19/d1a/d35/d47/d4b/d7a/f8c x:0 0 0 2024-02-15T18:27:07.546 INFO:tasks.workunit.client.0.smithi042.stdout:5/897: mkdir d0/d50/df7/d139 0 2024-02-15T18:27:07.550 INFO:tasks.workunit.client.1.smithi191.stdout:7/455: rename d0/db/d26/d5f/d77/df/d5d/f69 to d0/db/d26/d5f/d77/d35/d42/fa9 0 2024-02-15T18:27:07.551 INFO:tasks.workunit.client.0.smithi042.stdout:0/842: rename d4/d38/d39/df4/d114/dfe/l97 to d4/d38/d39/df4/d114/l115 0 2024-02-15T18:27:07.553 INFO:tasks.workunit.client.1.smithi191.stdout:2/406: unlink df/d12/d19/d1a/d35/f70 0 2024-02-15T18:27:07.557 INFO:tasks.workunit.client.0.smithi042.stdout:8/757: symlink dd/de/d33/d66/ldc 0 2024-02-15T18:27:07.558 INFO:tasks.workunit.client.0.smithi042.stdout:8/758: write dd/de/da4/fa3 [278128,130176] 0 2024-02-15T18:27:07.562 INFO:tasks.workunit.client.0.smithi042.stdout:9/880: creat d6/f12d x:0 0 0 2024-02-15T18:27:07.562 INFO:tasks.workunit.client.0.smithi042.stdout:9/881: readlink d6/da/d1d/d2d/d40/d5f/l99 0 2024-02-15T18:27:07.562 INFO:tasks.workunit.client.0.smithi042.stdout:9/882: chown d6/fa8 244 1 2024-02-15T18:27:07.562 INFO:tasks.workunit.client.0.smithi042.stdout:5/898: mknod d0/dda/dc7/d52/da0/d125/c13a 0 2024-02-15T18:27:07.564 INFO:tasks.workunit.client.0.smithi042.stdout:2/831: rename d0/d8/d10/d31/da6/cd0 to d0/d8/d10/d31/d3a/c108 0 2024-02-15T18:27:07.566 INFO:tasks.workunit.client.0.smithi042.stdout:0/843: creat d4/d9/d34/dab/db0/f116 x:0 0 0 2024-02-15T18:27:07.566 INFO:tasks.workunit.client.0.smithi042.stdout:0/844: write d4/d9/d34/d57/d88/fcf [600143,106852] 0 2024-02-15T18:27:07.566 INFO:tasks.workunit.client.0.smithi042.stdout:0/845: stat d4/d9/d34/d57/d88/dc9/fe3 0 2024-02-15T18:27:07.567 INFO:tasks.workunit.client.1.smithi191.stdout:1/565: rename d0/d8/df/d12/d2a/fb0 to d0/d8/df/d12/d2a/d2b/d78/d60/fbe 0 2024-02-15T18:27:07.567 INFO:tasks.workunit.client.1.smithi191.stdout:7/456: link d0/db/d26/d5f/d77/d35/f48 d0/db/d65/d11/d5a/d66/d1e/faa 0 2024-02-15T18:27:07.567 INFO:tasks.workunit.client.0.smithi042.stdout:8/759: mknod dd/de/da4/d81/cdd 0 2024-02-15T18:27:07.567 INFO:tasks.workunit.client.0.smithi042.stdout:8/760: fdatasync dd/de/da4/d67/d6f/fb0 0 2024-02-15T18:27:07.569 INFO:tasks.workunit.client.0.smithi042.stdout:5/899: creat d0/d38/d53/d91/d94/f13b x:0 0 0 2024-02-15T18:27:07.569 INFO:tasks.workunit.client.0.smithi042.stdout:2/832: read d0/d8/d10/d5a/fab [3764544,12140] 0 2024-02-15T18:27:07.570 INFO:tasks.workunit.client.0.smithi042.stdout:2/833: fdatasync d0/d8/d10/d31/d79/fea 0 2024-02-15T18:27:07.570 INFO:tasks.workunit.client.1.smithi191.stdout:9/479: rename d0/d7/dc/d30/c46 to d0/d1d/d3f/d80/ca6 0 2024-02-15T18:27:07.574 INFO:tasks.workunit.client.1.smithi191.stdout:7/457: symlink d0/db/d26/lab 0 2024-02-15T18:27:07.577 INFO:tasks.workunit.client.1.smithi191.stdout:9/480: link d0/c5a d0/d7/dc/d30/d4f/ca7 0 2024-02-15T18:27:07.577 INFO:tasks.workunit.client.0.smithi042.stdout:1/890: rename de/d15/d67/d63/f8b to de/d54/dbf/dd0/d7c/dd6/de8/d103/d10f/f154 0 2024-02-15T18:27:07.577 INFO:tasks.workunit.client.0.smithi042.stdout:0/846: readlink d4/d38/d39/df4/d114/lea 0 2024-02-15T18:27:07.577 INFO:tasks.workunit.client.0.smithi042.stdout:1/891: fsync de/d15/d67/d46/f59 0 2024-02-15T18:27:07.577 INFO:tasks.workunit.client.0.smithi042.stdout:0/847: write d4/d9/d34/fe7 [946314,69905] 0 2024-02-15T18:27:07.578 INFO:tasks.workunit.client.0.smithi042.stdout:0/848: chown d4/d38/c47 55 1 2024-02-15T18:27:07.578 INFO:tasks.workunit.client.0.smithi042.stdout:0/849: chown d4/d38/d5c/fa2 10 1 2024-02-15T18:27:07.579 INFO:tasks.workunit.client.1.smithi191.stdout:8/565: rename d3/d2c/d1a/d23/d2e/c4d to d3/d2c/d1a/d23/d2e/d5b/d69/d6d/ccc 0 2024-02-15T18:27:07.581 INFO:tasks.workunit.client.1.smithi191.stdout:8/566: symlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/lcd 0 2024-02-15T18:27:07.581 INFO:tasks.workunit.client.1.smithi191.stdout:8/567: write d3/d2c/d1a/d36/d9b/faf [666728,28761] 0 2024-02-15T18:27:07.585 INFO:tasks.workunit.client.0.smithi042.stdout:4/866: rename da/d3b/d41/d8d/f10a to da/d11/d85/db5/d103/f131 0 2024-02-15T18:27:07.585 INFO:tasks.workunit.client.0.smithi042.stdout:4/867: chown da/d3b/c8c 521854 1 2024-02-15T18:27:07.587 INFO:tasks.workunit.client.1.smithi191.stdout:8/568: creat d3/d2c/d1a/fce x:0 0 0 2024-02-15T18:27:07.589 INFO:tasks.workunit.client.1.smithi191.stdout:5/237: dwrite d0/d18/f45 [0,4194304] 0 2024-02-15T18:27:07.589 INFO:tasks.workunit.client.1.smithi191.stdout:5/238: truncate d0/d23/d41/d28/d31/f47 117438 0 2024-02-15T18:27:07.591 INFO:tasks.workunit.client.1.smithi191.stdout:8/569: mkdir d3/d2c/d1a/d23/d2e/d5b/d69/d6d/dcf 0 2024-02-15T18:27:07.592 INFO:tasks.workunit.client.0.smithi042.stdout:1/892: mkdir de/d54/dbf/dd0/d7c/dd6/d155 0 2024-02-15T18:27:07.592 INFO:tasks.workunit.client.0.smithi042.stdout:4/868: creat da/d11/d85/db5/d103/d66/d78/d2f/dca/f132 x:0 0 0 2024-02-15T18:27:07.592 INFO:tasks.workunit.client.1.smithi191.stdout:5/239: symlink d0/d23/d41/d28/d31/d38/l4f 0 2024-02-15T18:27:07.592 INFO:tasks.workunit.client.1.smithi191.stdout:5/240: stat d0/d23/d41/d28/d31/f47 0 2024-02-15T18:27:07.594 INFO:tasks.workunit.client.1.smithi191.stdout:0/311: dwrite d7/d1b/f46 [0,4194304] 0 2024-02-15T18:27:07.594 INFO:tasks.workunit.client.1.smithi191.stdout:0/312: fsync d7/da/d1a/d34/f5e 0 2024-02-15T18:27:07.594 INFO:tasks.workunit.client.1.smithi191.stdout:8/570: rename d3/d2c/d1a/d23/d2e/d5b/d69/c8c to d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/cd0 0 2024-02-15T18:27:07.595 INFO:tasks.workunit.client.0.smithi042.stdout:1/893: unlink de/d15/d1b/fc7 0 2024-02-15T18:27:07.596 INFO:tasks.workunit.client.0.smithi042.stdout:1/894: readlink de/d54/dbf/dd0/d7c/dd6/ddf/l5d 0 2024-02-15T18:27:07.596 INFO:tasks.workunit.client.0.smithi042.stdout:8/761: dread dd/de/d33/d66/f7f [0,4194304] 0 2024-02-15T18:27:07.597 INFO:tasks.workunit.client.1.smithi191.stdout:5/241: symlink d0/d23/d41/d28/d31/d38/l50 0 2024-02-15T18:27:07.597 INFO:tasks.workunit.client.1.smithi191.stdout:5/242: fsync d0/f1a 0 2024-02-15T18:27:07.597 INFO:tasks.workunit.client.1.smithi191.stdout:5/243: truncate d0/d23/d41/d28/d31/f47 939481 0 2024-02-15T18:27:07.597 INFO:tasks.workunit.client.0.smithi042.stdout:4/869: unlink da/l18 0 2024-02-15T18:27:07.597 INFO:tasks.workunit.client.0.smithi042.stdout:4/870: write da/d11/d85/db5/d103/d66/d78/d53/f9f [572575,50105] 0 2024-02-15T18:27:07.598 INFO:tasks.workunit.client.0.smithi042.stdout:1/895: write de/d15/d2c/f38 [3293341,57347] 0 2024-02-15T18:27:07.598 INFO:tasks.workunit.client.0.smithi042.stdout:8/762: dread - dd/d11/f9b zero size 2024-02-15T18:27:07.598 INFO:tasks.workunit.client.1.smithi191.stdout:8/571: write d3/d2c/d1a/f8f [928225,96009] 0 2024-02-15T18:27:07.599 INFO:tasks.workunit.client.1.smithi191.stdout:5/244: read d0/d23/d41/d28/d31/d38/f40 [1616445,105093] 0 2024-02-15T18:27:07.599 INFO:tasks.workunit.client.1.smithi191.stdout:5/245: write d0/d23/d41/d28/f36 [4316058,53727] 0 2024-02-15T18:27:07.599 INFO:tasks.workunit.client.0.smithi042.stdout:4/871: mkdir da/d11/d85/db5/d103/d66/d133 0 2024-02-15T18:27:07.601 INFO:tasks.workunit.client.0.smithi042.stdout:1/896: creat de/d54/d9a/f156 x:0 0 0 2024-02-15T18:27:07.602 INFO:tasks.workunit.client.0.smithi042.stdout:1/897: stat de/d15/d1b/d44/l62 0 2024-02-15T18:27:07.602 INFO:tasks.workunit.client.0.smithi042.stdout:1/898: chown de/d54/dbf/dd0/d7c/db3 3331 1 2024-02-15T18:27:07.602 INFO:tasks.workunit.client.0.smithi042.stdout:1/899: fdatasync de/d15/d1b/d44/fb2 0 2024-02-15T18:27:07.602 INFO:tasks.workunit.client.0.smithi042.stdout:1/900: fsync de/d15/d1b/d44/fb2 0 2024-02-15T18:27:07.602 INFO:tasks.workunit.client.1.smithi191.stdout:5/246: link d0/c3 d0/d23/d41/d28/d31/d38/c51 0 2024-02-15T18:27:07.602 INFO:tasks.workunit.client.1.smithi191.stdout:5/247: chown d0/f4c 7760 1 2024-02-15T18:27:07.603 INFO:tasks.workunit.client.0.smithi042.stdout:4/872: dread da/d11/d85/db5/d103/d66/d78/d2f/d4d/f48 [0,4194304] 0 2024-02-15T18:27:07.603 INFO:tasks.workunit.client.0.smithi042.stdout:8/763: rmdir dd/de/da4/d67/d5a/db2 39 2024-02-15T18:27:07.604 INFO:tasks.workunit.client.1.smithi191.stdout:0/313: dread d7/da/d1a/d34/f11 [0,4194304] 0 2024-02-15T18:27:07.604 INFO:tasks.workunit.client.1.smithi191.stdout:0/314: chown d7/da/d17/d31/d5f/f5b 19492 1 2024-02-15T18:27:07.605 INFO:tasks.workunit.client.1.smithi191.stdout:8/572: rename d3/d2c/d1a/f51 to d3/d5e/fd1 0 2024-02-15T18:27:07.607 INFO:tasks.workunit.client.0.smithi042.stdout:4/873: mkdir da/d11/d85/db5/d103/d66/d78/d2f/d4d/d134 0 2024-02-15T18:27:07.608 INFO:tasks.workunit.client.0.smithi042.stdout:8/764: getdents dd/de/d12 0 2024-02-15T18:27:07.610 INFO:tasks.workunit.client.1.smithi191.stdout:8/573: creat d3/d2c/d1a/d23/d2e/d5b/d69/d6d/fd2 x:0 0 0 2024-02-15T18:27:07.611 INFO:tasks.workunit.client.0.smithi042.stdout:4/874: mkdir da/d3b/d135 0 2024-02-15T18:27:07.613 INFO:tasks.workunit.client.1.smithi191.stdout:8/574: link d3/d2c/d1a/d23/d3e/c7b d3/d2c/d1a/d23/d2e/d47/cd3 0 2024-02-15T18:27:07.615 INFO:tasks.workunit.client.1.smithi191.stdout:8/575: unlink d3/d2c/d1a/d23/d2e/d5b/d69/l72 0 2024-02-15T18:27:07.615 INFO:tasks.workunit.client.1.smithi191.stdout:8/576: readlink d3/l71 0 2024-02-15T18:27:07.615 INFO:tasks.workunit.client.1.smithi191.stdout:8/577: write d3/d2c/f75 [4214898,31159] 0 2024-02-15T18:27:07.616 INFO:tasks.workunit.client.1.smithi191.stdout:8/578: rename d3/l2b to d3/d2c/d1a/d23/d2a/ld4 0 2024-02-15T18:27:07.616 INFO:tasks.workunit.client.1.smithi191.stdout:8/579: chown d3/d2c/f1f 2 1 2024-02-15T18:27:07.617 INFO:tasks.workunit.client.1.smithi191.stdout:8/580: mknod d3/d2c/d1a/cd5 0 2024-02-15T18:27:07.619 INFO:tasks.workunit.client.0.smithi042.stdout:8/765: dread dd/de/f1a [4194304,4194304] 0 2024-02-15T18:27:07.621 INFO:tasks.workunit.client.0.smithi042.stdout:8/766: write dd/de/da4/f8d [158425,30709] 0 2024-02-15T18:27:07.621 INFO:tasks.workunit.client.0.smithi042.stdout:8/767: chown dd/de/d33/d3b/d92/l96 919534 1 2024-02-15T18:27:07.623 INFO:tasks.workunit.client.0.smithi042.stdout:8/768: creat dd/de/d3c/fde x:0 0 0 2024-02-15T18:27:07.623 INFO:tasks.workunit.client.0.smithi042.stdout:7/877: dwrite d10/d49/d6d/d78/d83/fa3 [0,4194304] 0 2024-02-15T18:27:07.628 INFO:tasks.workunit.client.0.smithi042.stdout:8/769: mknod dd/de/d94/db1/cdf 0 2024-02-15T18:27:07.630 INFO:tasks.workunit.client.0.smithi042.stdout:8/770: dread dd/de/da4/d67/d5a/f8f [0,4194304] 0 2024-02-15T18:27:07.633 INFO:tasks.workunit.client.0.smithi042.stdout:7/878: rename d10/d49/d6d/d6e/db3/fc7 to d10/d15/d8f/da5/f12a 0 2024-02-15T18:27:07.638 INFO:tasks.workunit.client.0.smithi042.stdout:7/879: rename d10/d15/d8f/da5/dad/d104/fed to d10/d15/d110/f12b 0 2024-02-15T18:27:07.639 INFO:tasks.workunit.client.0.smithi042.stdout:7/880: unlink d10/c1e 0 2024-02-15T18:27:07.640 INFO:tasks.workunit.client.0.smithi042.stdout:7/881: creat d10/d15/d8f/da5/dad/de0/f12c x:0 0 0 2024-02-15T18:27:07.640 INFO:tasks.workunit.client.0.smithi042.stdout:7/882: stat d10/d49/d6d/d78/c10d 0 2024-02-15T18:27:07.642 INFO:tasks.workunit.client.0.smithi042.stdout:7/883: creat d10/d15/d39/d6c/dba/d117/d11f/f12d x:0 0 0 2024-02-15T18:27:07.642 INFO:tasks.workunit.client.0.smithi042.stdout:7/884: stat d10/c9d 0 2024-02-15T18:27:07.642 INFO:tasks.workunit.client.0.smithi042.stdout:7/885: fsync d10/d49/d6d/d78/fe1 0 2024-02-15T18:27:07.642 INFO:tasks.workunit.client.0.smithi042.stdout:7/886: chown d10/d49/d6d/db6/l119 524440 1 2024-02-15T18:27:07.642 INFO:tasks.workunit.client.0.smithi042.stdout:7/887: chown d10/d15/d8f/da5/dad/d104/d23/d51/dc9/ffe 184963 1 2024-02-15T18:27:07.643 INFO:tasks.workunit.client.0.smithi042.stdout:8/771: dwrite dd/de/fda [0,4194304] 0 2024-02-15T18:27:07.643 INFO:tasks.workunit.client.0.smithi042.stdout:8/772: read - dd/d11/f9b zero size 2024-02-15T18:27:07.643 INFO:tasks.workunit.client.0.smithi042.stdout:7/888: truncate d10/d49/f67 120863 0 2024-02-15T18:27:07.643 INFO:tasks.workunit.client.0.smithi042.stdout:7/889: write d10/d15/d8f/f9f [770574,106475] 0 2024-02-15T18:27:07.644 INFO:tasks.workunit.client.0.smithi042.stdout:8/773: write dd/de/da4/d67/d5a/f8f [1421620,96543] 0 2024-02-15T18:27:07.645 INFO:tasks.workunit.client.0.smithi042.stdout:8/774: write dd/d11/d75/f9d [4052803,100899] 0 2024-02-15T18:27:07.646 INFO:tasks.workunit.client.0.smithi042.stdout:3/844: dwrite d1/de/d19/fc9 [0,4194304] 0 2024-02-15T18:27:07.646 INFO:tasks.workunit.client.1.smithi191.stdout:2/407: dwrite df/d12/d19/d1a/d35/d5b/f6e [0,4194304] 0 2024-02-15T18:27:07.646 INFO:tasks.workunit.client.1.smithi191.stdout:2/408: write df/d12/d19/f77 [29411,105111] 0 2024-02-15T18:27:07.648 INFO:tasks.workunit.client.1.smithi191.stdout:0/315: dwrite d7/da/d1a/d34/f11 [0,4194304] 0 2024-02-15T18:27:07.648 INFO:tasks.workunit.client.0.smithi042.stdout:0/850: dwrite d4/d38/d77/f7c [0,4194304] 0 2024-02-15T18:27:07.648 INFO:tasks.workunit.client.0.smithi042.stdout:9/883: dwrite d6/da/d2f/d55/fa4 [0,4194304] 0 2024-02-15T18:27:07.651 INFO:tasks.workunit.client.0.smithi042.stdout:5/900: dwrite d0/d38/d53/d64/de5/fc9 [0,4194304] 0 2024-02-15T18:27:07.651 INFO:tasks.workunit.client.1.smithi191.stdout:4/453: dwrite d2/f4 [0,4194304] 0 2024-02-15T18:27:07.654 INFO:tasks.workunit.client.0.smithi042.stdout:3/845: dread d1/de/d1a/f6f [0,4194304] 0 2024-02-15T18:27:07.654 INFO:tasks.workunit.client.1.smithi191.stdout:2/409: mknod df/d12/c8d 0 2024-02-15T18:27:07.657 INFO:tasks.workunit.client.0.smithi042.stdout:1/901: dwrite de/d54/dbf/dd0/d7c/dd6/ddf/dca/f10a [0,4194304] 0 2024-02-15T18:27:07.658 INFO:tasks.workunit.client.1.smithi191.stdout:7/458: dwrite d0/db/d65/d11/d5a/d66/d1e/f8a [0,4194304] 0 2024-02-15T18:27:07.660 INFO:tasks.workunit.client.0.smithi042.stdout:0/851: symlink d4/d9/d34/d8b/l117 0 2024-02-15T18:27:07.660 INFO:tasks.workunit.client.1.smithi191.stdout:9/481: dwrite d0/d7/dc/d8c/f35 [0,4194304] 0 2024-02-15T18:27:07.661 INFO:tasks.workunit.client.1.smithi191.stdout:5/248: dwrite d0/d23/d41/d28/f3e [4194304,4194304] 0 2024-02-15T18:27:07.661 INFO:tasks.workunit.client.1.smithi191.stdout:2/410: unlink df/d12/d19/d1a/f82 0 2024-02-15T18:27:07.661 INFO:tasks.workunit.client.1.smithi191.stdout:9/482: write d0/d7/dc/d30/d3b/d56/d59/f97 [265267,126145] 0 2024-02-15T18:27:07.661 INFO:tasks.workunit.client.1.smithi191.stdout:8/581: dwrite d3/f31 [0,4194304] 0 2024-02-15T18:27:07.666 INFO:tasks.workunit.client.0.smithi042.stdout:6/948: sync 2024-02-15T18:27:07.666 INFO:tasks.workunit.client.0.smithi042.stdout:6/949: chown d9/d11/d46/d89/f12c 386 1 2024-02-15T18:27:07.667 INFO:tasks.workunit.client.1.smithi191.stdout:4/454: write d2/d5/d14/d47/d78/f6a [1805921,107416] 0 2024-02-15T18:27:07.667 INFO:tasks.workunit.client.1.smithi191.stdout:4/455: fsync d2/d5/d7/d2f/d41/f8f 0 2024-02-15T18:27:07.667 INFO:tasks.workunit.client.0.smithi042.stdout:8/775: dread dd/de/da4/d67/d5a/f64 [0,4194304] 0 2024-02-15T18:27:07.667 INFO:tasks.workunit.client.0.smithi042.stdout:8/776: chown dd/de/da4/d67/d5a/f91 57630606 1 2024-02-15T18:27:07.667 INFO:tasks.workunit.client.0.smithi042.stdout:6/950: read d9/d3e/f64 [693491,116275] 0 2024-02-15T18:27:07.667 INFO:tasks.workunit.client.0.smithi042.stdout:6/951: write d9/d3e/f112 [386577,108735] 0 2024-02-15T18:27:07.670 INFO:tasks.workunit.client.0.smithi042.stdout:3/846: write d1/d11/d14/d4d/d6c/f9b [520013,39769] 0 2024-02-15T18:27:07.670 INFO:tasks.workunit.client.0.smithi042.stdout:3/847: chown d1/l113 228 1 2024-02-15T18:27:07.680 INFO:tasks.workunit.client.1.smithi191.stdout:4/456: dread d2/d5/d14/d47/d78/f38 [0,4194304] 0 2024-02-15T18:27:07.689 INFO:tasks.workunit.client.1.smithi191.stdout:0/316: dwrite d7/d4a/f56 [0,4194304] 0 2024-02-15T18:27:07.690 INFO:tasks.workunit.client.0.smithi042.stdout:7/890: dwrite d10/d49/d6d/f86 [0,4194304] 0 2024-02-15T18:27:07.690 INFO:tasks.workunit.client.0.smithi042.stdout:7/891: chown d10/d49/c5c 9560636 1 2024-02-15T18:27:07.690 INFO:tasks.workunit.client.0.smithi042.stdout:8/777: dwrite dd/de/d33/d66/f34 [0,4194304] 0 2024-02-15T18:27:07.690 INFO:tasks.workunit.client.0.smithi042.stdout:8/778: write dd/de/d33/d3b/f46 [1478820,26226] 0 2024-02-15T18:27:07.690 INFO:tasks.workunit.client.0.smithi042.stdout:8/779: truncate dd/f7d 710853 0 2024-02-15T18:27:07.690 INFO:tasks.workunit.client.0.smithi042.stdout:8/780: fdatasync dd/de/da4/dd0/fdb 0 2024-02-15T18:27:07.693 INFO:tasks.workunit.client.0.smithi042.stdout:0/852: dwrite d4/d9/d5a/dfc/d92/fca [0,4194304] 0 2024-02-15T18:27:07.695 INFO:tasks.workunit.client.0.smithi042.stdout:7/892: write d10/d15/d8f/da5/dad/d104/d23/fa1 [381162,47989] 0 2024-02-15T18:27:07.708 INFO:tasks.workunit.client.0.smithi042.stdout:9/884: getdents d6/da/d1d/d2d/d40/de0 0 2024-02-15T18:27:07.708 INFO:tasks.workunit.client.0.smithi042.stdout:9/885: read - d6/dd/d9/d59/dbf/d128/f10a zero size 2024-02-15T18:27:07.709 INFO:tasks.workunit.client.1.smithi191.stdout:7/459: creat d0/db/d65/d11/fac x:0 0 0 2024-02-15T18:27:07.709 INFO:tasks.workunit.client.1.smithi191.stdout:7/460: dread - d0/f80 zero size 2024-02-15T18:27:07.709 INFO:tasks.workunit.client.0.smithi042.stdout:9/886: write d6/da/d1d/f38 [3799384,101879] 0 2024-02-15T18:27:07.712 INFO:tasks.workunit.client.1.smithi191.stdout:5/249: rmdir d0/d23/d41/d28/d31/d38/d3b 0 2024-02-15T18:27:07.712 INFO:tasks.workunit.client.1.smithi191.stdout:5/250: chown d0/d23/d41/d28/d31/d38/l4f 654 1 2024-02-15T18:27:07.713 INFO:tasks.workunit.client.0.smithi042.stdout:9/887: write d6/da/d1d/d2d/d40/f43 [3128742,67823] 0 2024-02-15T18:27:07.713 INFO:tasks.workunit.client.0.smithi042.stdout:5/901: creat d0/d14/d88/df8/f13c x:0 0 0 2024-02-15T18:27:07.714 INFO:tasks.workunit.client.0.smithi042.stdout:5/902: readlink d0/d38/d53/d64/lf6 0 2024-02-15T18:27:07.714 INFO:tasks.workunit.client.0.smithi042.stdout:0/853: dwrite d4/d38/d39/df4/d114/dfe/fe1 [0,4194304] 0 2024-02-15T18:27:07.714 INFO:tasks.workunit.client.0.smithi042.stdout:0/854: dread - d4/d9/d34/dec/f113 zero size 2024-02-15T18:27:07.714 INFO:tasks.workunit.client.0.smithi042.stdout:0/855: getdents d4/d38/d77/d2e 0 2024-02-15T18:27:07.714 INFO:tasks.workunit.client.0.smithi042.stdout:0/856: read - d4/d9/d5a/dfc/d92/fd2 zero size 2024-02-15T18:27:07.714 INFO:tasks.workunit.client.1.smithi191.stdout:2/411: creat df/d12/d19/d1a/d35/d47/f8e x:0 0 0 2024-02-15T18:27:07.714 INFO:tasks.workunit.client.1.smithi191.stdout:2/412: fsync df/d12/d19/d1a/d35/f74 0 2024-02-15T18:27:07.715 INFO:tasks.workunit.client.1.smithi191.stdout:2/413: dread df/d12/f55 [0,4194304] 0 2024-02-15T18:27:07.715 INFO:tasks.workunit.client.1.smithi191.stdout:2/414: chown df/d12/d32/d4d 37976 1 2024-02-15T18:27:07.715 INFO:tasks.workunit.client.1.smithi191.stdout:2/415: fdatasync df/d12/d1f/d42/f43 0 2024-02-15T18:27:07.715 INFO:tasks.workunit.client.1.smithi191.stdout:2/416: fdatasync df/d12/d19/d1a/d35/d5b/f66 0 2024-02-15T18:27:07.715 INFO:tasks.workunit.client.1.smithi191.stdout:2/417: chown df/d12/d19/d1a/d35/d47/c48 404790802 1 2024-02-15T18:27:07.716 INFO:tasks.workunit.client.1.smithi191.stdout:2/418: dread - df/d12/d19/d1a/d35/f74 zero size 2024-02-15T18:27:07.719 INFO:tasks.workunit.client.0.smithi042.stdout:1/902: mkdir de/d54/dbf/dd0/d7c/dd6/ddf/d88/d157 0 2024-02-15T18:27:07.719 INFO:tasks.workunit.client.0.smithi042.stdout:1/903: chown de/d15/ff6 1002 1 2024-02-15T18:27:07.723 INFO:tasks.workunit.client.1.smithi191.stdout:2/419: dread df/d12/d19/d1a/f3d [0,4194304] 0 2024-02-15T18:27:07.724 INFO:tasks.workunit.client.0.smithi042.stdout:6/952: symlink d9/d11/d19/d35/d49/d79/l151 0 2024-02-15T18:27:07.725 INFO:tasks.workunit.client.0.smithi042.stdout:3/848: mknod d1/d11/d14/d4d/d6c/d7b/c117 0 2024-02-15T18:27:07.729 INFO:tasks.workunit.client.1.smithi191.stdout:2/420: dread df/f10 [0,4194304] 0 2024-02-15T18:27:07.729 INFO:tasks.workunit.client.1.smithi191.stdout:2/421: chown f7 1 1 2024-02-15T18:27:07.735 INFO:tasks.workunit.client.0.smithi042.stdout:6/953: dwrite d9/d11/d46/f91 [0,4194304] 0 2024-02-15T18:27:07.746 INFO:tasks.workunit.client.1.smithi191.stdout:4/457: creat d2/fad x:0 0 0 2024-02-15T18:27:07.746 INFO:tasks.workunit.client.1.smithi191.stdout:4/458: fsync d2/d5/d14/d47/d78/f68 0 2024-02-15T18:27:07.746 INFO:tasks.workunit.client.1.smithi191.stdout:4/459: fdatasync d2/d5/d7/d2f/d41/f73 0 2024-02-15T18:27:07.746 INFO:tasks.workunit.client.1.smithi191.stdout:4/460: dread - d2/d5/d7/d3e/d63/d8c/da4/fa2 zero size 2024-02-15T18:27:07.746 INFO:tasks.workunit.client.1.smithi191.stdout:4/461: truncate d2/d5/d14/fa8 278191 0 2024-02-15T18:27:07.746 INFO:tasks.workunit.client.1.smithi191.stdout:4/462: write d2/d5/d7/d90/d52/f7b [218666,3869] 0 2024-02-15T18:27:07.759 INFO:tasks.workunit.client.0.smithi042.stdout:7/893: creat d10/d15/d39/d6c/dba/d117/f12e x:0 0 0 2024-02-15T18:27:07.763 INFO:tasks.workunit.client.0.smithi042.stdout:2/834: sync 2024-02-15T18:27:07.770 INFO:tasks.workunit.client.1.smithi191.stdout:0/317: symlink d7/da/d17/l65 0 2024-02-15T18:27:07.770 INFO:tasks.workunit.client.0.smithi042.stdout:9/888: mkdir d6/d12e 0 2024-02-15T18:27:07.770 INFO:tasks.workunit.client.0.smithi042.stdout:9/889: truncate d6/da/d2f/d55/d107/f108 4990356 0 2024-02-15T18:27:07.771 INFO:tasks.workunit.client.1.smithi191.stdout:7/461: mknod d0/db/d26/d5f/d77/d35/d42/cad 0 2024-02-15T18:27:07.775 INFO:tasks.workunit.client.0.smithi042.stdout:5/903: mkdir d0/dda/dc7/de1/dd8/d13d 0 2024-02-15T18:27:07.776 INFO:tasks.workunit.client.0.smithi042.stdout:5/904: fdatasync d0/d50/f5f 0 2024-02-15T18:27:07.776 INFO:tasks.workunit.client.0.smithi042.stdout:5/905: stat d0/dda/l129 0 2024-02-15T18:27:07.776 INFO:tasks.workunit.client.0.smithi042.stdout:5/906: chown d0/d89/fbe 3 1 2024-02-15T18:27:07.777 INFO:tasks.workunit.client.1.smithi191.stdout:8/582: rename d3/d2c/d1a/d23/d2e/d5b/f9c to d3/d2c/d1a/d23/d3e/fd6 0 2024-02-15T18:27:07.778 INFO:tasks.workunit.client.0.smithi042.stdout:0/857: unlink d4/d9/cce 0 2024-02-15T18:27:07.780 INFO:tasks.workunit.client.0.smithi042.stdout:1/904: rename de/d15/d67/d63/f11a to de/d6d/d9e/df4/d13e/f158 0 2024-02-15T18:27:07.781 INFO:tasks.workunit.client.1.smithi191.stdout:4/463: mknod d2/d5/d7/d3e/d88/cae 0 2024-02-15T18:27:07.786 INFO:tasks.workunit.client.0.smithi042.stdout:3/849: unlink d1/d11/d14/d4d/d73/d89/dd9/c10b 0 2024-02-15T18:27:07.788 INFO:tasks.workunit.client.1.smithi191.stdout:0/318: getdents d7/da/d1a/d34 0 2024-02-15T18:27:07.790 INFO:tasks.workunit.client.1.smithi191.stdout:7/462: mknod d0/db/d65/dc/cae 0 2024-02-15T18:27:07.792 INFO:tasks.workunit.client.0.smithi042.stdout:3/850: read d1/f34 [3579965,34488] 0 2024-02-15T18:27:07.793 INFO:tasks.workunit.client.1.smithi191.stdout:8/583: link d3/d2c/d1a/d23/d2e/d47/cd3 d3/d2c/d1a/d23/d2e/d5b/d69/cd7 0 2024-02-15T18:27:07.794 INFO:tasks.workunit.client.0.smithi042.stdout:6/954: unlink d9/d11/d19/d35/d49/l4a 0 2024-02-15T18:27:07.794 INFO:tasks.workunit.client.0.smithi042.stdout:6/955: readlink d9/d11/d46/df8/l111 0 2024-02-15T18:27:07.796 INFO:tasks.workunit.client.0.smithi042.stdout:7/894: fdatasync d10/d15/d8f/da5/dad/d104/d65/f88 0 2024-02-15T18:27:07.797 INFO:tasks.workunit.client.0.smithi042.stdout:7/895: chown d10/d43/d57/f66 2149 1 2024-02-15T18:27:07.798 INFO:tasks.workunit.client.1.smithi191.stdout:4/464: rmdir d2/d5/d7/d90/d52 39 2024-02-15T18:27:07.799 INFO:tasks.workunit.client.1.smithi191.stdout:2/422: rename df/f16 to df/d12/d32/f8f 0 2024-02-15T18:27:07.800 INFO:tasks.workunit.client.0.smithi042.stdout:2/835: mkdir d0/d40/d109 0 2024-02-15T18:27:07.800 INFO:tasks.workunit.client.0.smithi042.stdout:5/907: dwrite d0/d38/d53/d91/d94/f10e [0,4194304] 0 2024-02-15T18:27:07.800 INFO:tasks.workunit.client.1.smithi191.stdout:0/319: dread d7/d1b/f46 [0,4194304] 0 2024-02-15T18:27:07.802 INFO:tasks.workunit.client.1.smithi191.stdout:8/584: symlink d3/d2c/d1a/d23/d3e/db2/ld8 0 2024-02-15T18:27:07.804 INFO:tasks.workunit.client.0.smithi042.stdout:9/890: link d6/da/d75/lb5 d6/da/d2f/d55/d73/d8d/dcf/l12f 0 2024-02-15T18:27:07.804 INFO:tasks.workunit.client.0.smithi042.stdout:9/891: dread d6/da/d2f/d69/f7d [0,4194304] 0 2024-02-15T18:27:07.805 INFO:tasks.workunit.client.0.smithi042.stdout:9/892: write d6/da/d1d/f24 [4952932,119312] 0 2024-02-15T18:27:07.806 INFO:tasks.workunit.client.0.smithi042.stdout:9/893: write d6/dd/d9/d59/f63 [2340907,127488] 0 2024-02-15T18:27:07.806 INFO:tasks.workunit.client.0.smithi042.stdout:4/875: sync 2024-02-15T18:27:07.809 INFO:tasks.workunit.client.1.smithi191.stdout:2/423: rmdir df/d12/d1e 39 2024-02-15T18:27:07.809 INFO:tasks.workunit.client.1.smithi191.stdout:2/424: chown df/d12/d19/d1a/d35/c62 724104 1 2024-02-15T18:27:07.809 INFO:tasks.workunit.client.1.smithi191.stdout:2/425: readlink df/d12/d1e/l7b 0 2024-02-15T18:27:07.809 INFO:tasks.workunit.client.1.smithi191.stdout:7/463: dwrite d0/db/d26/d5f/d77/d35/d42/fa7 [0,4194304] 0 2024-02-15T18:27:07.810 INFO:tasks.workunit.client.0.smithi042.stdout:0/858: getdents d4/d9/d34/d57/d88/dc9 0 2024-02-15T18:27:07.810 INFO:tasks.workunit.client.0.smithi042.stdout:0/859: readlink d4/lb1 0 2024-02-15T18:27:07.812 INFO:tasks.workunit.client.1.smithi191.stdout:8/585: dread d3/d2c/d1a/d23/d3e/fb4 [0,4194304] 0 2024-02-15T18:27:07.812 INFO:tasks.workunit.client.1.smithi191.stdout:0/320: mknod d7/da/d1a/d34/c66 0 2024-02-15T18:27:07.816 INFO:tasks.workunit.client.0.smithi042.stdout:3/851: unlink d1/d11/d2e/d4c/f5b 0 2024-02-15T18:27:07.816 INFO:tasks.workunit.client.0.smithi042.stdout:6/956: creat d9/d11/d46/dc3/f152 x:0 0 0 2024-02-15T18:27:07.816 INFO:tasks.workunit.client.0.smithi042.stdout:6/957: write d9/d11/d46/f122 [3924045,66108] 0 2024-02-15T18:27:07.822 INFO:tasks.workunit.client.1.smithi191.stdout:0/321: rename d7/d1b/d30/c51 to d7/da/d1a/d2c/d4c/c67 0 2024-02-15T18:27:07.822 INFO:tasks.workunit.client.0.smithi042.stdout:2/836: mknod d0/d8/d20/c10a 0 2024-02-15T18:27:07.823 INFO:tasks.workunit.client.0.smithi042.stdout:7/896: mknod d10/d43/d57/c12f 0 2024-02-15T18:27:07.823 INFO:tasks.workunit.client.0.smithi042.stdout:2/837: truncate d0/d40/fc3 658708 0 2024-02-15T18:27:07.823 INFO:tasks.workunit.client.0.smithi042.stdout:1/905: dread de/f58 [0,4194304] 0 2024-02-15T18:27:07.823 INFO:tasks.workunit.client.0.smithi042.stdout:2/838: write d0/d8/f70 [2913616,107248] 0 2024-02-15T18:27:07.826 INFO:tasks.workunit.client.0.smithi042.stdout:4/876: dread da/d11/d29/db8/fb9 [0,4194304] 0 2024-02-15T18:27:07.826 INFO:tasks.workunit.client.0.smithi042.stdout:4/877: chown da/d11/d85/db5/d103/d66/d78/d2f/d4d/cc1 495690 1 2024-02-15T18:27:07.826 INFO:tasks.workunit.client.0.smithi042.stdout:5/908: rename d0/d14/d88/df8/f11a to d0/dda/dc7/de1/f13e 0 2024-02-15T18:27:07.827 INFO:tasks.workunit.client.0.smithi042.stdout:9/894: creat d6/da/d2f/d6c/f130 x:0 0 0 2024-02-15T18:27:07.828 INFO:tasks.workunit.client.0.smithi042.stdout:0/860: mkdir d4/d38/d118 0 2024-02-15T18:27:07.828 INFO:tasks.workunit.client.0.smithi042.stdout:0/861: fdatasync d4/f7 0 2024-02-15T18:27:07.828 INFO:tasks.workunit.client.0.smithi042.stdout:0/862: chown d4/d9/d5a/dfc/l1b 33 1 2024-02-15T18:27:07.829 INFO:tasks.workunit.client.0.smithi042.stdout:3/852: creat d1/d11/f118 x:0 0 0 2024-02-15T18:27:07.829 INFO:tasks.workunit.client.0.smithi042.stdout:3/853: readlink d1/l95 0 2024-02-15T18:27:07.829 INFO:tasks.workunit.client.0.smithi042.stdout:3/854: readlink d1/de/lab 0 2024-02-15T18:27:07.831 INFO:tasks.workunit.client.0.smithi042.stdout:9/895: write d6/dd/d9/d59/dbf/d128/fdf [564581,115974] 0 2024-02-15T18:27:07.833 INFO:tasks.workunit.client.0.smithi042.stdout:5/909: dread d0/d38/d53/d91/db0/fb8 [0,4194304] 0 2024-02-15T18:27:07.833 INFO:tasks.workunit.client.0.smithi042.stdout:5/910: chown d0/d12/d65/l67 38397627 1 2024-02-15T18:27:07.834 INFO:tasks.workunit.client.0.smithi042.stdout:1/906: symlink de/d15/d95/l159 0 2024-02-15T18:27:07.834 INFO:tasks.workunit.client.0.smithi042.stdout:2/839: creat d0/d8/d20/d7d/d94/f10b x:0 0 0 2024-02-15T18:27:07.834 INFO:tasks.workunit.client.0.smithi042.stdout:1/907: stat de/d15/d1b/f25 0 2024-02-15T18:27:07.834 INFO:tasks.workunit.client.0.smithi042.stdout:1/908: dread - de/d6d/f101 zero size 2024-02-15T18:27:07.834 INFO:tasks.workunit.client.0.smithi042.stdout:1/909: fdatasync de/d15/d67/d74/f75 0 2024-02-15T18:27:07.834 INFO:tasks.workunit.client.0.smithi042.stdout:6/958: link d9/d3e/c104 d9/d11/d46/df8/c153 0 2024-02-15T18:27:07.834 INFO:tasks.workunit.client.0.smithi042.stdout:0/863: symlink d4/d38/d39/df4/d114/dfe/l119 0 2024-02-15T18:27:07.836 INFO:tasks.workunit.client.0.smithi042.stdout:9/896: symlink d6/da/d1d/d2d/d40/de0/l131 0 2024-02-15T18:27:07.837 INFO:tasks.workunit.client.0.smithi042.stdout:2/840: truncate d0/d8/d20/d41/f85 581007 0 2024-02-15T18:27:07.839 INFO:tasks.workunit.client.0.smithi042.stdout:6/959: mkdir d9/d11/d19/d1c/d11a/d154 0 2024-02-15T18:27:07.840 INFO:tasks.workunit.client.0.smithi042.stdout:6/960: chown d9/d11/d19/d1c/fdb 761239 1 2024-02-15T18:27:07.841 INFO:tasks.workunit.client.0.smithi042.stdout:0/864: rename d4/d38/d39/df4/d114/d6c/d70/fad to d4/d9/d62/f11a 0 2024-02-15T18:27:07.841 INFO:tasks.workunit.client.0.smithi042.stdout:0/865: write d4/d38/d5c/fc6 [932740,51572] 0 2024-02-15T18:27:07.842 INFO:tasks.workunit.client.0.smithi042.stdout:2/841: truncate d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/f6f 4193330 0 2024-02-15T18:27:07.842 INFO:tasks.workunit.client.0.smithi042.stdout:2/842: chown d0/d8/d10/d31/fc0 51415 1 2024-02-15T18:27:07.843 INFO:tasks.workunit.client.0.smithi042.stdout:6/961: symlink d9/d11/d19/d35/dc4/d123/l155 0 2024-02-15T18:27:07.846 INFO:tasks.workunit.client.0.smithi042.stdout:0/866: mknod d4/d38/d39/df4/c11b 0 2024-02-15T18:27:07.848 INFO:tasks.workunit.client.0.smithi042.stdout:3/855: dread d1/d11/f2c [4194304,4194304] 0 2024-02-15T18:27:07.848 INFO:tasks.workunit.client.0.smithi042.stdout:9/897: read d6/dd/d9/fc3 [1096307,77992] 0 2024-02-15T18:27:07.848 INFO:tasks.workunit.client.0.smithi042.stdout:0/867: getdents d4 0 2024-02-15T18:27:07.850 INFO:tasks.workunit.client.0.smithi042.stdout:0/868: getdents d4/d38/d77/d2e/dae 0 2024-02-15T18:27:07.850 INFO:tasks.workunit.client.0.smithi042.stdout:0/869: readlink d4/d9/d62/d74/lc3 0 2024-02-15T18:27:07.851 INFO:tasks.workunit.client.1.smithi191.stdout:4/465: dwrite d2/f2c [0,4194304] 0 2024-02-15T18:27:07.852 INFO:tasks.workunit.client.0.smithi042.stdout:8/781: sync 2024-02-15T18:27:07.853 INFO:tasks.workunit.client.0.smithi042.stdout:8/782: read - dd/de/d3c/fd5 zero size 2024-02-15T18:27:07.853 INFO:tasks.workunit.client.0.smithi042.stdout:3/856: read d1/de/d1a/f5a [4087579,105572] 0 2024-02-15T18:27:07.853 INFO:tasks.workunit.client.0.smithi042.stdout:3/857: chown d1/d11/f8f 3143 1 2024-02-15T18:27:07.855 INFO:tasks.workunit.client.0.smithi042.stdout:3/858: write d1/d11/d14/f53 [171684,106451] 0 2024-02-15T18:27:07.856 INFO:tasks.workunit.client.0.smithi042.stdout:0/870: symlink d4/d9/d8d/l11c 0 2024-02-15T18:27:07.857 INFO:tasks.workunit.client.1.smithi191.stdout:4/466: creat d2/d5/d7/d3e/faf x:0 0 0 2024-02-15T18:27:07.857 INFO:tasks.workunit.client.1.smithi191.stdout:4/467: fdatasync d2/d5/d7/f18 0 2024-02-15T18:27:07.859 INFO:tasks.workunit.client.0.smithi042.stdout:8/783: creat dd/de/fe0 x:0 0 0 2024-02-15T18:27:07.860 INFO:tasks.workunit.client.1.smithi191.stdout:2/426: dwrite df/d12/d19/d1a/f44 [0,4194304] 0 2024-02-15T18:27:07.860 INFO:tasks.workunit.client.1.smithi191.stdout:4/468: creat d2/d5/d7/d3e/d63/d8c/fb0 x:0 0 0 2024-02-15T18:27:07.860 INFO:tasks.workunit.client.1.smithi191.stdout:0/322: dwrite d7/da/f20 [0,4194304] 0 2024-02-15T18:27:07.860 INFO:tasks.workunit.client.1.smithi191.stdout:0/323: readlink d7/da/d1a/d34/d27/l38 0 2024-02-15T18:27:07.860 INFO:tasks.workunit.client.1.smithi191.stdout:8/586: dwrite d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/f98 [0,4194304] 0 2024-02-15T18:27:07.861 INFO:tasks.workunit.client.0.smithi042.stdout:6/962: dread d9/d11/d19/d1c/f12a [0,4194304] 0 2024-02-15T18:27:07.861 INFO:tasks.workunit.client.0.smithi042.stdout:6/963: read - d9/d11/d19/d35/dc4/fe3 zero size 2024-02-15T18:27:07.863 INFO:tasks.workunit.client.0.smithi042.stdout:4/878: dwrite da/d11/d85/db5/d103/d66/d78/d2f/d4d/f48 [0,4194304] 0 2024-02-15T18:27:07.865 INFO:tasks.workunit.client.1.smithi191.stdout:2/427: rmdir df/d12/d19/d1a/d35/d47/d4b 39 2024-02-15T18:27:07.866 INFO:tasks.workunit.client.0.smithi042.stdout:5/911: dwrite d0/dda/dc7/fcc [0,4194304] 0 2024-02-15T18:27:07.866 INFO:tasks.workunit.client.0.smithi042.stdout:4/879: write da/d11/f3c [1945968,94980] 0 2024-02-15T18:27:07.867 INFO:tasks.workunit.client.0.smithi042.stdout:4/880: fsync da/d3a/dc9/f128 0 2024-02-15T18:27:07.868 INFO:tasks.workunit.client.1.smithi191.stdout:2/428: write df/d12/d19/d1a/d35/d5b/f6e [3188145,115153] 0 2024-02-15T18:27:07.868 INFO:tasks.workunit.client.1.smithi191.stdout:2/429: write df/d12/d1f/f31 [1412793,492] 0 2024-02-15T18:27:07.868 INFO:tasks.workunit.client.0.smithi042.stdout:2/843: dwrite d0/d35/f95 [0,4194304] 0 2024-02-15T18:27:07.871 INFO:tasks.workunit.client.1.smithi191.stdout:0/324: creat d7/da/f68 x:0 0 0 2024-02-15T18:27:07.872 INFO:tasks.workunit.client.0.smithi042.stdout:8/784: mknod dd/de/da4/d67/d6f/ce1 0 2024-02-15T18:27:07.872 INFO:tasks.workunit.client.0.smithi042.stdout:8/785: dread - dd/de/da4/d67/d6f/d78/d9a/fb5 zero size 2024-02-15T18:27:07.879 INFO:tasks.workunit.client.0.smithi042.stdout:3/859: dwrite d1/d11/d14/d4d/d6c/df4/f102 [0,4194304] 0 2024-02-15T18:27:07.880 INFO:tasks.workunit.client.0.smithi042.stdout:1/910: write de/d15/d67/d46/f68 [394812,11185] 0 2024-02-15T18:27:07.885 INFO:tasks.workunit.client.0.smithi042.stdout:0/871: dwrite d4/d38/d77/d2e/dda/d101/f10d [0,4194304] 0 2024-02-15T18:27:07.886 INFO:tasks.workunit.client.0.smithi042.stdout:8/786: dwrite dd/de/fe0 [0,4194304] 0 2024-02-15T18:27:07.888 INFO:tasks.workunit.client.0.smithi042.stdout:6/964: rmdir d9/d11/d19/d1c/d31/d9b/db4/d14e 39 2024-02-15T18:27:07.889 INFO:tasks.workunit.client.1.smithi191.stdout:8/587: mknod d3/d2c/d1a/d23/d2e/cd9 0 2024-02-15T18:27:07.892 INFO:tasks.workunit.client.1.smithi191.stdout:2/430: mknod df/d12/d1e/d6f/c90 0 2024-02-15T18:27:07.892 INFO:tasks.workunit.client.1.smithi191.stdout:2/431: write df/d12/d19/d1a/f3d [5118922,23347] 0 2024-02-15T18:27:07.893 INFO:tasks.workunit.client.1.smithi191.stdout:0/325: creat d7/d3b/f69 x:0 0 0 2024-02-15T18:27:07.897 INFO:tasks.workunit.client.1.smithi191.stdout:2/432: link df/d12/d19/d1a/c73 df/d12/d19/d1a/d35/d5b/c91 0 2024-02-15T18:27:07.898 INFO:tasks.workunit.client.1.smithi191.stdout:0/326: creat d7/f6a x:0 0 0 2024-02-15T18:27:07.899 INFO:tasks.workunit.client.1.smithi191.stdout:0/327: write d7/da/d17/d31/f41 [883688,95822] 0 2024-02-15T18:27:07.901 INFO:tasks.workunit.client.1.smithi191.stdout:0/328: getdents d7/d4a 0 2024-02-15T18:27:07.901 INFO:tasks.workunit.client.0.smithi042.stdout:6/965: dread d9/d11/d19/d35/d55/f65 [4194304,4194304] 0 2024-02-15T18:27:07.904 INFO:tasks.workunit.client.1.smithi191.stdout:0/329: creat d7/d3b/f6b x:0 0 0 2024-02-15T18:27:07.904 INFO:tasks.workunit.client.1.smithi191.stdout:2/433: dread df/d12/d19/d1a/f21 [0,4194304] 0 2024-02-15T18:27:07.905 INFO:tasks.workunit.client.0.smithi042.stdout:5/912: mkdir d0/d39/d13f 0 2024-02-15T18:27:07.905 INFO:tasks.workunit.client.1.smithi191.stdout:0/330: write d7/f39 [396421,65093] 0 2024-02-15T18:27:07.906 INFO:tasks.workunit.client.0.smithi042.stdout:4/881: mknod da/d11/d85/db5/d103/d66/d133/c136 0 2024-02-15T18:27:07.906 INFO:tasks.workunit.client.1.smithi191.stdout:2/434: write df/d12/f22 [1384765,42759] 0 2024-02-15T18:27:07.907 INFO:tasks.workunit.client.1.smithi191.stdout:0/331: dread d7/da/d1a/d34/f5e [0,4194304] 0 2024-02-15T18:27:07.908 INFO:tasks.workunit.client.0.smithi042.stdout:3/860: symlink d1/d11/d14/d4d/d6c/l119 0 2024-02-15T18:27:07.908 INFO:tasks.workunit.client.0.smithi042.stdout:3/861: fdatasync d1/d11/d14/d4d/d73/d89/f116 0 2024-02-15T18:27:07.909 INFO:tasks.workunit.client.0.smithi042.stdout:1/911: creat de/d54/dbf/dd0/d7c/f15a x:0 0 0 2024-02-15T18:27:07.912 INFO:tasks.workunit.client.1.smithi191.stdout:8/588: dread d3/fd [0,4194304] 0 2024-02-15T18:27:08.007 INFO:tasks.workunit.client.1.smithi191.stdout:0/332: mkdir d7/da/d6c 0 2024-02-15T18:27:08.014 INFO:tasks.workunit.client.0.smithi042.stdout:0/872: unlink d4/d9/d34/d57/d88/fcf 0 2024-02-15T18:27:08.014 INFO:tasks.workunit.client.0.smithi042.stdout:0/873: readlink d4/d38/d39/df4/d114/dfe/l119 0 2024-02-15T18:27:08.017 INFO:tasks.workunit.client.1.smithi191.stdout:8/589: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/fda x:0 0 0 2024-02-15T18:27:08.022 INFO:tasks.workunit.client.0.smithi042.stdout:8/787: mkdir dd/d11/d75/de2 0 2024-02-15T18:27:08.024 INFO:tasks.workunit.client.1.smithi191.stdout:0/333: creat d7/da/d1a/d34/f6d x:0 0 0 2024-02-15T18:27:08.024 INFO:tasks.workunit.client.1.smithi191.stdout:0/334: write d7/d3b/f6b [375916,22272] 0 2024-02-15T18:27:08.024 INFO:tasks.workunit.client.1.smithi191.stdout:0/335: write d7/da/f20 [4235619,130824] 0 2024-02-15T18:27:08.027 INFO:tasks.workunit.client.1.smithi191.stdout:8/590: readlink d3/l71 0 2024-02-15T18:27:08.027 INFO:tasks.workunit.client.1.smithi191.stdout:8/591: stat d3/d2c/d1a/d23/d2e/d64/f8d 0 2024-02-15T18:27:08.029 INFO:tasks.workunit.client.0.smithi042.stdout:4/882: truncate da/d11/d27/f69 1545913 0 2024-02-15T18:27:08.029 INFO:tasks.workunit.client.0.smithi042.stdout:4/883: readlink da/d11/d85/laf 0 2024-02-15T18:27:08.031 INFO:tasks.workunit.client.1.smithi191.stdout:0/336: link d7/da/d17/d31/f41 d7/d1b/f6e 0 2024-02-15T18:27:08.031 INFO:tasks.workunit.client.1.smithi191.stdout:0/337: write d7/d1b/d30/f43 [930451,86035] 0 2024-02-15T18:27:08.032 INFO:tasks.workunit.client.0.smithi042.stdout:2/844: truncate d0/d35/f95 4018016 0 2024-02-15T18:27:08.035 INFO:tasks.workunit.client.1.smithi191.stdout:8/592: dread d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/fc8 [0,4194304] 0 2024-02-15T18:27:08.035 INFO:tasks.workunit.client.1.smithi191.stdout:8/593: write d3/d2c/d1a/d23/d2a/f35 [1209158,70294] 0 2024-02-15T18:27:08.035 INFO:tasks.workunit.client.0.smithi042.stdout:3/862: mkdir d1/d11/d14/d4d/d73/d89/da8/d11a 0 2024-02-15T18:27:08.035 INFO:tasks.workunit.client.0.smithi042.stdout:3/863: fsync d1/d11/d14/d4d/d73/f82 0 2024-02-15T18:27:08.036 INFO:tasks.workunit.client.0.smithi042.stdout:3/864: fsync d1/d11/d7a/ff8 0 2024-02-15T18:27:08.036 INFO:tasks.workunit.client.1.smithi191.stdout:0/338: creat d7/da/d1a/d34/d27/f6f x:0 0 0 2024-02-15T18:27:08.036 INFO:tasks.workunit.client.1.smithi191.stdout:0/339: stat d7/da/d17/d31 0 2024-02-15T18:27:08.038 INFO:tasks.workunit.client.0.smithi042.stdout:7/897: sync 2024-02-15T18:27:08.038 INFO:tasks.workunit.client.0.smithi042.stdout:9/898: sync 2024-02-15T18:27:08.040 INFO:tasks.workunit.client.0.smithi042.stdout:1/912: rename de/d15/d67/f26 to de/d54/dbf/dd0/d7c/dd6/ddf/d88/d157/f15b 0 2024-02-15T18:27:08.040 INFO:tasks.workunit.client.1.smithi191.stdout:8/594: creat d3/d2c/d1a/fdb x:0 0 0 2024-02-15T18:27:08.042 INFO:tasks.workunit.client.0.smithi042.stdout:0/874: symlink d4/d9/d5a/dfc/l11d 0 2024-02-15T18:27:08.043 INFO:tasks.workunit.client.0.smithi042.stdout:0/875: dread d4/d9/d5a/dfc/f83 [0,4194304] 0 2024-02-15T18:27:08.044 INFO:tasks.workunit.client.1.smithi191.stdout:0/340: mknod d7/d4a/c70 0 2024-02-15T18:27:08.044 INFO:tasks.workunit.client.0.smithi042.stdout:8/788: symlink dd/de/d33/d66/le3 0 2024-02-15T18:27:08.044 INFO:tasks.workunit.client.0.smithi042.stdout:8/789: fsync dd/de/d33/d3b/f46 0 2024-02-15T18:27:08.046 INFO:tasks.workunit.client.0.smithi042.stdout:5/913: mkdir d0/d38/d53/d91/db0/d140 0 2024-02-15T18:27:08.047 INFO:tasks.workunit.client.1.smithi191.stdout:0/341: rename d7/da/f68 to d7/d3b/f71 0 2024-02-15T18:27:08.050 INFO:tasks.workunit.client.1.smithi191.stdout:0/342: unlink d7/d4a/f56 0 2024-02-15T18:27:08.050 INFO:tasks.workunit.client.0.smithi042.stdout:8/790: dread dd/de/d33/d66/f5f [0,4194304] 0 2024-02-15T18:27:08.050 INFO:tasks.workunit.client.0.smithi042.stdout:8/791: chown dd/de/d33/d66/d4e/c4a 0 1 2024-02-15T18:27:08.050 INFO:tasks.workunit.client.0.smithi042.stdout:8/792: chown dd/f13 74 1 2024-02-15T18:27:08.051 INFO:tasks.workunit.client.0.smithi042.stdout:3/865: symlink d1/d9c/l11b 0 2024-02-15T18:27:08.052 INFO:tasks.workunit.client.1.smithi191.stdout:0/343: symlink d7/da/d17/d31/d5f/l72 0 2024-02-15T18:27:08.054 INFO:tasks.workunit.client.0.smithi042.stdout:7/898: symlink d10/d15/d39/de3/d116/d11e/l130 0 2024-02-15T18:27:08.054 INFO:tasks.workunit.client.0.smithi042.stdout:7/899: truncate d10/d15/d39/dd9/dec/f11b 341865 0 2024-02-15T18:27:08.057 INFO:tasks.workunit.client.0.smithi042.stdout:2/845: rename d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/f4d to d0/d8/d20/d41/dbe/f10c 0 2024-02-15T18:27:08.057 INFO:tasks.workunit.client.0.smithi042.stdout:1/913: rmdir de/d15/d2c/d136 39 2024-02-15T18:27:08.058 INFO:tasks.workunit.client.0.smithi042.stdout:0/876: creat d4/d38/d77/d2e/dae/ddf/f11e x:0 0 0 2024-02-15T18:27:08.058 INFO:tasks.workunit.client.0.smithi042.stdout:0/877: chown d4/d9/d34/f40 953 1 2024-02-15T18:27:08.058 INFO:tasks.workunit.client.0.smithi042.stdout:5/914: mknod d0/d50/c141 0 2024-02-15T18:27:08.060 INFO:tasks.workunit.client.0.smithi042.stdout:8/793: unlink dd/de/fb9 0 2024-02-15T18:27:08.062 INFO:tasks.workunit.client.0.smithi042.stdout:3/866: link d1/d11/d2e/l108 d1/d11/d14/d4d/d6c/df4/l11c 0 2024-02-15T18:27:08.062 INFO:tasks.workunit.client.0.smithi042.stdout:3/867: dread - d1/d11/d14/dbf/f112 zero size 2024-02-15T18:27:08.062 INFO:tasks.workunit.client.0.smithi042.stdout:3/868: dread - d1/de/fe2 zero size 2024-02-15T18:27:08.063 INFO:tasks.workunit.client.0.smithi042.stdout:7/900: rename d10/d43/cb2 to d10/d15/d108/c131 0 2024-02-15T18:27:08.063 INFO:tasks.workunit.client.0.smithi042.stdout:1/914: truncate de/d54/dbf/fe1 898362 0 2024-02-15T18:27:08.064 INFO:tasks.workunit.client.0.smithi042.stdout:7/901: getdents d10/d15/d39/de3/d116 0 2024-02-15T18:27:08.065 INFO:tasks.workunit.client.0.smithi042.stdout:7/902: stat d10/d49/d6d/d78/d83/fe8 0 2024-02-15T18:27:08.065 INFO:tasks.workunit.client.0.smithi042.stdout:7/903: chown d10/d15/d39/caf 353 1 2024-02-15T18:27:08.066 INFO:tasks.workunit.client.0.smithi042.stdout:1/915: symlink de/d54/d9a/dc8/dd3/l15c 0 2024-02-15T18:27:08.066 INFO:tasks.workunit.client.0.smithi042.stdout:1/916: fsync de/d54/dbf/dd0/d7c/f118 0 2024-02-15T18:27:08.066 INFO:tasks.workunit.client.0.smithi042.stdout:1/917: stat de/d54/l6c 0 2024-02-15T18:27:08.067 INFO:tasks.workunit.client.0.smithi042.stdout:7/904: chown d10/d15/d8f/da5/dad/d104/d23/d51 3753128 1 2024-02-15T18:27:08.067 INFO:tasks.workunit.client.0.smithi042.stdout:7/905: truncate d10/d49/f121 245559 0 2024-02-15T18:27:08.067 INFO:tasks.workunit.client.0.smithi042.stdout:7/906: chown d10/d15/d8f/da5/dad/de0 603 1 2024-02-15T18:27:08.069 INFO:tasks.workunit.client.1.smithi191.stdout:0/344: dread d7/da/f16 [0,4194304] 0 2024-02-15T18:27:08.069 INFO:tasks.workunit.client.0.smithi042.stdout:8/794: write dd/de/d12/f45 [4158071,66495] 0 2024-02-15T18:27:08.069 INFO:tasks.workunit.client.0.smithi042.stdout:8/795: write dd/de/d12/f4c [1295204,4065] 0 2024-02-15T18:27:08.069 INFO:tasks.workunit.client.0.smithi042.stdout:0/878: dread d4/d9/d5a/dfc/ffd [0,4194304] 0 2024-02-15T18:27:08.069 INFO:tasks.workunit.client.0.smithi042.stdout:1/918: getdents de/d15/d1b/d44 0 2024-02-15T18:27:08.070 INFO:tasks.workunit.client.0.smithi042.stdout:3/869: dread d1/d11/d2e/fd7 [0,4194304] 0 2024-02-15T18:27:08.072 INFO:tasks.workunit.client.0.smithi042.stdout:7/907: getdents d10/d15/d8f/da5/dad/d104/d23 0 2024-02-15T18:27:08.072 INFO:tasks.workunit.client.0.smithi042.stdout:0/879: read d4/d9/d5a/dfc/d92/fca [2665007,41505] 0 2024-02-15T18:27:08.075 INFO:tasks.workunit.client.0.smithi042.stdout:8/796: rename dd/de/f2a to dd/fe4 0 2024-02-15T18:27:08.076 INFO:tasks.workunit.client.0.smithi042.stdout:3/870: mkdir d1/d11/d2e/d11d 0 2024-02-15T18:27:08.078 INFO:tasks.workunit.client.0.smithi042.stdout:4/884: dwrite da/d3b/fda [0,4194304] 0 2024-02-15T18:27:08.078 INFO:tasks.workunit.client.0.smithi042.stdout:0/880: unlink d4/d9/d62/fa6 0 2024-02-15T18:27:08.090 INFO:tasks.workunit.client.0.smithi042.stdout:9/899: dwrite d6/dd/d9/da1/fdc [4194304,4194304] 0 2024-02-15T18:27:08.093 INFO:tasks.workunit.client.1.smithi191.stdout:8/595: dwrite d3/d2c/d1a/d23/d2e/d5b/fc9 [0,4194304] 0 2024-02-15T18:27:08.093 INFO:tasks.workunit.client.1.smithi191.stdout:8/596: stat d3/d2c/d1a/d23/d2e/d64/dca 0 2024-02-15T18:27:08.093 INFO:tasks.workunit.client.0.smithi042.stdout:6/966: sync 2024-02-15T18:27:08.095 INFO:tasks.workunit.client.0.smithi042.stdout:2/846: dwrite d0/d8/d10/d31/d3a/d53/f58 [0,4194304] 0 2024-02-15T18:27:08.099 INFO:tasks.workunit.client.0.smithi042.stdout:5/915: dwrite d0/d22/fee [0,4194304] 0 2024-02-15T18:27:08.102 INFO:tasks.workunit.client.0.smithi042.stdout:7/908: dwrite d10/d43/d57/f66 [0,4194304] 0 2024-02-15T18:27:08.102 INFO:tasks.workunit.client.0.smithi042.stdout:7/909: truncate d10/d15/d8f/da5/dad/d104/d23/fdd 993595 0 2024-02-15T18:27:08.112 INFO:tasks.workunit.client.0.smithi042.stdout:6/967: rename d9/d11/d19/d1c/d31/dd2 to d9/d11/d46/dc3/d156 0 2024-02-15T18:27:08.115 INFO:tasks.workunit.client.1.smithi191.stdout:8/597: symlink d3/d2c/d1a/d23/d3e/db2/ldc 0 2024-02-15T18:27:08.115 INFO:tasks.workunit.client.1.smithi191.stdout:8/598: read d3/d2c/d1a/d23/d2a/f49 [58669,43830] 0 2024-02-15T18:27:08.116 INFO:tasks.workunit.client.0.smithi042.stdout:2/847: creat d0/d8/d10/d31/d79/dbd/f10d x:0 0 0 2024-02-15T18:27:08.119 INFO:tasks.workunit.client.0.smithi042.stdout:5/916: creat d0/f142 x:0 0 0 2024-02-15T18:27:08.121 INFO:tasks.workunit.client.0.smithi042.stdout:7/910: mknod d10/d15/c132 0 2024-02-15T18:27:08.121 INFO:tasks.workunit.client.0.smithi042.stdout:7/911: write d10/d15/d8f/da5/dad/d104/d65/ff7 [896182,109312] 0 2024-02-15T18:27:08.121 INFO:tasks.workunit.client.0.smithi042.stdout:6/968: truncate d9/f2f 3651476 0 2024-02-15T18:27:08.123 INFO:tasks.workunit.client.0.smithi042.stdout:7/912: symlink d10/d49/d6d/l133 0 2024-02-15T18:27:08.124 INFO:tasks.workunit.client.0.smithi042.stdout:6/969: unlink d9/d11/d19/fb5 0 2024-02-15T18:27:08.129 INFO:tasks.workunit.client.0.smithi042.stdout:1/919: dwrite de/d54/d9a/dc8/fdb [0,4194304] 0 2024-02-15T18:27:08.131 INFO:tasks.workunit.client.0.smithi042.stdout:1/920: creat de/d15/d95/f15d x:0 0 0 2024-02-15T18:27:08.131 INFO:tasks.workunit.client.0.smithi042.stdout:1/921: chown de/d6d/d9e/d109/f121 297488640 1 2024-02-15T18:27:08.132 INFO:tasks.workunit.client.0.smithi042.stdout:1/922: getdents de/d54/dbf/dd0/d7c/dd6/ddf/d88 0 2024-02-15T18:27:08.135 INFO:tasks.workunit.client.0.smithi042.stdout:1/923: mkdir de/d54/dbf/dd0/d7c/dd6/de8/dec/d100/d15e 0 2024-02-15T18:27:08.135 INFO:tasks.workunit.client.0.smithi042.stdout:1/924: dread - de/d54/d9a/dc8/dd3/f14d zero size 2024-02-15T18:27:08.135 INFO:tasks.workunit.client.0.smithi042.stdout:1/925: stat de/d15/d2c/f5f 0 2024-02-15T18:27:08.138 INFO:tasks.workunit.client.1.smithi191.stdout:0/345: dwrite d7/da/d1a/d34/f5e [0,4194304] 0 2024-02-15T18:27:08.140 INFO:tasks.workunit.client.0.smithi042.stdout:4/885: dread da/d11/d27/f89 [0,4194304] 0 2024-02-15T18:27:08.140 INFO:tasks.workunit.client.0.smithi042.stdout:4/886: chown da/d3b/d41/ff0 14519 1 2024-02-15T18:27:08.140 INFO:tasks.workunit.client.0.smithi042.stdout:8/797: dwrite dd/de/d94/f98 [0,4194304] 0 2024-02-15T18:27:08.140 INFO:tasks.workunit.client.0.smithi042.stdout:6/970: dread d9/d11/d19/d1c/d31/d53/fc2 [0,4194304] 0 2024-02-15T18:27:08.140 INFO:tasks.workunit.client.0.smithi042.stdout:8/798: stat dd/de/d94 0 2024-02-15T18:27:08.149 INFO:tasks.workunit.client.0.smithi042.stdout:3/871: dwrite d1/de/d19/f3e [0,4194304] 0 2024-02-15T18:27:08.149 INFO:tasks.workunit.client.0.smithi042.stdout:7/913: dread d10/d15/f3d [0,4194304] 0 2024-02-15T18:27:08.153 INFO:tasks.workunit.client.0.smithi042.stdout:6/971: write d9/d11/d19/d35/d55/fac [7495548,57859] 0 2024-02-15T18:27:08.153 INFO:tasks.workunit.client.0.smithi042.stdout:9/900: dwrite d6/da/d2f/d55/d107/f108 [4194304,4194304] 0 2024-02-15T18:27:08.153 INFO:tasks.workunit.client.0.smithi042.stdout:6/972: chown d9/d11/d19/d35/d55/la6 1 1 2024-02-15T18:27:08.153 INFO:tasks.workunit.client.0.smithi042.stdout:9/901: readlink d6/da/d1d/d2d/d40/d48/l66 0 2024-02-15T18:27:08.156 INFO:tasks.workunit.client.0.smithi042.stdout:2/848: dwrite d0/d8/d20/d7d/d7e/dc7/d8b/dd7/fe2 [0,4194304] 0 2024-02-15T18:27:08.156 INFO:tasks.workunit.client.0.smithi042.stdout:2/849: fsync d0/d8/d10/d7b/fa4 0 2024-02-15T18:27:08.161 INFO:tasks.workunit.client.1.smithi191.stdout:8/599: dwrite d3/d5e/f80 [0,4194304] 0 2024-02-15T18:27:08.166 INFO:tasks.workunit.client.0.smithi042.stdout:5/917: dwrite d0/dda/dc7/d52/f9e [0,4194304] 0 2024-02-15T18:27:08.175 INFO:tasks.workunit.client.0.smithi042.stdout:4/887: dwrite da/d11/d85/db5/d103/d66/d125/fa1 [0,4194304] 0 2024-02-15T18:27:08.175 INFO:tasks.workunit.client.0.smithi042.stdout:4/888: write da/d2b/d7e/fa6 [1194620,52405] 0 2024-02-15T18:27:08.177 INFO:tasks.workunit.client.0.smithi042.stdout:1/926: dwrite de/d6d/d9e/dcd/fe4 [8388608,4194304] 0 2024-02-15T18:27:08.192 INFO:tasks.workunit.client.1.smithi191.stdout:0/346: creat d7/da/d17/d31/d5f/f73 x:0 0 0 2024-02-15T18:27:08.193 INFO:tasks.workunit.client.0.smithi042.stdout:3/872: dwrite d1/d11/d14/d4d/d73/f8b [0,4194304] 0 2024-02-15T18:27:08.194 INFO:tasks.workunit.client.0.smithi042.stdout:6/973: dwrite d9/d11/d19/d1c/d31/f3a [0,4194304] 0 2024-02-15T18:27:08.194 INFO:tasks.workunit.client.0.smithi042.stdout:6/974: stat d9/d11/d19/d35/d55/d92/dfc/c144 0 2024-02-15T18:27:08.194 INFO:tasks.workunit.client.0.smithi042.stdout:6/975: stat d9/d11/d46/dc3/d156/lee 0 2024-02-15T18:27:08.194 INFO:tasks.workunit.client.0.smithi042.stdout:6/976: readlink d9/d11/d19/d35/dc4/d123/l155 0 2024-02-15T18:27:08.199 INFO:tasks.workunit.client.0.smithi042.stdout:2/850: dwrite d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/d44/f84 [0,4194304] 0 2024-02-15T18:27:08.224 INFO:tasks.workunit.client.0.smithi042.stdout:2/851: dwrite d0/d8/d20/d7d/d7e/f100 [0,4194304] 0 2024-02-15T18:27:08.238 INFO:tasks.workunit.client.0.smithi042.stdout:2/852: dread d0/d8/d20/d7d/d7e/dc7/f73 [0,4194304] 0 2024-02-15T18:27:08.238 INFO:tasks.workunit.client.0.smithi042.stdout:2/853: write d0/d8/d20/d41/f85 [1242143,113543] 0 2024-02-15T18:27:08.243 INFO:tasks.workunit.client.0.smithi042.stdout:2/854: dread d0/d8/d10/d5a/fab [4194304,4194304] 0 2024-02-15T18:27:08.252 INFO:tasks.workunit.client.0.smithi042.stdout:8/799: creat dd/de/d12/fe5 x:0 0 0 2024-02-15T18:27:08.256 INFO:tasks.workunit.client.0.smithi042.stdout:2/855: dread d0/d8/d10/d31/d3a/d53/f5d [4194304,4194304] 0 2024-02-15T18:27:08.300 INFO:tasks.workunit.client.1.smithi191.stdout:3/361: sync 2024-02-15T18:27:08.300 INFO:tasks.workunit.client.1.smithi191.stdout:3/362: fsync d6/d27/f3f 0 2024-02-15T18:27:08.301 INFO:tasks.workunit.client.1.smithi191.stdout:3/363: dread - d6/d3b/d30/f5c zero size 2024-02-15T18:27:08.301 INFO:tasks.workunit.client.1.smithi191.stdout:6/408: sync 2024-02-15T18:27:08.301 INFO:tasks.workunit.client.1.smithi191.stdout:5/251: sync 2024-02-15T18:27:08.301 INFO:tasks.workunit.client.1.smithi191.stdout:9/483: sync 2024-02-15T18:27:08.301 INFO:tasks.workunit.client.1.smithi191.stdout:1/566: sync 2024-02-15T18:27:08.303 INFO:tasks.workunit.client.1.smithi191.stdout:1/567: write d0/d8/df/f85 [412743,69302] 0 2024-02-15T18:27:08.313 INFO:tasks.workunit.client.1.smithi191.stdout:6/409: dread d8/d19/f35 [0,4194304] 0 2024-02-15T18:27:08.322 INFO:tasks.workunit.client.1.smithi191.stdout:6/410: write d8/d9/d2b/f2e [2318110,73134] 0 2024-02-15T18:27:08.323 INFO:tasks.workunit.client.1.smithi191.stdout:6/411: stat d8/d19/f1f 0 2024-02-15T18:27:08.326 INFO:tasks.workunit.client.1.smithi191.stdout:7/464: sync 2024-02-15T18:27:08.326 INFO:tasks.workunit.client.1.smithi191.stdout:7/465: write d0/db/f2f [712680,107709] 0 2024-02-15T18:27:08.328 INFO:tasks.workunit.client.1.smithi191.stdout:4/469: sync 2024-02-15T18:27:08.328 INFO:tasks.workunit.client.1.smithi191.stdout:2/435: sync 2024-02-15T18:27:08.328 INFO:tasks.workunit.client.1.smithi191.stdout:2/436: chown df/d12/d1e/c20 3119946 1 2024-02-15T18:27:08.331 INFO:tasks.workunit.client.1.smithi191.stdout:7/466: write d0/db/d65/f3e [55422,15683] 0 2024-02-15T18:27:08.336 INFO:tasks.workunit.client.1.smithi191.stdout:4/470: dread d2/d5/d7/d3e/f77 [0,4194304] 0 2024-02-15T18:27:08.383 INFO:tasks.workunit.client.0.smithi042.stdout:5/918: mkdir d0/d38/d53/d64/de5/d143 0 2024-02-15T18:27:08.385 INFO:tasks.workunit.client.1.smithi191.stdout:3/364: creat d6/d3b/d42/d49/d54/f7f x:0 0 0 2024-02-15T18:27:08.402 INFO:tasks.workunit.client.0.smithi042.stdout:9/902: symlink d6/da/d2f/d55/d73/l132 0 2024-02-15T18:27:08.402 INFO:tasks.workunit.client.0.smithi042.stdout:9/903: readlink d6/da/d2f/d6c/lc7 0 2024-02-15T18:27:08.405 INFO:tasks.workunit.client.0.smithi042.stdout:9/904: dread d6/da/d2f/f6a [0,4194304] 0 2024-02-15T18:27:08.407 INFO:tasks.workunit.client.0.smithi042.stdout:7/914: unlink d10/c100 0 2024-02-15T18:27:08.409 INFO:tasks.workunit.client.1.smithi191.stdout:0/347: link d7/da/d1a/d34/d27/l38 d7/da/d1a/l74 0 2024-02-15T18:27:08.413 INFO:tasks.workunit.client.0.smithi042.stdout:7/915: dread d10/f7e [0,4194304] 0 2024-02-15T18:27:08.413 INFO:tasks.workunit.client.0.smithi042.stdout:7/916: stat d10/d15/d8f/da5/dad/d104/d65/da7/cb9 0 2024-02-15T18:27:08.414 INFO:tasks.workunit.client.0.smithi042.stdout:7/917: fsync d10/d43/d57/f79 0 2024-02-15T18:27:08.414 INFO:tasks.workunit.client.0.smithi042.stdout:7/918: readlink d10/d15/d39/de3/lfa 0 2024-02-15T18:27:08.417 INFO:tasks.workunit.client.1.smithi191.stdout:5/252: rename d0/d23/d41/d28/d31/d38 to d0/d23/d41/d32/d52 0 2024-02-15T18:27:08.417 INFO:tasks.workunit.client.1.smithi191.stdout:5/253: fdatasync d0/d23/d41/d28/d31/f33 0 2024-02-15T18:27:08.419 INFO:tasks.workunit.client.0.smithi042.stdout:4/889: symlink da/d11/d29/d4a/d4f/dcf/l137 0 2024-02-15T18:27:08.420 INFO:tasks.workunit.client.1.smithi191.stdout:9/484: symlink d0/d7/dc/d30/d4f/la8 0 2024-02-15T18:27:08.424 INFO:tasks.workunit.client.0.smithi042.stdout:1/927: unlink de/d54/dbf/dd0/f143 0 2024-02-15T18:27:08.424 INFO:tasks.workunit.client.0.smithi042.stdout:1/928: write de/d15/d67/d89/f138 [190358,8461] 0 2024-02-15T18:27:08.425 INFO:tasks.workunit.client.1.smithi191.stdout:6/412: creat d8/d19/d6b/f75 x:0 0 0 2024-02-15T18:27:08.425 INFO:tasks.workunit.client.1.smithi191.stdout:6/413: fsync d8/d19/f1d 0 2024-02-15T18:27:08.429 INFO:tasks.workunit.client.1.smithi191.stdout:2/437: creat df/d12/d32/f92 x:0 0 0 2024-02-15T18:27:08.429 INFO:tasks.workunit.client.1.smithi191.stdout:9/485: dread d0/d7/f9 [0,4194304] 0 2024-02-15T18:27:08.430 INFO:tasks.workunit.client.0.smithi042.stdout:3/873: mkdir d1/d11e 0 2024-02-15T18:27:08.430 INFO:tasks.workunit.client.0.smithi042.stdout:3/874: chown d1/de/f42 223960 1 2024-02-15T18:27:08.431 INFO:tasks.workunit.client.1.smithi191.stdout:7/467: creat d0/db/d65/d11/d5a/d66/d1e/faf x:0 0 0 2024-02-15T18:27:08.432 INFO:tasks.workunit.client.1.smithi191.stdout:4/471: symlink d2/d5/d7/d90/da5/lb1 0 2024-02-15T18:27:08.433 INFO:tasks.workunit.client.0.smithi042.stdout:7/919: dwrite d10/d43/f58 [0,4194304] 0 2024-02-15T18:27:08.434 INFO:tasks.workunit.client.1.smithi191.stdout:3/365: truncate d6/d27/f3f 274983 0 2024-02-15T18:27:08.434 INFO:tasks.workunit.client.1.smithi191.stdout:3/366: chown d6/d3b/d30/f7c 0 1 2024-02-15T18:27:08.434 INFO:tasks.workunit.client.1.smithi191.stdout:3/367: stat d6/d1b/d60/d73/f75 0 2024-02-15T18:27:08.434 INFO:tasks.workunit.client.0.smithi042.stdout:6/977: write d9/d11/d1d/fd1 [193399,118148] 0 2024-02-15T18:27:08.435 INFO:tasks.workunit.client.1.smithi191.stdout:0/348: link d7/da/d1a/d34/d27/l2e d7/d1b/l75 0 2024-02-15T18:27:08.435 INFO:tasks.workunit.client.0.smithi042.stdout:3/875: write d1/de/f2d [9542609,123738] 0 2024-02-15T18:27:08.437 INFO:tasks.workunit.client.1.smithi191.stdout:5/254: creat d0/d23/d41/d28/d31/f53 x:0 0 0 2024-02-15T18:27:08.439 INFO:tasks.workunit.client.1.smithi191.stdout:6/414: mkdir d8/d19/d6b/d76 0 2024-02-15T18:27:08.439 INFO:tasks.workunit.client.1.smithi191.stdout:6/415: chown d8/d9/cf 36 1 2024-02-15T18:27:08.443 INFO:tasks.workunit.client.1.smithi191.stdout:9/486: write d0/d7/dc/d30/d3b/d56/d59/f97 [91776,88288] 0 2024-02-15T18:27:08.448 INFO:tasks.workunit.client.1.smithi191.stdout:6/416: dread d8/f33 [0,4194304] 0 2024-02-15T18:27:08.451 INFO:tasks.workunit.client.1.smithi191.stdout:7/468: dwrite d0/db/d26/d5f/d77/df/f8e [0,4194304] 0 2024-02-15T18:27:08.452 INFO:tasks.workunit.client.1.smithi191.stdout:3/368: dread d6/d1b/d60/d73/f75 [4194304,4194304] 0 2024-02-15T18:27:08.459 INFO:tasks.workunit.client.0.smithi042.stdout:3/876: dwrite d1/de/f42 [4194304,4194304] 0 2024-02-15T18:27:08.459 INFO:tasks.workunit.client.0.smithi042.stdout:3/877: chown d1/d11/d2e/fd6 13 1 2024-02-15T18:27:08.465 INFO:tasks.workunit.client.1.smithi191.stdout:9/487: dwrite d0/d7/dc/d8c/f6b [0,4194304] 0 2024-02-15T18:27:08.465 INFO:tasks.workunit.client.1.smithi191.stdout:9/488: write d0/d29/f70 [4622925,44663] 0 2024-02-15T18:27:08.595 INFO:tasks.workunit.client.1.smithi191.stdout:0/349: symlink d7/da/d17/l76 0 2024-02-15T18:27:08.595 INFO:tasks.workunit.client.1.smithi191.stdout:4/472: symlink d2/d5/d7/d3e/d88/lb2 0 2024-02-15T18:27:08.595 INFO:tasks.workunit.client.1.smithi191.stdout:0/350: chown d7/da/d1a/d2c 24792634 1 2024-02-15T18:27:08.595 INFO:tasks.workunit.client.1.smithi191.stdout:4/473: chown d2/d5/d7/d90/d52/c85 3 1 2024-02-15T18:27:08.596 INFO:tasks.workunit.client.1.smithi191.stdout:5/255: creat d0/d23/d41/d32/f54 x:0 0 0 2024-02-15T18:27:08.598 INFO:tasks.workunit.client.1.smithi191.stdout:6/417: mknod d8/d19/c77 0 2024-02-15T18:27:08.598 INFO:tasks.workunit.client.1.smithi191.stdout:6/418: chown d8/d9/d4a/c4d 11424938 1 2024-02-15T18:27:08.598 INFO:tasks.workunit.client.1.smithi191.stdout:5/256: dread d0/d23/d41/d28/d31/f47 [0,4194304] 0 2024-02-15T18:27:08.599 INFO:tasks.workunit.client.0.smithi042.stdout:8/800: dread dd/f7d [0,4194304] 0 2024-02-15T18:27:08.599 INFO:tasks.workunit.client.0.smithi042.stdout:8/801: dread - dd/de/da4/d67/d6f/fb0 zero size 2024-02-15T18:27:08.600 INFO:tasks.workunit.client.0.smithi042.stdout:2/856: rename d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d2e/f92 to d0/d8/d10/d7b/f10e 0 2024-02-15T18:27:08.606 INFO:tasks.workunit.client.0.smithi042.stdout:4/890: mkdir da/d2b/d7e/d138 0 2024-02-15T18:27:08.607 INFO:tasks.workunit.client.1.smithi191.stdout:3/369: mknod d6/d3b/c80 0 2024-02-15T18:27:08.607 INFO:tasks.workunit.client.1.smithi191.stdout:7/469: creat d0/db/d65/dc/fb0 x:0 0 0 2024-02-15T18:27:08.607 INFO:tasks.workunit.client.1.smithi191.stdout:7/470: dread - d0/db/d65/dc/fa0 zero size 2024-02-15T18:27:08.608 INFO:tasks.workunit.client.0.smithi042.stdout:1/929: creat de/d6d/d9e/df4/f15f x:0 0 0 2024-02-15T18:27:08.610 INFO:tasks.workunit.client.1.smithi191.stdout:9/489: symlink d0/d7/dc/d30/d3b/d56/la9 0 2024-02-15T18:27:08.610 INFO:tasks.workunit.client.1.smithi191.stdout:9/490: fdatasync d0/d7/dc/d17/f84 0 2024-02-15T18:27:08.610 INFO:tasks.workunit.client.1.smithi191.stdout:9/491: chown d0/d7/f11 2785 1 2024-02-15T18:27:08.610 INFO:tasks.workunit.client.1.smithi191.stdout:9/492: write d0/d7/dc/d8c/f78 [358774,57519] 0 2024-02-15T18:27:08.611 INFO:tasks.workunit.client.0.smithi042.stdout:7/920: link d10/d49/ff4 d10/d15/d39/dd9/dec/f134 0 2024-02-15T18:27:08.613 INFO:tasks.workunit.client.0.smithi042.stdout:2/857: read d0/d8/d20/d41/dbe/f10c [1543565,73998] 0 2024-02-15T18:27:08.615 INFO:tasks.workunit.client.0.smithi042.stdout:1/930: write de/d15/d95/fb9 [8302756,92285] 0 2024-02-15T18:27:08.615 INFO:tasks.workunit.client.0.smithi042.stdout:6/978: symlink d9/d3e/l157 0 2024-02-15T18:27:08.621 INFO:tasks.workunit.client.1.smithi191.stdout:2/438: mknod df/c93 0 2024-02-15T18:27:08.621 INFO:tasks.workunit.client.1.smithi191.stdout:2/439: chown df/d12/d19/d1a/d52/c7d 1 1 2024-02-15T18:27:08.621 INFO:tasks.workunit.client.1.smithi191.stdout:2/440: chown df/d12/d1f 6136175 1 2024-02-15T18:27:08.621 INFO:tasks.workunit.client.1.smithi191.stdout:2/441: readlink df/d12/d1f/d42/l64 0 2024-02-15T18:27:08.622 INFO:tasks.workunit.client.0.smithi042.stdout:3/878: mknod d1/d11/d14/d4d/d6c/d7b/c11f 0 2024-02-15T18:27:08.622 INFO:tasks.workunit.client.0.smithi042.stdout:3/879: fsync d1/de/d19/f48 0 2024-02-15T18:27:08.622 INFO:tasks.workunit.client.0.smithi042.stdout:3/880: read d1/d11/d7a/fe1 [577833,102550] 0 2024-02-15T18:27:08.623 INFO:tasks.workunit.client.1.smithi191.stdout:0/351: rename d7/da/d17/d31/d5f/l72 to d7/d3b/l77 0 2024-02-15T18:27:08.623 INFO:tasks.workunit.client.1.smithi191.stdout:0/352: fsync d7/da/d1a/d34/f52 0 2024-02-15T18:27:08.623 INFO:tasks.workunit.client.1.smithi191.stdout:0/353: truncate d7/da/d17/d31/d5f/f73 242331 0 2024-02-15T18:27:08.624 INFO:tasks.workunit.client.1.smithi191.stdout:5/257: creat d0/d23/d41/d28/f55 x:0 0 0 2024-02-15T18:27:08.625 INFO:tasks.workunit.client.1.smithi191.stdout:6/419: dwrite d8/d9/d2b/d57/f6c [0,4194304] 0 2024-02-15T18:27:08.628 INFO:tasks.workunit.client.0.smithi042.stdout:3/881: read d1/d11/d2e/f37 [3194556,94534] 0 2024-02-15T18:27:08.628 INFO:tasks.workunit.client.0.smithi042.stdout:3/882: write d1/d11/d14/d4d/d73/d89/dd9/f106 [619010,35805] 0 2024-02-15T18:27:08.629 INFO:tasks.workunit.client.0.smithi042.stdout:8/802: mknod dd/de/da5/ce6 0 2024-02-15T18:27:08.632 INFO:tasks.workunit.client.1.smithi191.stdout:3/370: unlink d6/d5d/l69 0 2024-02-15T18:27:08.633 INFO:tasks.workunit.client.0.smithi042.stdout:1/931: dwrite de/d54/dbf/fe1 [0,4194304] 0 2024-02-15T18:27:08.633 INFO:tasks.workunit.client.0.smithi042.stdout:1/932: fsync de/d54/dbf/dd0/d7c/dd6/de8/d103/f113 0 2024-02-15T18:27:08.634 INFO:tasks.workunit.client.0.smithi042.stdout:5/919: rename d0/d50 to d0/d39/d144 0 2024-02-15T18:27:08.634 INFO:tasks.workunit.client.0.smithi042.stdout:5/920: readlink d0/dda/d42/d7d/ld1 0 2024-02-15T18:27:08.635 INFO:tasks.workunit.client.0.smithi042.stdout:1/933: dread de/d54/dbf/dd0/d7c/fb1 [0,4194304] 0 2024-02-15T18:27:08.635 INFO:tasks.workunit.client.0.smithi042.stdout:1/934: write f6 [5565233,31507] 0 2024-02-15T18:27:08.635 INFO:tasks.workunit.client.0.smithi042.stdout:1/935: dread - de/d54/dbf/dd0/d7c/dd6/de8/d103/f113 zero size 2024-02-15T18:27:08.636 INFO:tasks.workunit.client.0.smithi042.stdout:0/881: sync 2024-02-15T18:27:08.637 INFO:tasks.workunit.client.0.smithi042.stdout:2/858: dread d0/d8/d10/d31/da6/fad [0,4194304] 0 2024-02-15T18:27:08.639 INFO:tasks.workunit.client.0.smithi042.stdout:7/921: dwrite d10/d15/d8f/da5/dad/de0/de2/ffd [0,4194304] 0 2024-02-15T18:27:08.654 INFO:tasks.workunit.client.0.smithi042.stdout:0/882: dread d4/f20 [0,4194304] 0 2024-02-15T18:27:08.657 INFO:tasks.workunit.client.0.smithi042.stdout:4/891: mkdir da/d11/d85/db5/d103/d66/d139 0 2024-02-15T18:27:08.659 INFO:tasks.workunit.client.1.smithi191.stdout:8/600: sync 2024-02-15T18:27:08.661 INFO:tasks.workunit.client.0.smithi042.stdout:6/979: mknod d9/d11/d19/d35/dc4/c158 0 2024-02-15T18:27:08.661 INFO:tasks.workunit.client.0.smithi042.stdout:9/905: sync 2024-02-15T18:27:08.662 INFO:tasks.workunit.client.1.smithi191.stdout:9/493: creat d0/d7/dc/d30/d3b/d8b/faa x:0 0 0 2024-02-15T18:27:08.662 INFO:tasks.workunit.client.1.smithi191.stdout:7/471: creat d0/db/d65/dc/d4b/fb1 x:0 0 0 2024-02-15T18:27:08.662 INFO:tasks.workunit.client.1.smithi191.stdout:9/494: chown d0/d7/dc/d17/c64 117715 1 2024-02-15T18:27:08.662 INFO:tasks.workunit.client.1.smithi191.stdout:2/442: mkdir df/d61/d94 0 2024-02-15T18:27:08.662 INFO:tasks.workunit.client.1.smithi191.stdout:2/443: fsync df/d12/d19/d1a/f21 0 2024-02-15T18:27:08.663 INFO:tasks.workunit.client.0.smithi042.stdout:9/906: write d6/da/d1d/f58 [3012482,90180] 0 2024-02-15T18:27:08.663 INFO:tasks.workunit.client.1.smithi191.stdout:0/354: mknod d7/d1b/c78 0 2024-02-15T18:27:08.665 INFO:tasks.workunit.client.1.smithi191.stdout:4/474: rename d2/d5/d7/d3e/d88/l9f to d2/d5/d7/d3e/d63/lb3 0 2024-02-15T18:27:08.665 INFO:tasks.workunit.client.1.smithi191.stdout:5/258: rmdir d0/d23/d41 39 2024-02-15T18:27:08.665 INFO:tasks.workunit.client.0.smithi042.stdout:2/859: dwrite d0/d40/fc3 [0,4194304] 0 2024-02-15T18:27:08.665 INFO:tasks.workunit.client.0.smithi042.stdout:3/883: symlink d1/l120 0 2024-02-15T18:27:08.665 INFO:tasks.workunit.client.0.smithi042.stdout:3/884: chown d1/d11/d14/df6/f114 77 1 2024-02-15T18:27:08.666 INFO:tasks.workunit.client.0.smithi042.stdout:3/885: chown d1/d11e 87700637 1 2024-02-15T18:27:08.667 INFO:tasks.workunit.client.0.smithi042.stdout:8/803: dwrite dd/de/d33/d66/f7f [0,4194304] 0 2024-02-15T18:27:08.667 INFO:tasks.workunit.client.0.smithi042.stdout:8/804: truncate dd/de/da4/d81/dae/fb6 4582502 0 2024-02-15T18:27:08.667 INFO:tasks.workunit.client.0.smithi042.stdout:5/921: getdents d0/d12/d2a/d105 0 2024-02-15T18:27:08.670 INFO:tasks.workunit.client.1.smithi191.stdout:3/371: link d6/d3b/d30/f50 d6/d1b/f81 0 2024-02-15T18:27:08.671 INFO:tasks.workunit.client.0.smithi042.stdout:1/936: creat de/d54/dbf/dd0/d7c/dd6/d155/f160 x:0 0 0 2024-02-15T18:27:08.673 INFO:tasks.workunit.client.1.smithi191.stdout:2/444: dread df/f1b [4194304,4194304] 0 2024-02-15T18:27:08.673 INFO:tasks.workunit.client.1.smithi191.stdout:2/445: fdatasync df/d12/d1f/f31 0 2024-02-15T18:27:08.673 INFO:tasks.workunit.client.1.smithi191.stdout:7/472: creat d0/db/d26/fb2 x:0 0 0 2024-02-15T18:27:08.676 INFO:tasks.workunit.client.1.smithi191.stdout:4/475: write d2/f3 [1988235,60559] 0 2024-02-15T18:27:08.677 INFO:tasks.workunit.client.0.smithi042.stdout:7/922: mkdir d10/d15/d39/d6c/dba/d117/d11f/d135 0 2024-02-15T18:27:08.678 INFO:tasks.workunit.client.1.smithi191.stdout:0/355: mkdir d7/da/d1a/d34/d45/d79 0 2024-02-15T18:27:08.679 INFO:tasks.workunit.client.0.smithi042.stdout:0/883: rmdir d4/d38/d39/df4/d114/d6c/df7 39 2024-02-15T18:27:08.679 INFO:tasks.workunit.client.1.smithi191.stdout:9/495: link d0/d1d/d3f/d80/fa2 d0/d7/fab 0 2024-02-15T18:27:08.680 INFO:tasks.workunit.client.1.smithi191.stdout:6/420: rename d8/d9/cf to d8/d19/d6b/d72/c78 0 2024-02-15T18:27:08.682 INFO:tasks.workunit.client.0.smithi042.stdout:4/892: mkdir da/d11/d85/db5/d103/d66/d78/d53/d82/dc0/d13a 0 2024-02-15T18:27:08.684 INFO:tasks.workunit.client.1.smithi191.stdout:8/601: dread d3/d2c/d1a/f8f [0,4194304] 0 2024-02-15T18:27:08.686 INFO:tasks.workunit.client.1.smithi191.stdout:5/259: getdents d0/d23/d41/d32/d52 0 2024-02-15T18:27:08.691 INFO:tasks.workunit.client.0.smithi042.stdout:6/980: rename d9/d11/d19/d35/d55/d92/ccb to d9/d11/d19/d35/d49/d79/c159 0 2024-02-15T18:27:08.692 INFO:tasks.workunit.client.1.smithi191.stdout:3/372: unlink d6/c4c 0 2024-02-15T18:27:08.695 INFO:tasks.workunit.client.1.smithi191.stdout:8/602: dread d3/d2c/d1a/f8f [0,4194304] 0 2024-02-15T18:27:08.695 INFO:tasks.workunit.client.1.smithi191.stdout:8/603: fsync d3/d2c/f17 0 2024-02-15T18:27:08.695 INFO:tasks.workunit.client.1.smithi191.stdout:8/604: read - d3/d2c/d1a/d23/d2e/d47/fbf zero size 2024-02-15T18:27:08.695 INFO:tasks.workunit.client.1.smithi191.stdout:2/446: creat df/d12/d32/d4d/d89/f95 x:0 0 0 2024-02-15T18:27:08.695 INFO:tasks.workunit.client.0.smithi042.stdout:9/907: truncate d6/f21 585103 0 2024-02-15T18:27:08.698 INFO:tasks.workunit.client.1.smithi191.stdout:7/473: creat d0/db/d65/d11/d1c/fb3 x:0 0 0 2024-02-15T18:27:08.698 INFO:tasks.workunit.client.0.smithi042.stdout:5/922: dwrite d0/dda/dc7/d52/dad/dba/dbd/dde/f138 [0,4194304] 0 2024-02-15T18:27:08.698 INFO:tasks.workunit.client.0.smithi042.stdout:5/923: chown d0/fce 7 1 2024-02-15T18:27:08.698 INFO:tasks.workunit.client.1.smithi191.stdout:0/356: symlink d7/d4a/l7a 0 2024-02-15T18:27:08.699 INFO:tasks.workunit.client.1.smithi191.stdout:0/357: read - d7/da/d1a/d34/f52 zero size 2024-02-15T18:27:08.700 INFO:tasks.workunit.client.0.smithi042.stdout:5/924: dread d0/d38/d53/db4/fe9 [0,4194304] 0 2024-02-15T18:27:08.704 INFO:tasks.workunit.client.1.smithi191.stdout:9/496: truncate d0/d7/dc/d17/d44/f6e 1229587 0 2024-02-15T18:27:08.705 INFO:tasks.workunit.client.0.smithi042.stdout:3/886: creat d1/d11/d2e/de7/f121 x:0 0 0 2024-02-15T18:27:08.707 INFO:tasks.workunit.client.1.smithi191.stdout:4/476: rename d2/d5/d14/d47/d78/c7c to d2/d5/d8b/cb4 0 2024-02-15T18:27:08.707 INFO:tasks.workunit.client.1.smithi191.stdout:4/477: chown d2/d5/d14/d44/la6 116596 1 2024-02-15T18:27:08.708 INFO:tasks.workunit.client.0.smithi042.stdout:7/923: dwrite d10/d15/d39/d6c/dba/d117/f12e [0,4194304] 0 2024-02-15T18:27:08.708 INFO:tasks.workunit.client.0.smithi042.stdout:8/805: symlink dd/le7 0 2024-02-15T18:27:08.708 INFO:tasks.workunit.client.0.smithi042.stdout:8/806: stat dd/de/da4/d67/d6f/d78/d9a/dce/fd9 0 2024-02-15T18:27:08.708 INFO:tasks.workunit.client.0.smithi042.stdout:7/924: write d10/d49/ff4 [2791903,38191] 0 2024-02-15T18:27:08.708 INFO:tasks.workunit.client.0.smithi042.stdout:7/925: chown d10/d43/d57/f101 1 1 2024-02-15T18:27:08.708 INFO:tasks.workunit.client.0.smithi042.stdout:7/926: readlink d10/d49/d6d/d6e/lb4 0 2024-02-15T18:27:08.709 INFO:tasks.workunit.client.1.smithi191.stdout:6/421: creat d8/d9/f79 x:0 0 0 2024-02-15T18:27:08.710 INFO:tasks.workunit.client.0.smithi042.stdout:7/927: dread d10/d15/d8f/da5/dad/d104/d65/f88 [0,4194304] 0 2024-02-15T18:27:08.710 INFO:tasks.workunit.client.0.smithi042.stdout:7/928: chown d10/d49/d6d/d78/d83 57321 1 2024-02-15T18:27:08.711 INFO:tasks.workunit.client.0.smithi042.stdout:7/929: write d10/d43/fa4 [415367,29359] 0 2024-02-15T18:27:08.712 INFO:tasks.workunit.client.1.smithi191.stdout:3/373: mkdir d6/d1b/d82 0 2024-02-15T18:27:08.712 INFO:tasks.workunit.client.1.smithi191.stdout:2/447: truncate fb 6094171 0 2024-02-15T18:27:08.712 INFO:tasks.workunit.client.1.smithi191.stdout:2/448: fdatasync df/d12/f22 0 2024-02-15T18:27:08.713 INFO:tasks.workunit.client.0.smithi042.stdout:0/884: dwrite d4/d38/f9f [0,4194304] 0 2024-02-15T18:27:08.713 INFO:tasks.workunit.client.1.smithi191.stdout:7/474: creat d0/db/d26/fb4 x:0 0 0 2024-02-15T18:27:08.713 INFO:tasks.workunit.client.1.smithi191.stdout:7/475: readlink d0/db/d65/d11/d57/l72 0 2024-02-15T18:27:08.713 INFO:tasks.workunit.client.1.smithi191.stdout:7/476: write d0/db/f2f [1200992,50379] 0 2024-02-15T18:27:08.714 INFO:tasks.workunit.client.1.smithi191.stdout:9/497: symlink d0/d1d/d3f/lac 0 2024-02-15T18:27:08.714 INFO:tasks.workunit.client.1.smithi191.stdout:9/498: chown d0/d29/f86 1 1 2024-02-15T18:27:08.719 INFO:tasks.workunit.client.0.smithi042.stdout:4/893: rmdir da/d3a/dc9 39 2024-02-15T18:27:08.720 INFO:tasks.workunit.client.0.smithi042.stdout:8/807: dwrite dd/de/da4/d67/f43 [0,4194304] 0 2024-02-15T18:27:08.724 INFO:tasks.workunit.client.1.smithi191.stdout:5/260: rename d0/d23/f3d to d0/d18/f56 0 2024-02-15T18:27:08.730 INFO:tasks.workunit.client.0.smithi042.stdout:1/937: rename de/d6d/c7a to de/d15/d1b/d44/df3/c161 0 2024-02-15T18:27:08.736 INFO:tasks.workunit.client.1.smithi191.stdout:4/478: mkdir d2/d5/d7/d90/d52/db5 0 2024-02-15T18:27:08.737 INFO:tasks.workunit.client.1.smithi191.stdout:6/422: creat d8/d19/f7a x:0 0 0 2024-02-15T18:27:08.738 INFO:tasks.workunit.client.0.smithi042.stdout:9/908: creat d6/dd/d9/da1/f133 x:0 0 0 2024-02-15T18:27:08.740 INFO:tasks.workunit.client.1.smithi191.stdout:3/374: link d6/d1b/f3e d6/d3b/d42/d49/d54/f83 0 2024-02-15T18:27:08.740 INFO:tasks.workunit.client.1.smithi191.stdout:3/375: readlink d6/l44 0 2024-02-15T18:27:08.740 INFO:tasks.workunit.client.1.smithi191.stdout:3/376: chown d6/d1b/d60/f68 0 1 2024-02-15T18:27:08.742 INFO:tasks.workunit.client.0.smithi042.stdout:6/981: write d9/d11/d1d/f88 [3557353,17854] 0 2024-02-15T18:27:08.743 INFO:tasks.workunit.client.1.smithi191.stdout:2/449: rmdir df/d61 39 2024-02-15T18:27:08.747 INFO:tasks.workunit.client.1.smithi191.stdout:9/499: dread d0/d7/dc/d17/d44/f6e [0,4194304] 0 2024-02-15T18:27:08.748 INFO:tasks.workunit.client.1.smithi191.stdout:7/477: symlink d0/db/d26/d5f/d77/d1d/d54/lb5 0 2024-02-15T18:27:08.750 INFO:tasks.workunit.client.0.smithi042.stdout:8/808: dwrite dd/d11/d75/f95 [0,4194304] 0 2024-02-15T18:27:08.750 INFO:tasks.workunit.client.1.smithi191.stdout:8/605: rename d3/d2c/d1a/d23/d3e/c7b to d3/d2c/d1a/d23/d3e/db2/cdd 0 2024-02-15T18:27:08.752 INFO:tasks.workunit.client.1.smithi191.stdout:5/261: dwrite d0/d18/f45 [0,4194304] 0 2024-02-15T18:27:08.754 INFO:tasks.workunit.client.0.smithi042.stdout:7/930: creat d10/d15/d39/de3/f136 x:0 0 0 2024-02-15T18:27:08.756 INFO:tasks.workunit.client.1.smithi191.stdout:4/479: link d2/d5/d7/d90/d52/c85 d2/d5/d7/cb6 0 2024-02-15T18:27:08.759 INFO:tasks.workunit.client.0.smithi042.stdout:0/885: mkdir d4/d38/d39/df4/d114/d6c/df9/d11f 0 2024-02-15T18:27:08.759 INFO:tasks.workunit.client.1.smithi191.stdout:6/423: getdents d8/d19/d6b/d72 0 2024-02-15T18:27:08.763 INFO:tasks.workunit.client.1.smithi191.stdout:3/377: link d6/d3b/f13 d6/d3b/f84 0 2024-02-15T18:27:08.764 INFO:tasks.workunit.client.1.smithi191.stdout:5/262: dread d0/d23/d41/d28/f3e [0,4194304] 0 2024-02-15T18:27:08.766 INFO:tasks.workunit.client.1.smithi191.stdout:4/480: write d2/d5/d7/d3e/f49 [354778,126108] 0 2024-02-15T18:27:08.766 INFO:tasks.workunit.client.0.smithi042.stdout:5/925: rename d0/dda/dc7/d52/dad/dba to d0/d38/d53/d64/d145 0 2024-02-15T18:27:08.768 INFO:tasks.workunit.client.1.smithi191.stdout:4/481: write d2/d5/f13 [4201026,1005] 0 2024-02-15T18:27:08.769 INFO:tasks.workunit.client.1.smithi191.stdout:2/450: symlink df/d12/d19/d1a/d35/d47/l96 0 2024-02-15T18:27:08.770 INFO:tasks.workunit.client.0.smithi042.stdout:1/938: symlink de/d15/d67/d89/l162 0 2024-02-15T18:27:08.770 INFO:tasks.workunit.client.1.smithi191.stdout:9/500: rmdir d0/d29 39 2024-02-15T18:27:08.770 INFO:tasks.workunit.client.1.smithi191.stdout:9/501: chown d0/c5a 394094 1 2024-02-15T18:27:08.771 INFO:tasks.workunit.client.1.smithi191.stdout:9/502: chown d0/d7/dc/d32/c81 0 1 2024-02-15T18:27:08.771 INFO:tasks.workunit.client.1.smithi191.stdout:9/503: write d0/d7/dc/d17/d33/f69 [1711221,28716] 0 2024-02-15T18:27:08.771 INFO:tasks.workunit.client.0.smithi042.stdout:9/909: creat d6/da/d1d/d2d/d40/d48/f134 x:0 0 0 2024-02-15T18:27:08.772 INFO:tasks.workunit.client.1.smithi191.stdout:7/478: creat d0/db/d65/d11/d5a/fb6 x:0 0 0 2024-02-15T18:27:08.773 INFO:tasks.workunit.client.0.smithi042.stdout:6/982: getdents d9/d11/d97 0 2024-02-15T18:27:08.773 INFO:tasks.workunit.client.0.smithi042.stdout:6/983: fsync d9/d11/d19/d35/d49/f4c 0 2024-02-15T18:27:08.774 INFO:tasks.workunit.client.1.smithi191.stdout:0/358: rename d7/da/f20 to d7/d4a/f7b 0 2024-02-15T18:27:08.776 INFO:tasks.workunit.client.1.smithi191.stdout:8/606: dwrite d3/d2c/d1a/fb1 [0,4194304] 0 2024-02-15T18:27:08.776 INFO:tasks.workunit.client.1.smithi191.stdout:6/424: unlink d8/d19/f3d 0 2024-02-15T18:27:08.776 INFO:tasks.workunit.client.1.smithi191.stdout:6/425: readlink d8/d19/d6b/d72/l74 0 2024-02-15T18:27:08.776 INFO:tasks.workunit.client.0.smithi042.stdout:1/939: dread de/d15/d2c/d51/d70/fba [0,4194304] 0 2024-02-15T18:27:08.782 INFO:tasks.workunit.client.1.smithi191.stdout:3/378: creat d6/d3b/d42/d49/f85 x:0 0 0 2024-02-15T18:27:08.782 INFO:tasks.workunit.client.1.smithi191.stdout:3/379: fsync d6/f8 0 2024-02-15T18:27:08.782 INFO:tasks.workunit.client.1.smithi191.stdout:3/380: read - d6/d27/f78 zero size 2024-02-15T18:27:08.783 INFO:tasks.workunit.client.1.smithi191.stdout:9/504: dread d0/d7/dc/d8c/f1c [0,4194304] 0 2024-02-15T18:27:08.783 INFO:tasks.workunit.client.0.smithi042.stdout:8/809: dwrite dd/de/da4/f8d [0,4194304] 0 2024-02-15T18:27:08.783 INFO:tasks.workunit.client.1.smithi191.stdout:5/263: truncate d0/d23/f27 1489821 0 2024-02-15T18:27:08.784 INFO:tasks.workunit.client.1.smithi191.stdout:3/381: write d6/d1b/d60/d73/f61 [810431,71731] 0 2024-02-15T18:27:08.786 INFO:tasks.workunit.client.1.smithi191.stdout:4/482: symlink d2/d5/d86/lb7 0 2024-02-15T18:27:08.787 INFO:tasks.workunit.client.0.smithi042.stdout:7/931: getdents d10/d15/d108 0 2024-02-15T18:27:08.787 INFO:tasks.workunit.client.0.smithi042.stdout:7/932: fdatasync d10/d15/f1d 0 2024-02-15T18:27:08.787 INFO:tasks.workunit.client.0.smithi042.stdout:7/933: stat d10/d15/d39/d6c/d92/cd1 0 2024-02-15T18:27:08.789 INFO:tasks.workunit.client.1.smithi191.stdout:2/451: getdents df/d12/d19 0 2024-02-15T18:27:08.794 INFO:tasks.workunit.client.1.smithi191.stdout:7/479: rmdir d0/db/d65/d11/d1c 39 2024-02-15T18:27:08.794 INFO:tasks.workunit.client.0.smithi042.stdout:0/886: unlink d4/d9/d34/dab/db0/ce9 0 2024-02-15T18:27:08.795 INFO:tasks.workunit.client.0.smithi042.stdout:0/887: dread - d4/d38/d77/d2e/dae/ddf/fe8 zero size 2024-02-15T18:27:08.795 INFO:tasks.workunit.client.1.smithi191.stdout:6/426: dwrite d8/d9/d2b/d57/f59 [0,4194304] 0 2024-02-15T18:27:08.797 INFO:tasks.workunit.client.0.smithi042.stdout:2/860: sync 2024-02-15T18:27:08.798 INFO:tasks.workunit.client.0.smithi042.stdout:2/861: readlink d0/d8/d20/d7d/lde 0 2024-02-15T18:27:08.799 INFO:tasks.workunit.client.0.smithi042.stdout:3/887: rename d1/d11/d14/la2 to d1/d9c/l122 0 2024-02-15T18:27:08.799 INFO:tasks.workunit.client.0.smithi042.stdout:3/888: dread - d1/d11/d2e/de7/f10d zero size 2024-02-15T18:27:08.802 INFO:tasks.workunit.client.1.smithi191.stdout:0/359: creat d7/da/d1a/d34/d27/f7c x:0 0 0 2024-02-15T18:27:08.804 INFO:tasks.workunit.client.0.smithi042.stdout:5/926: stat d0/d38/d53/d64/d145/dbd/dde/l110 0 2024-02-15T18:27:08.804 INFO:tasks.workunit.client.0.smithi042.stdout:5/927: write d0/d14/d88/f123 [906067,113857] 0 2024-02-15T18:27:08.804 INFO:tasks.workunit.client.0.smithi042.stdout:5/928: chown d0/dda/dc7/f6d 0 1 2024-02-15T18:27:08.804 INFO:tasks.workunit.client.1.smithi191.stdout:0/360: dread d7/da/d1a/d34/d27/f60 [0,4194304] 0 2024-02-15T18:27:08.805 INFO:tasks.workunit.client.0.smithi042.stdout:9/910: mknod d6/da/d2f/d55/d73/d8d/dcf/de1/c135 0 2024-02-15T18:27:08.807 INFO:tasks.workunit.client.0.smithi042.stdout:6/984: getdents d9/d11/d46/d89/df4 0 2024-02-15T18:27:08.810 INFO:tasks.workunit.client.1.smithi191.stdout:9/505: mknod d0/d29/d48/d66/d92/cad 0 2024-02-15T18:27:08.811 INFO:tasks.workunit.client.0.smithi042.stdout:1/940: mkdir de/d6d/d9e/df4/d163 0 2024-02-15T18:27:08.815 INFO:tasks.workunit.client.0.smithi042.stdout:8/810: symlink dd/de/da4/le8 0 2024-02-15T18:27:08.815 INFO:tasks.workunit.client.1.smithi191.stdout:3/382: mknod d6/d1b/d60/d73/d5e/c86 0 2024-02-15T18:27:08.816 INFO:tasks.workunit.client.0.smithi042.stdout:0/888: dwrite d4/d38/d39/f6e [4194304,4194304] 0 2024-02-15T18:27:08.816 INFO:tasks.workunit.client.0.smithi042.stdout:2/862: symlink d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/dfe/l10f 0 2024-02-15T18:27:08.817 INFO:tasks.workunit.client.1.smithi191.stdout:2/452: rmdir df/d61/d84 0 2024-02-15T18:27:08.817 INFO:tasks.workunit.client.1.smithi191.stdout:2/453: write df/d12/d19/f77 [754205,44913] 0 2024-02-15T18:27:08.818 INFO:tasks.workunit.client.1.smithi191.stdout:3/383: dread d6/d1b/f81 [0,4194304] 0 2024-02-15T18:27:08.819 INFO:tasks.workunit.client.1.smithi191.stdout:3/384: fsync d6/d3b/d42/f4a 0 2024-02-15T18:27:08.822 INFO:tasks.workunit.client.0.smithi042.stdout:4/894: rename da/d11/d29/db8/fb9 to da/d11/d85/db5/f13b 0 2024-02-15T18:27:08.822 INFO:tasks.workunit.client.0.smithi042.stdout:4/895: readlink da/lbc 0 2024-02-15T18:27:08.824 INFO:tasks.workunit.client.0.smithi042.stdout:5/929: symlink d0/l146 0 2024-02-15T18:27:08.827 INFO:tasks.workunit.client.0.smithi042.stdout:4/896: write da/d11/d85/db5/d103/d66/fa3 [430583,87804] 0 2024-02-15T18:27:08.828 INFO:tasks.workunit.client.0.smithi042.stdout:6/985: link d9/d11/d19/d1c/fad d9/d11/d97/df7/d12f/f15a 0 2024-02-15T18:27:08.828 INFO:tasks.workunit.client.0.smithi042.stdout:4/897: write da/d11/d85/ff8 [1502683,130917] 0 2024-02-15T18:27:08.828 INFO:tasks.workunit.client.0.smithi042.stdout:6/986: write d9/d11/d19/f99 [685993,81060] 0 2024-02-15T18:27:08.828 INFO:tasks.workunit.client.0.smithi042.stdout:4/898: dread - da/d3a/dc9/f114 zero size 2024-02-15T18:27:08.828 INFO:tasks.workunit.client.0.smithi042.stdout:6/987: chown d9/d11/d19/d35/dc4/fe0 3289114 1 2024-02-15T18:27:08.828 INFO:tasks.workunit.client.0.smithi042.stdout:6/988: read - d9/d11/d19/d1c/dcf/f150 zero size 2024-02-15T18:27:08.828 INFO:tasks.workunit.client.1.smithi191.stdout:6/427: creat d8/d9/d2b/f7b x:0 0 0 2024-02-15T18:27:08.829 INFO:tasks.workunit.client.0.smithi042.stdout:1/941: mknod de/d54/dbf/dd0/d7c/dd6/de8/dec/d100/c164 0 2024-02-15T18:27:08.830 INFO:tasks.workunit.client.0.smithi042.stdout:8/811: symlink dd/de/d12/le9 0 2024-02-15T18:27:08.831 INFO:tasks.workunit.client.1.smithi191.stdout:9/506: creat d0/d7/dc/fae x:0 0 0 2024-02-15T18:27:08.831 INFO:tasks.workunit.client.1.smithi191.stdout:9/507: fdatasync d0/d1d/d3f/d80/f5d 0 2024-02-15T18:27:08.831 INFO:tasks.workunit.client.1.smithi191.stdout:9/508: write d0/d7/dc/d8c/f78 [1317558,15160] 0 2024-02-15T18:27:08.833 INFO:tasks.workunit.client.1.smithi191.stdout:2/454: rename df/d12/d19/d1a/d35/c62 to df/d12/d19/c97 0 2024-02-15T18:27:08.835 INFO:tasks.workunit.client.0.smithi042.stdout:0/889: unlink d4/d38/d39/d8e/c55 0 2024-02-15T18:27:08.836 INFO:tasks.workunit.client.1.smithi191.stdout:3/385: unlink d6/d3b/d42/d49/d54/f7f 0 2024-02-15T18:27:08.836 INFO:tasks.workunit.client.1.smithi191.stdout:3/386: truncate d6/d3b/d42/f51 1513381 0 2024-02-15T18:27:08.836 INFO:tasks.workunit.client.1.smithi191.stdout:3/387: read - d6/d1b/d60/d73/f34 zero size 2024-02-15T18:27:08.837 INFO:tasks.workunit.client.1.smithi191.stdout:4/483: dwrite d2/d5/d7/f18 [0,4194304] 0 2024-02-15T18:27:08.837 INFO:tasks.workunit.client.1.smithi191.stdout:4/484: dread - d2/d5/d7/d2f/d41/f99 zero size 2024-02-15T18:27:08.837 INFO:tasks.workunit.client.0.smithi042.stdout:2/863: rename d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/f93 to d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d2e/f110 0 2024-02-15T18:27:08.838 INFO:tasks.workunit.client.1.smithi191.stdout:7/480: getdents d0/db/d65/d11/d1c 0 2024-02-15T18:27:08.839 INFO:tasks.workunit.client.0.smithi042.stdout:5/930: truncate d0/dda/f56 175522 0 2024-02-15T18:27:08.840 INFO:tasks.workunit.client.1.smithi191.stdout:6/428: creat d8/d9/d4a/d4c/d69/f7c x:0 0 0 2024-02-15T18:27:08.842 INFO:tasks.workunit.client.1.smithi191.stdout:2/455: rmdir df/d12/d1e 39 2024-02-15T18:27:08.842 INFO:tasks.workunit.client.1.smithi191.stdout:5/264: dwrite d0/d18/f56 [0,4194304] 0 2024-02-15T18:27:08.843 INFO:tasks.workunit.client.0.smithi042.stdout:7/934: dwrite d10/d15/d8f/da5/dad/d104/d65/f88 [0,4194304] 0 2024-02-15T18:27:08.843 INFO:tasks.workunit.client.0.smithi042.stdout:7/935: readlink d10/d15/d8f/da5/dad/d104/d23/d51/leb 0 2024-02-15T18:27:08.843 INFO:tasks.workunit.client.1.smithi191.stdout:8/607: dwrite d3/f31 [4194304,4194304] 0 2024-02-15T18:27:08.844 INFO:tasks.workunit.client.1.smithi191.stdout:3/388: symlink d6/d1b/d60/l87 0 2024-02-15T18:27:08.845 INFO:tasks.workunit.client.0.smithi042.stdout:9/911: dwrite d6/da/d2f/d69/f8b [4194304,4194304] 0 2024-02-15T18:27:08.845 INFO:tasks.workunit.client.0.smithi042.stdout:9/912: read - d6/da/d2f/d55/d73/d8d/dcf/de1/ffa zero size 2024-02-15T18:27:08.846 INFO:tasks.workunit.client.0.smithi042.stdout:3/889: dwrite d1/de/fe8 [0,4194304] 0 2024-02-15T18:27:08.846 INFO:tasks.workunit.client.0.smithi042.stdout:8/812: dread dd/d11/d75/f85 [0,4194304] 0 2024-02-15T18:27:08.846 INFO:tasks.workunit.client.0.smithi042.stdout:3/890: dread - d1/d11/d14/dbf/fe6 zero size 2024-02-15T18:27:08.846 INFO:tasks.workunit.client.0.smithi042.stdout:3/891: chown d1/de 1311695 1 2024-02-15T18:27:08.846 INFO:tasks.workunit.client.0.smithi042.stdout:3/892: truncate d1/d11/d14/df6/ff7 797470 0 2024-02-15T18:27:08.846 INFO:tasks.workunit.client.0.smithi042.stdout:3/893: write d1/d11/d2e/de7/f10d [396078,57193] 0 2024-02-15T18:27:08.846 INFO:tasks.workunit.client.0.smithi042.stdout:3/894: getdents d1/d11/d14/d4d/d73/d89/da8/d11a 0 2024-02-15T18:27:08.846 INFO:tasks.workunit.client.0.smithi042.stdout:6/989: unlink d9/d11/d19/d1c/fdb 0 2024-02-15T18:27:08.847 INFO:tasks.workunit.client.0.smithi042.stdout:6/990: write d9/d11/d19/f2a [3449895,6383] 0 2024-02-15T18:27:08.858 INFO:tasks.workunit.client.1.smithi191.stdout:9/509: dwrite d0/d7/f9 [0,4194304] 0 2024-02-15T18:27:08.858 INFO:tasks.workunit.client.1.smithi191.stdout:9/510: chown d0/d1d/f79 632 1 2024-02-15T18:27:08.859 INFO:tasks.workunit.client.1.smithi191.stdout:9/511: read d0/d29/d48/f5b [550479,74309] 0 2024-02-15T18:27:08.863 INFO:tasks.workunit.client.0.smithi042.stdout:4/899: mknod da/d11/d85/db5/d103/d66/d139/c13c 0 2024-02-15T18:27:08.864 INFO:tasks.workunit.client.1.smithi191.stdout:4/485: rmdir d2/d5/d7/d90 39 2024-02-15T18:27:08.867 INFO:tasks.workunit.client.0.smithi042.stdout:1/942: creat de/d54/dbf/dd0/d7c/dd6/ddf/dca/f165 x:0 0 0 2024-02-15T18:27:08.869 INFO:tasks.workunit.client.1.smithi191.stdout:7/481: link d0/db/d26/d5f/d77/df/f17 d0/db/d26/d5f/d77/fb7 0 2024-02-15T18:27:08.869 INFO:tasks.workunit.client.1.smithi191.stdout:7/482: read - d0/db/d65/dc/d4b/fb1 zero size 2024-02-15T18:27:08.870 INFO:tasks.workunit.client.0.smithi042.stdout:0/890: creat d4/d38/d77/d2e/dda/f120 x:0 0 0 2024-02-15T18:27:08.872 INFO:tasks.workunit.client.0.smithi042.stdout:2/864: symlink d0/d8/d20/d7d/d7e/l111 0 2024-02-15T18:27:08.873 INFO:tasks.workunit.client.1.smithi191.stdout:2/456: unlink df/c93 0 2024-02-15T18:27:08.874 INFO:tasks.workunit.client.0.smithi042.stdout:5/931: mknod d0/d14/d88/c147 0 2024-02-15T18:27:08.874 INFO:tasks.workunit.client.0.smithi042.stdout:5/932: write d0/dda/dc7/d52/dad/f132 [252372,30688] 0 2024-02-15T18:27:08.877 INFO:tasks.workunit.client.1.smithi191.stdout:5/265: link d0/d18/l1c d0/d23/d41/d28/d31/l57 0 2024-02-15T18:27:08.877 INFO:tasks.workunit.client.1.smithi191.stdout:5/266: readlink d0/d23/d41/d32/d52/l50 0 2024-02-15T18:27:08.882 INFO:tasks.workunit.client.0.smithi042.stdout:8/813: rmdir dd/de/da4/d81 39 2024-02-15T18:27:08.886 INFO:tasks.workunit.client.1.smithi191.stdout:8/608: chown d3/d5e/l84 72 1 2024-02-15T18:27:08.886 INFO:tasks.workunit.client.1.smithi191.stdout:2/457: dread df/d12/d19/d1a/d35/d5b/f6e [0,4194304] 0 2024-02-15T18:27:08.887 INFO:tasks.workunit.client.0.smithi042.stdout:3/895: mkdir d1/d123 0 2024-02-15T18:27:08.887 INFO:tasks.workunit.client.0.smithi042.stdout:3/896: fdatasync d1/f15 0 2024-02-15T18:27:08.887 INFO:tasks.workunit.client.0.smithi042.stdout:3/897: fdatasync d1/d11/d2e/f79 0 2024-02-15T18:27:08.887 INFO:tasks.workunit.client.0.smithi042.stdout:3/898: readlink d1/de/d1a/l8d 0 2024-02-15T18:27:08.888 INFO:tasks.workunit.client.0.smithi042.stdout:9/913: dwrite d6/da/d2f/d6c/f94 [0,4194304] 0 2024-02-15T18:27:08.888 INFO:tasks.workunit.client.0.smithi042.stdout:9/914: fsync d6/da/d2f/d69/f7d 0 2024-02-15T18:27:08.889 INFO:tasks.workunit.client.0.smithi042.stdout:3/899: write d1/d11/d14/d4d/d6c/d7b/f98 [874432,4107] 0 2024-02-15T18:27:08.889 INFO:tasks.workunit.client.0.smithi042.stdout:3/900: write d1/d11/d14/df6/f10e [277302,38162] 0 2024-02-15T18:27:08.889 INFO:tasks.workunit.client.0.smithi042.stdout:9/915: write d6/dd/d9/da1/fa5 [408012,67063] 0 2024-02-15T18:27:08.889 INFO:tasks.workunit.client.0.smithi042.stdout:9/916: read - d6/da/d2f/d55/d73/d8d/dcf/de1/ffa zero size 2024-02-15T18:27:08.889 INFO:tasks.workunit.client.0.smithi042.stdout:9/917: chown d6/dd/d9/f41 3030 1 2024-02-15T18:27:08.890 INFO:tasks.workunit.client.1.smithi191.stdout:3/389: unlink d6/d1b/d60/d73/c25 0 2024-02-15T18:27:08.891 INFO:tasks.workunit.client.0.smithi042.stdout:7/936: dwrite d10/d15/d39/d6c/f6f [0,4194304] 0 2024-02-15T18:27:08.891 INFO:tasks.workunit.client.0.smithi042.stdout:7/937: truncate d10/d15/d39/d6c/f9a 666427 0 2024-02-15T18:27:08.893 INFO:tasks.workunit.client.1.smithi191.stdout:4/486: dwrite d2/d5/d86/f96 [0,4194304] 0 2024-02-15T18:27:08.893 INFO:tasks.workunit.client.0.smithi042.stdout:6/991: symlink d9/d11/d46/d89/df4/d146/l15b 0 2024-02-15T18:27:08.893 INFO:tasks.workunit.client.0.smithi042.stdout:6/992: truncate d9/d11/d19/d1c/d31/d4f/fc8 473716 0 2024-02-15T18:27:08.897 INFO:tasks.workunit.client.1.smithi191.stdout:9/512: dwrite d0/d29/f86 [0,4194304] 0 2024-02-15T18:27:08.898 INFO:tasks.workunit.client.1.smithi191.stdout:9/513: write d0/f18 [444007,7641] 0 2024-02-15T18:27:08.899 INFO:tasks.workunit.client.1.smithi191.stdout:9/514: write d0/d7/dc/d30/d3b/fa1 [963226,104340] 0 2024-02-15T18:27:08.899 INFO:tasks.workunit.client.1.smithi191.stdout:9/515: chown d0/d7/fd 549054780 1 2024-02-15T18:27:08.899 INFO:tasks.workunit.client.1.smithi191.stdout:9/516: stat d0/f4 0 2024-02-15T18:27:08.899 INFO:tasks.workunit.client.1.smithi191.stdout:7/483: mknod d0/db/d26/d5f/d8c/cb8 0 2024-02-15T18:27:08.901 INFO:tasks.workunit.client.1.smithi191.stdout:7/484: write d0/db/d26/d5f/d77/d35/f50 [434328,124160] 0 2024-02-15T18:27:08.902 INFO:tasks.workunit.client.0.smithi042.stdout:3/901: dread d1/de/d19/f3e [0,4194304] 0 2024-02-15T18:27:08.902 INFO:tasks.workunit.client.0.smithi042.stdout:4/900: creat da/d11/d67/f13d x:0 0 0 2024-02-15T18:27:08.903 INFO:tasks.workunit.client.1.smithi191.stdout:6/429: symlink d8/d9/d2b/l7d 0 2024-02-15T18:27:08.903 INFO:tasks.workunit.client.1.smithi191.stdout:6/430: chown d8/c49 1303842 1 2024-02-15T18:27:08.904 INFO:tasks.workunit.client.0.smithi042.stdout:1/943: creat de/d15/d67/d63/f166 x:0 0 0 2024-02-15T18:27:08.904 INFO:tasks.workunit.client.0.smithi042.stdout:1/944: readlink de/d15/d2c/l3c 0 2024-02-15T18:27:08.905 INFO:tasks.workunit.client.1.smithi191.stdout:5/267: creat d0/d23/d48/f58 x:0 0 0 2024-02-15T18:27:08.907 INFO:tasks.workunit.client.0.smithi042.stdout:0/891: rename d4/d9/d5a to d4/d38/d39/df4/d114/d6c/df9/d11f/d121 0 2024-02-15T18:27:08.908 INFO:tasks.workunit.client.1.smithi191.stdout:8/609: rmdir d3/d2c/d1a/d23/d2e/d5b/d69 39 2024-02-15T18:27:08.909 INFO:tasks.workunit.client.0.smithi042.stdout:2/865: creat d0/d8/d10/f112 x:0 0 0 2024-02-15T18:27:08.910 INFO:tasks.workunit.client.1.smithi191.stdout:2/458: creat df/d61/d94/f98 x:0 0 0 2024-02-15T18:27:08.911 INFO:tasks.workunit.client.0.smithi042.stdout:5/933: unlink d0/d38/d53/d91/ca5 0 2024-02-15T18:27:08.913 INFO:tasks.workunit.client.0.smithi042.stdout:7/938: dwrite d10/f69 [0,4194304] 0 2024-02-15T18:27:08.978 INFO:tasks.workunit.client.1.smithi191.stdout:3/390: mknod d6/d3b/d42/d49/d54/c88 0 2024-02-15T18:27:08.979 INFO:tasks.workunit.client.0.smithi042.stdout:1/945: dwrite de/d54/dbf/dd0/d7c/dd6/de8/dec/d100/f119 [0,4194304] 0 2024-02-15T18:27:08.979 INFO:tasks.workunit.client.0.smithi042.stdout:1/946: fdatasync de/d54/d9a/fed 0 2024-02-15T18:27:08.984 INFO:tasks.workunit.client.0.smithi042.stdout:9/918: mkdir d6/da/d2f/d6c/d136 0 2024-02-15T18:27:08.984 INFO:tasks.workunit.client.0.smithi042.stdout:9/919: write d6/da/d1d/d2d/d40/d5f/ff9 [427299,110351] 0 2024-02-15T18:27:08.984 INFO:tasks.workunit.client.0.smithi042.stdout:9/920: chown d6/da/d75/leb 893 1 2024-02-15T18:27:08.984 INFO:tasks.workunit.client.1.smithi191.stdout:4/487: getdents d2/d5/d7/d90/da5 0 2024-02-15T18:27:08.990 INFO:tasks.workunit.client.1.smithi191.stdout:9/517: creat d0/d7/dc/d30/d4f/faf x:0 0 0 2024-02-15T18:27:08.990 INFO:tasks.workunit.client.1.smithi191.stdout:9/518: chown d0/d7/dc/d17/d33/c8f 423721 1 2024-02-15T18:27:08.990 INFO:tasks.workunit.client.1.smithi191.stdout:7/485: rmdir d0/db/d65/d11/d5a/d66 39 2024-02-15T18:27:08.995 INFO:tasks.workunit.client.1.smithi191.stdout:5/268: symlink d0/d23/d41/d32/l59 0 2024-02-15T18:27:08.997 INFO:tasks.workunit.client.1.smithi191.stdout:5/269: write d0/d23/d41/d28/f3f [3158458,26761] 0 2024-02-15T18:27:08.998 INFO:tasks.workunit.client.1.smithi191.stdout:5/270: write d0/d18/f2e [710404,67625] 0 2024-02-15T18:27:08.998 INFO:tasks.workunit.client.0.smithi042.stdout:1/947: dwrite de/d6d/d9e/df4/d13e/f158 [0,4194304] 0 2024-02-15T18:27:08.998 INFO:tasks.workunit.client.0.smithi042.stdout:1/948: fsync de/d6d/d9e/df4/f13c 0 2024-02-15T18:27:08.999 INFO:tasks.workunit.client.0.smithi042.stdout:1/949: write de/d15/d1b/f49 [4848442,73858] 0 2024-02-15T18:27:08.999 INFO:tasks.workunit.client.0.smithi042.stdout:1/950: dread - de/d54/dbf/f14a zero size 2024-02-15T18:27:09.000 INFO:tasks.workunit.client.1.smithi191.stdout:9/519: dread d0/d29/d48/f96 [0,4194304] 0 2024-02-15T18:27:09.000 INFO:tasks.workunit.client.1.smithi191.stdout:9/520: fdatasync d0/d1d/d3f/d80/f75 0 2024-02-15T18:27:09.000 INFO:tasks.workunit.client.1.smithi191.stdout:9/521: dread - d0/d7/dc/d17/f99 zero size 2024-02-15T18:27:09.005 INFO:tasks.workunit.client.0.smithi042.stdout:4/901: creat da/d11/f13e x:0 0 0 2024-02-15T18:27:09.007 INFO:tasks.workunit.client.0.smithi042.stdout:3/902: rename d1/d11/d14/fd8 to d1/d11e/f124 0 2024-02-15T18:27:09.007 INFO:tasks.workunit.client.1.smithi191.stdout:8/610: mknod d3/d2c/d1a/d23/d3e/cde 0 2024-02-15T18:27:09.008 INFO:tasks.workunit.client.0.smithi042.stdout:0/892: creat d4/d9/d62/d10c/f122 x:0 0 0 2024-02-15T18:27:09.009 INFO:tasks.workunit.client.1.smithi191.stdout:2/459: creat df/d12/d19/d1a/d35/d47/d4b/f99 x:0 0 0 2024-02-15T18:27:09.009 INFO:tasks.workunit.client.0.smithi042.stdout:5/934: creat d0/dda/d42/d7d/f148 x:0 0 0 2024-02-15T18:27:09.010 INFO:tasks.workunit.client.1.smithi191.stdout:3/391: creat d6/d27/d4d/d5a/f89 x:0 0 0 2024-02-15T18:27:09.012 INFO:tasks.workunit.client.0.smithi042.stdout:7/939: link d10/d49/fa9 d10/d15/d8f/da5/dad/d104/d65/da7/f137 0 2024-02-15T18:27:09.012 INFO:tasks.workunit.client.1.smithi191.stdout:4/488: creat d2/d5/d14/fb8 x:0 0 0 2024-02-15T18:27:09.012 INFO:tasks.workunit.client.1.smithi191.stdout:4/489: write d2/fad [976604,125181] 0 2024-02-15T18:27:09.014 INFO:tasks.workunit.client.1.smithi191.stdout:6/431: dwrite d8/d9/d2b/f51 [0,4194304] 0 2024-02-15T18:27:09.014 INFO:tasks.workunit.client.1.smithi191.stdout:6/432: dread - d8/d9/d2b/d5b/f5e zero size 2024-02-15T18:27:09.014 INFO:tasks.workunit.client.1.smithi191.stdout:7/486: truncate d0/f3a 542003 0 2024-02-15T18:27:09.019 INFO:tasks.workunit.client.1.smithi191.stdout:5/271: rmdir d0/d23/d41/d28/d31 39 2024-02-15T18:27:09.022 INFO:tasks.workunit.client.0.smithi042.stdout:9/921: rmdir d6/da/d2f/d6c 39 2024-02-15T18:27:09.022 INFO:tasks.workunit.client.0.smithi042.stdout:9/922: stat d6/da/d1d/d2d/l36 0 2024-02-15T18:27:09.023 INFO:tasks.workunit.client.1.smithi191.stdout:9/522: creat d0/d7/dc/d30/d4f/fb0 x:0 0 0 2024-02-15T18:27:09.023 INFO:tasks.workunit.client.1.smithi191.stdout:2/460: dread - df/f40 zero size 2024-02-15T18:27:09.024 INFO:tasks.workunit.client.1.smithi191.stdout:8/611: creat d3/d2c/d1a/d23/d2e/fdf x:0 0 0 2024-02-15T18:27:09.025 INFO:tasks.workunit.client.1.smithi191.stdout:3/392: rmdir d6/d1b/d60 39 2024-02-15T18:27:09.026 INFO:tasks.workunit.client.0.smithi042.stdout:6/993: mknod d9/d11/d1d/c15c 0 2024-02-15T18:27:09.030 INFO:tasks.workunit.client.1.smithi191.stdout:2/461: write f7 [4025189,10417] 0 2024-02-15T18:27:09.030 INFO:tasks.workunit.client.0.smithi042.stdout:1/951: mknod de/d54/dbf/dd0/d7c/dd6/ddf/dca/c167 0 2024-02-15T18:27:09.030 INFO:tasks.workunit.client.0.smithi042.stdout:1/952: fdatasync de/d15/d2c/d51/f6f 0 2024-02-15T18:27:09.031 INFO:tasks.workunit.client.1.smithi191.stdout:6/433: mknod d8/c7e 0 2024-02-15T18:27:09.032 INFO:tasks.workunit.client.0.smithi042.stdout:4/902: truncate da/d3a/dc9/fd2 52284 0 2024-02-15T18:27:09.032 INFO:tasks.workunit.client.0.smithi042.stdout:4/903: chown da/d11/da4 6490614 1 2024-02-15T18:27:09.037 INFO:tasks.workunit.client.1.smithi191.stdout:9/523: getdents d0/d29/d48/d66/d92 0 2024-02-15T18:27:09.038 INFO:tasks.workunit.client.1.smithi191.stdout:6/434: write d8/d9/d2b/f2e [2880098,32147] 0 2024-02-15T18:27:09.042 INFO:tasks.workunit.client.0.smithi042.stdout:8/814: sync 2024-02-15T18:27:09.042 INFO:tasks.workunit.client.0.smithi042.stdout:8/815: chown dd/de/d33/l83 548840 1 2024-02-15T18:27:09.042 INFO:tasks.workunit.client.0.smithi042.stdout:2/866: sync 2024-02-15T18:27:09.043 INFO:tasks.workunit.client.1.smithi191.stdout:8/612: write d3/f15 [2462663,61878] 0 2024-02-15T18:27:09.046 INFO:tasks.workunit.client.0.smithi042.stdout:3/903: creat d1/d11e/f125 x:0 0 0 2024-02-15T18:27:09.047 INFO:tasks.workunit.client.0.smithi042.stdout:3/904: fdatasync d1/f7 0 2024-02-15T18:27:09.047 INFO:tasks.workunit.client.0.smithi042.stdout:3/905: readlink d1/d11/d7a/lc3 0 2024-02-15T18:27:09.047 INFO:tasks.workunit.client.1.smithi191.stdout:3/393: fsync d6/d3b/d42/f31 0 2024-02-15T18:27:09.047 INFO:tasks.workunit.client.0.smithi042.stdout:3/906: write d1/d11/d14/d4d/d73/d89/da8/fc7 [188193,32665] 0 2024-02-15T18:27:09.048 INFO:tasks.workunit.client.1.smithi191.stdout:4/490: dwrite d2/d5/d7/d2f/d41/f98 [0,4194304] 0 2024-02-15T18:27:09.051 INFO:tasks.workunit.client.0.smithi042.stdout:8/816: dread dd/de/d12/f65 [0,4194304] 0 2024-02-15T18:27:09.051 INFO:tasks.workunit.client.0.smithi042.stdout:7/940: rename d10/d15/d8f/da5/dad/d104/d65/f88 to d10/d15/d8f/da5/dad/f138 0 2024-02-15T18:27:09.051 INFO:tasks.workunit.client.0.smithi042.stdout:9/923: symlink d6/da/d2f/d7c/l137 0 2024-02-15T18:27:09.051 INFO:tasks.workunit.client.0.smithi042.stdout:9/924: dread - d6/da/d1d/d2d/fd5 zero size 2024-02-15T18:27:09.052 INFO:tasks.workunit.client.1.smithi191.stdout:9/524: creat d0/d29/fb1 x:0 0 0 2024-02-15T18:27:09.052 INFO:tasks.workunit.client.1.smithi191.stdout:9/525: truncate d0/f51 4458195 0 2024-02-15T18:27:09.052 INFO:tasks.workunit.client.1.smithi191.stdout:9/526: write d0/d7/dc/d30/d3b/fa1 [1666433,61393] 0 2024-02-15T18:27:09.052 INFO:tasks.workunit.client.0.smithi042.stdout:6/994: rmdir d9/d11/d1d 39 2024-02-15T18:27:09.053 INFO:tasks.workunit.client.0.smithi042.stdout:6/995: write d9/d11/d19/d1c/d31/f145 [477978,96280] 0 2024-02-15T18:27:09.053 INFO:tasks.workunit.client.0.smithi042.stdout:6/996: write d9/f66 [4483758,5691] 0 2024-02-15T18:27:09.056 INFO:tasks.workunit.client.0.smithi042.stdout:4/904: mkdir da/d11/d85/db5/d103/d66/d78/d53/d82/dc0/d13a/d13f 0 2024-02-15T18:27:09.056 INFO:tasks.workunit.client.1.smithi191.stdout:6/435: mkdir d8/d9/d4a/d4c/d7f 0 2024-02-15T18:27:09.058 INFO:tasks.workunit.client.0.smithi042.stdout:2/867: link d0/d8/c16 d0/d8/d10/d31/d79/c113 0 2024-02-15T18:27:09.059 INFO:tasks.workunit.client.1.smithi191.stdout:8/613: symlink d3/d2c/d1a/d23/d3e/db2/le0 0 2024-02-15T18:27:09.060 INFO:tasks.workunit.client.0.smithi042.stdout:8/817: rename dd/de/da4/f8d to dd/d11/d75/de2/fea 0 2024-02-15T18:27:09.061 INFO:tasks.workunit.client.0.smithi042.stdout:8/818: read dd/de/d33/f53 [1327436,6845] 0 2024-02-15T18:27:09.061 INFO:tasks.workunit.client.1.smithi191.stdout:3/394: symlink d6/d3b/d42/l8a 0 2024-02-15T18:27:09.062 INFO:tasks.workunit.client.0.smithi042.stdout:9/925: mknod d6/da2/c138 0 2024-02-15T18:27:09.062 INFO:tasks.workunit.client.0.smithi042.stdout:9/926: chown d6/da/d1d/d2d/d40/d5f/f10c 5783558 1 2024-02-15T18:27:09.067 INFO:tasks.workunit.client.1.smithi191.stdout:5/272: dwrite d0/d23/d41/d28/f3e [0,4194304] 0 2024-02-15T18:27:09.067 INFO:tasks.workunit.client.0.smithi042.stdout:5/935: dwrite d0/d38/d53/d68/f10a [0,4194304] 0 2024-02-15T18:27:09.068 INFO:tasks.workunit.client.1.smithi191.stdout:4/491: rename d2/d5/d14/c19 to d2/d5/d7/d3e/d63/d8c/cb9 0 2024-02-15T18:27:09.068 INFO:tasks.workunit.client.0.smithi042.stdout:4/905: mknod da/d11/da4/c140 0 2024-02-15T18:27:09.068 INFO:tasks.workunit.client.0.smithi042.stdout:0/893: dwrite d4/d38/d39/f108 [0,4194304] 0 2024-02-15T18:27:09.069 INFO:tasks.workunit.client.0.smithi042.stdout:4/906: write da/d11/d29/d4a/d4f/fef [652651,125091] 0 2024-02-15T18:27:09.069 INFO:tasks.workunit.client.0.smithi042.stdout:4/907: dread - da/d11/d29/d4a/ff1 zero size 2024-02-15T18:27:09.069 INFO:tasks.workunit.client.0.smithi042.stdout:6/997: dread d9/d11/d19/d35/d55/fac [4194304,4194304] 0 2024-02-15T18:27:09.069 INFO:tasks.workunit.client.0.smithi042.stdout:0/894: write d4/d38/d39/f6e [3271774,56555] 0 2024-02-15T18:27:09.069 INFO:tasks.workunit.client.0.smithi042.stdout:8/819: symlink dd/de/d33/dbf/leb 0 2024-02-15T18:27:09.070 INFO:tasks.workunit.client.0.smithi042.stdout:1/953: dwrite de/d15/d2c/d51/f6f [0,4194304] 0 2024-02-15T18:27:09.070 INFO:tasks.workunit.client.1.smithi191.stdout:2/462: dwrite df/d12/d32/d4d/d89/f95 [0,4194304] 0 2024-02-15T18:27:09.071 INFO:tasks.workunit.client.1.smithi191.stdout:2/463: read df/d12/d19/d1a/f3d [683756,119740] 0 2024-02-15T18:27:09.071 INFO:tasks.workunit.client.1.smithi191.stdout:9/527: symlink d0/d7/dc/d17/lb2 0 2024-02-15T18:27:09.072 INFO:tasks.workunit.client.0.smithi042.stdout:9/927: mkdir d6/dd/d9/d59/dbf/d128/d139 0 2024-02-15T18:27:09.072 INFO:tasks.workunit.client.0.smithi042.stdout:3/907: dwrite d1/f15 [0,4194304] 0 2024-02-15T18:27:09.073 INFO:tasks.workunit.client.1.smithi191.stdout:6/436: unlink d8/l52 0 2024-02-15T18:27:09.084 INFO:tasks.workunit.client.0.smithi042.stdout:1/954: dread de/d15/f20 [4194304,4194304] 0 2024-02-15T18:27:09.090 INFO:tasks.workunit.client.0.smithi042.stdout:2/868: dwrite d0/d8/f65 [0,4194304] 0 2024-02-15T18:27:09.091 INFO:tasks.workunit.client.1.smithi191.stdout:6/437: dwrite d8/d9/f44 [0,4194304] 0 2024-02-15T18:27:09.110 INFO:tasks.workunit.client.0.smithi042.stdout:2/869: dwrite d0/f8c [0,4194304] 0 2024-02-15T18:27:09.112 INFO:tasks.workunit.client.0.smithi042.stdout:2/870: write d0/d8/d10/f17 [3197035,23138] 0 2024-02-15T18:27:09.115 INFO:tasks.workunit.client.0.smithi042.stdout:2/871: dread d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/fd [0,4194304] 0 2024-02-15T18:27:09.135 INFO:tasks.workunit.client.0.smithi042.stdout:2/872: dwrite d0/d8/f70 [0,4194304] 0 2024-02-15T18:27:09.145 INFO:tasks.workunit.client.0.smithi042.stdout:2/873: dread d0/d8/d10/d7b/fe7 [0,4194304] 0 2024-02-15T18:27:09.163 INFO:tasks.workunit.client.0.smithi042.stdout:2/874: dwrite d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d2e/ff2 [0,4194304] 0 2024-02-15T18:27:09.176 INFO:tasks.workunit.client.0.smithi042.stdout:2/875: dwrite d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d2e/f110 [4194304,4194304] 0 2024-02-15T18:27:09.193 INFO:tasks.workunit.client.1.smithi191.stdout:1/568: sync 2024-02-15T18:27:09.200 INFO:tasks.workunit.client.1.smithi191.stdout:8/614: getdents d3/d2c/d1a/d23/d2e/d5b 0 2024-02-15T18:27:09.204 INFO:tasks.workunit.client.0.smithi042.stdout:5/936: symlink d0/d39/d144/df7/l149 0 2024-02-15T18:27:09.204 INFO:tasks.workunit.client.0.smithi042.stdout:5/937: readlink d0/d39/l7e 0 2024-02-15T18:27:09.204 INFO:tasks.workunit.client.1.smithi191.stdout:5/273: rmdir d0/d23/d41 39 2024-02-15T18:27:09.206 INFO:tasks.workunit.client.1.smithi191.stdout:4/492: symlink d2/d5/d14/d44/lba 0 2024-02-15T18:27:09.206 INFO:tasks.workunit.client.1.smithi191.stdout:4/493: fdatasync d2/d5/d14/f2e 0 2024-02-15T18:27:09.209 INFO:tasks.workunit.client.0.smithi042.stdout:4/908: mkdir da/d3b/d41/d110/d141 0 2024-02-15T18:27:09.210 INFO:tasks.workunit.client.0.smithi042.stdout:4/909: write da/d3b/d41/ff0 [1720216,115854] 0 2024-02-15T18:27:09.211 INFO:tasks.workunit.client.0.smithi042.stdout:6/998: creat d9/d3e/dd9/f15d x:0 0 0 2024-02-15T18:27:09.211 INFO:tasks.workunit.client.0.smithi042.stdout:4/910: write da/d11/d29/d4a/d4f/d90/fae [969388,42528] 0 2024-02-15T18:27:09.211 INFO:tasks.workunit.client.0.smithi042.stdout:4/911: write da/d11/d29/d4a/d4f/d90/fdd [573551,9494] 0 2024-02-15T18:27:09.212 INFO:tasks.workunit.client.1.smithi191.stdout:5/274: dread d0/d18/f56 [0,4194304] 0 2024-02-15T18:27:09.214 INFO:tasks.workunit.client.1.smithi191.stdout:2/464: symlink df/d12/d1f/l9a 0 2024-02-15T18:27:09.215 INFO:tasks.workunit.client.0.smithi042.stdout:0/895: symlink d4/d38/d39/df4/d114/d6c/df9/l123 0 2024-02-15T18:27:09.215 INFO:tasks.workunit.client.0.smithi042.stdout:0/896: readlink d4/d9/d8d/le4 0 2024-02-15T18:27:09.216 INFO:tasks.workunit.client.1.smithi191.stdout:9/528: symlink d0/d7/dc/d30/d3b/lb3 0 2024-02-15T18:27:09.218 INFO:tasks.workunit.client.0.smithi042.stdout:8/820: symlink dd/de/da4/d67/d5a/lec 0 2024-02-15T18:27:09.218 INFO:tasks.workunit.client.0.smithi042.stdout:8/821: read - dd/de/da4/fad zero size 2024-02-15T18:27:09.220 INFO:tasks.workunit.client.0.smithi042.stdout:4/912: dread da/d11/d85/db5/d103/d66/f8a [0,4194304] 0 2024-02-15T18:27:09.220 INFO:tasks.workunit.client.0.smithi042.stdout:4/913: truncate da/d11/d85/db5/d103/d66/d78/d2f/dca/f132 1041317 0 2024-02-15T18:27:09.222 INFO:tasks.workunit.client.1.smithi191.stdout:6/438: mkdir d8/d19/d6b/d80 0 2024-02-15T18:27:09.222 INFO:tasks.workunit.client.1.smithi191.stdout:6/439: chown d8/d9/f44 3524673 1 2024-02-15T18:27:09.222 INFO:tasks.workunit.client.1.smithi191.stdout:6/440: getdents d8/d19/d6b/d76 0 2024-02-15T18:27:09.223 INFO:tasks.workunit.client.1.smithi191.stdout:1/569: symlink d0/d8/df/d12/d2a/d2b/lbf 0 2024-02-15T18:27:09.223 INFO:tasks.workunit.client.1.smithi191.stdout:1/570: write d0/d8/df/d12/d33/d49/fb9 [578015,87148] 0 2024-02-15T18:27:09.225 INFO:tasks.workunit.client.1.smithi191.stdout:8/615: dwrite d3/d2c/d1a/f81 [0,4194304] 0 2024-02-15T18:27:09.226 INFO:tasks.workunit.client.0.smithi042.stdout:9/928: creat d6/da/d75/f13a x:0 0 0 2024-02-15T18:27:09.231 INFO:tasks.workunit.client.0.smithi042.stdout:5/938: dwrite d0/dda/d42/fb9 [0,4194304] 0 2024-02-15T18:27:09.238 INFO:tasks.workunit.client.1.smithi191.stdout:2/465: dwrite df/d12/d1f/d42/f49 [0,4194304] 0 2024-02-15T18:27:09.238 INFO:tasks.workunit.client.1.smithi191.stdout:2/466: chown df/d12/d32/d4d/d89/f95 120 1 2024-02-15T18:27:09.239 INFO:tasks.workunit.client.0.smithi042.stdout:8/822: dwrite dd/d11/f9b [0,4194304] 0 2024-02-15T18:27:09.240 INFO:tasks.workunit.client.0.smithi042.stdout:4/914: dwrite da/d11/d85/db5/d103/d66/d125/fa9 [0,4194304] 0 2024-02-15T18:27:09.240 INFO:tasks.workunit.client.0.smithi042.stdout:4/915: readlink da/d11/d85/db5/d103/lf3 0 2024-02-15T18:27:09.242 INFO:tasks.workunit.client.0.smithi042.stdout:4/916: fsync da/d11/d29/d4a/d4f/d90/fdd 0 2024-02-15T18:27:09.339 INFO:tasks.workunit.client.1.smithi191.stdout:3/395: symlink d6/d3b/l8b 0 2024-02-15T18:27:09.344 INFO:tasks.workunit.client.1.smithi191.stdout:4/494: mkdir d2/d5/d7/d3e/d63/d8c/dbb 0 2024-02-15T18:27:09.346 INFO:tasks.workunit.client.1.smithi191.stdout:9/529: creat d0/d29/d48/d66/d92/fb4 x:0 0 0 2024-02-15T18:27:09.346 INFO:tasks.workunit.client.1.smithi191.stdout:9/530: write d0/d29/d48/d66/f90 [938613,46339] 0 2024-02-15T18:27:09.349 INFO:tasks.workunit.client.1.smithi191.stdout:1/571: unlink d0/d8/df/d12/fa0 0 2024-02-15T18:27:09.349 INFO:tasks.workunit.client.1.smithi191.stdout:1/572: truncate d0/d8/f30 425185 0 2024-02-15T18:27:09.350 INFO:tasks.workunit.client.1.smithi191.stdout:1/573: dread d0/fa6 [0,4194304] 0 2024-02-15T18:27:09.355 INFO:tasks.workunit.client.0.smithi042.stdout:7/941: sync 2024-02-15T18:27:09.357 INFO:tasks.workunit.client.1.smithi191.stdout:8/616: unlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/c3c 0 2024-02-15T18:27:09.358 INFO:tasks.workunit.client.1.smithi191.stdout:2/467: getdents df/d12 0 2024-02-15T18:27:09.358 INFO:tasks.workunit.client.1.smithi191.stdout:2/468: truncate df/d12/d32/d4d/d89/f95 4814007 0 2024-02-15T18:27:09.359 INFO:tasks.workunit.client.0.smithi042.stdout:2/876: symlink d0/d8/d20/d7d/d7e/dc7/d8b/l114 0 2024-02-15T18:27:09.359 INFO:tasks.workunit.client.1.smithi191.stdout:3/396: symlink d6/d1b/d60/d73/d5e/l8c 0 2024-02-15T18:27:09.359 INFO:tasks.workunit.client.1.smithi191.stdout:3/397: dread - d6/d3b/d42/d49/f85 zero size 2024-02-15T18:27:09.359 INFO:tasks.workunit.client.1.smithi191.stdout:3/398: write f4 [5707136,9681] 0 2024-02-15T18:27:09.361 INFO:tasks.workunit.client.1.smithi191.stdout:4/495: link d2/d5/d14/fb8 d2/d5/d7/d3e/d88/fbc 0 2024-02-15T18:27:09.363 INFO:tasks.workunit.client.0.smithi042.stdout:6/999: rename d9/d11/d19/d1c/l7e to d9/d11/d19/d1c/d31/d4f/l15e 0 2024-02-15T18:27:09.365 INFO:tasks.workunit.client.1.smithi191.stdout:5/275: mkdir d0/d23/d41/d28/d31/d5a 0 2024-02-15T18:27:09.366 INFO:tasks.workunit.client.0.smithi042.stdout:1/955: sync 2024-02-15T18:27:09.366 INFO:tasks.workunit.client.1.smithi191.stdout:5/276: read d0/d18/f2e [597922,61228] 0 2024-02-15T18:27:09.366 INFO:tasks.workunit.client.0.smithi042.stdout:3/908: dwrite d1/d11/d2e/d4c/f58 [0,4194304] 0 2024-02-15T18:27:09.368 INFO:tasks.workunit.client.1.smithi191.stdout:4/496: read d2/d5/d14/f58 [1753612,98490] 0 2024-02-15T18:27:09.368 INFO:tasks.workunit.client.1.smithi191.stdout:8/617: dread d3/d2c/d1a/d23/d3e/f68 [0,4194304] 0 2024-02-15T18:27:09.370 INFO:tasks.workunit.client.1.smithi191.stdout:9/531: creat d0/d7/dc/d30/d3b/d56/fb5 x:0 0 0 2024-02-15T18:27:09.370 INFO:tasks.workunit.client.0.smithi042.stdout:0/897: mknod d4/d38/c124 0 2024-02-15T18:27:09.370 INFO:tasks.workunit.client.0.smithi042.stdout:0/898: chown d4/l10 102328 1 2024-02-15T18:27:09.371 INFO:tasks.workunit.client.0.smithi042.stdout:2/877: dread d0/d8/d20/d7d/d7e/f100 [0,4194304] 0 2024-02-15T18:27:09.371 INFO:tasks.workunit.client.0.smithi042.stdout:2/878: fsync d0/d8/d10/d31/d79/dbd/f10d 0 2024-02-15T18:27:09.372 INFO:tasks.workunit.client.1.smithi191.stdout:6/441: mkdir d8/d9/d2b/d81 0 2024-02-15T18:27:09.372 INFO:tasks.workunit.client.1.smithi191.stdout:6/442: write d8/d9/d2b/f4e [346160,4211] 0 2024-02-15T18:27:09.374 INFO:tasks.workunit.client.0.smithi042.stdout:9/929: link d6/da/d1d/d2d/c50 d6/da/d2f/c13b 0 2024-02-15T18:27:09.375 INFO:tasks.workunit.client.0.smithi042.stdout:9/930: fsync d6/da/d2f/d55/f115 0 2024-02-15T18:27:09.375 INFO:tasks.workunit.client.0.smithi042.stdout:9/931: chown d6/da/d2f/d6c/dd8/d12a 65 1 2024-02-15T18:27:09.375 INFO:tasks.workunit.client.1.smithi191.stdout:1/574: mknod d0/d8/df/d41/cc0 0 2024-02-15T18:27:09.376 INFO:tasks.workunit.client.1.smithi191.stdout:5/277: dread d0/f1f [0,4194304] 0 2024-02-15T18:27:09.376 INFO:tasks.workunit.client.1.smithi191.stdout:5/278: read - d0/d23/d41/d28/f55 zero size 2024-02-15T18:27:09.376 INFO:tasks.workunit.client.1.smithi191.stdout:2/469: rmdir df/d12/d19/d1a/d35/d47/d4b/d7a 39 2024-02-15T18:27:09.380 INFO:tasks.workunit.client.1.smithi191.stdout:3/399: rmdir d6/d5d/d72 39 2024-02-15T18:27:09.380 INFO:tasks.workunit.client.0.smithi042.stdout:5/939: mkdir d0/d39/d8d/d9b/df1/d14a 0 2024-02-15T18:27:09.382 INFO:tasks.workunit.client.1.smithi191.stdout:1/575: read d0/d8/df/d12/f28 [480346,129330] 0 2024-02-15T18:27:09.384 INFO:tasks.workunit.client.1.smithi191.stdout:8/618: rmdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/da0 39 2024-02-15T18:27:09.384 INFO:tasks.workunit.client.1.smithi191.stdout:9/532: rename d0/d7/dc/d17/d44/f6e to d0/d7/dc/d32/da3/fb6 0 2024-02-15T18:27:09.386 INFO:tasks.workunit.client.1.smithi191.stdout:2/470: link df/f40 df/d12/d19/d1a/d35/d5b/f9b 0 2024-02-15T18:27:09.388 INFO:tasks.workunit.client.1.smithi191.stdout:3/400: creat d6/d1b/d60/d73/f8d x:0 0 0 2024-02-15T18:27:09.392 INFO:tasks.workunit.client.1.smithi191.stdout:6/443: rename d8/d9/d61/f6a to d8/d9/d2b/d5b/f82 0 2024-02-15T18:27:09.395 INFO:tasks.workunit.client.1.smithi191.stdout:1/576: dread d0/d2f/d77/f81 [0,4194304] 0 2024-02-15T18:27:09.402 INFO:tasks.workunit.client.0.smithi042.stdout:4/917: getdents da/d11/d85/db5/d103/d66/d78 0 2024-02-15T18:27:09.402 INFO:tasks.workunit.client.0.smithi042.stdout:4/918: fdatasync da/d11/d29/d4a/d4f/d90/fdd 0 2024-02-15T18:27:09.407 INFO:tasks.workunit.client.1.smithi191.stdout:4/497: dwrite d2/f4 [0,4194304] 0 2024-02-15T18:27:09.407 INFO:tasks.workunit.client.1.smithi191.stdout:2/471: mkdir df/d12/d1f/d9c 0 2024-02-15T18:27:09.408 INFO:tasks.workunit.client.0.smithi042.stdout:3/909: dwrite d1/d11/d14/d4d/d73/d89/f116 [0,4194304] 0 2024-02-15T18:27:09.408 INFO:tasks.workunit.client.0.smithi042.stdout:3/910: write d1/d11/d2e/d4c/fee [207791,102438] 0 2024-02-15T18:27:09.409 INFO:tasks.workunit.client.0.smithi042.stdout:3/911: fsync d1/d11/d14/d4d/fda 0 2024-02-15T18:27:09.409 INFO:tasks.workunit.client.1.smithi191.stdout:3/401: mkdir d6/d8e 0 2024-02-15T18:27:09.409 INFO:tasks.workunit.client.1.smithi191.stdout:3/402: stat d6/d1b/d60 0 2024-02-15T18:27:09.410 INFO:tasks.workunit.client.0.smithi042.stdout:7/942: mknod d10/d15/d8f/da5/dad/d104/d65/da7/c139 0 2024-02-15T18:27:09.412 INFO:tasks.workunit.client.1.smithi191.stdout:5/279: dwrite d0/d23/d41/d32/f54 [0,4194304] 0 2024-02-15T18:27:09.414 INFO:tasks.workunit.client.1.smithi191.stdout:8/619: rename d3/d2c/d1a/d23/d3e/db2/le0 to d3/d2c/d1a/d23/d3e/le1 0 2024-02-15T18:27:09.416 INFO:tasks.workunit.client.1.smithi191.stdout:8/620: write d3/d2c/d1a/f2d [5791192,45824] 0 2024-02-15T18:27:09.417 INFO:tasks.workunit.client.1.smithi191.stdout:6/444: mknod d8/d9/d2b/d81/c83 0 2024-02-15T18:27:09.417 INFO:tasks.workunit.client.1.smithi191.stdout:4/498: symlink d2/d5/d7/lbd 0 2024-02-15T18:27:09.418 INFO:tasks.workunit.client.0.smithi042.stdout:1/956: rename de/d6d/d9e/df4/f123 to de/d6d/d9e/d109/f168 0 2024-02-15T18:27:09.423 INFO:tasks.workunit.client.1.smithi191.stdout:2/472: creat df/d12/d1f/d9c/f9d x:0 0 0 2024-02-15T18:27:09.423 INFO:tasks.workunit.client.0.smithi042.stdout:0/899: symlink d4/d38/d39/df4/d114/dfe/l125 0 2024-02-15T18:27:09.424 INFO:tasks.workunit.client.0.smithi042.stdout:5/940: getdents d0/d38/d53 0 2024-02-15T18:27:09.425 INFO:tasks.workunit.client.0.smithi042.stdout:7/943: write d10/d49/d6d/d78/d83/fa3 [1707014,94828] 0 2024-02-15T18:27:09.425 INFO:tasks.workunit.client.0.smithi042.stdout:3/912: dread d1/de/d19/f1f [4194304,4194304] 0 2024-02-15T18:27:09.425 INFO:tasks.workunit.client.0.smithi042.stdout:3/913: chown d1/de/d19/c63 9276935 1 2024-02-15T18:27:09.425 INFO:tasks.workunit.client.0.smithi042.stdout:7/944: fdatasync d10/d15/d39/f5a 0 2024-02-15T18:27:09.426 INFO:tasks.workunit.client.0.smithi042.stdout:8/823: dwrite dd/d11/f9b [4194304,4194304] 0 2024-02-15T18:27:09.427 INFO:tasks.workunit.client.0.smithi042.stdout:8/824: chown dd/de/da4/d67/l41 14 1 2024-02-15T18:27:09.428 INFO:tasks.workunit.client.0.smithi042.stdout:2/879: rename d0/d8/c16 to d0/d8/d20/d41/c115 0 2024-02-15T18:27:09.431 INFO:tasks.workunit.client.0.smithi042.stdout:9/932: truncate d6/da/f25 288885 0 2024-02-15T18:27:09.433 INFO:tasks.workunit.client.0.smithi042.stdout:0/900: mknod d4/d38/d39/d8e/c126 0 2024-02-15T18:27:09.433 INFO:tasks.workunit.client.0.smithi042.stdout:0/901: fsync d4/d9/d62/d74/fc7 0 2024-02-15T18:27:09.435 INFO:tasks.workunit.client.0.smithi042.stdout:5/941: truncate d0/d38/d53/d91/fb6 597322 0 2024-02-15T18:27:09.436 INFO:tasks.workunit.client.0.smithi042.stdout:4/919: dwrite da/d11/d29/d4a/d4f/f6c [0,4194304] 0 2024-02-15T18:27:09.436 INFO:tasks.workunit.client.0.smithi042.stdout:4/920: fdatasync da/d11/d67/f6f 0 2024-02-15T18:27:09.437 INFO:tasks.workunit.client.0.smithi042.stdout:4/921: write da/d11/da4/f116 [775127,21529] 0 2024-02-15T18:27:09.437 INFO:tasks.workunit.client.1.smithi191.stdout:5/280: dwrite d0/d18/f45 [0,4194304] 0 2024-02-15T18:27:09.442 INFO:tasks.workunit.client.0.smithi042.stdout:3/914: unlink d1/d11/d14/d4d/d6c/d7b/cd4 0 2024-02-15T18:27:09.444 INFO:tasks.workunit.client.1.smithi191.stdout:3/403: rename d6/d1b/c23 to d6/d1b/d60/d73/d5e/c8f 0 2024-02-15T18:27:09.445 INFO:tasks.workunit.client.0.smithi042.stdout:8/825: dread dd/de/d12/f45 [0,4194304] 0 2024-02-15T18:27:09.445 INFO:tasks.workunit.client.0.smithi042.stdout:8/826: truncate dd/de/d12/faf 895155 0 2024-02-15T18:27:09.450 INFO:tasks.workunit.client.0.smithi042.stdout:7/945: getdents d10/d49/dd2 0 2024-02-15T18:27:09.454 INFO:tasks.workunit.client.1.smithi191.stdout:8/621: creat d3/d2c/d1a/d23/d2e/d47/d5f/fe2 x:0 0 0 2024-02-15T18:27:09.461 INFO:tasks.workunit.client.1.smithi191.stdout:2/473: mknod df/d12/d1f/d9c/c9e 0 2024-02-15T18:27:09.462 INFO:tasks.workunit.client.0.smithi042.stdout:9/933: symlink d6/da/d2f/d6c/dd8/d117/l13c 0 2024-02-15T18:27:09.463 INFO:tasks.workunit.client.0.smithi042.stdout:9/934: dread d6/da/d1d/d2d/d40/d5f/ff9 [0,4194304] 0 2024-02-15T18:27:09.463 INFO:tasks.workunit.client.0.smithi042.stdout:0/902: creat d4/d9/d34/d8b/f127 x:0 0 0 2024-02-15T18:27:09.464 INFO:tasks.workunit.client.0.smithi042.stdout:0/903: dread - d4/d9/d34/ffa zero size 2024-02-15T18:27:09.465 INFO:tasks.workunit.client.0.smithi042.stdout:1/957: dwrite de/d54/d9a/dc8/fdb [0,4194304] 0 2024-02-15T18:27:09.465 INFO:tasks.workunit.client.0.smithi042.stdout:1/958: fsync de/d54/dbf/dd0/d7c/dd6/ddf/d88/fff 0 2024-02-15T18:27:09.465 INFO:tasks.workunit.client.1.smithi191.stdout:1/577: dwrite d0/d8/df/d12/d33/d49/fb9 [0,4194304] 0 2024-02-15T18:27:09.465 INFO:tasks.workunit.client.1.smithi191.stdout:3/404: creat d6/d1b/d60/d73/f90 x:0 0 0 2024-02-15T18:27:09.467 INFO:tasks.workunit.client.0.smithi042.stdout:2/880: dwrite d0/d8/d10/d31/d79/fea [0,4194304] 0 2024-02-15T18:27:09.467 INFO:tasks.workunit.client.1.smithi191.stdout:8/622: truncate d3/d2c/d1a/d36/f6b 1997612 0 2024-02-15T18:27:09.467 INFO:tasks.workunit.client.1.smithi191.stdout:8/623: chown d3/d2c/d1a/d23/d2e/cd9 5028 1 2024-02-15T18:27:09.470 INFO:tasks.workunit.client.1.smithi191.stdout:1/578: write d0/f9 [177522,49350] 0 2024-02-15T18:27:09.470 INFO:tasks.workunit.client.1.smithi191.stdout:6/445: mkdir d8/d9/d2b/d84 0 2024-02-15T18:27:09.470 INFO:tasks.workunit.client.1.smithi191.stdout:6/446: write d8/d9/f65 [80488,96530] 0 2024-02-15T18:27:09.473 INFO:tasks.workunit.client.1.smithi191.stdout:5/281: symlink d0/d23/d41/d28/l5b 0 2024-02-15T18:27:09.477 INFO:tasks.workunit.client.0.smithi042.stdout:3/915: rename d1/d11/d14/d4d/d73/f8b to d1/d11/d14/d4d/f126 0 2024-02-15T18:27:09.477 INFO:tasks.workunit.client.0.smithi042.stdout:8/827: unlink dd/de/d33/d66/d4e/f6c 0 2024-02-15T18:27:09.479 INFO:tasks.workunit.client.0.smithi042.stdout:7/946: symlink d10/d49/dd2/l13a 0 2024-02-15T18:27:09.481 INFO:tasks.workunit.client.1.smithi191.stdout:3/405: mknod d6/d27/c91 0 2024-02-15T18:27:09.481 INFO:tasks.workunit.client.1.smithi191.stdout:3/406: chown d6/d27/d4d/d5a/f89 3 1 2024-02-15T18:27:09.481 INFO:tasks.workunit.client.1.smithi191.stdout:3/407: dread - d6/d3b/d42/f70 zero size 2024-02-15T18:27:09.482 INFO:tasks.workunit.client.1.smithi191.stdout:8/624: truncate d3/d2c/d1a/d23/f5c 125527 0 2024-02-15T18:27:09.489 INFO:tasks.workunit.client.0.smithi042.stdout:0/904: dwrite d4/d9/d34/dec/f113 [0,4194304] 0 2024-02-15T18:27:09.489 INFO:tasks.workunit.client.0.smithi042.stdout:0/905: fsync d4/d38/d77/d2e/dae/f110 0 2024-02-15T18:27:09.490 INFO:tasks.workunit.client.0.smithi042.stdout:0/906: fdatasync d4/d38/d5c/fc4 0 2024-02-15T18:27:09.494 INFO:tasks.workunit.client.0.smithi042.stdout:1/959: dwrite de/d15/d67/d63/f69 [0,4194304] 0 2024-02-15T18:27:09.494 INFO:tasks.workunit.client.1.smithi191.stdout:8/625: dread d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/fc8 [0,4194304] 0 2024-02-15T18:27:09.495 INFO:tasks.workunit.client.0.smithi042.stdout:9/935: rmdir d6/da/d2f/d55 39 2024-02-15T18:27:09.496 INFO:tasks.workunit.client.1.smithi191.stdout:5/282: mkdir d0/d18/d5c 0 2024-02-15T18:27:09.499 INFO:tasks.workunit.client.1.smithi191.stdout:1/579: dread d0/f9 [0,4194304] 0 2024-02-15T18:27:09.499 INFO:tasks.workunit.client.1.smithi191.stdout:1/580: dread - d0/d8/f69 zero size 2024-02-15T18:27:09.499 INFO:tasks.workunit.client.1.smithi191.stdout:1/581: chown d0/d8/df/d90/d94 3426361 1 2024-02-15T18:27:09.499 INFO:tasks.workunit.client.1.smithi191.stdout:8/626: dread d3/d2c/d1a/d23/f4c [0,4194304] 0 2024-02-15T18:27:09.500 INFO:tasks.workunit.client.1.smithi191.stdout:1/582: write d0/d2f/d51/f14 [1821443,59066] 0 2024-02-15T18:27:09.500 INFO:tasks.workunit.client.1.smithi191.stdout:3/408: mkdir d6/d27/d4d/d5a/d92 0 2024-02-15T18:27:09.504 INFO:tasks.workunit.client.1.smithi191.stdout:5/283: unlink d0/f4c 0 2024-02-15T18:27:09.505 INFO:tasks.workunit.client.1.smithi191.stdout:8/627: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/ce3 0 2024-02-15T18:27:09.506 INFO:tasks.workunit.client.0.smithi042.stdout:7/947: dwrite d10/d15/f34 [4194304,4194304] 0 2024-02-15T18:27:09.507 INFO:tasks.workunit.client.1.smithi191.stdout:1/583: rename d0/d2f/c88 to d0/d2f/d77/cc1 0 2024-02-15T18:27:09.507 INFO:tasks.workunit.client.1.smithi191.stdout:1/584: fsync d0/d2f/d77/faf 0 2024-02-15T18:27:09.508 INFO:tasks.workunit.client.1.smithi191.stdout:1/585: stat d0/d2f/d77 0 2024-02-15T18:27:09.508 INFO:tasks.workunit.client.1.smithi191.stdout:1/586: readlink d0/d8/df/d41/l31 0 2024-02-15T18:27:09.508 INFO:tasks.workunit.client.1.smithi191.stdout:3/409: write d6/d3b/d30/f7c [380782,30160] 0 2024-02-15T18:27:09.508 INFO:tasks.workunit.client.1.smithi191.stdout:3/410: readlink d6/d1b/d60/d73/d5e/l8c 0 2024-02-15T18:27:09.508 INFO:tasks.workunit.client.1.smithi191.stdout:3/411: dread - d6/d3b/d42/d49/f85 zero size 2024-02-15T18:27:09.510 INFO:tasks.workunit.client.0.smithi042.stdout:2/881: creat d0/d8/f116 x:0 0 0 2024-02-15T18:27:09.511 INFO:tasks.workunit.client.1.smithi191.stdout:8/628: rename d3/d2c/d1a/d23/d2e/d5b/d69/d6d/dcf to d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/de4 0 2024-02-15T18:27:09.513 INFO:tasks.workunit.client.0.smithi042.stdout:3/916: rename d1/de/d1a/f6f to d1/d11/d7a/f127 0 2024-02-15T18:27:09.515 INFO:tasks.workunit.client.0.smithi042.stdout:5/942: dread d0/d38/d53/d91/fb6 [0,4194304] 0 2024-02-15T18:27:09.516 INFO:tasks.workunit.client.1.smithi191.stdout:1/587: dread d0/d8/df/d12/d33/f8e [0,4194304] 0 2024-02-15T18:27:09.516 INFO:tasks.workunit.client.1.smithi191.stdout:1/588: stat d0/d2f/d51/d10/d9c 0 2024-02-15T18:27:09.517 INFO:tasks.workunit.client.1.smithi191.stdout:3/412: dread d6/d3b/d42/f4a [4194304,4194304] 0 2024-02-15T18:27:09.524 INFO:tasks.workunit.client.0.smithi042.stdout:9/936: link d6/da2/db1/lfd d6/da/d2f/d7c/l13d 0 2024-02-15T18:27:09.531 INFO:tasks.workunit.client.0.smithi042.stdout:7/948: mkdir d10/d15/d39/de3/d13b 0 2024-02-15T18:27:09.531 INFO:tasks.workunit.client.0.smithi042.stdout:2/882: mknod d0/d8/d20/d3f/d7c/c117 0 2024-02-15T18:27:09.531 INFO:tasks.workunit.client.0.smithi042.stdout:3/917: creat d1/de/f128 x:0 0 0 2024-02-15T18:27:09.531 INFO:tasks.workunit.client.0.smithi042.stdout:3/918: truncate d1/d11/d7a/f127 5162878 0 2024-02-15T18:27:09.531 INFO:tasks.workunit.client.0.smithi042.stdout:3/919: write d1/d11/d2e/de7/f10d [977869,43016] 0 2024-02-15T18:27:09.531 INFO:tasks.workunit.client.0.smithi042.stdout:8/828: rename dd/de/da4/dd0/fdb to dd/de/dc0/fed 0 2024-02-15T18:27:09.531 INFO:tasks.workunit.client.0.smithi042.stdout:8/829: dread - dd/de/da4/d67/d6f/d78/d9a/dce/fd9 zero size 2024-02-15T18:27:09.532 INFO:tasks.workunit.client.0.smithi042.stdout:5/943: mkdir d0/dda/dc7/d14b 0 2024-02-15T18:27:09.533 INFO:tasks.workunit.client.0.smithi042.stdout:7/949: creat d10/d15/d8f/f13c x:0 0 0 2024-02-15T18:27:09.533 INFO:tasks.workunit.client.0.smithi042.stdout:7/950: fsync d10/d49/d6d/fbb 0 2024-02-15T18:27:09.534 INFO:tasks.workunit.client.0.smithi042.stdout:3/920: creat d1/d11/d14/d4d/d73/d89/da8/f129 x:0 0 0 2024-02-15T18:27:09.534 INFO:tasks.workunit.client.0.smithi042.stdout:8/830: truncate dd/de/d12/f45 327502 0 2024-02-15T18:27:09.535 INFO:tasks.workunit.client.0.smithi042.stdout:9/937: truncate d6/dd/d9/f72 1476425 0 2024-02-15T18:27:09.538 INFO:tasks.workunit.client.0.smithi042.stdout:0/907: rename d4/d9/d62/d74/lc3 to d4/d38/d5c/l128 0 2024-02-15T18:27:09.540 INFO:tasks.workunit.client.0.smithi042.stdout:7/951: unlink d10/d49/d6d/f85 0 2024-02-15T18:27:09.540 INFO:tasks.workunit.client.0.smithi042.stdout:7/952: fsync d10/d15/d39/d6c/ff3 0 2024-02-15T18:27:09.542 INFO:tasks.workunit.client.0.smithi042.stdout:9/938: mkdir d6/da/d2f/d7c/d13e 0 2024-02-15T18:27:09.543 INFO:tasks.workunit.client.0.smithi042.stdout:7/953: write d10/d15/d8f/da5/dad/d104/d65/ff7 [639632,12342] 0 2024-02-15T18:27:09.543 INFO:tasks.workunit.client.0.smithi042.stdout:3/921: rename d1/d11/d2e/ffe to d1/de/d19/f12a 0 2024-02-15T18:27:09.545 INFO:tasks.workunit.client.0.smithi042.stdout:7/954: mkdir d10/d49/d6d/d13d 0 2024-02-15T18:27:09.545 INFO:tasks.workunit.client.0.smithi042.stdout:7/955: stat d10/d15/d39/de3/lfa 0 2024-02-15T18:27:09.545 INFO:tasks.workunit.client.0.smithi042.stdout:7/956: dread - d10/d49/d6d/d6e/db3/f10e zero size 2024-02-15T18:27:09.546 INFO:tasks.workunit.client.0.smithi042.stdout:7/957: chown d10/d15/d39/de3/d116/d11e/l130 1220 1 2024-02-15T18:27:09.546 INFO:tasks.workunit.client.0.smithi042.stdout:9/939: write d6/dd/f20 [6513193,110779] 0 2024-02-15T18:27:09.547 INFO:tasks.workunit.client.1.smithi191.stdout:3/413: dwrite d6/d3b/d42/d49/d54/f83 [0,4194304] 0 2024-02-15T18:27:09.547 INFO:tasks.workunit.client.1.smithi191.stdout:1/589: dwrite d0/d2f/f40 [0,4194304] 0 2024-02-15T18:27:09.548 INFO:tasks.workunit.client.0.smithi042.stdout:3/922: rmdir d1/d11/d14 39 2024-02-15T18:27:09.548 INFO:tasks.workunit.client.0.smithi042.stdout:4/922: sync 2024-02-15T18:27:09.552 INFO:tasks.workunit.client.1.smithi191.stdout:3/414: dread d6/d3b/d42/d49/f59 [0,4194304] 0 2024-02-15T18:27:09.552 INFO:tasks.workunit.client.1.smithi191.stdout:1/590: write d0/d8/df/d12/f3f [3099608,30351] 0 2024-02-15T18:27:09.553 INFO:tasks.workunit.client.0.smithi042.stdout:8/831: dread dd/de/da4/d67/d5a/f64 [0,4194304] 0 2024-02-15T18:27:09.554 INFO:tasks.workunit.client.0.smithi042.stdout:9/940: dread d6/dd/f2b [4194304,4194304] 0 2024-02-15T18:27:09.555 INFO:tasks.workunit.client.0.smithi042.stdout:7/958: dread d10/d49/d6d/d78/d83/fe8 [0,4194304] 0 2024-02-15T18:27:09.556 INFO:tasks.workunit.client.0.smithi042.stdout:4/923: mkdir da/d11/d85/db5/d103/d66/d125/d142 0 2024-02-15T18:27:09.556 INFO:tasks.workunit.client.0.smithi042.stdout:7/959: write d10/f53 [1326970,108996] 0 2024-02-15T18:27:09.557 INFO:tasks.workunit.client.0.smithi042.stdout:9/941: mkdir d6/da/d1d/d2d/d13f 0 2024-02-15T18:27:09.558 INFO:tasks.workunit.client.0.smithi042.stdout:4/924: creat da/d11/d29/d4a/d4f/dcf/f143 x:0 0 0 2024-02-15T18:27:09.559 INFO:tasks.workunit.client.0.smithi042.stdout:4/925: write da/d11/d85/db5/d103/d66/d78/d2f/dca/f132 [440984,56448] 0 2024-02-15T18:27:09.559 INFO:tasks.workunit.client.0.smithi042.stdout:4/926: fdatasync da/d11/d29/d4a/d4f/dcf/f143 0 2024-02-15T18:27:09.564 INFO:tasks.workunit.client.0.smithi042.stdout:0/908: dwrite d4/d38/d77/fe6 [0,4194304] 0 2024-02-15T18:27:09.565 INFO:tasks.workunit.client.0.smithi042.stdout:2/883: dwrite d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/feb [0,4194304] 0 2024-02-15T18:27:09.565 INFO:tasks.workunit.client.0.smithi042.stdout:2/884: stat d0/d8/d20/d3f/d7c/d102/l105 0 2024-02-15T18:27:09.566 INFO:tasks.workunit.client.1.smithi191.stdout:1/591: rename d0/d2f/d51/d10/l6f to d0/d8/df/d2e/lc2 0 2024-02-15T18:27:09.566 INFO:tasks.workunit.client.1.smithi191.stdout:1/592: readlink d0/d2f/d51/lc 0 2024-02-15T18:27:09.568 INFO:tasks.workunit.client.0.smithi042.stdout:7/960: mknod d10/d15/d8f/da5/dad/c13e 0 2024-02-15T18:27:09.568 INFO:tasks.workunit.client.0.smithi042.stdout:7/961: fdatasync d10/d49/d6d/d6e/db3/f10e 0 2024-02-15T18:27:09.568 INFO:tasks.workunit.client.0.smithi042.stdout:2/885: write d0/d8/d10/d5a/fab [3479296,5817] 0 2024-02-15T18:27:09.568 INFO:tasks.workunit.client.0.smithi042.stdout:2/886: truncate d0/d8/d10/d31/d3a/d53/f58 4555416 0 2024-02-15T18:27:09.568 INFO:tasks.workunit.client.1.smithi191.stdout:3/415: mknod d6/d3b/c93 0 2024-02-15T18:27:09.569 INFO:tasks.workunit.client.0.smithi042.stdout:9/942: creat d6/dd/d9/d59/dbf/d128/f140 x:0 0 0 2024-02-15T18:27:09.570 INFO:tasks.workunit.client.1.smithi191.stdout:3/416: mkdir d6/d27/d94 0 2024-02-15T18:27:09.570 INFO:tasks.workunit.client.1.smithi191.stdout:3/417: chown d6/d3b/d42/l24 119 1 2024-02-15T18:27:09.570 INFO:tasks.workunit.client.0.smithi042.stdout:8/832: dread dd/de/d12/f1e [0,4194304] 0 2024-02-15T18:27:09.570 INFO:tasks.workunit.client.0.smithi042.stdout:8/833: truncate dd/d11/d75/f95 4606186 0 2024-02-15T18:27:09.570 INFO:tasks.workunit.client.0.smithi042.stdout:5/944: dwrite d0/d38/d53/d91/fb6 [0,4194304] 0 2024-02-15T18:27:09.571 INFO:tasks.workunit.client.1.smithi191.stdout:3/418: write d6/d1b/f39 [1997860,10774] 0 2024-02-15T18:27:09.572 INFO:tasks.workunit.client.0.smithi042.stdout:9/943: dread d6/da/d2f/f6a [0,4194304] 0 2024-02-15T18:27:09.572 INFO:tasks.workunit.client.0.smithi042.stdout:9/944: stat d6/da/d2f/d55/d73/d8d/dcf/l122 0 2024-02-15T18:27:09.573 INFO:tasks.workunit.client.0.smithi042.stdout:4/927: symlink da/d11/d85/db5/d103/d66/d78/d53/l144 0 2024-02-15T18:27:09.574 INFO:tasks.workunit.client.1.smithi191.stdout:3/419: symlink d6/d3b/d42/d49/d54/l95 0 2024-02-15T18:27:09.575 INFO:tasks.workunit.client.0.smithi042.stdout:0/909: creat d4/d9/d34/d8b/f129 x:0 0 0 2024-02-15T18:27:09.575 INFO:tasks.workunit.client.1.smithi191.stdout:3/420: symlink d6/d1b/d60/d73/d5e/l96 0 2024-02-15T18:27:09.575 INFO:tasks.workunit.client.1.smithi191.stdout:3/421: dread - d6/d1b/d60/d73/f8d zero size 2024-02-15T18:27:09.576 INFO:tasks.workunit.client.1.smithi191.stdout:1/593: dread d0/d2f/d51/f1d [0,4194304] 0 2024-02-15T18:27:09.577 INFO:tasks.workunit.client.1.smithi191.stdout:3/422: creat d6/d5d/d72/f97 x:0 0 0 2024-02-15T18:27:09.581 INFO:tasks.workunit.client.1.smithi191.stdout:1/594: truncate d0/d8/df/d12/d2a/d2b/f5a 618947 0 2024-02-15T18:27:09.581 INFO:tasks.workunit.client.1.smithi191.stdout:1/595: fsync d0/d2f/d51/d10/f59 0 2024-02-15T18:27:09.584 INFO:tasks.workunit.client.0.smithi042.stdout:7/962: mkdir d10/d15/d39/de3/d116/d13f 0 2024-02-15T18:27:09.584 INFO:tasks.workunit.client.0.smithi042.stdout:7/963: write d10/d49/d6d/d78/d83/fe8 [2764474,66847] 0 2024-02-15T18:27:09.595 INFO:tasks.workunit.client.1.smithi191.stdout:1/596: mknod d0/d8/df/d12/cc3 0 2024-02-15T18:27:09.598 INFO:tasks.workunit.client.0.smithi042.stdout:2/887: dwrite d0/fef [0,4194304] 0 2024-02-15T18:27:09.599 INFO:tasks.workunit.client.1.smithi191.stdout:1/597: mknod d0/d8/df/d12/d2a/d2b/d78/d60/d74/cc4 0 2024-02-15T18:27:09.600 INFO:tasks.workunit.client.1.smithi191.stdout:3/423: dwrite d6/d27/f3f [0,4194304] 0 2024-02-15T18:27:09.600 INFO:tasks.workunit.client.1.smithi191.stdout:3/424: fdatasync d6/d1b/d60/d73/f34 0 2024-02-15T18:27:09.605 INFO:tasks.workunit.client.0.smithi042.stdout:2/888: dread d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/f14 [0,4194304] 0 2024-02-15T18:27:09.608 INFO:tasks.workunit.client.1.smithi191.stdout:1/598: link d0/d2f/d66/c97 d0/d2f/d51/d54/cc5 0 2024-02-15T18:27:09.613 INFO:tasks.workunit.client.0.smithi042.stdout:8/834: symlink dd/de/da4/dd0/lee 0 2024-02-15T18:27:09.614 INFO:tasks.workunit.client.0.smithi042.stdout:8/835: write dd/de/da4/d67/d6f/d78/d9a/dce/fd9 [591481,108088] 0 2024-02-15T18:27:09.614 INFO:tasks.workunit.client.0.smithi042.stdout:8/836: write dd/de/d33/d66/f5f [1483022,90684] 0 2024-02-15T18:27:09.614 INFO:tasks.workunit.client.0.smithi042.stdout:5/945: unlink d0/d14/f63 0 2024-02-15T18:27:09.614 INFO:tasks.workunit.client.0.smithi042.stdout:9/945: creat d6/da/d1d/d2d/d40/d48/f141 x:0 0 0 2024-02-15T18:27:09.615 INFO:tasks.workunit.client.0.smithi042.stdout:4/928: creat da/d11/d85/db5/f145 x:0 0 0 2024-02-15T18:27:09.618 INFO:tasks.workunit.client.0.smithi042.stdout:0/910: symlink d4/d38/d77/d2e/dda/l12a 0 2024-02-15T18:27:09.621 INFO:tasks.workunit.client.0.smithi042.stdout:2/889: unlink d0/d8/l5c 0 2024-02-15T18:27:09.623 INFO:tasks.workunit.client.1.smithi191.stdout:3/425: dread d6/d3b/f84 [0,4194304] 0 2024-02-15T18:27:09.624 INFO:tasks.workunit.client.1.smithi191.stdout:1/599: dread d0/d2f/f40 [0,4194304] 0 2024-02-15T18:27:09.625 INFO:tasks.workunit.client.0.smithi042.stdout:8/837: creat dd/de/d12/fef x:0 0 0 2024-02-15T18:27:09.625 INFO:tasks.workunit.client.0.smithi042.stdout:9/946: getdents d6/da/d2f/d55/d73/d119 0 2024-02-15T18:27:09.625 INFO:tasks.workunit.client.0.smithi042.stdout:4/929: creat da/d3a/dbb/f146 x:0 0 0 2024-02-15T18:27:09.625 INFO:tasks.workunit.client.0.smithi042.stdout:4/930: chown da/d11/d29/d4a/d4f/le0 111505 1 2024-02-15T18:27:09.626 INFO:tasks.workunit.client.1.smithi191.stdout:3/426: stat d6/d3b/d42/d49/l77 0 2024-02-15T18:27:09.627 INFO:tasks.workunit.client.0.smithi042.stdout:1/960: sync 2024-02-15T18:27:09.627 INFO:tasks.workunit.client.1.smithi191.stdout:0/361: sync 2024-02-15T18:27:09.630 INFO:tasks.workunit.client.0.smithi042.stdout:0/911: mknod d4/d38/d39/df4/d114/d6c/df9/d11f/d121/dfc/d92/c12b 0 2024-02-15T18:27:09.635 INFO:tasks.workunit.client.1.smithi191.stdout:1/600: creat d0/d2f/d51/d54/d7b/fc6 x:0 0 0 2024-02-15T18:27:09.637 INFO:tasks.workunit.client.1.smithi191.stdout:0/362: mkdir d7/da/d17/d31/d7d 0 2024-02-15T18:27:09.638 INFO:tasks.workunit.client.0.smithi042.stdout:9/947: getdents d6 0 2024-02-15T18:27:09.638 INFO:tasks.workunit.client.0.smithi042.stdout:4/931: rename da/d11/d85/db5/d103/d66/fb0 to da/d11/d85/db5/d103/d66/d133/f147 0 2024-02-15T18:27:09.643 INFO:tasks.workunit.client.0.smithi042.stdout:1/961: readlink de/d15/d67/l2b 0 2024-02-15T18:27:09.643 INFO:tasks.workunit.client.0.smithi042.stdout:2/890: dread d0/f8c [0,4194304] 0 2024-02-15T18:27:09.646 INFO:tasks.workunit.client.1.smithi191.stdout:0/363: rename d7/da/f18 to d7/da/d1a/d2c/f7e 0 2024-02-15T18:27:09.649 INFO:tasks.workunit.client.0.smithi042.stdout:0/912: link d4/d38/d39/df4/d114/dfe/l119 d4/d38/d39/df4/d114/d6c/l12c 0 2024-02-15T18:27:09.651 INFO:tasks.workunit.client.0.smithi042.stdout:5/946: dwrite d0/d22/d7b/f66 [0,4194304] 0 2024-02-15T18:27:09.652 INFO:tasks.workunit.client.0.smithi042.stdout:1/962: getdents de/d6d 0 2024-02-15T18:27:09.652 INFO:tasks.workunit.client.0.smithi042.stdout:1/963: read - de/d15/d1b/d44/f140 zero size 2024-02-15T18:27:09.652 INFO:tasks.workunit.client.0.smithi042.stdout:1/964: chown de/d15/d2c/c73 2500229 1 2024-02-15T18:27:09.653 INFO:tasks.workunit.client.1.smithi191.stdout:1/601: rmdir d0 39 2024-02-15T18:27:09.653 INFO:tasks.workunit.client.0.smithi042.stdout:1/965: write de/d15/d2c/d51/d70/fba [2690756,55560] 0 2024-02-15T18:27:09.653 INFO:tasks.workunit.client.0.smithi042.stdout:1/966: write de/d54/dbf/dd0/d7c/dd6/ddf/d88/d111/f142 [991854,67342] 0 2024-02-15T18:27:09.655 INFO:tasks.workunit.client.0.smithi042.stdout:9/948: dread d6/dd/f4a [0,4194304] 0 2024-02-15T18:27:09.657 INFO:tasks.workunit.client.0.smithi042.stdout:8/838: dwrite dd/d11/f21 [0,4194304] 0 2024-02-15T18:27:09.669 INFO:tasks.workunit.client.0.smithi042.stdout:4/932: dwrite da/fa7 [0,4194304] 0 2024-02-15T18:27:09.669 INFO:tasks.workunit.client.0.smithi042.stdout:3/923: sync 2024-02-15T18:27:09.670 INFO:tasks.workunit.client.0.smithi042.stdout:3/924: stat d1/d11/d14/d4d/d6c/d7b 0 2024-02-15T18:27:09.672 INFO:tasks.workunit.client.0.smithi042.stdout:2/891: dwrite d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/fb5 [0,4194304] 0 2024-02-15T18:27:09.675 INFO:tasks.workunit.client.0.smithi042.stdout:4/933: creat da/d11/d27/f148 x:0 0 0 2024-02-15T18:27:09.675 INFO:tasks.workunit.client.0.smithi042.stdout:5/947: dread d0/d12/f21 [0,4194304] 0 2024-02-15T18:27:09.678 INFO:tasks.workunit.client.0.smithi042.stdout:4/934: truncate da/d11/d67/fd5 2548036 0 2024-02-15T18:27:09.678 INFO:tasks.workunit.client.0.smithi042.stdout:4/935: chown da/d11/d85/db5/d103/d66/d125/fa9 23656 1 2024-02-15T18:27:09.678 INFO:tasks.workunit.client.0.smithi042.stdout:5/948: unlink d0/d38/d53/d91/d94/cab 0 2024-02-15T18:27:09.680 INFO:tasks.workunit.client.0.smithi042.stdout:4/936: mknod da/d11/c149 0 2024-02-15T18:27:09.680 INFO:tasks.workunit.client.0.smithi042.stdout:4/937: chown da/d11/da4/f116 1237493 1 2024-02-15T18:27:09.680 INFO:tasks.workunit.client.0.smithi042.stdout:4/938: readlink da/d2b/lee 0 2024-02-15T18:27:09.681 INFO:tasks.workunit.client.0.smithi042.stdout:5/949: getdents d0/d12/d65 0 2024-02-15T18:27:09.681 INFO:tasks.workunit.client.0.smithi042.stdout:5/950: chown d0/d22/d7b 320072322 1 2024-02-15T18:27:09.682 INFO:tasks.workunit.client.0.smithi042.stdout:4/939: rmdir da/d11/d85/db5/d103/d66/d78/d53/d82/dc0/d13a 39 2024-02-15T18:27:09.682 INFO:tasks.workunit.client.0.smithi042.stdout:4/940: fsync da/d11/da4/f116 0 2024-02-15T18:27:09.682 INFO:tasks.workunit.client.0.smithi042.stdout:0/913: dwrite d4/d38/d39/d8e/f67 [0,4194304] 0 2024-02-15T18:27:09.683 INFO:tasks.workunit.client.0.smithi042.stdout:2/892: write d0/d8/d10/d7b/fe7 [2367800,98797] 0 2024-02-15T18:27:09.684 INFO:tasks.workunit.client.0.smithi042.stdout:5/951: rename d0/d38/d53/d91/db0/ld5 to d0/dda/dc7/de1/l14c 0 2024-02-15T18:27:09.686 INFO:tasks.workunit.client.0.smithi042.stdout:4/941: rmdir da/d11/d29/d4a/d4f/d12d 0 2024-02-15T18:27:09.687 INFO:tasks.workunit.client.0.smithi042.stdout:4/942: dread da/d3a/dc9/fd2 [0,4194304] 0 2024-02-15T18:27:09.687 INFO:tasks.workunit.client.0.smithi042.stdout:4/943: truncate da/d11/d29/d11e/d12c/ff5 568655 0 2024-02-15T18:27:09.688 INFO:tasks.workunit.client.0.smithi042.stdout:0/914: truncate d4/d9/d34/dab/db0/fed 31502 0 2024-02-15T18:27:09.689 INFO:tasks.workunit.client.0.smithi042.stdout:2/893: creat d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/dfe/f118 x:0 0 0 2024-02-15T18:27:09.690 INFO:tasks.workunit.client.0.smithi042.stdout:5/952: link d0/d38/d53/d64/ff3 d0/d38/d53/d64/de5/d143/f14d 0 2024-02-15T18:27:09.691 INFO:tasks.workunit.client.0.smithi042.stdout:4/944: mkdir da/d11/d85/db5/d103/d66/d78/d53/d82/dc0/d14a 0 2024-02-15T18:27:09.695 INFO:tasks.workunit.client.0.smithi042.stdout:7/964: sync 2024-02-15T18:27:09.695 INFO:tasks.workunit.client.0.smithi042.stdout:7/965: chown d10/d15/d8f/da5/dad/d104/d23/f55 4 1 2024-02-15T18:27:09.695 INFO:tasks.workunit.client.0.smithi042.stdout:7/966: truncate d10/d15/d39/dd9/dec/f11b 564538 0 2024-02-15T18:27:09.696 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:27:08.334867+0000 mgr.smithi042.kfoihb (mgr.24435) 43 : cephadm [INF] Upgrade: Updating mon.smithi191 2024-02-15T18:27:09.696 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:27:08.347830+0000 mon.smithi042 (mon.0) 12 : audit [INF] from='mgr.24435 ' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:09.696 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:27:08.354806+0000 mon.smithi191 (mon.1) 77 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:27:09.696 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:27:08.355605+0000 mon.smithi191 (mon.1) 78 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:27:09.696 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:27:08.356998+0000 mon.smithi191 (mon.1) 79 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:27:09.696 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cephadm 2024-02-15T18:27:08.357678+0000 mgr.smithi042.kfoihb (mgr.24435) 44 : cephadm [INF] Deploying daemon mon.smithi191 on smithi191 2024-02-15T18:27:09.696 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:27:08.683570+0000 mgr.smithi042.kfoihb (mgr.24435) 45 : cluster [DBG] pgmap v33: 65 pgs: 65 active+clean; 2.7 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 174 MiB/s rd, 379 MiB/s wr, 590 op/s 2024-02-15T18:27:09.697 INFO:tasks.workunit.client.0.smithi042.stdout:2/894: rename d0/d8/d10/fa2 to d0/d8/d20/d3f/f119 0 2024-02-15T18:27:09.698 INFO:tasks.workunit.client.0.smithi042.stdout:5/953: mknod d0/d14/ddb/c14e 0 2024-02-15T18:27:09.700 INFO:tasks.workunit.client.0.smithi042.stdout:4/945: unlink da/d11/d85/db5/f115 0 2024-02-15T18:27:09.701 INFO:tasks.workunit.client.0.smithi042.stdout:4/946: write da/d3b/d41/d8d/fb1 [745575,80032] 0 2024-02-15T18:27:09.701 INFO:tasks.workunit.client.0.smithi042.stdout:7/967: truncate d10/d15/d39/f5d 6159903 0 2024-02-15T18:27:09.702 INFO:tasks.workunit.client.0.smithi042.stdout:9/949: dwrite d6/da/d2f/d55/d73/d8d/dcf/de1/ffa [0,4194304] 0 2024-02-15T18:27:09.704 INFO:tasks.workunit.client.0.smithi042.stdout:5/954: rename d0/fce to d0/dda/dc7/f14f 0 2024-02-15T18:27:09.706 INFO:tasks.workunit.client.0.smithi042.stdout:4/947: mkdir da/d2b/d7e/d138/d14b 0 2024-02-15T18:27:09.708 INFO:tasks.workunit.client.0.smithi042.stdout:3/925: dwrite d1/d11/d14/d4d/d6c/df4/f102 [4194304,4194304] 0 2024-02-15T18:27:09.708 INFO:tasks.workunit.client.0.smithi042.stdout:8/839: dwrite dd/de/da4/d81/f88 [0,4194304] 0 2024-02-15T18:27:09.712 INFO:tasks.workunit.client.0.smithi042.stdout:7/968: unlink d10/d15/f1d 0 2024-02-15T18:27:09.712 INFO:tasks.workunit.client.0.smithi042.stdout:7/969: dread - d10/d15/d8f/da5/dad/dda/f120 zero size 2024-02-15T18:27:09.714 INFO:tasks.workunit.client.0.smithi042.stdout:9/950: creat d6/da/d1d/d2d/d40/d5f/f142 x:0 0 0 2024-02-15T18:27:09.717 INFO:tasks.workunit.client.0.smithi042.stdout:4/948: rename da/d11/d29/d4a/d4f/d90/fe4 to da/d11/d29/d4a/d4f/f14c 0 2024-02-15T18:27:09.717 INFO:tasks.workunit.client.0.smithi042.stdout:4/949: write da/d11/d85/db5/fed [283243,69201] 0 2024-02-15T18:27:09.717 INFO:tasks.workunit.client.0.smithi042.stdout:4/950: truncate da/d11/d27/f148 300094 0 2024-02-15T18:27:09.718 INFO:tasks.workunit.client.0.smithi042.stdout:4/951: write da/d11/d85/db5/d103/d66/f8a [2240411,96262] 0 2024-02-15T18:27:09.719 INFO:tasks.workunit.client.0.smithi042.stdout:8/840: read dd/de/d3c/f7e [641029,65322] 0 2024-02-15T18:27:09.726 INFO:tasks.workunit.client.0.smithi042.stdout:9/951: link d6/da/d1d/d2d/d40/c116 d6/da/d2f/c143 0 2024-02-15T18:27:09.726 INFO:tasks.workunit.client.0.smithi042.stdout:9/952: truncate d6/da/d2f/d55/d73/d8d/fe9 19757 0 2024-02-15T18:27:09.727 INFO:tasks.workunit.client.0.smithi042.stdout:5/955: dread d0/dda/dc7/de1/fd [0,4194304] 0 2024-02-15T18:27:09.727 INFO:tasks.workunit.client.0.smithi042.stdout:4/952: creat da/d11/d29/d4a/d4f/d90/f14d x:0 0 0 2024-02-15T18:27:09.728 INFO:tasks.workunit.client.0.smithi042.stdout:8/841: mkdir dd/de/d33/dbf/df0 0 2024-02-15T18:27:09.729 INFO:tasks.workunit.client.0.smithi042.stdout:9/953: link d6/dd/d9/d59/d5c/c7a d6/dd/d9/c144 0 2024-02-15T18:27:09.729 INFO:tasks.workunit.client.0.smithi042.stdout:9/954: chown d6/dd/f2b 699891846 1 2024-02-15T18:27:09.733 INFO:tasks.workunit.client.0.smithi042.stdout:4/953: getdents da/d11/d29/d11e/d12c 0 2024-02-15T18:27:09.734 INFO:tasks.workunit.client.0.smithi042.stdout:8/842: creat dd/de/da4/d67/d5a/db2/ff1 x:0 0 0 2024-02-15T18:27:09.739 INFO:tasks.workunit.client.0.smithi042.stdout:3/926: dwrite d1/d11/d14/d4d/d6c/f9b [0,4194304] 0 2024-02-15T18:27:09.739 INFO:tasks.workunit.client.0.smithi042.stdout:5/956: write d0/d14/f77 [138798,58230] 0 2024-02-15T18:27:09.739 INFO:tasks.workunit.client.0.smithi042.stdout:7/970: dwrite d10/d49/fd6 [0,4194304] 0 2024-02-15T18:27:09.740 INFO:tasks.workunit.client.0.smithi042.stdout:2/895: dwrite d0/d8/f70 [0,4194304] 0 2024-02-15T18:27:09.747 INFO:tasks.workunit.client.0.smithi042.stdout:3/927: creat d1/d11/d14/d4d/d6c/df4/f12b x:0 0 0 2024-02-15T18:27:09.747 INFO:tasks.workunit.client.0.smithi042.stdout:3/928: truncate d1/d11/d7a/fad 427862 0 2024-02-15T18:27:09.751 INFO:tasks.workunit.client.0.smithi042.stdout:5/957: mkdir d0/dda/dc7/d150 0 2024-02-15T18:27:09.754 INFO:tasks.workunit.client.0.smithi042.stdout:5/958: mkdir d0/d12/d65/d151 0 2024-02-15T18:27:09.755 INFO:tasks.workunit.client.0.smithi042.stdout:5/959: fdatasync d0/d38/d53/d64/d145/dbd/dde/f138 0 2024-02-15T18:27:09.755 INFO:tasks.workunit.client.0.smithi042.stdout:5/960: write d0/d14/d88/da1/fb1 [1826470,130054] 0 2024-02-15T18:27:09.756 INFO:tasks.workunit.client.0.smithi042.stdout:9/955: dwrite d6/da/d1d/d2d/fb4 [0,4194304] 0 2024-02-15T18:27:09.765 INFO:tasks.workunit.client.0.smithi042.stdout:1/967: sync 2024-02-15T18:27:09.766 INFO:tasks.workunit.client.0.smithi042.stdout:5/961: dread d0/dda/dc7/de1/dd8/ff4 [0,4194304] 0 2024-02-15T18:27:09.766 INFO:tasks.workunit.client.0.smithi042.stdout:9/956: creat d6/da/d1d/d2d/d40/d48/f145 x:0 0 0 2024-02-15T18:27:09.767 INFO:tasks.workunit.client.0.smithi042.stdout:1/968: readlink de/d15/d1b/l2f 0 2024-02-15T18:27:09.768 INFO:tasks.workunit.client.0.smithi042.stdout:5/962: mkdir d0/d89/d152 0 2024-02-15T18:27:09.769 INFO:tasks.workunit.client.0.smithi042.stdout:9/957: mknod d6/da/d2f/d69/c146 0 2024-02-15T18:27:09.770 INFO:tasks.workunit.client.0.smithi042.stdout:5/963: symlink d0/d38/d53/d91/db0/dd4/l153 0 2024-02-15T18:27:09.770 INFO:tasks.workunit.client.0.smithi042.stdout:5/964: chown d0/dda/d42/d7d/ld1 0 1 2024-02-15T18:27:09.771 INFO:tasks.workunit.client.0.smithi042.stdout:5/965: symlink d0/dda/dc7/de1/dd8/d13d/l154 0 2024-02-15T18:27:09.785 INFO:tasks.workunit.client.0.smithi042.stdout:2/896: dwrite d0/d8/d20/d7d/d7e/dc7/f73 [4194304,4194304] 0 2024-02-15T18:27:09.786 INFO:tasks.workunit.client.0.smithi042.stdout:2/897: mkdir d0/d8/d10/d11a 0 2024-02-15T18:27:09.787 INFO:tasks.workunit.client.0.smithi042.stdout:2/898: mkdir d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d11b 0 2024-02-15T18:27:09.788 INFO:tasks.workunit.client.0.smithi042.stdout:2/899: symlink d0/d8/d20/l11c 0 2024-02-15T18:27:09.793 INFO:tasks.workunit.client.0.smithi042.stdout:7/971: dwrite d10/d15/d8f/da5/dad/de0/de2/ffd [4194304,4194304] 0 2024-02-15T18:27:09.794 INFO:tasks.workunit.client.0.smithi042.stdout:8/843: dwrite dd/de/d33/d66/d4e/f7c [0,4194304] 0 2024-02-15T18:27:09.796 INFO:tasks.workunit.client.0.smithi042.stdout:7/972: creat d10/d15/d8f/da5/dad/d104/d23/d51/dc9/f140 x:0 0 0 2024-02-15T18:27:09.796 INFO:tasks.workunit.client.0.smithi042.stdout:3/929: dwrite d1/fd5 [0,4194304] 0 2024-02-15T18:27:09.796 INFO:tasks.workunit.client.0.smithi042.stdout:1/969: dwrite de/d15/d2c/f5a [0,4194304] 0 2024-02-15T18:27:09.796 INFO:tasks.workunit.client.0.smithi042.stdout:9/958: dwrite d6/dd/d9/da1/fdc [4194304,4194304] 0 2024-02-15T18:27:09.802 INFO:tasks.workunit.client.0.smithi042.stdout:3/930: creat d1/d11/d14/d4d/d73/d89/da8/f12c x:0 0 0 2024-02-15T18:27:09.803 INFO:tasks.workunit.client.0.smithi042.stdout:8/844: dread dd/de/da4/fa3 [0,4194304] 0 2024-02-15T18:27:09.804 INFO:tasks.workunit.client.0.smithi042.stdout:2/900: dwrite d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d2e/f110 [0,4194304] 0 2024-02-15T18:27:09.805 INFO:tasks.workunit.client.0.smithi042.stdout:2/901: write d0/d8/d20/d7d/d94/fca [942120,65335] 0 2024-02-15T18:27:09.805 INFO:tasks.workunit.client.0.smithi042.stdout:2/902: truncate d0/d8/d10/d7b/faa 5137385 0 2024-02-15T18:27:09.805 INFO:tasks.workunit.client.0.smithi042.stdout:2/903: chown d0/d8/d10/d7b/fa4 1 1 2024-02-15T18:27:09.805 INFO:tasks.workunit.client.0.smithi042.stdout:2/904: dread - d0/d8/f116 zero size 2024-02-15T18:27:09.805 INFO:tasks.workunit.client.0.smithi042.stdout:2/905: readlink d0/d8/d20/d3f/d7c/lb1 0 2024-02-15T18:27:09.805 INFO:tasks.workunit.client.0.smithi042.stdout:1/970: creat de/d54/dbf/dd0/d7c/dd6/de8/d103/d10f/f169 x:0 0 0 2024-02-15T18:27:09.805 INFO:tasks.workunit.client.0.smithi042.stdout:1/971: chown de/d54/dbf/dd0/d7c/dd6/ddf/d88/lae 1 1 2024-02-15T18:27:09.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:27:08.334867+0000 mgr.smithi042.kfoihb (mgr.24435) 43 : cephadm [INF] Upgrade: Updating mon.smithi191 2024-02-15T18:27:09.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:08.347830+0000 mon.smithi042 (mon.0) 12 : audit [INF] from='mgr.24435 ' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:09.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:08.354806+0000 mon.smithi191 (mon.1) 77 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-15T18:27:09.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:08.355605+0000 mon.smithi191 (mon.1) 78 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-15T18:27:09.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:08.356998+0000 mon.smithi191 (mon.1) 79 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:27:09.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:27:08.357678+0000 mgr.smithi042.kfoihb (mgr.24435) 44 : cephadm [INF] Deploying daemon mon.smithi191 on smithi191 2024-02-15T18:27:09.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:08.683570+0000 mgr.smithi042.kfoihb (mgr.24435) 45 : cluster [DBG] pgmap v33: 65 pgs: 65 active+clean; 2.7 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 174 MiB/s rd, 379 MiB/s wr, 590 op/s 2024-02-15T18:27:09.809 INFO:tasks.workunit.client.0.smithi042.stdout:9/959: mknod d6/da/d2f/d55/d73/d8d/dcf/c147 0 2024-02-15T18:27:09.811 INFO:tasks.workunit.client.0.smithi042.stdout:3/931: symlink d1/d11/d2e/dd1/l12d 0 2024-02-15T18:27:09.812 INFO:tasks.workunit.client.0.smithi042.stdout:7/973: dread d10/d15/d39/f5d [0,4194304] 0 2024-02-15T18:27:09.814 INFO:tasks.workunit.client.0.smithi042.stdout:8/845: truncate dd/de/fe0 2310633 0 2024-02-15T18:27:09.815 INFO:tasks.workunit.client.0.smithi042.stdout:2/906: mkdir d0/d8/d20/d7d/d7e/dc7/d8b/dd7/d11d 0 2024-02-15T18:27:09.817 INFO:tasks.workunit.client.0.smithi042.stdout:0/915: sync 2024-02-15T18:27:09.817 INFO:tasks.workunit.client.0.smithi042.stdout:0/916: write d4/d38/d39/df4/d114/d6c/df9/d11f/d121/dfc/d92/fca [4663662,101623] 0 2024-02-15T18:27:09.817 INFO:tasks.workunit.client.0.smithi042.stdout:4/954: sync 2024-02-15T18:27:09.819 INFO:tasks.workunit.client.0.smithi042.stdout:1/972: rename de/d15/d67/d74/c9b to de/d54/dbf/dd0/d7c/dd6/ddf/d88/d111/c16a 0 2024-02-15T18:27:09.821 INFO:tasks.workunit.client.0.smithi042.stdout:8/846: dread dd/f3e [0,4194304] 0 2024-02-15T18:27:09.825 INFO:tasks.workunit.client.0.smithi042.stdout:9/960: creat d6/da/d2f/d55/d73/f148 x:0 0 0 2024-02-15T18:27:09.826 INFO:tasks.workunit.client.0.smithi042.stdout:7/974: creat d10/d49/dc4/f141 x:0 0 0 2024-02-15T18:27:09.826 INFO:tasks.workunit.client.0.smithi042.stdout:0/917: creat d4/dc2/f12d x:0 0 0 2024-02-15T18:27:09.826 INFO:tasks.workunit.client.0.smithi042.stdout:0/918: fsync d4/d9/d62/d10c/f122 0 2024-02-15T18:27:09.827 INFO:tasks.workunit.client.0.smithi042.stdout:4/955: mknod da/d3a/dc9/c14e 0 2024-02-15T18:27:09.828 INFO:tasks.workunit.client.0.smithi042.stdout:3/932: rename d1/d11/d2e/ca3 to d1/de/d1a/de9/c12e 0 2024-02-15T18:27:09.830 INFO:tasks.workunit.client.0.smithi042.stdout:1/973: link de/d54/de6/df7/c115 de/d54/dbf/dd0/d7c/dd6/ddf/d88/c16b 0 2024-02-15T18:27:09.832 INFO:tasks.workunit.client.0.smithi042.stdout:9/961: dread d6/dd/f20 [0,4194304] 0 2024-02-15T18:27:09.835 INFO:tasks.workunit.client.0.smithi042.stdout:8/847: rename dd/de/d12/fc8 to dd/de/d12/ff2 0 2024-02-15T18:27:09.835 INFO:tasks.workunit.client.0.smithi042.stdout:8/848: write dd/de/d33/d3b/f56 [4253611,61758] 0 2024-02-15T18:27:09.836 INFO:tasks.workunit.client.0.smithi042.stdout:4/956: creat da/d11/d29/f14f x:0 0 0 2024-02-15T18:27:09.840 INFO:tasks.workunit.client.0.smithi042.stdout:2/907: rename d0/d8/d20/d7d/d7e/dc7/d8b/dd7/d11d to d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d11e 0 2024-02-15T18:27:09.840 INFO:tasks.workunit.client.0.smithi042.stdout:3/933: creat d1/d11/d14/dbf/f12f x:0 0 0 2024-02-15T18:27:09.840 INFO:tasks.workunit.client.0.smithi042.stdout:3/934: readlink d1/d11/d14/d4d/l55 0 2024-02-15T18:27:09.841 INFO:tasks.workunit.client.0.smithi042.stdout:1/974: creat de/f16c x:0 0 0 2024-02-15T18:27:09.842 INFO:tasks.workunit.client.0.smithi042.stdout:8/849: symlink dd/de/da4/d67/d5a/db2/lf3 0 2024-02-15T18:27:09.844 INFO:tasks.workunit.client.0.smithi042.stdout:4/957: creat da/d11/d85/db5/d103/d66/d78/d2f/d4d/f150 x:0 0 0 2024-02-15T18:27:09.845 INFO:tasks.workunit.client.0.smithi042.stdout:7/975: rename d10/d49/d6d/db6/l119 to d10/d15/d39/d6c/dba/d117/d11f/l142 0 2024-02-15T18:27:09.846 INFO:tasks.workunit.client.0.smithi042.stdout:1/975: unlink de/d15/d1b/l2f 0 2024-02-15T18:27:09.847 INFO:tasks.workunit.client.0.smithi042.stdout:1/976: chown de/d6d/ffa 9569193 1 2024-02-15T18:27:09.847 INFO:tasks.workunit.client.0.smithi042.stdout:8/850: symlink dd/de/da4/d67/d5a/db2/lf4 0 2024-02-15T18:27:09.848 INFO:tasks.workunit.client.0.smithi042.stdout:1/977: creat de/d15/d67/d46/f16d x:0 0 0 2024-02-15T18:27:09.848 INFO:tasks.workunit.client.0.smithi042.stdout:8/851: symlink dd/de/da4/d67/d6f/lf5 0 2024-02-15T18:27:09.849 INFO:tasks.workunit.client.0.smithi042.stdout:2/908: write d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d2e/fcf [351923,36368] 0 2024-02-15T18:27:09.849 INFO:tasks.workunit.client.0.smithi042.stdout:2/909: write d0/d8/d20/d7d/d94/f87 [400356,110583] 0 2024-02-15T18:27:09.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.853+0000 7fda63f31700 1 -- 172.21.15.42:0/3867045690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda5c0f55c0 msgr2=0x7fda5c0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:09.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.853+0000 7fda63f31700 1 --2- 172.21.15.42:0/3867045690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda5c0f55c0 0x7fda5c0f59e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fda58004660 tx=0x7fda58009b30).stop 2024-02-15T18:27:09.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.853+0000 7fda63f31700 1 -- 172.21.15.42:0/3867045690 shutdown_connections 2024-02-15T18:27:09.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.853+0000 7fda63f31700 1 --2- 172.21.15.42:0/3867045690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda5c0f6860 0x7fda5c0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:09.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.853+0000 7fda63f31700 1 --2- 172.21.15.42:0/3867045690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda5c0f55c0 0x7fda5c0f59e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:09.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.853+0000 7fda63f31700 1 -- 172.21.15.42:0/3867045690 >> 172.21.15.42:0/3867045690 conn(0x7fda5c0f0d40 msgr2=0x7fda5c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:09.850 INFO:tasks.workunit.client.0.smithi042.stdout:1/978: mkdir de/d54/de6/d16e 0 2024-02-15T18:27:09.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.853+0000 7fda63f31700 1 -- 172.21.15.42:0/3867045690 shutdown_connections 2024-02-15T18:27:09.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.853+0000 7fda63f31700 1 -- 172.21.15.42:0/3867045690 wait complete. 2024-02-15T18:27:09.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.854+0000 7fda63f31700 1 Processor -- start 2024-02-15T18:27:09.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.854+0000 7fda63f31700 1 -- start start 2024-02-15T18:27:09.851 INFO:tasks.workunit.client.0.smithi042.stdout:3/935: dread d1/d11/f8f [0,4194304] 0 2024-02-15T18:27:09.851 INFO:tasks.workunit.client.0.smithi042.stdout:3/936: write d1/d11/d2e/fd7 [4463393,32085] 0 2024-02-15T18:27:09.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.854+0000 7fda63f31700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda5c0f55c0 0x7fda5c105f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:09.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.854+0000 7fda63f31700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda5c0f6860 0x7fda5c106440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:09.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.854+0000 7fda63f31700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c1069f0 con 0x7fda5c0f6860 2024-02-15T18:27:09.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.854+0000 7fda63f31700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c106b30 con 0x7fda5c0f55c0 2024-02-15T18:27:09.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.854+0000 7fda614cc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda5c0f6860 0x7fda5c106440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:09.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.854+0000 7fda614cc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda5c0f6860 0x7fda5c106440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34184/0 (socket says 172.21.15.42:34184) 2024-02-15T18:27:09.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.854+0000 7fda614cc700 1 -- 172.21.15.42:0/1468148555 learned_addr learned my addr 172.21.15.42:0/1468148555 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:27:09.853 INFO:tasks.workunit.client.0.smithi042.stdout:0/919: dwrite d4/d9/d34/dab/f100 [0,4194304] 0 2024-02-15T18:27:09.853 INFO:tasks.workunit.client.0.smithi042.stdout:0/920: fsync d4/d38/d39/df4/d114/dfe/ff1 0 2024-02-15T18:27:09.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.855+0000 7fda61ccd700 1 --2- 172.21.15.42:0/1468148555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda5c0f55c0 0x7fda5c105f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:09.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.855+0000 7fda614cc700 1 -- 172.21.15.42:0/1468148555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda5c0f55c0 msgr2=0x7fda5c105f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:09.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.855+0000 7fda614cc700 1 --2- 172.21.15.42:0/1468148555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda5c0f55c0 0x7fda5c105f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:09.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.855+0000 7fda614cc700 1 -- 172.21.15.42:0/1468148555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c005040 con 0x7fda5c0f6860 2024-02-15T18:27:09.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.855+0000 7fda614cc700 1 --2- 172.21.15.42:0/1468148555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda5c0f6860 0x7fda5c106440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fda4c004870 tx=0x7fda4c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:09.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.855+0000 7fda527fc700 1 -- 172.21.15.42:0/1468148555 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda4c00e030 con 0x7fda5c0f6860 2024-02-15T18:27:09.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.855+0000 7fda527fc700 1 -- 172.21.15.42:0/1468148555 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fda4c00f040 con 0x7fda5c0f6860 2024-02-15T18:27:09.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.855+0000 7fda63f31700 1 -- 172.21.15.42:0/1468148555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda58005040 con 0x7fda5c0f6860 2024-02-15T18:27:09.854 INFO:tasks.workunit.client.0.smithi042.stdout:8/852: rename dd/de/d33/d66/d4e/f7c to dd/de/da5/dc6/ff6 0 2024-02-15T18:27:09.854 INFO:tasks.workunit.client.0.smithi042.stdout:3/937: creat d1/d11/d7a/f130 x:0 0 0 2024-02-15T18:27:09.855 INFO:tasks.workunit.client.0.smithi042.stdout:3/938: chown d1/d11/l61 8290 1 2024-02-15T18:27:09.855 INFO:tasks.workunit.client.0.smithi042.stdout:3/939: dread - d1/de/d1a/fe4 zero size 2024-02-15T18:27:09.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.855+0000 7fda63f31700 1 -- 172.21.15.42:0/1468148555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c107900 con 0x7fda5c0f6860 2024-02-15T18:27:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.856+0000 7fda527fc700 1 -- 172.21.15.42:0/1468148555 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda4c014940 con 0x7fda5c0f6860 2024-02-15T18:27:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.857+0000 7fda63f31700 1 -- 172.21.15.42:0/1468148555 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda44004fa0 con 0x7fda5c0f6860 2024-02-15T18:27:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.857+0000 7fda527fc700 1 -- 172.21.15.42:0/1468148555 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7fda4c014b90 con 0x7fda5c0f6860 2024-02-15T18:27:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.857+0000 7fda527fc700 1 --2- 172.21.15.42:0/1468148555 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fda480653c0 0x7fda48067880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.858+0000 7fda527fc700 1 -- 172.21.15.42:0/1468148555 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fda4c07c6b0 con 0x7fda5c0f6860 2024-02-15T18:27:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.858+0000 7fda61ccd700 1 --2- 172.21.15.42:0/1468148555 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fda480653c0 0x7fda48067880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:09.856 INFO:tasks.workunit.client.0.smithi042.stdout:8/853: rename dd/le7 to dd/de/da4/d67/lf7 0 2024-02-15T18:27:09.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.858+0000 7fda61ccd700 1 --2- 172.21.15.42:0/1468148555 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fda480653c0 0x7fda48067880 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fda58004630 tx=0x7fda58016040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:09.857 INFO:tasks.workunit.client.0.smithi042.stdout:3/940: mknod d1/d11e/c131 0 2024-02-15T18:27:09.857 INFO:tasks.workunit.client.0.smithi042.stdout:9/962: dwrite d6/dd/d9/f41 [4194304,4194304] 0 2024-02-15T18:27:09.857 INFO:tasks.workunit.client.0.smithi042.stdout:3/941: symlink d1/d9c/l132 0 2024-02-15T18:27:09.857 INFO:tasks.workunit.client.1.smithi191.stdout:7/487: sync 2024-02-15T18:27:09.857 INFO:tasks.workunit.client.1.smithi191.stdout:9/533: sync 2024-02-15T18:27:09.858 INFO:tasks.workunit.client.0.smithi042.stdout:0/921: dread d4/d38/d5c/fc6 [0,4194304] 0 2024-02-15T18:27:09.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:09.863+0000 7fda527fc700 1 -- 172.21.15.42:0/1468148555 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fda4c050710 con 0x7fda5c0f6860 2024-02-15T18:27:09.861 INFO:tasks.workunit.client.0.smithi042.stdout:9/963: rename d6/c60 to d6/da/d1d/d2d/d13f/c149 0 2024-02-15T18:27:09.861 INFO:tasks.workunit.client.0.smithi042.stdout:9/964: chown d6/da/d2f/d55/d73/d8d 57015165 1 2024-02-15T18:27:09.862 INFO:tasks.workunit.client.0.smithi042.stdout:3/942: rename d1/d11/f45 to d1/d11/f133 0 2024-02-15T18:27:09.862 INFO:tasks.workunit.client.0.smithi042.stdout:3/943: chown d1/d11/d14/df6/f10e 54576280 1 2024-02-15T18:27:09.862 INFO:tasks.workunit.client.0.smithi042.stdout:3/944: readlink d1/d11/d2e/dd1/l12d 0 2024-02-15T18:27:09.867 INFO:tasks.workunit.client.1.smithi191.stdout:7/488: dread d0/db/d26/d5f/d77/df/f1a [0,4194304] 0 2024-02-15T18:27:09.870 INFO:tasks.workunit.client.1.smithi191.stdout:7/489: mkdir d0/db/d26/d5f/d77/db9 0 2024-02-15T18:27:09.870 INFO:tasks.workunit.client.0.smithi042.stdout:9/965: symlink d6/da2/l14a 0 2024-02-15T18:27:09.875 INFO:tasks.workunit.client.1.smithi191.stdout:7/490: link d0/db/d65/d11/d5a/d66/d1e/l67 d0/db/d26/d5f/lba 0 2024-02-15T18:27:09.876 INFO:tasks.workunit.client.1.smithi191.stdout:7/491: rmdir d0/db/d65/d11/d88 39 2024-02-15T18:27:09.877 INFO:tasks.workunit.client.0.smithi042.stdout:1/979: dread de/d54/dbf/dd0/d7c/dd6/ddf/d88/d157/f15b [0,4194304] 0 2024-02-15T18:27:09.879 INFO:tasks.workunit.client.0.smithi042.stdout:1/980: unlink de/d15/d1b/d44/df3/l137 0 2024-02-15T18:27:09.880 INFO:tasks.workunit.client.0.smithi042.stdout:3/945: dwrite d1/de/d19/fb4 [0,4194304] 0 2024-02-15T18:27:09.883 INFO:tasks.workunit.client.0.smithi042.stdout:1/981: rename de/f5b to de/d54/dbf/dd0/d7c/dd6/de8/d103/d10f/f16f 0 2024-02-15T18:27:09.883 INFO:tasks.workunit.client.0.smithi042.stdout:1/982: stat de/d54/dbf/dd0/d7c/dd6/ddf/dca/f165 0 2024-02-15T18:27:09.883 INFO:tasks.workunit.client.0.smithi042.stdout:1/983: write de/d6d/f101 [800977,3873] 0 2024-02-15T18:27:09.883 INFO:tasks.workunit.client.0.smithi042.stdout:1/984: truncate de/d6d/d9e/d109/f168 699273 0 2024-02-15T18:27:09.886 INFO:tasks.workunit.client.0.smithi042.stdout:1/985: creat de/d15/d1b/f170 x:0 0 0 2024-02-15T18:27:09.886 INFO:tasks.workunit.client.0.smithi042.stdout:1/986: write de/d54/dbf/dd0/d7c/dd6/f151 [98937,78733] 0 2024-02-15T18:27:09.889 INFO:tasks.workunit.client.0.smithi042.stdout:5/966: sync 2024-02-15T18:27:09.889 INFO:tasks.workunit.client.0.smithi042.stdout:5/967: chown d0/c8 113323 1 2024-02-15T18:27:09.891 INFO:tasks.workunit.client.0.smithi042.stdout:5/968: creat d0/d14/d88/df8/f155 x:0 0 0 2024-02-15T18:27:09.894 INFO:tasks.workunit.client.0.smithi042.stdout:8/854: dwrite dd/de/d12/f45 [0,4194304] 0 2024-02-15T18:27:09.894 INFO:tasks.workunit.client.0.smithi042.stdout:8/855: write dd/de/da4/d67/d6f/fbe [991888,56517] 0 2024-02-15T18:27:09.894 INFO:tasks.workunit.client.0.smithi042.stdout:8/856: chown dd/de/da4/d81/f8c 1752 1 2024-02-15T18:27:09.902 INFO:tasks.workunit.client.0.smithi042.stdout:5/969: write d0/d12/f104 [3620141,46631] 0 2024-02-15T18:27:09.902 INFO:tasks.workunit.client.0.smithi042.stdout:5/970: chown d0/d12/d65/d151 20 1 2024-02-15T18:27:09.902 INFO:tasks.workunit.client.0.smithi042.stdout:5/971: chown d0/d14 38 1 2024-02-15T18:27:09.902 INFO:tasks.workunit.client.1.smithi191.stdout:9/534: dwrite d0/d7/f19 [0,4194304] 0 2024-02-15T18:27:09.905 INFO:tasks.workunit.client.1.smithi191.stdout:9/535: symlink d0/d7/dc/d30/lb7 0 2024-02-15T18:27:09.905 INFO:tasks.workunit.client.1.smithi191.stdout:9/536: fdatasync d0/d7/dc/d17/f25 0 2024-02-15T18:27:09.906 INFO:tasks.workunit.client.0.smithi042.stdout:1/987: dread de/d54/dbf/dd0/d7c/dd6/ddf/d88/d157/f15b [0,4194304] 0 2024-02-15T18:27:09.906 INFO:tasks.workunit.client.0.smithi042.stdout:1/988: write de/d15/d67/d74/f75 [240488,128187] 0 2024-02-15T18:27:09.907 INFO:tasks.workunit.client.0.smithi042.stdout:8/857: dread dd/d11/f15 [0,4194304] 0 2024-02-15T18:27:09.908 INFO:tasks.workunit.client.1.smithi191.stdout:9/537: dread d0/d29/d48/f5b [0,4194304] 0 2024-02-15T18:27:09.908 INFO:tasks.workunit.client.1.smithi191.stdout:9/538: dread - d0/d7/dc/d30/d4f/faf zero size 2024-02-15T18:27:09.908 INFO:tasks.workunit.client.0.smithi042.stdout:1/989: creat de/d6d/d9e/df4/d163/f171 x:0 0 0 2024-02-15T18:27:09.909 INFO:tasks.workunit.client.0.smithi042.stdout:5/972: dread d0/dda/dc7/de1/f10 [0,4194304] 0 2024-02-15T18:27:09.909 INFO:tasks.workunit.client.0.smithi042.stdout:1/990: write de/d54/d9a/fed [1002340,49969] 0 2024-02-15T18:27:09.909 INFO:tasks.workunit.client.1.smithi191.stdout:9/539: getdents d0/d7/dc/d8c 0 2024-02-15T18:27:09.912 INFO:tasks.workunit.client.0.smithi042.stdout:5/973: unlink d0/c33 0 2024-02-15T18:27:09.912 INFO:tasks.workunit.client.0.smithi042.stdout:1/991: mkdir de/d54/dbf/dd0/d7c/dd6/de8/d103/d14f/d172 0 2024-02-15T18:27:09.914 INFO:tasks.workunit.client.0.smithi042.stdout:5/974: link d0/dda/dc7/d52/dad/ff0 d0/d14/ddb/f156 0 2024-02-15T18:27:09.916 INFO:tasks.workunit.client.1.smithi191.stdout:7/492: dwrite d0/f3a [0,4194304] 0 2024-02-15T18:27:09.916 INFO:tasks.workunit.client.1.smithi191.stdout:7/493: write d0/db/d65/d11/d1c/fb3 [926597,101021] 0 2024-02-15T18:27:09.917 INFO:tasks.workunit.client.0.smithi042.stdout:0/922: dwrite d4/d38/d77/d2e/f32 [4194304,4194304] 0 2024-02-15T18:27:09.917 INFO:tasks.workunit.client.0.smithi042.stdout:0/923: truncate d4/d9/d34/dab/db0/fed 389251 0 2024-02-15T18:27:09.917 INFO:tasks.workunit.client.0.smithi042.stdout:0/924: dread - d4/d9/d62/d10c/f122 zero size 2024-02-15T18:27:09.917 INFO:tasks.workunit.client.0.smithi042.stdout:0/925: truncate d4/d38/d77/d2e/dda/d101/f105 804074 0 2024-02-15T18:27:09.917 INFO:tasks.workunit.client.0.smithi042.stdout:9/966: dwrite d6/da/d75/f12b [0,4194304] 0 2024-02-15T18:27:09.918 INFO:tasks.workunit.client.0.smithi042.stdout:1/992: dread de/d15/f43 [0,4194304] 0 2024-02-15T18:27:09.918 INFO:tasks.workunit.client.0.smithi042.stdout:3/946: dwrite d1/d11/f66 [0,4194304] 0 2024-02-15T18:27:09.918 INFO:tasks.workunit.client.0.smithi042.stdout:3/947: write d1/de/d19/f12a [595791,15512] 0 2024-02-15T18:27:09.919 INFO:tasks.workunit.client.1.smithi191.stdout:7/494: rename d0/db/d26/d5f/d77/l38 to d0/db/d26/d5f/d77/lbb 0 2024-02-15T18:27:09.920 INFO:tasks.workunit.client.0.smithi042.stdout:5/975: dread d0/d38/f5a [0,4194304] 0 2024-02-15T18:27:09.921 INFO:tasks.workunit.client.0.smithi042.stdout:0/926: creat d4/d38/d77/d2e/f12e x:0 0 0 2024-02-15T18:27:09.921 INFO:tasks.workunit.client.0.smithi042.stdout:0/927: readlink d4/d9/d34/d57/dc1/ld1 0 2024-02-15T18:27:09.921 INFO:tasks.workunit.client.0.smithi042.stdout:3/948: read d1/d11/d14/d4d/d73/d89/f116 [2874143,60946] 0 2024-02-15T18:27:09.922 INFO:tasks.workunit.client.0.smithi042.stdout:3/949: stat d1/d11/d14/f43 0 2024-02-15T18:27:09.922 INFO:tasks.workunit.client.0.smithi042.stdout:3/950: readlink d1/de/d1a/l36 0 2024-02-15T18:27:09.925 INFO:tasks.workunit.client.0.smithi042.stdout:5/976: rmdir d0/dda/dc7/d14b 0 2024-02-15T18:27:09.925 INFO:tasks.workunit.client.0.smithi042.stdout:5/977: fdatasync d0/f142 0 2024-02-15T18:27:09.926 INFO:tasks.workunit.client.0.smithi042.stdout:3/951: creat d1/d11/d2e/d4c/f134 x:0 0 0 2024-02-15T18:27:09.927 INFO:tasks.workunit.client.0.smithi042.stdout:3/952: readlink d1/d9c/l11b 0 2024-02-15T18:27:09.927 INFO:tasks.workunit.client.0.smithi042.stdout:3/953: fdatasync d1/d11/d2e/fd6 0 2024-02-15T18:27:09.928 INFO:tasks.workunit.client.0.smithi042.stdout:9/967: getdents d6/da/d2f/d6c/dd8/d117 0 2024-02-15T18:27:09.928 INFO:tasks.workunit.client.1.smithi191.stdout:9/540: dwrite d0/d7/dc/d17/f25 [4194304,4194304] 0 2024-02-15T18:27:09.929 INFO:tasks.workunit.client.0.smithi042.stdout:0/928: write d4/d38/d39/df4/d114/d6c/df9/d11f/d121/dfc/f78 [3576950,6999] 0 2024-02-15T18:27:09.929 INFO:tasks.workunit.client.0.smithi042.stdout:0/929: stat d4/d38/d39/d8e/f67 0 2024-02-15T18:27:09.929 INFO:tasks.workunit.client.0.smithi042.stdout:0/930: dread - d4/d38/d39/d8e/fbc zero size 2024-02-15T18:27:09.930 INFO:tasks.workunit.client.0.smithi042.stdout:5/978: truncate d0/d38/d53/d64/d145/dbd/dde/f138 3322391 0 2024-02-15T18:27:09.931 INFO:tasks.workunit.client.0.smithi042.stdout:5/979: write d0/d39/d8d/fc6 [134324,31871] 0 2024-02-15T18:27:09.931 INFO:tasks.workunit.client.0.smithi042.stdout:5/980: chown d0/d38/d53/d91/d94/f10e 4116677 1 2024-02-15T18:27:09.932 INFO:tasks.workunit.client.0.smithi042.stdout:3/954: mkdir d1/de/d1a/d135 0 2024-02-15T18:27:09.932 INFO:tasks.workunit.client.1.smithi191.stdout:7/495: dread d0/db/d65/d11/d1c/fb3 [0,4194304] 0 2024-02-15T18:27:09.932 INFO:tasks.workunit.client.1.smithi191.stdout:7/496: truncate d0/db/d26/fb2 736921 0 2024-02-15T18:27:09.934 INFO:tasks.workunit.client.0.smithi042.stdout:1/993: dwrite de/d54/dbf/dd0/fd4 [4194304,4194304] 0 2024-02-15T18:27:09.938 INFO:tasks.workunit.client.0.smithi042.stdout:9/968: symlink d6/da/d2f/d54/l14b 0 2024-02-15T18:27:09.939 INFO:tasks.workunit.client.1.smithi191.stdout:9/541: mknod d0/d7/dc/d32/da3/cb8 0 2024-02-15T18:27:09.942 INFO:tasks.workunit.client.0.smithi042.stdout:0/931: mknod d4/d9/d34/dec/c12f 0 2024-02-15T18:27:09.942 INFO:tasks.workunit.client.1.smithi191.stdout:7/497: mknod d0/db/d65/d11/d5a/cbc 0 2024-02-15T18:27:09.944 INFO:tasks.workunit.client.0.smithi042.stdout:5/981: link d0/d39/ddd/de0/f10b d0/dda/dc7/de1/dd8/d13d/f157 0 2024-02-15T18:27:09.946 INFO:tasks.workunit.client.1.smithi191.stdout:9/542: rename d0/d1d/d3f/d80/f5d to d0/d7/dc/d32/da3/fb9 0 2024-02-15T18:27:09.946 INFO:tasks.workunit.client.0.smithi042.stdout:3/955: symlink d1/d11/d14/d4d/d6c/l136 0 2024-02-15T18:27:09.950 INFO:tasks.workunit.client.1.smithi191.stdout:6/447: sync 2024-02-15T18:27:09.950 INFO:tasks.workunit.client.1.smithi191.stdout:2/474: sync 2024-02-15T18:27:09.950 INFO:tasks.workunit.client.1.smithi191.stdout:3/427: sync 2024-02-15T18:27:09.950 INFO:tasks.workunit.client.1.smithi191.stdout:0/364: sync 2024-02-15T18:27:09.951 INFO:tasks.workunit.client.1.smithi191.stdout:4/499: sync 2024-02-15T18:27:09.951 INFO:tasks.workunit.client.1.smithi191.stdout:4/500: dread - d2/d5/d7/d3e/d63/d8c/da4/fa2 zero size 2024-02-15T18:27:09.951 INFO:tasks.workunit.client.1.smithi191.stdout:5/284: sync 2024-02-15T18:27:09.951 INFO:tasks.workunit.client.1.smithi191.stdout:5/285: chown d0/d23/l2c 382146288 1 2024-02-15T18:27:09.951 INFO:tasks.workunit.client.1.smithi191.stdout:5/286: chown d0/d18 6023 1 2024-02-15T18:27:09.951 INFO:tasks.workunit.client.1.smithi191.stdout:1/602: sync 2024-02-15T18:27:09.951 INFO:tasks.workunit.client.1.smithi191.stdout:8/629: sync 2024-02-15T18:27:09.952 INFO:tasks.workunit.client.1.smithi191.stdout:7/498: rename d0/db/d65/d11/d1c/c44 to d0/db/d26/d5f/d77/df/cbd 0 2024-02-15T18:27:09.954 INFO:tasks.workunit.client.0.smithi042.stdout:1/994: creat de/d54/dbf/dd0/d7c/dd6/ddf/dca/f173 x:0 0 0 2024-02-15T18:27:09.955 INFO:tasks.workunit.client.1.smithi191.stdout:9/543: link d0/d7/dc/d30/d4f/faf d0/d7/dc/d17/d44/fba 0 2024-02-15T18:27:09.956 INFO:tasks.workunit.client.0.smithi042.stdout:9/969: mkdir d6/dd/d9/d59/dbf/d128/d14c 0 2024-02-15T18:27:09.956 INFO:tasks.workunit.client.0.smithi042.stdout:9/970: readlink d6/da/d1d/d2d/d40/l53 0 2024-02-15T18:27:09.957 INFO:tasks.workunit.client.0.smithi042.stdout:9/971: fsync d6/da/d75/fe7 0 2024-02-15T18:27:09.957 INFO:tasks.workunit.client.1.smithi191.stdout:2/475: mkdir df/d61/d9f 0 2024-02-15T18:27:09.958 INFO:tasks.workunit.client.1.smithi191.stdout:3/428: creat d6/d5d/d72/f98 x:0 0 0 2024-02-15T18:27:09.958 INFO:tasks.workunit.client.0.smithi042.stdout:0/932: rmdir d4/d38/d39/df4/d114/d6c 39 2024-02-15T18:27:09.959 INFO:tasks.workunit.client.0.smithi042.stdout:0/933: chown d4/d38/d77/d2e/dae 919325 1 2024-02-15T18:27:09.959 INFO:tasks.workunit.client.0.smithi042.stdout:5/982: creat d0/d38/d53/d64/f158 x:0 0 0 2024-02-15T18:27:09.959 INFO:tasks.workunit.client.0.smithi042.stdout:5/983: chown d0/d38/d53 1 1 2024-02-15T18:27:09.960 INFO:tasks.workunit.client.1.smithi191.stdout:5/287: getdents d0/d23/d48 0 2024-02-15T18:27:09.960 INFO:tasks.workunit.client.1.smithi191.stdout:5/288: chown d0/d18/f42 124440052 1 2024-02-15T18:27:09.960 INFO:tasks.workunit.client.1.smithi191.stdout:5/289: fsync d0/d23/d41/d28/d31/f33 0 2024-02-15T18:27:09.961 INFO:tasks.workunit.client.1.smithi191.stdout:6/448: dread d8/d9/d2b/f2e [0,4194304] 0 2024-02-15T18:27:09.961 INFO:tasks.workunit.client.1.smithi191.stdout:8/630: chown d3/l22 3 1 2024-02-15T18:27:09.961 INFO:tasks.workunit.client.1.smithi191.stdout:8/631: fsync d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/fa8 0 2024-02-15T18:27:09.962 INFO:tasks.workunit.client.1.smithi191.stdout:1/603: link d0/d8/f35 d0/d2f/d51/fc7 0 2024-02-15T18:27:09.964 INFO:tasks.workunit.client.1.smithi191.stdout:0/365: rename d7/da/d17/c33 to d7/d1b/c7f 0 2024-02-15T18:27:09.964 INFO:tasks.workunit.client.1.smithi191.stdout:0/366: write d7/da/d17/d31/d5f/f5b [361676,124375] 0 2024-02-15T18:27:09.966 INFO:tasks.workunit.client.1.smithi191.stdout:7/499: creat d0/db/d26/d5f/d77/d1d/d54/fbe x:0 0 0 2024-02-15T18:27:09.966 INFO:tasks.workunit.client.1.smithi191.stdout:7/500: dread - d0/db/d26/d5f/d77/d1d/d54/fbe zero size 2024-02-15T18:27:09.968 INFO:tasks.workunit.client.0.smithi042.stdout:9/972: dread d6/da/d111/f112 [0,4194304] 0 2024-02-15T18:27:09.972 INFO:tasks.workunit.client.1.smithi191.stdout:9/544: creat d0/d7/dc/fbb x:0 0 0 2024-02-15T18:27:09.973 INFO:tasks.workunit.client.1.smithi191.stdout:4/501: dwrite d2/d5/d7/f3b [0,4194304] 0 2024-02-15T18:27:09.975 INFO:tasks.workunit.client.1.smithi191.stdout:2/476: creat df/d12/d19/d1a/d35/fa0 x:0 0 0 2024-02-15T18:27:09.977 INFO:tasks.workunit.client.1.smithi191.stdout:3/429: write d6/d3b/f6f [725726,3911] 0 2024-02-15T18:27:09.977 INFO:tasks.workunit.client.1.smithi191.stdout:3/430: truncate d6/d3b/d30/f5c 133686 0 2024-02-15T18:27:09.978 INFO:tasks.workunit.client.1.smithi191.stdout:5/290: truncate d0/d23/d41/d28/f3f 868802 0 2024-02-15T18:27:09.978 INFO:tasks.workunit.client.1.smithi191.stdout:5/291: chown d0/d23/d41/d32/d52/l4f 342590927 1 2024-02-15T18:27:09.979 INFO:tasks.workunit.client.1.smithi191.stdout:6/449: mknod d8/c85 0 2024-02-15T18:27:09.980 INFO:tasks.workunit.client.1.smithi191.stdout:6/450: chown d8/d9/d4a/l56 82565 1 2024-02-15T18:27:09.982 INFO:tasks.workunit.client.1.smithi191.stdout:5/292: write d0/f1a [4453104,13488] 0 2024-02-15T18:27:09.982 INFO:tasks.workunit.client.1.smithi191.stdout:1/604: mkdir d0/d8/df/d90/d94/dc8 0 2024-02-15T18:27:09.988 INFO:tasks.workunit.client.0.smithi042.stdout:5/984: dwrite d0/d38/d53/d68/f10a [0,4194304] 0 2024-02-15T18:27:09.990 INFO:tasks.workunit.client.1.smithi191.stdout:0/367: dwrite d7/d4a/f59 [4194304,4194304] 0 2024-02-15T18:27:09.990 INFO:tasks.workunit.client.1.smithi191.stdout:0/368: readlink d7/da/d1a/d2c/l54 0 2024-02-15T18:27:09.991 INFO:tasks.workunit.client.1.smithi191.stdout:7/501: rmdir d0/db/d26 39 2024-02-15T18:27:09.992 INFO:tasks.workunit.client.1.smithi191.stdout:9/545: dwrite d0/d7/dc/fae [0,4194304] 0 2024-02-15T18:27:09.993 INFO:tasks.workunit.client.1.smithi191.stdout:0/369: read d7/f63 [2097943,46795] 0 2024-02-15T18:27:09.994 INFO:tasks.workunit.client.0.smithi042.stdout:1/995: creat de/d15/d67/d63/d11e/f174 x:0 0 0 2024-02-15T18:27:09.994 INFO:tasks.workunit.client.0.smithi042.stdout:1/996: stat de/d15/d2c/f144 0 2024-02-15T18:27:09.997 INFO:tasks.workunit.client.0.smithi042.stdout:0/934: rename d4/c91 to d4/d9/d34/c130 0 2024-02-15T18:27:09.997 INFO:tasks.workunit.client.0.smithi042.stdout:0/935: chown d4/f3b 512385 1 2024-02-15T18:27:09.998 INFO:tasks.workunit.client.1.smithi191.stdout:2/477: mknod df/d61/d94/ca1 0 2024-02-15T18:27:09.998 INFO:tasks.workunit.client.0.smithi042.stdout:0/936: write d4/d38/d5c/fc4 [18670,56103] 0 2024-02-15T18:27:09.999 INFO:tasks.workunit.client.0.smithi042.stdout:9/973: mkdir d6/dd/d9/d59/d5c/d14d 0 2024-02-15T18:27:09.999 INFO:tasks.workunit.client.0.smithi042.stdout:9/974: write d6/da/d2f/d55/d73/d8d/dcf/f113 [298728,92252] 0 2024-02-15T18:27:10.000 INFO:tasks.workunit.client.1.smithi191.stdout:3/431: link d6/d3b/d30/f7c d6/d8e/f99 0 2024-02-15T18:27:10.000 INFO:tasks.workunit.client.1.smithi191.stdout:3/432: dread - d6/d3b/d42/d49/f6d zero size 2024-02-15T18:27:10.002 INFO:tasks.workunit.client.0.smithi042.stdout:1/997: read de/d6d/d9e/df4/d13e/f158 [1811397,125356] 0 2024-02-15T18:27:10.002 INFO:tasks.workunit.client.0.smithi042.stdout:1/998: readlink de/d15/d67/d89/l162 0 2024-02-15T18:27:10.002 INFO:tasks.workunit.client.0.smithi042.stdout:1/999: write de/d15/d1b/f170 [739545,52213] 0 2024-02-15T18:27:10.004 INFO:tasks.workunit.client.1.smithi191.stdout:5/293: mknod d0/d23/d48/d49/c5d 0 2024-02-15T18:27:10.005 INFO:tasks.workunit.client.0.smithi042.stdout:4/958: sync 2024-02-15T18:27:10.005 INFO:tasks.workunit.client.0.smithi042.stdout:5/985: creat d0/d39/d8d/d9b/f159 x:0 0 0 2024-02-15T18:27:10.006 INFO:tasks.workunit.client.1.smithi191.stdout:4/502: dwrite d2/d5/d14/fa8 [0,4194304] 0 2024-02-15T18:27:10.006 INFO:tasks.workunit.client.1.smithi191.stdout:1/605: mkdir d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/dc9 0 2024-02-15T18:27:10.010 INFO:tasks.workunit.client.1.smithi191.stdout:6/451: dwrite d8/f15 [4194304,4194304] 0 2024-02-15T18:27:10.012 INFO:tasks.workunit.client.1.smithi191.stdout:7/502: unlink d0/db/d65/f3e 0 2024-02-15T18:27:10.012 INFO:tasks.workunit.client.1.smithi191.stdout:7/503: chown d0/db/d26/d5f/d77/f9 954803397 1 2024-02-15T18:27:10.014 INFO:tasks.workunit.client.0.smithi042.stdout:7/976: sync 2024-02-15T18:27:10.014 INFO:tasks.workunit.client.0.smithi042.stdout:2/910: sync 2024-02-15T18:27:10.017 INFO:tasks.workunit.client.1.smithi191.stdout:9/546: truncate d0/f4 4047408 0 2024-02-15T18:27:10.019 INFO:tasks.workunit.client.1.smithi191.stdout:0/370: rename d7/d3b/f6b to d7/da/d17/d31/d7d/f80 0 2024-02-15T18:27:10.025 INFO:tasks.workunit.client.1.smithi191.stdout:2/478: dwrite df/d12/d1f/d42/f49 [0,4194304] 0 2024-02-15T18:27:10.026 INFO:tasks.workunit.client.0.smithi042.stdout:9/975: unlink d6/da/d1d/d2d/c6b 0 2024-02-15T18:27:10.026 INFO:tasks.workunit.client.0.smithi042.stdout:4/959: creat da/d11/d29/d11e/d12c/f151 x:0 0 0 2024-02-15T18:27:10.026 INFO:tasks.workunit.client.0.smithi042.stdout:5/986: link d0/dda/dc7/ffd d0/d38/d53/db4/f15a 0 2024-02-15T18:27:10.027 INFO:tasks.workunit.client.0.smithi042.stdout:8/858: sync 2024-02-15T18:27:10.027 INFO:tasks.workunit.client.0.smithi042.stdout:3/956: sync 2024-02-15T18:27:10.027 INFO:tasks.workunit.client.0.smithi042.stdout:8/859: dread - dd/de/d94/fcb zero size 2024-02-15T18:27:10.027 INFO:tasks.workunit.client.0.smithi042.stdout:8/860: fdatasync dd/de/da4/d67/d6f/d78/d9a/dce/fd9 0 2024-02-15T18:27:10.029 INFO:tasks.workunit.client.1.smithi191.stdout:3/433: symlink d6/d3b/d42/l9a 0 2024-02-15T18:27:10.030 INFO:tasks.workunit.client.0.smithi042.stdout:7/977: mknod d10/d15/d8f/da5/dad/dda/c143 0 2024-02-15T18:27:10.033 INFO:tasks.workunit.client.0.smithi042.stdout:8/861: write dd/de/d3c/f54 [4074378,37398] 0 2024-02-15T18:27:10.033 INFO:tasks.workunit.client.0.smithi042.stdout:8/862: fsync dd/de/d3c/f7e 0 2024-02-15T18:27:10.033 INFO:tasks.workunit.client.0.smithi042.stdout:8/863: fdatasync dd/de/d33/d66/d4e/fd1 0 2024-02-15T18:27:10.033 INFO:tasks.workunit.client.0.smithi042.stdout:3/957: creat d1/de/f137 x:0 0 0 2024-02-15T18:27:10.034 INFO:tasks.workunit.client.1.smithi191.stdout:4/503: mknod d2/d5/d14/d71/cbe 0 2024-02-15T18:27:10.034 INFO:tasks.workunit.client.0.smithi042.stdout:2/911: link d0/d8/d10/d31/d3a/d53/fcc d0/d8/d20/d7d/d7e/dc7/f11f 0 2024-02-15T18:27:10.036 INFO:tasks.workunit.client.0.smithi042.stdout:7/978: read d10/d15/d8f/da5/dad/d104/f5f [788885,64873] 0 2024-02-15T18:27:10.039 INFO:tasks.workunit.client.0.smithi042.stdout:3/958: creat d1/de/d1a/f138 x:0 0 0 2024-02-15T18:27:10.040 INFO:tasks.workunit.client.0.smithi042.stdout:9/976: write d6/dd/f2b [4259719,84993] 0 2024-02-15T18:27:10.041 INFO:tasks.workunit.client.0.smithi042.stdout:2/912: link d0/fc8 d0/d8/d20/d41/dbe/f120 0 2024-02-15T18:27:10.042 INFO:tasks.workunit.client.1.smithi191.stdout:3/434: dread d6/d3b/d30/f45 [0,4194304] 0 2024-02-15T18:27:10.042 INFO:tasks.workunit.client.1.smithi191.stdout:3/435: readlink d6/d1b/d60/d73/d5e/l8c 0 2024-02-15T18:27:10.043 INFO:tasks.workunit.client.1.smithi191.stdout:3/436: truncate d6/d1b/d60/d73/f34 628416 0 2024-02-15T18:27:10.044 INFO:tasks.workunit.client.1.smithi191.stdout:9/547: mknod d0/d7/dc/d17/cbc 0 2024-02-15T18:27:10.045 INFO:tasks.workunit.client.1.smithi191.stdout:2/479: mkdir df/d12/d19/d1a/d35/d47/da2 0 2024-02-15T18:27:10.045 INFO:tasks.workunit.client.1.smithi191.stdout:2/480: fdatasync df/d61/f79 0 2024-02-15T18:27:10.045 INFO:tasks.workunit.client.0.smithi042.stdout:3/959: creat d1/d11/d14/df6/f139 x:0 0 0 2024-02-15T18:27:10.046 INFO:tasks.workunit.client.0.smithi042.stdout:7/979: dread d10/d15/d8f/da5/dad/de0/ff1 [0,4194304] 0 2024-02-15T18:27:10.046 INFO:tasks.workunit.client.0.smithi042.stdout:7/980: dread - d10/d15/d8f/da5/dad/de0/f12c zero size 2024-02-15T18:27:10.046 INFO:tasks.workunit.client.0.smithi042.stdout:7/981: readlink d10/d49/dd2/l13a 0 2024-02-15T18:27:10.049 INFO:tasks.workunit.client.1.smithi191.stdout:5/294: dwrite d0/f1b [0,4194304] 0 2024-02-15T18:27:10.050 INFO:tasks.workunit.client.1.smithi191.stdout:5/295: fdatasync d0/d23/d41/d28/d31/f33 0 2024-02-15T18:27:10.053 INFO:tasks.workunit.client.0.smithi042.stdout:0/937: dwrite d4/d38/d39/df4/d114/d6c/df9/d11f/d121/dfc/f78 [0,4194304] 0 2024-02-15T18:27:10.056 INFO:tasks.workunit.client.1.smithi191.stdout:4/504: symlink d2/d5/d14/d44/lbf 0 2024-02-15T18:27:10.057 INFO:tasks.workunit.client.0.smithi042.stdout:9/977: creat d6/dd/d9/d59/dbf/d128/d139/f14e x:0 0 0 2024-02-15T18:27:10.058 INFO:tasks.workunit.client.1.smithi191.stdout:3/437: link d6/d1b/d60/d73/d5e/c8f d6/d1b/c9b 0 2024-02-15T18:27:10.059 INFO:tasks.workunit.client.0.smithi042.stdout:2/913: mkdir d0/d8/d20/d3f/d7c/d102/d121 0 2024-02-15T18:27:10.060 INFO:tasks.workunit.client.1.smithi191.stdout:9/548: creat d0/d29/d48/d66/fbd x:0 0 0 2024-02-15T18:27:10.060 INFO:tasks.workunit.client.1.smithi191.stdout:9/549: fsync d0/d7/dc/d17/d33/f43 0 2024-02-15T18:27:10.060 INFO:tasks.workunit.client.0.smithi042.stdout:3/960: mknod d1/d11/c13a 0 2024-02-15T18:27:10.060 INFO:tasks.workunit.client.0.smithi042.stdout:3/961: truncate d1/d11/d2e/fd6 323459 0 2024-02-15T18:27:10.061 INFO:tasks.workunit.client.1.smithi191.stdout:2/481: mkdir df/d61/d9f/da3 0 2024-02-15T18:27:10.062 INFO:tasks.workunit.client.0.smithi042.stdout:8/864: write dd/d11/d75/de2/fea [2689637,91743] 0 2024-02-15T18:27:10.062 INFO:tasks.workunit.client.0.smithi042.stdout:8/865: write dd/d11/f74 [385117,46594] 0 2024-02-15T18:27:10.062 INFO:tasks.workunit.client.0.smithi042.stdout:0/938: creat d4/d38/d39/df4/d114/dfe/f131 x:0 0 0 2024-02-15T18:27:10.064 INFO:tasks.workunit.client.1.smithi191.stdout:5/296: link d0/d23/d41/d32/d52/l50 d0/d23/d48/d49/l5e 0 2024-02-15T18:27:10.067 INFO:tasks.workunit.client.0.smithi042.stdout:2/914: unlink d0/d8/d20/d7d/l89 0 2024-02-15T18:27:10.073 INFO:tasks.workunit.client.1.smithi191.stdout:5/297: link d0/d23/d41/d32/d52/c4d d0/d23/d41/d28/d31/c5f 0 2024-02-15T18:27:10.073 INFO:tasks.workunit.client.1.smithi191.stdout:5/298: fdatasync d0/d23/f27 0 2024-02-15T18:27:10.075 INFO:tasks.workunit.client.0.smithi042.stdout:3/962: mkdir d1/d11/d14/d4d/d6c/d7b/d13b 0 2024-02-15T18:27:10.076 INFO:tasks.workunit.client.0.smithi042.stdout:3/963: stat d1/d11/d14/d4d/d73 0 2024-02-15T18:27:10.076 INFO:tasks.workunit.client.0.smithi042.stdout:3/964: chown d1/d11/d2e/d4c/f76 103490287 1 2024-02-15T18:27:10.076 INFO:tasks.workunit.client.1.smithi191.stdout:3/438: link d6/d3b/l14 d6/d5d/d72/l9c 0 2024-02-15T18:27:10.076 INFO:tasks.workunit.client.1.smithi191.stdout:3/439: write d6/d1b/d60/d73/f8d [1028382,105908] 0 2024-02-15T18:27:10.076 INFO:tasks.workunit.client.1.smithi191.stdout:4/505: dread d2/d5/d7/d3e/d63/d8c/da4/f76 [0,4194304] 0 2024-02-15T18:27:10.077 INFO:tasks.workunit.client.0.smithi042.stdout:2/915: dread d0/d8/d10/d31/fa1 [4194304,4194304] 0 2024-02-15T18:27:10.077 INFO:tasks.workunit.client.0.smithi042.stdout:2/916: fdatasync d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/d44/f84 0 2024-02-15T18:27:10.077 INFO:tasks.workunit.client.0.smithi042.stdout:2/917: write d0/d8/d20/d7d/d7e/dc7/d8b/fb6 [6082,79987] 0 2024-02-15T18:27:10.078 INFO:tasks.workunit.client.0.smithi042.stdout:8/866: unlink dd/de/d33/d66/f5f 0 2024-02-15T18:27:10.080 INFO:tasks.workunit.client.1.smithi191.stdout:4/506: dread d2/d5/d7/d90/d52/f7b [0,4194304] 0 2024-02-15T18:27:10.080 INFO:tasks.workunit.client.1.smithi191.stdout:4/507: write d2/d5/d14/d47/d78/f68 [546207,117383] 0 2024-02-15T18:27:10.080 INFO:tasks.workunit.client.1.smithi191.stdout:7/504: dwrite d0/f99 [0,4194304] 0 2024-02-15T18:27:10.080 INFO:tasks.workunit.client.1.smithi191.stdout:7/505: fdatasync d0/db/d65/d11/d5a/d66/d1e/f9b 0 2024-02-15T18:27:10.080 INFO:tasks.workunit.client.1.smithi191.stdout:7/506: write d0/f80 [705377,117075] 0 2024-02-15T18:27:10.080 INFO:tasks.workunit.client.1.smithi191.stdout:5/299: creat d0/d23/d41/d32/d52/f60 x:0 0 0 2024-02-15T18:27:10.080 INFO:tasks.workunit.client.0.smithi042.stdout:0/939: symlink d4/d9/d8d/l132 0 2024-02-15T18:27:10.081 INFO:tasks.workunit.client.1.smithi191.stdout:3/440: mkdir d6/d1b/d82/d9d 0 2024-02-15T18:27:10.081 INFO:tasks.workunit.client.1.smithi191.stdout:3/441: write d6/d3b/d42/d49/f6d [92512,9773] 0 2024-02-15T18:27:10.081 INFO:tasks.workunit.client.0.smithi042.stdout:9/978: dread d6/da/d2f/d55/d73/d8d/fb2 [0,4194304] 0 2024-02-15T18:27:10.081 INFO:tasks.workunit.client.0.smithi042.stdout:9/979: dread - d6/dd/d9/d59/dbf/d128/f10a zero size 2024-02-15T18:27:10.082 INFO:tasks.workunit.client.0.smithi042.stdout:8/867: mkdir dd/de/d33/d66/d4e/df8 0 2024-02-15T18:27:10.083 INFO:tasks.workunit.client.1.smithi191.stdout:4/508: creat d2/d5/d7/d3e/fc0 x:0 0 0 2024-02-15T18:27:10.087 INFO:tasks.workunit.client.1.smithi191.stdout:7/507: symlink d0/lbf 0 2024-02-15T18:27:10.088 INFO:tasks.workunit.client.0.smithi042.stdout:0/940: truncate d4/d38/d39/f6e 3124145 0 2024-02-15T18:27:10.088 INFO:tasks.workunit.client.0.smithi042.stdout:0/941: chown d4/d38/d39/ca5 169 1 2024-02-15T18:27:10.088 INFO:tasks.workunit.client.0.smithi042.stdout:0/942: fsync d4/d38/d39/d8e/fbc 0 2024-02-15T18:27:10.088 INFO:tasks.workunit.client.0.smithi042.stdout:0/943: write d4/d38/d39/d8e/fbb [5850889,16934] 0 2024-02-15T18:27:10.088 INFO:tasks.workunit.client.0.smithi042.stdout:9/980: rmdir d6/da2/dfc 39 2024-02-15T18:27:10.088 INFO:tasks.workunit.client.1.smithi191.stdout:5/300: truncate d0/d23/d41/d28/d31/f47 195639 0 2024-02-15T18:27:10.090 INFO:tasks.workunit.client.1.smithi191.stdout:3/442: rename d6/d1b/d60/d73/d5e/c86 to d6/d27/d4d/d5a/d92/c9e 0 2024-02-15T18:27:10.092 INFO:tasks.workunit.client.0.smithi042.stdout:3/965: dread d1/f7 [0,4194304] 0 2024-02-15T18:27:10.092 INFO:tasks.workunit.client.0.smithi042.stdout:5/987: dwrite d0/d38/d53/d91/fb6 [0,4194304] 0 2024-02-15T18:27:10.093 INFO:tasks.workunit.client.0.smithi042.stdout:5/988: chown d0/d38/d53/d64/f117 232 1 2024-02-15T18:27:10.093 INFO:tasks.workunit.client.0.smithi042.stdout:3/966: write d1/de/d19/fc9 [4741468,86668] 0 2024-02-15T18:27:10.093 INFO:tasks.workunit.client.0.smithi042.stdout:3/967: write d1/de/d19/f12a [1253562,80872] 0 2024-02-15T18:27:10.093 INFO:tasks.workunit.client.1.smithi191.stdout:4/509: unlink d2/d5/c1c 0 2024-02-15T18:27:10.093 INFO:tasks.workunit.client.1.smithi191.stdout:0/371: dwrite d7/d4a/f59 [8388608,4194304] 0 2024-02-15T18:27:10.093 INFO:tasks.workunit.client.1.smithi191.stdout:0/372: dread - d7/da/d1a/d34/d27/f6f zero size 2024-02-15T18:27:10.094 INFO:tasks.workunit.client.0.smithi042.stdout:9/981: truncate d6/da2/fac 553806 0 2024-02-15T18:27:10.094 INFO:tasks.workunit.client.0.smithi042.stdout:4/960: dwrite da/d11/d85/db5/d103/d66/d125/fa1 [4194304,4194304] 0 2024-02-15T18:27:10.094 INFO:tasks.workunit.client.0.smithi042.stdout:9/982: write d6/dd/dcc/ffb [1733583,87171] 0 2024-02-15T18:27:10.095 INFO:tasks.workunit.client.1.smithi191.stdout:7/508: symlink d0/db/d65/d11/d5a/lc0 0 2024-02-15T18:27:10.095 INFO:tasks.workunit.client.1.smithi191.stdout:5/301: mkdir d0/d23/d41/d28/d31/d5a/d61 0 2024-02-15T18:27:10.096 INFO:tasks.workunit.client.1.smithi191.stdout:5/302: chown d0/d23/d41/d28/d31 12776672 1 2024-02-15T18:27:10.097 INFO:tasks.workunit.client.1.smithi191.stdout:3/443: creat d6/d1b/d60/d73/f9f x:0 0 0 2024-02-15T18:27:10.098 INFO:tasks.workunit.client.0.smithi042.stdout:5/989: truncate d0/d22/fee 1602940 0 2024-02-15T18:27:10.099 INFO:tasks.workunit.client.1.smithi191.stdout:6/452: dwrite d8/d9/d2b/f4b [0,4194304] 0 2024-02-15T18:27:10.099 INFO:tasks.workunit.client.1.smithi191.stdout:2/482: dwrite df/d12/d1f/f34 [0,4194304] 0 2024-02-15T18:27:10.099 INFO:tasks.workunit.client.1.smithi191.stdout:9/550: dwrite d0/d7/dc/f15 [4194304,4194304] 0 2024-02-15T18:27:10.099 INFO:tasks.workunit.client.1.smithi191.stdout:9/551: chown d0/d7/dc/fbb 63 1 2024-02-15T18:27:10.100 INFO:tasks.workunit.client.0.smithi042.stdout:2/918: dwrite d0/d8/d10/fd9 [0,4194304] 0 2024-02-15T18:27:10.101 INFO:tasks.workunit.client.1.smithi191.stdout:4/510: creat d2/d5/fc1 x:0 0 0 2024-02-15T18:27:10.105 INFO:tasks.workunit.client.0.smithi042.stdout:8/868: dwrite dd/de/d12/f62 [0,4194304] 0 2024-02-15T18:27:10.105 INFO:tasks.workunit.client.0.smithi042.stdout:8/869: chown dd/de/d33/d66/ca6 200009486 1 2024-02-15T18:27:10.111 INFO:tasks.workunit.client.0.smithi042.stdout:3/968: rename d1/de/d1a/f9e to d1/f13c 0 2024-02-15T18:27:10.112 INFO:tasks.workunit.client.1.smithi191.stdout:2/483: dread df/d12/d1f/f34 [0,4194304] 0 2024-02-15T18:27:10.113 INFO:tasks.workunit.client.1.smithi191.stdout:2/484: read df/f10 [1083661,121040] 0 2024-02-15T18:27:10.119 INFO:tasks.workunit.client.1.smithi191.stdout:4/511: dread d2/d5/d7/d2f/f51 [0,4194304] 0 2024-02-15T18:27:10.119 INFO:tasks.workunit.client.1.smithi191.stdout:4/512: fsync d2/d5/d14/d47/d78/f6a 0 2024-02-15T18:27:10.119 INFO:tasks.workunit.client.0.smithi042.stdout:0/944: dwrite d4/d9/d62/fa0 [0,4194304] 0 2024-02-15T18:27:10.120 INFO:tasks.workunit.client.1.smithi191.stdout:7/509: mkdir d0/db/d65/dc/d4b/dc1 0 2024-02-15T18:27:10.120 INFO:tasks.workunit.client.1.smithi191.stdout:7/510: fsync d0/db/d26/d5f/d77/df/f19 0 2024-02-15T18:27:10.121 INFO:tasks.workunit.client.1.smithi191.stdout:9/552: dwrite d0/d7/fab [0,4194304] 0 2024-02-15T18:27:10.121 INFO:tasks.workunit.client.1.smithi191.stdout:9/553: write d0/d7/f21 [4954536,25242] 0 2024-02-15T18:27:10.126 INFO:tasks.workunit.client.1.smithi191.stdout:5/303: getdents d0/d18 0 2024-02-15T18:27:10.126 INFO:tasks.workunit.client.1.smithi191.stdout:5/304: chown d0/d23/d41/d32/f54 39 1 2024-02-15T18:27:10.131 INFO:tasks.workunit.client.1.smithi191.stdout:5/305: dread d0/d18/f56 [0,4194304] 0 2024-02-15T18:27:10.131 INFO:tasks.workunit.client.1.smithi191.stdout:6/453: creat d8/d9/f86 x:0 0 0 2024-02-15T18:27:10.132 INFO:tasks.workunit.client.0.smithi042.stdout:0/945: dread d4/f20 [0,4194304] 0 2024-02-15T18:27:10.132 INFO:tasks.workunit.client.0.smithi042.stdout:0/946: write d4/f41 [4988155,68860] 0 2024-02-15T18:27:10.136 INFO:tasks.workunit.client.0.smithi042.stdout:9/983: unlink d6/da/d2f/d55/d73/d8d/cd6 0 2024-02-15T18:27:10.138 INFO:tasks.workunit.client.1.smithi191.stdout:2/485: mknod df/d12/d1f/d42/ca4 0 2024-02-15T18:27:10.138 INFO:tasks.workunit.client.1.smithi191.stdout:2/486: write df/d12/f55 [49210,126212] 0 2024-02-15T18:27:10.140 INFO:tasks.workunit.client.1.smithi191.stdout:4/513: truncate d2/d5/d7/f93 2688826 0 2024-02-15T18:27:10.140 INFO:tasks.workunit.client.1.smithi191.stdout:4/514: write d2/d5/d7/d3e/d63/d8c/da4/fa2 [506405,87925] 0 2024-02-15T18:27:10.141 INFO:tasks.workunit.client.0.smithi042.stdout:2/919: getdents d0/d8/d20/d7d/d7e/dc7 0 2024-02-15T18:27:10.141 INFO:tasks.workunit.client.0.smithi042.stdout:2/920: chown d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/c9f 101022991 1 2024-02-15T18:27:10.144 INFO:tasks.workunit.client.1.smithi191.stdout:7/511: link d0/db/l41 d0/db/d26/d5f/d77/d1d/d54/lc2 0 2024-02-15T18:27:10.144 INFO:tasks.workunit.client.1.smithi191.stdout:7/512: chown d0/db/d65/d11/d5a/d66/d1e/l67 1 1 2024-02-15T18:27:10.147 INFO:tasks.workunit.client.1.smithi191.stdout:9/554: mknod d0/d7/dc/d30/d3b/d8b/cbe 0 2024-02-15T18:27:10.149 INFO:tasks.workunit.client.0.smithi042.stdout:4/961: rename da/d11/d29/f14f to da/d11/d85/db5/f152 0 2024-02-15T18:27:10.149 INFO:tasks.workunit.client.1.smithi191.stdout:6/454: truncate d8/d9/d2b/d57/f6c 1237877 0 2024-02-15T18:27:10.150 INFO:tasks.workunit.client.1.smithi191.stdout:5/306: dwrite d0/f46 [0,4194304] 0 2024-02-15T18:27:10.151 INFO:tasks.workunit.client.1.smithi191.stdout:4/515: dread d2/d5/d86/f9d [0,4194304] 0 2024-02-15T18:27:10.151 INFO:tasks.workunit.client.1.smithi191.stdout:4/516: fdatasync d2/d5/fc1 0 2024-02-15T18:27:10.151 INFO:tasks.workunit.client.1.smithi191.stdout:4/517: chown d2/d5/d14 11108 1 2024-02-15T18:27:10.152 INFO:tasks.workunit.client.1.smithi191.stdout:6/455: read d8/d9/d2b/f4e [275268,127536] 0 2024-02-15T18:27:10.152 INFO:tasks.workunit.client.1.smithi191.stdout:2/487: rmdir df/d12/d32 39 2024-02-15T18:27:10.153 INFO:tasks.workunit.client.1.smithi191.stdout:2/488: write df/d12/d19/d1a/d35/d5b/f6e [2916264,81336] 0 2024-02-15T18:27:10.161 INFO:tasks.workunit.client.1.smithi191.stdout:7/513: truncate d0/db/d26/d5f/d77/f51 7738076 0 2024-02-15T18:27:10.165 INFO:tasks.workunit.client.1.smithi191.stdout:6/456: symlink d8/d19/d6b/d72/l87 0 2024-02-15T18:27:10.165 INFO:tasks.workunit.client.1.smithi191.stdout:6/457: readlink d8/d9/d4a/d4c/l70 0 2024-02-15T18:27:10.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.170+0000 7fda63f31700 1 -- 172.21.15.42:0/1468148555 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fda44000bc0 con 0x7fda480653c0 2024-02-15T18:27:10.167 INFO:tasks.workunit.client.0.smithi042.stdout:0/947: rmdir d4/dc2 39 2024-02-15T18:27:10.167 INFO:tasks.workunit.client.0.smithi042.stdout:0/948: readlink d4/l11 0 2024-02-15T18:27:10.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.172+0000 7fda527fc700 1 -- 172.21.15.42:0/1468148555 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fda44000bc0 con 0x7fda480653c0 2024-02-15T18:27:10.172 INFO:tasks.workunit.client.0.smithi042.stdout:2/921: mkdir d0/d8/d10/d11a/d122 0 2024-02-15T18:27:10.172 INFO:tasks.workunit.client.0.smithi042.stdout:2/922: read - d0/d8/d10/d31/fc0 zero size 2024-02-15T18:27:10.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.176+0000 7fda63f31700 1 -- 172.21.15.42:0/1468148555 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fda480653c0 msgr2=0x7fda48067880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:10.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.176+0000 7fda63f31700 1 --2- 172.21.15.42:0/1468148555 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fda480653c0 0x7fda48067880 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fda58004630 tx=0x7fda58016040).stop 2024-02-15T18:27:10.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.176+0000 7fda63f31700 1 -- 172.21.15.42:0/1468148555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda5c0f6860 msgr2=0x7fda5c106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:10.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.176+0000 7fda63f31700 1 --2- 172.21.15.42:0/1468148555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda5c0f6860 0x7fda5c106440 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fda4c004870 tx=0x7fda4c009b30).stop 2024-02-15T18:27:10.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.176+0000 7fda63f31700 1 -- 172.21.15.42:0/1468148555 shutdown_connections 2024-02-15T18:27:10.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.176+0000 7fda63f31700 1 --2- 172.21.15.42:0/1468148555 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fda480653c0 0x7fda48067880 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:10.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.176+0000 7fda63f31700 1 --2- 172.21.15.42:0/1468148555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda5c0f55c0 0x7fda5c105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:10.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.176+0000 7fda63f31700 1 --2- 172.21.15.42:0/1468148555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fda5c0f6860 0x7fda5c106440 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:10.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.176+0000 7fda63f31700 1 -- 172.21.15.42:0/1468148555 >> 172.21.15.42:0/1468148555 conn(0x7fda5c0f0d40 msgr2=0x7fda5c0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:10.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.176+0000 7fda63f31700 1 -- 172.21.15.42:0/1468148555 shutdown_connections 2024-02-15T18:27:10.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.176+0000 7fda63f31700 1 -- 172.21.15.42:0/1468148555 wait complete. 2024-02-15T18:27:10.176 INFO:tasks.workunit.client.0.smithi042.stdout:8/870: rename dd/de/da5/la7 to dd/de/da4/d67/d5a/lf9 0 2024-02-15T18:27:10.177 INFO:tasks.workunit.client.0.smithi042.stdout:4/962: rmdir da/d11/d85/db5/d103/d66/d78/d2f 39 2024-02-15T18:27:10.177 INFO:tasks.workunit.client.0.smithi042.stdout:4/963: dread - da/d3a/dc9/f128 zero size 2024-02-15T18:27:10.177 INFO:tasks.workunit.client.0.smithi042.stdout:4/964: chown da/d2b/lc4 10793 1 2024-02-15T18:27:10.177 INFO:tasks.workunit.client.1.smithi191.stdout:2/489: mkdir df/d12/d1e/da5 0 2024-02-15T18:27:10.178 INFO:tasks.workunit.client.0.smithi042.stdout:0/949: mknod d4/d38/d39/df4/d114/d6c/df9/d11f/c133 0 2024-02-15T18:27:10.179 INFO:tasks.workunit.client.1.smithi191.stdout:7/514: creat d0/db/d26/d5f/d77/d35/d42/fc3 x:0 0 0 2024-02-15T18:27:10.179 INFO:tasks.workunit.client.1.smithi191.stdout:7/515: write d0/db/d26/d5f/d77/f6a [515535,89654] 0 2024-02-15T18:27:10.181 INFO:tasks.workunit.client.0.smithi042.stdout:5/990: rmdir d0/d38/d53/d91/d94/d107 0 2024-02-15T18:27:10.181 INFO:tasks.workunit.client.0.smithi042.stdout:5/991: dread - d0/d39/d144/ff2 zero size 2024-02-15T18:27:10.181 INFO:tasks.workunit.client.1.smithi191.stdout:2/490: stat df/d61/c72 0 2024-02-15T18:27:10.184 INFO:tasks.workunit.client.0.smithi042.stdout:3/969: rename d1/d11/d14/d4d/d73/d89/da8/f12c to d1/d11/d14/d4d/d6c/d7b/f13d 0 2024-02-15T18:27:10.184 INFO:tasks.workunit.client.0.smithi042.stdout:3/970: fsync d1/d11/d2e/fd7 0 2024-02-15T18:27:10.184 INFO:tasks.workunit.client.0.smithi042.stdout:3/971: chown d1/d9c/cba 3136 1 2024-02-15T18:27:10.184 INFO:tasks.workunit.client.1.smithi191.stdout:2/491: mknod df/d12/d32/d4d/d89/ca6 0 2024-02-15T18:27:10.185 INFO:tasks.workunit.client.1.smithi191.stdout:4/518: dwrite d2/fad [0,4194304] 0 2024-02-15T18:27:10.186 INFO:tasks.workunit.client.0.smithi042.stdout:8/871: link dd/de/d33/d3b/c3d dd/de/d3c/cfa 0 2024-02-15T18:27:10.186 INFO:tasks.workunit.client.0.smithi042.stdout:8/872: truncate dd/fca 55495 0 2024-02-15T18:27:10.186 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:27:10.187 INFO:tasks.workunit.client.0.smithi042.stdout:4/965: mknod da/d11/d85/db5/d103/d66/d78/c153 0 2024-02-15T18:27:10.188 INFO:tasks.workunit.client.1.smithi191.stdout:6/458: dwrite d8/d19/f35 [0,4194304] 0 2024-02-15T18:27:10.188 INFO:tasks.workunit.client.1.smithi191.stdout:2/492: getdents df/d12/d19/d1a/d35/d47 0 2024-02-15T18:27:10.188 INFO:tasks.workunit.client.1.smithi191.stdout:2/493: chown df/d12/d19/d1a/d35/d47/l96 1 1 2024-02-15T18:27:10.188 INFO:tasks.workunit.client.1.smithi191.stdout:2/494: chown df/d12/d1f/d42/f49 39531374 1 2024-02-15T18:27:10.190 INFO:tasks.workunit.client.0.smithi042.stdout:0/950: truncate d4/dc2/fd3 63808 0 2024-02-15T18:27:10.192 INFO:tasks.workunit.client.1.smithi191.stdout:6/459: dread d8/d19/f38 [0,4194304] 0 2024-02-15T18:27:10.192 INFO:tasks.workunit.client.1.smithi191.stdout:6/460: write d8/d19/f62 [934674,20021] 0 2024-02-15T18:27:10.192 INFO:tasks.workunit.client.0.smithi042.stdout:5/992: creat d0/dda/d42/d7d/db5/f15b x:0 0 0 2024-02-15T18:27:10.193 INFO:tasks.workunit.client.1.smithi191.stdout:7/516: dread d0/f99 [0,4194304] 0 2024-02-15T18:27:10.195 INFO:tasks.workunit.client.0.smithi042.stdout:9/984: rename d6/da2/cb8 to d6/da/d2f/d55/d73/d8d/c14f 0 2024-02-15T18:27:10.197 INFO:tasks.workunit.client.1.smithi191.stdout:4/519: creat d2/d5/d7/d2f/fc2 x:0 0 0 2024-02-15T18:27:10.201 INFO:tasks.workunit.client.0.smithi042.stdout:3/972: unlink d1/d11/d14/d4d/d73/cb6 0 2024-02-15T18:27:10.202 INFO:tasks.workunit.client.0.smithi042.stdout:7/982: sync 2024-02-15T18:27:10.206 INFO:tasks.workunit.client.0.smithi042.stdout:4/966: getdents da 0 2024-02-15T18:27:10.209 INFO:tasks.workunit.client.0.smithi042.stdout:4/967: dread da/d3a/dc9/fd7 [0,4194304] 0 2024-02-15T18:27:10.209 INFO:tasks.workunit.client.1.smithi191.stdout:6/461: dwrite d8/d19/f1f [0,4194304] 0 2024-02-15T18:27:10.209 INFO:tasks.workunit.client.1.smithi191.stdout:2/495: mknod df/d12/d19/d1a/d35/d47/da2/ca7 0 2024-02-15T18:27:10.209 INFO:tasks.workunit.client.1.smithi191.stdout:7/517: creat d0/db/d65/d11/fc4 x:0 0 0 2024-02-15T18:27:10.211 INFO:tasks.workunit.client.1.smithi191.stdout:4/520: creat d2/d5/d7/d90/d52/db5/fc3 x:0 0 0 2024-02-15T18:27:10.211 INFO:tasks.workunit.client.1.smithi191.stdout:4/521: dread - d2/d5/d7/d3e/d88/fbc zero size 2024-02-15T18:27:10.212 INFO:tasks.workunit.client.0.smithi042.stdout:0/951: creat d4/d9/d34/d57/dc1/f134 x:0 0 0 2024-02-15T18:27:10.212 INFO:tasks.workunit.client.0.smithi042.stdout:0/952: write d4/d38/d39/df4/d114/dfe/f64 [2228782,37599] 0 2024-02-15T18:27:10.212 INFO:tasks.workunit.client.0.smithi042.stdout:0/953: getdents d4/d38/d77/d2e/dae 0 2024-02-15T18:27:10.213 INFO:tasks.workunit.client.1.smithi191.stdout:6/462: symlink d8/d19/l88 0 2024-02-15T18:27:10.219 INFO:tasks.workunit.client.1.smithi191.stdout:4/522: write d2/d5/d14/d47/d78/f38 [3204846,101474] 0 2024-02-15T18:27:10.220 INFO:tasks.workunit.client.0.smithi042.stdout:5/993: mkdir d0/d38/d53/db4/d15c 0 2024-02-15T18:27:10.222 INFO:tasks.workunit.client.0.smithi042.stdout:9/985: truncate d6/da/d2f/d69/fd4 467279 0 2024-02-15T18:27:10.223 INFO:tasks.workunit.client.1.smithi191.stdout:6/463: symlink d8/l89 0 2024-02-15T18:27:10.226 INFO:tasks.workunit.client.1.smithi191.stdout:4/523: getdents d2 0 2024-02-15T18:27:10.227 INFO:tasks.workunit.client.0.smithi042.stdout:8/873: rename dd/d11/f39 to dd/de/da4/d67/d5a/ffb 0 2024-02-15T18:27:10.227 INFO:tasks.workunit.client.0.smithi042.stdout:7/983: creat d10/d15/d39/d6c/dba/d117/f144 x:0 0 0 2024-02-15T18:27:10.228 INFO:tasks.workunit.client.1.smithi191.stdout:6/464: mknod d8/d19/d6b/d76/c8a 0 2024-02-15T18:27:10.229 INFO:tasks.workunit.client.0.smithi042.stdout:3/973: dwrite d1/d11/d2e/de7/f121 [0,4194304] 0 2024-02-15T18:27:10.230 INFO:tasks.workunit.client.0.smithi042.stdout:4/968: getdents da/d11/d85/db5/d103/d66/d78/d53/d82/dc0 0 2024-02-15T18:27:10.233 INFO:tasks.workunit.client.1.smithi191.stdout:2/496: dwrite df/d12/d19/d1a/f21 [0,4194304] 0 2024-02-15T18:27:10.240 INFO:tasks.workunit.client.1.smithi191.stdout:2/497: rename df/d12/d19/d1a/d35/d47/f8e to df/d12/d19/d1a/d35/d47/fa8 0 2024-02-15T18:27:10.240 INFO:tasks.workunit.client.1.smithi191.stdout:4/524: symlink d2/d5/d7/d3e/d63/lc4 0 2024-02-15T18:27:10.243 INFO:tasks.workunit.client.0.smithi042.stdout:5/994: mknod d0/d38/d53/db4/d15c/c15d 0 2024-02-15T18:27:10.243 INFO:tasks.workunit.client.1.smithi191.stdout:2/498: read df/d12/d19/d1a/f29 [1104806,27995] 0 2024-02-15T18:27:10.243 INFO:tasks.workunit.client.1.smithi191.stdout:2/499: truncate df/d12/d19/d1a/f5e 1262204 0 2024-02-15T18:27:10.244 INFO:tasks.workunit.client.0.smithi042.stdout:0/954: dwrite d4/dc2/f12d [0,4194304] 0 2024-02-15T18:27:10.244 INFO:tasks.workunit.client.0.smithi042.stdout:0/955: fsync d4/d38/d39/d8e/fbc 0 2024-02-15T18:27:10.244 INFO:tasks.workunit.client.0.smithi042.stdout:9/986: symlink d6/da/d2f/d55/d73/d8d/dcf/de1/df7/l150 0 2024-02-15T18:27:10.245 INFO:tasks.workunit.client.0.smithi042.stdout:9/987: write d6/da/d2f/d69/f5b [373926,48243] 0 2024-02-15T18:27:10.245 INFO:tasks.workunit.client.0.smithi042.stdout:9/988: chown d6/dd/d9/fc3 28199 1 2024-02-15T18:27:10.256 INFO:tasks.workunit.client.0.smithi042.stdout:5/995: write d0/d14/fa3 [5198909,34496] 0 2024-02-15T18:27:10.256 INFO:tasks.workunit.client.0.smithi042.stdout:5/996: write d0/dda/dc7/d52/dad/f132 [402432,12401] 0 2024-02-15T18:27:10.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.301+0000 7f3a73def700 1 -- 172.21.15.42:0/1027572160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a6c0f6860 msgr2=0x7f3a6c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:10.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.301+0000 7f3a73def700 1 --2- 172.21.15.42:0/1027572160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a6c0f6860 0x7f3a6c0f6cc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f3a5c004660 tx=0x7f3a5c009b30).stop 2024-02-15T18:27:10.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.301+0000 7f3a73def700 1 -- 172.21.15.42:0/1027572160 shutdown_connections 2024-02-15T18:27:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.301+0000 7f3a73def700 1 --2- 172.21.15.42:0/1027572160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a6c0f6860 0x7f3a6c0f6cc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.301+0000 7f3a73def700 1 --2- 172.21.15.42:0/1027572160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a6c0f55c0 0x7f3a6c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.301+0000 7f3a73def700 1 -- 172.21.15.42:0/1027572160 >> 172.21.15.42:0/1027572160 conn(0x7f3a6c0f0d40 msgr2=0x7f3a6c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.301+0000 7f3a73def700 1 -- 172.21.15.42:0/1027572160 shutdown_connections 2024-02-15T18:27:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.301+0000 7f3a73def700 1 -- 172.21.15.42:0/1027572160 wait complete. 2024-02-15T18:27:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.302+0000 7f3a73def700 1 Processor -- start 2024-02-15T18:27:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.302+0000 7f3a73def700 1 -- start start 2024-02-15T18:27:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.302+0000 7f3a73def700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a6c0f55c0 0x7f3a6c109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.302+0000 7f3a73def700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a6c0f6860 0x7f3a6c10c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.302+0000 7f3a73def700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c060420 con 0x7f3a6c0f55c0 2024-02-15T18:27:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.302+0000 7f3a73def700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c060560 con 0x7f3a6c0f6860 2024-02-15T18:27:10.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.302+0000 7f3a71b8b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a6c0f55c0 0x7f3a6c109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.302+0000 7f3a71b8b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a6c0f55c0 0x7f3a6c109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34200/0 (socket says 172.21.15.42:34200) 2024-02-15T18:27:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.302+0000 7f3a71b8b700 1 -- 172.21.15.42:0/552804446 learned_addr learned my addr 172.21.15.42:0/552804446 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:27:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.302+0000 7f3a7138a700 1 --2- 172.21.15.42:0/552804446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a6c0f6860 0x7f3a6c10c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.302+0000 7f3a71b8b700 1 -- 172.21.15.42:0/552804446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a6c0f6860 msgr2=0x7f3a6c10c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.302+0000 7f3a71b8b700 1 --2- 172.21.15.42:0/552804446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a6c0f6860 0x7f3a6c10c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.302+0000 7f3a71b8b700 1 -- 172.21.15.42:0/552804446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c005040 con 0x7f3a6c0f55c0 2024-02-15T18:27:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.302+0000 7f3a71b8b700 1 --2- 172.21.15.42:0/552804446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a6c0f55c0 0x7f3a6c109c70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f3a68002700 tx=0x7f3a68009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.303+0000 7f3a627fc700 1 -- 172.21.15.42:0/552804446 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a680079c0 con 0x7f3a6c0f55c0 2024-02-15T18:27:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.303+0000 7f3a627fc700 1 -- 172.21.15.42:0/552804446 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f3a6800f040 con 0x7f3a6c0f55c0 2024-02-15T18:27:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.303+0000 7f3a73def700 1 -- 172.21.15.42:0/552804446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c10c780 con 0x7f3a6c0f55c0 2024-02-15T18:27:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.303+0000 7f3a627fc700 1 -- 172.21.15.42:0/552804446 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a68014940 con 0x7f3a6c0f55c0 2024-02-15T18:27:10.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.303+0000 7f3a73def700 1 -- 172.21.15.42:0/552804446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c10cd10 con 0x7f3a6c0f55c0 2024-02-15T18:27:10.302 INFO:tasks.workunit.client.1.smithi191.stdout:4/525: creat d2/d5/d14/d47/fc5 x:0 0 0 2024-02-15T18:27:10.303 INFO:tasks.workunit.client.1.smithi191.stdout:7/518: dwrite d0/db/f8d [0,4194304] 0 2024-02-15T18:27:10.303 INFO:tasks.workunit.client.1.smithi191.stdout:6/465: dwrite d8/d19/d6b/f75 [0,4194304] 0 2024-02-15T18:27:10.303 INFO:tasks.workunit.client.1.smithi191.stdout:2/500: creat df/d12/d32/fa9 x:0 0 0 2024-02-15T18:27:10.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.304+0000 7f3a73def700 1 -- 172.21.15.42:0/552804446 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6c0f49b0 con 0x7f3a6c0f55c0 2024-02-15T18:27:10.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.304+0000 7f3a627fc700 1 -- 172.21.15.42:0/552804446 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7f3a6801b060 con 0x7f3a6c0f55c0 2024-02-15T18:27:10.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.304+0000 7f3a627fc700 1 --2- 172.21.15.42:0/552804446 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3a58060fb0 0x7f3a58063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:10.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.304+0000 7f3a627fc700 1 -- 172.21.15.42:0/552804446 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f3a6807bf40 con 0x7f3a6c0f55c0 2024-02-15T18:27:10.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.304+0000 7f3a7138a700 1 --2- 172.21.15.42:0/552804446 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3a58060fb0 0x7f3a58063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:10.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.305+0000 7f3a7138a700 1 --2- 172.21.15.42:0/552804446 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3a58060fb0 0x7f3a58063470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f3a5c004630 tx=0x7f3a5c009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:10.305 INFO:tasks.workunit.client.0.smithi042.stdout:8/874: dwrite dd/de/da4/d67/f43 [4194304,4194304] 0 2024-02-15T18:27:10.305 INFO:tasks.workunit.client.0.smithi042.stdout:8/875: stat dd/de/d33/l55 0 2024-02-15T18:27:10.305 INFO:tasks.workunit.client.0.smithi042.stdout:3/974: dwrite d1/d11/d2e/fd6 [0,4194304] 0 2024-02-15T18:27:10.305 INFO:tasks.workunit.client.0.smithi042.stdout:7/984: getdents d10/d15/d8f/da5/dad/dda 0 2024-02-15T18:27:10.305 INFO:tasks.workunit.client.0.smithi042.stdout:7/985: chown d10/d43/d57/fb8 4 1 2024-02-15T18:27:10.305 INFO:tasks.workunit.client.0.smithi042.stdout:7/986: readlink d10/d15/d39/de3/d116/d11e/l130 0 2024-02-15T18:27:10.305 INFO:tasks.workunit.client.0.smithi042.stdout:7/987: chown d10/d49/f8e 44876058 1 2024-02-15T18:27:10.305 INFO:tasks.workunit.client.0.smithi042.stdout:7/988: chown d10/d15/d8f/da5/dad/fd0 217 1 2024-02-15T18:27:10.305 INFO:tasks.workunit.client.0.smithi042.stdout:7/989: fsync d10/d15/d110/f12b 0 2024-02-15T18:27:10.306 INFO:tasks.workunit.client.1.smithi191.stdout:4/526: mknod d2/d5/d14/d47/cc6 0 2024-02-15T18:27:10.306 INFO:tasks.workunit.client.1.smithi191.stdout:4/527: write d2/d5/d7/d90/d52/f54 [781920,122309] 0 2024-02-15T18:27:10.306 INFO:tasks.workunit.client.1.smithi191.stdout:4/528: write d2/d5/d7/d90/d52/db5/fc3 [993934,15820] 0 2024-02-15T18:27:10.306 INFO:tasks.workunit.client.1.smithi191.stdout:7/519: creat d0/fc5 x:0 0 0 2024-02-15T18:27:10.307 INFO:tasks.workunit.client.0.smithi042.stdout:7/990: write d10/d15/f3d [3249722,72528] 0 2024-02-15T18:27:10.307 INFO:tasks.workunit.client.0.smithi042.stdout:7/991: write d10/d43/d57/fc1 [990910,1832] 0 2024-02-15T18:27:10.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.310+0000 7f3a627fc700 1 -- 172.21.15.42:0/552804446 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3a6804c3b0 con 0x7f3a6c0f55c0 2024-02-15T18:27:10.307 INFO:tasks.workunit.client.0.smithi042.stdout:4/969: creat da/d11/d85/db5/d103/d66/d78/d53/d82/f154 x:0 0 0 2024-02-15T18:27:10.311 INFO:tasks.workunit.client.1.smithi191.stdout:6/466: symlink d8/d19/d6b/d80/l8b 0 2024-02-15T18:27:10.312 INFO:tasks.workunit.client.1.smithi191.stdout:2/501: rmdir df/d12/d19/d1a/d35/d5b 39 2024-02-15T18:27:10.314 INFO:tasks.workunit.client.1.smithi191.stdout:6/467: write d8/d19/f40 [3165958,95747] 0 2024-02-15T18:27:10.314 INFO:tasks.workunit.client.1.smithi191.stdout:4/529: creat d2/d5/d7/d2f/fc7 x:0 0 0 2024-02-15T18:27:10.315 INFO:tasks.workunit.client.0.smithi042.stdout:7/992: dread d10/d15/d8f/da5/dad/d104/d65/ff7 [0,4194304] 0 2024-02-15T18:27:10.317 INFO:tasks.workunit.client.0.smithi042.stdout:9/989: rename d6/dd/f129 to d6/dd/d9/d59/f151 0 2024-02-15T18:27:10.320 INFO:tasks.workunit.client.1.smithi191.stdout:2/502: symlink df/d12/d1f/d9c/laa 0 2024-02-15T18:27:10.326 INFO:tasks.workunit.client.0.smithi042.stdout:2/923: sync 2024-02-15T18:27:10.329 INFO:tasks.workunit.client.0.smithi042.stdout:8/876: dwrite dd/de/fc4 [0,4194304] 0 2024-02-15T18:27:10.330 INFO:tasks.workunit.client.0.smithi042.stdout:8/877: chown dd/de/da4/d67/l38 14102 1 2024-02-15T18:27:10.330 INFO:tasks.workunit.client.0.smithi042.stdout:8/878: write dd/de/d33/f42 [127032,1945] 0 2024-02-15T18:27:10.330 INFO:tasks.workunit.client.0.smithi042.stdout:8/879: write dd/de/da4/d67/f9f [671770,74439] 0 2024-02-15T18:27:10.331 INFO:tasks.workunit.client.0.smithi042.stdout:8/880: stat dd/de/da4/d81/f88 0 2024-02-15T18:27:10.331 INFO:tasks.workunit.client.0.smithi042.stdout:8/881: write dd/de/da4/d67/d6f/d78/d9a/dce/fd9 [1709301,77934] 0 2024-02-15T18:27:10.331 INFO:tasks.workunit.client.0.smithi042.stdout:5/997: dwrite d0/d14/fa3 [8388608,4194304] 0 2024-02-15T18:27:10.331 INFO:tasks.workunit.client.1.smithi191.stdout:6/468: mkdir d8/d9/d4a/d4c/d7f/d8c 0 2024-02-15T18:27:10.332 INFO:tasks.workunit.client.1.smithi191.stdout:4/530: creat d2/d5/d7/d3e/fc8 x:0 0 0 2024-02-15T18:27:10.333 INFO:tasks.workunit.client.1.smithi191.stdout:6/469: write d8/d9/d2b/f2e [4019827,5442] 0 2024-02-15T18:27:10.336 INFO:tasks.workunit.client.0.smithi042.stdout:3/975: symlink d1/l13e 0 2024-02-15T18:27:10.336 INFO:tasks.workunit.client.0.smithi042.stdout:3/976: write d1/d11/d14/f47 [3568827,107978] 0 2024-02-15T18:27:10.337 INFO:tasks.workunit.client.0.smithi042.stdout:4/970: symlink da/d11/d67/l155 0 2024-02-15T18:27:10.338 INFO:tasks.workunit.client.0.smithi042.stdout:7/993: rmdir d10/d15/d39/d6c 39 2024-02-15T18:27:10.338 INFO:tasks.workunit.client.0.smithi042.stdout:7/994: chown d10/d49/dd2 2 1 2024-02-15T18:27:10.338 INFO:tasks.workunit.client.0.smithi042.stdout:7/995: write d10/d15/fa6 [315964,129422] 0 2024-02-15T18:27:10.338 INFO:tasks.workunit.client.0.smithi042.stdout:7/996: stat d10/d49/d6d/d6e/lca 0 2024-02-15T18:27:10.340 INFO:tasks.workunit.client.0.smithi042.stdout:9/990: creat d6/dd/d9/d59/d5c/f152 x:0 0 0 2024-02-15T18:27:10.341 INFO:tasks.workunit.client.1.smithi191.stdout:4/531: mkdir d2/d5/d7/d90/d52/db5/dc9 0 2024-02-15T18:27:10.341 INFO:tasks.workunit.client.1.smithi191.stdout:4/532: chown d2/d5/d7/d2f/d41/f73 890741471 1 2024-02-15T18:27:10.341 INFO:tasks.workunit.client.1.smithi191.stdout:4/533: write d2/d5/f13 [3695988,30151] 0 2024-02-15T18:27:10.342 INFO:tasks.workunit.client.1.smithi191.stdout:6/470: rename d8/d9/d2b/c6f to d8/d19/d6b/c8d 0 2024-02-15T18:27:10.342 INFO:tasks.workunit.client.1.smithi191.stdout:6/471: fsync d8/d9/d2b/d5b/f5e 0 2024-02-15T18:27:10.342 INFO:tasks.workunit.client.1.smithi191.stdout:6/472: stat d8/d9/d2b/d5b/f82 0 2024-02-15T18:27:10.345 INFO:tasks.workunit.client.0.smithi042.stdout:2/924: link d0/d8/d20/d7d/lde d0/d8/d10/d31/d3a/l123 0 2024-02-15T18:27:10.347 INFO:tasks.workunit.client.1.smithi191.stdout:4/534: write d2/d5/d7/d2f/f6b [441990,65820] 0 2024-02-15T18:27:10.347 INFO:tasks.workunit.client.1.smithi191.stdout:2/503: dwrite df/d12/d1f/f34 [0,4194304] 0 2024-02-15T18:27:10.347 INFO:tasks.workunit.client.0.smithi042.stdout:2/925: write d0/d8/d10/d31/d3a/d53/f66 [4160798,24360] 0 2024-02-15T18:27:10.348 INFO:tasks.workunit.client.1.smithi191.stdout:4/535: dread d2/d5/d7/d2f/d41/f92 [0,4194304] 0 2024-02-15T18:27:10.349 INFO:tasks.workunit.client.1.smithi191.stdout:6/473: mkdir d8/d9/d2b/d57/d8e 0 2024-02-15T18:27:10.349 INFO:tasks.workunit.client.1.smithi191.stdout:6/474: stat d8/d19/c55 0 2024-02-15T18:27:10.349 INFO:tasks.workunit.client.1.smithi191.stdout:6/475: fdatasync d8/d9/d2b/f2e 0 2024-02-15T18:27:10.349 INFO:tasks.workunit.client.1.smithi191.stdout:6/476: chown d8/d9/c3b 0 1 2024-02-15T18:27:10.351 INFO:tasks.workunit.client.1.smithi191.stdout:2/504: mkdir df/d61/d9f/da3/dab 0 2024-02-15T18:27:10.351 INFO:tasks.workunit.client.0.smithi042.stdout:2/926: dread d0/d8/d10/d5a/f88 [0,4194304] 0 2024-02-15T18:27:10.351 INFO:tasks.workunit.client.0.smithi042.stdout:8/882: creat dd/de/da4/d67/d6f/d78/d9a/dce/ffc x:0 0 0 2024-02-15T18:27:10.352 INFO:tasks.workunit.client.1.smithi191.stdout:4/536: getdents d2/d5/d7/d2f 0 2024-02-15T18:27:10.360 INFO:tasks.workunit.client.1.smithi191.stdout:6/477: symlink d8/d9/d2b/d81/l8f 0 2024-02-15T18:27:10.360 INFO:tasks.workunit.client.1.smithi191.stdout:6/478: write d8/f33 [443583,84882] 0 2024-02-15T18:27:10.362 INFO:tasks.workunit.client.0.smithi042.stdout:5/998: link d0/dda/lb7 d0/d12/d2a/d102/l15e 0 2024-02-15T18:27:10.365 INFO:tasks.workunit.client.0.smithi042.stdout:0/956: sync 2024-02-15T18:27:10.365 INFO:tasks.workunit.client.0.smithi042.stdout:3/977: symlink d1/d11/d10c/l13f 0 2024-02-15T18:27:10.367 INFO:tasks.workunit.client.0.smithi042.stdout:4/971: mknod da/d2b/d7e/d138/c156 0 2024-02-15T18:27:10.370 INFO:tasks.workunit.client.0.smithi042.stdout:8/883: mkdir dd/de/dc0/dfd 0 2024-02-15T18:27:10.371 INFO:tasks.workunit.client.0.smithi042.stdout:8/884: dread dd/de/da4/d67/f2d [0,4194304] 0 2024-02-15T18:27:10.371 INFO:tasks.workunit.client.0.smithi042.stdout:8/885: rename dd/de to dd/de/da4/d67/d6f/d78/d9a/dfe 22 2024-02-15T18:27:10.372 INFO:tasks.workunit.client.0.smithi042.stdout:2/927: creat d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d11e/f124 x:0 0 0 2024-02-15T18:27:10.372 INFO:tasks.workunit.client.0.smithi042.stdout:2/928: fsync d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/d44/f103 0 2024-02-15T18:27:10.372 INFO:tasks.workunit.client.0.smithi042.stdout:2/929: write d0/d8/d20/d7d/d7e/dc7/f7f [4753073,130915] 0 2024-02-15T18:27:10.372 INFO:tasks.workunit.client.0.smithi042.stdout:5/999: creat d0/d39/ddd/de0/f15f x:0 0 0 2024-02-15T18:27:10.373 INFO:tasks.workunit.client.0.smithi042.stdout:0/957: creat d4/d38/d77/d2e/f135 x:0 0 0 2024-02-15T18:27:10.373 INFO:tasks.workunit.client.0.smithi042.stdout:0/958: write d4/d9/d62/d74/fc7 [840897,5461] 0 2024-02-15T18:27:10.374 INFO:tasks.workunit.client.0.smithi042.stdout:3/978: symlink d1/de/d1a/l140 0 2024-02-15T18:27:10.376 INFO:tasks.workunit.client.0.smithi042.stdout:4/972: creat da/d11/f157 x:0 0 0 2024-02-15T18:27:10.377 INFO:tasks.workunit.client.1.smithi191.stdout:4/537: rename d2/d5/d7/d2f/f51 to d2/d5/d7/d3e/d63/d8c/da4/fca 0 2024-02-15T18:27:10.379 INFO:tasks.workunit.client.0.smithi042.stdout:8/886: dread dd/de/da5/dc6/ff6 [0,4194304] 0 2024-02-15T18:27:10.379 INFO:tasks.workunit.client.0.smithi042.stdout:2/930: creat d0/d8/d20/d7d/d7e/dc7/d8b/dd7/f125 x:0 0 0 2024-02-15T18:27:10.379 INFO:tasks.workunit.client.0.smithi042.stdout:2/931: chown d0/d35/f95 65 1 2024-02-15T18:27:10.379 INFO:tasks.workunit.client.0.smithi042.stdout:8/887: dread dd/de/da4/d81/dae/fb6 [4194304,4194304] 0 2024-02-15T18:27:10.380 INFO:tasks.workunit.client.0.smithi042.stdout:2/932: write d0/d35/f95 [341230,23254] 0 2024-02-15T18:27:10.380 INFO:tasks.workunit.client.0.smithi042.stdout:2/933: write d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d11e/f124 [631025,112589] 0 2024-02-15T18:27:10.380 INFO:tasks.workunit.client.0.smithi042.stdout:2/934: readlink d0/d8/d20/l97 0 2024-02-15T18:27:10.381 INFO:tasks.workunit.client.1.smithi191.stdout:6/479: dwrite d8/d9/d2b/f4b [0,4194304] 0 2024-02-15T18:27:10.381 INFO:tasks.workunit.client.1.smithi191.stdout:6/480: read d8/f15 [2289672,33837] 0 2024-02-15T18:27:10.381 INFO:tasks.workunit.client.0.smithi042.stdout:0/959: mknod d4/d38/d118/c136 0 2024-02-15T18:27:10.384 INFO:tasks.workunit.client.1.smithi191.stdout:4/538: write d2/d5/d7/d2f/d41/f8f [41229,9530] 0 2024-02-15T18:27:10.384 INFO:tasks.workunit.client.0.smithi042.stdout:3/979: symlink d1/d11/d14/d4d/d6c/df4/l141 0 2024-02-15T18:27:10.386 INFO:tasks.workunit.client.1.smithi191.stdout:6/481: mkdir d8/d9/d4a/d4c/d7f/d8c/d90 0 2024-02-15T18:27:10.388 INFO:tasks.workunit.client.0.smithi042.stdout:4/973: rename da/d3b to da/d3a/dbb/d158 0 2024-02-15T18:27:10.389 INFO:tasks.workunit.client.1.smithi191.stdout:6/482: fdatasync d8/f30 0 2024-02-15T18:27:10.389 INFO:tasks.workunit.client.1.smithi191.stdout:6/483: write d8/d19/f1d [3682347,108985] 0 2024-02-15T18:27:10.389 INFO:tasks.workunit.client.1.smithi191.stdout:6/484: dread - d8/d9/d4a/d4c/f64 zero size 2024-02-15T18:27:10.390 INFO:tasks.workunit.client.1.smithi191.stdout:6/485: fsync d8/f15 0 2024-02-15T18:27:10.390 INFO:tasks.workunit.client.1.smithi191.stdout:6/486: read d8/d9/d4a/d4c/f5d [272524,16019] 0 2024-02-15T18:27:10.390 INFO:tasks.workunit.client.1.smithi191.stdout:4/539: mkdir d2/d5/d7/d90/d52/dcb 0 2024-02-15T18:27:10.392 INFO:tasks.workunit.client.0.smithi042.stdout:8/888: symlink dd/de/d94/db1/lff 0 2024-02-15T18:27:10.395 INFO:tasks.workunit.client.0.smithi042.stdout:2/935: symlink d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d11b/l126 0 2024-02-15T18:27:10.397 INFO:tasks.workunit.client.0.smithi042.stdout:3/980: mkdir d1/d11/d14/dbf/d142 0 2024-02-15T18:27:10.399 INFO:tasks.workunit.client.0.smithi042.stdout:2/936: link d0/d8/d20/d7d/d7e/dc7/l5b d0/d8/d20/d7d/d7e/l127 0 2024-02-15T18:27:10.399 INFO:tasks.workunit.client.0.smithi042.stdout:2/937: write d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/d44/f103 [460264,25113] 0 2024-02-15T18:27:10.399 INFO:tasks.workunit.client.0.smithi042.stdout:2/938: dread - d0/d8/d20/d41/dbe/f120 zero size 2024-02-15T18:27:10.400 INFO:tasks.workunit.client.0.smithi042.stdout:3/981: rename d1/d11/d14/d4d/cde to d1/de/d19/c143 0 2024-02-15T18:27:10.401 INFO:tasks.workunit.client.0.smithi042.stdout:2/939: creat d0/d8/d10/d5a/f128 x:0 0 0 2024-02-15T18:27:10.403 INFO:tasks.workunit.client.0.smithi042.stdout:2/940: creat d0/d8/d20/d3f/d7c/d102/d121/f129 x:0 0 0 2024-02-15T18:27:10.407 INFO:tasks.workunit.client.0.smithi042.stdout:9/991: dwrite d6/da/d2f/d55/f6d [4194304,4194304] 0 2024-02-15T18:27:10.407 INFO:tasks.workunit.client.0.smithi042.stdout:9/992: readlink d6/da/d2f/d6c/l6f 0 2024-02-15T18:27:10.407 INFO:tasks.workunit.client.1.smithi191.stdout:6/487: dread d8/d19/d6b/f75 [0,4194304] 0 2024-02-15T18:27:10.408 INFO:tasks.workunit.client.0.smithi042.stdout:0/960: read d4/d9/f22 [1701734,89954] 0 2024-02-15T18:27:10.413 INFO:tasks.workunit.client.0.smithi042.stdout:2/941: read d0/d8/f70 [628509,30623] 0 2024-02-15T18:27:10.413 INFO:tasks.workunit.client.0.smithi042.stdout:2/942: chown d0/d35/d9e/dd2 796629 1 2024-02-15T18:27:10.413 INFO:tasks.workunit.client.0.smithi042.stdout:8/889: dread dd/de/d12/ff2 [0,4194304] 0 2024-02-15T18:27:10.413 INFO:tasks.workunit.client.0.smithi042.stdout:9/993: dread d6/dd/d9/d59/d5c/fa6 [0,4194304] 0 2024-02-15T18:27:10.415 INFO:tasks.workunit.client.1.smithi191.stdout:4/540: dwrite d2/d5/d7/d90/d52/f7b [0,4194304] 0 2024-02-15T18:27:10.416 INFO:tasks.workunit.client.0.smithi042.stdout:0/961: dread d4/d38/d77/d2e/dda/d101/f105 [0,4194304] 0 2024-02-15T18:27:10.417 INFO:tasks.workunit.client.0.smithi042.stdout:2/943: creat d0/d40/d109/f12a x:0 0 0 2024-02-15T18:27:10.417 INFO:tasks.workunit.client.0.smithi042.stdout:2/944: chown d0/d8/d10/d7b/fa4 3762 1 2024-02-15T18:27:10.417 INFO:tasks.workunit.client.0.smithi042.stdout:2/945: chown d0/d8/d20/d7d/d7e/f107 242 1 2024-02-15T18:27:10.417 INFO:tasks.workunit.client.0.smithi042.stdout:8/890: creat dd/de/da5/f100 x:0 0 0 2024-02-15T18:27:10.418 INFO:tasks.workunit.client.0.smithi042.stdout:9/994: write d6/dd/f2b [1362442,102933] 0 2024-02-15T18:27:10.420 INFO:tasks.workunit.client.0.smithi042.stdout:2/946: rename d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/f2b to d0/d40/d109/f12b 0 2024-02-15T18:27:10.420 INFO:tasks.workunit.client.0.smithi042.stdout:2/947: chown d0/c91 913789796 1 2024-02-15T18:27:10.420 INFO:tasks.workunit.client.0.smithi042.stdout:4/974: dwrite da/d11/d85/db5/f152 [0,4194304] 0 2024-02-15T18:27:10.421 INFO:tasks.workunit.client.0.smithi042.stdout:4/975: chown da/d11/d85/db5/d103/f131 991005141 1 2024-02-15T18:27:10.421 INFO:tasks.workunit.client.0.smithi042.stdout:8/891: chown dd/d11/l6b 54 1 2024-02-15T18:27:10.422 INFO:tasks.workunit.client.1.smithi191.stdout:4/541: creat d2/d5/d7/fcc x:0 0 0 2024-02-15T18:27:10.425 INFO:tasks.workunit.client.0.smithi042.stdout:0/962: dread d4/d38/d39/f108 [0,4194304] 0 2024-02-15T18:27:10.430 INFO:tasks.workunit.client.0.smithi042.stdout:9/995: creat d6/da/d2f/d54/f153 x:0 0 0 2024-02-15T18:27:10.431 INFO:tasks.workunit.client.1.smithi191.stdout:4/542: truncate d2/d5/f4d 2353506 0 2024-02-15T18:27:10.431 INFO:tasks.workunit.client.0.smithi042.stdout:2/948: creat d0/d8/d10/d31/d79/f12c x:0 0 0 2024-02-15T18:27:10.431 INFO:tasks.workunit.client.0.smithi042.stdout:2/949: readlink d0/d8/d10/d31/d3a/l6c 0 2024-02-15T18:27:10.433 INFO:tasks.workunit.client.1.smithi191.stdout:4/543: mkdir d2/d5/d14/d71/dcd 0 2024-02-15T18:27:10.435 INFO:tasks.workunit.client.1.smithi191.stdout:4/544: write d2/d5/d14/f58 [1828233,125881] 0 2024-02-15T18:27:10.436 INFO:tasks.workunit.client.0.smithi042.stdout:8/892: rename dd/de/fda to dd/f101 0 2024-02-15T18:27:10.436 INFO:tasks.workunit.client.0.smithi042.stdout:8/893: chown dd/de/da4/d67/d5a 1 1 2024-02-15T18:27:10.436 INFO:tasks.workunit.client.1.smithi191.stdout:6/488: dwrite d8/d9/d2b/d57/f6c [0,4194304] 0 2024-02-15T18:27:10.438 INFO:tasks.workunit.client.0.smithi042.stdout:3/982: dwrite d1/d11/d14/d4d/d73/f107 [0,4194304] 0 2024-02-15T18:27:10.438 INFO:tasks.workunit.client.0.smithi042.stdout:3/983: stat d1/de/d19/f4b 0 2024-02-15T18:27:10.440 INFO:tasks.workunit.client.0.smithi042.stdout:0/963: truncate d4/d38/d39/df4/d114/d6c/df9/d11f/d121/dfc/f21 2466885 0 2024-02-15T18:27:10.440 INFO:tasks.workunit.client.0.smithi042.stdout:0/964: write d4/d38/d77/d2e/f135 [539526,68282] 0 2024-02-15T18:27:10.442 INFO:tasks.workunit.client.1.smithi191.stdout:4/545: rmdir d2/d5/d14/d44 39 2024-02-15T18:27:10.442 INFO:tasks.workunit.client.1.smithi191.stdout:4/546: dread - d2/d5/d14/fb8 zero size 2024-02-15T18:27:10.443 INFO:tasks.workunit.client.0.smithi042.stdout:9/996: creat d6/da/d2f/d6c/f154 x:0 0 0 2024-02-15T18:27:10.443 INFO:tasks.workunit.client.0.smithi042.stdout:3/984: write d1/de/d19/f1f [7230340,27728] 0 2024-02-15T18:27:10.443 INFO:tasks.workunit.client.1.smithi191.stdout:6/489: mkdir d8/d9/d2b/d57/d8e/d91 0 2024-02-15T18:27:10.447 INFO:tasks.workunit.client.0.smithi042.stdout:2/950: dread d0/d8/d20/d7d/d7e/dc7/d8b/dd7/fe2 [0,4194304] 0 2024-02-15T18:27:10.448 INFO:tasks.workunit.client.1.smithi191.stdout:4/547: link d2/d5/d86/f96 d2/d5/d14/d47/d48/fce 0 2024-02-15T18:27:10.450 INFO:tasks.workunit.client.1.smithi191.stdout:6/490: creat d8/d9/d2b/d57/d8e/f92 x:0 0 0 2024-02-15T18:27:10.450 INFO:tasks.workunit.client.1.smithi191.stdout:6/491: write d8/d19/f7a [829099,82826] 0 2024-02-15T18:27:10.452 INFO:tasks.workunit.client.0.smithi042.stdout:8/894: dwrite dd/de/da4/d67/d5a/f93 [0,4194304] 0 2024-02-15T18:27:10.453 INFO:tasks.workunit.client.1.smithi191.stdout:6/492: write d8/f33 [881282,104425] 0 2024-02-15T18:27:10.454 INFO:tasks.workunit.client.1.smithi191.stdout:6/493: readlink d8/d19/d6b/d72/l74 0 2024-02-15T18:27:10.458 INFO:tasks.workunit.client.1.smithi191.stdout:4/548: mknod d2/d5/d7/d90/d52/dcb/ccf 0 2024-02-15T18:27:10.460 INFO:tasks.workunit.client.1.smithi191.stdout:4/549: chown d2/d5/d7/d2f/f31 1010 1 2024-02-15T18:27:10.461 INFO:tasks.workunit.client.0.smithi042.stdout:0/965: mkdir d4/d38/d118/d137 0 2024-02-15T18:27:10.461 INFO:tasks.workunit.client.0.smithi042.stdout:9/997: truncate d6/da/d2f/d69/f8b 4858306 0 2024-02-15T18:27:10.462 INFO:tasks.workunit.client.0.smithi042.stdout:9/998: readlink d6/dd/l1b 0 2024-02-15T18:27:10.463 INFO:tasks.workunit.client.0.smithi042.stdout:3/985: rmdir d1/d11/d14/d4d/d73/d89/da8 39 2024-02-15T18:27:10.466 INFO:tasks.workunit.client.0.smithi042.stdout:3/986: dread d1/d11/d14/df6/ff7 [0,4194304] 0 2024-02-15T18:27:10.471 INFO:tasks.workunit.client.0.smithi042.stdout:7/997: sync 2024-02-15T18:27:10.471 INFO:tasks.workunit.client.0.smithi042.stdout:2/951: rename d0/d8/d20/c32 to d0/d8/d20/d7d/d7e/dc7/d8b/dd7/c12d 0 2024-02-15T18:27:10.471 INFO:tasks.workunit.client.0.smithi042.stdout:7/998: write d10/d15/d8f/da5/dad/d104/d23/d51/dc9/ffe [6611,59444] 0 2024-02-15T18:27:10.472 INFO:tasks.workunit.client.0.smithi042.stdout:8/895: creat dd/de/d3c/f102 x:0 0 0 2024-02-15T18:27:10.474 INFO:tasks.workunit.client.0.smithi042.stdout:0/966: link d4/c23 d4/d38/d118/d137/c138 0 2024-02-15T18:27:10.474 INFO:tasks.workunit.client.0.smithi042.stdout:0/967: chown d4/d38/d39/df4/d114/lea 69 1 2024-02-15T18:27:10.475 INFO:tasks.workunit.client.0.smithi042.stdout:9/999: mknod d6/dd/d9/d59/d5c/c155 0 2024-02-15T18:27:10.477 INFO:tasks.workunit.client.0.smithi042.stdout:2/952: rename d0/d8/d10/d31/d3a/l123 to d0/d8/d20/l12e 0 2024-02-15T18:27:10.477 INFO:tasks.workunit.client.0.smithi042.stdout:2/953: fsync d0/d8/d10/d31/d3a/d53/fcc 0 2024-02-15T18:27:10.477 INFO:tasks.workunit.client.0.smithi042.stdout:7/999: getdents d10/d15/d8f/da5/dad/d104/d23/d51 0 2024-02-15T18:27:10.478 INFO:tasks.workunit.client.0.smithi042.stdout:2/954: mkdir d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda/de/d1a/d11e/d12f 0 2024-02-15T18:27:10.484 INFO:tasks.workunit.client.1.smithi191.stdout:4/550: dwrite d2/d5/d14/d47/d78/f61 [0,4194304] 0 2024-02-15T18:27:10.484 INFO:tasks.workunit.client.1.smithi191.stdout:4/551: chown d2/d5/d7/d90/da5 225061 1 2024-02-15T18:27:10.486 INFO:tasks.workunit.client.1.smithi191.stdout:6/494: dwrite d8/d9/d4a/d4c/f66 [0,4194304] 0 2024-02-15T18:27:10.486 INFO:tasks.workunit.client.1.smithi191.stdout:6/495: fdatasync d8/d9/f79 0 2024-02-15T18:27:10.486 INFO:tasks.workunit.client.1.smithi191.stdout:6/496: dread - d8/d9/d4a/d4c/d69/f7c zero size 2024-02-15T18:27:10.486 INFO:tasks.workunit.client.1.smithi191.stdout:6/497: readlink d8/d19/l1b 0 2024-02-15T18:27:10.487 INFO:tasks.workunit.client.1.smithi191.stdout:4/552: mkdir d2/d5/d14/d47/d78/dd0 0 2024-02-15T18:27:10.487 INFO:tasks.workunit.client.1.smithi191.stdout:4/553: dread - d2/d5/d14/d71/fac zero size 2024-02-15T18:27:10.490 INFO:tasks.workunit.client.1.smithi191.stdout:6/498: creat d8/d9/d2b/d57/f93 x:0 0 0 2024-02-15T18:27:10.490 INFO:tasks.workunit.client.1.smithi191.stdout:4/554: creat d2/d5/d14/d47/d78/dd0/fd1 x:0 0 0 2024-02-15T18:27:10.493 INFO:tasks.workunit.client.0.smithi042.stdout:3/987: dwrite d1/de/d19/f48 [0,4194304] 0 2024-02-15T18:27:10.493 INFO:tasks.workunit.client.1.smithi191.stdout:6/499: dread d8/d9/d4a/d4c/f5d [0,4194304] 0 2024-02-15T18:27:10.494 INFO:tasks.workunit.client.1.smithi191.stdout:6/500: truncate d8/d9/d2b/d57/f63 1665342 0 2024-02-15T18:27:10.496 INFO:tasks.workunit.client.0.smithi042.stdout:3/988: mknod d1/de/d1a/d135/c144 0 2024-02-15T18:27:10.497 INFO:tasks.workunit.client.0.smithi042.stdout:8/896: dread dd/d11/f15 [4194304,4194304] 0 2024-02-15T18:27:10.498 INFO:tasks.workunit.client.0.smithi042.stdout:3/989: link d1/d11/d7a/l7c d1/d11/d2e/d11d/l145 0 2024-02-15T18:27:10.499 INFO:tasks.workunit.client.0.smithi042.stdout:8/897: rename dd/de/d12/f45 to dd/d11/f103 0 2024-02-15T18:27:10.501 INFO:tasks.workunit.client.0.smithi042.stdout:8/898: creat dd/de/d33/dbf/df0/f104 x:0 0 0 2024-02-15T18:27:10.501 INFO:tasks.workunit.client.0.smithi042.stdout:8/899: readlink dd/de/d33/d3b/l7b 0 2024-02-15T18:27:10.501 INFO:tasks.workunit.client.0.smithi042.stdout:8/900: read - dd/de/dc0/fed zero size 2024-02-15T18:27:10.501 INFO:tasks.workunit.client.0.smithi042.stdout:8/901: read - dd/de/da4/d67/d5a/db2/fc3 zero size 2024-02-15T18:27:10.501 INFO:tasks.workunit.client.0.smithi042.stdout:8/902: fdatasync dd/de/d94/fcb 0 2024-02-15T18:27:10.508 INFO:tasks.workunit.client.1.smithi191.stdout:6/501: dread d8/d19/f35 [0,4194304] 0 2024-02-15T18:27:10.508 INFO:tasks.workunit.client.1.smithi191.stdout:6/502: chown d8/f30 113946 1 2024-02-15T18:27:10.512 INFO:tasks.workunit.client.0.smithi042.stdout:0/968: dwrite d4/d9/d34/fe7 [0,4194304] 0 2024-02-15T18:27:10.515 INFO:tasks.workunit.client.0.smithi042.stdout:0/969: rename d4/d9/d62/f53 to d4/d9/d34/d57/f139 0 2024-02-15T18:27:10.517 INFO:tasks.workunit.client.0.smithi042.stdout:0/970: rename d4/d9/f46 to d4/d38/d39/d8e/d8f/f13a 0 2024-02-15T18:27:10.517 INFO:tasks.workunit.client.0.smithi042.stdout:0/971: write d4/dc2/fd3 [567610,26168] 0 2024-02-15T18:27:10.519 INFO:tasks.workunit.client.0.smithi042.stdout:0/972: write d4/d38/d39/df4/d114/d6c/df9/d11f/d121/dfc/f78 [3556531,12268] 0 2024-02-15T18:27:10.519 INFO:tasks.workunit.client.0.smithi042.stdout:0/973: write d4/d38/d39/df4/d114/d6c/df9/d11f/d121/dfc/f83 [1394909,66318] 0 2024-02-15T18:27:10.520 INFO:tasks.workunit.client.1.smithi191.stdout:6/503: write d8/f33 [4171424,50116] 0 2024-02-15T18:27:10.521 INFO:tasks.workunit.client.0.smithi042.stdout:2/955: dwrite d0/d8/d10/d31/d3a/d53/fcc [0,4194304] 0 2024-02-15T18:27:10.521 INFO:tasks.workunit.client.0.smithi042.stdout:0/974: mknod d4/d38/d77/d2e/c13b 0 2024-02-15T18:27:10.522 INFO:tasks.workunit.client.1.smithi191.stdout:6/504: mknod d8/d9/d4a/c94 0 2024-02-15T18:27:10.525 INFO:tasks.workunit.client.0.smithi042.stdout:8/903: write dd/f7d [1277090,16355] 0 2024-02-15T18:27:10.525 INFO:tasks.workunit.client.0.smithi042.stdout:8/904: truncate dd/de/da4/d67/d6f/fb0 246192 0 2024-02-15T18:27:10.527 INFO:tasks.workunit.client.0.smithi042.stdout:2/956: creat d0/d8/d20/d41/dbe/f130 x:0 0 0 2024-02-15T18:27:10.527 INFO:tasks.workunit.client.1.smithi191.stdout:6/505: dread d8/d9/d2b/d57/f60 [0,4194304] 0 2024-02-15T18:27:10.527 INFO:tasks.workunit.client.1.smithi191.stdout:6/506: write d8/d9/d2b/f4b [4633342,92424] 0 2024-02-15T18:27:10.528 INFO:tasks.workunit.client.1.smithi191.stdout:6/507: chown d8/d9/f79 22 1 2024-02-15T18:27:10.528 INFO:tasks.workunit.client.1.smithi191.stdout:6/508: read - d8/d9/d2b/d57/d8e/f92 zero size 2024-02-15T18:27:10.528 INFO:tasks.workunit.client.1.smithi191.stdout:4/555: dwrite d2/d5/d7/d3e/d63/d8c/fb0 [0,4194304] 0 2024-02-15T18:27:10.528 INFO:tasks.workunit.client.0.smithi042.stdout:8/905: mknod dd/de/d33/c105 0 2024-02-15T18:27:10.529 INFO:tasks.workunit.client.0.smithi042.stdout:0/975: dread d4/d38/d39/df4/d114/dfe/ff1 [0,4194304] 0 2024-02-15T18:27:10.529 INFO:tasks.workunit.client.0.smithi042.stdout:3/990: dwrite d1/de/d1a/fa4 [0,4194304] 0 2024-02-15T18:27:10.533 INFO:tasks.workunit.client.1.smithi191.stdout:6/509: rename d8/d9/d4a/d4c/d7f/d8c to d8/d9/d61/d95 0 2024-02-15T18:27:10.535 INFO:tasks.workunit.client.0.smithi042.stdout:8/906: unlink dd/c14 0 2024-02-15T18:27:10.540 INFO:tasks.workunit.client.1.smithi191.stdout:4/556: getdents d2/d5/d7/d3e/d88 0 2024-02-15T18:27:10.543 INFO:tasks.workunit.client.0.smithi042.stdout:0/976: dread d4/d9/d34/d8b/f90 [0,4194304] 0 2024-02-15T18:27:10.550 INFO:tasks.workunit.client.0.smithi042.stdout:3/991: stat d1/d11/d14/d4d/l68 0 2024-02-15T18:27:10.554 INFO:tasks.workunit.client.1.smithi191.stdout:4/557: dread d2/d5/d7/d3e/f94 [0,4194304] 0 2024-02-15T18:27:10.554 INFO:tasks.workunit.client.0.smithi042.stdout:8/907: rename dd/de/d3c/f102 to dd/de/d33/d66/d4e/df8/f106 0 2024-02-15T18:27:10.555 INFO:tasks.workunit.client.0.smithi042.stdout:0/977: rmdir d4/d38/d39 39 2024-02-15T18:27:10.556 INFO:tasks.workunit.client.0.smithi042.stdout:2/957: dwrite d0/d8/d20/d7d/d94/f87 [0,4194304] 0 2024-02-15T18:27:10.557 INFO:tasks.workunit.client.1.smithi191.stdout:4/558: link d2/d5/d7/lbd d2/d5/d7/d90/d52/db5/ld2 0 2024-02-15T18:27:10.559 INFO:tasks.workunit.client.1.smithi191.stdout:4/559: mkdir d2/d5/d14/d47/d78/dd0/dd3 0 2024-02-15T18:27:10.561 INFO:tasks.workunit.client.1.smithi191.stdout:4/560: creat d2/d5/d7/d90/fd4 x:0 0 0 2024-02-15T18:27:10.563 INFO:tasks.workunit.client.0.smithi042.stdout:4/976: sync 2024-02-15T18:27:10.564 INFO:tasks.workunit.client.1.smithi191.stdout:6/510: dwrite d8/d9/d4a/d4c/f66 [0,4194304] 0 2024-02-15T18:27:10.569 INFO:tasks.workunit.client.0.smithi042.stdout:2/958: symlink d0/d8/d20/d7d/d7e/dc7/d8b/l131 0 2024-02-15T18:27:10.573 INFO:tasks.workunit.client.0.smithi042.stdout:4/977: getdents da/d11/d67 0 2024-02-15T18:27:10.573 INFO:tasks.workunit.client.1.smithi191.stdout:4/561: dread d2/d5/d7/d90/d52/db5/fc3 [0,4194304] 0 2024-02-15T18:27:10.573 INFO:tasks.workunit.client.1.smithi191.stdout:4/562: chown d2/f81 3 1 2024-02-15T18:27:10.573 INFO:tasks.workunit.client.1.smithi191.stdout:6/511: creat d8/d9/d61/d95/d90/f96 x:0 0 0 2024-02-15T18:27:10.577 INFO:tasks.workunit.client.0.smithi042.stdout:2/959: rename d0/d8/d20/d7d/d7e/dc7/d8b/dd7/dda to d0/d8/d20/d3f/d132 0 2024-02-15T18:27:10.578 INFO:tasks.workunit.client.1.smithi191.stdout:4/563: mknod d2/d5/d14/d71/dcd/cd5 0 2024-02-15T18:27:10.579 INFO:tasks.workunit.client.0.smithi042.stdout:4/978: creat da/d11/d85/db5/d103/d66/d78/f159 x:0 0 0 2024-02-15T18:27:10.580 INFO:tasks.workunit.client.0.smithi042.stdout:4/979: write da/d11/d85/db5/d103/d66/d125/fa9 [4149035,14270] 0 2024-02-15T18:27:10.582 INFO:tasks.workunit.client.0.smithi042.stdout:0/978: dwrite d4/d38/d5c/f61 [0,4194304] 0 2024-02-15T18:27:10.582 INFO:tasks.workunit.client.0.smithi042.stdout:0/979: fsync d4/dc2/fd3 0 2024-02-15T18:27:10.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.586+0000 7f3a73def700 1 -- 172.21.15.42:0/552804446 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a6c057f10 con 0x7f3a58060fb0 2024-02-15T18:27:10.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.588+0000 7f3a627fc700 1 -- 172.21.15.42:0/552804446 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f3a6c057f10 con 0x7f3a58060fb0 2024-02-15T18:27:10.585 INFO:tasks.workunit.client.1.smithi191.stdout:6/512: rename d8/d19/f35 to d8/f97 0 2024-02-15T18:27:10.586 INFO:tasks.workunit.client.0.smithi042.stdout:8/908: dwrite dd/de/f1a [0,4194304] 0 2024-02-15T18:27:10.586 INFO:tasks.workunit.client.0.smithi042.stdout:8/909: stat dd/de/d94/db1/lff 0 2024-02-15T18:27:10.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.590+0000 7f3a73def700 1 -- 172.21.15.42:0/552804446 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3a58060fb0 msgr2=0x7f3a58063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:10.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.590+0000 7f3a73def700 1 --2- 172.21.15.42:0/552804446 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3a58060fb0 0x7f3a58063470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f3a5c004630 tx=0x7f3a5c009b30).stop 2024-02-15T18:27:10.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.590+0000 7f3a73def700 1 -- 172.21.15.42:0/552804446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a6c0f55c0 msgr2=0x7f3a6c109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:10.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.590+0000 7f3a73def700 1 --2- 172.21.15.42:0/552804446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a6c0f55c0 0x7f3a6c109c70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f3a68002700 tx=0x7f3a68009d80).stop 2024-02-15T18:27:10.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.590+0000 7f3a73def700 1 -- 172.21.15.42:0/552804446 shutdown_connections 2024-02-15T18:27:10.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.590+0000 7f3a73def700 1 --2- 172.21.15.42:0/552804446 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3a58060fb0 0x7f3a58063470 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:10.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.590+0000 7f3a73def700 1 --2- 172.21.15.42:0/552804446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a6c0f55c0 0x7f3a6c109c70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:10.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.590+0000 7f3a73def700 1 --2- 172.21.15.42:0/552804446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3a6c0f6860 0x7f3a6c10c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:10.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.590+0000 7f3a73def700 1 -- 172.21.15.42:0/552804446 >> 172.21.15.42:0/552804446 conn(0x7f3a6c0f0d40 msgr2=0x7f3a6c0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:10.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.590+0000 7f3a73def700 1 -- 172.21.15.42:0/552804446 shutdown_connections 2024-02-15T18:27:10.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.590+0000 7f3a73def700 1 -- 172.21.15.42:0/552804446 wait complete. 2024-02-15T18:27:10.587 INFO:tasks.workunit.client.0.smithi042.stdout:2/960: rename d0/c9a to d0/d8/d20/d3f/d132/de/d1a/d2e/c133 0 2024-02-15T18:27:10.589 INFO:tasks.workunit.client.0.smithi042.stdout:4/980: rename da/f7c to da/f15a 0 2024-02-15T18:27:10.591 INFO:tasks.workunit.client.1.smithi191.stdout:4/564: rename d2/d5/d14/fa8 to d2/d5/d7/d90/d52/dcb/fd6 0 2024-02-15T18:27:10.591 INFO:tasks.workunit.client.0.smithi042.stdout:0/980: truncate d4/d38/d39/f108 3829726 0 2024-02-15T18:27:10.591 INFO:tasks.workunit.client.0.smithi042.stdout:0/981: readlink d4/d9/d62/l65 0 2024-02-15T18:27:10.594 INFO:tasks.workunit.client.0.smithi042.stdout:8/910: mkdir dd/de/dc0/dfd/d107 0 2024-02-15T18:27:10.594 INFO:tasks.workunit.client.1.smithi191.stdout:4/565: creat d2/d5/d7/d3e/d63/d8c/da4/fd7 x:0 0 0 2024-02-15T18:27:10.595 INFO:tasks.workunit.client.1.smithi191.stdout:4/566: write d2/d5/d14/d71/fac [547935,64599] 0 2024-02-15T18:27:10.596 INFO:tasks.workunit.client.0.smithi042.stdout:4/981: symlink da/d11/d29/d4a/l15b 0 2024-02-15T18:27:10.597 INFO:tasks.workunit.client.1.smithi191.stdout:6/513: dread d8/d9/f44 [0,4194304] 0 2024-02-15T18:27:10.597 INFO:tasks.workunit.client.1.smithi191.stdout:6/514: chown d8/l41 0 1 2024-02-15T18:27:10.597 INFO:tasks.workunit.client.1.smithi191.stdout:6/515: write d8/d9/d4a/d4c/d69/f7c [696661,64792] 0 2024-02-15T18:27:10.598 INFO:tasks.workunit.client.0.smithi042.stdout:0/982: unlink d4/d9/d34/dab/db0/f116 0 2024-02-15T18:27:10.598 INFO:tasks.workunit.client.0.smithi042.stdout:8/911: symlink dd/de/d33/dbf/df0/l108 0 2024-02-15T18:27:10.598 INFO:tasks.workunit.client.0.smithi042.stdout:8/912: write dd/f29 [3403140,90448] 0 2024-02-15T18:27:10.600 INFO:tasks.workunit.client.1.smithi191.stdout:6/516: mkdir d8/d19/d6b/d98 0 2024-02-15T18:27:10.600 INFO:tasks.workunit.client.1.smithi191.stdout:6/517: write d8/d9/d2b/d57/d8e/f92 [984473,66947] 0 2024-02-15T18:27:10.603 INFO:tasks.workunit.client.0.smithi042.stdout:4/982: unlink da/d11/d29/d4a/d4f/dcf/cff 0 2024-02-15T18:27:10.603 INFO:tasks.workunit.client.0.smithi042.stdout:0/983: creat d4/d9/d62/d74/f13c x:0 0 0 2024-02-15T18:27:10.603 INFO:tasks.workunit.client.0.smithi042.stdout:0/984: write d4/d38/d5c/fa2 [891087,72515] 0 2024-02-15T18:27:10.612 INFO:tasks.workunit.client.0.smithi042.stdout:8/913: rename dd/de/da4/d67/daa/cc5 to dd/de/da4/d67/c109 0 2024-02-15T18:27:10.613 INFO:tasks.workunit.client.0.smithi042.stdout:2/961: dwrite d0/d40/d109/f12b [0,4194304] 0 2024-02-15T18:27:10.614 INFO:tasks.workunit.client.0.smithi042.stdout:4/983: link da/d2b/f127 da/d11/d29/d11e/d12b/f15c 0 2024-02-15T18:27:10.617 INFO:tasks.workunit.client.1.smithi191.stdout:4/567: dwrite d2/d5/d7/d90/f6f [0,4194304] 0 2024-02-15T18:27:10.618 INFO:tasks.workunit.client.1.smithi191.stdout:4/568: dread d2/d5/d14/d47/f84 [0,4194304] 0 2024-02-15T18:27:10.618 INFO:tasks.workunit.client.0.smithi042.stdout:2/962: symlink d0/d8/d10/d5a/l134 0 2024-02-15T18:27:10.619 INFO:tasks.workunit.client.0.smithi042.stdout:2/963: dread - d0/d8/d20/d7d/d7e/f107 zero size 2024-02-15T18:27:10.620 INFO:tasks.workunit.client.0.smithi042.stdout:4/984: mknod da/d11/d85/db5/d103/d66/d78/d53/d82/dc0/c15d 0 2024-02-15T18:27:10.621 INFO:tasks.workunit.client.1.smithi191.stdout:4/569: mknod d2/d5/d14/d47/d48/cd8 0 2024-02-15T18:27:10.621 INFO:tasks.workunit.client.1.smithi191.stdout:4/570: fsync d2/d5/d14/d47/d78/f61 0 2024-02-15T18:27:10.622 INFO:tasks.workunit.client.0.smithi042.stdout:2/964: creat d0/d8/d20/d7d/d7e/dc7/d8b/dd7/f135 x:0 0 0 2024-02-15T18:27:10.622 INFO:tasks.workunit.client.0.smithi042.stdout:2/965: readlink d0/d8/d10/d31/d3a/l6c 0 2024-02-15T18:27:10.622 INFO:tasks.workunit.client.0.smithi042.stdout:0/985: dwrite d4/d9/d62/fa0 [4194304,4194304] 0 2024-02-15T18:27:10.623 INFO:tasks.workunit.client.0.smithi042.stdout:8/914: dread dd/de/da4/d67/d5a/f93 [0,4194304] 0 2024-02-15T18:27:10.627 INFO:tasks.workunit.client.1.smithi191.stdout:8/632: sync 2024-02-15T18:27:10.627 INFO:tasks.workunit.client.0.smithi042.stdout:2/966: truncate d0/d8/d20/d7d/d7e/dc7/f73 792414 0 2024-02-15T18:27:10.629 INFO:tasks.workunit.client.0.smithi042.stdout:3/992: sync 2024-02-15T18:27:10.630 INFO:tasks.workunit.client.1.smithi191.stdout:4/571: link d2/d5/d7/d3e/d88/l7e d2/d5/d7/d2f/ld9 0 2024-02-15T18:27:10.632 INFO:tasks.workunit.client.0.smithi042.stdout:8/915: rmdir dd/de/d12 39 2024-02-15T18:27:10.633 INFO:tasks.workunit.client.0.smithi042.stdout:4/985: dwrite da/d11/d29/d4a/ff1 [0,4194304] 0 2024-02-15T18:27:10.635 INFO:tasks.workunit.client.0.smithi042.stdout:2/967: mknod d0/d8/d10/d31/da6/c136 0 2024-02-15T18:27:10.637 INFO:tasks.workunit.client.1.smithi191.stdout:4/572: write d2/d5/d7/d2f/f75 [468674,128693] 0 2024-02-15T18:27:10.637 INFO:tasks.workunit.client.1.smithi191.stdout:4/573: chown d2/d5/c43 2 1 2024-02-15T18:27:10.637 INFO:tasks.workunit.client.1.smithi191.stdout:8/633: symlink d3/d2c/d1a/d23/d2e/d64/le5 0 2024-02-15T18:27:10.639 INFO:tasks.workunit.client.0.smithi042.stdout:3/993: mknod d1/de/d1a/c146 0 2024-02-15T18:27:10.641 INFO:tasks.workunit.client.1.smithi191.stdout:4/574: mkdir d2/d5/d7/d90/da5/dda 0 2024-02-15T18:27:10.642 INFO:tasks.workunit.client.0.smithi042.stdout:0/986: link d4/d9/d62/d74/f13c d4/d38/d118/d137/f13d 0 2024-02-15T18:27:10.643 INFO:tasks.workunit.client.1.smithi191.stdout:8/634: creat d3/d2c/d1a/d23/d2e/d47/fe6 x:0 0 0 2024-02-15T18:27:10.644 INFO:tasks.workunit.client.1.smithi191.stdout:8/635: dread d3/d2c/d1a/d23/d2e/d47/f63 [0,4194304] 0 2024-02-15T18:27:10.645 INFO:tasks.workunit.client.0.smithi042.stdout:8/916: symlink dd/de/da4/l10a 0 2024-02-15T18:27:10.647 INFO:tasks.workunit.client.1.smithi191.stdout:4/575: mknod d2/d5/d14/d47/d48/cdb 0 2024-02-15T18:27:10.647 INFO:tasks.workunit.client.1.smithi191.stdout:4/576: truncate d2/d5/d7/f29 4648236 0 2024-02-15T18:27:10.648 INFO:tasks.workunit.client.1.smithi191.stdout:8/636: symlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/le7 0 2024-02-15T18:27:10.648 INFO:tasks.workunit.client.1.smithi191.stdout:8/637: write d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/fad [603676,85159] 0 2024-02-15T18:27:10.649 INFO:tasks.workunit.client.1.smithi191.stdout:8/638: chown d3/l71 15474 1 2024-02-15T18:27:10.649 INFO:tasks.workunit.client.1.smithi191.stdout:8/639: stat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95 0 2024-02-15T18:27:10.650 INFO:tasks.workunit.client.0.smithi042.stdout:4/986: mkdir da/d2b/d7e/d15e 0 2024-02-15T18:27:10.653 INFO:tasks.workunit.client.0.smithi042.stdout:2/968: creat d0/d8/d20/d3f/d132/d44/f137 x:0 0 0 2024-02-15T18:27:10.654 INFO:tasks.workunit.client.0.smithi042.stdout:3/994: symlink d1/d11/d14/d4d/d73/d89/da8/d11a/l147 0 2024-02-15T18:27:10.655 INFO:tasks.workunit.client.0.smithi042.stdout:0/987: getdents d4/d38/d39/df4 0 2024-02-15T18:27:10.655 INFO:tasks.workunit.client.0.smithi042.stdout:8/917: symlink dd/de/da4/d67/d6f/l10b 0 2024-02-15T18:27:10.656 INFO:tasks.workunit.client.0.smithi042.stdout:8/918: fdatasync dd/de/d33/d3b/d92/fa8 0 2024-02-15T18:27:10.656 INFO:tasks.workunit.client.0.smithi042.stdout:8/919: fsync dd/de/da4/d67/d6f/fbe 0 2024-02-15T18:27:10.656 INFO:tasks.workunit.client.0.smithi042.stdout:8/920: stat dd/de/da4/d67/d5a/db2/ff1 0 2024-02-15T18:27:10.657 INFO:tasks.workunit.client.0.smithi042.stdout:8/921: mkdir dd/de/d3c/d10c 0 2024-02-15T18:27:10.657 INFO:tasks.workunit.client.0.smithi042.stdout:8/922: dread - dd/de/d12/fe5 zero size 2024-02-15T18:27:10.662 INFO:tasks.workunit.client.0.smithi042.stdout:8/923: unlink dd/de/da4/d67/d6f/d78/d9a/dce/fd9 0 2024-02-15T18:27:10.663 INFO:tasks.workunit.client.0.smithi042.stdout:8/924: creat dd/de/da4/d81/dae/f10d x:0 0 0 2024-02-15T18:27:10.667 INFO:tasks.workunit.client.0.smithi042.stdout:3/995: dread d1/de/d19/f3e [0,4194304] 0 2024-02-15T18:27:10.670 INFO:tasks.workunit.client.0.smithi042.stdout:3/996: rmdir d1/d11/d2e/dd1 39 2024-02-15T18:27:10.703 INFO:tasks.workunit.client.1.smithi191.stdout:8/640: dwrite d3/d2c/d1a/d23/d2e/f3a [0,4194304] 0 2024-02-15T18:27:10.704 INFO:tasks.workunit.client.1.smithi191.stdout:8/641: read d3/f18 [3078817,123648] 0 2024-02-15T18:27:10.705 INFO:tasks.workunit.client.1.smithi191.stdout:8/642: truncate d3/d2c/d1a/f8f 1616574 0 2024-02-15T18:27:10.705 INFO:tasks.workunit.client.1.smithi191.stdout:8/643: fdatasync d3/d2c/d1a/d23/d3e/fb4 0 2024-02-15T18:27:10.709 INFO:tasks.workunit.client.1.smithi191.stdout:8/644: write d3/d2c/d1a/fb1 [3338654,58792] 0 2024-02-15T18:27:10.710 INFO:tasks.workunit.client.1.smithi191.stdout:4/577: dwrite d2/d5/d7/d90/d52/db5/fc3 [0,4194304] 0 2024-02-15T18:27:10.711 INFO:tasks.workunit.client.1.smithi191.stdout:8/645: mkdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/de8 0 2024-02-15T18:27:10.713 INFO:tasks.workunit.client.1.smithi191.stdout:4/578: mkdir d2/d5/d7/d3e/d63/d8c/dbb/ddc 0 2024-02-15T18:27:10.714 INFO:tasks.workunit.client.1.smithi191.stdout:8/646: mkdir d3/d2c/d1a/d23/d2e/de9 0 2024-02-15T18:27:10.714 INFO:tasks.workunit.client.1.smithi191.stdout:8/647: chown d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/fc6 4211 1 2024-02-15T18:27:10.715 INFO:tasks.workunit.client.1.smithi191.stdout:4/579: symlink d2/d5/d14/d47/d78/dd0/ldd 0 2024-02-15T18:27:10.715 INFO:tasks.workunit.client.1.smithi191.stdout:4/580: fdatasync d2/d5/d7/d2f/d41/f92 0 2024-02-15T18:27:10.715 INFO:tasks.workunit.client.1.smithi191.stdout:4/581: symlink d2/d5/d7/d90/d52/lde 0 2024-02-15T18:27:10.718 INFO:tasks.workunit.client.0.smithi042.stdout:0/988: dwrite d4/d38/d39/df4/d114/d6c/f71 [4194304,4194304] 0 2024-02-15T18:27:10.718 INFO:tasks.workunit.client.0.smithi042.stdout:4/987: dwrite da/d11/d29/d11e/d12c/ff5 [0,4194304] 0 2024-02-15T18:27:10.719 INFO:tasks.workunit.client.1.smithi191.stdout:4/582: truncate d2/d5/d7/f29 487806 0 2024-02-15T18:27:10.719 INFO:tasks.workunit.client.0.smithi042.stdout:2/969: dwrite d0/d8/d20/d3f/d132/f80 [0,4194304] 0 2024-02-15T18:27:10.721 INFO:tasks.workunit.client.1.smithi191.stdout:4/583: creat d2/d5/d7/d3e/d63/d8c/dbb/ddc/fdf x:0 0 0 2024-02-15T18:27:10.723 INFO:tasks.workunit.client.1.smithi191.stdout:4/584: link d2/d5/l11 d2/d5/d7/d3e/d88/le0 0 2024-02-15T18:27:10.724 INFO:tasks.workunit.client.1.smithi191.stdout:4/585: rmdir d2/d5/d7/d90/d52 39 2024-02-15T18:27:10.724 INFO:tasks.workunit.client.0.smithi042.stdout:4/988: mknod da/d11/d29/d11e/d12b/c15f 0 2024-02-15T18:27:10.724 INFO:tasks.workunit.client.0.smithi042.stdout:3/997: dwrite d1/de/d1a/f77 [0,4194304] 0 2024-02-15T18:27:10.727 INFO:tasks.workunit.client.1.smithi191.stdout:4/586: rename d2/d5/d7/d3e/d63/d8c/da4/f76 to d2/d5/d7/fe1 0 2024-02-15T18:27:10.730 INFO:tasks.workunit.client.0.smithi042.stdout:3/998: rename d1/d11/d14/d4d/l55 to d1/d11/d2e/dd1/l148 0 2024-02-15T18:27:10.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.736+0000 7fc857353700 1 -- 172.21.15.42:0/1985349342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8500f55e0 msgr2=0x7fc8500f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:10.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.736+0000 7fc857353700 1 --2- 172.21.15.42:0/1985349342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8500f55e0 0x7fc8500f5a00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fc84c004660 tx=0x7fc84c009b30).stop 2024-02-15T18:27:10.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.736+0000 7fc857353700 1 -- 172.21.15.42:0/1985349342 shutdown_connections 2024-02-15T18:27:10.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.736+0000 7fc857353700 1 --2- 172.21.15.42:0/1985349342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8500f5ef0 0x7fc8500f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:27:10.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.736+0000 7fc857353700 1 --2- 172.21.15.42:0/1985349342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8500f55e0 0x7fc8500f5a00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:10.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.736+0000 7fc857353700 1 -- 172.21.15.42:0/1985349342 >> 172.21.15.42:0/1985349342 conn(0x7fc8500f0de0 msgr2=0x7fc8500f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:10.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.736+0000 7fc857353700 1 -- 172.21.15.42:0/1985349342 shutdown_connections 2024-02-15T18:27:10.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.736+0000 7fc857353700 1 -- 172.21.15.42:0/1985349342 wait complete. 2024-02-15T18:27:10.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.737+0000 7fc857353700 1 Processor -- start 2024-02-15T18:27:10.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.737+0000 7fc857353700 1 -- start start 2024-02-15T18:27:10.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.737+0000 7fc857353700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8500f55e0 0x7fc85010d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:10.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.737+0000 7fc857353700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8500f5ef0 0x7fc85010db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:10.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.737+0000 7fc857353700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85010e1b0 con 0x7fc8500f55e0 2024-02-15T18:27:10.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.737+0000 7fc857353700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85010ea60 con 0x7fc8500f5ef0 2024-02-15T18:27:10.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.738+0000 7fc8548ee700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8500f5ef0 0x7fc85010db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:10.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.738+0000 7fc8548ee700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8500f5ef0 0x7fc85010db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.42:57930/0 (socket says 172.21.15.42:57930) 2024-02-15T18:27:10.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.738+0000 7fc8548ee700 1 -- 172.21.15.42:0/805238366 learned_addr learned my addr 172.21.15.42:0/805238366 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:27:10.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.738+0000 7fc8550ef700 1 --2- 172.21.15.42:0/805238366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8500f55e0 0x7fc85010d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:10.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.739+0000 7fc8548ee700 1 -- 172.21.15.42:0/805238366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8500f55e0 msgr2=0x7fc85010d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:10.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.739+0000 7fc8548ee700 1 --2- 172.21.15.42:0/805238366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8500f55e0 0x7fc85010d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:10.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.739+0000 7fc8548ee700 1 -- 172.21.15.42:0/805238366 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c005040 con 0x7fc8500f5ef0 2024-02-15T18:27:10.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.739+0000 7fc8548ee700 1 --2- 172.21.15.42:0/805238366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8500f5ef0 0x7fc85010db70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fc84000b410 tx=0x7fc840006ef0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:10.736 INFO:tasks.workunit.client.0.smithi042.stdout:4/989: mknod da/d3a/dc9/c160 0 2024-02-15T18:27:10.736 INFO:tasks.workunit.client.0.smithi042.stdout:3/999: write d1/de/d19/f48 [3163548,35414] 0 2024-02-15T18:27:10.737 INFO:tasks.workunit.client.0.smithi042.stdout:0/989: dread d4/d9/f22 [0,4194304] 0 2024-02-15T18:27:10.737 INFO:tasks.workunit.client.0.smithi042.stdout:4/990: symlink da/d2b/d7e/l161 0 2024-02-15T18:27:10.738 INFO:tasks.workunit.client.0.smithi042.stdout:0/990: rmdir d4/d38/d39/df4/d114 39 2024-02-15T18:27:10.738 INFO:tasks.workunit.client.0.smithi042.stdout:0/991: dread - d4/d9/d62/d10c/f122 zero size 2024-02-15T18:27:10.738 INFO:tasks.workunit.client.0.smithi042.stdout:0/992: mkdir d4/d38/d5c/d13e 0 2024-02-15T18:27:10.739 INFO:tasks.workunit.client.1.smithi191.stdout:4/587: dread d2/d5/d7/d90/d52/f7b [0,4194304] 0 2024-02-15T18:27:10.739 INFO:tasks.workunit.client.1.smithi191.stdout:4/588: truncate d2/d5/d7/d3e/faf 754427 0 2024-02-15T18:27:10.739 INFO:tasks.workunit.client.1.smithi191.stdout:4/589: stat d2/d5/d14/d47/d78/dd0 0 2024-02-15T18:27:10.739 INFO:tasks.workunit.client.1.smithi191.stdout:4/590: fdatasync d2/d5/d14/d47/d78/dd0/fd1 0 2024-02-15T18:27:10.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.746+0000 7fc845ffb700 1 -- 172.21.15.42:0/805238366 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8400125f0 con 0x7fc8500f5ef0 2024-02-15T18:27:10.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.746+0000 7fc845ffb700 1 -- 172.21.15.42:0/805238366 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fc84000be30 con 0x7fc8500f5ef0 2024-02-15T18:27:10.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.746+0000 7fc857353700 1 -- 172.21.15.42:0/805238366 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85010ed40 con 0x7fc8500f5ef0 2024-02-15T18:27:10.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.747+0000 7fc857353700 1 -- 172.21.15.42:0/805238366 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85010f280 con 0x7fc8500f5ef0 2024-02-15T18:27:10.743 INFO:tasks.workunit.client.0.smithi042.stdout:4/991: write da/d11/d27/f89 [54011,40621] 0 2024-02-15T18:27:10.743 INFO:tasks.workunit.client.0.smithi042.stdout:4/992: dread - da/d2b/d6e/f106 zero size 2024-02-15T18:27:10.744 INFO:tasks.workunit.client.0.smithi042.stdout:0/993: rmdir d4/d38/d39/df4/d114/d6c/d70 39 2024-02-15T18:27:10.744 INFO:tasks.workunit.client.0.smithi042.stdout:0/994: chown d4/d9/d34/d57/faf 66 1 2024-02-15T18:27:10.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.747+0000 7fc845ffb700 1 -- 172.21.15.42:0/805238366 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc840002b70 con 0x7fc8500f5ef0 2024-02-15T18:27:10.744 INFO:tasks.workunit.client.0.smithi042.stdout:4/993: dread da/d3a/f100 [0,4194304] 0 2024-02-15T18:27:10.744 INFO:tasks.workunit.client.0.smithi042.stdout:4/994: chown da/d11/d85/db5/d103/d66/d78/d2f/d4d/f47 0 1 2024-02-15T18:27:10.745 INFO:tasks.workunit.client.0.smithi042.stdout:4/995: chown da/d11/d67/c76 65 1 2024-02-15T18:27:10.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.748+0000 7fc857353700 1 -- 172.21.15.42:0/805238366 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc830004fa0 con 0x7fc8500f5ef0 2024-02-15T18:27:10.745 INFO:tasks.workunit.client.0.smithi042.stdout:2/970: dwrite d0/d8/d20/d7d/d7e/dc7/f73 [0,4194304] 0 2024-02-15T18:27:10.745 INFO:tasks.workunit.client.0.smithi042.stdout:4/996: mknod da/d3a/dbb/c162 0 2024-02-15T18:27:10.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.749+0000 7fc845ffb700 1 -- 172.21.15.42:0/805238366 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 30) v1 ==== 77194+0+0 (secure 0 0 0) 0x7fc84000f030 con 0x7fc8500f5ef0 2024-02-15T18:27:10.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.749+0000 7fc845ffb700 1 --2- 172.21.15.42:0/805238366 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc83c060fb0 0x7fc83c063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:10.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.749+0000 7fc845ffb700 1 -- 172.21.15.42:0/805238366 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fc84007c4c0 con 0x7fc8500f5ef0 2024-02-15T18:27:10.747 INFO:tasks.workunit.client.0.smithi042.stdout:0/995: truncate d4/d38/d39/df4/d114/f87 3546459 0 2024-02-15T18:27:10.747 INFO:tasks.workunit.client.1.smithi191.stdout:8/648: dwrite d3/d5e/fd1 [0,4194304] 0 2024-02-15T18:27:10.747 INFO:tasks.workunit.client.1.smithi191.stdout:8/649: readlink d3/d2c/d1a/d23/d3e/db2/ldc 0 2024-02-15T18:27:10.748 INFO:tasks.workunit.client.1.smithi191.stdout:8/650: dread - d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/fc7 zero size 2024-02-15T18:27:10.748 INFO:tasks.workunit.client.1.smithi191.stdout:8/651: dread d3/d2c/d1a/d23/d2e/f9f [0,4194304] 0 2024-02-15T18:27:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.750+0000 7fc8550ef700 1 --2- 172.21.15.42:0/805238366 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc83c060fb0 0x7fc83c063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.750+0000 7fc8550ef700 1 --2- 172.21.15.42:0/805238366 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc83c060fb0 0x7fc83c063470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc84c004630 tx=0x7fc84c009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:10.749 INFO:tasks.workunit.client.1.smithi191.stdout:8/652: symlink d3/d2c/d1a/d23/d2e/d47/lea 0 2024-02-15T18:27:10.749 INFO:tasks.workunit.client.1.smithi191.stdout:8/653: write d3/d2c/d1a/d23/d2e/d47/d5f/f50 [4418805,2896] 0 2024-02-15T18:27:10.750 INFO:tasks.workunit.client.0.smithi042.stdout:4/997: dread - da/d11/d29/d4a/d4f/d90/f14d zero size 2024-02-15T18:27:10.750 INFO:tasks.workunit.client.0.smithi042.stdout:4/998: chown da/f63 157094 1 2024-02-15T18:27:10.751 INFO:tasks.workunit.client.1.smithi191.stdout:8/654: read d3/d2c/d1a/d23/d2e/f56 [432347,84048] 0 2024-02-15T18:27:10.752 INFO:tasks.workunit.client.0.smithi042.stdout:0/996: creat d4/d9/d34/d57/f13f x:0 0 0 2024-02-15T18:27:10.753 INFO:tasks.workunit.client.0.smithi042.stdout:0/997: chown d4/dc2 106 1 2024-02-15T18:27:10.754 INFO:tasks.workunit.client.0.smithi042.stdout:4/999: mkdir da/d11/d85/db5/d103/d66/d78/d2f/d4d/d134/d163 0 2024-02-15T18:27:10.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:10.760+0000 7fc845ffb700 1 -- 172.21.15.42:0/805238366 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc840050360 con 0x7fc8500f5ef0 2024-02-15T18:27:10.757 INFO:tasks.workunit.client.1.smithi191.stdout:8/655: link d3/d2c/d1a/fdb d3/d2c/d1a/d23/d2e/d47/feb 0 2024-02-15T18:27:10.757 INFO:tasks.workunit.client.1.smithi191.stdout:8/656: chown d3/d2c/d1a/d36/f6b 0 1 2024-02-15T18:27:10.763 INFO:tasks.workunit.client.1.smithi191.stdout:5/307: sync 2024-02-15T18:27:10.763 INFO:tasks.workunit.client.1.smithi191.stdout:9/555: sync 2024-02-15T18:27:10.763 INFO:tasks.workunit.client.1.smithi191.stdout:9/556: truncate d0/d7/dc/d8c/f27 762801 0 2024-02-15T18:27:10.764 INFO:tasks.workunit.client.1.smithi191.stdout:3/444: sync 2024-02-15T18:27:10.764 INFO:tasks.workunit.client.1.smithi191.stdout:2/505: sync 2024-02-15T18:27:10.764 INFO:tasks.workunit.client.1.smithi191.stdout:3/445: readlink d6/d3b/d42/l8a 0 2024-02-15T18:27:10.764 INFO:tasks.workunit.client.1.smithi191.stdout:3/446: chown d6/d1b/d60/d73/f34 26906569 1 2024-02-15T18:27:10.764 INFO:tasks.workunit.client.1.smithi191.stdout:0/373: sync 2024-02-15T18:27:10.764 INFO:tasks.workunit.client.1.smithi191.stdout:7/520: sync 2024-02-15T18:27:10.764 INFO:tasks.workunit.client.1.smithi191.stdout:7/521: fdatasync d0/db/d65/dc/d4b/f7f 0 2024-02-15T18:27:10.764 INFO:tasks.workunit.client.1.smithi191.stdout:7/522: getdents d0/db/d26/d5f/d77/db9 0 2024-02-15T18:27:10.765 INFO:tasks.workunit.client.1.smithi191.stdout:1/606: sync 2024-02-15T18:27:10.770 INFO:tasks.workunit.client.1.smithi191.stdout:0/374: read d7/f39 [154687,46614] 0 2024-02-15T18:27:10.770 INFO:tasks.workunit.client.1.smithi191.stdout:2/506: write df/d12/d19/d1a/d35/d5b/f6e [3763743,115235] 0 2024-02-15T18:27:10.771 INFO:tasks.workunit.client.1.smithi191.stdout:2/507: chown df/d12/d19/d1a/d35 0 1 2024-02-15T18:27:10.771 INFO:tasks.workunit.client.1.smithi191.stdout:3/447: unlink l0 0 2024-02-15T18:27:10.771 INFO:tasks.workunit.client.1.smithi191.stdout:3/448: fsync d6/d3b/d30/f50 0 2024-02-15T18:27:10.771 INFO:tasks.workunit.client.1.smithi191.stdout:7/523: getdents d0/db/d26/d5f/d77/df 0 2024-02-15T18:27:10.776 INFO:tasks.workunit.client.0.smithi042.stdout:8/925: sync 2024-02-15T18:27:10.778 INFO:tasks.workunit.client.1.smithi191.stdout:0/375: creat d7/d1b/f81 x:0 0 0 2024-02-15T18:27:10.778 INFO:tasks.workunit.client.1.smithi191.stdout:0/376: stat d7/d1b 0 2024-02-15T18:27:10.792 INFO:tasks.workunit.client.0.smithi042.stdout:0/998: dwrite d4/d9/d34/f40 [0,4194304] 0 2024-02-15T18:27:10.796 INFO:tasks.workunit.client.1.smithi191.stdout:5/308: mkdir d0/d62 0 2024-02-15T18:27:10.796 INFO:tasks.workunit.client.1.smithi191.stdout:5/309: dread - d0/d23/d41/d28/f55 zero size 2024-02-15T18:27:10.797 INFO:tasks.workunit.client.1.smithi191.stdout:8/657: dwrite d3/d2c/d1a/d23/d2e/d64/dbc/fc0 [0,4194304] 0 2024-02-15T18:27:10.797 INFO:tasks.workunit.client.1.smithi191.stdout:7/524: mknod d0/db/d65/d11/d57/cc6 0 2024-02-15T18:27:10.797 INFO:tasks.workunit.client.1.smithi191.stdout:7/525: dread - d0/db/d65/f75 zero size 2024-02-15T18:27:10.798 INFO:tasks.workunit.client.1.smithi191.stdout:7/526: fdatasync d0/db/d26/d5f/d77/d35/d42/f7b 0 2024-02-15T18:27:10.799 INFO:tasks.workunit.client.1.smithi191.stdout:8/658: read d3/d2c/d1a/d23/d2e/d5b/fa1 [2091518,50181] 0 2024-02-15T18:27:10.800 INFO:tasks.workunit.client.1.smithi191.stdout:0/377: mkdir d7/da/d17/d31/d5f/d82 0 2024-02-15T18:27:10.801 INFO:tasks.workunit.client.1.smithi191.stdout:2/508: dwrite df/d12/d19/d1a/d35/d47/fa8 [0,4194304] 0 2024-02-15T18:27:10.801 INFO:tasks.workunit.client.0.smithi042.stdout:0/999: creat d4/dc2/f140 x:0 0 0 2024-02-15T18:27:10.804 INFO:tasks.workunit.client.0.smithi042.stdout:2/971: dwrite d0/d8/d10/d31/da6/fac [0,4194304] 0 2024-02-15T18:27:10.804 INFO:tasks.workunit.client.0.smithi042.stdout:8/926: dread dd/de/d12/ff2 [0,4194304] 0 2024-02-15T18:27:10.806 INFO:tasks.workunit.client.1.smithi191.stdout:5/310: unlink d0/f1f 0 2024-02-15T18:27:10.806 INFO:tasks.workunit.client.1.smithi191.stdout:5/311: write d0/d23/d41/d28/d31/f53 [193474,86898] 0 2024-02-15T18:27:10.806 INFO:tasks.workunit.client.1.smithi191.stdout:5/312: chown d0/d23/d41/d32/f54 9 1 2024-02-15T18:27:10.808 INFO:tasks.workunit.client.1.smithi191.stdout:7/527: mkdir d0/db/d26/d5f/d77/d35/dc7 0 2024-02-15T18:27:10.811 INFO:tasks.workunit.client.0.smithi042.stdout:2/972: mknod d0/d8/d20/d3f/d132/de/d1a/d11e/c138 0 2024-02-15T18:27:10.812 INFO:tasks.workunit.client.1.smithi191.stdout:2/509: symlink df/d12/d1e/lac 0 2024-02-15T18:27:10.812 INFO:tasks.workunit.client.1.smithi191.stdout:3/449: dwrite d6/d3b/d42/d49/f59 [0,4194304] 0 2024-02-15T18:27:10.812 INFO:tasks.workunit.client.1.smithi191.stdout:3/450: write d6/f65 [1647903,80188] 0 2024-02-15T18:27:10.813 INFO:tasks.workunit.client.1.smithi191.stdout:8/659: dread d3/d2c/fb5 [0,4194304] 0 2024-02-15T18:27:10.814 INFO:tasks.workunit.client.1.smithi191.stdout:9/557: dwrite d0/d7/dc/d17/d33/f43 [0,4194304] 0 2024-02-15T18:27:10.815 INFO:tasks.workunit.client.0.smithi042.stdout:2/973: stat d0/d8/d20/l12e 0 2024-02-15T18:27:10.816 INFO:tasks.workunit.client.1.smithi191.stdout:2/510: creat df/d12/d1e/d6f/fad x:0 0 0 2024-02-15T18:27:10.821 INFO:tasks.workunit.client.1.smithi191.stdout:3/451: unlink d6/d1b/d60/d73/f90 0 2024-02-15T18:27:10.822 INFO:tasks.workunit.client.1.smithi191.stdout:8/660: getdents d3/d5e 0 2024-02-15T18:27:10.825 INFO:tasks.workunit.client.1.smithi191.stdout:9/558: mkdir d0/d7/dbf 0 2024-02-15T18:27:10.826 INFO:tasks.workunit.client.1.smithi191.stdout:2/511: creat df/d12/d19/d1a/fae x:0 0 0 2024-02-15T18:27:10.827 INFO:tasks.workunit.client.1.smithi191.stdout:8/661: mkdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/dec 0 2024-02-15T18:27:10.827 INFO:tasks.workunit.client.1.smithi191.stdout:8/662: stat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/ce3 0 2024-02-15T18:27:10.828 INFO:tasks.workunit.client.1.smithi191.stdout:9/559: mknod d0/d7/dc/d8c/cc0 0 2024-02-15T18:27:10.828 INFO:tasks.workunit.client.1.smithi191.stdout:9/560: fdatasync d0/d29/fb1 0 2024-02-15T18:27:10.828 INFO:tasks.workunit.client.1.smithi191.stdout:7/528: dwrite d0/db/d26/fb4 [0,4194304] 0 2024-02-15T18:27:10.831 INFO:tasks.workunit.client.1.smithi191.stdout:9/561: dread d0/d29/f95 [0,4194304] 0 2024-02-15T18:27:10.831 INFO:tasks.workunit.client.1.smithi191.stdout:9/562: truncate d0/d7/dc/d17/d44/fba 1024588 0 2024-02-15T18:27:10.831 INFO:tasks.workunit.client.1.smithi191.stdout:2/512: rename df/d12/d19/d1a/d35/c65 to df/d61/d9f/da3/caf 0 2024-02-15T18:27:10.831 INFO:tasks.workunit.client.1.smithi191.stdout:2/513: read df/d12/d19/f77 [644205,130089] 0 2024-02-15T18:27:10.832 INFO:tasks.workunit.client.1.smithi191.stdout:8/663: creat d3/d2c/d1a/d23/d2e/d64/dca/fed x:0 0 0 2024-02-15T18:27:10.832 INFO:tasks.workunit.client.1.smithi191.stdout:2/514: read df/d12/d1f/f26 [589545,39133] 0 2024-02-15T18:27:10.833 INFO:tasks.workunit.client.1.smithi191.stdout:7/529: creat d0/db/d26/d5f/d77/d1d/fc8 x:0 0 0 2024-02-15T18:27:10.833 INFO:tasks.workunit.client.1.smithi191.stdout:7/530: stat d0/db/d65/dc/d6e/c71 0 2024-02-15T18:27:10.835 INFO:tasks.workunit.client.1.smithi191.stdout:9/563: link d0/d7/dc/c10 d0/d7/dc/d32/da3/cc1 0 2024-02-15T18:27:10.836 INFO:tasks.workunit.client.1.smithi191.stdout:8/664: dread d3/d5e/f79 [0,4194304] 0 2024-02-15T18:27:10.836 INFO:tasks.workunit.client.1.smithi191.stdout:2/515: mkdir df/d12/d19/d1a/d52/db0 0 2024-02-15T18:27:10.836 INFO:tasks.workunit.client.1.smithi191.stdout:2/516: chown df/d12/d19/d1a/d35/d47/l96 6558 1 2024-02-15T18:27:10.837 INFO:tasks.workunit.client.1.smithi191.stdout:9/564: symlink d0/d7/dc/d30/d3b/d56/d59/lc2 0 2024-02-15T18:27:10.838 INFO:tasks.workunit.client.1.smithi191.stdout:8/665: mknod d3/d2c/d1a/d23/d2e/cee 0 2024-02-15T18:27:10.841 INFO:tasks.workunit.client.1.smithi191.stdout:2/517: creat df/d12/d19/fb1 x:0 0 0 2024-02-15T18:27:10.841 INFO:tasks.workunit.client.1.smithi191.stdout:2/518: readlink df/d12/d1f/l69 0 2024-02-15T18:27:10.842 INFO:tasks.workunit.client.1.smithi191.stdout:2/519: rename df/l87 to df/lb2 0 2024-02-15T18:27:10.842 INFO:tasks.workunit.client.1.smithi191.stdout:2/520: chown df/d12/d1e/c20 402 1 2024-02-15T18:27:10.843 INFO:tasks.workunit.client.1.smithi191.stdout:2/521: read - df/d61/d94/f98 zero size 2024-02-15T18:27:10.843 INFO:tasks.workunit.client.1.smithi191.stdout:2/522: fsync df/d12/d1f/f31 0 2024-02-15T18:27:10.844 INFO:tasks.workunit.client.1.smithi191.stdout:2/523: write df/d12/d32/d4d/d89/f95 [63620,73389] 0 2024-02-15T18:27:10.844 INFO:tasks.workunit.client.1.smithi191.stdout:2/524: chown df/d12/d19/d1a/f5e 58 1 2024-02-15T18:27:10.845 INFO:tasks.workunit.client.1.smithi191.stdout:2/525: creat df/d61/d9f/da3/fb3 x:0 0 0 2024-02-15T18:27:10.845 INFO:tasks.workunit.client.1.smithi191.stdout:2/526: fdatasync df/d61/f67 0 2024-02-15T18:27:10.845 INFO:tasks.workunit.client.1.smithi191.stdout:2/527: getdents df/d12/d1e/d6f 0 2024-02-15T18:27:10.855 INFO:tasks.workunit.client.1.smithi191.stdout:7/531: dwrite d0/db/d65/dc/d4b/f59 [0,4194304] 0 2024-02-15T18:27:10.856 INFO:tasks.workunit.client.1.smithi191.stdout:7/532: stat d0/db/d65/d11/d5a/lc0 0 2024-02-15T18:27:10.856 INFO:tasks.workunit.client.1.smithi191.stdout:7/533: write d0/db/d26/d5f/d77/d35/d42/f6c [4629765,64937] 0 2024-02-15T18:27:10.856 INFO:tasks.workunit.client.1.smithi191.stdout:7/534: creat d0/db/d65/dc/d4b/dc1/fc9 x:0 0 0 2024-02-15T18:27:10.857 INFO:tasks.workunit.client.1.smithi191.stdout:7/535: symlink d0/db/d26/d5f/d77/df/lca 0 2024-02-15T18:27:10.859 INFO:tasks.workunit.client.1.smithi191.stdout:7/536: dread d0/db/d26/d5f/d77/f6a [0,4194304] 0 2024-02-15T18:27:10.862 INFO:tasks.workunit.client.1.smithi191.stdout:7/537: creat d0/fcb x:0 0 0 2024-02-15T18:27:10.863 INFO:tasks.workunit.client.1.smithi191.stdout:7/538: creat d0/db/d26/d5f/d77/d1d/fcc x:0 0 0 2024-02-15T18:27:10.863 INFO:tasks.workunit.client.1.smithi191.stdout:7/539: dread - d0/db/d26/d5f/d77/d1d/fcc zero size 2024-02-15T18:27:10.889 INFO:tasks.workunit.client.0.smithi042.stdout:8/927: sync 2024-02-15T18:27:10.889 INFO:tasks.workunit.client.0.smithi042.stdout:8/928: write dd/de/fe0 [2140370,122207] 0 2024-02-15T18:27:10.892 INFO:tasks.workunit.client.0.smithi042.stdout:8/929: mkdir dd/de/d33/dbf/d10e 0 2024-02-15T18:27:10.927 INFO:tasks.workunit.client.1.smithi191.stdout:5/313: dread d0/d23/d41/d28/f3f [0,4194304] 0 2024-02-15T18:27:10.927 INFO:tasks.workunit.client.1.smithi191.stdout:1/607: dwrite d0/d8/df/d12/d2a/d2b/f7a [0,4194304] 0 2024-02-15T18:27:10.927 INFO:tasks.workunit.client.1.smithi191.stdout:3/452: read d6/d8e/f99 [362694,56278] 0 2024-02-15T18:27:10.928 INFO:tasks.workunit.client.1.smithi191.stdout:3/453: chown d6/d8e 172 1 2024-02-15T18:27:10.931 INFO:tasks.workunit.client.1.smithi191.stdout:1/608: creat d0/d8/df/d12/d33/fca x:0 0 0 2024-02-15T18:27:10.931 INFO:tasks.workunit.client.1.smithi191.stdout:1/609: fdatasync d0/d8/f57 0 2024-02-15T18:27:10.933 INFO:tasks.workunit.client.1.smithi191.stdout:3/454: rename d6/d1b/d60/d73/f61 to d6/fa0 0 2024-02-15T18:27:10.934 INFO:tasks.workunit.client.1.smithi191.stdout:1/610: symlink d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/lcb 0 2024-02-15T18:27:10.934 INFO:tasks.workunit.client.0.smithi042.stdout:2/974: dread d0/d8/d10/d31/fa1 [0,4194304] 0 2024-02-15T18:27:10.935 INFO:tasks.workunit.client.1.smithi191.stdout:3/455: creat d6/d3b/d42/fa1 x:0 0 0 2024-02-15T18:27:10.935 INFO:tasks.workunit.client.1.smithi191.stdout:3/456: stat d6/d1b/c40 0 2024-02-15T18:27:10.935 INFO:tasks.workunit.client.1.smithi191.stdout:1/611: rmdir d0/d8/df 39 2024-02-15T18:27:10.935 INFO:tasks.workunit.client.1.smithi191.stdout:1/612: chown d0/d8/df/d12/d33/d49/f67 102005 1 2024-02-15T18:27:10.936 INFO:tasks.workunit.client.1.smithi191.stdout:1/613: write d0/d8/f69 [429704,123039] 0 2024-02-15T18:27:10.938 INFO:tasks.workunit.client.0.smithi042.stdout:2/975: mkdir d0/d8/d20/d3f/d132/dfe/d139 0 2024-02-15T18:27:10.938 INFO:tasks.workunit.client.0.smithi042.stdout:2/976: fsync d0/d8/d20/d3f/d132/de/f14 0 2024-02-15T18:27:10.942 INFO:tasks.workunit.client.0.smithi042.stdout:2/977: dread d0/d8/d20/d3f/d132/f42 [0,4194304] 0 2024-02-15T18:27:10.942 INFO:tasks.workunit.client.0.smithi042.stdout:2/978: fsync d0/d8/d20/d41/dbe/f10c 0 2024-02-15T18:27:10.943 INFO:tasks.workunit.client.1.smithi191.stdout:7/540: dread d0/f8 [0,4194304] 0 2024-02-15T18:27:10.943 INFO:tasks.workunit.client.1.smithi191.stdout:7/541: write d0/db/d65/d11/d5a/d66/d1e/fa5 [1401685,118219] 0 2024-02-15T18:27:10.944 INFO:tasks.workunit.client.1.smithi191.stdout:5/314: write d0/d18/f45 [567203,3894] 0 2024-02-15T18:27:10.944 INFO:tasks.workunit.client.1.smithi191.stdout:5/315: chown d0/f1b 1637888 1 2024-02-15T18:27:10.944 INFO:tasks.workunit.client.1.smithi191.stdout:7/542: rmdir d0/db/d26/d5f/d8c 39 2024-02-15T18:27:10.945 INFO:tasks.workunit.client.1.smithi191.stdout:5/316: creat d0/d18/d5c/f63 x:0 0 0 2024-02-15T18:27:10.945 INFO:tasks.workunit.client.1.smithi191.stdout:5/317: write d0/d18/f2e [739840,83647] 0 2024-02-15T18:27:10.947 INFO:tasks.workunit.client.1.smithi191.stdout:5/318: symlink d0/d23/d41/d3c/l64 0 2024-02-15T18:27:10.947 INFO:tasks.workunit.client.1.smithi191.stdout:5/319: truncate d0/d23/f27 2025105 0 2024-02-15T18:27:10.947 INFO:tasks.workunit.client.1.smithi191.stdout:5/320: fsync d0/d23/d41/d32/d52/f40 0 2024-02-15T18:27:10.947 INFO:tasks.workunit.client.1.smithi191.stdout:8/666: dwrite d3/d2c/d1a/f44 [0,4194304] 0 2024-02-15T18:27:10.949 INFO:tasks.workunit.client.1.smithi191.stdout:9/565: dwrite d0/d7/dc/d8c/f72 [0,4194304] 0 2024-02-15T18:27:10.949 INFO:tasks.workunit.client.1.smithi191.stdout:8/667: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/cef 0 2024-02-15T18:27:10.951 INFO:tasks.workunit.client.1.smithi191.stdout:2/528: dwrite df/d61/f79 [0,4194304] 0 2024-02-15T18:27:10.951 INFO:tasks.workunit.client.1.smithi191.stdout:9/566: creat d0/d29/d48/fc3 x:0 0 0 2024-02-15T18:27:10.951 INFO:tasks.workunit.client.1.smithi191.stdout:0/378: dwrite d7/da/d17/f47 [4194304,4194304] 0 2024-02-15T18:27:10.953 INFO:tasks.workunit.client.1.smithi191.stdout:8/668: rename d3/d2c/d1a/d23/d2a/f49 to d3/d2c/d1a/d23/d2e/de9/ff0 0 2024-02-15T18:27:10.956 INFO:tasks.workunit.client.1.smithi191.stdout:2/529: mkdir df/d12/db4 0 2024-02-15T18:27:10.956 INFO:tasks.workunit.client.1.smithi191.stdout:2/530: write df/d12/d19/d1a/d35/f85 [832922,93156] 0 2024-02-15T18:27:10.956 INFO:tasks.workunit.client.1.smithi191.stdout:2/531: read - df/d12/d32/d4d/f50 zero size 2024-02-15T18:27:10.956 INFO:tasks.workunit.client.1.smithi191.stdout:2/532: dread - df/d12/d32/d4d/f50 zero size 2024-02-15T18:27:10.957 INFO:tasks.workunit.client.1.smithi191.stdout:9/567: symlink d0/d7/dc/d17/lc4 0 2024-02-15T18:27:10.957 INFO:tasks.workunit.client.1.smithi191.stdout:9/568: write d0/d7/dc/d30/d3b/d56/fb5 [1022554,129074] 0 2024-02-15T18:27:10.962 INFO:tasks.workunit.client.1.smithi191.stdout:2/533: rename df/d12/d1e/da5 to df/d12/d1f/d42/db5 0 2024-02-15T18:27:10.963 INFO:tasks.workunit.client.1.smithi191.stdout:2/534: creat df/d61/d9f/da3/fb6 x:0 0 0 2024-02-15T18:27:10.963 INFO:tasks.workunit.client.1.smithi191.stdout:2/535: write df/d12/d19/d1a/d35/d5b/f66 [2104,103687] 0 2024-02-15T18:27:10.964 INFO:tasks.workunit.client.1.smithi191.stdout:2/536: truncate df/d12/d19/fb1 454408 0 2024-02-15T18:27:10.981 INFO:tasks.workunit.client.1.smithi191.stdout:1/614: dwrite d0/d2f/d5c/f71 [0,4194304] 0 2024-02-15T18:27:10.981 INFO:tasks.workunit.client.1.smithi191.stdout:8/669: dread d3/d5e/fd1 [0,4194304] 0 2024-02-15T18:27:10.981 INFO:tasks.workunit.client.1.smithi191.stdout:8/670: fdatasync d3/d2c/d1a/d23/f7c 0 2024-02-15T18:27:10.981 INFO:tasks.workunit.client.1.smithi191.stdout:8/671: chown d3/d2c/d1a/ca2 137549 1 2024-02-15T18:27:10.981 INFO:tasks.workunit.client.1.smithi191.stdout:8/672: chown d3/d2c/d1a/d23/d2e/de9 23601 1 2024-02-15T18:27:10.981 INFO:tasks.workunit.client.1.smithi191.stdout:8/673: chown d3/d2c/d1a/d36 249252478 1 2024-02-15T18:27:10.984 INFO:tasks.workunit.client.1.smithi191.stdout:1/615: creat d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/dc9/fcc x:0 0 0 2024-02-15T18:27:10.985 INFO:tasks.workunit.client.1.smithi191.stdout:8/674: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/da0/ff1 x:0 0 0 2024-02-15T18:27:10.985 INFO:tasks.workunit.client.1.smithi191.stdout:8/675: getdents d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/dec 0 2024-02-15T18:27:10.985 INFO:tasks.workunit.client.1.smithi191.stdout:8/676: getdents d3/d2c/d1a/d23/d2e/de9 0 2024-02-15T18:27:10.985 INFO:tasks.workunit.client.1.smithi191.stdout:3/457: dwrite d6/d3b/d30/f7a [0,4194304] 0 2024-02-15T18:27:10.985 INFO:tasks.workunit.client.1.smithi191.stdout:5/321: dread d0/d23/d41/d32/d52/f40 [0,4194304] 0 2024-02-15T18:27:10.987 INFO:tasks.workunit.client.1.smithi191.stdout:8/677: rename d3/d2c/d1a/f44 to d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/ff2 0 2024-02-15T18:27:10.987 INFO:tasks.workunit.client.1.smithi191.stdout:8/678: chown d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/cef 873920 1 2024-02-15T18:27:10.990 INFO:tasks.workunit.client.1.smithi191.stdout:3/458: mkdir d6/d1b/d60/da2 0 2024-02-15T18:27:10.990 INFO:tasks.workunit.client.1.smithi191.stdout:8/679: creat d3/d2c/d1a/d23/d2e/d47/d5f/ff3 x:0 0 0 2024-02-15T18:27:10.991 INFO:tasks.workunit.client.1.smithi191.stdout:3/459: fdatasync d6/d3b/d42/f31 0 2024-02-15T18:27:10.991 INFO:tasks.workunit.client.1.smithi191.stdout:3/460: readlink d6/d1b/d60/d73/d5e/l8c 0 2024-02-15T18:27:10.991 INFO:tasks.workunit.client.1.smithi191.stdout:3/461: rmdir d6/d1b 39 2024-02-15T18:27:10.995 INFO:tasks.workunit.client.0.smithi042.stdout:2/979: dwrite d0/d8/d20/d3f/d7c/ffc [4194304,4194304] 0 2024-02-15T18:27:10.995 INFO:tasks.workunit.client.1.smithi191.stdout:3/462: unlink d6/d3b/c19 0 2024-02-15T18:27:10.995 INFO:tasks.workunit.client.1.smithi191.stdout:3/463: chown d6/d3b/d30/c79 287493497 1 2024-02-15T18:27:10.996 INFO:tasks.workunit.client.0.smithi042.stdout:8/930: sync 2024-02-15T18:27:10.996 INFO:tasks.workunit.client.0.smithi042.stdout:8/931: write dd/de/d33/dbf/fc7 [503957,126666] 0 2024-02-15T18:27:10.997 INFO:tasks.workunit.client.1.smithi191.stdout:9/569: dwrite d0/d7/dc/d30/f5c [0,4194304] 0 2024-02-15T18:27:10.997 INFO:tasks.workunit.client.1.smithi191.stdout:9/570: chown d0/d7/dc/d30/d3b/d56/fb5 20266 1 2024-02-15T18:27:10.997 INFO:tasks.workunit.client.1.smithi191.stdout:2/537: dwrite df/d12/d32/fa9 [0,4194304] 0 2024-02-15T18:27:10.997 INFO:tasks.workunit.client.0.smithi042.stdout:2/980: creat d0/d8/d20/d7d/d7e/dc7/d8b/f13a x:0 0 0 2024-02-15T18:27:10.997 INFO:tasks.workunit.client.0.smithi042.stdout:2/981: write d0/d8/d10/d5a/f90 [1396079,39435] 0 2024-02-15T18:27:10.997 INFO:tasks.workunit.client.0.smithi042.stdout:2/982: write d0/d8/d10/d31/d79/f8e [496765,45496] 0 2024-02-15T18:27:10.998 INFO:tasks.workunit.client.1.smithi191.stdout:7/543: dwrite d0/db/d26/d5f/d77/d1d/d54/fbe [0,4194304] 0 2024-02-15T18:27:10.998 INFO:tasks.workunit.client.1.smithi191.stdout:9/571: read d0/d7/dc/d32/da3/fb9 [2001449,70398] 0 2024-02-15T18:27:10.998 INFO:tasks.workunit.client.1.smithi191.stdout:9/572: chown d0/d7/dc/d32/da3/cc1 55 1 2024-02-15T18:27:10.999 INFO:tasks.workunit.client.1.smithi191.stdout:9/573: chown d0/d29/d48/d66/d92/c8a 28515 1 2024-02-15T18:27:10.999 INFO:tasks.workunit.client.1.smithi191.stdout:3/464: dread d6/d3b/d42/d49/f6d [0,4194304] 0 2024-02-15T18:27:10.999 INFO:tasks.workunit.client.1.smithi191.stdout:3/465: fsync d6/d3b/d42/d49/d54/f6b 0 2024-02-15T18:27:10.999 INFO:tasks.workunit.client.1.smithi191.stdout:3/466: readlink d6/d27/l4f 0 2024-02-15T18:27:10.999 INFO:tasks.workunit.client.1.smithi191.stdout:3/467: fdatasync d6/f65 0 2024-02-15T18:27:10.999 INFO:tasks.workunit.client.1.smithi191.stdout:3/468: write d6/d1b/d60/d73/f9f [845620,95024] 0 2024-02-15T18:27:10.999 INFO:tasks.workunit.client.0.smithi042.stdout:8/932: rename dd/f7d to dd/de/d33/f10f 0 2024-02-15T18:27:11.000 INFO:tasks.workunit.client.1.smithi191.stdout:0/379: dwrite d7/d3b/f69 [0,4194304] 0 2024-02-15T18:27:11.000 INFO:tasks.workunit.client.1.smithi191.stdout:3/469: write d6/d3b/f84 [4780071,4335] 0 2024-02-15T18:27:11.000 INFO:tasks.workunit.client.1.smithi191.stdout:0/380: write d7/da/d1a/d2c/f4b [776579,21967] 0 2024-02-15T18:27:11.004 INFO:tasks.workunit.client.0.smithi042.stdout:2/983: creat d0/d8/d20/d7d/d7e/dc7/d8b/dd7/f13b x:0 0 0 2024-02-15T18:27:11.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.008+0000 7fc857353700 1 -- 172.21.15.42:0/805238366 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc830000bc0 con 0x7fc83c060fb0 2024-02-15T18:27:11.006 INFO:tasks.workunit.client.0.smithi042.stdout:8/933: unlink dd/de/dc0/fed 0 2024-02-15T18:27:11.008 INFO:tasks.workunit.client.0.smithi042.stdout:2/984: rename d0/d8/d20/d41/f85 to d0/f13c 0 2024-02-15T18:27:11.012 INFO:tasks.workunit.client.0.smithi042.stdout:8/934: symlink dd/de/d33/d66/d4e/l110 0 2024-02-15T18:27:11.012 INFO:tasks.workunit.client.1.smithi191.stdout:2/538: mknod df/d12/d19/d1a/d52/cb7 0 2024-02-15T18:27:11.014 INFO:tasks.workunit.client.0.smithi042.stdout:8/935: mknod dd/de/da4/d67/d6f/d78/d9a/dce/c111 0 2024-02-15T18:27:11.015 INFO:tasks.workunit.client.0.smithi042.stdout:8/936: read dd/de/d12/f1e [1981073,17873] 0 2024-02-15T18:27:11.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.021+0000 7fc845ffb700 1 -- 172.21.15.42:0/805238366 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fc830000bc0 con 0x7fc83c060fb0 2024-02-15T18:27:11.017 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:27:11.017 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (6m) 11s ago 8m 20.6M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:27:11.017 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (8m) 11s ago 8m 6400k - 16.2.4 8d91d370c2b8 94d74c33c758 2024-02-15T18:27:11.017 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (7m) 33s ago 7m 6405k - 16.2.4 8d91d370c2b8 ff0f56f9475a 2024-02-15T18:27:11.017 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (6m) 11s ago 8m 33.7M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:27:11.017 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (3m) 11s ago 3m 13.9M - 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:27:11.018 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (3m) 11s ago 3m 16.6M - 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:27:11.018 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (3m) 33s ago 3m 75.1M - 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:27:11.018 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (3m) 33s ago 3m 143M - 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:27:11.018 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (69s) 11s ago 10m 425M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:27:11.018 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (46s) 33s ago 7m 40.5M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:27:11.018 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (25s) 11s ago 10m 23.5M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:27:11.018 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (7m) 33s ago 7m 49.5M 2048M 16.2.4 8d91d370c2b8 c3dd4902e2a2 2024-02-15T18:27:11.018 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (8m) 11s ago 8m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:27:11.018 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (7m) 33s ago 7m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:27:11.018 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (6m) 11s ago 6m 146M 4096M 16.2.4 8d91d370c2b8 973745b31d5a 2024-02-15T18:27:11.018 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (6m) 11s ago 6m 161M 4096M 16.2.4 8d91d370c2b8 2b3ae86111e8 2024-02-15T18:27:11.018 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (6m) 11s ago 6m 144M 4096M 16.2.4 8d91d370c2b8 3706071e4698 2024-02-15T18:27:11.018 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (5m) 33s ago 5m 79.8M 4096M 16.2.4 8d91d370c2b8 e6fe8e77c6fe 2024-02-15T18:27:11.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (5m) 33s ago 5m 80.6M 4096M 16.2.4 8d91d370c2b8 24582012e234 2024-02-15T18:27:11.019 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (5m) 33s ago 5m 74.7M 4096M 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:27:11.019 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (76s) 11s ago 7m 48.2M - 2.18.1 de242295e225 63b6dfb825ed 2024-02-15T18:27:11.019 INFO:tasks.workunit.client.1.smithi191.stdout:3/470: dwrite d6/d27/d4d/d5a/f89 [0,4194304] 0 2024-02-15T18:27:11.020 INFO:tasks.workunit.client.0.smithi042.stdout:8/937: mknod dd/de/da5/c112 0 2024-02-15T18:27:11.020 INFO:tasks.workunit.client.0.smithi042.stdout:8/938: fdatasync dd/de/d12/fef 0 2024-02-15T18:27:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.023+0000 7fc857353700 1 -- 172.21.15.42:0/805238366 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc83c060fb0 msgr2=0x7fc83c063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.023+0000 7fc857353700 1 --2- 172.21.15.42:0/805238366 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc83c060fb0 0x7fc83c063470 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fc84c004630 tx=0x7fc84c009b30).stop 2024-02-15T18:27:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.023+0000 7fc857353700 1 -- 172.21.15.42:0/805238366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8500f5ef0 msgr2=0x7fc85010db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.023+0000 7fc857353700 1 --2- 172.21.15.42:0/805238366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8500f5ef0 0x7fc85010db70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fc84000b410 tx=0x7fc840006ef0).stop 2024-02-15T18:27:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.023+0000 7fc857353700 1 -- 172.21.15.42:0/805238366 shutdown_connections 2024-02-15T18:27:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.023+0000 7fc857353700 1 --2- 172.21.15.42:0/805238366 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc83c060fb0 0x7fc83c063470 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.023+0000 7fc857353700 1 --2- 172.21.15.42:0/805238366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8500f55e0 0x7fc85010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.023+0000 7fc857353700 1 --2- 172.21.15.42:0/805238366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8500f5ef0 0x7fc85010db70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.023+0000 7fc857353700 1 -- 172.21.15.42:0/805238366 >> 172.21.15.42:0/805238366 conn(0x7fc8500f0de0 msgr2=0x7fc85005ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.023+0000 7fc857353700 1 -- 172.21.15.42:0/805238366 shutdown_connections 2024-02-15T18:27:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.023+0000 7fc857353700 1 -- 172.21.15.42:0/805238366 wait complete. 2024-02-15T18:27:11.024 INFO:tasks.workunit.client.1.smithi191.stdout:7/544: creat d0/db/d26/d5f/d77/fcd x:0 0 0 2024-02-15T18:27:11.026 INFO:tasks.workunit.client.1.smithi191.stdout:9/574: unlink d0/f51 0 2024-02-15T18:27:11.026 INFO:tasks.workunit.client.1.smithi191.stdout:9/575: truncate d0/d7/dc/d30/d4f/fb0 972989 0 2024-02-15T18:27:11.028 INFO:tasks.workunit.client.1.smithi191.stdout:0/381: link d7/da/d1a/d34/d27/l64 d7/da/d1a/l83 0 2024-02-15T18:27:11.028 INFO:tasks.workunit.client.1.smithi191.stdout:0/382: write d7/da/d1a/d34/f52 [748776,63111] 0 2024-02-15T18:27:11.035 INFO:tasks.workunit.client.1.smithi191.stdout:2/539: unlink df/d61/d9f/da3/fb6 0 2024-02-15T18:27:11.035 INFO:tasks.workunit.client.1.smithi191.stdout:2/540: chown df/d12/d19/d1a/c23 411089 1 2024-02-15T18:27:11.036 INFO:tasks.workunit.client.1.smithi191.stdout:7/545: dread d0/db/d26/d5f/d77/d35/f48 [0,4194304] 0 2024-02-15T18:27:11.036 INFO:tasks.workunit.client.1.smithi191.stdout:7/546: write d0/db/d26/f3c [1356480,91891] 0 2024-02-15T18:27:11.041 INFO:tasks.workunit.client.1.smithi191.stdout:3/471: getdents d6/d1b 0 2024-02-15T18:27:11.041 INFO:tasks.workunit.client.1.smithi191.stdout:0/383: symlink d7/da/d1a/l84 0 2024-02-15T18:27:11.043 INFO:tasks.workunit.client.1.smithi191.stdout:0/384: write d7/da/d17/d31/f55 [2649942,66738] 0 2024-02-15T18:27:11.043 INFO:tasks.workunit.client.1.smithi191.stdout:0/385: truncate d7/d1b/f81 120096 0 2024-02-15T18:27:11.043 INFO:tasks.workunit.client.1.smithi191.stdout:0/386: chown d7/da/d17/l76 30173987 1 2024-02-15T18:27:11.043 INFO:tasks.workunit.client.1.smithi191.stdout:0/387: chown d7/d1b/f46 173839 1 2024-02-15T18:27:11.044 INFO:tasks.workunit.client.1.smithi191.stdout:2/541: link df/d12/d1f/d42/ca4 df/d12/d1e/d6f/cb8 0 2024-02-15T18:27:11.045 INFO:tasks.workunit.client.1.smithi191.stdout:2/542: write df/d12/f2b [3002942,66582] 0 2024-02-15T18:27:11.045 INFO:tasks.workunit.client.1.smithi191.stdout:7/547: mknod d0/db/d26/d5f/d77/d35/dc7/cce 0 2024-02-15T18:27:11.045 INFO:tasks.workunit.client.1.smithi191.stdout:7/548: fsync d0/db/d26/d5f/d77/df/f1a 0 2024-02-15T18:27:11.046 INFO:tasks.workunit.client.1.smithi191.stdout:3/472: getdents d6/d27 0 2024-02-15T18:27:11.046 INFO:tasks.workunit.client.1.smithi191.stdout:3/473: chown d6 6964 1 2024-02-15T18:27:11.059 INFO:tasks.workunit.client.1.smithi191.stdout:2/543: dwrite df/d12/d32/f8f [0,4194304] 0 2024-02-15T18:27:11.076 INFO:tasks.workunit.client.1.smithi191.stdout:2/544: dwrite df/d12/d32/d4d/d89/f95 [0,4194304] 0 2024-02-15T18:27:11.076 INFO:tasks.workunit.client.1.smithi191.stdout:2/545: stat df/d12/d32/d4d 0 2024-02-15T18:27:11.076 INFO:tasks.workunit.client.1.smithi191.stdout:2/546: write df/d12/d19/d1a/f2c [489137,121915] 0 2024-02-15T18:27:11.089 INFO:tasks.workunit.client.0.smithi042.stdout:2/985: sync 2024-02-15T18:27:11.089 INFO:tasks.workunit.client.0.smithi042.stdout:2/986: getdents d0/d8/d20/d3f/dd6/df3/df6 0 2024-02-15T18:27:11.090 INFO:tasks.workunit.client.0.smithi042.stdout:2/987: write d0/d8/f65 [5007788,115868] 0 2024-02-15T18:27:11.097 INFO:tasks.workunit.client.1.smithi191.stdout:6/518: sync 2024-02-15T18:27:11.107 INFO:tasks.workunit.client.1.smithi191.stdout:0/388: creat d7/d4a/f85 x:0 0 0 2024-02-15T18:27:11.109 INFO:tasks.workunit.client.1.smithi191.stdout:0/389: dread d7/f39 [0,4194304] 0 2024-02-15T18:27:11.110 INFO:tasks.workunit.client.1.smithi191.stdout:0/390: write d7/da/d1a/f58 [251543,106864] 0 2024-02-15T18:27:11.112 INFO:tasks.workunit.client.1.smithi191.stdout:7/549: rename d0/db/d26/f85 to d0/db/d65/d11/d57/fcf 0 2024-02-15T18:27:11.112 INFO:tasks.workunit.client.1.smithi191.stdout:3/474: creat d6/d1b/fa3 x:0 0 0 2024-02-15T18:27:11.113 INFO:tasks.workunit.client.1.smithi191.stdout:2/547: mknod df/d12/d1f/d42/cb9 0 2024-02-15T18:27:11.115 INFO:tasks.workunit.client.1.smithi191.stdout:6/519: symlink d8/d9/d4a/d4c/d7f/l99 0 2024-02-15T18:27:11.116 INFO:tasks.workunit.client.1.smithi191.stdout:6/520: read d8/d9/f65 [75859,77991] 0 2024-02-15T18:27:11.119 INFO:tasks.workunit.client.1.smithi191.stdout:0/391: creat d7/da/d1a/d2c/f86 x:0 0 0 2024-02-15T18:27:11.120 INFO:tasks.workunit.client.1.smithi191.stdout:3/475: unlink d6/f65 0 2024-02-15T18:27:11.121 INFO:tasks.workunit.client.1.smithi191.stdout:6/521: creat d8/d9/d4a/f9a x:0 0 0 2024-02-15T18:27:11.121 INFO:tasks.workunit.client.1.smithi191.stdout:2/548: dread df/d12/d1f/d42/f43 [0,4194304] 0 2024-02-15T18:27:11.122 INFO:tasks.workunit.client.1.smithi191.stdout:0/392: creat d7/da/d17/d31/f87 x:0 0 0 2024-02-15T18:27:11.122 INFO:tasks.workunit.client.1.smithi191.stdout:3/476: mkdir d6/d27/d4d/d5a/da4 0 2024-02-15T18:27:11.123 INFO:tasks.workunit.client.1.smithi191.stdout:6/522: dread d8/d9/f65 [0,4194304] 0 2024-02-15T18:27:11.132 INFO:tasks.workunit.client.1.smithi191.stdout:0/393: rmdir d7/da/d17/d31/d5f/d82 0 2024-02-15T18:27:11.134 INFO:tasks.workunit.client.1.smithi191.stdout:3/477: dread d6/d3b/d42/d49/f59 [0,4194304] 0 2024-02-15T18:27:11.135 INFO:tasks.workunit.client.1.smithi191.stdout:3/478: creat d6/d27/fa5 x:0 0 0 2024-02-15T18:27:11.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.142+0000 7f729d380700 1 -- 172.21.15.42:0/2910265795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729806b9c0 msgr2=0x7f7298069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:11.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.142+0000 7f729d380700 1 --2- 172.21.15.42:0/2910265795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729806b9c0 0x7f7298069ec0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f7288004660 tx=0x7f7288009b30).stop 2024-02-15T18:27:11.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.142+0000 7f729d380700 1 -- 172.21.15.42:0/2910265795 shutdown_connections 2024-02-15T18:27:11.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.142+0000 7f729d380700 1 --2- 172.21.15.42:0/2910265795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f729806a3b0 0x7f729806a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.142+0000 7f729d380700 1 --2- 172.21.15.42:0/2910265795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729806b9c0 0x7f7298069ec0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.142+0000 7f729d380700 1 -- 172.21.15.42:0/2910265795 >> 172.21.15.42:0/2910265795 conn(0x7f72980f1e10 msgr2=0x7f72980f4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:11.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.142+0000 7f729d380700 1 -- 172.21.15.42:0/2910265795 shutdown_connections 2024-02-15T18:27:11.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.142+0000 7f729d380700 1 -- 172.21.15.42:0/2910265795 wait complete. 2024-02-15T18:27:11.139 INFO:tasks.workunit.client.1.smithi191.stdout:3/479: rename d6/d27/d4d/d5a/f89 to d6/d3b/d42/d49/d54/fa6 0 2024-02-15T18:27:11.139 INFO:tasks.workunit.client.1.smithi191.stdout:3/480: chown d6/d27 215 1 2024-02-15T18:27:11.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.143+0000 7f729d380700 1 Processor -- start 2024-02-15T18:27:11.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.143+0000 7f729d380700 1 -- start start 2024-02-15T18:27:11.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.143+0000 7f729d380700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f729806a3b0 0x7f729810d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:11.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.143+0000 7f729d380700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729806b9c0 0x7f729810dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:11.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.143+0000 7f729d380700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729810e060 con 0x7f729806b9c0 2024-02-15T18:27:11.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.143+0000 7f729d380700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729810e1a0 con 0x7f729806a3b0 2024-02-15T18:27:11.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.143+0000 7f72967fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729806b9c0 0x7f729810dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:11.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.143+0000 7f72967fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729806b9c0 0x7f729810dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34236/0 (socket says 172.21.15.42:34236) 2024-02-15T18:27:11.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.143+0000 7f72967fc700 1 -- 172.21.15.42:0/598158119 learned_addr learned my addr 172.21.15.42:0/598158119 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:27:11.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.143+0000 7f7296ffd700 1 --2- 172.21.15.42:0/598158119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f729806a3b0 0x7f729810d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:11.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.144+0000 7f72967fc700 1 -- 172.21.15.42:0/598158119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f729806a3b0 msgr2=0x7f729810d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:11.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.144+0000 7f72967fc700 1 --2- 172.21.15.42:0/598158119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f729806a3b0 0x7f729810d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.144+0000 7f72967fc700 1 -- 172.21.15.42:0/598158119 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7288005040 con 0x7f729806b9c0 2024-02-15T18:27:11.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.144+0000 7f72967fc700 1 --2- 172.21.15.42:0/598158119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729806b9c0 0x7f729810dab0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f728c00e7f0 tx=0x7f728c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:11.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.144+0000 7f727f7fe700 1 -- 172.21.15.42:0/598158119 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f728c0092e0 con 0x7f729806b9c0 2024-02-15T18:27:11.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.144+0000 7f727f7fe700 1 -- 172.21.15.42:0/598158119 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f728c00f040 con 0x7f729806b9c0 2024-02-15T18:27:11.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.144+0000 7f729d380700 1 -- 172.21.15.42:0/598158119 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729810eb40 con 0x7f729806b9c0 2024-02-15T18:27:11.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.144+0000 7f727f7fe700 1 -- 172.21.15.42:0/598158119 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f728c014850 con 0x7f729806b9c0 2024-02-15T18:27:11.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.144+0000 7f729d380700 1 -- 172.21.15.42:0/598158119 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729810f100 con 0x7f729806b9c0 2024-02-15T18:27:11.143 INFO:tasks.workunit.client.1.smithi191.stdout:3/481: creat d6/d3b/fa7 x:0 0 0 2024-02-15T18:27:11.143 INFO:tasks.workunit.client.1.smithi191.stdout:3/482: mkdir d6/d27/d4d/da8 0 2024-02-15T18:27:11.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.145+0000 7f729d380700 1 -- 172.21.15.42:0/598158119 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7298045ad0 con 0x7f729806b9c0 2024-02-15T18:27:11.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.146+0000 7f727f7fe700 1 -- 172.21.15.42:0/598158119 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7f728c01b030 con 0x7f729806b9c0 2024-02-15T18:27:11.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.146+0000 7f727f7fe700 1 --2- 172.21.15.42:0/598158119 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f7280060fb0 0x7f7280063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:11.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.146+0000 7f727f7fe700 1 -- 172.21.15.42:0/598158119 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f728c07bf20 con 0x7f729806b9c0 2024-02-15T18:27:11.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.146+0000 7f7296ffd700 1 --2- 172.21.15.42:0/598158119 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f7280060fb0 0x7f7280063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:11.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.147+0000 7f7296ffd700 1 --2- 172.21.15.42:0/598158119 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f7280060fb0 0x7f7280063470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f7288009720 tx=0x7f7288016040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:11.145 INFO:tasks.workunit.client.0.smithi042.stdout:8/939: sync 2024-02-15T18:27:11.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.151+0000 7f727f7fe700 1 -- 172.21.15.42:0/598158119 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f728c0aaf60 con 0x7f729806b9c0 2024-02-15T18:27:11.149 INFO:tasks.workunit.client.0.smithi042.stdout:8/940: dread dd/de/fe0 [0,4194304] 0 2024-02-15T18:27:11.149 INFO:tasks.workunit.client.0.smithi042.stdout:8/941: truncate dd/de/d12/f1e 493528 0 2024-02-15T18:27:11.151 INFO:tasks.workunit.client.1.smithi191.stdout:2/549: dwrite df/d12/f2b [0,4194304] 0 2024-02-15T18:27:11.151 INFO:tasks.workunit.client.0.smithi042.stdout:8/942: mkdir dd/de/da4/d81/d113 0 2024-02-15T18:27:11.152 INFO:tasks.workunit.client.1.smithi191.stdout:7/550: dwrite d0/db/d65/dc/d4b/f59 [0,4194304] 0 2024-02-15T18:27:11.152 INFO:tasks.workunit.client.1.smithi191.stdout:6/523: dwrite d8/d19/f40 [4194304,4194304] 0 2024-02-15T18:27:11.153 INFO:tasks.workunit.client.0.smithi042.stdout:8/943: mkdir dd/de/da4/d67/d5a/db2/d114 0 2024-02-15T18:27:11.153 INFO:tasks.workunit.client.1.smithi191.stdout:7/551: read d0/db/d26/d5f/d77/d35/d42/f6c [3837346,61379] 0 2024-02-15T18:27:11.153 INFO:tasks.workunit.client.1.smithi191.stdout:7/552: dread - d0/db/d65/dc/d4b/f5b zero size 2024-02-15T18:27:11.156 INFO:tasks.workunit.client.0.smithi042.stdout:8/944: creat dd/de/da4/d67/d5a/db2/d114/f115 x:0 0 0 2024-02-15T18:27:11.157 INFO:tasks.workunit.client.0.smithi042.stdout:8/945: chown dd/de/d12/l72 0 1 2024-02-15T18:27:11.157 INFO:tasks.workunit.client.1.smithi191.stdout:6/524: mknod d8/d9/d4a/c9b 0 2024-02-15T18:27:11.157 INFO:tasks.workunit.client.1.smithi191.stdout:6/525: write d8/d19/f7a [1152296,68890] 0 2024-02-15T18:27:11.159 INFO:tasks.workunit.client.1.smithi191.stdout:7/553: link d0/db/d65/d11/d1c/fb3 d0/db/d65/dc/d6e/fd0 0 2024-02-15T18:27:11.159 INFO:tasks.workunit.client.0.smithi042.stdout:8/946: truncate dd/de/f1a 240719 0 2024-02-15T18:27:11.162 INFO:tasks.workunit.client.1.smithi191.stdout:7/554: mknod d0/db/d65/cd1 0 2024-02-15T18:27:11.164 INFO:tasks.workunit.client.1.smithi191.stdout:2/550: dread df/d12/f5d [0,4194304] 0 2024-02-15T18:27:11.164 INFO:tasks.workunit.client.1.smithi191.stdout:2/551: fsync df/d12/d19/d1a/fae 0 2024-02-15T18:27:11.164 INFO:tasks.workunit.client.1.smithi191.stdout:2/552: creat df/d12/d1f/d42/fba x:0 0 0 2024-02-15T18:27:11.164 INFO:tasks.workunit.client.1.smithi191.stdout:2/553: truncate df/d12/d19/d1a/d35/fa0 628458 0 2024-02-15T18:27:11.167 INFO:tasks.workunit.client.1.smithi191.stdout:2/554: dread df/d12/d19/d1a/f5a [0,4194304] 0 2024-02-15T18:27:11.168 INFO:tasks.workunit.client.1.smithi191.stdout:2/555: creat df/d12/d1e/fbb x:0 0 0 2024-02-15T18:27:11.171 INFO:tasks.workunit.client.1.smithi191.stdout:2/556: rmdir df/d61/d9f/da3 39 2024-02-15T18:27:11.176 INFO:tasks.workunit.client.1.smithi191.stdout:2/557: unlink df/d12/d19/d1a/d35/d47/c48 0 2024-02-15T18:27:11.179 INFO:tasks.workunit.client.1.smithi191.stdout:2/558: creat df/d12/d19/d1a/d35/d47/da2/fbc x:0 0 0 2024-02-15T18:27:11.179 INFO:tasks.workunit.client.1.smithi191.stdout:2/559: dread - df/d12/d19/d1a/d35/f74 zero size 2024-02-15T18:27:11.181 INFO:tasks.workunit.client.1.smithi191.stdout:2/560: creat df/d12/fbd x:0 0 0 2024-02-15T18:27:11.182 INFO:tasks.workunit.client.1.smithi191.stdout:2/561: mknod df/d12/d1f/d42/db5/cbe 0 2024-02-15T18:27:11.196 INFO:tasks.workunit.client.0.smithi042.stdout:8/947: dwrite dd/de/da4/d67/d6f/fbe [0,4194304] 0 2024-02-15T18:27:11.196 INFO:tasks.workunit.client.0.smithi042.stdout:8/948: read dd/de/d3c/f54 [3651795,78199] 0 2024-02-15T18:27:11.196 INFO:tasks.workunit.client.0.smithi042.stdout:8/949: fsync dd/d11/d75/f9d 0 2024-02-15T18:27:11.196 INFO:tasks.workunit.client.0.smithi042.stdout:2/988: sync 2024-02-15T18:27:11.198 INFO:tasks.workunit.client.0.smithi042.stdout:2/989: creat d0/d35/f13d x:0 0 0 2024-02-15T18:27:11.198 INFO:tasks.workunit.client.1.smithi191.stdout:6/526: dwrite d8/d9/d2b/f42 [0,4194304] 0 2024-02-15T18:27:11.199 INFO:tasks.workunit.client.1.smithi191.stdout:7/555: dwrite d0/db/d65/d11/d5a/d66/d1e/f8a [4194304,4194304] 0 2024-02-15T18:27:11.200 INFO:tasks.workunit.client.1.smithi191.stdout:2/562: dwrite df/d12/f5d [0,4194304] 0 2024-02-15T18:27:11.200 INFO:tasks.workunit.client.1.smithi191.stdout:2/563: readlink df/d12/d1f/l69 0 2024-02-15T18:27:11.203 INFO:tasks.workunit.client.1.smithi191.stdout:7/556: truncate d0/db/d65/d11/d1c/f45 709089 0 2024-02-15T18:27:11.203 INFO:tasks.workunit.client.1.smithi191.stdout:6/527: dread d8/d9/f46 [0,4194304] 0 2024-02-15T18:27:11.206 INFO:tasks.workunit.client.1.smithi191.stdout:2/564: rename df/d12/d1f/d9c to df/d12/d19/d1a/d35/d5b/dbf 0 2024-02-15T18:27:11.206 INFO:tasks.workunit.client.1.smithi191.stdout:2/565: readlink df/d12/d1f/l56 0 2024-02-15T18:27:11.210 INFO:tasks.workunit.client.1.smithi191.stdout:7/557: rename d0/db/d26/d5f/d77/d1d/d54 to d0/db/d65/d11/d88/d9d/dd2 0 2024-02-15T18:27:11.212 INFO:tasks.workunit.client.1.smithi191.stdout:2/566: getdents df/d12/d19/d1a/d35 0 2024-02-15T18:27:11.213 INFO:tasks.workunit.client.1.smithi191.stdout:6/528: dread d8/f30 [0,4194304] 0 2024-02-15T18:27:11.215 INFO:tasks.workunit.client.1.smithi191.stdout:2/567: creat df/d12/d19/d1a/d52/fc0 x:0 0 0 2024-02-15T18:27:11.219 INFO:tasks.workunit.client.1.smithi191.stdout:6/529: rename d8/d9/d4a/l56 to d8/l9c 0 2024-02-15T18:27:11.220 INFO:tasks.workunit.client.1.smithi191.stdout:2/568: getdents df/d12/d19/d1a/d35/d47/d4b 0 2024-02-15T18:27:11.221 INFO:tasks.workunit.client.0.smithi042.stdout:2/990: dwrite d0/d8/d10/d31/d79/f8e [0,4194304] 0 2024-02-15T18:27:11.223 INFO:tasks.workunit.client.0.smithi042.stdout:8/950: dwrite dd/de/d12/ff2 [0,4194304] 0 2024-02-15T18:27:11.223 INFO:tasks.workunit.client.0.smithi042.stdout:8/951: fdatasync dd/de/fe0 0 2024-02-15T18:27:11.224 INFO:tasks.workunit.client.1.smithi191.stdout:6/530: rename d8/d19 to d8/d9/d61/d95/d9d 0 2024-02-15T18:27:11.224 INFO:tasks.workunit.client.1.smithi191.stdout:6/531: read d8/d9/f46 [636463,53760] 0 2024-02-15T18:27:11.225 INFO:tasks.workunit.client.1.smithi191.stdout:2/569: symlink df/d12/d19/d1a/d35/lc1 0 2024-02-15T18:27:11.225 INFO:tasks.workunit.client.1.smithi191.stdout:2/570: readlink df/d12/d1f/d42/l64 0 2024-02-15T18:27:11.228 INFO:tasks.workunit.client.1.smithi191.stdout:6/532: dread d8/d9/f46 [0,4194304] 0 2024-02-15T18:27:11.228 INFO:tasks.workunit.client.1.smithi191.stdout:6/533: write d8/f45 [286293,87771] 0 2024-02-15T18:27:11.228 INFO:tasks.workunit.client.1.smithi191.stdout:6/534: chown d8/d9/d2b/d81/l8f 78266 1 2024-02-15T18:27:11.229 INFO:tasks.workunit.client.0.smithi042.stdout:2/991: rename d0/d8/d20/d3f/d132/de/d1a/l1f to d0/d8/d20/d3f/dd6/df3/df6/l13e 0 2024-02-15T18:27:11.229 INFO:tasks.workunit.client.1.smithi191.stdout:7/558: dwrite d0/db/d65/dc/d4b/fb1 [0,4194304] 0 2024-02-15T18:27:11.229 INFO:tasks.workunit.client.1.smithi191.stdout:7/559: stat d0/db/d26/d5f/d77/df/f64 0 2024-02-15T18:27:11.230 INFO:tasks.workunit.client.1.smithi191.stdout:6/535: creat d8/d9/d4a/d4c/f9e x:0 0 0 2024-02-15T18:27:11.230 INFO:tasks.workunit.client.1.smithi191.stdout:6/536: truncate d8/d9/d4a/d4c/f5d 1728136 0 2024-02-15T18:27:11.230 INFO:tasks.workunit.client.1.smithi191.stdout:6/537: dread - d8/d9/d2b/f7b zero size 2024-02-15T18:27:11.231 INFO:tasks.workunit.client.0.smithi042.stdout:8/952: rename dd/de/d12/f62 to dd/de/d3c/d10c/f116 0 2024-02-15T18:27:11.233 INFO:tasks.workunit.client.0.smithi042.stdout:2/992: mknod d0/c13f 0 2024-02-15T18:27:11.234 INFO:tasks.workunit.client.1.smithi191.stdout:7/560: creat d0/db/d26/d5f/d77/df/fd3 x:0 0 0 2024-02-15T18:27:11.234 INFO:tasks.workunit.client.1.smithi191.stdout:7/561: chown d0/db/d65/dc/fa0 35 1 2024-02-15T18:27:11.235 INFO:tasks.workunit.client.0.smithi042.stdout:2/993: unlink d0/d8/d20/d7d/d7e/dc7/d8b/fb6 0 2024-02-15T18:27:11.241 INFO:tasks.workunit.client.1.smithi191.stdout:7/562: creat d0/db/d26/d5f/d77/d35/fd4 x:0 0 0 2024-02-15T18:27:11.241 INFO:tasks.workunit.client.1.smithi191.stdout:7/563: stat d0/db/d65/c4a 0 2024-02-15T18:27:11.242 INFO:tasks.workunit.client.0.smithi042.stdout:2/994: unlink d0/d8/d10/d31/d3a/d53/l33 0 2024-02-15T18:27:11.244 INFO:tasks.workunit.client.0.smithi042.stdout:2/995: rmdir d0/d40 39 2024-02-15T18:27:11.246 INFO:tasks.workunit.client.0.smithi042.stdout:2/996: rmdir d0/d8/d10/d31/da6 39 2024-02-15T18:27:11.246 INFO:tasks.workunit.client.0.smithi042.stdout:2/997: chown d0/d8/d10/d31/fc0 17279 1 2024-02-15T18:27:11.247 INFO:tasks.workunit.client.0.smithi042.stdout:2/998: creat d0/d8/d20/d41/df1/f140 x:0 0 0 2024-02-15T18:27:11.249 INFO:tasks.workunit.client.1.smithi191.stdout:7/564: creat d0/db/d26/d5f/fd5 x:0 0 0 2024-02-15T18:27:11.249 INFO:tasks.workunit.client.0.smithi042.stdout:8/953: dwrite dd/de/d12/f1e [0,4194304] 0 2024-02-15T18:27:11.251 INFO:tasks.workunit.client.1.smithi191.stdout:7/565: mkdir d0/db/d65/d11/d5a/d66/dd6 0 2024-02-15T18:27:11.251 INFO:tasks.workunit.client.1.smithi191.stdout:7/566: stat d0/db/d26/d5f/d77/d1d/fcc 0 2024-02-15T18:27:11.251 INFO:tasks.workunit.client.1.smithi191.stdout:7/567: getdents d0/db 0 2024-02-15T18:27:11.252 INFO:tasks.workunit.client.0.smithi042.stdout:8/954: link dd/de/d33/c105 dd/de/dc0/dfd/d107/c117 0 2024-02-15T18:27:11.258 INFO:tasks.workunit.client.1.smithi191.stdout:7/568: dread d0/f9e [0,4194304] 0 2024-02-15T18:27:11.260 INFO:tasks.workunit.client.1.smithi191.stdout:7/569: read d0/db/d26/d5f/d77/df/f1a [294032,28849] 0 2024-02-15T18:27:11.260 INFO:tasks.workunit.client.1.smithi191.stdout:7/570: truncate d0/db/d65/f82 1025415 0 2024-02-15T18:27:11.260 INFO:tasks.workunit.client.1.smithi191.stdout:7/571: fdatasync d0/db/d26/d5f/d77/f97 0 2024-02-15T18:27:11.260 INFO:tasks.workunit.client.1.smithi191.stdout:7/572: chown d0/db/d26/c2c 214623 1 2024-02-15T18:27:11.262 INFO:tasks.workunit.client.1.smithi191.stdout:7/573: dread d0/f80 [0,4194304] 0 2024-02-15T18:27:11.262 INFO:tasks.workunit.client.0.smithi042.stdout:2/999: dwrite d0/d8/d20/d3f/d132/f80 [4194304,4194304] 0 2024-02-15T18:27:11.356 INFO:tasks.workunit.client.0.smithi042.stdout:8/955: creat dd/de/dc0/dfd/d107/f118 x:0 0 0 2024-02-15T18:27:11.356 INFO:tasks.workunit.client.0.smithi042.stdout:8/956: write dd/de/da4/d67/d5a/db2/fc3 [140091,37978] 0 2024-02-15T18:27:11.357 INFO:tasks.workunit.client.1.smithi191.stdout:1/616: sync 2024-02-15T18:27:11.357 INFO:tasks.workunit.client.1.smithi191.stdout:4/591: sync 2024-02-15T18:27:11.357 INFO:tasks.workunit.client.1.smithi191.stdout:1/617: readlink d0/d8/df/d41/l62 0 2024-02-15T18:27:11.357 INFO:tasks.workunit.client.1.smithi191.stdout:1/618: getdents d0/d8/df/d90/d94/dc8 0 2024-02-15T18:27:11.357 INFO:tasks.workunit.client.1.smithi191.stdout:7/574: mkdir d0/db/d65/d11/dd7 0 2024-02-15T18:27:11.359 INFO:tasks.workunit.client.0.smithi042.stdout:8/957: truncate dd/f101 3833385 0 2024-02-15T18:27:11.360 INFO:tasks.workunit.client.1.smithi191.stdout:4/592: creat d2/fe2 x:0 0 0 2024-02-15T18:27:11.360 INFO:tasks.workunit.client.1.smithi191.stdout:1/619: symlink d0/d2f/d5c/lcd 0 2024-02-15T18:27:11.361 INFO:tasks.workunit.client.0.smithi042.stdout:8/958: getdents dd/de/d33 0 2024-02-15T18:27:11.361 INFO:tasks.workunit.client.0.smithi042.stdout:8/959: write dd/d11/d75/f9d [4873309,24364] 0 2024-02-15T18:27:11.362 INFO:tasks.workunit.client.1.smithi191.stdout:7/575: rename d0/db/d65/ce to d0/db/d65/dc/d4b/dc1/cd8 0 2024-02-15T18:27:11.362 INFO:tasks.workunit.client.1.smithi191.stdout:7/576: write d0/db/d26/d5f/d77/f6a [1576051,23343] 0 2024-02-15T18:27:11.363 INFO:tasks.workunit.client.1.smithi191.stdout:1/620: truncate d0/f9 1847008 0 2024-02-15T18:27:11.364 INFO:tasks.workunit.client.1.smithi191.stdout:4/593: rename d2/d5/d7/d3e/d88 to d2/d5/d8b/de3 0 2024-02-15T18:27:11.368 INFO:tasks.workunit.client.0.smithi042.stdout:8/960: mknod dd/c119 0 2024-02-15T18:27:11.370 INFO:tasks.workunit.client.1.smithi191.stdout:4/594: dread d2/d5/d7/f29 [0,4194304] 0 2024-02-15T18:27:11.371 INFO:tasks.workunit.client.1.smithi191.stdout:7/577: link d0/db/d26/d5f/d77/fcd d0/db/d65/d11/d57/fd9 0 2024-02-15T18:27:11.371 INFO:tasks.workunit.client.1.smithi191.stdout:7/578: stat d0/db/d65/d11/d5a/d66/f53 0 2024-02-15T18:27:11.373 INFO:tasks.workunit.client.1.smithi191.stdout:1/621: creat d0/d2f/d51/d5e/d68/fce x:0 0 0 2024-02-15T18:27:11.373 INFO:tasks.workunit.client.1.smithi191.stdout:1/622: truncate d0/d8/df/d2e/f7f 377234 0 2024-02-15T18:27:11.375 INFO:tasks.workunit.client.1.smithi191.stdout:7/579: rename d0/db/d26/fb4 to d0/db/d65/d11/d88/d9d/fda 0 2024-02-15T18:27:11.375 INFO:tasks.workunit.client.0.smithi042.stdout:8/961: dread dd/de/da4/d81/f8c [0,4194304] 0 2024-02-15T18:27:11.377 INFO:tasks.workunit.client.1.smithi191.stdout:1/623: creat d0/d2f/d5c/fcf x:0 0 0 2024-02-15T18:27:11.377 INFO:tasks.workunit.client.1.smithi191.stdout:1/624: chown d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/dc9 1186308973 1 2024-02-15T18:27:11.378 INFO:tasks.workunit.client.1.smithi191.stdout:7/580: getdents d0/db/d26/d5f/d77/d35/d42 0 2024-02-15T18:27:11.379 INFO:tasks.workunit.client.1.smithi191.stdout:1/625: unlink d0/d2f/d51/d10/f2d 0 2024-02-15T18:27:11.379 INFO:tasks.workunit.client.1.smithi191.stdout:1/626: chown d0/d8/df/d12/d2a/d2b/d78/d60/fbe 850 1 2024-02-15T18:27:11.379 INFO:tasks.workunit.client.1.smithi191.stdout:1/627: chown d0/d2f/d51/d10/fb4 0 1 2024-02-15T18:27:11.380 INFO:tasks.workunit.client.1.smithi191.stdout:7/581: symlink d0/db/d65/d11/d1c/ldb 0 2024-02-15T18:27:11.384 INFO:tasks.workunit.client.1.smithi191.stdout:1/628: write d0/d8/df/d12/f3f [2302342,74595] 0 2024-02-15T18:27:11.384 INFO:tasks.workunit.client.1.smithi191.stdout:1/629: chown d0/d2f/d51/d10/f5b 19 1 2024-02-15T18:27:11.385 INFO:tasks.workunit.client.1.smithi191.stdout:1/630: creat d0/d8/df/d12/d2a/fd0 x:0 0 0 2024-02-15T18:27:11.385 INFO:tasks.workunit.client.1.smithi191.stdout:1/631: readlink d0/d8/df/d41/l62 0 2024-02-15T18:27:11.387 INFO:tasks.workunit.client.1.smithi191.stdout:4/595: dwrite d2/d5/d14/d71/fac [0,4194304] 0 2024-02-15T18:27:11.390 INFO:tasks.workunit.client.1.smithi191.stdout:7/582: getdents d0/db/d26 0 2024-02-15T18:27:11.390 INFO:tasks.workunit.client.1.smithi191.stdout:7/583: truncate d0/db/d26/d5f/d77/f97 978478 0 2024-02-15T18:27:11.390 INFO:tasks.workunit.client.0.smithi042.stdout:8/962: dread dd/f17 [0,4194304] 0 2024-02-15T18:27:11.394 INFO:tasks.workunit.client.1.smithi191.stdout:4/596: mkdir d2/d5/d14/d47/d78/dd0/dd3/de4 0 2024-02-15T18:27:11.396 INFO:tasks.workunit.client.1.smithi191.stdout:7/584: rename d0/db/d26/d5f/c76 to d0/db/cdc 0 2024-02-15T18:27:11.400 INFO:tasks.workunit.client.1.smithi191.stdout:7/585: write d0/db/d26/d5f/d77/df/f17 [775264,57492] 0 2024-02-15T18:27:11.416 INFO:tasks.workunit.client.1.smithi191.stdout:8/680: sync 2024-02-15T18:27:11.416 INFO:tasks.workunit.client.1.smithi191.stdout:9/576: sync 2024-02-15T18:27:11.416 INFO:tasks.workunit.client.1.smithi191.stdout:5/322: sync 2024-02-15T18:27:11.416 INFO:tasks.workunit.client.1.smithi191.stdout:1/632: dwrite d0/d2f/d77/faf [0,4194304] 0 2024-02-15T18:27:11.416 INFO:tasks.workunit.client.1.smithi191.stdout:9/577: fsync d0/d7/dc/d17/d33/f43 0 2024-02-15T18:27:11.417 INFO:tasks.workunit.client.1.smithi191.stdout:9/578: write d0/d7/dc/d30/d3b/f55 [2329335,12750] 0 2024-02-15T18:27:11.417 INFO:tasks.workunit.client.1.smithi191.stdout:9/579: chown d0/d29/d48/d66 925 1 2024-02-15T18:27:11.417 INFO:tasks.workunit.client.0.smithi042.stdout:8/963: dwrite dd/de/da4/d67/f9f [0,4194304] 0 2024-02-15T18:27:11.417 INFO:tasks.workunit.client.0.smithi042.stdout:8/964: chown dd/de/da4/dd0 11023305 1 2024-02-15T18:27:11.420 INFO:tasks.workunit.client.1.smithi191.stdout:8/681: getdents d3/d2c/d1a/d23/d2e/d5b/d69/d6d 0 2024-02-15T18:27:11.421 INFO:tasks.workunit.client.1.smithi191.stdout:1/633: creat d0/d2f/d51/fd1 x:0 0 0 2024-02-15T18:27:11.422 INFO:tasks.workunit.client.1.smithi191.stdout:9/580: mknod d0/d7/dbf/cc5 0 2024-02-15T18:27:11.424 INFO:tasks.workunit.client.1.smithi191.stdout:5/323: mkdir d0/d23/d41/d28/d65 0 2024-02-15T18:27:11.425 INFO:tasks.workunit.client.1.smithi191.stdout:4/597: dwrite d2/d5/d7/d2f/f6b [0,4194304] 0 2024-02-15T18:27:11.425 INFO:tasks.workunit.client.1.smithi191.stdout:4/598: write d2/d5/d7/d90/d52/dcb/fd6 [4294862,88718] 0 2024-02-15T18:27:11.425 INFO:tasks.workunit.client.1.smithi191.stdout:4/599: stat d2/d5/d7/c79 0 2024-02-15T18:27:11.425 INFO:tasks.workunit.client.1.smithi191.stdout:4/600: chown d2/d5/d14/d47/d78/f61 428377 1 2024-02-15T18:27:11.425 INFO:tasks.workunit.client.1.smithi191.stdout:8/682: truncate d3/d5e/f80 3331147 0 2024-02-15T18:27:11.425 INFO:tasks.workunit.client.1.smithi191.stdout:8/683: truncate d3/d2c/d1a/d23/d2a/f6e 1417453 0 2024-02-15T18:27:11.428 INFO:tasks.workunit.client.1.smithi191.stdout:5/324: dread d0/d23/f43 [0,4194304] 0 2024-02-15T18:27:11.428 INFO:tasks.workunit.client.1.smithi191.stdout:8/684: write d3/d2c/d1a/f8f [62538,41506] 0 2024-02-15T18:27:11.430 INFO:tasks.workunit.client.1.smithi191.stdout:4/601: dread d2/d5/d7/d90/d52/f7b [0,4194304] 0 2024-02-15T18:27:11.431 INFO:tasks.workunit.client.1.smithi191.stdout:1/634: dread d0/d8/df/d12/f23 [0,4194304] 0 2024-02-15T18:27:11.433 INFO:tasks.workunit.client.1.smithi191.stdout:9/581: dread d0/d7/dc/d17/f61 [0,4194304] 0 2024-02-15T18:27:11.434 INFO:tasks.workunit.client.1.smithi191.stdout:8/685: write d3/d2c/d1a/f2d [779929,43360] 0 2024-02-15T18:27:11.436 INFO:tasks.workunit.client.1.smithi191.stdout:7/586: dwrite d0/db/d26/d5f/d77/df/f64 [0,4194304] 0 2024-02-15T18:27:11.436 INFO:tasks.workunit.client.1.smithi191.stdout:7/587: fsync d0/db/f8b 0 2024-02-15T18:27:11.438 INFO:tasks.workunit.client.1.smithi191.stdout:5/325: unlink d0/d23/d41/c4e 0 2024-02-15T18:27:11.440 INFO:tasks.workunit.client.1.smithi191.stdout:1/635: symlink d0/d8/df/ld2 0 2024-02-15T18:27:11.440 INFO:tasks.workunit.client.1.smithi191.stdout:1/636: chown d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/dc9/fcc 387 1 2024-02-15T18:27:11.440 INFO:tasks.workunit.client.1.smithi191.stdout:1/637: dread - d0/fa8 zero size 2024-02-15T18:27:11.440 INFO:tasks.workunit.client.1.smithi191.stdout:1/638: readlink d0/d8/df/d2e/l46 0 2024-02-15T18:27:11.440 INFO:tasks.workunit.client.1.smithi191.stdout:1/639: write d0/d8/df/d12/d2a/fb8 [540377,106249] 0 2024-02-15T18:27:11.441 INFO:tasks.workunit.client.1.smithi191.stdout:8/686: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/ff4 x:0 0 0 2024-02-15T18:27:11.443 INFO:tasks.workunit.client.1.smithi191.stdout:1/640: truncate d0/d8/df/d12/f32 4523115 0 2024-02-15T18:27:11.445 INFO:tasks.workunit.client.1.smithi191.stdout:8/687: rename d3/d2c/d1a/d23/d2e/d47/c7f to d3/d2c/d1a/d23/d2e/d5b/d69/d6d/cf5 0 2024-02-15T18:27:11.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.451+0000 7f729d380700 1 -- 172.21.15.42:0/598158119 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7298027a00 con 0x7f729806b9c0 2024-02-15T18:27:11.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.451+0000 7f727f7fe700 1 -- 172.21.15.42:0/598158119 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+806 (secure 0 0 0) 0x7f728c019070 con 0x7f729806b9c0 2024-02-15T18:27:11.448 INFO:tasks.workunit.client.1.smithi191.stdout:5/326: symlink d0/l66 0 2024-02-15T18:27:11.448 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:27:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:27:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 1, 2024-02-15T18:27:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 1 2024-02-15T18:27:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:27:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:27:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:27:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:27:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:27:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:27:11.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:27:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:27:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:27:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:27:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:27:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 3, 2024-02-15T18:27:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 11 2024-02-15T18:27:11.449 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:27:11.449 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:27:11.450 INFO:tasks.workunit.client.1.smithi191.stdout:7/588: dread d0/db/d65/f22 [0,4194304] 0 2024-02-15T18:27:11.450 INFO:tasks.workunit.client.1.smithi191.stdout:7/589: chown d0/db/d26/d5f/d77/df/f8e 5526 1 2024-02-15T18:27:11.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.453+0000 7f729d380700 1 -- 172.21.15.42:0/598158119 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f7280060fb0 msgr2=0x7f7280063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:11.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.453+0000 7f729d380700 1 --2- 172.21.15.42:0/598158119 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f7280060fb0 0x7f7280063470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f7288009720 tx=0x7f7288016040).stop 2024-02-15T18:27:11.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.453+0000 7f729d380700 1 -- 172.21.15.42:0/598158119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729806b9c0 msgr2=0x7f729810dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:11.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.453+0000 7f729d380700 1 --2- 172.21.15.42:0/598158119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729806b9c0 0x7f729810dab0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f728c00e7f0 tx=0x7f728c00b040).stop 2024-02-15T18:27:11.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.453+0000 7f729d380700 1 -- 172.21.15.42:0/598158119 shutdown_connections 2024-02-15T18:27:11.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.453+0000 7f729d380700 1 --2- 172.21.15.42:0/598158119 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f7280060fb0 0x7f7280063470 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.453+0000 7f729d380700 1 --2- 172.21.15.42:0/598158119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f729806a3b0 0x7f729810d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.453+0000 7f729d380700 1 --2- 172.21.15.42:0/598158119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729806b9c0 0x7f729810dab0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.453+0000 7f729d380700 1 -- 172.21.15.42:0/598158119 >> 172.21.15.42:0/598158119 conn(0x7f72980f1e10 msgr2=0x7f72980f89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:11.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.454+0000 7f729d380700 1 -- 172.21.15.42:0/598158119 shutdown_connections 2024-02-15T18:27:11.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.454+0000 7f729d380700 1 -- 172.21.15.42:0/598158119 wait complete. 2024-02-15T18:27:11.451 INFO:tasks.workunit.client.1.smithi191.stdout:1/641: mkdir d0/d2f/d51/d54/dd3 0 2024-02-15T18:27:11.452 INFO:tasks.workunit.client.1.smithi191.stdout:8/688: creat d3/d2c/d1a/d23/ff6 x:0 0 0 2024-02-15T18:27:11.453 INFO:tasks.workunit.client.1.smithi191.stdout:5/327: creat d0/d23/d48/f67 x:0 0 0 2024-02-15T18:27:11.455 INFO:tasks.workunit.client.1.smithi191.stdout:7/590: mkdir d0/db/ddd 0 2024-02-15T18:27:11.456 INFO:tasks.workunit.client.1.smithi191.stdout:4/602: dwrite d2/f81 [0,4194304] 0 2024-02-15T18:27:11.457 INFO:tasks.workunit.client.1.smithi191.stdout:1/642: creat d0/d8/df/d12/fd4 x:0 0 0 2024-02-15T18:27:11.463 INFO:tasks.workunit.client.1.smithi191.stdout:8/689: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/ff7 x:0 0 0 2024-02-15T18:27:11.464 INFO:tasks.workunit.client.1.smithi191.stdout:5/328: creat d0/d23/d41/d28/d31/d5a/f68 x:0 0 0 2024-02-15T18:27:11.467 INFO:tasks.workunit.client.1.smithi191.stdout:1/643: creat d0/d8/df/d90/d94/fd5 x:0 0 0 2024-02-15T18:27:11.467 INFO:tasks.workunit.client.1.smithi191.stdout:1/644: read d0/d8/df/d12/d2a/d2b/f5a [479731,24302] 0 2024-02-15T18:27:11.469 INFO:tasks.workunit.client.1.smithi191.stdout:1/645: creat d0/d2f/d51/d5e/fd6 x:0 0 0 2024-02-15T18:27:11.473 INFO:tasks.workunit.client.1.smithi191.stdout:5/329: dread d0/d23/f27 [0,4194304] 0 2024-02-15T18:27:11.473 INFO:tasks.workunit.client.1.smithi191.stdout:5/330: chown d0/d18/l1c 604129 1 2024-02-15T18:27:11.476 INFO:tasks.workunit.client.1.smithi191.stdout:8/690: rmdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67 39 2024-02-15T18:27:11.478 INFO:tasks.workunit.client.1.smithi191.stdout:5/331: dread d0/d18/f2e [0,4194304] 0 2024-02-15T18:27:11.478 INFO:tasks.workunit.client.1.smithi191.stdout:5/332: stat d0/f1a 0 2024-02-15T18:27:11.478 INFO:tasks.workunit.client.1.smithi191.stdout:5/333: write d0/d23/d41/d28/d31/f47 [668328,116540] 0 2024-02-15T18:27:11.479 INFO:tasks.workunit.client.1.smithi191.stdout:0/394: sync 2024-02-15T18:27:11.480 INFO:tasks.workunit.client.1.smithi191.stdout:3/483: sync 2024-02-15T18:27:11.480 INFO:tasks.workunit.client.1.smithi191.stdout:5/334: mknod d0/d23/d41/d3c/c69 0 2024-02-15T18:27:11.480 INFO:tasks.workunit.client.1.smithi191.stdout:5/335: dread - d0/d23/d41/d28/d31/d5a/f68 zero size 2024-02-15T18:27:11.481 INFO:tasks.workunit.client.1.smithi191.stdout:4/603: dwrite d2/d5/d7/d2f/f31 [4194304,4194304] 0 2024-02-15T18:27:11.481 INFO:tasks.workunit.client.1.smithi191.stdout:4/604: readlink d2/l4b 0 2024-02-15T18:27:11.481 INFO:tasks.workunit.client.1.smithi191.stdout:4/605: readlink d2/d5/d14/d44/la6 0 2024-02-15T18:27:11.481 INFO:tasks.workunit.client.1.smithi191.stdout:7/591: dwrite d0/db/d65/dc/fb0 [0,4194304] 0 2024-02-15T18:27:11.481 INFO:tasks.workunit.client.1.smithi191.stdout:7/592: write d0/db/d26/d5f/fd5 [799357,112090] 0 2024-02-15T18:27:11.482 INFO:tasks.workunit.client.1.smithi191.stdout:3/484: rename d6/f8 to d6/d1b/d60/da2/fa9 0 2024-02-15T18:27:11.482 INFO:tasks.workunit.client.0.smithi042.stdout:8/965: sync 2024-02-15T18:27:11.482 INFO:tasks.workunit.client.0.smithi042.stdout:8/966: fsync dd/de/d33/dbf/df0/f104 0 2024-02-15T18:27:11.483 INFO:tasks.workunit.client.1.smithi191.stdout:0/395: symlink d7/da/d17/d31/d5f/l88 0 2024-02-15T18:27:11.485 INFO:tasks.workunit.client.1.smithi191.stdout:5/336: symlink d0/d23/d41/d28/d31/l6a 0 2024-02-15T18:27:11.486 INFO:tasks.workunit.client.1.smithi191.stdout:4/606: getdents d2/d5/d7/d90/d52 0 2024-02-15T18:27:11.486 INFO:tasks.workunit.client.1.smithi191.stdout:4/607: readlink d2/l4b 0 2024-02-15T18:27:11.488 INFO:tasks.workunit.client.1.smithi191.stdout:7/593: rmdir d0/db/d65/d11/d88/d9d 39 2024-02-15T18:27:11.488 INFO:tasks.workunit.client.1.smithi191.stdout:7/594: fdatasync d0/db/d26/d5f/d77/fcd 0 2024-02-15T18:27:11.489 INFO:tasks.workunit.client.1.smithi191.stdout:3/485: mkdir d6/d5d/daa 0 2024-02-15T18:27:11.491 INFO:tasks.workunit.client.1.smithi191.stdout:4/608: rename d2/d5/d7/d3e/d63/d8c/fb0 to d2/d5/d8b/fe5 0 2024-02-15T18:27:11.496 INFO:tasks.workunit.client.1.smithi191.stdout:7/595: creat d0/db/d65/d11/d57/fde x:0 0 0 2024-02-15T18:27:11.497 INFO:tasks.workunit.client.1.smithi191.stdout:7/596: chown d0/db/d65/d11/d57/fcf 24 1 2024-02-15T18:27:11.498 INFO:tasks.workunit.client.1.smithi191.stdout:4/609: creat d2/d5/d7/d90/d52/fe6 x:0 0 0 2024-02-15T18:27:11.499 INFO:tasks.workunit.client.1.smithi191.stdout:4/610: truncate d2/d5/d14/d47/fc5 77422 0 2024-02-15T18:27:11.499 INFO:tasks.workunit.client.1.smithi191.stdout:4/611: readlink d2/d5/d14/d44/l7a 0 2024-02-15T18:27:11.499 INFO:tasks.workunit.client.1.smithi191.stdout:4/612: chown d2/d5/d7/fa9 22119828 1 2024-02-15T18:27:11.499 INFO:tasks.workunit.client.1.smithi191.stdout:4/613: chown d2/d5/d7/d3e/faf 370 1 2024-02-15T18:27:11.499 INFO:tasks.workunit.client.1.smithi191.stdout:3/486: write d6/d3b/d42/d49/f3d [3359502,30447] 0 2024-02-15T18:27:11.499 INFO:tasks.workunit.client.1.smithi191.stdout:7/597: mknod d0/db/d65/d11/d88/d9d/cdf 0 2024-02-15T18:27:11.500 INFO:tasks.workunit.client.1.smithi191.stdout:3/487: fdatasync d6/d3b/d42/d49/d54/f83 0 2024-02-15T18:27:11.501 INFO:tasks.workunit.client.1.smithi191.stdout:7/598: mknod d0/db/d65/d11/d57/ce0 0 2024-02-15T18:27:11.502 INFO:tasks.workunit.client.1.smithi191.stdout:3/488: rmdir d6/d1b/d60/d73 39 2024-02-15T18:27:11.502 INFO:tasks.workunit.client.1.smithi191.stdout:3/489: chown d6/d1b/d82/d9d 1795 1 2024-02-15T18:27:11.502 INFO:tasks.workunit.client.1.smithi191.stdout:3/490: truncate d6/d1b/d60/d73/f34 1447523 0 2024-02-15T18:27:11.502 INFO:tasks.workunit.client.1.smithi191.stdout:3/491: chown d6/d1b/d60/d73 1 1 2024-02-15T18:27:11.503 INFO:tasks.workunit.client.1.smithi191.stdout:4/614: write d2/d5/d14/d47/d78/f4f [910317,121927] 0 2024-02-15T18:27:11.503 INFO:tasks.workunit.client.1.smithi191.stdout:7/599: symlink d0/le1 0 2024-02-15T18:27:11.504 INFO:tasks.workunit.client.1.smithi191.stdout:3/492: dread d6/d3b/d42/f2a [0,4194304] 0 2024-02-15T18:27:11.505 INFO:tasks.workunit.client.1.smithi191.stdout:4/615: mknod d2/d5/d7/d90/da5/ce7 0 2024-02-15T18:27:11.507 INFO:tasks.workunit.client.1.smithi191.stdout:7/600: dread d0/f9e [0,4194304] 0 2024-02-15T18:27:11.507 INFO:tasks.workunit.client.1.smithi191.stdout:7/601: fdatasync d0/db/d65/f22 0 2024-02-15T18:27:11.507 INFO:tasks.workunit.client.1.smithi191.stdout:4/616: mkdir d2/d5/d7/d3e/de8 0 2024-02-15T18:27:11.508 INFO:tasks.workunit.client.1.smithi191.stdout:7/602: unlink d0/db/d26/d5f/d77/d1d/fcc 0 2024-02-15T18:27:11.515 INFO:tasks.workunit.client.1.smithi191.stdout:5/337: dwrite d0/d23/d48/f58 [0,4194304] 0 2024-02-15T18:27:11.515 INFO:tasks.workunit.client.1.smithi191.stdout:5/338: write d0/d23/d41/d28/f4a [437229,12078] 0 2024-02-15T18:27:11.515 INFO:tasks.workunit.client.1.smithi191.stdout:5/339: readlink d0/d23/d41/l34 0 2024-02-15T18:27:11.517 INFO:tasks.workunit.client.1.smithi191.stdout:5/340: truncate d0/d23/d41/d28/f3e 4377515 0 2024-02-15T18:27:11.517 INFO:tasks.workunit.client.1.smithi191.stdout:5/341: chown d0/d23/d41/d32/d52/c51 12 1 2024-02-15T18:27:11.517 INFO:tasks.workunit.client.1.smithi191.stdout:7/603: write d0/db/d26/d5f/d77/d35/f50 [3521946,42483] 0 2024-02-15T18:27:11.517 INFO:tasks.workunit.client.1.smithi191.stdout:4/617: dread d2/d5/d7/d3e/d63/d8c/da4/fca [0,4194304] 0 2024-02-15T18:27:11.517 INFO:tasks.workunit.client.1.smithi191.stdout:8/691: dwrite d3/d2c/d1a/d23/d2e/d5b/fa1 [0,4194304] 0 2024-02-15T18:27:11.517 INFO:tasks.workunit.client.1.smithi191.stdout:8/692: dread - d3/d2c/d1a/fce zero size 2024-02-15T18:27:11.517 INFO:tasks.workunit.client.1.smithi191.stdout:8/693: dread - d3/d2c/d1a/d23/d2e/d47/feb zero size 2024-02-15T18:27:11.517 INFO:tasks.workunit.client.1.smithi191.stdout:8/694: fsync d3/d2c/f48 0 2024-02-15T18:27:11.518 INFO:tasks.workunit.client.1.smithi191.stdout:8/695: read d3/d2c/d1a/d23/d2e/f9f [12845,69257] 0 2024-02-15T18:27:11.518 INFO:tasks.workunit.client.1.smithi191.stdout:8/696: write d3/fd [4401174,94788] 0 2024-02-15T18:27:11.518 INFO:tasks.workunit.client.1.smithi191.stdout:8/697: chown d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/de4 147175445 1 2024-02-15T18:27:11.518 INFO:tasks.workunit.client.0.smithi042.stdout:8/967: dwrite dd/fe4 [0,4194304] 0 2024-02-15T18:27:11.518 INFO:tasks.workunit.client.0.smithi042.stdout:8/968: write dd/de/da4/d67/d5a/db2/fc3 [36232,118546] 0 2024-02-15T18:27:11.521 INFO:tasks.workunit.client.1.smithi191.stdout:7/604: creat d0/db/ddd/fe2 x:0 0 0 2024-02-15T18:27:11.523 INFO:tasks.workunit.client.0.smithi042.stdout:8/969: write dd/f29 [1824371,96692] 0 2024-02-15T18:27:11.523 INFO:tasks.workunit.client.0.smithi042.stdout:8/970: fdatasync dd/de/d3c/f7e 0 2024-02-15T18:27:11.523 INFO:tasks.workunit.client.1.smithi191.stdout:4/618: symlink d2/d5/d7/le9 0 2024-02-15T18:27:11.527 INFO:tasks.workunit.client.1.smithi191.stdout:8/698: symlink d3/d2c/lf8 0 2024-02-15T18:27:11.529 INFO:tasks.workunit.client.1.smithi191.stdout:7/605: mknod d0/db/d26/d5f/d77/df/d5d/ce3 0 2024-02-15T18:27:11.533 INFO:tasks.workunit.client.1.smithi191.stdout:8/699: rename d3/d5e/lc3 to d3/d2c/d1a/d36/d9b/lf9 0 2024-02-15T18:27:11.534 INFO:tasks.workunit.client.1.smithi191.stdout:5/342: dwrite d0/f46 [0,4194304] 0 2024-02-15T18:27:11.534 INFO:tasks.workunit.client.1.smithi191.stdout:5/343: readlink d0/l1d 0 2024-02-15T18:27:11.535 INFO:tasks.workunit.client.1.smithi191.stdout:7/606: unlink d0/db/f2f 0 2024-02-15T18:27:11.535 INFO:tasks.workunit.client.1.smithi191.stdout:4/619: mknod d2/d5/d7/d2f/cea 0 2024-02-15T18:27:11.536 INFO:tasks.workunit.client.1.smithi191.stdout:8/700: rmdir d3 39 2024-02-15T18:27:11.540 INFO:tasks.workunit.client.1.smithi191.stdout:7/607: mknod d0/db/d26/d5f/d77/df/d5d/ce4 0 2024-02-15T18:27:11.540 INFO:tasks.workunit.client.1.smithi191.stdout:4/620: write d2/d5/d7/d2f/fc2 [338081,37887] 0 2024-02-15T18:27:11.540 INFO:tasks.workunit.client.1.smithi191.stdout:4/621: readlink d2/d5/d14/l65 0 2024-02-15T18:27:11.542 INFO:tasks.workunit.client.0.smithi042.stdout:8/971: dwrite dd/de/d33/d66/f34 [0,4194304] 0 2024-02-15T18:27:11.544 INFO:tasks.workunit.client.1.smithi191.stdout:4/622: link d2/d5/d14/f58 d2/d5/d8b/feb 0 2024-02-15T18:27:11.546 INFO:tasks.workunit.client.0.smithi042.stdout:8/972: mkdir dd/de/da5/dc6/d11a 0 2024-02-15T18:27:11.552 INFO:tasks.workunit.client.0.smithi042.stdout:8/973: creat dd/de/d33/d66/d4e/f11b x:0 0 0 2024-02-15T18:27:11.557 INFO:tasks.workunit.client.1.smithi191.stdout:5/344: dwrite d0/d18/f2e [0,4194304] 0 2024-02-15T18:27:11.557 INFO:tasks.workunit.client.1.smithi191.stdout:5/345: readlink d0/d18/l1c 0 2024-02-15T18:27:11.557 INFO:tasks.workunit.client.1.smithi191.stdout:5/346: chown d0/f46 6558 1 2024-02-15T18:27:11.557 INFO:tasks.workunit.client.1.smithi191.stdout:5/347: fsync d0/f1b 0 2024-02-15T18:27:11.557 INFO:tasks.workunit.client.1.smithi191.stdout:5/348: getdents d0/d23/d41/d28/d65 0 2024-02-15T18:27:11.558 INFO:tasks.workunit.client.0.smithi042.stdout:8/974: write dd/de/da4/d67/f9f [3484006,76194] 0 2024-02-15T18:27:11.558 INFO:tasks.workunit.client.0.smithi042.stdout:8/975: chown dd/de/d12/fef 757 1 2024-02-15T18:27:11.558 INFO:tasks.workunit.client.0.smithi042.stdout:8/976: stat fc 0 2024-02-15T18:27:11.560 INFO:tasks.workunit.client.1.smithi191.stdout:5/349: symlink d0/d23/d41/l6b 0 2024-02-15T18:27:11.560 INFO:tasks.workunit.client.1.smithi191.stdout:5/350: fsync d0/d23/f43 0 2024-02-15T18:27:11.560 INFO:tasks.workunit.client.1.smithi191.stdout:5/351: dread - d0/d23/d48/f67 zero size 2024-02-15T18:27:11.561 INFO:tasks.workunit.client.0.smithi042.stdout:8/977: creat dd/de/da4/d67/d6f/f11c x:0 0 0 2024-02-15T18:27:11.562 INFO:tasks.workunit.client.1.smithi191.stdout:4/623: dwrite d2/d5/d7/fa9 [0,4194304] 0 2024-02-15T18:27:11.562 INFO:tasks.workunit.client.1.smithi191.stdout:5/352: mknod d0/d23/d41/d28/d31/c6c 0 2024-02-15T18:27:11.562 INFO:tasks.workunit.client.1.smithi191.stdout:4/624: fsync d2/d5/d7/d90/d52/db5/fc3 0 2024-02-15T18:27:11.563 INFO:tasks.workunit.client.1.smithi191.stdout:5/353: chown d0/c3 12523457 1 2024-02-15T18:27:11.563 INFO:tasks.workunit.client.1.smithi191.stdout:4/625: dread - d2/d5/d7/d3e/d63/d8c/da4/fd7 zero size 2024-02-15T18:27:11.563 INFO:tasks.workunit.client.1.smithi191.stdout:5/354: write d0/d23/d41/d28/d31/d5a/f68 [255241,103535] 0 2024-02-15T18:27:11.564 INFO:tasks.workunit.client.1.smithi191.stdout:7/608: dwrite d0/db/d65/dc/d4b/f59 [0,4194304] 0 2024-02-15T18:27:11.565 INFO:tasks.workunit.client.0.smithi042.stdout:8/978: truncate dd/de/d12/fd4 668757 0 2024-02-15T18:27:11.568 INFO:tasks.workunit.client.0.smithi042.stdout:8/979: read dd/de/d12/faf [758991,17557] 0 2024-02-15T18:27:11.568 INFO:tasks.workunit.client.0.smithi042.stdout:8/980: dread - dd/de/d12/fef zero size 2024-02-15T18:27:11.569 INFO:tasks.workunit.client.1.smithi191.stdout:4/626: unlink d2/d5/d86/c9c 0 2024-02-15T18:27:11.571 INFO:tasks.workunit.client.1.smithi191.stdout:5/355: creat d0/d18/d5c/f6d x:0 0 0 2024-02-15T18:27:11.572 INFO:tasks.workunit.client.0.smithi042.stdout:8/981: creat dd/de/da4/d67/f11d x:0 0 0 2024-02-15T18:27:11.572 INFO:tasks.workunit.client.0.smithi042.stdout:8/982: dread - dd/de/d33/d66/d4e/f87 zero size 2024-02-15T18:27:11.573 INFO:tasks.workunit.client.0.smithi042.stdout:8/983: fdatasync dd/de/d33/dbf/df0/f104 0 2024-02-15T18:27:11.573 INFO:tasks.workunit.client.1.smithi191.stdout:4/627: mknod d2/d5/d7/d90/cec 0 2024-02-15T18:27:11.574 INFO:tasks.workunit.client.0.smithi042.stdout:8/984: mknod dd/de/d33/d66/c11e 0 2024-02-15T18:27:11.575 INFO:tasks.workunit.client.1.smithi191.stdout:5/356: symlink d0/d18/d5c/l6e 0 2024-02-15T18:27:11.576 INFO:tasks.workunit.client.1.smithi191.stdout:4/628: mkdir d2/d5/d14/d71/dcd/ded 0 2024-02-15T18:27:11.576 INFO:tasks.workunit.client.1.smithi191.stdout:4/629: dread - d2/d5/d8b/de3/f9a zero size 2024-02-15T18:27:11.577 INFO:tasks.workunit.client.1.smithi191.stdout:4/630: read d2/d5/d7/d90/d52/f53 [33953,100690] 0 2024-02-15T18:27:11.578 INFO:tasks.workunit.client.1.smithi191.stdout:4/631: write d2/d5/d7/f18 [1330919,28592] 0 2024-02-15T18:27:11.578 INFO:tasks.workunit.client.1.smithi191.stdout:4/632: rmdir d2/d5/d7/d2f 39 2024-02-15T18:27:11.579 INFO:tasks.workunit.client.1.smithi191.stdout:4/633: getdents d2/d5/d14 0 2024-02-15T18:27:11.581 INFO:tasks.workunit.client.1.smithi191.stdout:4/634: dread d2/d5/d7/f80 [0,4194304] 0 2024-02-15T18:27:11.582 INFO:tasks.workunit.client.1.smithi191.stdout:2/571: sync 2024-02-15T18:27:11.583 INFO:tasks.workunit.client.1.smithi191.stdout:6/538: sync 2024-02-15T18:27:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.587+0000 7fbc9e280700 1 -- 172.21.15.42:0/794283208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc980f77d0 msgr2=0x7fbc980f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.587+0000 7fbc9e280700 1 --2- 172.21.15.42:0/794283208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc980f77d0 0x7fbc980f7c30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fbc88004660 tx=0x7fbc88009b30).stop 2024-02-15T18:27:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.587+0000 7fbc9e280700 1 -- 172.21.15.42:0/794283208 shutdown_connections 2024-02-15T18:27:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.587+0000 7fbc9e280700 1 --2- 172.21.15.42:0/794283208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc980f77d0 0x7fbc980f7c30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.587+0000 7fbc9e280700 1 --2- 172.21.15.42:0/794283208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc980f6530 0x7fbc980f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.587+0000 7fbc9e280700 1 -- 172.21.15.42:0/794283208 >> 172.21.15.42:0/794283208 conn(0x7fbc980f1c90 msgr2=0x7fbc980f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.587+0000 7fbc9e280700 1 -- 172.21.15.42:0/794283208 shutdown_connections 2024-02-15T18:27:11.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.587+0000 7fbc9e280700 1 -- 172.21.15.42:0/794283208 wait complete. 2024-02-15T18:27:11.585 INFO:tasks.workunit.client.1.smithi191.stdout:4/635: mkdir d2/d5/d7/dee 0 2024-02-15T18:27:11.585 INFO:tasks.workunit.client.1.smithi191.stdout:4/636: truncate d2/f28 4700239 0 2024-02-15T18:27:11.585 INFO:tasks.workunit.client.1.smithi191.stdout:2/572: rmdir df/d12/d1e 39 2024-02-15T18:27:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.588+0000 7fbc9e280700 1 Processor -- start 2024-02-15T18:27:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.588+0000 7fbc9e280700 1 -- start start 2024-02-15T18:27:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.588+0000 7fbc9e280700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc980f6530 0x7fbc98106e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.588+0000 7fbc9e280700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc980f77d0 0x7fbc98107310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.588+0000 7fbc9e280700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc98107910 con 0x7fbc980f6530 2024-02-15T18:27:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.588+0000 7fbc9e280700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc98108120 con 0x7fbc980f77d0 2024-02-15T18:27:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.588+0000 7fbc9d27e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc980f6530 0x7fbc98106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.588+0000 7fbc9d27e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc980f6530 0x7fbc98106e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45066/0 (socket says 172.21.15.42:45066) 2024-02-15T18:27:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.588+0000 7fbc9d27e700 1 -- 172.21.15.42:0/2126640308 learned_addr learned my addr 172.21.15.42:0/2126640308 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:27:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.589+0000 7fbc9d27e700 1 -- 172.21.15.42:0/2126640308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc980f77d0 msgr2=0x7fbc98107310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.589+0000 7fbc9d27e700 1 --2- 172.21.15.42:0/2126640308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc980f77d0 0x7fbc98107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:27:11.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.589+0000 7fbc9d27e700 1 -- 172.21.15.42:0/2126640308 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc88005040 con 0x7fbc980f6530 2024-02-15T18:27:11.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.589+0000 7fbc9d27e700 1 --2- 172.21.15.42:0/2126640308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc980f6530 0x7fbc98106e20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fbc9400e7c0 tx=0x7fbc9400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:11.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.589+0000 7fbc8dffb700 1 -- 172.21.15.42:0/2126640308 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc940092e0 con 0x7fbc980f6530 2024-02-15T18:27:11.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.589+0000 7fbc9e280700 1 -- 172.21.15.42:0/2126640308 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc981083a0 con 0x7fbc980f6530 2024-02-15T18:27:11.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.589+0000 7fbc9e280700 1 -- 172.21.15.42:0/2126640308 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc98108960 con 0x7fbc980f6530 2024-02-15T18:27:11.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.590+0000 7fbc8dffb700 1 -- 172.21.15.42:0/2126640308 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fbc9400f040 con 0x7fbc980f6530 2024-02-15T18:27:11.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.590+0000 7fbc8dffb700 1 -- 172.21.15.42:0/2126640308 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc94014880 con 0x7fbc980f6530 2024-02-15T18:27:11.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.591+0000 7fbc9e280700 1 -- 172.21.15.42:0/2126640308 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc98059100 con 0x7fbc980f6530 2024-02-15T18:27:11.588 INFO:tasks.workunit.client.1.smithi191.stdout:6/539: mkdir d8/d9/d2b/d5b/d9f 0 2024-02-15T18:27:11.588 INFO:tasks.workunit.client.1.smithi191.stdout:6/540: write d8/d9/d2b/d57/d8e/f92 [16392,33583] 0 2024-02-15T18:27:11.588 INFO:tasks.workunit.client.1.smithi191.stdout:6/541: readlink d8/l2a 0 2024-02-15T18:27:11.588 INFO:tasks.workunit.client.1.smithi191.stdout:6/542: truncate d8/d9/d4a/d4c/f5d 2740671 0 2024-02-15T18:27:11.588 INFO:tasks.workunit.client.1.smithi191.stdout:4/637: getdents d2/d5/d7/d3e/d63/d8c/da4 0 2024-02-15T18:27:11.589 INFO:tasks.workunit.client.1.smithi191.stdout:6/543: write d8/d9/f36 [2935862,54779] 0 2024-02-15T18:27:11.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.594+0000 7fbc8dffb700 1 -- 172.21.15.42:0/2126640308 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 30) v1 ==== 77246+0+0 (secure 0 0 0) 0x7fbc9401b030 con 0x7fbc980f6530 2024-02-15T18:27:11.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.594+0000 7fbc8dffb700 1 --2- 172.21.15.42:0/2126640308 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc84061000 0x7fbc840634c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:11.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.594+0000 7fbc8dffb700 1 -- 172.21.15.42:0/2126640308 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fbc9407c270 con 0x7fbc980f6530 2024-02-15T18:27:11.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.594+0000 7fbc9ca7d700 1 --2- 172.21.15.42:0/2126640308 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc84061000 0x7fbc840634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:11.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.595+0000 7fbc9ca7d700 1 --2- 172.21.15.42:0/2126640308 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc84061000 0x7fbc840634c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbc88005010 tx=0x7fbc8800b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:11.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.597+0000 7fbc8dffb700 1 -- 172.21.15.42:0/2126640308 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbc9404c790 con 0x7fbc980f6530 2024-02-15T18:27:11.597 INFO:tasks.workunit.client.1.smithi191.stdout:2/573: dread df/d12/d1f/f34 [0,4194304] 0 2024-02-15T18:27:11.598 INFO:tasks.workunit.client.1.smithi191.stdout:2/574: mknod df/d12/d19/cc2 0 2024-02-15T18:27:11.598 INFO:tasks.workunit.client.1.smithi191.stdout:2/575: fsync df/d12/d1f/d42/fba 0 2024-02-15T18:27:11.599 INFO:tasks.workunit.client.1.smithi191.stdout:2/576: creat df/d12/d1e/d6f/fc3 x:0 0 0 2024-02-15T18:27:11.601 INFO:tasks.workunit.client.1.smithi191.stdout:6/544: dread d8/d9/d61/d95/d9d/d6b/f75 [0,4194304] 0 2024-02-15T18:27:11.611 INFO:tasks.workunit.client.0.smithi042.stdout:8/985: dwrite dd/de/d12/fd4 [0,4194304] 0 2024-02-15T18:27:11.612 INFO:tasks.workunit.client.1.smithi191.stdout:4/638: dwrite d2/d5/d14/d47/d78/dd0/fd1 [0,4194304] 0 2024-02-15T18:27:11.612 INFO:tasks.workunit.client.1.smithi191.stdout:4/639: getdents d2/d5/d14/d71/dcd/ded 0 2024-02-15T18:27:11.616 INFO:tasks.workunit.client.1.smithi191.stdout:4/640: creat d2/d5/d7/d90/fef x:0 0 0 2024-02-15T18:27:11.616 INFO:tasks.workunit.client.1.smithi191.stdout:4/641: getdents d2/d5/d7/d3e/d63/d8c/da4 0 2024-02-15T18:27:11.618 INFO:tasks.workunit.client.1.smithi191.stdout:4/642: mkdir d2/d5/d7/d3e/d63/df0 0 2024-02-15T18:27:11.634 INFO:tasks.workunit.client.1.smithi191.stdout:4/643: dwrite d2/d5/d14/d47/f8e [0,4194304] 0 2024-02-15T18:27:11.634 INFO:tasks.workunit.client.1.smithi191.stdout:4/644: readlink d2/d5/d7/d90/d52/l91 0 2024-02-15T18:27:11.634 INFO:tasks.workunit.client.1.smithi191.stdout:4/645: readlink d2/d5/d7/le9 0 2024-02-15T18:27:11.636 INFO:tasks.workunit.client.1.smithi191.stdout:4/646: mknod d2/d5/d7/d3e/d63/d8c/dbb/cf1 0 2024-02-15T18:27:11.637 INFO:tasks.workunit.client.1.smithi191.stdout:4/647: mknod d2/d5/d7/d3e/d63/cf2 0 2024-02-15T18:27:11.638 INFO:tasks.workunit.client.1.smithi191.stdout:4/648: symlink d2/d5/d7/dee/lf3 0 2024-02-15T18:27:11.638 INFO:tasks.workunit.client.1.smithi191.stdout:4/649: write d2/d5/d7/d3e/d63/d8c/da4/fca [517277,75651] 0 2024-02-15T18:27:11.652 INFO:tasks.workunit.client.1.smithi191.stdout:4/650: dwrite d2/d5/d7/d2f/d41/f98 [0,4194304] 0 2024-02-15T18:27:11.653 INFO:tasks.workunit.client.1.smithi191.stdout:4/651: write d2/d5/d14/f2e [570051,31608] 0 2024-02-15T18:27:11.653 INFO:tasks.workunit.client.1.smithi191.stdout:4/652: dread - d2/d5/d14/fb8 zero size 2024-02-15T18:27:11.654 INFO:tasks.workunit.client.1.smithi191.stdout:4/653: truncate d2/f3 2916971 0 2024-02-15T18:27:11.654 INFO:tasks.workunit.client.1.smithi191.stdout:4/654: stat d2/d5/d7/d3e/d63/d8c/dbb/ddc 0 2024-02-15T18:27:11.655 INFO:tasks.workunit.client.1.smithi191.stdout:4/655: dread d2/d5/d14/d47/fc5 [0,4194304] 0 2024-02-15T18:27:11.655 INFO:tasks.workunit.client.1.smithi191.stdout:4/656: chown d2/d5/d86/lb7 0 1 2024-02-15T18:27:11.660 INFO:tasks.workunit.client.1.smithi191.stdout:4/657: dread d2/d5/d7/d2f/f75 [0,4194304] 0 2024-02-15T18:27:11.660 INFO:tasks.workunit.client.1.smithi191.stdout:4/658: write d2/d5/d7/d90/fef [255890,110490] 0 2024-02-15T18:27:11.661 INFO:tasks.workunit.client.1.smithi191.stdout:4/659: mknod d2/d5/d14/cf4 0 2024-02-15T18:27:11.663 INFO:tasks.workunit.client.1.smithi191.stdout:4/660: mknod d2/d5/d14/d47/cf5 0 2024-02-15T18:27:11.664 INFO:tasks.workunit.client.1.smithi191.stdout:4/661: creat d2/d5/d14/d47/d78/dd0/dd3/de4/ff6 x:0 0 0 2024-02-15T18:27:11.679 INFO:tasks.workunit.client.1.smithi191.stdout:4/662: dwrite d2/f3 [0,4194304] 0 2024-02-15T18:27:11.681 INFO:tasks.workunit.client.0.smithi042.stdout:8/986: sync 2024-02-15T18:27:11.683 INFO:tasks.workunit.client.0.smithi042.stdout:8/987: getdents dd/de/d3c 0 2024-02-15T18:27:11.684 INFO:tasks.workunit.client.0.smithi042.stdout:8/988: creat dd/de/da4/d67/d5a/db2/f11f x:0 0 0 2024-02-15T18:27:11.685 INFO:tasks.workunit.client.0.smithi042.stdout:8/989: mknod dd/de/da5/dc6/d11a/c120 0 2024-02-15T18:27:11.687 INFO:tasks.workunit.client.0.smithi042.stdout:8/990: creat dd/de/da4/d81/dae/f121 x:0 0 0 2024-02-15T18:27:11.687 INFO:tasks.workunit.client.0.smithi042.stdout:8/991: stat dd/de/dc0/ld3 0 2024-02-15T18:27:11.687 INFO:tasks.workunit.client.0.smithi042.stdout:8/992: stat dd/de/dc0/dfd/d107/f118 0 2024-02-15T18:27:11.688 INFO:tasks.workunit.client.0.smithi042.stdout:8/993: truncate dd/de/d33/d66/f7f 1362604 0 2024-02-15T18:27:11.688 INFO:tasks.workunit.client.0.smithi042.stdout:8/994: chown dd/de/d33/d3b/f46 139714 1 2024-02-15T18:27:11.689 INFO:tasks.workunit.client.1.smithi191.stdout:4/663: dread d2/d5/d7/f93 [0,4194304] 0 2024-02-15T18:27:11.690 INFO:tasks.workunit.client.0.smithi042.stdout:8/995: unlink dd/de/d33/l55 0 2024-02-15T18:27:11.691 INFO:tasks.workunit.client.0.smithi042.stdout:8/996: rename dd/de/da4/d81/d113 to dd/de/da4/d67/d6f/d122 0 2024-02-15T18:27:11.701 INFO:tasks.workunit.client.0.smithi042.stdout:8/997: write dd/de/d12/faf [208589,49621] 0 2024-02-15T18:27:11.703 INFO:tasks.workunit.client.0.smithi042.stdout:8/998: mkdir dd/de/dc0/d123 0 2024-02-15T18:27:11.703 INFO:tasks.workunit.client.0.smithi042.stdout:8/999: fdatasync dd/de/da4/d67/d6f/fbe 0 2024-02-15T18:27:11.706 INFO:tasks.workunit.client.1.smithi191.stdout:4/664: dwrite d2/d5/d14/f58 [0,4194304] 0 2024-02-15T18:27:11.706 INFO:tasks.workunit.client.0.smithi042.stderr:+ rm -rf -- ./tmp.JrUCqlHuGi 2024-02-15T18:27:11.762 INFO:tasks.workunit.client.1.smithi191.stdout:9/582: sync 2024-02-15T18:27:11.764 INFO:tasks.workunit.client.1.smithi191.stdout:9/583: symlink d0/d29/d48/d66/d92/lc6 0 2024-02-15T18:27:11.787 INFO:tasks.workunit.client.1.smithi191.stdout:9/584: dwrite d0/d29/d48/f5b [0,4194304] 0 2024-02-15T18:27:11.789 INFO:tasks.workunit.client.1.smithi191.stdout:9/585: dread d0/d7/dc/d17/f49 [0,4194304] 0 2024-02-15T18:27:11.802 INFO:tasks.workunit.client.1.smithi191.stdout:9/586: dwrite d0/d7/dc/d30/d3b/fa1 [0,4194304] 0 2024-02-15T18:27:11.805 INFO:tasks.workunit.client.1.smithi191.stdout:9/587: getdents d0/d7/dc/d17/d44 0 2024-02-15T18:27:11.805 INFO:tasks.workunit.client.1.smithi191.stdout:9/588: write d0/d7/dc/d17/d33/f77 [803154,121783] 0 2024-02-15T18:27:11.807 INFO:tasks.workunit.client.1.smithi191.stdout:9/589: creat d0/d7/dc/d30/d3b/d56/fc7 x:0 0 0 2024-02-15T18:27:11.808 INFO:tasks.workunit.client.1.smithi191.stdout:9/590: write d0/d1d/d3f/d80/fa2 [2985231,8376] 0 2024-02-15T18:27:11.808 INFO:tasks.workunit.client.1.smithi191.stdout:9/591: chown d0/d7/dc/d32/c9d 2039 1 2024-02-15T18:27:11.810 INFO:tasks.workunit.client.1.smithi191.stdout:9/592: symlink d0/d7/dbf/lc8 0 2024-02-15T18:27:11.813 INFO:tasks.workunit.client.1.smithi191.stdout:9/593: rename d0/d7/dc/d8c/l98 to d0/d29/lc9 0 2024-02-15T18:27:11.813 INFO:tasks.workunit.client.1.smithi191.stdout:9/594: write d0/d7/dc/d17/d44/fa4 [317344,123910] 0 2024-02-15T18:27:11.814 INFO:tasks.workunit.client.1.smithi191.stdout:9/595: creat d0/d29/d48/d66/d92/fca x:0 0 0 2024-02-15T18:27:11.816 INFO:tasks.workunit.client.1.smithi191.stdout:9/596: mkdir d0/d29/d48/dcb 0 2024-02-15T18:27:11.835 INFO:tasks.workunit.client.1.smithi191.stdout:9/597: dwrite d0/d7/dc/d17/d33/f69 [0,4194304] 0 2024-02-15T18:27:11.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.846+0000 7fbc9e280700 1 -- 172.21.15.42:0/2126640308 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbc980fee60 con 0x7fbc980f6530 2024-02-15T18:27:11.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.846+0000 7fbc8dffb700 1 -- 172.21.15.42:0/2126640308 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1773 (secure 0 0 0) 0x7fbc94019700 con 0x7fbc980f6530 2024-02-15T18:27:11.843 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:27:11.843 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:27:11.843 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:27:11.843 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:27:11.843 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:27:11.843 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:27:11.843 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:27:11.843 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:27:11.843 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:27:11.843 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:27:11.843 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:27:11.844 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:27:11.844 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:27:11.844 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:27:11.844 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:27:11.844 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:27:11.844 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:27:11.844 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:27:11.844 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:27:11.844 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:27:11.844 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:27:11.844 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:27:11.844 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:27:11.844 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:27:11.844 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:27:11.845 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:27:11.845 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:27:11.845 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:27:11.845 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:27:11.845 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:27:11.845 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:27:11.845 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:27:11.845 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:27:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:27:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:27:11.845 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:27:11.845 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:27:11.845 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:27:11.845 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:27:11.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.850+0000 7fbc9e280700 1 -- 172.21.15.42:0/2126640308 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc84061000 msgr2=0x7fbc840634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:11.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.850+0000 7fbc9e280700 1 --2- 172.21.15.42:0/2126640308 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc84061000 0x7fbc840634c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbc88005010 tx=0x7fbc8800b040).stop 2024-02-15T18:27:11.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.850+0000 7fbc9e280700 1 -- 172.21.15.42:0/2126640308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc980f6530 msgr2=0x7fbc98106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:11.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.850+0000 7fbc9e280700 1 --2- 172.21.15.42:0/2126640308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc980f6530 0x7fbc98106e20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fbc9400e7c0 tx=0x7fbc9400b040).stop 2024-02-15T18:27:11.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.851+0000 7fbc9e280700 1 -- 172.21.15.42:0/2126640308 shutdown_connections 2024-02-15T18:27:11.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.851+0000 7fbc9e280700 1 --2- 172.21.15.42:0/2126640308 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc84061000 0x7fbc840634c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.851+0000 7fbc9e280700 1 --2- 172.21.15.42:0/2126640308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc980f6530 0x7fbc98106e20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.851+0000 7fbc9e280700 1 --2- 172.21.15.42:0/2126640308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc980f77d0 0x7fbc98107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:27:11.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.851+0000 7fbc9e280700 1 -- 172.21.15.42:0/2126640308 >> 172.21.15.42:0/2126640308 conn(0x7fbc980f1c90 msgr2=0x7fbc980fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:11.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.851+0000 7fbc9e280700 1 -- 172.21.15.42:0/2126640308 shutdown_connections 2024-02-15T18:27:11.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.851+0000 7fbc9e280700 1 -- 172.21.15.42:0/2126640308 wait complete. 2024-02-15T18:27:11.848 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:27:11.856 INFO:tasks.workunit.client.1.smithi191.stdout:9/598: dwrite d0/d7/dc/d8c/f72 [4194304,4194304] 0 2024-02-15T18:27:11.864 INFO:tasks.workunit.client.1.smithi191.stdout:9/599: creat d0/d7/dc/d17/d44/fcc x:0 0 0 2024-02-15T18:27:11.871 INFO:tasks.workunit.client.1.smithi191.stdout:1/646: sync 2024-02-15T18:27:11.873 INFO:tasks.workunit.client.1.smithi191.stdout:1/647: mknod d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/cd7 0 2024-02-15T18:27:11.875 INFO:tasks.workunit.client.1.smithi191.stdout:1/648: truncate d0/d2f/f40 829788 0 2024-02-15T18:27:11.875 INFO:tasks.workunit.client.1.smithi191.stdout:1/649: fsync d0/d2f/d77/f81 0 2024-02-15T18:27:11.877 INFO:tasks.workunit.client.1.smithi191.stdout:1/650: mknod d0/d2f/d51/d54/dd3/cd8 0 2024-02-15T18:27:11.877 INFO:tasks.workunit.client.1.smithi191.stdout:1/651: dread - d0/d8/df/d12/fd4 zero size 2024-02-15T18:27:11.877 INFO:tasks.workunit.client.1.smithi191.stdout:9/600: dwrite d0/d7/dc/d30/f4e [4194304,4194304] 0 2024-02-15T18:27:11.878 INFO:tasks.workunit.client.1.smithi191.stdout:1/652: rename d0/d8/df/d41/cc0 to d0/d8/df/d12/cd9 0 2024-02-15T18:27:11.883 INFO:tasks.workunit.client.1.smithi191.stdout:9/601: symlink d0/d7/dbf/lcd 0 2024-02-15T18:27:11.885 INFO:tasks.workunit.client.1.smithi191.stdout:1/653: unlink d0/d8/df/d12/d33/f63 0 2024-02-15T18:27:11.886 INFO:tasks.workunit.client.1.smithi191.stdout:9/602: mkdir d0/d7/dc/d30/d3b/d56/d59/dce 0 2024-02-15T18:27:11.886 INFO:tasks.workunit.client.1.smithi191.stdout:9/603: fdatasync d0/d7/dc/d17/f99 0 2024-02-15T18:27:11.886 INFO:tasks.workunit.client.1.smithi191.stdout:1/654: creat d0/d8/df/d12/fda x:0 0 0 2024-02-15T18:27:11.887 INFO:tasks.workunit.client.1.smithi191.stdout:9/604: mknod d0/d7/dc/d30/ccf 0 2024-02-15T18:27:11.888 INFO:tasks.workunit.client.1.smithi191.stdout:1/655: dread d0/d2f/d77/fbd [0,4194304] 0 2024-02-15T18:27:11.889 INFO:tasks.workunit.client.1.smithi191.stdout:9/605: dread d0/d29/d48/d66/f90 [0,4194304] 0 2024-02-15T18:27:11.890 INFO:tasks.workunit.client.1.smithi191.stdout:9/606: write d0/d7/dc/d30/f5c [872902,37983] 0 2024-02-15T18:27:11.895 INFO:tasks.workunit.client.1.smithi191.stdout:1/656: symlink d0/d8/ldb 0 2024-02-15T18:27:11.896 INFO:tasks.workunit.client.1.smithi191.stdout:1/657: read d0/d2f/d77/faf [2086760,120961] 0 2024-02-15T18:27:11.897 INFO:tasks.workunit.client.1.smithi191.stdout:9/607: rename d0/d29/d48/d66/c9c to d0/d7/dc/d32/cd0 0 2024-02-15T18:27:11.899 INFO:tasks.workunit.client.1.smithi191.stdout:1/658: mknod d0/d8/df/d12/d33/cdc 0 2024-02-15T18:27:11.900 INFO:tasks.workunit.client.1.smithi191.stdout:1/659: fdatasync d0/d2f/d51/f87 0 2024-02-15T18:27:11.901 INFO:tasks.workunit.client.1.smithi191.stdout:9/608: rmdir d0/d29/d48/dcb 0 2024-02-15T18:27:11.903 INFO:tasks.workunit.client.1.smithi191.stdout:1/660: creat d0/d8/df/d12/d2a/d2b/fdd x:0 0 0 2024-02-15T18:27:11.903 INFO:tasks.workunit.client.1.smithi191.stdout:9/609: creat d0/d7/dc/d30/d3b/d56/d59/fd1 x:0 0 0 2024-02-15T18:27:11.903 INFO:tasks.workunit.client.1.smithi191.stdout:9/610: chown d0/d7/dc/c10 26232 1 2024-02-15T18:27:11.903 INFO:tasks.workunit.client.1.smithi191.stdout:1/661: mknod d0/d8/df/d90/d94/d9a/cde 0 2024-02-15T18:27:11.904 INFO:tasks.workunit.client.1.smithi191.stdout:9/611: mknod d0/d29/d48/cd2 0 2024-02-15T18:27:11.905 INFO:tasks.workunit.client.1.smithi191.stdout:9/612: fdatasync d0/d7/dc/d30/d3b/d56/d59/fd1 0 2024-02-15T18:27:11.907 INFO:tasks.workunit.client.1.smithi191.stdout:3/493: sync 2024-02-15T18:27:11.907 INFO:tasks.workunit.client.1.smithi191.stdout:8/701: sync 2024-02-15T18:27:11.907 INFO:tasks.workunit.client.1.smithi191.stdout:0/396: sync 2024-02-15T18:27:11.907 INFO:tasks.workunit.client.1.smithi191.stdout:7/609: sync 2024-02-15T18:27:11.908 INFO:tasks.workunit.client.1.smithi191.stdout:9/613: dread d0/d29/f3d [0,4194304] 0 2024-02-15T18:27:11.909 INFO:tasks.workunit.client.1.smithi191.stdout:3/494: unlink d6/d27/d4d/l63 0 2024-02-15T18:27:11.909 INFO:tasks.workunit.client.1.smithi191.stdout:8/702: write d3/d2c/d1a/d23/d3e/fb4 [694307,51971] 0 2024-02-15T18:27:11.909 INFO:tasks.workunit.client.1.smithi191.stdout:0/397: creat d7/da/d17/d31/f89 x:0 0 0 2024-02-15T18:27:11.912 INFO:tasks.workunit.client.1.smithi191.stdout:9/614: dread d0/d29/f95 [0,4194304] 0 2024-02-15T18:27:11.916 INFO:tasks.workunit.client.1.smithi191.stdout:7/610: link d0/db/f2d d0/db/d26/d5f/d77/d35/dc7/fe5 0 2024-02-15T18:27:11.918 INFO:tasks.workunit.client.1.smithi191.stdout:6/545: sync 2024-02-15T18:27:11.918 INFO:tasks.workunit.client.1.smithi191.stdout:2/577: sync 2024-02-15T18:27:11.918 INFO:tasks.workunit.client.1.smithi191.stdout:5/357: sync 2024-02-15T18:27:11.918 INFO:tasks.workunit.client.1.smithi191.stdout:4/665: sync 2024-02-15T18:27:11.918 INFO:tasks.workunit.client.1.smithi191.stdout:2/578: write df/d61/d9f/da3/fb3 [620805,92781] 0 2024-02-15T18:27:11.918 INFO:tasks.workunit.client.1.smithi191.stdout:2/579: chown df/d12/d32 13016501 1 2024-02-15T18:27:11.921 INFO:tasks.workunit.client.1.smithi191.stdout:5/358: dread d0/d18/f45 [0,4194304] 0 2024-02-15T18:27:11.922 INFO:tasks.workunit.client.1.smithi191.stdout:8/703: unlink d3/d2c/d1a/d23/d2e/d47/d5f/l86 0 2024-02-15T18:27:11.923 INFO:tasks.workunit.client.1.smithi191.stdout:9/615: write d0/d7/dc/d8c/f78 [1690862,55608] 0 2024-02-15T18:27:11.924 INFO:tasks.workunit.client.1.smithi191.stdout:7/611: symlink d0/db/ddd/le6 0 2024-02-15T18:27:11.924 INFO:tasks.workunit.client.1.smithi191.stdout:7/612: fdatasync d0/db/d65/d11/f18 0 2024-02-15T18:27:11.925 INFO:tasks.workunit.client.1.smithi191.stdout:2/580: creat df/d12/d32/fc4 x:0 0 0 2024-02-15T18:27:11.930 INFO:tasks.workunit.client.1.smithi191.stdout:9/616: rmdir d0 39 2024-02-15T18:27:11.931 INFO:tasks.workunit.client.1.smithi191.stdout:6/546: dread d8/d9/d2b/f42 [0,4194304] 0 2024-02-15T18:27:11.932 INFO:tasks.workunit.client.1.smithi191.stdout:6/547: fdatasync d8/d9/d61/d95/d9d/f62 0 2024-02-15T18:27:11.935 INFO:tasks.workunit.client.1.smithi191.stdout:3/495: dwrite d6/d1b/d60/d73/f74 [0,4194304] 0 2024-02-15T18:27:11.935 INFO:tasks.workunit.client.1.smithi191.stdout:3/496: truncate d6/d3b/d30/f50 1249410 0 2024-02-15T18:27:11.937 INFO:tasks.workunit.client.1.smithi191.stdout:4/666: dread d2/d5/d7/f3b [0,4194304] 0 2024-02-15T18:27:11.937 INFO:tasks.workunit.client.1.smithi191.stdout:7/613: unlink d0/db/d65/dc/d6e/fd0 0 2024-02-15T18:27:11.941 INFO:tasks.workunit.client.1.smithi191.stdout:4/667: read d2/d5/d14/d47/d78/f4f [372946,80254] 0 2024-02-15T18:27:11.942 INFO:tasks.workunit.client.1.smithi191.stdout:9/617: creat d0/d7/dc/d30/d3b/d56/fd3 x:0 0 0 2024-02-15T18:27:11.943 INFO:tasks.workunit.client.1.smithi191.stdout:1/662: dwrite d0/d8/df/d12/f32 [0,4194304] 0 2024-02-15T18:27:11.944 INFO:tasks.workunit.client.1.smithi191.stdout:6/548: rename d8/d9/d61/d95/d9d/d6b/l6e to d8/d9/d61/d95/d9d/d6b/la0 0 2024-02-15T18:27:11.948 INFO:tasks.workunit.client.1.smithi191.stdout:3/497: unlink d6/d3b/c93 0 2024-02-15T18:27:11.950 INFO:tasks.workunit.client.1.smithi191.stdout:2/581: rename df/d12/d1f/d42/l64 to df/lc5 0 2024-02-15T18:27:11.950 INFO:tasks.workunit.client.1.smithi191.stdout:2/582: getdents df/d61/d9f/da3/dab 0 2024-02-15T18:27:11.950 INFO:tasks.workunit.client.1.smithi191.stdout:2/583: readlink df/d12/d19/d1a/d35/d47/l68 0 2024-02-15T18:27:11.950 INFO:tasks.workunit.client.1.smithi191.stdout:4/668: mknod d2/d5/d7/d2f/d41/cf7 0 2024-02-15T18:27:11.951 INFO:tasks.workunit.client.1.smithi191.stdout:6/549: creat d8/d9/d61/d95/d90/fa1 x:0 0 0 2024-02-15T18:27:11.952 INFO:tasks.workunit.client.1.smithi191.stdout:3/498: rmdir d6/d27/d94 0 2024-02-15T18:27:11.953 INFO:tasks.workunit.client.1.smithi191.stdout:1/663: rename d0/d8/df/d12/d33/d49/l8a to d0/d2f/d51/ldf 0 2024-02-15T18:27:11.954 INFO:tasks.workunit.client.1.smithi191.stdout:2/584: mknod df/d12/d19/cc6 0 2024-02-15T18:27:11.954 INFO:tasks.workunit.client.1.smithi191.stdout:4/669: symlink d2/d5/d7/d90/da5/dda/lf8 0 2024-02-15T18:27:11.955 INFO:tasks.workunit.client.1.smithi191.stdout:3/499: mknod d6/d27/d4d/d5a/cab 0 2024-02-15T18:27:11.956 INFO:tasks.workunit.client.1.smithi191.stdout:3/500: read d6/d1b/d60/f68 [129411,40371] 0 2024-02-15T18:27:11.956 INFO:tasks.workunit.client.1.smithi191.stdout:3/501: write d6/d27/d4d/f58 [8899657,123315] 0 2024-02-15T18:27:11.956 INFO:tasks.workunit.client.1.smithi191.stdout:3/502: fsync d6/d1b/d60/d73/f41 0 2024-02-15T18:27:11.956 INFO:tasks.workunit.client.1.smithi191.stdout:1/664: symlink d0/d2f/d51/d5e/d68/le0 0 2024-02-15T18:27:11.956 INFO:tasks.workunit.client.1.smithi191.stdout:1/665: chown d0/d8/df/d12/d2a/f42 15905742 1 2024-02-15T18:27:11.957 INFO:tasks.workunit.client.1.smithi191.stdout:1/666: read d0/d8/df/d12/d2a/d2b/f91 [2343383,1114] 0 2024-02-15T18:27:11.957 INFO:tasks.workunit.client.1.smithi191.stdout:2/585: mknod df/d61/d94/cc7 0 2024-02-15T18:27:11.957 INFO:tasks.workunit.client.1.smithi191.stdout:2/586: readlink df/d12/d1e/l7b 0 2024-02-15T18:27:11.958 INFO:tasks.workunit.client.1.smithi191.stdout:1/667: write d0/f5 [3420801,11093] 0 2024-02-15T18:27:11.959 INFO:tasks.workunit.client.1.smithi191.stdout:4/670: rename d2/l4b to d2/d5/d14/lf9 0 2024-02-15T18:27:11.959 INFO:tasks.workunit.client.1.smithi191.stdout:7/614: dwrite d0/db/d65/d11/d5a/d66/d1e/f8a [0,4194304] 0 2024-02-15T18:27:11.960 INFO:tasks.workunit.client.1.smithi191.stdout:9/618: dwrite d0/d7/dc/d17/d44/fa4 [0,4194304] 0 2024-02-15T18:27:11.960 INFO:tasks.workunit.client.1.smithi191.stdout:9/619: stat d0/d7/dc/d30/d4f/fb0 0 2024-02-15T18:27:11.960 INFO:tasks.workunit.client.1.smithi191.stdout:6/550: symlink d8/d9/d2b/la2 0 2024-02-15T18:27:11.965 INFO:tasks.workunit.client.1.smithi191.stdout:3/503: unlink d6/d3b/d30/f7c 0 2024-02-15T18:27:11.966 INFO:tasks.workunit.client.1.smithi191.stdout:4/671: dread - d2/d5/d7/d2f/fc7 zero size 2024-02-15T18:27:11.971 INFO:tasks.workunit.client.1.smithi191.stdout:7/615: mknod d0/db/d65/d11/d5a/d66/d1e/ce7 0 2024-02-15T18:27:11.975 INFO:tasks.workunit.client.1.smithi191.stdout:3/504: mkdir d6/d1b/d60/da2/dac 0 2024-02-15T18:27:11.976 INFO:tasks.workunit.client.1.smithi191.stdout:4/672: truncate d2/d5/d7/f6c 3645152 0 2024-02-15T18:27:11.976 INFO:tasks.workunit.client.1.smithi191.stdout:4/673: write d2/d5/d7/d3e/f94 [4282685,109266] 0 2024-02-15T18:27:11.978 INFO:tasks.workunit.client.1.smithi191.stdout:7/616: mkdir d0/db/d65/dc/d4b/de8 0 2024-02-15T18:27:11.978 INFO:tasks.workunit.client.1.smithi191.stdout:7/617: fdatasync d0/db/d65/d11/d5a/d66/d1e/faf 0 2024-02-15T18:27:11.980 INFO:tasks.workunit.client.1.smithi191.stdout:7/618: getdents d0/db/d65/d11/d5a/d66 0 2024-02-15T18:27:11.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.984+0000 7fcb256a7700 1 -- 172.21.15.42:0/4259031279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb200f6860 msgr2=0x7fcb200f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:11.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.984+0000 7fcb256a7700 1 --2- 172.21.15.42:0/4259031279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb200f6860 0x7fcb200f6cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fcb14004660 tx=0x7fcb14009b30).stop 2024-02-15T18:27:11.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.984+0000 7fcb256a7700 1 -- 172.21.15.42:0/4259031279 shutdown_connections 2024-02-15T18:27:11.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.984+0000 7fcb256a7700 1 --2- 172.21.15.42:0/4259031279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb200f6860 0x7fcb200f6cc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.984+0000 7fcb256a7700 1 --2- 172.21.15.42:0/4259031279 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb200f55c0 0x7fcb200f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:27:11.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.984+0000 7fcb256a7700 1 -- 172.21.15.42:0/4259031279 >> 172.21.15.42:0/4259031279 conn(0x7fcb200f0d40 msgr2=0x7fcb200f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:11.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.984+0000 7fcb256a7700 1 -- 172.21.15.42:0/4259031279 shutdown_connections 2024-02-15T18:27:11.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.984+0000 7fcb256a7700 1 -- 172.21.15.42:0/4259031279 wait complete. 2024-02-15T18:27:11.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.985+0000 7fcb256a7700 1 Processor -- start 2024-02-15T18:27:11.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.985+0000 7fcb256a7700 1 -- start start 2024-02-15T18:27:11.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.985+0000 7fcb256a7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb200f55c0 0x7fcb2010c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:11.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.985+0000 7fcb256a7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb200f6860 0x7fcb2010e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:11.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.985+0000 7fcb256a7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb20060310 con 0x7fcb200f6860 2024-02-15T18:27:11.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.985+0000 7fcb256a7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb20060450 con 0x7fcb200f55c0 2024-02-15T18:27:11.982 INFO:tasks.workunit.client.1.smithi191.stdout:4/674: write d2/d5/d7/d2f/f31 [5136813,11959] 0 2024-02-15T18:27:11.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.985+0000 7fcb1effd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb200f55c0 0x7fcb2010c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:11.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.985+0000 7fcb1effd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb200f55c0 0x7fcb2010c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.42:33656/0 (socket says 172.21.15.42:33656) 2024-02-15T18:27:11.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.985+0000 7fcb1effd700 1 -- 172.21.15.42:0/1589686770 learned_addr learned my addr 172.21.15.42:0/1589686770 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:27:11.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.986+0000 7fcb1e7fc700 1 --2- 172.21.15.42:0/1589686770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb200f6860 0x7fcb2010e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:11.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.986+0000 7fcb1effd700 1 -- 172.21.15.42:0/1589686770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb200f6860 msgr2=0x7fcb2010e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:11.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.986+0000 7fcb1effd700 1 --2- 172.21.15.42:0/1589686770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb200f6860 0x7fcb2010e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.986+0000 7fcb1effd700 1 -- 172.21.15.42:0/1589686770 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb14005040 con 0x7fcb200f55c0 2024-02-15T18:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.986+0000 7fcb1effd700 1 --2- 172.21.15.42:0/1589686770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb200f55c0 0x7fcb2010c130 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fcb100093f0 tx=0x7fcb10006ec0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.986+0000 7fcb077fe700 1 -- 172.21.15.42:0/1589686770 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb10013640 con 0x7fcb200f55c0 2024-02-15T18:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.986+0000 7fcb256a7700 1 -- 172.21.15.42:0/1589686770 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2010eb20 con 0x7fcb200f55c0 2024-02-15T18:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.986+0000 7fcb256a7700 1 -- 172.21.15.42:0/1589686770 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2010efd0 con 0x7fcb200f55c0 2024-02-15T18:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.986+0000 7fcb077fe700 1 -- 172.21.15.42:0/1589686770 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fcb10013c90 con 0x7fcb200f55c0 2024-02-15T18:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.986+0000 7fcb077fe700 1 -- 172.21.15.42:0/1589686770 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb10012de0 con 0x7fcb200f55c0 2024-02-15T18:27:11.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.988+0000 7fcb256a7700 1 -- 172.21.15.42:0/1589686770 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb00004fa0 con 0x7fcb200f55c0 2024-02-15T18:27:11.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.989+0000 7fcb077fe700 1 -- 172.21.15.42:0/1589686770 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 30) v1 ==== 77194+0+0 (secure 0 0 0) 0x7fcb1001a060 con 0x7fcb200f55c0 2024-02-15T18:27:11.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.990+0000 7fcb077fe700 1 --2- 172.21.15.42:0/1589686770 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb08060fb0 0x7fcb08063470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:11.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.990+0000 7fcb077fe700 1 -- 172.21.15.42:0/1589686770 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fcb1007c680 con 0x7fcb200f55c0 2024-02-15T18:27:11.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.990+0000 7fcb1e7fc700 1 --2- 172.21.15.42:0/1589686770 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb08060fb0 0x7fcb08063470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:11.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.990+0000 7fcb1e7fc700 1 --2- 172.21.15.42:0/1589686770 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb08060fb0 0x7fcb08063470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fcb14004630 tx=0x7fcb14009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:11.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:11.995+0000 7fcb077fe700 1 -- 172.21.15.42:0/1589686770 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcb1004ca30 con 0x7fcb200f55c0 2024-02-15T18:27:12.002 INFO:tasks.workunit.client.1.smithi191.stdout:1/668: dwrite d0/d2f/d77/f81 [0,4194304] 0 2024-02-15T18:27:12.003 INFO:tasks.workunit.client.1.smithi191.stdout:2/587: dwrite df/d12/d32/fc4 [0,4194304] 0 2024-02-15T18:27:12.003 INFO:tasks.workunit.client.1.smithi191.stdout:9/620: dwrite d0/d1d/d3f/d80/fa2 [0,4194304] 0 2024-02-15T18:27:12.005 INFO:tasks.workunit.client.1.smithi191.stdout:6/551: dwrite d8/d9/f46 [0,4194304] 0 2024-02-15T18:27:12.005 INFO:tasks.workunit.client.1.smithi191.stdout:7/619: dwrite d0/db/d26/d5f/d77/df/d5d/fa1 [0,4194304] 0 2024-02-15T18:27:12.005 INFO:tasks.workunit.client.1.smithi191.stdout:6/552: truncate d8/d9/d4a/f9a 87946 0 2024-02-15T18:27:12.013 INFO:tasks.workunit.client.1.smithi191.stdout:4/675: dwrite d2/d5/d14/f2e [0,4194304] 0 2024-02-15T18:27:12.028 INFO:tasks.workunit.client.1.smithi191.stdout:9/621: dwrite d0/d7/dc/d17/d44/fa4 [0,4194304] 0 2024-02-15T18:27:12.028 INFO:tasks.workunit.client.1.smithi191.stdout:9/622: fdatasync d0/d7/dc/d30/d4f/fb0 0 2024-02-15T18:27:12.029 INFO:tasks.workunit.client.1.smithi191.stdout:7/620: dwrite d0/db/d65/dc/fa0 [0,4194304] 0 2024-02-15T18:27:12.038 INFO:tasks.workunit.client.1.smithi191.stdout:9/623: dread d0/d7/dc/d30/f4e [4194304,4194304] 0 2024-02-15T18:27:12.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:10.172957+0000 mgr.smithi042.kfoihb (mgr.24435) 46 : audit [DBG] from='client.34100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:12.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:10.588513+0000 mgr.smithi042.kfoihb (mgr.24435) 47 : audit [DBG] from='client.34104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:12.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:10.683910+0000 mgr.smithi042.kfoihb (mgr.24435) 48 : cluster [DBG] pgmap v34: 65 pgs: 65 active+clean; 2.7 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 113 MiB/s rd, 254 MiB/s wr, 415 op/s 2024-02-15T18:27:12.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:11.452498+0000 mon.smithi042 (mon.0) 13 : audit [DBG] from='client.? 172.21.15.42:0/598158119' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:12.077 INFO:tasks.workunit.client.1.smithi191.stdout:1/669: symlink d0/d8/df/d12/d2a/d2b/d78/d60/le1 0 2024-02-15T18:27:12.081 INFO:tasks.workunit.client.1.smithi191.stdout:2/588: creat df/d12/fc8 x:0 0 0 2024-02-15T18:27:12.081 INFO:tasks.workunit.client.1.smithi191.stdout:2/589: chown df/d12/d19/d1a/d35/d5b/f6e 654 1 2024-02-15T18:27:12.082 INFO:tasks.workunit.client.1.smithi191.stdout:2/590: write df/d12/d1f/f31 [311960,55907] 0 2024-02-15T18:27:12.082 INFO:tasks.workunit.client.1.smithi191.stdout:2/591: chown df/d12/d19/cc6 365 1 2024-02-15T18:27:12.082 INFO:tasks.workunit.client.1.smithi191.stdout:2/592: write df/d12/d19/d1a/d35/fa0 [1227572,37409] 0 2024-02-15T18:27:12.082 INFO:tasks.workunit.client.1.smithi191.stdout:2/593: fsync df/d12/f22 0 2024-02-15T18:27:12.082 INFO:tasks.workunit.client.1.smithi191.stdout:2/594: dread - df/d12/d19/d1a/d35/d47/d4b/d7a/f86 zero size 2024-02-15T18:27:12.088 INFO:tasks.workunit.client.1.smithi191.stdout:2/595: dread df/d12/f2b [0,4194304] 0 2024-02-15T18:27:12.092 INFO:tasks.workunit.client.1.smithi191.stdout:6/553: creat d8/d9/d2b/d57/d8e/d91/fa3 x:0 0 0 2024-02-15T18:27:12.097 INFO:tasks.workunit.client.1.smithi191.stdout:4/676: link d2/d5/d7/d3e/d63/cf2 d2/d5/d7/d3e/d63/df0/cfa 0 2024-02-15T18:27:12.101 INFO:tasks.workunit.client.1.smithi191.stdout:7/621: creat d0/db/d65/d11/fe9 x:0 0 0 2024-02-15T18:27:12.102 INFO:tasks.workunit.client.1.smithi191.stdout:9/624: unlink d0/d7/dc/d17/c9e 0 2024-02-15T18:27:12.110 INFO:tasks.workunit.client.1.smithi191.stdout:1/670: rename d0/d2f/d51/ldf to d0/d2f/d51/d10/d9c/le2 0 2024-02-15T18:27:12.114 INFO:tasks.workunit.client.1.smithi191.stdout:2/596: unlink cc 0 2024-02-15T18:27:12.117 INFO:tasks.workunit.client.1.smithi191.stdout:6/554: rmdir d8/d9/d61/d95 39 2024-02-15T18:27:12.118 INFO:tasks.workunit.client.1.smithi191.stdout:4/677: symlink d2/d5/d14/lfb 0 2024-02-15T18:27:12.118 INFO:tasks.workunit.client.1.smithi191.stdout:4/678: fdatasync d2/d5/d7/d90/d52/f7b 0 2024-02-15T18:27:12.118 INFO:tasks.workunit.client.1.smithi191.stdout:4/679: chown d2/d5/d14/d47/fc5 62871975 1 2024-02-15T18:27:12.119 INFO:tasks.workunit.client.1.smithi191.stdout:7/622: mkdir d0/db/ddd/dea 0 2024-02-15T18:27:12.119 INFO:tasks.workunit.client.1.smithi191.stdout:7/623: stat d0/db/f2d 0 2024-02-15T18:27:12.120 INFO:tasks.workunit.client.1.smithi191.stdout:9/625: rename d0/d7/dc/d17/f49 to d0/d29/d48/d66/d92/fd4 0 2024-02-15T18:27:12.123 INFO:tasks.workunit.client.1.smithi191.stdout:6/555: rename d8/d9/d4a/d4c/d69 to d8/d9/d4a/d4c/da4 0 2024-02-15T18:27:12.124 INFO:tasks.workunit.client.1.smithi191.stdout:6/556: write d8/d9/f86 [668365,97583] 0 2024-02-15T18:27:12.124 INFO:tasks.workunit.client.1.smithi191.stdout:2/597: dread df/d12/f36 [0,4194304] 0 2024-02-15T18:27:12.124 INFO:tasks.workunit.client.1.smithi191.stdout:2/598: truncate df/d61/d94/f98 66857 0 2024-02-15T18:27:12.125 INFO:tasks.workunit.client.1.smithi191.stdout:7/624: unlink d0/db/d65/d11/f18 0 2024-02-15T18:27:12.126 INFO:tasks.workunit.client.1.smithi191.stdout:9/626: symlink d0/d7/dc/d30/d3b/d56/d59/dce/ld5 0 2024-02-15T18:27:12.126 INFO:tasks.workunit.client.1.smithi191.stdout:6/557: rmdir d8/d9/d61 39 2024-02-15T18:27:12.127 INFO:tasks.workunit.client.1.smithi191.stdout:2/599: read f7 [723335,50459] 0 2024-02-15T18:27:12.128 INFO:tasks.workunit.client.1.smithi191.stdout:7/625: symlink d0/db/ddd/dea/leb 0 2024-02-15T18:27:12.129 INFO:tasks.workunit.client.1.smithi191.stdout:9/627: mknod d0/d29/d48/d66/cd6 0 2024-02-15T18:27:12.133 INFO:tasks.workunit.client.1.smithi191.stdout:7/626: dread d0/db/d65/d11/f90 [0,4194304] 0 2024-02-15T18:27:12.134 INFO:tasks.workunit.client.1.smithi191.stdout:7/627: write d0/db/d65/d11/d88/d9d/fda [3953326,10917] 0 2024-02-15T18:27:12.135 INFO:tasks.workunit.client.1.smithi191.stdout:9/628: creat d0/d7/dc/d30/d3b/d8b/fd7 x:0 0 0 2024-02-15T18:27:12.137 INFO:tasks.workunit.client.1.smithi191.stdout:7/628: creat d0/db/d65/d11/d88/fec x:0 0 0 2024-02-15T18:27:12.137 INFO:tasks.workunit.client.1.smithi191.stdout:1/671: dwrite d0/d2f/d51/d54/d7b/fb6 [0,4194304] 0 2024-02-15T18:27:12.137 INFO:tasks.workunit.client.1.smithi191.stdout:1/672: stat d0/d2f/d66/c7d 0 2024-02-15T18:27:12.137 INFO:tasks.workunit.client.1.smithi191.stdout:1/673: truncate d0/d8/df/d41/f9e 600991 0 2024-02-15T18:27:12.137 INFO:tasks.workunit.client.1.smithi191.stdout:1/674: chown d0/d2f/d51/f93 118 1 2024-02-15T18:27:12.137 INFO:tasks.workunit.client.1.smithi191.stdout:1/675: truncate d0/d8/df/d12/f28 4533998 0 2024-02-15T18:27:12.139 INFO:tasks.workunit.client.1.smithi191.stdout:7/629: truncate d0/db/d26/d5f/d77/f51 7240660 0 2024-02-15T18:27:12.142 INFO:tasks.workunit.client.1.smithi191.stdout:2/600: dread df/d12/d1f/f31 [0,4194304] 0 2024-02-15T18:27:12.143 INFO:tasks.workunit.client.1.smithi191.stdout:2/601: readlink df/d12/d1f/l56 0 2024-02-15T18:27:12.143 INFO:tasks.workunit.client.1.smithi191.stdout:1/676: rename d0/d8/df/d12/d33/fca to d0/d8/df/d12/d2a/fe3 0 2024-02-15T18:27:12.144 INFO:tasks.workunit.client.1.smithi191.stdout:1/677: chown d0/d8/d48 0 1 2024-02-15T18:27:12.145 INFO:tasks.workunit.client.1.smithi191.stdout:7/630: dread d0/db/d26/d5f/fd5 [0,4194304] 0 2024-02-15T18:27:12.145 INFO:tasks.workunit.client.1.smithi191.stdout:4/680: dwrite d2/d5/d14/d47/fc5 [0,4194304] 0 2024-02-15T18:27:12.145 INFO:tasks.workunit.client.1.smithi191.stdout:2/602: symlink df/d61/d9f/da3/lc9 0 2024-02-15T18:27:12.146 INFO:tasks.workunit.client.1.smithi191.stdout:2/603: write df/d12/d32/f92 [947887,121499] 0 2024-02-15T18:27:12.146 INFO:tasks.workunit.client.1.smithi191.stdout:6/558: dwrite d8/d9/d4a/d4c/f53 [0,4194304] 0 2024-02-15T18:27:12.147 INFO:tasks.workunit.client.1.smithi191.stdout:1/678: rename d0/d8/f35 to d0/d2f/d51/d5e/fe4 0 2024-02-15T18:27:12.147 INFO:tasks.workunit.client.1.smithi191.stdout:1/679: readlink d0/d8/ldb 0 2024-02-15T18:27:12.148 INFO:tasks.workunit.client.1.smithi191.stdout:1/680: dread d0/d8/df/d12/d33/d49/f4a [0,4194304] 0 2024-02-15T18:27:12.148 INFO:tasks.workunit.client.1.smithi191.stdout:1/681: fsync d0/d2f/d5c/f71 0 2024-02-15T18:27:12.149 INFO:tasks.workunit.client.1.smithi191.stdout:7/631: mknod d0/db/d26/d5f/d77/df/d5d/ced 0 2024-02-15T18:27:12.154 INFO:tasks.workunit.client.1.smithi191.stdout:4/681: creat d2/d5/d7/dee/ffc x:0 0 0 2024-02-15T18:27:12.155 INFO:tasks.workunit.client.1.smithi191.stdout:2/604: truncate df/d12/d32/fa9 2075601 0 2024-02-15T18:27:12.155 INFO:tasks.workunit.client.1.smithi191.stdout:2/605: write df/d12/d19/d1a/d52/fc0 [509409,96323] 0 2024-02-15T18:27:12.174 INFO:tasks.workunit.client.1.smithi191.stdout:6/559: creat d8/fa5 x:0 0 0 2024-02-15T18:27:12.175 INFO:tasks.workunit.client.1.smithi191.stdout:1/682: creat d0/d8/df/d2e/fe5 x:0 0 0 2024-02-15T18:27:12.175 INFO:tasks.workunit.client.1.smithi191.stdout:1/683: read - d0/d8/df/d12/fd4 zero size 2024-02-15T18:27:12.175 INFO:tasks.workunit.client.1.smithi191.stdout:1/684: write d0/d8/df/d90/d94/fba [934839,89857] 0 2024-02-15T18:27:12.175 INFO:tasks.workunit.client.1.smithi191.stdout:1/685: readlink d0/d8/d48/lad 0 2024-02-15T18:27:12.176 INFO:tasks.workunit.client.1.smithi191.stdout:2/606: dwrite df/d12/d19/d1a/d35/d5b/f6e [0,4194304] 0 2024-02-15T18:27:12.177 INFO:tasks.workunit.client.1.smithi191.stdout:9/629: dwrite d0/d7/dc/d8c/f27 [0,4194304] 0 2024-02-15T18:27:12.177 INFO:tasks.workunit.client.1.smithi191.stdout:9/630: fsync d0/d7/dc/fbb 0 2024-02-15T18:27:12.178 INFO:tasks.workunit.client.1.smithi191.stdout:7/632: link d0/db/d65/d11/fc4 d0/db/d65/dc/fee 0 2024-02-15T18:27:12.178 INFO:tasks.workunit.client.1.smithi191.stdout:7/633: chown d0/db/d65/d11/d5a/d66 1969 1 2024-02-15T18:27:12.182 INFO:tasks.workunit.client.1.smithi191.stdout:4/682: mknod d2/d5/d14/d47/d78/cfd 0 2024-02-15T18:27:12.185 INFO:tasks.workunit.client.1.smithi191.stdout:6/560: rename d8/d9/d61/d95/d9d/c24 to d8/d9/d61/ca6 0 2024-02-15T18:27:12.185 INFO:tasks.workunit.client.1.smithi191.stdout:6/561: fsync d8/d9/d4a/d4c/f9e 0 2024-02-15T18:27:12.185 INFO:tasks.workunit.client.1.smithi191.stdout:1/686: creat d0/d8/df/d90/d94/fe6 x:0 0 0 2024-02-15T18:27:12.186 INFO:tasks.workunit.client.1.smithi191.stdout:2/607: creat df/d12/fca x:0 0 0 2024-02-15T18:27:12.189 INFO:tasks.workunit.client.1.smithi191.stdout:7/634: dread d0/db/d65/d11/d88/d9d/fda [0,4194304] 0 2024-02-15T18:27:12.189 INFO:tasks.workunit.client.1.smithi191.stdout:9/631: creat d0/d7/dc/d17/d44/fd8 x:0 0 0 2024-02-15T18:27:12.190 INFO:tasks.workunit.client.1.smithi191.stdout:7/635: dread d0/db/d26/d5f/d77/f97 [0,4194304] 0 2024-02-15T18:27:12.191 INFO:tasks.workunit.client.1.smithi191.stdout:3/505: sync 2024-02-15T18:27:12.191 INFO:tasks.workunit.client.1.smithi191.stdout:0/398: sync 2024-02-15T18:27:12.191 INFO:tasks.workunit.client.1.smithi191.stdout:8/704: sync 2024-02-15T18:27:12.193 INFO:tasks.workunit.client.1.smithi191.stdout:4/683: mknod d2/d5/d7/cfe 0 2024-02-15T18:27:12.195 INFO:tasks.workunit.client.1.smithi191.stdout:6/562: symlink d8/d9/d2b/d5b/d9f/la7 0 2024-02-15T18:27:12.195 INFO:tasks.workunit.client.1.smithi191.stdout:6/563: stat d8/d9/d4a/c9b 0 2024-02-15T18:27:12.195 INFO:tasks.workunit.client.1.smithi191.stdout:6/564: truncate d8/f97 4911362 0 2024-02-15T18:27:12.197 INFO:tasks.workunit.client.1.smithi191.stdout:1/687: mkdir d0/de7 0 2024-02-15T18:27:12.197 INFO:tasks.workunit.client.1.smithi191.stdout:1/688: chown d0/d2f/d51/d10/f25 8994904 1 2024-02-15T18:27:12.197 INFO:tasks.workunit.client.1.smithi191.stdout:1/689: truncate d0/d8/df/d12/d33/d49/f4a 699977 0 2024-02-15T18:27:12.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:27:10.172957+0000 mgr.smithi042.kfoihb (mgr.24435) 46 : audit [DBG] from='client.34100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:12.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:27:10.588513+0000 mgr.smithi042.kfoihb (mgr.24435) 47 : audit [DBG] from='client.34104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:12.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:27:10.683910+0000 mgr.smithi042.kfoihb (mgr.24435) 48 : cluster [DBG] pgmap v34: 65 pgs: 65 active+clean; 2.7 GiB data, 7.9 GiB used, 529 GiB / 536 GiB avail; 113 MiB/s rd, 254 MiB/s wr, 415 op/s 2024-02-15T18:27:12.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:27:11.452498+0000 mon.smithi042 (mon.0) 13 : audit [DBG] from='client.? 172.21.15.42:0/598158119' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:12.199 INFO:tasks.workunit.client.1.smithi191.stdout:2/608: mknod df/d12/d19/d1a/d35/d47/d4b/d7a/ccb 0 2024-02-15T18:27:12.199 INFO:tasks.workunit.client.1.smithi191.stdout:2/609: dread - df/d12/fc8 zero size 2024-02-15T18:27:12.203 INFO:tasks.workunit.client.1.smithi191.stdout:9/632: unlink d0/d7/dc/d32/c9d 0 2024-02-15T18:27:12.205 INFO:tasks.workunit.client.1.smithi191.stdout:7/636: mkdir d0/db/d65/dc/d4b/dc1/def 0 2024-02-15T18:27:12.206 INFO:tasks.workunit.client.1.smithi191.stdout:3/506: readlink d6/d3b/d42/d49/d54/l95 0 2024-02-15T18:27:12.207 INFO:tasks.workunit.client.1.smithi191.stdout:8/705: mknod d3/d2c/d1a/d23/d2e/d5b/d69/cfa 0 2024-02-15T18:27:12.207 INFO:tasks.workunit.client.1.smithi191.stdout:8/706: write d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/ff4 [178077,20688] 0 2024-02-15T18:27:12.209 INFO:tasks.workunit.client.1.smithi191.stdout:1/690: rename d0/d2f/d5c/lcd to d0/d8/df/d12/le8 0 2024-02-15T18:27:12.210 INFO:tasks.workunit.client.1.smithi191.stdout:9/633: creat d0/d1d/d3f/fd9 x:0 0 0 2024-02-15T18:27:12.211 INFO:tasks.workunit.client.1.smithi191.stdout:7/637: mkdir d0/db/d65/dc/d4b/de8/df0 0 2024-02-15T18:27:12.212 INFO:tasks.workunit.client.1.smithi191.stdout:9/634: symlink d0/d7/dc/d17/lda 0 2024-02-15T18:27:12.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:12.216+0000 7fcb256a7700 1 -- 172.21.15.42:0/1589686770 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb00000bc0 con 0x7fcb08060fb0 2024-02-15T18:27:12.213 INFO:tasks.workunit.client.1.smithi191.stdout:7/638: mkdir d0/db/d65/df1 0 2024-02-15T18:27:12.213 INFO:tasks.workunit.client.1.smithi191.stdout:7/639: chown d0/db/d26/d5f/d77/d35/d42/fa7 62 1 2024-02-15T18:27:12.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:12.217+0000 7fcb077fe700 1 -- 172.21.15.42:0/1589686770 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fcb00000bc0 con 0x7fcb08060fb0 2024-02-15T18:27:12.214 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:27:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:27:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:27:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:27:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:27:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-02-15T18:27:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:27:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "3/21 daemons upgraded", 2024-02-15T18:27:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mon daemons", 2024-02-15T18:27:12.214 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:27:12.214 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:27:12.215 INFO:tasks.workunit.client.1.smithi191.stdout:9/635: symlink d0/d7/ldb 0 2024-02-15T18:27:12.215 INFO:tasks.workunit.client.1.smithi191.stdout:9/636: readlink d0/d29/d48/d66/d92/lc6 0 2024-02-15T18:27:12.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:12.220+0000 7fcb256a7700 1 -- 172.21.15.42:0/1589686770 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb08060fb0 msgr2=0x7fcb08063470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:12.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:12.220+0000 7fcb256a7700 1 --2- 172.21.15.42:0/1589686770 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb08060fb0 0x7fcb08063470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fcb14004630 tx=0x7fcb14009b30).stop 2024-02-15T18:27:12.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:12.220+0000 7fcb256a7700 1 -- 172.21.15.42:0/1589686770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb200f55c0 msgr2=0x7fcb2010c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:12.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:12.220+0000 7fcb256a7700 1 --2- 172.21.15.42:0/1589686770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb200f55c0 0x7fcb2010c130 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fcb100093f0 tx=0x7fcb10006ec0).stop 2024-02-15T18:27:12.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:12.221+0000 7fcb256a7700 1 -- 172.21.15.42:0/1589686770 shutdown_connections 2024-02-15T18:27:12.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:12.221+0000 7fcb256a7700 1 --2- 172.21.15.42:0/1589686770 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb08060fb0 0x7fcb08063470 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:12.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:12.221+0000 7fcb256a7700 1 --2- 172.21.15.42:0/1589686770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb200f55c0 0x7fcb2010c130 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0x7fcb100093f0 tx=0x7fcb10006ec0).stop 2024-02-15T18:27:12.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:12.221+0000 7fcb256a7700 1 --2- 172.21.15.42:0/1589686770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb200f6860 0x7fcb2010e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:12.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:12.221+0000 7fcb256a7700 1 -- 172.21.15.42:0/1589686770 >> 172.21.15.42:0/1589686770 conn(0x7fcb200f0d40 msgr2=0x7fcb200fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:12.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:12.221+0000 7fcb256a7700 1 -- 172.21.15.42:0/1589686770 shutdown_connections 2024-02-15T18:27:12.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:12.221+0000 7fcb256a7700 1 -- 172.21.15.42:0/1589686770 wait complete. 2024-02-15T18:27:12.223 INFO:tasks.workunit.client.1.smithi191.stdout:9/637: write d0/d29/d48/f5b [3697449,15777] 0 2024-02-15T18:27:12.223 INFO:tasks.workunit.client.1.smithi191.stdout:3/507: dread d6/d27/d4d/f58 [0,4194304] 0 2024-02-15T18:27:12.223 INFO:tasks.workunit.client.1.smithi191.stdout:3/508: chown c1 94 1 2024-02-15T18:27:12.223 INFO:tasks.workunit.client.1.smithi191.stdout:4/684: dwrite d2/d5/d7/d2f/fc2 [0,4194304] 0 2024-02-15T18:27:12.224 INFO:tasks.workunit.client.1.smithi191.stdout:3/509: truncate d6/d3b/d30/f45 714038 0 2024-02-15T18:27:12.228 INFO:tasks.workunit.client.1.smithi191.stdout:4/685: symlink d2/d5/d7/d3e/d63/d8c/dbb/ddc/lff 0 2024-02-15T18:27:12.229 INFO:tasks.workunit.client.1.smithi191.stdout:5/359: sync 2024-02-15T18:27:12.229 INFO:tasks.workunit.client.1.smithi191.stdout:4/686: write d2/d5/d8b/feb [1281734,48873] 0 2024-02-15T18:27:12.230 INFO:tasks.workunit.client.1.smithi191.stdout:3/510: symlink d6/d1b/d60/lad 0 2024-02-15T18:27:12.230 INFO:tasks.workunit.client.1.smithi191.stdout:4/687: read d2/d5/d86/f9d [1931449,59633] 0 2024-02-15T18:27:12.230 INFO:tasks.workunit.client.1.smithi191.stdout:4/688: read - d2/d5/d7/d3e/d63/f7d zero size 2024-02-15T18:27:12.233 INFO:tasks.workunit.client.1.smithi191.stdout:5/360: symlink d0/d23/d41/d28/d31/d5a/d61/l6f 0 2024-02-15T18:27:12.235 INFO:tasks.workunit.client.1.smithi191.stdout:3/511: stat d6/d3b/c80 0 2024-02-15T18:27:12.236 INFO:tasks.workunit.client.1.smithi191.stdout:6/565: dwrite d8/d9/d4a/f9a [0,4194304] 0 2024-02-15T18:27:12.236 INFO:tasks.workunit.client.1.smithi191.stdout:6/566: fsync d8/d9/d2b/d57/f60 0 2024-02-15T18:27:12.236 INFO:tasks.workunit.client.1.smithi191.stdout:1/691: dwrite d0/fa6 [0,4194304] 0 2024-02-15T18:27:12.236 INFO:tasks.workunit.client.1.smithi191.stdout:5/361: creat d0/d62/f70 x:0 0 0 2024-02-15T18:27:12.237 INFO:tasks.workunit.client.1.smithi191.stdout:0/399: dwrite d7/f63 [4194304,4194304] 0 2024-02-15T18:27:12.237 INFO:tasks.workunit.client.1.smithi191.stdout:1/692: dread d0/d2f/d51/d10/f59 [0,4194304] 0 2024-02-15T18:27:12.238 INFO:tasks.workunit.client.1.smithi191.stdout:2/610: dwrite df/d12/d1f/f31 [0,4194304] 0 2024-02-15T18:27:12.238 INFO:tasks.workunit.client.1.smithi191.stdout:2/611: stat df/d12/d19/f77 0 2024-02-15T18:27:12.238 INFO:tasks.workunit.client.1.smithi191.stdout:3/512: creat d6/d1b/d82/d9d/fae x:0 0 0 2024-02-15T18:27:12.238 INFO:tasks.workunit.client.1.smithi191.stdout:8/707: dwrite d3/d2c/d1a/d23/d2e/d47/feb [0,4194304] 0 2024-02-15T18:27:12.238 INFO:tasks.workunit.client.1.smithi191.stdout:8/708: dread - d3/d2c/d1a/fce zero size 2024-02-15T18:27:12.239 INFO:tasks.workunit.client.1.smithi191.stdout:6/567: write d8/d9/d61/d95/d9d/f26 [3430047,41992] 0 2024-02-15T18:27:12.244 INFO:tasks.workunit.client.1.smithi191.stdout:1/693: write d0/d2f/d51/fe [3757023,20280] 0 2024-02-15T18:27:12.247 INFO:tasks.workunit.client.1.smithi191.stdout:4/689: dwrite d2/d5/d7/fa9 [4194304,4194304] 0 2024-02-15T18:27:12.248 INFO:tasks.workunit.client.1.smithi191.stdout:6/568: dread d8/d9/f44 [0,4194304] 0 2024-02-15T18:27:12.256 INFO:tasks.workunit.client.1.smithi191.stdout:5/362: chown d0/d23/d41/d28/c29 23973827 1 2024-02-15T18:27:12.256 INFO:tasks.workunit.client.1.smithi191.stdout:5/363: chown d0/d23/d41/d28/d31/f53 17585 1 2024-02-15T18:27:12.260 INFO:tasks.workunit.client.1.smithi191.stdout:0/400: link d7/d1b/l75 d7/d4a/l8a 0 2024-02-15T18:27:12.269 INFO:tasks.workunit.client.1.smithi191.stdout:2/612: unlink df/d12/d1e/c33 0 2024-02-15T18:27:12.270 INFO:tasks.workunit.client.1.smithi191.stdout:3/513: write d6/d3b/d42/f51 [158788,77925] 0 2024-02-15T18:27:12.270 INFO:tasks.workunit.client.1.smithi191.stdout:3/514: chown d6/d3b/d42/d49/d54 253045239 1 2024-02-15T18:27:12.276 INFO:tasks.workunit.client.1.smithi191.stdout:8/709: rename d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/ff2 to d3/d2c/d1a/d23/d2e/ffb 0 2024-02-15T18:27:12.276 INFO:tasks.workunit.client.1.smithi191.stdout:8/710: chown d3/d2c/d1a/d23/d3e/f99 3182683 1 2024-02-15T18:27:12.276 INFO:tasks.workunit.client.1.smithi191.stdout:8/711: truncate d3/d2c/d1a/d23/d2e/d47/fbf 873436 0 2024-02-15T18:27:12.279 INFO:tasks.workunit.client.1.smithi191.stdout:6/569: creat d8/d9/d2b/d5b/d9f/fa8 x:0 0 0 2024-02-15T18:27:12.280 INFO:tasks.workunit.client.1.smithi191.stdout:8/712: dread d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/ff4 [0,4194304] 0 2024-02-15T18:27:12.280 INFO:tasks.workunit.client.1.smithi191.stdout:8/713: dread - d3/d2c/d1a/d23/d2e/fdf zero size 2024-02-15T18:27:12.282 INFO:tasks.workunit.client.1.smithi191.stdout:5/364: creat d0/d18/d5c/f71 x:0 0 0 2024-02-15T18:27:12.283 INFO:tasks.workunit.client.1.smithi191.stdout:0/401: symlink d7/d3b/l8b 0 2024-02-15T18:27:12.283 INFO:tasks.workunit.client.1.smithi191.stdout:0/402: write d7/da/d1a/f58 [306829,126390] 0 2024-02-15T18:27:12.286 INFO:tasks.workunit.client.1.smithi191.stdout:2/613: dwrite df/d12/fc8 [0,4194304] 0 2024-02-15T18:27:12.286 INFO:tasks.workunit.client.1.smithi191.stdout:2/614: write df/d12/f22 [5006403,59982] 0 2024-02-15T18:27:12.288 INFO:tasks.workunit.client.1.smithi191.stdout:2/615: write df/d12/d19/d1a/f3d [3021136,94645] 0 2024-02-15T18:27:12.288 INFO:tasks.workunit.client.1.smithi191.stdout:2/616: chown df/d12/d19/d1a/d52/c7d 1 1 2024-02-15T18:27:12.289 INFO:tasks.workunit.client.1.smithi191.stdout:3/515: creat d6/d3b/d42/d49/d54/faf x:0 0 0 2024-02-15T18:27:12.289 INFO:tasks.workunit.client.1.smithi191.stdout:3/516: write d6/d27/fa5 [122510,79678] 0 2024-02-15T18:27:12.291 INFO:tasks.workunit.client.1.smithi191.stdout:6/570: rename d8/d9/d61/d95/d9d/f26 to d8/d9/d2b/d57/fa9 0 2024-02-15T18:27:12.293 INFO:tasks.workunit.client.1.smithi191.stdout:5/365: unlink d0/d23/d41/d32/d52/l44 0 2024-02-15T18:27:12.295 INFO:tasks.workunit.client.1.smithi191.stdout:2/617: mkdir df/d12/d19/d1a/d35/d47/d4b/d7a/dcc 0 2024-02-15T18:27:12.296 INFO:tasks.workunit.client.1.smithi191.stdout:3/517: unlink d6/d1b/c40 0 2024-02-15T18:27:12.297 INFO:tasks.workunit.client.1.smithi191.stdout:6/571: creat d8/d9/faa x:0 0 0 2024-02-15T18:27:12.298 INFO:tasks.workunit.client.1.smithi191.stdout:5/366: symlink d0/d23/d41/d32/l72 0 2024-02-15T18:27:12.298 INFO:tasks.workunit.client.1.smithi191.stdout:2/618: write df/d12/d19/d1a/f44 [2665523,416] 0 2024-02-15T18:27:12.299 INFO:tasks.workunit.client.1.smithi191.stdout:5/367: mkdir d0/d18/d5c/d73 0 2024-02-15T18:27:12.299 INFO:tasks.workunit.client.1.smithi191.stdout:5/368: fsync d0/d23/d41/d32/d52/f40 0 2024-02-15T18:27:12.300 INFO:tasks.workunit.client.1.smithi191.stdout:8/714: dwrite d3/d2c/d1a/d23/d2e/d64/f76 [0,4194304] 0 2024-02-15T18:27:12.301 INFO:tasks.workunit.client.1.smithi191.stdout:8/715: write d3/d2c/f75 [2388176,39905] 0 2024-02-15T18:27:12.301 INFO:tasks.workunit.client.1.smithi191.stdout:8/716: fdatasync d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/f96 0 2024-02-15T18:27:12.301 INFO:tasks.workunit.client.1.smithi191.stdout:5/369: dread d0/d23/d41/d28/f4a [0,4194304] 0 2024-02-15T18:27:12.301 INFO:tasks.workunit.client.1.smithi191.stdout:5/370: write d0/d23/d41/d28/d31/f47 [484161,122173] 0 2024-02-15T18:27:12.304 INFO:tasks.workunit.client.1.smithi191.stdout:2/619: rename df/d12/d1f/l76 to df/d61/d9f/lcd 0 2024-02-15T18:27:12.305 INFO:tasks.workunit.client.1.smithi191.stdout:2/620: dread - df/d12/d19/d1a/d35/f74 zero size 2024-02-15T18:27:12.305 INFO:tasks.workunit.client.1.smithi191.stdout:2/621: dread - df/d12/d19/d1a/d35/d47/d4b/d7a/f8c zero size 2024-02-15T18:27:12.307 INFO:tasks.workunit.client.1.smithi191.stdout:8/717: creat d3/d2c/d1a/d23/d2e/de9/ffc x:0 0 0 2024-02-15T18:27:12.309 INFO:tasks.workunit.client.1.smithi191.stdout:5/371: creat d0/d23/d41/d28/d31/f74 x:0 0 0 2024-02-15T18:27:12.310 INFO:tasks.workunit.client.1.smithi191.stdout:2/622: mkdir df/d12/d19/d1a/d35/d47/d4b/d7a/dce 0 2024-02-15T18:27:12.313 INFO:tasks.workunit.client.1.smithi191.stdout:5/372: symlink d0/d18/l75 0 2024-02-15T18:27:12.313 INFO:tasks.workunit.client.1.smithi191.stdout:2/623: symlink df/d12/d32/lcf 0 2024-02-15T18:27:12.313 INFO:tasks.workunit.client.1.smithi191.stdout:2/624: fsync df/d12/d19/d1a/d35/d47/d4b/f7e 0 2024-02-15T18:27:12.313 INFO:tasks.workunit.client.1.smithi191.stdout:2/625: dread - df/d12/d19/d1a/d35/d47/d4b/f7e zero size 2024-02-15T18:27:12.314 INFO:tasks.workunit.client.1.smithi191.stdout:5/373: mkdir d0/d23/d48/d49/d76 0 2024-02-15T18:27:12.316 INFO:tasks.workunit.client.1.smithi191.stdout:2/626: mknod df/d12/d19/d1a/d35/d5b/dbf/cd0 0 2024-02-15T18:27:12.316 INFO:tasks.workunit.client.1.smithi191.stdout:2/627: truncate df/d12/d19/d1a/d35/d47/d4b/f7e 173176 0 2024-02-15T18:27:12.320 INFO:tasks.workunit.client.1.smithi191.stdout:2/628: link df/d12/d1f/d42/ca4 df/d12/cd1 0 2024-02-15T18:27:12.320 INFO:tasks.workunit.client.1.smithi191.stdout:2/629: fsync df/d12/f36 0 2024-02-15T18:27:12.325 INFO:tasks.workunit.client.1.smithi191.stdout:2/630: dread df/d12/d19/d1a/d35/fa0 [0,4194304] 0 2024-02-15T18:27:12.331 INFO:tasks.workunit.client.1.smithi191.stdout:5/374: getdents d0/d23/d41/d28/d31 0 2024-02-15T18:27:12.331 INFO:tasks.workunit.client.1.smithi191.stdout:5/375: write d0/d18/d5c/f6d [972420,25339] 0 2024-02-15T18:27:12.331 INFO:tasks.workunit.client.1.smithi191.stdout:5/376: fdatasync d0/d23/d41/d32/f54 0 2024-02-15T18:27:12.333 INFO:tasks.workunit.client.1.smithi191.stdout:2/631: mknod df/d12/d19/d1a/d35/d47/d4b/d7a/cd2 0 2024-02-15T18:27:12.337 INFO:tasks.workunit.client.1.smithi191.stdout:5/377: write d0/f1b [817253,96180] 0 2024-02-15T18:27:12.338 INFO:tasks.workunit.client.1.smithi191.stdout:5/378: getdents d0/d23/d41 0 2024-02-15T18:27:12.340 INFO:tasks.workunit.client.1.smithi191.stdout:8/718: dwrite d3/fa [0,4194304] 0 2024-02-15T18:27:12.341 INFO:tasks.workunit.client.1.smithi191.stdout:5/379: unlink d0/d23/d41/d28/d31/f74 0 2024-02-15T18:27:12.341 INFO:tasks.workunit.client.1.smithi191.stdout:2/632: dread df/d12/d32/fa9 [0,4194304] 0 2024-02-15T18:27:12.341 INFO:tasks.workunit.client.1.smithi191.stdout:8/719: read d3/d2c/d1a/d23/d3e/f68 [2406848,61663] 0 2024-02-15T18:27:12.341 INFO:tasks.workunit.client.1.smithi191.stdout:8/720: write d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d8e/f90 [591358,60941] 0 2024-02-15T18:27:12.342 INFO:tasks.workunit.client.1.smithi191.stdout:8/721: rmdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/de4 0 2024-02-15T18:27:12.343 INFO:tasks.workunit.client.1.smithi191.stdout:5/380: write d0/d18/f2e [222666,90535] 0 2024-02-15T18:27:12.345 INFO:tasks.workunit.client.1.smithi191.stdout:2/633: rename df/d12/d32/c8a to df/cd3 0 2024-02-15T18:27:12.345 INFO:tasks.workunit.client.1.smithi191.stdout:2/634: stat df/d12/d1e/l24 0 2024-02-15T18:27:12.346 INFO:tasks.workunit.client.1.smithi191.stdout:8/722: unlink d3/d2c/d1a/d23/d3e/cde 0 2024-02-15T18:27:12.349 INFO:tasks.workunit.client.1.smithi191.stdout:5/381: dread d0/d18/f42 [0,4194304] 0 2024-02-15T18:27:12.351 INFO:tasks.workunit.client.1.smithi191.stdout:5/382: write d0/d23/d41/d28/f3e [3008888,24271] 0 2024-02-15T18:27:12.351 INFO:tasks.workunit.client.1.smithi191.stdout:2/635: link df/d12/d1f/c30 df/d12/d1e/cd4 0 2024-02-15T18:27:12.352 INFO:tasks.workunit.client.1.smithi191.stdout:8/723: unlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/l4a 0 2024-02-15T18:27:12.353 INFO:tasks.workunit.client.1.smithi191.stdout:5/383: unlink d0/d18/f45 0 2024-02-15T18:27:12.354 INFO:tasks.workunit.client.1.smithi191.stdout:2/636: symlink df/d12/d1f/ld5 0 2024-02-15T18:27:12.355 INFO:tasks.workunit.client.1.smithi191.stdout:8/724: rename d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/lcd to d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/da0/lfd 0 2024-02-15T18:27:12.357 INFO:tasks.workunit.client.1.smithi191.stdout:2/637: mkdir df/d61/d9f/da3/dab/dd6 0 2024-02-15T18:27:12.359 INFO:tasks.workunit.client.1.smithi191.stdout:8/725: unlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/da0/lb0 0 2024-02-15T18:27:12.363 INFO:tasks.workunit.client.1.smithi191.stdout:8/726: rename d3/d2c/d1a/d23/d2e/f56 to d3/ffe 0 2024-02-15T18:27:12.364 INFO:tasks.workunit.client.1.smithi191.stdout:8/727: mkdir d3/d2c/d1a/d23/d3e/db2/dff 0 2024-02-15T18:27:12.372 INFO:tasks.workunit.client.1.smithi191.stdout:5/384: truncate d0/f1b 3300122 0 2024-02-15T18:27:12.372 INFO:tasks.workunit.client.1.smithi191.stdout:5/385: fsync d0/d23/d48/f67 0 2024-02-15T18:27:12.373 INFO:tasks.workunit.client.1.smithi191.stdout:8/728: creat d3/d2c/d1a/d23/d2a/f100 x:0 0 0 2024-02-15T18:27:12.373 INFO:tasks.workunit.client.1.smithi191.stdout:8/729: truncate d3/d2c/d1a/d23/d3e/db2/fb9 703064 0 2024-02-15T18:27:12.374 INFO:tasks.workunit.client.1.smithi191.stdout:5/386: mknod d0/d23/d41/d28/c77 0 2024-02-15T18:27:12.374 INFO:tasks.workunit.client.1.smithi191.stdout:5/387: readlink d0/d23/d41/d32/d52/l4f 0 2024-02-15T18:27:12.379 INFO:tasks.workunit.client.1.smithi191.stdout:2/638: dwrite df/d12/d19/d1a/d35/d47/d4b/f7e [0,4194304] 0 2024-02-15T18:27:12.384 INFO:tasks.workunit.client.1.smithi191.stdout:2/639: symlink df/d12/d1f/ld7 0 2024-02-15T18:27:12.384 INFO:tasks.workunit.client.1.smithi191.stdout:2/640: chown df/d12/d32/lcf 56095 1 2024-02-15T18:27:12.385 INFO:tasks.workunit.client.1.smithi191.stdout:2/641: write df/d12/d19/d1a/f3d [2422258,78488] 0 2024-02-15T18:27:12.398 INFO:tasks.workunit.client.1.smithi191.stdout:8/730: dwrite d3/fa [0,4194304] 0 2024-02-15T18:27:12.476 INFO:tasks.workunit.client.1.smithi191.stdout:8/731: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/c101 0 2024-02-15T18:27:12.485 INFO:tasks.workunit.client.1.smithi191.stdout:8/732: creat d3/d5e/f102 x:0 0 0 2024-02-15T18:27:12.488 INFO:tasks.workunit.client.1.smithi191.stdout:8/733: chown d3/c10 19640 1 2024-02-15T18:27:12.490 INFO:tasks.workunit.client.1.smithi191.stdout:8/734: symlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/da0/l103 0 2024-02-15T18:27:12.491 INFO:tasks.workunit.client.1.smithi191.stdout:8/735: read d3/d2c/d1a/f81 [3126401,62691] 0 2024-02-15T18:27:12.492 INFO:tasks.workunit.client.1.smithi191.stdout:8/736: read d3/d2c/d1a/d23/d2e/d5b/d69/d6d/fc1 [96758,54469] 0 2024-02-15T18:27:12.492 INFO:tasks.workunit.client.1.smithi191.stdout:8/737: fsync d3/d2c/d1a/d23/d2e/d47/d5f/fc5 0 2024-02-15T18:27:12.493 INFO:tasks.workunit.client.1.smithi191.stdout:8/738: symlink d3/d5e/l104 0 2024-02-15T18:27:12.494 INFO:tasks.workunit.client.1.smithi191.stdout:8/739: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d8e/f105 x:0 0 0 2024-02-15T18:27:12.494 INFO:tasks.workunit.client.1.smithi191.stdout:8/740: read - d3/d2c/d1a/d23/d2e/d47/d5f/ff3 zero size 2024-02-15T18:27:12.495 INFO:tasks.workunit.client.1.smithi191.stdout:8/741: mkdir d3/d2c/d1a/d23/d2e/d64/dca/d106 0 2024-02-15T18:27:12.498 INFO:tasks.workunit.client.1.smithi191.stdout:8/742: rename d3/d2c/d1a/d23/d2e/f3a to d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/de8/f107 0 2024-02-15T18:27:12.498 INFO:tasks.workunit.client.1.smithi191.stdout:8/743: fdatasync d3/d2c/d1a/f81 0 2024-02-15T18:27:12.499 INFO:tasks.workunit.client.1.smithi191.stdout:8/744: symlink d3/d2c/d1a/d23/d3e/db2/l108 0 2024-02-15T18:27:12.499 INFO:tasks.workunit.client.1.smithi191.stdout:8/745: write d3/d2c/d1a/d23/d2e/d64/fc4 [948395,10992] 0 2024-02-15T18:27:12.500 INFO:tasks.workunit.client.1.smithi191.stdout:8/746: write d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/de8/f107 [3880620,54382] 0 2024-02-15T18:27:12.501 INFO:tasks.workunit.client.1.smithi191.stdout:8/747: rmdir d3/d2c/d1a/d23/d2e/d5b/d69/d6d 39 2024-02-15T18:27:12.501 INFO:tasks.workunit.client.1.smithi191.stdout:8/748: chown d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/fda 2106292 1 2024-02-15T18:27:12.502 INFO:tasks.workunit.client.1.smithi191.stdout:8/749: mkdir d3/d2c/d1a/d23/d2e/d64/dbc/d109 0 2024-02-15T18:27:12.507 INFO:tasks.workunit.client.1.smithi191.stdout:8/750: dread d3/d2c/d1a/d23/d2a/f6e [0,4194304] 0 2024-02-15T18:27:12.507 INFO:tasks.workunit.client.1.smithi191.stdout:8/751: fsync d3/d2c/fb5 0 2024-02-15T18:27:12.508 INFO:tasks.workunit.client.1.smithi191.stdout:8/752: link d3/d2c/d1a/d23/d2a/f6e d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/f10a 0 2024-02-15T18:27:12.515 INFO:tasks.workunit.client.1.smithi191.stdout:8/753: dread d3/d2c/d1a/fb1 [0,4194304] 0 2024-02-15T18:27:12.522 INFO:tasks.workunit.client.1.smithi191.stdout:8/754: dread d3/d2c/d1a/f81 [0,4194304] 0 2024-02-15T18:27:12.536 INFO:tasks.workunit.client.1.smithi191.stdout:8/755: dwrite d3/d2c/d1a/d23/d2e/d47/f88 [0,4194304] 0 2024-02-15T18:27:12.537 INFO:tasks.workunit.client.1.smithi191.stdout:8/756: dread d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/ff4 [0,4194304] 0 2024-02-15T18:27:12.537 INFO:tasks.workunit.client.1.smithi191.stdout:8/757: chown d3/d2c/d1a/d23/d2e/d47/d5f/f50 1219030 1 2024-02-15T18:27:12.600 INFO:tasks.workunit.client.1.smithi191.stdout:9/638: sync 2024-02-15T18:27:12.635 INFO:tasks.workunit.client.1.smithi191.stdout:7/640: sync 2024-02-15T18:27:12.636 INFO:tasks.workunit.client.1.smithi191.stdout:9/639: unlink d0/d7/dc/d30/d4f/faf 0 2024-02-15T18:27:12.637 INFO:tasks.workunit.client.1.smithi191.stdout:7/641: link d0/db/d26/d5f/d77/d35/d42/cad d0/db/d65/d11/d88/cf2 0 2024-02-15T18:27:12.637 INFO:tasks.workunit.client.1.smithi191.stdout:7/642: link d0/db/d65/f82 d0/db/ff3 0 2024-02-15T18:27:12.637 INFO:tasks.workunit.client.1.smithi191.stdout:7/643: write d0/db/d65/dc/fa0 [571411,38695] 0 2024-02-15T18:27:12.637 INFO:tasks.workunit.client.1.smithi191.stdout:9/640: link d0/d7/dc/d17/d44/fba d0/fdc 0 2024-02-15T18:27:12.637 INFO:tasks.workunit.client.1.smithi191.stdout:7/644: symlink d0/db/d65/df1/lf4 0 2024-02-15T18:27:12.637 INFO:tasks.workunit.client.1.smithi191.stdout:9/641: dread d0/d7/dc/d32/da3/fb6 [0,4194304] 0 2024-02-15T18:27:12.637 INFO:tasks.workunit.client.1.smithi191.stdout:9/642: read d0/d1d/f79 [205479,104852] 0 2024-02-15T18:27:12.637 INFO:tasks.workunit.client.1.smithi191.stdout:7/645: rename d0/db/d26/d5f/d77/d1d to d0/db/d65/d11/d5a/d66/d1e/df5 0 2024-02-15T18:27:12.637 INFO:tasks.workunit.client.1.smithi191.stdout:7/646: fdatasync d0/db/d65/dc/d4b/fb1 0 2024-02-15T18:27:12.639 INFO:tasks.workunit.client.1.smithi191.stdout:9/643: dwrite d0/d7/dc/d17/d44/fa4 [0,4194304] 0 2024-02-15T18:27:12.641 INFO:tasks.workunit.client.1.smithi191.stdout:7/647: rmdir d0/db/d65/d11/d5a 39 2024-02-15T18:27:12.641 INFO:tasks.workunit.client.1.smithi191.stdout:7/648: write d0/db/d65/d11/d57/fcf [293934,33290] 0 2024-02-15T18:27:12.641 INFO:tasks.workunit.client.1.smithi191.stdout:7/649: fdatasync d0/db/d26/d5f/d77/d35/d42/f6c 0 2024-02-15T18:27:12.649 INFO:tasks.workunit.client.1.smithi191.stdout:7/650: stat d0/l5e 0 2024-02-15T18:27:12.651 INFO:tasks.workunit.client.1.smithi191.stdout:7/651: dread d0/f80 [0,4194304] 0 2024-02-15T18:27:12.653 INFO:tasks.workunit.client.1.smithi191.stdout:7/652: rename d0/db/d65/d11/f90 to d0/db/d65/dc/d4b/dc1/def/ff6 0 2024-02-15T18:27:12.655 INFO:tasks.workunit.client.1.smithi191.stdout:7/653: read d0/db/d26/d5f/d77/f97 [523849,56860] 0 2024-02-15T18:27:12.656 INFO:tasks.workunit.client.1.smithi191.stdout:9/644: dwrite d0/d7/dc/d30/d3b/d8b/fd7 [0,4194304] 0 2024-02-15T18:27:12.661 INFO:tasks.workunit.client.1.smithi191.stdout:9/645: read d0/d29/f70 [724415,41165] 0 2024-02-15T18:27:12.669 INFO:tasks.workunit.client.1.smithi191.stdout:9/646: getdents d0/d7/dc/d17 0 2024-02-15T18:27:12.687 INFO:tasks.workunit.client.1.smithi191.stdout:9/647: dwrite d0/d7/dc/d8c/f6b [4194304,4194304] 0 2024-02-15T18:27:12.688 INFO:tasks.workunit.client.1.smithi191.stdout:9/648: truncate d0/d7/dc/d17/d44/fcc 787215 0 2024-02-15T18:27:12.690 INFO:tasks.workunit.client.1.smithi191.stdout:9/649: link d0/d7/dc/d32/da3/fb6 d0/d29/d48/fdd 0 2024-02-15T18:27:12.690 INFO:tasks.workunit.client.1.smithi191.stdout:9/650: chown d0/d7/dc/d17/d33/f77 0 1 2024-02-15T18:27:12.690 INFO:tasks.workunit.client.1.smithi191.stdout:9/651: truncate d0/d7/dc/d17/f99 150765 0 2024-02-15T18:27:12.708 INFO:tasks.workunit.client.1.smithi191.stdout:9/652: dwrite d0/d1d/d3f/fd9 [0,4194304] 0 2024-02-15T18:27:12.708 INFO:tasks.workunit.client.1.smithi191.stdout:9/653: fsync d0/d29/d48/d66/d92/fd4 0 2024-02-15T18:27:12.708 INFO:tasks.workunit.client.1.smithi191.stdout:9/654: getdents d0/d7/dc/d17 0 2024-02-15T18:27:12.712 INFO:tasks.workunit.client.1.smithi191.stdout:9/655: getdents d0/d29/d48/d66/d92 0 2024-02-15T18:27:12.720 INFO:tasks.workunit.client.1.smithi191.stdout:9/656: link d0/c94 d0/d29/d48/cde 0 2024-02-15T18:27:12.723 INFO:tasks.workunit.client.1.smithi191.stdout:9/657: mknod d0/d1d/d3f/d80/cdf 0 2024-02-15T18:27:12.723 INFO:tasks.workunit.client.1.smithi191.stdout:9/658: write d0/d29/d48/d66/d92/fca [82914,85069] 0 2024-02-15T18:27:12.725 INFO:tasks.workunit.client.1.smithi191.stdout:9/659: unlink d0/d7/dc/d32/l6c 0 2024-02-15T18:27:12.728 INFO:tasks.workunit.client.1.smithi191.stdout:9/660: rmdir d0/d7/dc/d30/d4f 39 2024-02-15T18:27:12.732 INFO:tasks.workunit.client.1.smithi191.stdout:9/661: rename d0/d29/d48/d66/d92/cad to d0/d7/dc/d17/d33/ce0 0 2024-02-15T18:27:12.738 INFO:tasks.workunit.client.1.smithi191.stdout:9/662: rename d0/d7/dc/d17/d33/f77 to d0/d7/dc/d32/d89/fe1 0 2024-02-15T18:27:12.738 INFO:tasks.workunit.client.1.smithi191.stdout:9/663: write d0/d7/dc/d17/d44/fcc [1076163,40719] 0 2024-02-15T18:27:12.739 INFO:tasks.workunit.client.1.smithi191.stdout:9/664: read d0/d7/dc/d32/da3/fb6 [1065935,21474] 0 2024-02-15T18:27:12.745 INFO:tasks.workunit.client.1.smithi191.stdout:9/665: dread d0/d29/f70 [0,4194304] 0 2024-02-15T18:27:12.745 INFO:tasks.workunit.client.1.smithi191.stdout:9/666: readlink d0/d29/d48/d66/l91 0 2024-02-15T18:27:12.761 INFO:tasks.workunit.client.1.smithi191.stdout:9/667: dwrite d0/d7/dc/d30/f5c [0,4194304] 0 2024-02-15T18:27:12.769 INFO:tasks.workunit.client.1.smithi191.stdout:9/668: chown d0/d7/dc/d17/d33/f40 0 1 2024-02-15T18:27:12.771 INFO:tasks.workunit.client.1.smithi191.stdout:9/669: mkdir d0/d29/de2 0 2024-02-15T18:27:12.787 INFO:tasks.workunit.client.1.smithi191.stdout:1/694: sync 2024-02-15T18:27:12.787 INFO:tasks.workunit.client.1.smithi191.stdout:4/690: sync 2024-02-15T18:27:12.790 INFO:tasks.workunit.client.1.smithi191.stdout:1/695: symlink d0/d8/df/d12/d2a/le9 0 2024-02-15T18:27:12.791 INFO:tasks.workunit.client.1.smithi191.stdout:4/691: mknod d2/d5/d7/d3e/d63/df0/c100 0 2024-02-15T18:27:12.793 INFO:tasks.workunit.client.1.smithi191.stdout:9/670: dwrite d0/d7/f21 [4194304,4194304] 0 2024-02-15T18:27:12.811 INFO:tasks.workunit.client.1.smithi191.stdout:9/671: dread d0/d7/f19 [0,4194304] 0 2024-02-15T18:27:12.816 INFO:tasks.workunit.client.1.smithi191.stdout:4/692: dwrite d2/fad [0,4194304] 0 2024-02-15T18:27:12.830 INFO:tasks.workunit.client.1.smithi191.stdout:4/693: dwrite d2/f5c [0,4194304] 0 2024-02-15T18:27:12.830 INFO:tasks.workunit.client.1.smithi191.stdout:4/694: readlink d2/d5/d7/d2f/ld9 0 2024-02-15T18:27:12.830 INFO:tasks.workunit.client.1.smithi191.stdout:4/695: fsync d2/f2c 0 2024-02-15T18:27:12.836 INFO:tasks.workunit.client.1.smithi191.stdout:4/696: dread d2/d5/d14/d71/fac [0,4194304] 0 2024-02-15T18:27:12.886 INFO:tasks.workunit.client.1.smithi191.stdout:9/672: creat d0/d1d/d3f/fe3 x:0 0 0 2024-02-15T18:27:12.913 INFO:tasks.workunit.client.1.smithi191.stdout:9/673: dwrite d0/d29/d48/f5b [0,4194304] 0 2024-02-15T18:27:12.945 INFO:tasks.workunit.client.1.smithi191.stdout:4/697: creat d2/d5/d7/d90/d52/f101 x:0 0 0 2024-02-15T18:27:12.951 INFO:tasks.workunit.client.1.smithi191.stdout:4/698: creat d2/d5/d14/d47/d48/f102 x:0 0 0 2024-02-15T18:27:12.955 INFO:tasks.workunit.client.1.smithi191.stdout:9/674: dread d0/d7/dc/d17/f61 [0,4194304] 0 2024-02-15T18:27:12.957 INFO:tasks.workunit.client.1.smithi191.stdout:0/403: sync 2024-02-15T18:27:12.957 INFO:tasks.workunit.client.1.smithi191.stdout:6/572: sync 2024-02-15T18:27:12.957 INFO:tasks.workunit.client.1.smithi191.stdout:3/518: sync 2024-02-15T18:27:12.958 INFO:tasks.workunit.client.1.smithi191.stdout:9/675: write d0/d7/dc/d17/d33/f43 [1380049,3850] 0 2024-02-15T18:27:12.958 INFO:tasks.workunit.client.1.smithi191.stdout:9/676: write d0/d29/d48/fc3 [752636,74815] 0 2024-02-15T18:27:12.960 INFO:tasks.workunit.client.1.smithi191.stdout:6/573: rename d8/d9/d4a/d4c/f64 to d8/fab 0 2024-02-15T18:27:12.960 INFO:tasks.workunit.client.1.smithi191.stdout:0/404: creat d7/da/d1a/d2c/f8c x:0 0 0 2024-02-15T18:27:12.960 INFO:tasks.workunit.client.1.smithi191.stdout:0/405: chown d7/da/d17/d31/d5f/l88 7 1 2024-02-15T18:27:12.962 INFO:tasks.workunit.client.1.smithi191.stdout:9/677: mkdir d0/d7/dc/d32/de4 0 2024-02-15T18:27:12.963 INFO:tasks.workunit.client.1.smithi191.stdout:0/406: mknod d7/d3b/c8d 0 2024-02-15T18:27:12.965 INFO:tasks.workunit.client.1.smithi191.stdout:6/574: mkdir d8/d9/dac 0 2024-02-15T18:27:12.966 INFO:tasks.workunit.client.1.smithi191.stdout:6/575: symlink d8/d9/d61/d95/lad 0 2024-02-15T18:27:12.968 INFO:tasks.workunit.client.1.smithi191.stdout:6/576: rmdir d8/d9/d2b/d5b/d9f 39 2024-02-15T18:27:12.983 INFO:tasks.workunit.client.1.smithi191.stdout:4/699: dwrite d2/d5/d7/d90/d52/f7b [0,4194304] 0 2024-02-15T18:27:12.985 INFO:tasks.workunit.client.1.smithi191.stdout:4/700: creat d2/d5/d14/d44/f103 x:0 0 0 2024-02-15T18:27:12.986 INFO:tasks.workunit.client.1.smithi191.stdout:4/701: symlink d2/d5/d14/d44/l104 0 2024-02-15T18:27:12.987 INFO:tasks.workunit.client.1.smithi191.stdout:4/702: mkdir d2/d5/d14/d47/d78/dd0/d105 0 2024-02-15T18:27:12.989 INFO:tasks.workunit.client.1.smithi191.stdout:3/519: dwrite d6/d3b/d42/d49/f59 [0,4194304] 0 2024-02-15T18:27:12.990 INFO:tasks.workunit.client.1.smithi191.stdout:3/520: mkdir d6/d1b/db0 0 2024-02-15T18:27:12.994 INFO:tasks.workunit.client.1.smithi191.stdout:9/678: dwrite d0/d7/dc/d30/d3b/d8b/fd7 [0,4194304] 0 2024-02-15T18:27:12.995 INFO:tasks.workunit.client.1.smithi191.stdout:4/703: write d2/f3 [2441131,111809] 0 2024-02-15T18:27:12.995 INFO:tasks.workunit.client.1.smithi191.stdout:4/704: fsync d2/d5/d14/d47/d78/f4f 0 2024-02-15T18:27:12.996 INFO:tasks.workunit.client.1.smithi191.stdout:9/679: rmdir d0/d29/d9a 0 2024-02-15T18:27:12.998 INFO:tasks.workunit.client.1.smithi191.stdout:9/680: mknod d0/d29/d48/d66/ce5 0 2024-02-15T18:27:13.004 INFO:tasks.workunit.client.1.smithi191.stdout:6/577: dwrite d8/d9/d2b/d57/f60 [0,4194304] 0 2024-02-15T18:27:13.005 INFO:tasks.workunit.client.1.smithi191.stdout:0/407: dwrite d7/d4a/f59 [8388608,4194304] 0 2024-02-15T18:27:13.005 INFO:tasks.workunit.client.1.smithi191.stdout:0/408: dread - d7/da/d1a/d2c/f32 zero size 2024-02-15T18:27:13.007 INFO:tasks.workunit.client.1.smithi191.stdout:9/681: rename d0/d7/dc/d32/d89 to d0/d7/dc/d17/d44/de6 0 2024-02-15T18:27:13.007 INFO:tasks.workunit.client.1.smithi191.stdout:9/682: dread - d0/d7/dc/fbb zero size 2024-02-15T18:27:13.007 INFO:tasks.workunit.client.1.smithi191.stdout:9/683: fdatasync d0/d7/dc/d8c/f72 0 2024-02-15T18:27:13.009 INFO:tasks.workunit.client.1.smithi191.stdout:6/578: mknod d8/d9/d2b/d57/d8e/d91/cae 0 2024-02-15T18:27:13.011 INFO:tasks.workunit.client.1.smithi191.stdout:0/409: creat d7/da/f8e x:0 0 0 2024-02-15T18:27:13.011 INFO:tasks.workunit.client.1.smithi191.stdout:0/410: fsync d7/da/d1a/f58 0 2024-02-15T18:27:13.012 INFO:tasks.workunit.client.1.smithi191.stdout:9/684: getdents d0/d7/dc/d30/d4f 0 2024-02-15T18:27:13.015 INFO:tasks.workunit.client.1.smithi191.stdout:6/579: write d8/d9/d2b/d57/f59 [27691,87806] 0 2024-02-15T18:27:13.015 INFO:tasks.workunit.client.1.smithi191.stdout:9/685: read d0/d1d/d3f/d80/fa2 [3370982,116354] 0 2024-02-15T18:27:13.016 INFO:tasks.workunit.client.1.smithi191.stdout:0/411: rename d7/da/d1a/d34/d27/l38 to d7/da/d1a/d2c/d4c/l8f 0 2024-02-15T18:27:13.019 INFO:tasks.workunit.client.1.smithi191.stdout:6/580: rmdir d8/d9/d4a/d4c/d7f 39 2024-02-15T18:27:13.022 INFO:tasks.workunit.client.1.smithi191.stdout:9/686: symlink d0/d7/dc/d32/le7 0 2024-02-15T18:27:13.022 INFO:tasks.workunit.client.1.smithi191.stdout:9/687: chown d0/d7/dc/d17/d33 6808976 1 2024-02-15T18:27:13.022 INFO:tasks.workunit.client.1.smithi191.stdout:3/521: dwrite d6/fa0 [0,4194304] 0 2024-02-15T18:27:13.022 INFO:tasks.workunit.client.1.smithi191.stdout:4/705: dwrite d2/d5/d7/dee/ffc [0,4194304] 0 2024-02-15T18:27:13.023 INFO:tasks.workunit.client.1.smithi191.stdout:3/522: write d6/d3b/d42/d49/d54/f83 [952471,17866] 0 2024-02-15T18:27:13.026 INFO:tasks.workunit.client.1.smithi191.stdout:6/581: write d8/f1c [2219636,84094] 0 2024-02-15T18:27:13.032 INFO:tasks.workunit.client.1.smithi191.stdout:4/706: truncate d2/d5/d86/f9d 1852525 0 2024-02-15T18:27:13.033 INFO:tasks.workunit.client.1.smithi191.stdout:3/523: creat d6/d27/d4d/da8/fb1 x:0 0 0 2024-02-15T18:27:13.035 INFO:tasks.workunit.client.1.smithi191.stdout:4/707: write d2/d5/d8b/fe5 [2345326,124640] 0 2024-02-15T18:27:13.040 INFO:tasks.workunit.client.1.smithi191.stdout:6/582: creat d8/d9/d2b/d57/d8e/faf x:0 0 0 2024-02-15T18:27:13.041 INFO:tasks.workunit.client.1.smithi191.stdout:3/524: truncate d6/d3b/d42/f2a 1747647 0 2024-02-15T18:27:13.043 INFO:tasks.workunit.client.1.smithi191.stdout:4/708: mkdir d2/d5/d7/d106 0 2024-02-15T18:27:13.043 INFO:tasks.workunit.client.1.smithi191.stdout:4/709: write d2/d5/d14/d47/d78/f68 [1462055,56547] 0 2024-02-15T18:27:13.043 INFO:tasks.workunit.client.1.smithi191.stdout:6/583: write d8/d9/d4a/d4c/f66 [1350154,107701] 0 2024-02-15T18:27:13.043 INFO:tasks.workunit.client.1.smithi191.stdout:0/412: dwrite d7/f2b [0,4194304] 0 2024-02-15T18:27:13.043 INFO:tasks.workunit.client.1.smithi191.stdout:4/710: creat d2/d5/d7/d3e/d63/d8c/f107 x:0 0 0 2024-02-15T18:27:13.053 INFO:tasks.workunit.client.1.smithi191.stdout:0/413: dread d7/da/d17/d31/d7d/f80 [0,4194304] 0 2024-02-15T18:27:13.053 INFO:tasks.workunit.client.1.smithi191.stdout:3/525: dread d6/d27/f32 [0,4194304] 0 2024-02-15T18:27:13.057 INFO:tasks.workunit.client.1.smithi191.stdout:6/584: rename d8/d9/d61 to d8/d9/dac/db0 0 2024-02-15T18:27:13.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:11.010617+0000 mgr.smithi042.kfoihb (mgr.24435) 49 : audit [DBG] from='client.24513 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:13.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:11.847790+0000 mon.smithi042 (mon.0) 14 : audit [DBG] from='client.? 172.21.15.42:0/2126640308' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:27:13.062 INFO:tasks.workunit.client.1.smithi191.stdout:9/688: dwrite d0/d7/dc/d17/d33/f40 [0,4194304] 0 2024-02-15T18:27:13.063 INFO:tasks.workunit.client.1.smithi191.stdout:0/414: unlink d7/da/f25 0 2024-02-15T18:27:13.063 INFO:tasks.workunit.client.1.smithi191.stdout:4/711: dwrite d2/d5/d7/d3e/f77 [0,4194304] 0 2024-02-15T18:27:13.063 INFO:tasks.workunit.client.1.smithi191.stdout:4/712: getdents d2/d5/d7/d90/d52/db5/dc9 0 2024-02-15T18:27:13.064 INFO:tasks.workunit.client.1.smithi191.stdout:3/526: rename d6/d1b/d82/d9d to d6/d1b/d82/db2 0 2024-02-15T18:27:13.071 INFO:tasks.workunit.client.1.smithi191.stdout:9/689: mknod d0/d7/dc/d30/d3b/d8b/ce8 0 2024-02-15T18:27:13.072 INFO:tasks.workunit.client.1.smithi191.stdout:0/415: creat d7/da/d1a/f90 x:0 0 0 2024-02-15T18:27:13.074 INFO:tasks.workunit.client.1.smithi191.stdout:4/713: dread d2/f3 [0,4194304] 0 2024-02-15T18:27:13.074 INFO:tasks.workunit.client.1.smithi191.stdout:4/714: chown d2/d5/d14/d47/la1 1032615 1 2024-02-15T18:27:13.076 INFO:tasks.workunit.client.1.smithi191.stdout:3/527: write d6/d3b/f13 [3860338,107163] 0 2024-02-15T18:27:13.076 INFO:tasks.workunit.client.1.smithi191.stdout:3/528: readlink d6/d1b/d60/l87 0 2024-02-15T18:27:13.077 INFO:tasks.workunit.client.1.smithi191.stdout:6/585: dwrite d8/d9/d2b/d57/f60 [0,4194304] 0 2024-02-15T18:27:13.077 INFO:tasks.workunit.client.1.smithi191.stdout:9/690: symlink d0/d7/dc/d17/d44/de6/le9 0 2024-02-15T18:27:13.080 INFO:tasks.workunit.client.1.smithi191.stdout:0/416: mknod d7/da/d1a/d2c/c91 0 2024-02-15T18:27:13.088 INFO:tasks.workunit.client.1.smithi191.stdout:4/715: getdents d2/d5/d7/d2f 0 2024-02-15T18:27:13.089 INFO:tasks.workunit.client.1.smithi191.stdout:3/529: unlink d6/d3b/d42/d49/l77 0 2024-02-15T18:27:13.091 INFO:tasks.workunit.client.1.smithi191.stdout:6/586: stat d8/d9/dac/db0/d95/d9d/d6b/la0 0 2024-02-15T18:27:13.092 INFO:tasks.workunit.client.1.smithi191.stdout:9/691: creat d0/d7/dc/d17/d44/de6/fea x:0 0 0 2024-02-15T18:27:13.094 INFO:tasks.workunit.client.1.smithi191.stdout:3/530: link d6/d27/f32 d6/d27/d4d/da8/fb3 0 2024-02-15T18:27:13.094 INFO:tasks.workunit.client.1.smithi191.stdout:3/531: fsync d6/d1b/f39 0 2024-02-15T18:27:13.094 INFO:tasks.workunit.client.1.smithi191.stdout:5/388: sync 2024-02-15T18:27:13.094 INFO:tasks.workunit.client.1.smithi191.stdout:2/642: sync 2024-02-15T18:27:13.095 INFO:tasks.workunit.client.1.smithi191.stdout:2/643: write df/d12/d19/d1a/f3d [6104859,74470] 0 2024-02-15T18:27:13.096 INFO:tasks.workunit.client.1.smithi191.stdout:6/587: link d8/d9/d2b/d57/f6c d8/d9/dac/db0/d95/d9d/d6b/d76/fb1 0 2024-02-15T18:27:13.098 INFO:tasks.workunit.client.1.smithi191.stdout:3/532: readlink d6/d3b/d42/l8a 0 2024-02-15T18:27:13.098 INFO:tasks.workunit.client.1.smithi191.stdout:3/533: stat d6/d27/d4d/d5a/da4 0 2024-02-15T18:27:13.098 INFO:tasks.workunit.client.1.smithi191.stdout:3/534: truncate d6/fa0 6715529 0 2024-02-15T18:27:13.098 INFO:tasks.workunit.client.1.smithi191.stdout:6/588: write d8/d9/d2b/d57/f59 [1421629,9987] 0 2024-02-15T18:27:13.103 INFO:tasks.workunit.client.1.smithi191.stdout:5/389: unlink d0/f46 0 2024-02-15T18:27:13.105 INFO:tasks.workunit.client.1.smithi191.stdout:9/692: link d0/d7/dc/d8c/f78 d0/d7/dc/d17/d44/feb 0 2024-02-15T18:27:13.108 INFO:tasks.workunit.client.1.smithi191.stdout:6/589: unlink d8/f15 0 2024-02-15T18:27:13.109 INFO:tasks.workunit.client.1.smithi191.stdout:0/417: dwrite d7/da/d1a/f90 [0,4194304] 0 2024-02-15T18:27:13.110 INFO:tasks.workunit.client.1.smithi191.stdout:4/716: dwrite d2/d5/d14/d71/fac [0,4194304] 0 2024-02-15T18:27:13.110 INFO:tasks.workunit.client.1.smithi191.stdout:4/717: chown d2/d5/d7/d3e/d63/d8c/da4/fd7 67 1 2024-02-15T18:27:13.111 INFO:tasks.workunit.client.1.smithi191.stdout:5/390: creat d0/d23/d48/d49/f78 x:0 0 0 2024-02-15T18:27:13.111 INFO:tasks.workunit.client.1.smithi191.stdout:5/391: fsync d0/d23/d41/d28/f36 0 2024-02-15T18:27:13.113 INFO:tasks.workunit.client.1.smithi191.stdout:6/590: creat d8/d9/d2b/fb2 x:0 0 0 2024-02-15T18:27:13.115 INFO:tasks.workunit.client.1.smithi191.stdout:9/693: read d0/d7/dc/d8c/f1c [3181549,109874] 0 2024-02-15T18:27:13.115 INFO:tasks.workunit.client.1.smithi191.stdout:2/644: dwrite df/d12/d1e/f41 [0,4194304] 0 2024-02-15T18:27:13.117 INFO:tasks.workunit.client.1.smithi191.stdout:0/418: rmdir d7/da/d1a/d34/d45/d79 0 2024-02-15T18:27:13.128 INFO:tasks.workunit.client.1.smithi191.stdout:6/591: dwrite d8/d9/dac/db0/d95/d9d/f1d [0,4194304] 0 2024-02-15T18:27:13.128 INFO:tasks.workunit.client.1.smithi191.stdout:6/592: dread - d8/d9/faa zero size 2024-02-15T18:27:13.128 INFO:tasks.workunit.client.1.smithi191.stdout:4/718: mkdir d2/d5/d7/d3e/de8/d108 0 2024-02-15T18:27:13.128 INFO:tasks.workunit.client.1.smithi191.stdout:4/719: chown d2/d5/d14/l2a 17 1 2024-02-15T18:27:13.131 INFO:tasks.workunit.client.1.smithi191.stdout:9/694: unlink d0/d7/dc/d30/d3b/d56/d59/f97 0 2024-02-15T18:27:13.134 INFO:tasks.workunit.client.1.smithi191.stdout:2/645: creat df/d12/d19/d1a/fd8 x:0 0 0 2024-02-15T18:27:13.134 INFO:tasks.workunit.client.1.smithi191.stdout:2/646: write df/d12/d19/d1a/fd8 [92619,11397] 0 2024-02-15T18:27:13.134 INFO:tasks.workunit.client.1.smithi191.stdout:2/647: chown df/d12/d1f/ld5 13137 1 2024-02-15T18:27:13.135 INFO:tasks.workunit.client.1.smithi191.stdout:0/419: getdents d7/da/d1a/d34/d27 0 2024-02-15T18:27:13.137 INFO:tasks.workunit.client.1.smithi191.stdout:6/593: rename d8/f45 to d8/d9/d4a/d4c/d7f/fb3 0 2024-02-15T18:27:13.138 INFO:tasks.workunit.client.1.smithi191.stdout:5/392: link d0/d23/d41/d32/d52/f60 d0/f79 0 2024-02-15T18:27:13.140 INFO:tasks.workunit.client.1.smithi191.stdout:2/648: link df/d12/d19/d1a/d35/f85 df/d12/d19/d1a/d35/d47/d4b/fd9 0 2024-02-15T18:27:13.140 INFO:tasks.workunit.client.1.smithi191.stdout:4/720: dread d2/d5/d14/d47/d48/fce [0,4194304] 0 2024-02-15T18:27:13.143 INFO:tasks.workunit.client.1.smithi191.stdout:4/721: write d2/d5/d7/fe1 [868270,94640] 0 2024-02-15T18:27:13.143 INFO:tasks.workunit.client.1.smithi191.stdout:4/722: fsync d2/d5/d7/d3e/d63/f7d 0 2024-02-15T18:27:13.165 INFO:tasks.workunit.client.1.smithi191.stdout:9/695: dwrite d0/d7/f28 [0,4194304] 0 2024-02-15T18:27:13.170 INFO:tasks.workunit.client.1.smithi191.stdout:0/420: dwrite d7/d1b/f23 [0,4194304] 0 2024-02-15T18:27:13.170 INFO:tasks.workunit.client.1.smithi191.stdout:0/421: read d7/da/d17/d31/d5f/f5b [66102,10973] 0 2024-02-15T18:27:13.177 INFO:tasks.workunit.client.1.smithi191.stdout:4/723: dwrite d2/f5c [0,4194304] 0 2024-02-15T18:27:13.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:27:11.010617+0000 mgr.smithi042.kfoihb (mgr.24435) 49 : audit [DBG] from='client.24513 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:13.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:27:11.847790+0000 mon.smithi042 (mon.0) 14 : audit [DBG] from='client.? 172.21.15.42:0/2126640308' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:27:13.209 INFO:tasks.workunit.client.1.smithi191.stdout:6/594: symlink d8/d9/dac/db0/d95/d9d/d6b/d72/lb4 0 2024-02-15T18:27:13.281 INFO:tasks.workunit.client.1.smithi191.stdout:5/393: creat d0/d18/f7a x:0 0 0 2024-02-15T18:27:13.302 INFO:tasks.workunit.client.1.smithi191.stdout:5/394: dwrite d0/d23/d41/d28/f4a [0,4194304] 0 2024-02-15T18:27:13.320 INFO:tasks.workunit.client.1.smithi191.stdout:5/395: dwrite d0/d18/f7a [0,4194304] 0 2024-02-15T18:27:13.320 INFO:tasks.workunit.client.1.smithi191.stdout:5/396: dread - d0/d18/d5c/f71 zero size 2024-02-15T18:27:13.320 INFO:tasks.workunit.client.1.smithi191.stdout:5/397: fsync d0/d23/d41/d32/d52/f60 0 2024-02-15T18:27:13.336 INFO:tasks.workunit.client.1.smithi191.stdout:2/649: getdents df/d12/d32/d4d/d89 0 2024-02-15T18:27:13.338 INFO:tasks.workunit.client.1.smithi191.stdout:2/650: dread df/d61/d9f/da3/fb3 [0,4194304] 0 2024-02-15T18:27:13.339 INFO:tasks.workunit.client.1.smithi191.stdout:2/651: dread df/d12/d19/d1a/d35/d5b/f66 [0,4194304] 0 2024-02-15T18:27:13.422 INFO:tasks.workunit.client.1.smithi191.stdout:9/696: mkdir d0/d7/dc/d32/de4/dec 0 2024-02-15T18:27:13.423 INFO:tasks.workunit.client.1.smithi191.stdout:0/422: mknod d7/da/d1a/d2c/c92 0 2024-02-15T18:27:13.424 INFO:tasks.workunit.client.1.smithi191.stdout:4/724: creat d2/d5/d7/d106/f109 x:0 0 0 2024-02-15T18:27:13.428 INFO:tasks.workunit.client.1.smithi191.stdout:6/595: link d8/d9/c47 d8/d9/d2b/d57/cb5 0 2024-02-15T18:27:13.434 INFO:tasks.workunit.client.1.smithi191.stdout:5/398: rename d0/d18/l75 to d0/d23/d41/l7b 0 2024-02-15T18:27:13.435 INFO:tasks.workunit.client.1.smithi191.stdout:2/652: mkdir df/d12/d19/d1a/dda 0 2024-02-15T18:27:13.435 INFO:tasks.workunit.client.1.smithi191.stdout:2/653: readlink df/lc5 0 2024-02-15T18:27:13.435 INFO:tasks.workunit.client.1.smithi191.stdout:9/697: getdents d0/d1d 0 2024-02-15T18:27:13.436 INFO:tasks.workunit.client.1.smithi191.stdout:8/758: sync 2024-02-15T18:27:13.439 INFO:tasks.workunit.client.1.smithi191.stdout:0/423: creat d7/da/d6c/f93 x:0 0 0 2024-02-15T18:27:13.439 INFO:tasks.workunit.client.1.smithi191.stdout:6/596: rename d8/d9/d2b/d57/f63 to d8/d9/dac/db0/fb6 0 2024-02-15T18:27:13.440 INFO:tasks.workunit.client.1.smithi191.stdout:4/725: dwrite d2/d5/d7/d2f/d41/f92 [0,4194304] 0 2024-02-15T18:27:13.441 INFO:tasks.workunit.client.1.smithi191.stdout:2/654: unlink df/d12/d1e/l7b 0 2024-02-15T18:27:13.442 INFO:tasks.workunit.client.1.smithi191.stdout:7/654: sync 2024-02-15T18:27:13.442 INFO:tasks.workunit.client.1.smithi191.stdout:1/696: sync 2024-02-15T18:27:13.442 INFO:tasks.workunit.client.1.smithi191.stdout:3/535: sync 2024-02-15T18:27:13.442 INFO:tasks.workunit.client.1.smithi191.stdout:1/697: chown d0/d8/df/d12/l56 131355440 1 2024-02-15T18:27:13.445 INFO:tasks.workunit.client.1.smithi191.stdout:8/759: mkdir d3/d2c/d1a/d10b 0 2024-02-15T18:27:13.446 INFO:tasks.workunit.client.1.smithi191.stdout:0/424: mkdir d7/d94 0 2024-02-15T18:27:13.448 INFO:tasks.workunit.client.1.smithi191.stdout:6/597: unlink d8/d9/d4a/c9b 0 2024-02-15T18:27:13.449 INFO:tasks.workunit.client.1.smithi191.stdout:7/655: dread d0/db/d65/d11/d88/d9d/dd2/f95 [0,4194304] 0 2024-02-15T18:27:13.450 INFO:tasks.workunit.client.1.smithi191.stdout:5/399: rename d0/d23/d48 to d0/d18/d5c/d7c 0 2024-02-15T18:27:13.453 INFO:tasks.workunit.client.1.smithi191.stdout:4/726: mknod d2/d5/d86/c10a 0 2024-02-15T18:27:13.454 INFO:tasks.workunit.client.1.smithi191.stdout:4/727: readlink d2/d5/d7/le9 0 2024-02-15T18:27:13.461 INFO:tasks.workunit.client.1.smithi191.stdout:2/655: mkdir df/d12/d19/d1a/d35/d47/d4b/d7a/ddb 0 2024-02-15T18:27:13.464 INFO:tasks.workunit.client.1.smithi191.stdout:1/698: mkdir d0/d8/df/d12/d33/d49/dea 0 2024-02-15T18:27:13.467 INFO:tasks.workunit.client.1.smithi191.stdout:9/698: truncate d0/f4 3839564 0 2024-02-15T18:27:13.467 INFO:tasks.workunit.client.1.smithi191.stdout:9/699: fsync d0/d7/dc/d8c/f1c 0 2024-02-15T18:27:13.469 INFO:tasks.workunit.client.1.smithi191.stdout:8/760: unlink d3/d2c/d1a/d23/d2e/d5b/d69/cd7 0 2024-02-15T18:27:13.469 INFO:tasks.workunit.client.1.smithi191.stdout:8/761: dread - d3/d2c/d1a/d23/d2e/fdf zero size 2024-02-15T18:27:13.469 INFO:tasks.workunit.client.1.smithi191.stdout:8/762: write d3/d2c/d1a/d23/ff6 [1039177,66876] 0 2024-02-15T18:27:13.469 INFO:tasks.workunit.client.1.smithi191.stdout:8/763: write d3/d2c/d1a/d23/d2e/d47/feb [3712066,118107] 0 2024-02-15T18:27:13.470 INFO:tasks.workunit.client.1.smithi191.stdout:8/764: write d3/fd [2455189,5363] 0 2024-02-15T18:27:13.473 INFO:tasks.workunit.client.1.smithi191.stdout:0/425: symlink d7/d4a/l95 0 2024-02-15T18:27:13.474 INFO:tasks.workunit.client.1.smithi191.stdout:7/656: getdents d0/db/d65 0 2024-02-15T18:27:13.477 INFO:tasks.workunit.client.1.smithi191.stdout:3/536: rename d6/d3b/d30/f50 to d6/fb4 0 2024-02-15T18:27:13.479 INFO:tasks.workunit.client.1.smithi191.stdout:5/400: mknod d0/d23/d41/d3c/c7d 0 2024-02-15T18:27:13.479 INFO:tasks.workunit.client.1.smithi191.stdout:2/656: dwrite df/d12/d32/f8f [0,4194304] 0 2024-02-15T18:27:13.481 INFO:tasks.workunit.client.1.smithi191.stdout:4/728: mkdir d2/d5/d7/d90/d10b 0 2024-02-15T18:27:13.482 INFO:tasks.workunit.client.1.smithi191.stdout:4/729: chown d2/d5/d14/d47/d78/dd0 30 1 2024-02-15T18:27:13.483 INFO:tasks.workunit.client.1.smithi191.stdout:1/699: unlink d0/d2f/d51/d54/d7b/la7 0 2024-02-15T18:27:13.491 INFO:tasks.workunit.client.1.smithi191.stdout:8/765: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/f10c x:0 0 0 2024-02-15T18:27:13.493 INFO:tasks.workunit.client.1.smithi191.stdout:7/657: unlink d0/db/d65/dc/fa0 0 2024-02-15T18:27:13.493 INFO:tasks.workunit.client.1.smithi191.stdout:7/658: read - d0/db/d26/d5f/d77/d35/fd4 zero size 2024-02-15T18:27:13.493 INFO:tasks.workunit.client.1.smithi191.stdout:7/659: stat d0/db/d26/d5f/d77/c36 0 2024-02-15T18:27:13.494 INFO:tasks.workunit.client.1.smithi191.stdout:3/537: dwrite f4 [0,4194304] 0 2024-02-15T18:27:13.496 INFO:tasks.workunit.client.1.smithi191.stdout:6/598: rename d8/f1c to d8/d9/dac/db0/fb7 0 2024-02-15T18:27:13.496 INFO:tasks.workunit.client.1.smithi191.stdout:6/599: chown d8/d9/d2b/d5b 249 1 2024-02-15T18:27:13.498 INFO:tasks.workunit.client.1.smithi191.stdout:5/401: symlink d0/d23/d41/d28/d31/d5a/d61/l7e 0 2024-02-15T18:27:13.500 INFO:tasks.workunit.client.1.smithi191.stdout:1/700: creat d0/d8/df/d41/feb x:0 0 0 2024-02-15T18:27:13.505 INFO:tasks.workunit.client.1.smithi191.stdout:3/538: dread d6/d3b/f13 [0,4194304] 0 2024-02-15T18:27:13.506 INFO:tasks.workunit.client.1.smithi191.stdout:3/539: write d6/d27/f32 [2130562,85076] 0 2024-02-15T18:27:13.506 INFO:tasks.workunit.client.1.smithi191.stdout:8/766: getdents d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/da0 0 2024-02-15T18:27:13.510 INFO:tasks.workunit.client.1.smithi191.stdout:6/600: readlink d8/d9/d2b/l43 0 2024-02-15T18:27:13.510 INFO:tasks.workunit.client.1.smithi191.stdout:1/701: dread d0/f1b [0,4194304] 0 2024-02-15T18:27:13.512 INFO:tasks.workunit.client.1.smithi191.stdout:9/700: rename d0/d7/dc/d30/d3b/d56/la9 to d0/d7/dc/d30/d4f/led 0 2024-02-15T18:27:13.514 INFO:tasks.workunit.client.1.smithi191.stdout:7/660: dwrite d0/db/d26/d5f/d77/df/f19 [0,4194304] 0 2024-02-15T18:27:13.519 INFO:tasks.workunit.client.1.smithi191.stdout:6/601: dread d8/d9/f46 [0,4194304] 0 2024-02-15T18:27:13.523 INFO:tasks.workunit.client.1.smithi191.stdout:5/402: getdents d0/d62 0 2024-02-15T18:27:13.528 INFO:tasks.workunit.client.1.smithi191.stdout:8/767: dwrite d3/d2c/d1a/d23/d2e/d5b/d69/d6d/fbd [4194304,4194304] 0 2024-02-15T18:27:13.528 INFO:tasks.workunit.client.1.smithi191.stdout:3/540: unlink d6/d5d/d72/l9c 0 2024-02-15T18:27:13.529 INFO:tasks.workunit.client.1.smithi191.stdout:9/701: rename d0/d29/f3d to d0/d7/dc/d30/fee 0 2024-02-15T18:27:13.532 INFO:tasks.workunit.client.1.smithi191.stdout:7/661: creat d0/db/d26/d5f/d77/d35/dc7/ff7 x:0 0 0 2024-02-15T18:27:13.534 INFO:tasks.workunit.client.1.smithi191.stdout:8/768: mkdir d3/d2c/d1a/d10d 0 2024-02-15T18:27:13.535 INFO:tasks.workunit.client.1.smithi191.stdout:8/769: readlink d3/d2c/d1a/d23/d3e/db2/l108 0 2024-02-15T18:27:13.539 INFO:tasks.workunit.client.1.smithi191.stdout:9/702: unlink d0/d7/dc/d30/d3b/d8b/faa 0 2024-02-15T18:27:13.543 INFO:tasks.workunit.client.1.smithi191.stdout:0/426: rename d7/f63 to d7/d1b/f96 0 2024-02-15T18:27:13.544 INFO:tasks.workunit.client.1.smithi191.stdout:0/427: write d7/da/d1a/d2c/f32 [178886,103859] 0 2024-02-15T18:27:13.545 INFO:tasks.workunit.client.1.smithi191.stdout:0/428: chown d7/fb 0 1 2024-02-15T18:27:13.545 INFO:tasks.workunit.client.1.smithi191.stdout:1/702: dwrite d0/d8/df/d12/d33/d49/f67 [0,4194304] 0 2024-02-15T18:27:13.545 INFO:tasks.workunit.client.1.smithi191.stdout:3/541: dread d6/fa0 [0,4194304] 0 2024-02-15T18:27:13.545 INFO:tasks.workunit.client.1.smithi191.stdout:7/662: unlink d0/db/d65/d11/d88/d9d/dd2/l9a 0 2024-02-15T18:27:13.545 INFO:tasks.workunit.client.1.smithi191.stdout:7/663: chown d0/db/d26/d5f/d77/d35/fd4 83395888 1 2024-02-15T18:27:13.546 INFO:tasks.workunit.client.1.smithi191.stdout:9/703: rmdir d0/d7/dc/d17/d44/de6 39 2024-02-15T18:27:13.546 INFO:tasks.workunit.client.1.smithi191.stdout:0/429: dread d7/da/d1a/d34/f52 [0,4194304] 0 2024-02-15T18:27:13.546 INFO:tasks.workunit.client.1.smithi191.stdout:0/430: stat d7/da/d1a/d2c/f4f 0 2024-02-15T18:27:13.549 INFO:tasks.workunit.client.1.smithi191.stdout:6/602: dwrite d8/d9/f46 [0,4194304] 0 2024-02-15T18:27:13.549 INFO:tasks.workunit.client.1.smithi191.stdout:6/603: fdatasync d8/d9/f86 0 2024-02-15T18:27:13.554 INFO:tasks.workunit.client.1.smithi191.stdout:5/403: dwrite d0/d18/d5c/f71 [0,4194304] 0 2024-02-15T18:27:13.555 INFO:tasks.workunit.client.1.smithi191.stdout:8/770: dwrite d3/d5e/f73 [4194304,4194304] 0 2024-02-15T18:27:13.595 INFO:tasks.workunit.client.1.smithi191.stdout:2/657: rename df/d12/d19/d1a/d35/d5b/f6e to df/d61/fdc 0 2024-02-15T18:27:13.633 INFO:tasks.workunit.client.1.smithi191.stdout:1/703: mknod d0/d2f/d51/d54/cec 0 2024-02-15T18:27:13.635 INFO:tasks.workunit.client.1.smithi191.stdout:7/664: creat d0/db/d65/dc/d4b/de8/ff8 x:0 0 0 2024-02-15T18:27:13.637 INFO:tasks.workunit.client.1.smithi191.stdout:3/542: creat d6/d1b/d60/da2/fb5 x:0 0 0 2024-02-15T18:27:13.640 INFO:tasks.workunit.client.1.smithi191.stdout:0/431: creat d7/d4a/f97 x:0 0 0 2024-02-15T18:27:13.640 INFO:tasks.workunit.client.1.smithi191.stdout:9/704: creat d0/d1d/fef x:0 0 0 2024-02-15T18:27:13.641 INFO:tasks.workunit.client.1.smithi191.stdout:6/604: rmdir d8/d9/dac/db0 39 2024-02-15T18:27:13.641 INFO:tasks.workunit.client.1.smithi191.stdout:6/605: chown d8/f97 1662 1 2024-02-15T18:27:13.647 INFO:tasks.workunit.client.1.smithi191.stdout:5/404: mkdir d0/d18/d5c/d73/d7f 0 2024-02-15T18:27:13.649 INFO:tasks.workunit.client.1.smithi191.stdout:8/771: getdents d3/d2c/d1a/d23/d2e/d5b/d69 0 2024-02-15T18:27:13.649 INFO:tasks.workunit.client.1.smithi191.stdout:8/772: chown d3/d2c/d1a/d23/d3e/fb4 738837 1 2024-02-15T18:27:13.650 INFO:tasks.workunit.client.1.smithi191.stdout:2/658: truncate df/f1b 6191779 0 2024-02-15T18:27:13.653 INFO:tasks.workunit.client.1.smithi191.stdout:1/704: rename d0/d8/df/d90/d94/d9a/cde to d0/d8/df/d12/d2a/d2b/d78/d60/d74/ced 0 2024-02-15T18:27:13.654 INFO:tasks.workunit.client.1.smithi191.stdout:7/665: symlink d0/db/d65/d11/d88/d9d/dd2/lf9 0 2024-02-15T18:27:13.654 INFO:tasks.workunit.client.1.smithi191.stdout:7/666: chown d0/db/d26/d5f/d77/df/d5d/c74 173 1 2024-02-15T18:27:13.657 INFO:tasks.workunit.client.1.smithi191.stdout:3/543: creat d6/d27/d4d/fb6 x:0 0 0 2024-02-15T18:27:13.657 INFO:tasks.workunit.client.1.smithi191.stdout:0/432: mkdir d7/da/d1a/d2c/d4c/d98 0 2024-02-15T18:27:13.658 INFO:tasks.workunit.client.1.smithi191.stdout:9/705: symlink d0/d7/dc/d17/d33/lf0 0 2024-02-15T18:27:13.661 INFO:tasks.workunit.client.1.smithi191.stdout:1/705: dread d0/d2f/d51/d10/f59 [0,4194304] 0 2024-02-15T18:27:13.662 INFO:tasks.workunit.client.1.smithi191.stdout:1/706: fsync d0/d2f/d51/d10/f25 0 2024-02-15T18:27:13.662 INFO:tasks.workunit.client.1.smithi191.stdout:6/606: symlink d8/d9/dac/db0/d95/d9d/lb8 0 2024-02-15T18:27:13.663 INFO:tasks.workunit.client.1.smithi191.stdout:5/405: creat d0/d18/d5c/f80 x:0 0 0 2024-02-15T18:27:13.663 INFO:tasks.workunit.client.1.smithi191.stdout:5/406: chown d0/d23/d41/d32/l59 647 1 2024-02-15T18:27:13.665 INFO:tasks.workunit.client.1.smithi191.stdout:6/607: write d8/d9/d2b/d57/d8e/f92 [520458,57149] 0 2024-02-15T18:27:13.667 INFO:tasks.workunit.client.1.smithi191.stdout:6/608: dread d8/d9/d4a/d4c/da4/f7c [0,4194304] 0 2024-02-15T18:27:13.667 INFO:tasks.workunit.client.1.smithi191.stdout:8/773: rmdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d8e 39 2024-02-15T18:27:13.669 INFO:tasks.workunit.client.1.smithi191.stdout:2/659: unlink df/d61/f79 0 2024-02-15T18:27:13.674 INFO:tasks.workunit.client.1.smithi191.stdout:3/544: symlink d6/d27/d4d/da8/lb7 0 2024-02-15T18:27:13.675 INFO:tasks.workunit.client.1.smithi191.stdout:0/433: creat d7/da/d1a/d34/f99 x:0 0 0 2024-02-15T18:27:13.678 INFO:tasks.workunit.client.1.smithi191.stdout:9/706: symlink d0/d7/dc/d32/lf1 0 2024-02-15T18:27:13.683 INFO:tasks.workunit.client.1.smithi191.stdout:1/707: creat d0/d2f/d51/d5e/fee x:0 0 0 2024-02-15T18:27:13.688 INFO:tasks.workunit.client.1.smithi191.stdout:1/708: dread d0/d8/df/d12/d2a/fb8 [0,4194304] 0 2024-02-15T18:27:13.688 INFO:tasks.workunit.client.1.smithi191.stdout:1/709: fdatasync d0/d8/df/d12/d2a/f3a 0 2024-02-15T18:27:13.688 INFO:tasks.workunit.client.1.smithi191.stdout:9/707: dread d0/d29/f86 [0,4194304] 0 2024-02-15T18:27:13.688 INFO:tasks.workunit.client.1.smithi191.stdout:9/708: read d0/d7/dc/d30/f5c [2644675,38094] 0 2024-02-15T18:27:13.692 INFO:tasks.workunit.client.1.smithi191.stdout:9/709: read d0/d1d/d3f/d80/f75 [690574,54608] 0 2024-02-15T18:27:13.694 INFO:tasks.workunit.client.1.smithi191.stdout:0/434: dread d7/d1b/f96 [0,4194304] 0 2024-02-15T18:27:13.694 INFO:tasks.workunit.client.1.smithi191.stdout:0/435: readlink d7/da/d1a/l44 0 2024-02-15T18:27:13.700 INFO:tasks.workunit.client.1.smithi191.stdout:5/407: dwrite d0/d23/d41/d28/d31/d5a/f68 [0,4194304] 0 2024-02-15T18:27:13.702 INFO:tasks.workunit.client.1.smithi191.stdout:8/774: dwrite d3/d5e/f80 [0,4194304] 0 2024-02-15T18:27:13.702 INFO:tasks.workunit.client.1.smithi191.stdout:8/775: write d3/d2c/d1a/d23/d2a/f6e [1754766,68113] 0 2024-02-15T18:27:13.703 INFO:tasks.workunit.client.1.smithi191.stdout:1/710: dread d0/d8/df/d12/d2a/d2b/f61 [0,4194304] 0 2024-02-15T18:27:13.708 INFO:tasks.workunit.client.1.smithi191.stdout:7/667: rmdir d0/db/d65/d11/d88 39 2024-02-15T18:27:13.719 INFO:tasks.workunit.client.1.smithi191.stdout:6/609: unlink d8/d9/dac/db0/d95/d9d/d6b/la0 0 2024-02-15T18:27:13.721 INFO:tasks.workunit.client.1.smithi191.stdout:3/545: mkdir d6/db8 0 2024-02-15T18:27:13.723 INFO:tasks.workunit.client.1.smithi191.stdout:2/660: rename df/d12/f5d to df/d61/d9f/da3/dab/fdd 0 2024-02-15T18:27:13.723 INFO:tasks.workunit.client.1.smithi191.stdout:9/710: unlink d0/d29/d48/d66/fbd 0 2024-02-15T18:27:13.725 INFO:tasks.workunit.client.1.smithi191.stdout:0/436: link d7/da/d17/d31/d5f/f61 d7/d3b/f9a 0 2024-02-15T18:27:13.726 INFO:tasks.workunit.client.1.smithi191.stdout:9/711: read d0/d7/dc/f15 [3023235,34864] 0 2024-02-15T18:27:13.733 INFO:tasks.workunit.client.1.smithi191.stdout:5/408: link d0/d23/d41/d28/d31/f53 d0/d23/d41/d32/f81 0 2024-02-15T18:27:13.741 INFO:tasks.workunit.client.1.smithi191.stdout:8/776: truncate d3/d2c/f17 3608973 0 2024-02-15T18:27:13.741 INFO:tasks.workunit.client.1.smithi191.stdout:1/711: symlink d0/d2f/d5c/lef 0 2024-02-15T18:27:13.743 INFO:tasks.workunit.client.1.smithi191.stdout:7/668: symlink d0/db/d65/d11/d57/lfa 0 2024-02-15T18:27:13.746 INFO:tasks.workunit.client.1.smithi191.stdout:3/546: getdents d6/d5d 0 2024-02-15T18:27:13.747 INFO:tasks.workunit.client.1.smithi191.stdout:0/437: chown l2 986020697 1 2024-02-15T18:27:13.747 INFO:tasks.workunit.client.1.smithi191.stdout:0/438: readlink d7/da/d17/d31/d5f/l88 0 2024-02-15T18:27:13.747 INFO:tasks.workunit.client.1.smithi191.stdout:0/439: readlink d7/da/d1a/l2d 0 2024-02-15T18:27:13.749 INFO:tasks.workunit.client.1.smithi191.stdout:9/712: rmdir d0/d7/dc 39 2024-02-15T18:27:13.749 INFO:tasks.workunit.client.1.smithi191.stdout:2/661: mknod df/d12/d1f/cde 0 2024-02-15T18:27:13.749 INFO:tasks.workunit.client.1.smithi191.stdout:2/662: truncate df/d12/d19/d1a/d35/f80 757892 0 2024-02-15T18:27:13.749 INFO:tasks.workunit.client.1.smithi191.stdout:2/663: chown df/d12/d19/d1a/f21 165824901 1 2024-02-15T18:27:13.750 INFO:tasks.workunit.client.1.smithi191.stdout:5/409: creat d0/d23/d41/d28/d31/d5a/d61/f82 x:0 0 0 2024-02-15T18:27:13.751 INFO:tasks.workunit.client.1.smithi191.stdout:8/777: creat d3/d2c/d1a/d10b/f10e x:0 0 0 2024-02-15T18:27:13.753 INFO:tasks.workunit.client.1.smithi191.stdout:1/712: symlink d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/lf0 0 2024-02-15T18:27:13.755 INFO:tasks.workunit.client.1.smithi191.stdout:7/669: creat d0/db/d26/d5f/ffb x:0 0 0 2024-02-15T18:27:13.756 INFO:tasks.workunit.client.1.smithi191.stdout:7/670: chown d0/db/d26/d5f/d77/d35/d42/f6c 1061 1 2024-02-15T18:27:13.757 INFO:tasks.workunit.client.1.smithi191.stdout:6/610: link d8/d9/d4a/d4c/l70 d8/d9/lb9 0 2024-02-15T18:27:13.758 INFO:tasks.workunit.client.1.smithi191.stdout:3/547: creat d6/d5d/daa/fb9 x:0 0 0 2024-02-15T18:27:13.761 INFO:tasks.workunit.client.1.smithi191.stdout:0/440: dread d7/d1b/f96 [4194304,4194304] 0 2024-02-15T18:27:13.761 INFO:tasks.workunit.client.1.smithi191.stdout:0/441: chown d7/da/d17/d31/d5f/f73 106674 1 2024-02-15T18:27:13.761 INFO:tasks.workunit.client.1.smithi191.stdout:9/713: creat d0/d7/dc/d17/d33/ff2 x:0 0 0 2024-02-15T18:27:13.761 INFO:tasks.workunit.client.1.smithi191.stdout:2/664: rename df/d12/d19/d1a/fd8 to df/d12/d19/d1a/fdf 0 2024-02-15T18:27:13.763 INFO:tasks.workunit.client.1.smithi191.stdout:8/778: mkdir d3/d2c/d1a/db3/d10f 0 2024-02-15T18:27:13.763 INFO:tasks.workunit.client.1.smithi191.stdout:8/779: write d3/d2c/d1a/d23/d3e/db2/fbb [981233,14757] 0 2024-02-15T18:27:13.764 INFO:tasks.workunit.client.1.smithi191.stdout:8/780: readlink d3/l71 0 2024-02-15T18:27:13.765 INFO:tasks.workunit.client.1.smithi191.stdout:1/713: creat d0/ff1 x:0 0 0 2024-02-15T18:27:13.775 INFO:tasks.workunit.client.1.smithi191.stdout:7/671: rename d0/db/d26/d5f/d77/df/d5d/fa1 to d0/db/d26/d5f/d77/db9/ffc 0 2024-02-15T18:27:13.779 INFO:tasks.workunit.client.1.smithi191.stdout:6/611: truncate d8/d9/d4a/f9a 42359 0 2024-02-15T18:27:13.779 INFO:tasks.workunit.client.1.smithi191.stdout:6/612: truncate d8/d9/d2b/d57/f93 105213 0 2024-02-15T18:27:13.779 INFO:tasks.workunit.client.1.smithi191.stdout:3/548: getdents d6/d1b/d82 0 2024-02-15T18:27:13.779 INFO:tasks.workunit.client.1.smithi191.stdout:2/665: creat df/d12/d19/d1a/d35/d47/d4b/d7a/ddb/fe0 x:0 0 0 2024-02-15T18:27:13.780 INFO:tasks.workunit.client.1.smithi191.stdout:2/666: write df/d12/d32/fa9 [3020935,123598] 0 2024-02-15T18:27:13.780 INFO:tasks.workunit.client.1.smithi191.stdout:2/667: chown df/d12/d19/d1a/d35/d5b/dbf/cd0 474476 1 2024-02-15T18:27:13.781 INFO:tasks.workunit.client.1.smithi191.stdout:9/714: dread d0/d7/fab [0,4194304] 0 2024-02-15T18:27:13.781 INFO:tasks.workunit.client.1.smithi191.stdout:8/781: creat d3/d2c/d1a/d23/d3e/db2/dff/f110 x:0 0 0 2024-02-15T18:27:13.781 INFO:tasks.workunit.client.1.smithi191.stdout:8/782: dread - d3/d2c/d1a/d23/d2e/d5b/d69/d6d/fd2 zero size 2024-02-15T18:27:13.783 INFO:tasks.workunit.client.1.smithi191.stdout:7/672: mknod d0/db/d65/d11/d57/cfd 0 2024-02-15T18:27:13.783 INFO:tasks.workunit.client.1.smithi191.stdout:6/613: symlink d8/d9/d4a/lba 0 2024-02-15T18:27:13.783 INFO:tasks.workunit.client.1.smithi191.stdout:6/614: write d8/d9/d2b/d5b/d9f/fa8 [70381,110495] 0 2024-02-15T18:27:13.786 INFO:tasks.workunit.client.1.smithi191.stdout:9/715: dread d0/d7/dc/d30/d3b/d56/fb5 [0,4194304] 0 2024-02-15T18:27:13.786 INFO:tasks.workunit.client.1.smithi191.stdout:7/673: mknod d0/db/d26/d5f/d8c/cfe 0 2024-02-15T18:27:13.786 INFO:tasks.workunit.client.1.smithi191.stdout:7/674: write d0/db/d65/dc/d4b/f7f [2431435,6020] 0 2024-02-15T18:27:13.787 INFO:tasks.workunit.client.1.smithi191.stdout:9/716: mkdir d0/d7/dbf/df3 0 2024-02-15T18:27:13.787 INFO:tasks.workunit.client.1.smithi191.stdout:6/615: write d8/d9/dac/db0/d95/d9d/f1d [4159205,122538] 0 2024-02-15T18:27:13.791 INFO:tasks.workunit.client.1.smithi191.stdout:7/675: creat d0/db/d26/d5f/d77/d35/fff x:0 0 0 2024-02-15T18:27:13.791 INFO:tasks.workunit.client.1.smithi191.stdout:7/676: fsync d0/db/d65/d11/d1c/fb3 0 2024-02-15T18:27:13.791 INFO:tasks.workunit.client.1.smithi191.stdout:9/717: mknod d0/d7/cf4 0 2024-02-15T18:27:13.792 INFO:tasks.workunit.client.1.smithi191.stdout:6/616: creat d8/d9/d2b/d57/d8e/d91/fbb x:0 0 0 2024-02-15T18:27:13.793 INFO:tasks.workunit.client.1.smithi191.stdout:3/549: write d6/d1b/d60/da2/fa9 [298035,36080] 0 2024-02-15T18:27:13.795 INFO:tasks.workunit.client.1.smithi191.stdout:9/718: mknod d0/d7/dc/d30/d3b/d56/d59/dce/cf5 0 2024-02-15T18:27:13.802 INFO:tasks.workunit.client.1.smithi191.stdout:8/783: dread d3/f15 [0,4194304] 0 2024-02-15T18:27:13.802 INFO:tasks.workunit.client.1.smithi191.stdout:2/668: dread df/d12/d1f/d42/f49 [0,4194304] 0 2024-02-15T18:27:13.802 INFO:tasks.workunit.client.1.smithi191.stdout:2/669: readlink df/d12/d1f/l9a 0 2024-02-15T18:27:13.802 INFO:tasks.workunit.client.1.smithi191.stdout:2/670: fdatasync df/d12/d19/d1a/f29 0 2024-02-15T18:27:13.803 INFO:tasks.workunit.client.1.smithi191.stdout:6/617: getdents d8/d9/d2b/d5b/d9f 0 2024-02-15T18:27:13.806 INFO:tasks.workunit.client.1.smithi191.stdout:0/442: dwrite d7/da/d17/d31/f41 [0,4194304] 0 2024-02-15T18:27:13.807 INFO:tasks.workunit.client.1.smithi191.stdout:1/714: dwrite d0/d2f/d51/d10/fb4 [0,4194304] 0 2024-02-15T18:27:13.807 INFO:tasks.workunit.client.1.smithi191.stdout:1/715: dread - d0/d8/df/d2e/f8d zero size 2024-02-15T18:27:13.807 INFO:tasks.workunit.client.1.smithi191.stdout:1/716: read - d0/d2f/d51/fd1 zero size 2024-02-15T18:27:13.807 INFO:tasks.workunit.client.1.smithi191.stdout:2/671: creat df/d12/d19/d1a/d35/d47/d4b/d7a/fe1 x:0 0 0 2024-02-15T18:27:13.811 INFO:tasks.workunit.client.1.smithi191.stdout:1/717: write d0/d8/df/d12/d2a/d2b/f37 [991110,11872] 0 2024-02-15T18:27:13.811 INFO:tasks.workunit.client.1.smithi191.stdout:1/718: chown d0/d8/df/cb5 486 1 2024-02-15T18:27:13.815 INFO:tasks.workunit.client.1.smithi191.stdout:5/410: dwrite d0/d23/d41/d28/d31/d5a/d61/f82 [0,4194304] 0 2024-02-15T18:27:13.815 INFO:tasks.workunit.client.1.smithi191.stdout:5/411: write d0/d23/d41/d32/d52/f60 [567330,29587] 0 2024-02-15T18:27:13.815 INFO:tasks.workunit.client.1.smithi191.stdout:5/412: truncate d0/f79 673652 0 2024-02-15T18:27:13.815 INFO:tasks.workunit.client.1.smithi191.stdout:2/672: dread df/d12/f39 [0,4194304] 0 2024-02-15T18:27:13.815 INFO:tasks.workunit.client.1.smithi191.stdout:0/443: symlink d7/da/d17/l9b 0 2024-02-15T18:27:13.816 INFO:tasks.workunit.client.1.smithi191.stdout:0/444: dread d7/d1b/f81 [0,4194304] 0 2024-02-15T18:27:13.819 INFO:tasks.workunit.client.1.smithi191.stdout:1/719: mknod d0/d2f/d66/cf2 0 2024-02-15T18:27:13.819 INFO:tasks.workunit.client.1.smithi191.stdout:8/784: write d3/d2c/f43 [2587114,16850] 0 2024-02-15T18:27:13.820 INFO:tasks.workunit.client.1.smithi191.stdout:5/413: dread d0/d23/d41/d28/d31/f53 [0,4194304] 0 2024-02-15T18:27:13.826 INFO:tasks.workunit.client.1.smithi191.stdout:3/550: dwrite d6/d3b/d42/fa1 [0,4194304] 0 2024-02-15T18:27:13.826 INFO:tasks.workunit.client.1.smithi191.stdout:6/618: dwrite d8/f30 [0,4194304] 0 2024-02-15T18:27:13.826 INFO:tasks.workunit.client.1.smithi191.stdout:6/619: readlink d8/d9/dac/db0/d95/d9d/l88 0 2024-02-15T18:27:13.826 INFO:tasks.workunit.client.1.smithi191.stdout:6/620: chown d8/l25 134049189 1 2024-02-15T18:27:13.827 INFO:tasks.workunit.client.1.smithi191.stdout:6/621: chown d8/d9/d2b/f42 128424185 1 2024-02-15T18:27:13.836 INFO:tasks.workunit.client.1.smithi191.stdout:8/785: dread d3/d2c/d1a/d23/d2e/d47/d5f/f50 [0,4194304] 0 2024-02-15T18:27:13.843 INFO:tasks.workunit.client.1.smithi191.stdout:2/673: mknod df/d12/d19/d1a/d35/d47/d4b/d7a/dce/ce2 0 2024-02-15T18:27:13.845 INFO:tasks.workunit.client.1.smithi191.stdout:0/445: rename l2 to d7/d94/l9c 0 2024-02-15T18:27:13.849 INFO:tasks.workunit.client.1.smithi191.stdout:2/674: write df/d12/f22 [3997996,60940] 0 2024-02-15T18:27:13.850 INFO:tasks.workunit.client.1.smithi191.stdout:1/720: creat d0/d8/df/ff3 x:0 0 0 2024-02-15T18:27:13.859 INFO:tasks.workunit.client.1.smithi191.stdout:5/414: symlink d0/d18/d5c/l83 0 2024-02-15T18:27:13.865 INFO:tasks.workunit.client.1.smithi191.stdout:3/551: truncate d6/d3b/f84 1242908 0 2024-02-15T18:27:13.870 INFO:tasks.workunit.client.1.smithi191.stdout:6/622: mknod d8/d9/d2b/d84/cbc 0 2024-02-15T18:27:13.871 INFO:tasks.workunit.client.1.smithi191.stdout:8/786: rmdir d3/d2c/d1a/db3 39 2024-02-15T18:27:13.871 INFO:tasks.workunit.client.1.smithi191.stdout:7/677: dwrite d0/db/ff3 [0,4194304] 0 2024-02-15T18:27:13.874 INFO:tasks.workunit.client.1.smithi191.stdout:7/678: dread d0/db/d65/d11/d5a/d66/d1e/fa5 [0,4194304] 0 2024-02-15T18:27:13.877 INFO:tasks.workunit.client.1.smithi191.stdout:0/446: rename d7/da/d17/d31/f55 to d7/da/d1a/d34/f9d 0 2024-02-15T18:27:13.879 INFO:tasks.workunit.client.1.smithi191.stdout:2/675: mknod df/d12/ce3 0 2024-02-15T18:27:13.882 INFO:tasks.workunit.client.1.smithi191.stdout:1/721: creat d0/d2f/d51/d10/ff4 x:0 0 0 2024-02-15T18:27:13.882 INFO:tasks.workunit.client.1.smithi191.stdout:5/415: mknod d0/d23/d41/d32/c84 0 2024-02-15T18:27:13.882 INFO:tasks.workunit.client.1.smithi191.stdout:8/787: getdents d3 0 2024-02-15T18:27:13.883 INFO:tasks.workunit.client.1.smithi191.stdout:8/788: dread d3/d2c/d1a/d23/f5c [0,4194304] 0 2024-02-15T18:27:13.886 INFO:tasks.workunit.client.1.smithi191.stdout:6/623: rename d8/d9/d2b/f4b to d8/d9/d2b/d84/fbd 0 2024-02-15T18:27:13.887 INFO:tasks.workunit.client.1.smithi191.stdout:2/676: chown df/l54 117652 1 2024-02-15T18:27:13.888 INFO:tasks.workunit.client.1.smithi191.stdout:2/677: readlink df/d12/d19/d1a/d52/l5c 0 2024-02-15T18:27:13.888 INFO:tasks.workunit.client.1.smithi191.stdout:1/722: read d0/d8/f17 [139576,98317] 0 2024-02-15T18:27:13.889 INFO:tasks.workunit.client.1.smithi191.stdout:3/552: link d6/d27/f32 d6/d3b/fba 0 2024-02-15T18:27:13.890 INFO:tasks.workunit.client.1.smithi191.stdout:8/789: unlink d3/d2c/d1a/d23/f7c 0 2024-02-15T18:27:13.892 INFO:tasks.workunit.client.1.smithi191.stdout:2/678: dread df/d12/d19/d1a/f29 [0,4194304] 0 2024-02-15T18:27:13.894 INFO:tasks.workunit.client.1.smithi191.stdout:0/447: dwrite d7/d1b/f46 [0,4194304] 0 2024-02-15T18:27:13.894 INFO:tasks.workunit.client.1.smithi191.stdout:0/448: write d7/da/d17/d31/d5f/f73 [1040473,118235] 0 2024-02-15T18:27:13.900 INFO:tasks.workunit.client.1.smithi191.stdout:5/416: rename d0/d23/d41/d32 to d0/d23/d41/d28/d65/d85 0 2024-02-15T18:27:13.901 INFO:tasks.workunit.client.1.smithi191.stdout:5/417: write d0/d18/f42 [2302991,24312] 0 2024-02-15T18:27:13.901 INFO:tasks.workunit.client.1.smithi191.stdout:6/624: unlink d8/d9/d2b/f4e 0 2024-02-15T18:27:13.901 INFO:tasks.workunit.client.1.smithi191.stdout:6/625: chown d8/d9/dac/db0 456 1 2024-02-15T18:27:13.903 INFO:tasks.workunit.client.1.smithi191.stdout:1/723: symlink d0/lf5 0 2024-02-15T18:27:13.907 INFO:tasks.workunit.client.1.smithi191.stdout:3/553: rmdir d6/d1b/d82 39 2024-02-15T18:27:13.909 INFO:tasks.workunit.client.1.smithi191.stdout:8/790: mknod d3/d2c/d1a/d23/d2e/c111 0 2024-02-15T18:27:13.909 INFO:tasks.workunit.client.1.smithi191.stdout:8/791: chown d3/l40 515347484 1 2024-02-15T18:27:13.910 INFO:tasks.workunit.client.1.smithi191.stdout:2/679: truncate df/d12/d19/d1a/d35/d47/d4b/f7e 2950434 0 2024-02-15T18:27:13.910 INFO:tasks.workunit.client.1.smithi191.stdout:8/792: dread d3/d2c/d1a/d23/d2e/f9f [0,4194304] 0 2024-02-15T18:27:13.913 INFO:tasks.workunit.client.1.smithi191.stdout:0/449: mkdir d7/da/d17/d31/d5f/d9e 0 2024-02-15T18:27:13.913 INFO:tasks.workunit.client.1.smithi191.stdout:0/450: dread - d7/da/f8e zero size 2024-02-15T18:27:13.918 INFO:tasks.workunit.client.1.smithi191.stdout:5/418: link d0/d18/d5c/f6d d0/d18/d5c/d7c/f86 0 2024-02-15T18:27:13.922 INFO:tasks.workunit.client.1.smithi191.stdout:6/626: creat d8/d9/dac/db0/d95/d90/fbe x:0 0 0 2024-02-15T18:27:13.922 INFO:tasks.workunit.client.1.smithi191.stdout:6/627: fdatasync d8/d9/d2b/d57/f6c 0 2024-02-15T18:27:13.924 INFO:tasks.workunit.client.1.smithi191.stdout:1/724: link d0/fa8 d0/de7/ff6 0 2024-02-15T18:27:13.927 INFO:tasks.workunit.client.1.smithi191.stdout:2/680: mknod df/d61/d94/ce4 0 2024-02-15T18:27:13.927 INFO:tasks.workunit.client.1.smithi191.stdout:8/793: mkdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/da0/d112 0 2024-02-15T18:27:13.927 INFO:tasks.workunit.client.1.smithi191.stdout:8/794: chown d3/d2c/d1a/l1e 351 1 2024-02-15T18:27:13.932 INFO:tasks.workunit.client.1.smithi191.stdout:0/451: rename d7/da/d1a/d34/f9d to d7/da/d1a/d2c/d4c/f9f 0 2024-02-15T18:27:13.932 INFO:tasks.workunit.client.1.smithi191.stdout:3/554: mkdir d6/d3b/d42/dbb 0 2024-02-15T18:27:13.932 INFO:tasks.workunit.client.1.smithi191.stdout:3/555: chown d6/d27/d4d/f58 3 1 2024-02-15T18:27:13.932 INFO:tasks.workunit.client.1.smithi191.stdout:3/556: chown d6/d27/c91 7 1 2024-02-15T18:27:13.932 INFO:tasks.workunit.client.1.smithi191.stdout:3/557: fsync d6/d1b/d60/d73/f41 0 2024-02-15T18:27:13.932 INFO:tasks.workunit.client.1.smithi191.stdout:0/452: creat d7/da/d1a/d34/fa0 x:0 0 0 2024-02-15T18:27:13.932 INFO:tasks.workunit.client.1.smithi191.stdout:0/453: chown d7/fb 6157 1 2024-02-15T18:27:13.934 INFO:tasks.workunit.client.1.smithi191.stdout:3/558: unlink d6/d1b/d60/da2/fb5 0 2024-02-15T18:27:13.934 INFO:tasks.workunit.client.1.smithi191.stdout:3/559: fsync d6/d3b/f13 0 2024-02-15T18:27:13.939 INFO:tasks.workunit.client.1.smithi191.stdout:3/560: creat d6/d27/d4d/d5a/da4/fbc x:0 0 0 2024-02-15T18:27:13.939 INFO:tasks.workunit.client.1.smithi191.stdout:3/561: truncate d6/d27/d4d/d5a/da4/fbc 277664 0 2024-02-15T18:27:13.941 INFO:tasks.workunit.client.1.smithi191.stdout:0/454: symlink d7/da/d1a/d2c/d4c/la1 0 2024-02-15T18:27:13.956 INFO:tasks.workunit.client.1.smithi191.stdout:0/455: dread d7/da/d17/d31/f41 [0,4194304] 0 2024-02-15T18:27:13.963 INFO:tasks.workunit.client.1.smithi191.stdout:0/456: write d7/d1b/f46 [2753991,3478] 0 2024-02-15T18:27:13.964 INFO:tasks.workunit.client.1.smithi191.stdout:0/457: getdents d7/d1b 0 2024-02-15T18:27:13.965 INFO:tasks.workunit.client.1.smithi191.stdout:0/458: read d7/d4a/f59 [11981271,127876] 0 2024-02-15T18:27:13.972 INFO:tasks.workunit.client.1.smithi191.stdout:5/419: dwrite d0/d18/d5c/d7c/f58 [0,4194304] 0 2024-02-15T18:27:13.973 INFO:tasks.workunit.client.1.smithi191.stdout:5/420: read d0/d23/f27 [367326,62310] 0 2024-02-15T18:27:13.974 INFO:tasks.workunit.client.1.smithi191.stdout:5/421: creat d0/d23/d41/f87 x:0 0 0 2024-02-15T18:27:13.974 INFO:tasks.workunit.client.1.smithi191.stdout:5/422: stat d0/d23/d41/d3c/l64 0 2024-02-15T18:27:13.975 INFO:tasks.workunit.client.1.smithi191.stdout:5/423: mknod d0/d23/d41/d28/d31/d5a/d61/c88 0 2024-02-15T18:27:13.977 INFO:tasks.workunit.client.1.smithi191.stdout:1/725: dwrite d0/d8/df/d12/d2a/d2b/f7a [0,4194304] 0 2024-02-15T18:27:13.978 INFO:tasks.workunit.client.1.smithi191.stdout:6/628: dwrite d8/d9/d4a/f9a [0,4194304] 0 2024-02-15T18:27:13.979 INFO:tasks.workunit.client.1.smithi191.stdout:8/795: dwrite d3/d2c/d1a/d23/d2e/d64/f8d [0,4194304] 0 2024-02-15T18:27:13.980 INFO:tasks.workunit.client.1.smithi191.stdout:8/796: dread - d3/d2c/d1a/fce zero size 2024-02-15T18:27:13.982 INFO:tasks.workunit.client.1.smithi191.stdout:3/562: dwrite d6/d27/f32 [4194304,4194304] 0 2024-02-15T18:27:13.983 INFO:tasks.workunit.client.1.smithi191.stdout:2/681: dwrite df/d12/d19/d1a/d35/d47/d4b/d7a/fe1 [0,4194304] 0 2024-02-15T18:27:13.986 INFO:tasks.workunit.client.1.smithi191.stdout:5/424: dread d0/d18/f7a [0,4194304] 0 2024-02-15T18:27:13.986 INFO:tasks.workunit.client.1.smithi191.stdout:6/629: creat d8/d9/dac/db0/d95/d9d/d6b/d98/fbf x:0 0 0 2024-02-15T18:27:13.988 INFO:tasks.workunit.client.1.smithi191.stdout:6/630: write d8/d9/dac/db0/d95/d9d/d6b/d76/fb1 [2225723,59965] 0 2024-02-15T18:27:13.989 INFO:tasks.workunit.client.1.smithi191.stdout:1/726: dread d0/d8/df/d12/d2a/d2b/f37 [0,4194304] 0 2024-02-15T18:27:13.990 INFO:tasks.workunit.client.1.smithi191.stdout:0/459: dwrite d7/da/d1a/d2c/f32 [0,4194304] 0 2024-02-15T18:27:13.992 INFO:tasks.workunit.client.1.smithi191.stdout:3/563: unlink d6/d1b/d82/db2/fae 0 2024-02-15T18:27:13.996 INFO:tasks.workunit.client.1.smithi191.stdout:2/682: creat df/d61/d9f/da3/dab/dd6/fe5 x:0 0 0 2024-02-15T18:27:14.000 INFO:tasks.workunit.client.1.smithi191.stdout:6/631: rename d8/d9/d2b/d84/fbd to d8/d9/d4a/d4c/fc0 0 2024-02-15T18:27:14.003 INFO:tasks.workunit.client.1.smithi191.stdout:3/564: creat d6/d3b/fbd x:0 0 0 2024-02-15T18:27:14.007 INFO:tasks.workunit.client.1.smithi191.stdout:5/425: dwrite d0/d18/d5c/d7c/f58 [0,4194304] 0 2024-02-15T18:27:14.008 INFO:tasks.workunit.client.1.smithi191.stdout:5/426: write d0/f1a [2047428,20081] 0 2024-02-15T18:27:14.008 INFO:tasks.workunit.client.1.smithi191.stdout:5/427: dread - d0/d18/d5c/f63 zero size 2024-02-15T18:27:14.009 INFO:tasks.workunit.client.1.smithi191.stdout:2/683: mkdir df/d12/d1f/de6 0 2024-02-15T18:27:14.009 INFO:tasks.workunit.client.1.smithi191.stdout:2/684: chown df/d12/d19/d1a/d35/f60 742577 1 2024-02-15T18:27:14.010 INFO:tasks.workunit.client.1.smithi191.stdout:0/460: rename d7/da/d17/d31/d5f/c5c to d7/da/d17/ca2 0 2024-02-15T18:27:14.010 INFO:tasks.workunit.client.1.smithi191.stdout:1/727: dwrite d0/d8/df/d12/d2a/d2b/f37 [0,4194304] 0 2024-02-15T18:27:14.011 INFO:tasks.workunit.client.1.smithi191.stdout:1/728: truncate d0/d2f/d5c/fcf 511340 0 2024-02-15T18:27:14.012 INFO:tasks.workunit.client.1.smithi191.stdout:6/632: creat d8/d9/dac/db0/d95/fc1 x:0 0 0 2024-02-15T18:27:14.012 INFO:tasks.workunit.client.1.smithi191.stdout:1/729: write d0/d2f/d51/f14 [4448817,92397] 0 2024-02-15T18:27:14.013 INFO:tasks.workunit.client.1.smithi191.stdout:1/730: write d0/d2f/d5c/fcf [116528,3155] 0 2024-02-15T18:27:14.013 INFO:tasks.workunit.client.1.smithi191.stdout:1/731: chown d0/d8/df/d12/d2a/d2b/d78/d60/le1 7 1 2024-02-15T18:27:14.018 INFO:tasks.workunit.client.1.smithi191.stdout:3/565: mkdir d6/d27/d4d/d5a/d92/dbe 0 2024-02-15T18:27:14.018 INFO:tasks.workunit.client.1.smithi191.stdout:5/428: symlink d0/d18/d5c/d7c/d49/d76/l89 0 2024-02-15T18:27:14.020 INFO:tasks.workunit.client.1.smithi191.stdout:2/685: link df/d12/d19/d1a/c27 df/d12/d1f/d42/ce7 0 2024-02-15T18:27:14.020 INFO:tasks.workunit.client.1.smithi191.stdout:0/461: stat d7/da/d17/l37 0 2024-02-15T18:27:14.021 INFO:tasks.workunit.client.1.smithi191.stdout:1/732: symlink d0/d8/df/d41/lf7 0 2024-02-15T18:27:14.021 INFO:tasks.workunit.client.1.smithi191.stdout:1/733: stat d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa 0 2024-02-15T18:27:14.021 INFO:tasks.workunit.client.1.smithi191.stdout:1/734: write d0/d8/df/d2e/f8d [769631,93176] 0 2024-02-15T18:27:14.022 INFO:tasks.workunit.client.1.smithi191.stdout:3/566: fdatasync d6/d8e/f99 0 2024-02-15T18:27:14.022 INFO:tasks.workunit.client.1.smithi191.stdout:3/567: write d6/d3b/fa7 [965780,44607] 0 2024-02-15T18:27:14.025 INFO:tasks.workunit.client.1.smithi191.stdout:2/686: creat df/d12/d1f/d42/db5/fe8 x:0 0 0 2024-02-15T18:27:14.025 INFO:tasks.workunit.client.1.smithi191.stdout:0/462: mknod d7/da/d17/d31/d7d/ca3 0 2024-02-15T18:27:14.026 INFO:tasks.workunit.client.1.smithi191.stdout:0/463: write d7/da/d1a/d34/d27/f60 [1068787,73504] 0 2024-02-15T18:27:14.028 INFO:tasks.workunit.client.1.smithi191.stdout:1/735: getdents d0/d8/df/d12/d2a/d2b 0 2024-02-15T18:27:14.028 INFO:tasks.workunit.client.1.smithi191.stdout:1/736: dread - d0/d8/df/d12/d2a/fe3 zero size 2024-02-15T18:27:14.028 INFO:tasks.workunit.client.1.smithi191.stdout:1/737: chown d0/d8/df/d41/f36 0 1 2024-02-15T18:27:14.029 INFO:tasks.workunit.client.1.smithi191.stdout:3/568: mkdir d6/d3b/d42/d49/dbf 0 2024-02-15T18:27:14.029 INFO:tasks.workunit.client.1.smithi191.stdout:3/569: write d6/d1b/d60/d73/f75 [1304992,16915] 0 2024-02-15T18:27:14.030 INFO:tasks.workunit.client.1.smithi191.stdout:2/687: symlink df/d12/d19/d1a/dda/le9 0 2024-02-15T18:27:14.030 INFO:tasks.workunit.client.1.smithi191.stdout:1/738: getdents d0/d8/df/d90/d94/d9a 0 2024-02-15T18:27:14.030 INFO:tasks.workunit.client.1.smithi191.stdout:1/739: write d0/d2f/d51/d5e/d68/fce [129795,1692] 0 2024-02-15T18:27:14.031 INFO:tasks.workunit.client.1.smithi191.stdout:1/740: readlink d0/l1e 0 2024-02-15T18:27:14.032 INFO:tasks.workunit.client.1.smithi191.stdout:2/688: mknod df/d12/d19/d1a/d52/cea 0 2024-02-15T18:27:14.032 INFO:tasks.workunit.client.1.smithi191.stdout:3/570: dread d6/d8e/f99 [0,4194304] 0 2024-02-15T18:27:14.033 INFO:tasks.workunit.client.1.smithi191.stdout:6/633: dwrite d8/d9/dac/db0/d95/d9d/f38 [0,4194304] 0 2024-02-15T18:27:14.034 INFO:tasks.workunit.client.1.smithi191.stdout:1/741: creat d0/d8/df/ff8 x:0 0 0 2024-02-15T18:27:14.034 INFO:tasks.workunit.client.1.smithi191.stdout:1/742: write d0/d8/f17 [613150,33244] 0 2024-02-15T18:27:14.034 INFO:tasks.workunit.client.1.smithi191.stdout:1/743: chown d0/d2f/d51/d5e/d68/le0 994 1 2024-02-15T18:27:14.034 INFO:tasks.workunit.client.1.smithi191.stdout:1/744: fsync d0/d8/d48/f4d 0 2024-02-15T18:27:14.034 INFO:tasks.workunit.client.1.smithi191.stdout:3/571: getdents d6/d27/d4d 0 2024-02-15T18:27:14.035 INFO:tasks.workunit.client.1.smithi191.stdout:0/464: dread d7/d3b/f69 [0,4194304] 0 2024-02-15T18:27:14.035 INFO:tasks.workunit.client.1.smithi191.stdout:2/689: write df/d12/d1f/f26 [244699,3956] 0 2024-02-15T18:27:14.036 INFO:tasks.workunit.client.1.smithi191.stdout:3/572: write d6/d27/d4d/f58 [4946487,681] 0 2024-02-15T18:27:14.036 INFO:tasks.workunit.client.1.smithi191.stdout:1/745: symlink d0/d8/d48/lf9 0 2024-02-15T18:27:14.036 INFO:tasks.workunit.client.1.smithi191.stdout:1/746: write d0/d8/df/d12/d2a/d2b/d78/d60/fbe [783320,48818] 0 2024-02-15T18:27:14.036 INFO:tasks.workunit.client.1.smithi191.stdout:6/634: write d8/d9/d2b/d57/f6c [3987370,92075] 0 2024-02-15T18:27:14.037 INFO:tasks.workunit.client.1.smithi191.stdout:5/429: dwrite d0/d23/d41/d28/d65/d85/d52/f60 [0,4194304] 0 2024-02-15T18:27:14.037 INFO:tasks.workunit.client.1.smithi191.stdout:0/465: symlink d7/da/d1a/d34/la4 0 2024-02-15T18:27:14.037 INFO:tasks.workunit.client.1.smithi191.stdout:2/690: creat df/d12/d19/d1a/feb x:0 0 0 2024-02-15T18:27:14.038 INFO:tasks.workunit.client.1.smithi191.stdout:5/430: dread d0/d23/d41/d28/d31/f53 [0,4194304] 0 2024-02-15T18:27:14.040 INFO:tasks.workunit.client.1.smithi191.stdout:3/573: rename d6/d3b/d42/d49/f3d to d6/d27/d4d/d5a/da4/fc0 0 2024-02-15T18:27:14.042 INFO:tasks.workunit.client.1.smithi191.stdout:1/747: symlink d0/d2f/d66/lfa 0 2024-02-15T18:27:14.053 INFO:tasks.workunit.client.1.smithi191.stdout:0/466: creat d7/da/d1a/fa5 x:0 0 0 2024-02-15T18:27:14.053 INFO:tasks.workunit.client.1.smithi191.stdout:0/467: stat d7/d1b/c7f 0 2024-02-15T18:27:14.054 INFO:tasks.workunit.client.1.smithi191.stdout:0/468: write d7/da/d1a/d34/f52 [255123,50082] 0 2024-02-15T18:27:14.055 INFO:tasks.workunit.client.1.smithi191.stdout:5/431: mknod d0/d18/d5c/c8a 0 2024-02-15T18:27:14.056 INFO:tasks.workunit.client.1.smithi191.stdout:3/574: unlink d6/d27/d4d/da8/lb7 0 2024-02-15T18:27:14.057 INFO:tasks.workunit.client.1.smithi191.stdout:2/691: dwrite df/d12/fca [0,4194304] 0 2024-02-15T18:27:14.058 INFO:tasks.workunit.client.1.smithi191.stdout:1/748: rmdir d0/d2f/d51/d5e 39 2024-02-15T18:27:14.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:12.217944+0000 mgr.smithi042.kfoihb (mgr.24435) 50 : audit [DBG] from='client.24523 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:14.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:12.684344+0000 mgr.smithi042.kfoihb (mgr.24435) 51 : cluster [DBG] pgmap v35: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 173 MiB/s rd, 369 MiB/s wr, 600 op/s 2024-02-15T18:27:14.061 INFO:tasks.workunit.client.1.smithi191.stdout:6/635: symlink d8/d9/lc2 0 2024-02-15T18:27:14.061 INFO:tasks.workunit.client.1.smithi191.stdout:6/636: chown d8/d9/dac/db0/d95/d9d/d6b/d76/c8a 401087671 1 2024-02-15T18:27:14.061 INFO:tasks.workunit.client.1.smithi191.stdout:6/637: write d8/d9/d2b/fb2 [628495,115285] 0 2024-02-15T18:27:14.061 INFO:tasks.workunit.client.1.smithi191.stdout:6/638: dread - d8/d9/d2b/d5b/f5e zero size 2024-02-15T18:27:14.061 INFO:tasks.workunit.client.1.smithi191.stdout:6/639: stat d8/d9/dac/db0/d95/d9d/d6b/d72/lb4 0 2024-02-15T18:27:14.061 INFO:tasks.workunit.client.1.smithi191.stdout:6/640: dread - d8/fa5 zero size 2024-02-15T18:27:14.064 INFO:tasks.workunit.client.1.smithi191.stdout:0/469: symlink d7/da/d17/la6 0 2024-02-15T18:27:14.066 INFO:tasks.workunit.client.1.smithi191.stdout:5/432: mknod d0/d18/d5c/d7c/d49/d76/c8b 0 2024-02-15T18:27:14.067 INFO:tasks.workunit.client.1.smithi191.stdout:3/575: mkdir d6/d1b/db0/dc1 0 2024-02-15T18:27:14.073 INFO:tasks.workunit.client.1.smithi191.stdout:1/749: dwrite d0/d8/df/d90/d94/fe6 [0,4194304] 0 2024-02-15T18:27:14.074 INFO:tasks.workunit.client.1.smithi191.stdout:2/692: creat df/d12/d19/d1a/dda/fec x:0 0 0 2024-02-15T18:27:14.075 INFO:tasks.workunit.client.1.smithi191.stdout:0/470: mknod d7/da/d17/d31/d5f/d9e/ca7 0 2024-02-15T18:27:14.080 INFO:tasks.workunit.client.1.smithi191.stdout:5/433: link d0/d23/d41/d28/f36 d0/d23/d41/d28/d31/d5a/f8c 0 2024-02-15T18:27:14.080 INFO:tasks.workunit.client.1.smithi191.stdout:5/434: truncate d0/d23/d41/d28/f4a 4995607 0 2024-02-15T18:27:14.081 INFO:tasks.workunit.client.1.smithi191.stdout:3/576: creat d6/d3b/d42/d49/d54/fc2 x:0 0 0 2024-02-15T18:27:14.085 INFO:tasks.workunit.client.1.smithi191.stdout:1/750: link d0/d8/df/d12/l56 d0/d8/df/d12/d2a/d2b/d78/lfb 0 2024-02-15T18:27:14.085 INFO:tasks.workunit.client.1.smithi191.stdout:6/641: dwrite d8/d9/f29 [0,4194304] 0 2024-02-15T18:27:14.088 INFO:tasks.workunit.client.1.smithi191.stdout:2/693: symlink df/d12/d19/d1a/led 0 2024-02-15T18:27:14.090 INFO:tasks.workunit.client.1.smithi191.stdout:0/471: link d7/da/d1a/d34/c3d d7/da/d17/ca8 0 2024-02-15T18:27:14.091 INFO:tasks.workunit.client.1.smithi191.stdout:5/435: link d0/d23/d41/c16 d0/d23/d41/d28/d31/d5a/d61/c8d 0 2024-02-15T18:27:14.094 INFO:tasks.workunit.client.1.smithi191.stdout:1/751: creat d0/d2f/d51/d10/ffc x:0 0 0 2024-02-15T18:27:14.095 INFO:tasks.workunit.client.1.smithi191.stdout:2/694: mkdir df/d12/d32/dee 0 2024-02-15T18:27:14.096 INFO:tasks.workunit.client.1.smithi191.stdout:0/472: rmdir d7/d4a 39 2024-02-15T18:27:14.096 INFO:tasks.workunit.client.1.smithi191.stdout:0/473: write d7/da/d1a/d34/d27/f7c [173739,114225] 0 2024-02-15T18:27:14.097 INFO:tasks.workunit.client.1.smithi191.stdout:2/695: rmdir df/d12/d19/d1a/d35/d5b/dbf 39 2024-02-15T18:27:14.097 INFO:tasks.workunit.client.1.smithi191.stdout:2/696: stat df/d12/d1e/d6f/fad 0 2024-02-15T18:27:14.099 INFO:tasks.workunit.client.1.smithi191.stdout:0/474: mkdir d7/da/d17/d31/d5f/da9 0 2024-02-15T18:27:14.099 INFO:tasks.workunit.client.1.smithi191.stdout:0/475: truncate d7/da/d1a/d34/d27/f60 1532787 0 2024-02-15T18:27:14.100 INFO:tasks.workunit.client.1.smithi191.stdout:5/436: mknod d0/c8e 0 2024-02-15T18:27:14.102 INFO:tasks.workunit.client.1.smithi191.stdout:0/476: unlink d7/da/d1a/d34/c66 0 2024-02-15T18:27:14.104 INFO:tasks.workunit.client.1.smithi191.stdout:5/437: mknod d0/d23/d41/d28/d31/d5a/d61/c8f 0 2024-02-15T18:27:14.104 INFO:tasks.workunit.client.1.smithi191.stdout:5/438: write d0/d23/d41/d28/d65/d85/f81 [546672,99447] 0 2024-02-15T18:27:14.106 INFO:tasks.workunit.client.1.smithi191.stdout:2/697: dread df/d12/d19/d1a/d35/d47/d4b/f7e [0,4194304] 0 2024-02-15T18:27:14.108 INFO:tasks.workunit.client.1.smithi191.stdout:2/698: write df/d12/d19/d1a/f44 [2747944,3957] 0 2024-02-15T18:27:14.108 INFO:tasks.workunit.client.1.smithi191.stdout:0/477: rmdir d7/da/d6c 39 2024-02-15T18:27:14.109 INFO:tasks.workunit.client.1.smithi191.stdout:0/478: rmdir d7/da/d1a/d2c/d4c 39 2024-02-15T18:27:14.109 INFO:tasks.workunit.client.1.smithi191.stdout:3/577: dwrite d6/d27/f32 [4194304,4194304] 0 2024-02-15T18:27:14.109 INFO:tasks.workunit.client.1.smithi191.stdout:0/479: write d7/da/d1a/d2c/f8c [200710,83378] 0 2024-02-15T18:27:14.109 INFO:tasks.workunit.client.1.smithi191.stdout:0/480: fsync d7/da/d17/d31/d5f/f61 0 2024-02-15T18:27:14.114 INFO:tasks.workunit.client.1.smithi191.stdout:5/439: mknod d0/d23/d41/d28/c90 0 2024-02-15T18:27:14.114 INFO:tasks.workunit.client.1.smithi191.stdout:5/440: chown d0/d23/d41/d28/d31/c5f 28859 1 2024-02-15T18:27:14.114 INFO:tasks.workunit.client.1.smithi191.stdout:5/441: truncate d0/d23/d41/d28/d31/f47 1832334 0 2024-02-15T18:27:14.118 INFO:tasks.workunit.client.1.smithi191.stdout:3/578: mknod d6/d3b/d42/dbb/cc3 0 2024-02-15T18:27:14.119 INFO:tasks.workunit.client.1.smithi191.stdout:0/481: mkdir d7/da/d17/daa 0 2024-02-15T18:27:14.119 INFO:tasks.workunit.client.1.smithi191.stdout:0/482: write d7/da/d17/d31/f89 [293529,99135] 0 2024-02-15T18:27:14.122 INFO:tasks.workunit.client.1.smithi191.stdout:3/579: creat d6/d27/d4d/da8/fc4 x:0 0 0 2024-02-15T18:27:14.122 INFO:tasks.workunit.client.1.smithi191.stdout:2/699: dread df/d12/fca [0,4194304] 0 2024-02-15T18:27:14.124 INFO:tasks.workunit.client.1.smithi191.stdout:2/700: mknod df/d61/d94/cef 0 2024-02-15T18:27:14.132 INFO:tasks.workunit.client.1.smithi191.stdout:6/642: dwrite d8/d9/dac/db0/d95/d90/fbe [0,4194304] 0 2024-02-15T18:27:14.132 INFO:tasks.workunit.client.1.smithi191.stdout:6/643: chown d8/d9/dac/db0/d95/d9d/l2c 18069 1 2024-02-15T18:27:14.132 INFO:tasks.workunit.client.1.smithi191.stdout:5/442: dread d0/d18/f42 [0,4194304] 0 2024-02-15T18:27:14.132 INFO:tasks.workunit.client.1.smithi191.stdout:5/443: fsync d0/d23/d41/d28/f36 0 2024-02-15T18:27:14.132 INFO:tasks.workunit.client.1.smithi191.stdout:6/644: write d8/d9/d2b/d57/f59 [4895945,116298] 0 2024-02-15T18:27:14.137 INFO:tasks.workunit.client.1.smithi191.stdout:5/444: dread d0/d23/f43 [0,4194304] 0 2024-02-15T18:27:14.140 INFO:tasks.workunit.client.1.smithi191.stdout:6/645: symlink d8/d9/dac/db0/d95/d9d/lc3 0 2024-02-15T18:27:14.140 INFO:tasks.workunit.client.1.smithi191.stdout:2/701: dread df/d61/fdc [0,4194304] 0 2024-02-15T18:27:14.141 INFO:tasks.workunit.client.1.smithi191.stdout:2/702: chown df/d12/d19/fb1 57 1 2024-02-15T18:27:14.141 INFO:tasks.workunit.client.1.smithi191.stdout:2/703: chown df/d12/d32/d4d/d89 21 1 2024-02-15T18:27:14.148 INFO:tasks.workunit.client.1.smithi191.stdout:2/704: symlink df/d12/d1f/lf0 0 2024-02-15T18:27:14.152 INFO:tasks.workunit.client.1.smithi191.stdout:2/705: rename df/d12/d19/d1a/f5a to df/d12/db4/ff1 0 2024-02-15T18:27:14.152 INFO:tasks.workunit.client.1.smithi191.stdout:5/445: dread d0/d23/d41/d28/f3e [0,4194304] 0 2024-02-15T18:27:14.152 INFO:tasks.workunit.client.1.smithi191.stdout:5/446: truncate d0/d23/d41/f87 1044384 0 2024-02-15T18:27:14.152 INFO:tasks.workunit.client.1.smithi191.stdout:1/752: dwrite d0/d2f/d5c/fcf [0,4194304] 0 2024-02-15T18:27:14.154 INFO:tasks.workunit.client.1.smithi191.stdout:3/580: dwrite d6/d3b/d42/d49/f6d [0,4194304] 0 2024-02-15T18:27:14.154 INFO:tasks.workunit.client.1.smithi191.stdout:3/581: chown d6/d3b/d42/d49/d54/faf 265832221 1 2024-02-15T18:27:14.154 INFO:tasks.workunit.client.1.smithi191.stdout:3/582: stat d6/d3b/c80 0 2024-02-15T18:27:14.154 INFO:tasks.workunit.client.1.smithi191.stdout:6/646: dread d8/d9/d4a/f9a [0,4194304] 0 2024-02-15T18:27:14.154 INFO:tasks.workunit.client.1.smithi191.stdout:6/647: chown d8/d9/c47 6 1 2024-02-15T18:27:14.154 INFO:tasks.workunit.client.1.smithi191.stdout:0/483: dwrite d7/da/d1a/d34/f99 [0,4194304] 0 2024-02-15T18:27:14.155 INFO:tasks.workunit.client.1.smithi191.stdout:6/648: read d8/d9/f29 [3250755,6901] 0 2024-02-15T18:27:14.163 INFO:tasks.workunit.client.1.smithi191.stdout:5/447: dread d0/d23/d41/d28/d65/d85/d52/f40 [0,4194304] 0 2024-02-15T18:27:14.163 INFO:tasks.workunit.client.1.smithi191.stdout:5/448: readlink d0/d18/d5c/d7c/d49/d76/l89 0 2024-02-15T18:27:14.163 INFO:tasks.workunit.client.1.smithi191.stdout:5/449: fdatasync d0/d23/f43 0 2024-02-15T18:27:14.163 INFO:tasks.workunit.client.1.smithi191.stdout:5/450: fdatasync d0/d23/d41/d28/d31/f53 0 2024-02-15T18:27:14.163 INFO:tasks.workunit.client.1.smithi191.stdout:5/451: chown d0/d18/d5c/d7c/d49 3753 1 2024-02-15T18:27:14.173 INFO:tasks.workunit.client.1.smithi191.stdout:1/753: creat d0/d2f/d66/ffd x:0 0 0 2024-02-15T18:27:14.178 INFO:tasks.workunit.client.1.smithi191.stdout:3/583: write d6/d3b/d42/d49/d54/f6b [747856,24929] 0 2024-02-15T18:27:14.178 INFO:tasks.workunit.client.1.smithi191.stdout:2/706: dwrite df/d12/d1f/f26 [0,4194304] 0 2024-02-15T18:27:14.182 INFO:tasks.workunit.client.1.smithi191.stdout:5/452: dwrite d0/d23/f43 [0,4194304] 0 2024-02-15T18:27:14.187 INFO:tasks.workunit.client.1.smithi191.stdout:1/754: dread d0/d8/df/d12/d2a/f3a [0,4194304] 0 2024-02-15T18:27:14.193 INFO:tasks.workunit.client.1.smithi191.stdout:0/484: mkdir d7/da/d1a/d34/d27/dab 0 2024-02-15T18:27:14.194 INFO:tasks.workunit.client.1.smithi191.stdout:6/649: mkdir d8/d9/d4a/d4c/dc4 0 2024-02-15T18:27:14.195 INFO:tasks.workunit.client.1.smithi191.stdout:3/584: creat d6/d1b/d60/d73/d5e/fc5 x:0 0 0 2024-02-15T18:27:14.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: audit 2024-02-15T18:27:12.217944+0000 mgr.smithi042.kfoihb (mgr.24435) 50 : audit [DBG] from='client.24523 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:14.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:27:12.684344+0000 mgr.smithi042.kfoihb (mgr.24435) 51 : cluster [DBG] pgmap v35: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 173 MiB/s rd, 369 MiB/s wr, 600 op/s 2024-02-15T18:27:14.199 INFO:tasks.workunit.client.1.smithi191.stdout:2/707: mkdir df/d12/d19/d1a/d52/df2 0 2024-02-15T18:27:14.200 INFO:tasks.workunit.client.1.smithi191.stdout:0/485: dread d7/da/d1a/d34/f29 [4194304,4194304] 0 2024-02-15T18:27:14.204 INFO:tasks.workunit.client.1.smithi191.stdout:3/585: dread d6/d1b/f3e [0,4194304] 0 2024-02-15T18:27:14.204 INFO:tasks.workunit.client.1.smithi191.stdout:5/453: creat d0/d18/d5c/f91 x:0 0 0 2024-02-15T18:27:14.204 INFO:tasks.workunit.client.1.smithi191.stdout:5/454: fsync d0/d18/d5c/f80 0 2024-02-15T18:27:14.206 INFO:tasks.workunit.client.1.smithi191.stdout:1/755: rename d0/d8/df/ff3 to d0/d2f/d51/d5e/ffe 0 2024-02-15T18:27:14.206 INFO:tasks.workunit.client.1.smithi191.stdout:1/756: getdents d0/d8/df/d12/d33/d49/dea 0 2024-02-15T18:27:14.207 INFO:tasks.workunit.client.1.smithi191.stdout:5/455: dread d0/d23/d41/d28/f3e [0,4194304] 0 2024-02-15T18:27:14.212 INFO:tasks.workunit.client.1.smithi191.stdout:6/650: unlink d8/d9/d4a/d4c/f53 0 2024-02-15T18:27:14.212 INFO:tasks.workunit.client.1.smithi191.stdout:6/651: readlink d8/d9/dac/db0/d95/d9d/d6b/d72/l74 0 2024-02-15T18:27:14.212 INFO:tasks.workunit.client.1.smithi191.stdout:6/652: write d8/d9/dac/db0/d95/d90/fa1 [176520,110495] 0 2024-02-15T18:27:14.213 INFO:tasks.workunit.client.1.smithi191.stdout:2/708: creat df/d12/d19/d1a/dda/ff3 x:0 0 0 2024-02-15T18:27:14.220 INFO:tasks.workunit.client.1.smithi191.stdout:0/486: creat d7/da/d1a/d2c/d4c/fac x:0 0 0 2024-02-15T18:27:14.220 INFO:tasks.workunit.client.1.smithi191.stdout:0/487: chown d7/d1b/f6e 211891 1 2024-02-15T18:27:14.220 INFO:tasks.workunit.client.1.smithi191.stdout:0/488: chown d7/da/c35 20826 1 2024-02-15T18:27:14.221 INFO:tasks.workunit.client.1.smithi191.stdout:3/586: getdents d6/d27 0 2024-02-15T18:27:14.222 INFO:tasks.workunit.client.1.smithi191.stdout:1/757: mknod d0/d8/df/d12/d33/d49/dea/cff 0 2024-02-15T18:27:14.223 INFO:tasks.workunit.client.1.smithi191.stdout:5/456: creat d0/d23/d41/d3c/f92 x:0 0 0 2024-02-15T18:27:14.224 INFO:tasks.workunit.client.1.smithi191.stdout:0/489: creat d7/d94/fad x:0 0 0 2024-02-15T18:27:14.225 INFO:tasks.workunit.client.1.smithi191.stdout:1/758: creat d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/f100 x:0 0 0 2024-02-15T18:27:14.226 INFO:tasks.workunit.client.1.smithi191.stdout:5/457: rename d0/d23/d41/d3c/c7d to d0/d23/d41/d3c/c93 0 2024-02-15T18:27:14.227 INFO:tasks.workunit.client.1.smithi191.stdout:0/490: mkdir d7/da/d17/d31/d5f/dae 0 2024-02-15T18:27:14.228 INFO:tasks.workunit.client.1.smithi191.stdout:3/587: rename d6/d1b/d60/da2/fa9 to d6/d8e/fc6 0 2024-02-15T18:27:14.228 INFO:tasks.workunit.client.1.smithi191.stdout:3/588: readlink d6/d1b/l57 0 2024-02-15T18:27:14.229 INFO:tasks.workunit.client.1.smithi191.stdout:1/759: creat d0/d8/df/d90/d94/dc8/f101 x:0 0 0 2024-02-15T18:27:14.229 INFO:tasks.workunit.client.1.smithi191.stdout:5/458: creat d0/d23/d41/d28/f94 x:0 0 0 2024-02-15T18:27:14.231 INFO:tasks.workunit.client.1.smithi191.stdout:5/459: unlink d0/l10 0 2024-02-15T18:27:14.231 INFO:tasks.workunit.client.1.smithi191.stdout:5/460: truncate d0/d18/d5c/d7c/d49/f78 946253 0 2024-02-15T18:27:14.232 INFO:tasks.workunit.client.1.smithi191.stdout:5/461: unlink d0/d18/f56 0 2024-02-15T18:27:14.234 INFO:tasks.workunit.client.1.smithi191.stdout:5/462: creat d0/d23/d41/d28/d31/d5a/d61/f95 x:0 0 0 2024-02-15T18:27:14.235 INFO:tasks.workunit.client.1.smithi191.stdout:5/463: creat d0/d18/d5c/d7c/f96 x:0 0 0 2024-02-15T18:27:14.236 INFO:tasks.workunit.client.1.smithi191.stdout:4/730: sync 2024-02-15T18:27:14.241 INFO:tasks.workunit.client.1.smithi191.stdout:4/731: mkdir d2/d5/d7/d90/d52/dcb/d10c 0 2024-02-15T18:27:14.241 INFO:tasks.workunit.client.1.smithi191.stdout:4/732: chown d2/d5/d14/d47/d78/dd0 387941886 1 2024-02-15T18:27:14.242 INFO:tasks.workunit.client.1.smithi191.stdout:4/733: fsync d2/d5/d7/d2f/f75 0 2024-02-15T18:27:14.242 INFO:tasks.workunit.client.1.smithi191.stdout:4/734: dread - d2/d5/d86/faa zero size 2024-02-15T18:27:14.242 INFO:tasks.workunit.client.1.smithi191.stdout:6/653: dwrite d8/d9/dac/db0/d95/d9d/f1f [0,4194304] 0 2024-02-15T18:27:14.242 INFO:tasks.workunit.client.1.smithi191.stdout:6/654: chown d8/d9/dac/db0/d95/d9d/d6b/d72/l74 811 1 2024-02-15T18:27:14.242 INFO:tasks.workunit.client.1.smithi191.stdout:4/735: creat d2/d5/d14/d47/d78/dd0/f10d x:0 0 0 2024-02-15T18:27:14.249 INFO:tasks.workunit.client.1.smithi191.stdout:6/655: unlink d8/d9/d2b/l2d 0 2024-02-15T18:27:14.249 INFO:tasks.workunit.client.1.smithi191.stdout:6/656: chown d8/d9/dac/db0/d95/d9d 4164310 1 2024-02-15T18:27:14.249 INFO:tasks.workunit.client.1.smithi191.stdout:6/657: fsync d8/fab 0 2024-02-15T18:27:14.251 INFO:tasks.workunit.client.1.smithi191.stdout:6/658: rename d8/d9/d2b/d57/d8e/d91/cae to d8/d9/d4a/d4c/d7f/cc5 0 2024-02-15T18:27:14.253 INFO:tasks.workunit.client.1.smithi191.stdout:6/659: creat d8/d9/dac/db0/fc6 x:0 0 0 2024-02-15T18:27:14.254 INFO:tasks.workunit.client.1.smithi191.stdout:2/709: dwrite df/d61/f67 [0,4194304] 0 2024-02-15T18:27:14.255 INFO:tasks.workunit.client.1.smithi191.stdout:6/660: symlink d8/d9/dac/db0/d95/d9d/d6b/d98/lc7 0 2024-02-15T18:27:14.257 INFO:tasks.workunit.client.1.smithi191.stdout:2/710: symlink df/d12/d19/d1a/d35/d47/d4b/d7a/lf4 0 2024-02-15T18:27:14.258 INFO:tasks.workunit.client.1.smithi191.stdout:2/711: symlink df/d12/d19/d1a/d35/d5b/lf5 0 2024-02-15T18:27:14.259 INFO:tasks.workunit.client.1.smithi191.stdout:2/712: dread - df/d12/d1f/d42/fba zero size 2024-02-15T18:27:14.259 INFO:tasks.workunit.client.1.smithi191.stdout:2/713: fsync df/d12/d19/d1a/dda/ff3 0 2024-02-15T18:27:14.263 INFO:tasks.workunit.client.1.smithi191.stdout:3/589: dwrite d6/d3b/d42/f4a [4194304,4194304] 0 2024-02-15T18:27:14.265 INFO:tasks.workunit.client.1.smithi191.stdout:3/590: rmdir d6/d8e 39 2024-02-15T18:27:14.272 INFO:tasks.workunit.client.1.smithi191.stdout:6/661: dwrite d8/d9/d2b/f51 [0,4194304] 0 2024-02-15T18:27:14.272 INFO:tasks.workunit.client.1.smithi191.stdout:6/662: dread - d8/d9/d4a/d4c/f9e zero size 2024-02-15T18:27:14.272 INFO:tasks.workunit.client.1.smithi191.stdout:6/663: dread - d8/d9/d2b/d57/d8e/faf zero size 2024-02-15T18:27:14.272 INFO:tasks.workunit.client.1.smithi191.stdout:6/664: chown d8/d9/d2b/d5b/f82 1182 1 2024-02-15T18:27:14.274 INFO:tasks.workunit.client.1.smithi191.stdout:6/665: mknod d8/d9/d4a/cc8 0 2024-02-15T18:27:14.275 INFO:tasks.workunit.client.1.smithi191.stdout:6/666: read d8/d9/d2b/d5b/f82 [3425380,99633] 0 2024-02-15T18:27:14.275 INFO:tasks.workunit.client.1.smithi191.stdout:6/667: dread - d8/d9/d2b/f7b zero size 2024-02-15T18:27:14.276 INFO:tasks.workunit.client.1.smithi191.stdout:6/668: symlink d8/d9/dac/lc9 0 2024-02-15T18:27:14.276 INFO:tasks.workunit.client.1.smithi191.stdout:6/669: readlink d8/l16 0 2024-02-15T18:27:14.277 INFO:tasks.workunit.client.1.smithi191.stdout:6/670: truncate d8/d9/d2b/f2e 3932719 0 2024-02-15T18:27:14.278 INFO:tasks.workunit.client.1.smithi191.stdout:6/671: creat d8/d9/dac/db0/fca x:0 0 0 2024-02-15T18:27:14.278 INFO:tasks.workunit.client.1.smithi191.stdout:6/672: chown d8/d9/dac/db0/d95/d9d/d6b/d76/c8a 1 1 2024-02-15T18:27:14.278 INFO:tasks.workunit.client.1.smithi191.stdout:4/736: dwrite d2/d5/d8b/f8d [0,4194304] 0 2024-02-15T18:27:14.279 INFO:tasks.workunit.client.1.smithi191.stdout:5/464: dwrite d0/d23/d41/d28/d31/f53 [0,4194304] 0 2024-02-15T18:27:14.279 INFO:tasks.workunit.client.1.smithi191.stdout:6/673: dread d8/d9/f65 [0,4194304] 0 2024-02-15T18:27:14.281 INFO:tasks.workunit.client.1.smithi191.stdout:4/737: write d2/d5/d8b/f8d [377851,98558] 0 2024-02-15T18:27:14.284 INFO:tasks.workunit.client.1.smithi191.stdout:6/674: rename d8/d9/d2b/d57/d8e/f92 to d8/d9/d2b/d57/d8e/d91/fcb 0 2024-02-15T18:27:14.286 INFO:tasks.workunit.client.1.smithi191.stdout:4/738: unlink d2/d5/d7/d90/d52/dcb/fd6 0 2024-02-15T18:27:14.286 INFO:tasks.workunit.client.1.smithi191.stdout:4/739: fdatasync d2/fe2 0 2024-02-15T18:27:14.287 INFO:tasks.workunit.client.1.smithi191.stdout:4/740: creat d2/d5/d14/d47/d78/dd0/d105/f10e x:0 0 0 2024-02-15T18:27:14.287 INFO:tasks.workunit.client.1.smithi191.stdout:4/741: fdatasync d2/d5/d14/d47/d78/f6a 0 2024-02-15T18:27:14.289 INFO:tasks.workunit.client.1.smithi191.stdout:4/742: mknod d2/d5/d7/dee/c10f 0 2024-02-15T18:27:14.291 INFO:tasks.workunit.client.1.smithi191.stdout:4/743: dread d2/d5/d7/f15 [0,4194304] 0 2024-02-15T18:27:14.293 INFO:tasks.workunit.client.1.smithi191.stdout:4/744: creat d2/d5/d8b/f110 x:0 0 0 2024-02-15T18:27:14.295 INFO:tasks.workunit.client.1.smithi191.stdout:6/675: dread d8/d9/dac/db0/d95/d9d/f1d [0,4194304] 0 2024-02-15T18:27:14.295 INFO:tasks.workunit.client.1.smithi191.stdout:6/676: write d8/d9/d2b/d57/f93 [920519,26434] 0 2024-02-15T18:27:14.298 INFO:tasks.workunit.client.1.smithi191.stdout:4/745: write d2/d5/d7/d2f/d41/f98 [254030,22511] 0 2024-02-15T18:27:14.301 INFO:tasks.workunit.client.1.smithi191.stdout:1/760: dwrite d0/d8/df/d12/d2a/d2b/fdd [0,4194304] 0 2024-02-15T18:27:14.302 INFO:tasks.workunit.client.1.smithi191.stdout:0/491: dwrite d7/d1b/f96 [0,4194304] 0 2024-02-15T18:27:14.302 INFO:tasks.workunit.client.1.smithi191.stdout:0/492: chown d7/da/d1a/d2c/f4b 1 1 2024-02-15T18:27:14.306 INFO:tasks.workunit.client.1.smithi191.stdout:2/714: dwrite df/d12/d19/f77 [0,4194304] 0 2024-02-15T18:27:14.308 INFO:tasks.workunit.client.1.smithi191.stdout:3/591: dwrite d6/d3b/fba [0,4194304] 0 2024-02-15T18:27:14.308 INFO:tasks.workunit.client.1.smithi191.stdout:5/465: dwrite d0/d23/d41/d28/f4a [0,4194304] 0 2024-02-15T18:27:14.309 INFO:tasks.workunit.client.1.smithi191.stdout:0/493: rename d7/da/d1a/d2c/d4c/l8f to d7/d94/laf 0 2024-02-15T18:27:14.309 INFO:tasks.workunit.client.1.smithi191.stdout:1/761: creat d0/d8/df/d90/d94/dc8/f102 x:0 0 0 2024-02-15T18:27:14.313 INFO:tasks.workunit.client.1.smithi191.stdout:3/592: mkdir d6/d3b/d42/d49/dc7 0 2024-02-15T18:27:14.314 INFO:tasks.workunit.client.1.smithi191.stdout:3/593: dread - d6/d27/d4d/da8/fb1 zero size 2024-02-15T18:27:14.317 INFO:tasks.workunit.client.1.smithi191.stdout:6/677: dwrite d8/fa5 [0,4194304] 0 2024-02-15T18:27:14.317 INFO:tasks.workunit.client.1.smithi191.stdout:6/678: write d8/d9/f44 [4349047,45744] 0 2024-02-15T18:27:14.317 INFO:tasks.workunit.client.1.smithi191.stdout:2/715: rename df/d12/d19/d1a/d35/d47/fa8 to df/d12/d32/d4d/d89/ff6 0 2024-02-15T18:27:14.317 INFO:tasks.workunit.client.1.smithi191.stdout:2/716: write df/d12/d19/d1a/d35/d5b/f66 [723492,37126] 0 2024-02-15T18:27:14.318 INFO:tasks.workunit.client.1.smithi191.stdout:2/717: chown df/d12/d19/d1a/dda/le9 3 1 2024-02-15T18:27:14.321 INFO:tasks.workunit.client.1.smithi191.stdout:0/494: mknod d7/da/d17/daa/cb0 0 2024-02-15T18:27:14.323 INFO:tasks.workunit.client.1.smithi191.stdout:2/718: write df/d12/d19/d1a/f44 [4115996,60927] 0 2024-02-15T18:27:14.323 INFO:tasks.workunit.client.1.smithi191.stdout:2/719: dread - df/d61/d9f/da3/dab/dd6/fe5 zero size 2024-02-15T18:27:14.323 INFO:tasks.workunit.client.1.smithi191.stdout:2/720: fsync df/d61/fdc 0 2024-02-15T18:27:14.328 INFO:tasks.workunit.client.1.smithi191.stdout:3/594: mknod d6/d5d/cc8 0 2024-02-15T18:27:14.328 INFO:tasks.workunit.client.1.smithi191.stdout:3/595: chown d6/d3b/d42/d49/f85 0 1 2024-02-15T18:27:14.328 INFO:tasks.workunit.client.1.smithi191.stdout:3/596: read d6/fb4 [305895,18133] 0 2024-02-15T18:27:14.338 INFO:tasks.workunit.client.1.smithi191.stdout:5/466: dwrite d0/d18/d5c/d7c/f58 [0,4194304] 0 2024-02-15T18:27:14.338 INFO:tasks.workunit.client.1.smithi191.stdout:5/467: fsync d0/d18/f42 0 2024-02-15T18:27:14.338 INFO:tasks.workunit.client.1.smithi191.stdout:5/468: stat d0/c8e 0 2024-02-15T18:27:14.338 INFO:tasks.workunit.client.1.smithi191.stdout:5/469: stat d0/d23/d41/d28/d65/d85/d52 0 2024-02-15T18:27:14.338 INFO:tasks.workunit.client.1.smithi191.stdout:5/470: write d0/d18/d5c/d7c/f96 [496921,98289] 0 2024-02-15T18:27:14.338 INFO:tasks.workunit.client.1.smithi191.stdout:1/762: dwrite d0/f3 [0,4194304] 0 2024-02-15T18:27:14.343 INFO:tasks.workunit.client.1.smithi191.stdout:6/679: symlink d8/d9/dac/db0/d95/lcc 0 2024-02-15T18:27:14.343 INFO:tasks.workunit.client.1.smithi191.stdout:6/680: chown d8/d9/d2b/d81/l8f 305 1 2024-02-15T18:27:14.343 INFO:tasks.workunit.client.1.smithi191.stdout:6/681: readlink d8/d9/dac/db0/d95/d9d/lb8 0 2024-02-15T18:27:14.343 INFO:tasks.workunit.client.1.smithi191.stdout:6/682: chown d8/d9/f79 9942584 1 2024-02-15T18:27:14.347 INFO:tasks.workunit.client.1.smithi191.stdout:0/495: unlink d7/da/d17/d31/d7d/ca3 0 2024-02-15T18:27:14.347 INFO:tasks.workunit.client.1.smithi191.stdout:0/496: stat d7/da/d1a/d34/l19 0 2024-02-15T18:27:14.347 INFO:tasks.workunit.client.1.smithi191.stdout:0/497: fdatasync d7/d3b/f69 0 2024-02-15T18:27:14.350 INFO:tasks.workunit.client.1.smithi191.stdout:2/721: unlink df/f10 0 2024-02-15T18:27:14.351 INFO:tasks.workunit.client.1.smithi191.stdout:3/597: creat d6/d27/d4d/fc9 x:0 0 0 2024-02-15T18:27:14.356 INFO:tasks.workunit.client.1.smithi191.stdout:1/763: symlink d0/d2f/d5c/l103 0 2024-02-15T18:27:14.356 INFO:tasks.workunit.client.1.smithi191.stdout:1/764: chown d0/d2f/d51/d5e/d68/le0 1526281 1 2024-02-15T18:27:14.356 INFO:tasks.workunit.client.1.smithi191.stdout:1/765: stat d0/d8/df/d12/d2a/d2b/f7a 0 2024-02-15T18:27:14.357 INFO:tasks.workunit.client.1.smithi191.stdout:5/471: dwrite d0/f1b [0,4194304] 0 2024-02-15T18:27:14.358 INFO:tasks.workunit.client.1.smithi191.stdout:6/683: link d8/d9/d4a/d4c/f5d d8/fcd 0 2024-02-15T18:27:14.359 INFO:tasks.workunit.client.1.smithi191.stdout:0/498: getdents d7/d94 0 2024-02-15T18:27:14.359 INFO:tasks.workunit.client.1.smithi191.stdout:0/499: chown d7/d3b/f69 0 1 2024-02-15T18:27:14.361 INFO:tasks.workunit.client.1.smithi191.stdout:1/766: mknod d0/d8/df/d90/d94/d9a/c104 0 2024-02-15T18:27:14.361 INFO:tasks.workunit.client.1.smithi191.stdout:1/767: fdatasync d0/d2f/d51/d10/ff4 0 2024-02-15T18:27:14.362 INFO:tasks.workunit.client.1.smithi191.stdout:5/472: symlink d0/d23/d41/d28/d65/d85/l97 0 2024-02-15T18:27:14.364 INFO:tasks.workunit.client.1.smithi191.stdout:6/684: creat d8/d9/d4a/d4c/da4/fce x:0 0 0 2024-02-15T18:27:14.366 INFO:tasks.workunit.client.1.smithi191.stdout:0/500: getdents d7/da/d1a/d34/d27 0 2024-02-15T18:27:14.367 INFO:tasks.workunit.client.1.smithi191.stdout:1/768: link d0/c6 d0/d8/df/d2e/c105 0 2024-02-15T18:27:14.370 INFO:tasks.workunit.client.1.smithi191.stdout:5/473: unlink d0/d23/d41/d28/d31/c35 0 2024-02-15T18:27:14.371 INFO:tasks.workunit.client.1.smithi191.stdout:0/501: creat d7/fb1 x:0 0 0 2024-02-15T18:27:14.371 INFO:tasks.workunit.client.1.smithi191.stdout:0/502: truncate d7/da/d1a/d2c/f7e 2089904 0 2024-02-15T18:27:14.371 INFO:tasks.workunit.client.1.smithi191.stdout:5/474: creat d0/d23/d41/d28/f98 x:0 0 0 2024-02-15T18:27:14.371 INFO:tasks.workunit.client.1.smithi191.stdout:5/475: fsync d0/d18/f42 0 2024-02-15T18:27:14.372 INFO:tasks.workunit.client.1.smithi191.stdout:5/476: write d0/d23/d41/d28/d31/f53 [5236178,46503] 0 2024-02-15T18:27:14.372 INFO:tasks.workunit.client.1.smithi191.stdout:6/685: dread d8/d9/dac/db0/d95/d9d/d6b/f75 [0,4194304] 0 2024-02-15T18:27:14.373 INFO:tasks.workunit.client.1.smithi191.stdout:6/686: fsync d8/d9/f65 0 2024-02-15T18:27:14.373 INFO:tasks.workunit.client.1.smithi191.stdout:5/477: mknod d0/d23/d41/d28/d31/d5a/d61/c99 0 2024-02-15T18:27:14.373 INFO:tasks.workunit.client.1.smithi191.stdout:5/478: readlink d0/d18/d5c/d7c/d49/d76/l89 0 2024-02-15T18:27:14.375 INFO:tasks.workunit.client.1.smithi191.stdout:6/687: mkdir d8/d9/d4a/d4c/d7f/dcf 0 2024-02-15T18:27:14.375 INFO:tasks.workunit.client.1.smithi191.stdout:6/688: readlink d8/d9/dac/db0/d95/d9d/l88 0 2024-02-15T18:27:14.375 INFO:tasks.workunit.client.1.smithi191.stdout:6/689: dread - d8/d9/d4a/d4c/da4/fce zero size 2024-02-15T18:27:14.375 INFO:tasks.workunit.client.1.smithi191.stdout:6/690: stat d8/d9/d2b/d57/d8e/d91/fa3 0 2024-02-15T18:27:14.376 INFO:tasks.workunit.client.1.smithi191.stdout:0/503: dread d7/da/d1a/d2c/f7e [0,4194304] 0 2024-02-15T18:27:14.381 INFO:tasks.workunit.client.1.smithi191.stdout:6/691: creat d8/d9/dac/db0/d95/d90/fd0 x:0 0 0 2024-02-15T18:27:14.391 INFO:tasks.workunit.client.1.smithi191.stdout:1/769: dwrite d0/d8/df/d12/d2a/f3a [4194304,4194304] 0 2024-02-15T18:27:14.395 INFO:tasks.workunit.client.1.smithi191.stdout:1/770: unlink d0/d8/df/d90/d94/fe6 0 2024-02-15T18:27:14.400 INFO:tasks.workunit.client.1.smithi191.stdout:1/771: rename d0/d2f/d77/f81 to d0/d8/df/d90/d94/dc8/f106 0 2024-02-15T18:27:14.400 INFO:tasks.workunit.client.1.smithi191.stdout:1/772: write d0/d8/df/f85 [1075579,49653] 0 2024-02-15T18:27:14.407 INFO:tasks.workunit.client.1.smithi191.stdout:1/773: creat d0/d8/df/d12/d33/d49/f107 x:0 0 0 2024-02-15T18:27:14.424 INFO:tasks.workunit.client.1.smithi191.stdout:6/692: dread d8/d9/dac/db0/d95/d90/fa1 [0,4194304] 0 2024-02-15T18:27:14.424 INFO:tasks.workunit.client.1.smithi191.stdout:6/693: readlink d8/l3f 0 2024-02-15T18:27:14.424 INFO:tasks.workunit.client.1.smithi191.stdout:2/722: dwrite df/d12/d19/d1a/d35/d47/d4b/f99 [0,4194304] 0 2024-02-15T18:27:14.426 INFO:tasks.workunit.client.1.smithi191.stdout:6/694: creat d8/d9/dac/db0/d95/d9d/d6b/d98/fd1 x:0 0 0 2024-02-15T18:27:14.431 INFO:tasks.workunit.client.1.smithi191.stdout:6/695: link d8/d9/d4a/d4c/fc0 d8/d9/d4a/d4c/dc4/fd2 0 2024-02-15T18:27:14.434 INFO:tasks.workunit.client.1.smithi191.stdout:2/723: link df/d12/d19/d1a/d35/d47/da2/ca7 df/d12/db4/cf7 0 2024-02-15T18:27:14.444 INFO:tasks.workunit.client.1.smithi191.stdout:5/479: dwrite d0/d18/d5c/f71 [0,4194304] 0 2024-02-15T18:27:14.445 INFO:tasks.workunit.client.1.smithi191.stdout:0/504: dwrite d7/da/d1a/f58 [0,4194304] 0 2024-02-15T18:27:14.451 INFO:tasks.workunit.client.1.smithi191.stdout:2/724: truncate df/d12/d19/d1a/d35/d47/f51 967393 0 2024-02-15T18:27:14.451 INFO:tasks.workunit.client.1.smithi191.stdout:2/725: dread - df/d12/d19/d1a/dda/ff3 zero size 2024-02-15T18:27:14.451 INFO:tasks.workunit.client.1.smithi191.stdout:5/480: creat d0/d23/d41/f9a x:0 0 0 2024-02-15T18:27:14.451 INFO:tasks.workunit.client.1.smithi191.stdout:5/481: truncate d0/d18/d5c/f80 970350 0 2024-02-15T18:27:14.452 INFO:tasks.workunit.client.1.smithi191.stdout:2/726: dread df/d61/d94/f98 [0,4194304] 0 2024-02-15T18:27:14.454 INFO:tasks.workunit.client.1.smithi191.stdout:6/696: dwrite d8/fab [0,4194304] 0 2024-02-15T18:27:14.460 INFO:tasks.workunit.client.1.smithi191.stdout:7/679: sync 2024-02-15T18:27:14.461 INFO:tasks.workunit.client.1.smithi191.stdout:9/719: sync 2024-02-15T18:27:14.462 INFO:tasks.workunit.client.1.smithi191.stdout:5/482: rmdir d0/d18/d5c/d7c/d49/d76 39 2024-02-15T18:27:14.462 INFO:tasks.workunit.client.1.smithi191.stdout:5/483: write d0/d23/d41/d28/d31/d5a/d61/f95 [242445,24410] 0 2024-02-15T18:27:14.466 INFO:tasks.workunit.client.1.smithi191.stdout:6/697: creat d8/d9/d4a/d4c/fd3 x:0 0 0 2024-02-15T18:27:14.466 INFO:tasks.workunit.client.1.smithi191.stdout:7/680: symlink d0/db/d26/d5f/d77/l100 0 2024-02-15T18:27:14.471 INFO:tasks.workunit.client.1.smithi191.stdout:6/698: mknod d8/d9/dac/db0/cd4 0 2024-02-15T18:27:14.472 INFO:tasks.workunit.client.1.smithi191.stdout:6/699: write d8/d9/d2b/d57/d8e/d91/fcb [1471950,80283] 0 2024-02-15T18:27:14.472 INFO:tasks.workunit.client.1.smithi191.stdout:6/700: truncate d8/d9/d4a/d4c/f9e 255155 0 2024-02-15T18:27:14.473 INFO:tasks.workunit.client.1.smithi191.stdout:7/681: creat d0/db/d65/d11/d88/f101 x:0 0 0 2024-02-15T18:27:14.473 INFO:tasks.workunit.client.1.smithi191.stdout:7/682: chown d0/db/d26/d5f/d77/d35/dc7/fe5 185041799 1 2024-02-15T18:27:14.476 INFO:tasks.workunit.client.1.smithi191.stdout:6/701: unlink d8/d9/dac/db0/c71 0 2024-02-15T18:27:14.477 INFO:tasks.workunit.client.1.smithi191.stdout:7/683: mknod d0/db/d65/d11/dd7/c102 0 2024-02-15T18:27:14.479 INFO:tasks.workunit.client.1.smithi191.stdout:6/702: creat d8/d9/d2b/d84/fd5 x:0 0 0 2024-02-15T18:27:14.479 INFO:tasks.workunit.client.1.smithi191.stdout:6/703: fsync d8/d9/f65 0 2024-02-15T18:27:14.481 INFO:tasks.workunit.client.1.smithi191.stdout:7/684: creat d0/db/d26/f103 x:0 0 0 2024-02-15T18:27:14.483 INFO:tasks.workunit.client.1.smithi191.stdout:6/704: symlink d8/d9/dac/db0/d95/d9d/d6b/d98/ld6 0 2024-02-15T18:27:14.491 INFO:tasks.workunit.client.1.smithi191.stdout:7/685: truncate d0/db/d26/d5f/d77/f97 141305 0 2024-02-15T18:27:14.492 INFO:tasks.workunit.client.1.smithi191.stdout:6/705: mkdir d8/d9/d2b/d5b/d9f/dd7 0 2024-02-15T18:27:14.496 INFO:tasks.workunit.client.1.smithi191.stdout:0/505: dwrite d7/da/d1a/d34/f99 [0,4194304] 0 2024-02-15T18:27:14.496 INFO:tasks.workunit.client.1.smithi191.stdout:6/706: write d8/f33 [3076077,63375] 0 2024-02-15T18:27:14.498 INFO:tasks.workunit.client.1.smithi191.stdout:0/506: creat d7/d3b/fb2 x:0 0 0 2024-02-15T18:27:14.498 INFO:tasks.workunit.client.1.smithi191.stdout:0/507: write d7/f6a [491139,96852] 0 2024-02-15T18:27:14.499 INFO:tasks.workunit.client.1.smithi191.stdout:0/508: write d7/d3b/f71 [852535,27513] 0 2024-02-15T18:27:14.499 INFO:tasks.workunit.client.1.smithi191.stdout:0/509: write d7/da/d1a/d34/d27/f60 [1763661,123197] 0 2024-02-15T18:27:14.499 INFO:tasks.workunit.client.1.smithi191.stdout:0/510: dread - d7/da/d17/d31/d5f/f61 zero size 2024-02-15T18:27:14.500 INFO:tasks.workunit.client.1.smithi191.stdout:0/511: mknod d7/da/d1a/d2c/d4c/d98/cb3 0 2024-02-15T18:27:14.503 INFO:tasks.workunit.client.1.smithi191.stdout:0/512: write d7/da/d1a/d2c/d4c/f9f [1126560,127700] 0 2024-02-15T18:27:14.512 INFO:tasks.workunit.client.1.smithi191.stdout:0/513: dread d7/da/d1a/f90 [0,4194304] 0 2024-02-15T18:27:14.512 INFO:tasks.workunit.client.1.smithi191.stdout:0/514: fdatasync d7/d4a/f85 0 2024-02-15T18:27:14.513 INFO:tasks.workunit.client.1.smithi191.stdout:0/515: link d7/d1b/c7f d7/da/d17/d31/cb4 0 2024-02-15T18:27:14.514 INFO:tasks.workunit.client.1.smithi191.stdout:6/707: dwrite d8/d9/d2b/f51 [0,4194304] 0 2024-02-15T18:27:14.514 INFO:tasks.workunit.client.1.smithi191.stdout:5/484: dwrite d0/d23/f43 [0,4194304] 0 2024-02-15T18:27:14.514 INFO:tasks.workunit.client.1.smithi191.stdout:0/516: read d7/da/d1a/d34/d27/f7c [132937,21282] 0 2024-02-15T18:27:14.514 INFO:tasks.workunit.client.1.smithi191.stdout:9/720: dwrite d0/d7/dc/fae [0,4194304] 0 2024-02-15T18:27:14.516 INFO:tasks.workunit.client.1.smithi191.stdout:6/708: link d8/d9/f86 d8/d9/d4a/d4c/d7f/fd8 0 2024-02-15T18:27:14.516 INFO:tasks.workunit.client.1.smithi191.stdout:7/686: dwrite d0/db/d65/d11/d5a/d66/d1e/faf [0,4194304] 0 2024-02-15T18:27:14.519 INFO:tasks.workunit.client.1.smithi191.stdout:5/485: rmdir d0/d23/d41/d28/d31/d5a 39 2024-02-15T18:27:14.520 INFO:tasks.workunit.client.1.smithi191.stdout:5/486: write d0/d18/f42 [459251,21035] 0 2024-02-15T18:27:14.523 INFO:tasks.workunit.client.1.smithi191.stdout:0/517: unlink d7/da/d1a/l74 0 2024-02-15T18:27:14.525 INFO:tasks.workunit.client.1.smithi191.stdout:9/721: truncate d0/d7/f9 3259307 0 2024-02-15T18:27:14.525 INFO:tasks.workunit.client.1.smithi191.stdout:9/722: write d0/d7/dc/fa5 [409288,125662] 0 2024-02-15T18:27:14.526 INFO:tasks.workunit.client.1.smithi191.stdout:9/723: read d0/d29/d48/fdd [346193,32930] 0 2024-02-15T18:27:14.533 INFO:tasks.workunit.client.1.smithi191.stdout:6/709: symlink d8/d9/dac/db0/d95/d9d/d6b/d72/ld9 0 2024-02-15T18:27:14.533 INFO:tasks.workunit.client.1.smithi191.stdout:6/710: readlink d8/d9/dac/db0/d95/lcc 0 2024-02-15T18:27:14.535 INFO:tasks.workunit.client.1.smithi191.stdout:5/487: symlink d0/d18/l9b 0 2024-02-15T18:27:14.536 INFO:tasks.workunit.client.1.smithi191.stdout:0/518: getdents d7 0 2024-02-15T18:27:14.549 INFO:tasks.workunit.client.1.smithi191.stdout:9/724: unlink d0/d7/dc/c1e 0 2024-02-15T18:27:14.549 INFO:tasks.workunit.client.1.smithi191.stdout:9/725: fdatasync d0/d7/dc/d17/d33/ff2 0 2024-02-15T18:27:14.550 INFO:tasks.workunit.client.1.smithi191.stdout:9/726: stat d0/d7/dc/d17/d44/de6 0 2024-02-15T18:27:14.550 INFO:tasks.workunit.client.1.smithi191.stdout:9/727: fdatasync d0/d29/fb1 0 2024-02-15T18:27:14.550 INFO:tasks.workunit.client.1.smithi191.stdout:6/711: creat d8/d9/d2b/d57/fda x:0 0 0 2024-02-15T18:27:14.550 INFO:tasks.workunit.client.1.smithi191.stdout:6/712: write d8/d9/d2b/d57/fa9 [4901346,111197] 0 2024-02-15T18:27:14.550 INFO:tasks.workunit.client.1.smithi191.stdout:7/687: truncate d0/db/d65/d11/d1c/f45 720474 0 2024-02-15T18:27:14.553 INFO:tasks.workunit.client.1.smithi191.stdout:9/728: truncate d0/d7/dc/d30/d3b/d8b/fd7 1476642 0 2024-02-15T18:27:14.553 INFO:tasks.workunit.client.1.smithi191.stdout:9/729: readlink d0/d29/d48/d66/d92/l7c 0 2024-02-15T18:27:14.553 INFO:tasks.workunit.client.1.smithi191.stdout:9/730: stat d0/d7/dc/d30/d3b/d56/fc7 0 2024-02-15T18:27:14.555 INFO:tasks.workunit.client.1.smithi191.stdout:6/713: rename d8/d9/d2b/d5b/d9f to d8/d9/dac/ddb 0 2024-02-15T18:27:14.555 INFO:tasks.workunit.client.1.smithi191.stdout:6/714: truncate d8/d9/f65 699951 0 2024-02-15T18:27:14.556 INFO:tasks.workunit.client.1.smithi191.stdout:7/688: unlink d0/db/d26/d5f/d77/c36 0 2024-02-15T18:27:14.557 INFO:tasks.workunit.client.1.smithi191.stdout:5/488: dwrite d0/d23/d41/d28/d65/d85/d52/f60 [0,4194304] 0 2024-02-15T18:27:14.557 INFO:tasks.workunit.client.1.smithi191.stdout:9/731: unlink d0/d1d/fef 0 2024-02-15T18:27:14.558 INFO:tasks.workunit.client.1.smithi191.stdout:6/715: unlink d8/d9/d2b/d84/fd5 0 2024-02-15T18:27:14.561 INFO:tasks.workunit.client.1.smithi191.stdout:7/689: rmdir d0/db/d65/d11/dd7 39 2024-02-15T18:27:14.562 INFO:tasks.workunit.client.1.smithi191.stdout:7/690: write d0/db/d65/d11/fc4 [644596,115517] 0 2024-02-15T18:27:14.562 INFO:tasks.workunit.client.1.smithi191.stdout:7/691: chown d0/db/d65/d11/d1c/ldb 530804 1 2024-02-15T18:27:14.562 INFO:tasks.workunit.client.1.smithi191.stdout:5/489: creat d0/d18/d5c/d7c/f9c x:0 0 0 2024-02-15T18:27:14.564 INFO:tasks.workunit.client.1.smithi191.stdout:6/716: creat d8/d9/d4a/d4c/d7f/dcf/fdc x:0 0 0 2024-02-15T18:27:14.566 INFO:tasks.workunit.client.1.smithi191.stdout:5/490: link d0/d18/d5c/l6e d0/d23/d41/d28/d31/l9d 0 2024-02-15T18:27:14.566 INFO:tasks.workunit.client.1.smithi191.stdout:0/519: dwrite d7/f6a [0,4194304] 0 2024-02-15T18:27:14.566 INFO:tasks.workunit.client.1.smithi191.stdout:0/520: stat d7/d3b/l8b 0 2024-02-15T18:27:14.568 INFO:tasks.workunit.client.1.smithi191.stdout:5/491: write d0/d18/d5c/d7c/d49/f78 [457794,48386] 0 2024-02-15T18:27:14.578 INFO:tasks.workunit.client.1.smithi191.stdout:5/492: unlink d0/d23/d41/d28/d65/d85/d52/c4d 0 2024-02-15T18:27:14.578 INFO:tasks.workunit.client.1.smithi191.stdout:5/493: readlink d0/d18/d5c/l6e 0 2024-02-15T18:27:14.580 INFO:tasks.workunit.client.1.smithi191.stdout:7/692: dread d0/db/f8d [0,4194304] 0 2024-02-15T18:27:14.580 INFO:tasks.workunit.client.1.smithi191.stdout:7/693: write d0/db/d65/d11/d88/fec [401771,101524] 0 2024-02-15T18:27:14.582 INFO:tasks.workunit.client.1.smithi191.stdout:9/732: dwrite d0/d29/d48/d66/d92/fb4 [0,4194304] 0 2024-02-15T18:27:14.583 INFO:tasks.workunit.client.1.smithi191.stdout:0/521: fsync d7/d3b/f71 0 2024-02-15T18:27:14.583 INFO:tasks.workunit.client.1.smithi191.stdout:0/522: write d7/da/d1a/d2c/f4f [548903,71827] 0 2024-02-15T18:27:14.583 INFO:tasks.workunit.client.1.smithi191.stdout:7/694: mkdir d0/db/d26/d5f/d77/d35/d42/d104 0 2024-02-15T18:27:14.594 INFO:tasks.workunit.client.1.smithi191.stdout:0/523: rename d7/da/d17/d31/d5f/d9e/ca7 to d7/da/d17/d31/d5f/dae/cb5 0 2024-02-15T18:27:14.595 INFO:tasks.workunit.client.1.smithi191.stdout:7/695: mknod d0/db/d26/d5f/d77/d35/c105 0 2024-02-15T18:27:14.596 INFO:tasks.workunit.client.1.smithi191.stdout:7/696: chown d0/db/d65/d11/d88/d9d/dd2/c81 32453 1 2024-02-15T18:27:14.596 INFO:tasks.workunit.client.1.smithi191.stdout:7/697: stat d0/db/d65/d11/d5a/fb6 0 2024-02-15T18:27:14.603 INFO:tasks.workunit.client.1.smithi191.stdout:0/524: symlink d7/d4a/lb6 0 2024-02-15T18:27:14.603 INFO:tasks.workunit.client.1.smithi191.stdout:0/525: dread - d7/da/d1a/d34/f6d zero size 2024-02-15T18:27:14.603 INFO:tasks.workunit.client.1.smithi191.stdout:0/526: fsync d7/f39 0 2024-02-15T18:27:14.607 INFO:tasks.workunit.client.1.smithi191.stdout:7/698: creat d0/db/d26/d5f/d8c/f106 x:0 0 0 2024-02-15T18:27:14.608 INFO:tasks.workunit.client.1.smithi191.stdout:9/733: dwrite d0/d7/dc/d17/d44/fd8 [0,4194304] 0 2024-02-15T18:27:14.608 INFO:tasks.workunit.client.1.smithi191.stdout:9/734: fdatasync d0/d7/fd 0 2024-02-15T18:27:14.609 INFO:tasks.workunit.client.1.smithi191.stdout:7/699: rename d0/db/d26/d5f/d77/l2b to d0/db/d65/dc/l107 0 2024-02-15T18:27:14.611 INFO:tasks.workunit.client.1.smithi191.stdout:9/735: dread d0/d7/dc/d32/da3/fb6 [0,4194304] 0 2024-02-15T18:27:14.613 INFO:tasks.workunit.client.1.smithi191.stdout:7/700: mknod d0/db/ddd/c108 0 2024-02-15T18:27:14.615 INFO:tasks.workunit.client.1.smithi191.stdout:9/736: rename d0/d7/dc/d30/d3b/d8b/cbe to d0/d7/dc/d17/d44/cf6 0 2024-02-15T18:27:14.616 INFO:tasks.workunit.client.1.smithi191.stdout:7/701: dread d0/db/d26/d5f/d77/d35/d42/fa9 [0,4194304] 0 2024-02-15T18:27:14.616 INFO:tasks.workunit.client.1.smithi191.stdout:7/702: dread - d0/db/d65/dc/fa8 zero size 2024-02-15T18:27:14.616 INFO:tasks.workunit.client.1.smithi191.stdout:9/737: write d0/d7/dc/d8c/f3e [3162305,56019] 0 2024-02-15T18:27:14.618 INFO:tasks.workunit.client.1.smithi191.stdout:7/703: symlink d0/db/d65/d11/d88/l109 0 2024-02-15T18:27:14.624 INFO:tasks.workunit.client.1.smithi191.stdout:9/738: mknod d0/d1d/d3f/d80/cf7 0 2024-02-15T18:27:14.625 INFO:tasks.workunit.client.1.smithi191.stdout:7/704: symlink d0/db/d65/d11/d1c/l10a 0 2024-02-15T18:27:14.627 INFO:tasks.workunit.client.1.smithi191.stdout:0/527: dwrite d7/da/d1a/d2c/f8c [0,4194304] 0 2024-02-15T18:27:14.628 INFO:tasks.workunit.client.1.smithi191.stdout:9/739: creat d0/d7/dc/d17/d44/de6/ff8 x:0 0 0 2024-02-15T18:27:14.628 INFO:tasks.workunit.client.1.smithi191.stdout:9/740: write d0/d7/dc/d30/d3b/d56/fb5 [1076394,47867] 0 2024-02-15T18:27:14.629 INFO:tasks.workunit.client.1.smithi191.stdout:7/705: creat d0/db/ddd/f10b x:0 0 0 2024-02-15T18:27:14.629 INFO:tasks.workunit.client.1.smithi191.stdout:7/706: fdatasync d0/db/d65/d11/d88/d9d/dd2/f95 0 2024-02-15T18:27:14.629 INFO:tasks.workunit.client.1.smithi191.stdout:7/707: chown d0/db/d65/dc/d4b/f5b 16237 1 2024-02-15T18:27:14.633 INFO:tasks.workunit.client.1.smithi191.stdout:0/528: mknod d7/d4a/cb7 0 2024-02-15T18:27:14.635 INFO:tasks.workunit.client.1.smithi191.stdout:9/741: unlink d0/d7/dc/d30/f4e 0 2024-02-15T18:27:14.644 INFO:tasks.workunit.client.1.smithi191.stdout:7/708: mknod d0/db/d65/d11/d5a/d66/d1e/df5/c10c 0 2024-02-15T18:27:14.644 INFO:tasks.workunit.client.1.smithi191.stdout:9/742: creat d0/d29/d48/d66/d92/ff9 x:0 0 0 2024-02-15T18:27:14.646 INFO:tasks.workunit.client.1.smithi191.stdout:0/529: dread d7/d4a/f7b [0,4194304] 0 2024-02-15T18:27:14.646 INFO:tasks.workunit.client.1.smithi191.stdout:0/530: truncate d7/fb1 396068 0 2024-02-15T18:27:14.684 INFO:tasks.workunit.client.1.smithi191.stdout:7/709: mknod d0/db/d26/d5f/d77/df/d5d/c10d 0 2024-02-15T18:27:14.693 INFO:tasks.workunit.client.1.smithi191.stdout:9/743: readlink d0/d7/dc/d17/lda 0 2024-02-15T18:27:14.693 INFO:tasks.workunit.client.1.smithi191.stdout:0/531: mknod d7/da/d1a/d34/d27/cb8 0 2024-02-15T18:27:14.693 INFO:tasks.workunit.client.1.smithi191.stdout:0/532: fsync d7/da/d1a/d2c/d4c/fac 0 2024-02-15T18:27:14.697 INFO:tasks.workunit.client.1.smithi191.stdout:7/710: link d0/db/d26/l86 d0/db/ddd/dea/l10e 0 2024-02-15T18:27:14.699 INFO:tasks.workunit.client.1.smithi191.stdout:0/533: creat d7/d94/fb9 x:0 0 0 2024-02-15T18:27:14.699 INFO:tasks.workunit.client.1.smithi191.stdout:0/534: write d7/da/d6c/f93 [925680,100362] 0 2024-02-15T18:27:14.704 INFO:tasks.workunit.client.1.smithi191.stdout:7/711: rename d0/db/d65/d11/d57/cfd to d0/db/d26/d5f/d77/db9/c10f 0 2024-02-15T18:27:14.706 INFO:tasks.workunit.client.1.smithi191.stdout:7/712: write d0/f9e [3842656,112390] 0 2024-02-15T18:27:14.706 INFO:tasks.workunit.client.1.smithi191.stdout:7/713: truncate d0/db/d26/fb2 1291687 0 2024-02-15T18:27:14.708 INFO:tasks.workunit.client.1.smithi191.stdout:9/744: dread d0/d7/dc/d8c/f72 [4194304,4194304] 0 2024-02-15T18:27:14.710 INFO:tasks.workunit.client.1.smithi191.stdout:0/535: symlink d7/da/d6c/lba 0 2024-02-15T18:27:14.710 INFO:tasks.workunit.client.1.smithi191.stdout:0/536: fdatasync d7/da/d1a/d2c/f4b 0 2024-02-15T18:27:14.711 INFO:tasks.workunit.client.1.smithi191.stdout:0/537: mkdir d7/da/d17/d31/d5f/dbb 0 2024-02-15T18:27:14.712 INFO:tasks.workunit.client.1.smithi191.stdout:9/745: link d0/c65 d0/d7/cfa 0 2024-02-15T18:27:14.714 INFO:tasks.workunit.client.1.smithi191.stdout:0/538: creat d7/da/d17/d31/d5f/fbc x:0 0 0 2024-02-15T18:27:14.715 INFO:tasks.workunit.client.1.smithi191.stdout:9/746: creat d0/d7/dc/d30/d3b/ffb x:0 0 0 2024-02-15T18:27:14.716 INFO:tasks.workunit.client.1.smithi191.stdout:0/539: symlink d7/d3b/lbd 0 2024-02-15T18:27:14.716 INFO:tasks.workunit.client.1.smithi191.stdout:0/540: chown d7/da/d1a/d2c/d4c/c67 35266616 1 2024-02-15T18:27:14.718 INFO:tasks.workunit.client.1.smithi191.stdout:9/747: mknod d0/d7/dc/d30/d3b/d56/cfc 0 2024-02-15T18:27:14.718 INFO:tasks.workunit.client.1.smithi191.stdout:9/748: write d0/d7/dc/f9b [177530,115208] 0 2024-02-15T18:27:14.718 INFO:tasks.workunit.client.1.smithi191.stdout:9/749: read d0/d7/dc/f9b [198184,55212] 0 2024-02-15T18:27:14.718 INFO:tasks.workunit.client.1.smithi191.stdout:0/541: creat d7/da/d17/d31/d5f/dae/fbe x:0 0 0 2024-02-15T18:27:14.720 INFO:tasks.workunit.client.1.smithi191.stdout:9/750: mkdir d0/d29/d48/d66/d92/dfd 0 2024-02-15T18:27:14.732 INFO:tasks.workunit.client.1.smithi191.stdout:7/714: dwrite d0/db/d65/d11/d5a/d66/d1e/f8a [4194304,4194304] 0 2024-02-15T18:27:14.732 INFO:tasks.workunit.client.1.smithi191.stdout:9/751: dread d0/d7/dc/d8c/f72 [0,4194304] 0 2024-02-15T18:27:14.732 INFO:tasks.workunit.client.1.smithi191.stdout:9/752: write d0/d7/dc/d30/f5c [4231255,99406] 0 2024-02-15T18:27:14.732 INFO:tasks.workunit.client.1.smithi191.stdout:9/753: fsync d0/d7/dc/d30/d3b/d56/fb5 0 2024-02-15T18:27:14.732 INFO:tasks.workunit.client.1.smithi191.stdout:9/754: write d0/d7/dc/f9b [784618,68510] 0 2024-02-15T18:27:14.732 INFO:tasks.workunit.client.1.smithi191.stdout:9/755: dread - d0/d7/dc/d17/d44/de6/fea zero size 2024-02-15T18:27:14.733 INFO:tasks.workunit.client.1.smithi191.stdout:7/715: mkdir d0/db/d26/d5f/d77/d35/d42/d104/d110 0 2024-02-15T18:27:14.734 INFO:tasks.workunit.client.1.smithi191.stdout:9/756: chown d0/d7/dc/d17/f25 898286248 1 2024-02-15T18:27:14.734 INFO:tasks.workunit.client.1.smithi191.stdout:9/757: stat d0/d29/d48/d66/d92/fd4 0 2024-02-15T18:27:14.734 INFO:tasks.workunit.client.1.smithi191.stdout:8/797: sync 2024-02-15T18:27:14.735 INFO:tasks.workunit.client.1.smithi191.stdout:7/716: getdents d0/db 0 2024-02-15T18:27:14.739 INFO:tasks.workunit.client.1.smithi191.stdout:8/798: link d3/ffe d3/d2c/d1a/d23/d2a/f113 0 2024-02-15T18:27:14.739 INFO:tasks.workunit.client.1.smithi191.stdout:8/799: chown d3/d2c/d1a/d23/d3e/f68 31 1 2024-02-15T18:27:14.740 INFO:tasks.workunit.client.1.smithi191.stdout:0/542: dwrite d7/da/d1a/d34/f11 [0,4194304] 0 2024-02-15T18:27:14.743 INFO:tasks.workunit.client.1.smithi191.stdout:9/758: write d0/d29/d48/f96 [1419109,57788] 0 2024-02-15T18:27:14.743 INFO:tasks.workunit.client.1.smithi191.stdout:7/717: mkdir d0/db/d65/d11/d5a/d111 0 2024-02-15T18:27:14.744 INFO:tasks.workunit.client.1.smithi191.stdout:8/800: link d3/d2c/d1a/fce d3/d2c/f114 0 2024-02-15T18:27:14.744 INFO:tasks.workunit.client.1.smithi191.stdout:8/801: fdatasync d3/d2c/d1a/f8f 0 2024-02-15T18:27:14.747 INFO:tasks.workunit.client.1.smithi191.stdout:0/543: rename d7/da/d17/f5d to d7/da/d6c/fbf 0 2024-02-15T18:27:14.750 INFO:tasks.workunit.client.1.smithi191.stdout:9/759: dread d0/d7/dc/d17/f84 [0,4194304] 0 2024-02-15T18:27:14.750 INFO:tasks.workunit.client.1.smithi191.stdout:7/718: creat d0/db/ddd/dea/f112 x:0 0 0 2024-02-15T18:27:14.751 INFO:tasks.workunit.client.1.smithi191.stdout:9/760: write d0/d7/dc/d30/fee [162478,83117] 0 2024-02-15T18:27:14.752 INFO:tasks.workunit.client.1.smithi191.stdout:7/719: getdents d0/db/d26/d5f/d77/d35/d42/d104 0 2024-02-15T18:27:14.752 INFO:tasks.workunit.client.1.smithi191.stdout:9/761: write d0/d7/dc/d30/d3b/f55 [694523,47671] 0 2024-02-15T18:27:14.753 INFO:tasks.workunit.client.1.smithi191.stdout:7/720: mkdir d0/db/d26/d5f/d77/d35/d113 0 2024-02-15T18:27:14.758 INFO:tasks.workunit.client.1.smithi191.stdout:0/544: dread d7/d1b/f46 [0,4194304] 0 2024-02-15T18:27:14.770 INFO:tasks.workunit.client.1.smithi191.stdout:7/721: dread d0/db/d26/d5f/d77/d35/d42/fa7 [0,4194304] 0 2024-02-15T18:27:14.770 INFO:tasks.workunit.client.1.smithi191.stdout:7/722: dread - d0/db/d26/d5f/d77/d35/fd4 zero size 2024-02-15T18:27:14.771 INFO:tasks.workunit.client.1.smithi191.stdout:8/802: dwrite d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/da0/ff1 [0,4194304] 0 2024-02-15T18:27:14.773 INFO:tasks.workunit.client.1.smithi191.stdout:7/723: dread d0/db/d65/d11/d5a/d66/d1e/fa5 [0,4194304] 0 2024-02-15T18:27:14.773 INFO:tasks.workunit.client.1.smithi191.stdout:8/803: write d3/d5e/fd1 [1932980,4828] 0 2024-02-15T18:27:14.773 INFO:tasks.workunit.client.1.smithi191.stdout:7/724: read d0/db/d65/d11/d57/fcf [31354,11113] 0 2024-02-15T18:27:14.774 INFO:tasks.workunit.client.1.smithi191.stdout:7/725: creat d0/db/d65/dc/d4b/dc1/f114 x:0 0 0 2024-02-15T18:27:14.775 INFO:tasks.workunit.client.1.smithi191.stdout:7/726: truncate d0/db/d26/d5f/d77/d35/d42/fa9 741223 0 2024-02-15T18:27:14.775 INFO:tasks.workunit.client.1.smithi191.stdout:7/727: write d0/db/d26/d5f/d77/df/f19 [4578807,35493] 0 2024-02-15T18:27:14.781 INFO:tasks.workunit.client.1.smithi191.stdout:8/804: dread d3/d2c/d1a/d23/d2e/d5b/fc9 [0,4194304] 0 2024-02-15T18:27:14.782 INFO:tasks.workunit.client.1.smithi191.stdout:8/805: chown d3/l40 1 1 2024-02-15T18:27:14.785 INFO:tasks.workunit.client.1.smithi191.stdout:0/545: dwrite d7/d4a/f59 [4194304,4194304] 0 2024-02-15T18:27:14.785 INFO:tasks.workunit.client.1.smithi191.stdout:0/546: chown d7/d3b 897455 1 2024-02-15T18:27:14.785 INFO:tasks.workunit.client.1.smithi191.stdout:0/547: fsync d7/da/d1a/d2c/f7e 0 2024-02-15T18:27:14.786 INFO:tasks.workunit.client.1.smithi191.stdout:9/762: dwrite d0/d7/dc/d17/d44/fba [0,4194304] 0 2024-02-15T18:27:14.786 INFO:tasks.workunit.client.1.smithi191.stdout:8/806: truncate d3/d5e/f80 2756970 0 2024-02-15T18:27:14.788 INFO:tasks.workunit.client.1.smithi191.stdout:0/548: mkdir d7/d4a/dc0 0 2024-02-15T18:27:14.792 INFO:tasks.workunit.client.1.smithi191.stdout:9/763: rename d0/d29/d48/cd2 to d0/d7/dbf/cfe 0 2024-02-15T18:27:14.792 INFO:tasks.workunit.client.1.smithi191.stdout:0/549: symlink d7/da/d1a/lc1 0 2024-02-15T18:27:14.794 INFO:tasks.workunit.client.1.smithi191.stdout:7/728: dread d0/f9e [0,4194304] 0 2024-02-15T18:27:14.798 INFO:tasks.workunit.client.1.smithi191.stdout:9/764: getdents d0/d7/dc/d30/d3b/d8b 0 2024-02-15T18:27:14.801 INFO:tasks.workunit.client.1.smithi191.stdout:7/729: dread d0/db/f8d [0,4194304] 0 2024-02-15T18:27:14.801 INFO:tasks.workunit.client.1.smithi191.stdout:8/807: dread d3/d2c/d1a/d23/d2e/d64/f76 [0,4194304] 0 2024-02-15T18:27:14.801 INFO:tasks.workunit.client.1.smithi191.stdout:9/765: write d0/d7/dc/d8c/f3e [3453849,7207] 0 2024-02-15T18:27:14.804 INFO:tasks.workunit.client.1.smithi191.stdout:8/808: stat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/ccb 0 2024-02-15T18:27:14.805 INFO:tasks.workunit.client.1.smithi191.stdout:0/550: truncate d7/da/d6c/f93 510911 0 2024-02-15T18:27:14.807 INFO:tasks.workunit.client.1.smithi191.stdout:9/766: truncate d0/d7/dc/d8c/f27 2088197 0 2024-02-15T18:27:14.808 INFO:tasks.workunit.client.1.smithi191.stdout:0/551: link d7/d1b/c7f d7/da/d1a/d34/cc2 0 2024-02-15T18:27:14.810 INFO:tasks.workunit.client.1.smithi191.stdout:7/730: dread d0/db/d26/d5f/d77/d35/dc7/fe5 [4194304,4194304] 0 2024-02-15T18:27:14.813 INFO:tasks.workunit.client.1.smithi191.stdout:0/552: truncate d7/da/d1a/d2c/f8c 2812786 0 2024-02-15T18:27:14.813 INFO:tasks.workunit.client.1.smithi191.stdout:0/553: write d7/d4a/f85 [856472,57947] 0 2024-02-15T18:27:14.817 INFO:tasks.workunit.client.1.smithi191.stdout:7/731: creat d0/db/d26/d5f/d77/d35/d42/d104/d110/f115 x:0 0 0 2024-02-15T18:27:14.819 INFO:tasks.workunit.client.1.smithi191.stdout:7/732: unlink d0/l5e 0 2024-02-15T18:27:14.821 INFO:tasks.workunit.client.1.smithi191.stdout:7/733: write d0/db/f8d [835153,68984] 0 2024-02-15T18:27:14.821 INFO:tasks.workunit.client.1.smithi191.stdout:7/734: read d0/db/d65/d11/d5a/d66/f79 [240069,3509] 0 2024-02-15T18:27:14.821 INFO:tasks.workunit.client.1.smithi191.stdout:7/735: write d0/db/ddd/dea/f112 [899348,7820] 0 2024-02-15T18:27:14.821 INFO:tasks.workunit.client.1.smithi191.stdout:7/736: dread - d0/db/d26/d5f/d77/fcd zero size 2024-02-15T18:27:14.825 INFO:tasks.workunit.client.1.smithi191.stdout:0/554: dread d7/d1b/f96 [0,4194304] 0 2024-02-15T18:27:14.825 INFO:tasks.workunit.client.1.smithi191.stdout:0/555: chown d7/d4a/f85 0 1 2024-02-15T18:27:14.825 INFO:tasks.workunit.client.1.smithi191.stdout:7/737: creat d0/db/d65/dc/d4b/dc1/def/f116 x:0 0 0 2024-02-15T18:27:14.825 INFO:tasks.workunit.client.1.smithi191.stdout:7/738: read d0/db/d26/d5f/d77/f21 [1074676,64577] 0 2024-02-15T18:27:14.826 INFO:tasks.workunit.client.1.smithi191.stdout:0/556: mkdir d7/da/d17/d31/d5f/d9e/dc3 0 2024-02-15T18:27:14.827 INFO:tasks.workunit.client.1.smithi191.stdout:7/739: creat d0/db/d65/df1/f117 x:0 0 0 2024-02-15T18:27:14.828 INFO:tasks.workunit.client.1.smithi191.stdout:8/809: dwrite d3/d2c/d1a/f8f [0,4194304] 0 2024-02-15T18:27:14.831 INFO:tasks.workunit.client.1.smithi191.stdout:9/767: dwrite d0/d1d/f79 [0,4194304] 0 2024-02-15T18:27:14.834 INFO:tasks.workunit.client.1.smithi191.stdout:7/740: mknod d0/db/d65/df1/c118 0 2024-02-15T18:27:14.834 INFO:tasks.workunit.client.1.smithi191.stdout:7/741: truncate d0/db/d65/d11/d57/fcf 476101 0 2024-02-15T18:27:14.843 INFO:tasks.workunit.client.1.smithi191.stdout:0/557: dwrite d7/da/d1a/d34/f52 [0,4194304] 0 2024-02-15T18:27:14.843 INFO:tasks.workunit.client.1.smithi191.stdout:0/558: chown d7/da/d17/ca8 109251 1 2024-02-15T18:27:14.843 INFO:tasks.workunit.client.1.smithi191.stdout:0/559: truncate d7/da/d1a/d34/d27/f6f 435461 0 2024-02-15T18:27:14.843 INFO:tasks.workunit.client.1.smithi191.stdout:0/560: truncate d7/d1b/f23 4801845 0 2024-02-15T18:27:14.846 INFO:tasks.workunit.client.1.smithi191.stdout:0/561: write d7/da/d1a/d34/f99 [2510808,112165] 0 2024-02-15T18:27:14.846 INFO:tasks.workunit.client.1.smithi191.stdout:0/562: rename d7/da/d1a/d34 to d7/da/d1a/d34/d45/dc4 22 2024-02-15T18:27:14.846 INFO:tasks.workunit.client.1.smithi191.stdout:0/563: fsync d7/da/d17/d31/f87 0 2024-02-15T18:27:14.857 INFO:tasks.workunit.client.1.smithi191.stdout:9/768: rmdir d0/d7/dc/d17/d44/de6 39 2024-02-15T18:27:14.860 INFO:tasks.workunit.client.1.smithi191.stdout:7/742: truncate d0/db/d65/d11/d5a/d66/d1e/fa5 705218 0 2024-02-15T18:27:14.865 INFO:tasks.workunit.client.1.smithi191.stdout:9/769: dread d0/d29/d48/d66/d92/fb4 [0,4194304] 0 2024-02-15T18:27:14.865 INFO:tasks.workunit.client.1.smithi191.stdout:9/770: write d0/d29/d48/fc3 [1134624,109888] 0 2024-02-15T18:27:14.865 INFO:tasks.workunit.client.1.smithi191.stdout:9/771: write d0/d7/dc/d30/d3b/d56/fc7 [773777,23583] 0 2024-02-15T18:27:14.870 INFO:tasks.workunit.client.1.smithi191.stdout:4/746: sync 2024-02-15T18:27:14.870 INFO:tasks.workunit.client.1.smithi191.stdout:3/598: sync 2024-02-15T18:27:14.876 INFO:tasks.workunit.client.1.smithi191.stdout:0/564: unlink d7/d3b/lbd 0 2024-02-15T18:27:14.878 INFO:tasks.workunit.client.1.smithi191.stdout:9/772: creat d0/d29/d48/d66/fff x:0 0 0 2024-02-15T18:27:14.881 INFO:tasks.workunit.client.1.smithi191.stdout:4/747: unlink d2/d5/d7/d3e/d63/df0/c100 0 2024-02-15T18:27:14.881 INFO:tasks.workunit.client.1.smithi191.stdout:4/748: readlink d2/l1d 0 2024-02-15T18:27:14.881 INFO:tasks.workunit.client.1.smithi191.stdout:4/749: dread - d2/d5/d7/d3e/fc0 zero size 2024-02-15T18:27:14.881 INFO:tasks.workunit.client.1.smithi191.stdout:4/750: dread - d2/d5/d7/fcc zero size 2024-02-15T18:27:14.887 INFO:tasks.workunit.client.1.smithi191.stdout:3/599: mknod d6/d5d/daa/cca 0 2024-02-15T18:27:14.887 INFO:tasks.workunit.client.1.smithi191.stdout:9/773: creat d0/d1d/d3f/d80/f100 x:0 0 0 2024-02-15T18:27:14.903 INFO:tasks.workunit.client.1.smithi191.stdout:4/751: dwrite d2/d5/d14/d47/d78/dd0/dd3/de4/ff6 [0,4194304] 0 2024-02-15T18:27:14.903 INFO:tasks.workunit.client.1.smithi191.stdout:4/752: chown d2/d5/fc1 1 1 2024-02-15T18:27:14.903 INFO:tasks.workunit.client.1.smithi191.stdout:4/753: dread - d2/d5/d7/d90/fd4 zero size 2024-02-15T18:27:14.903 INFO:tasks.workunit.client.1.smithi191.stdout:4/754: chown d2/d5/d14/d47/d78/dd0/dd3/de4 27850567 1 2024-02-15T18:27:14.971 INFO:tasks.workunit.client.1.smithi191.stdout:0/565: dwrite d7/f6a [0,4194304] 0 2024-02-15T18:27:14.971 INFO:tasks.workunit.client.1.smithi191.stdout:0/566: dread - d7/da/d1a/d2c/d4c/fac zero size 2024-02-15T18:27:14.973 INFO:tasks.workunit.client.1.smithi191.stdout:0/567: rename d7/da/d1a/l44 to d7/da/d17/d31/d5f/d9e/dc3/lc5 0 2024-02-15T18:27:14.977 INFO:tasks.workunit.client.1.smithi191.stdout:9/774: dwrite d0/d7/dc/d30/d3b/ffb [0,4194304] 0 2024-02-15T18:27:14.977 INFO:tasks.workunit.client.1.smithi191.stdout:9/775: stat d0/d7/dc 0 2024-02-15T18:27:14.977 INFO:tasks.workunit.client.1.smithi191.stdout:7/743: dwrite d0/db/d65/d11/d5a/d66/d1e/fa5 [0,4194304] 0 2024-02-15T18:27:14.977 INFO:tasks.workunit.client.1.smithi191.stdout:7/744: getdents d0/db/d65/d11/d5a/d66/dd6 0 2024-02-15T18:27:14.977 INFO:tasks.workunit.client.1.smithi191.stdout:7/745: chown d0/db/d65/d11/d5a/d66 124790 1 2024-02-15T18:27:14.977 INFO:tasks.workunit.client.1.smithi191.stdout:7/746: stat d0/db/d65/d11/d5a/d66/f53 0 2024-02-15T18:27:14.977 INFO:tasks.workunit.client.1.smithi191.stdout:4/755: dwrite d2/d5/d7/d2f/fc7 [0,4194304] 0 2024-02-15T18:27:14.977 INFO:tasks.workunit.client.1.smithi191.stdout:0/568: creat d7/da/d1a/d34/fc6 x:0 0 0 2024-02-15T18:27:14.977 INFO:tasks.workunit.client.1.smithi191.stdout:3/600: dwrite d6/d1b/d60/d73/f75 [8388608,4194304] 0 2024-02-15T18:27:14.978 INFO:tasks.workunit.client.1.smithi191.stdout:9/776: mknod d0/d1d/d3f/c101 0 2024-02-15T18:27:14.990 INFO:tasks.workunit.client.1.smithi191.stdout:7/747: link d0/db/d65/d11/d88/l109 d0/db/d65/d11/d5a/d66/dd6/l119 0 2024-02-15T18:27:14.990 INFO:tasks.workunit.client.1.smithi191.stdout:7/748: stat d0/db/d65/d11/d57/fcf 0 2024-02-15T18:27:14.990 INFO:tasks.workunit.client.1.smithi191.stdout:7/749: readlink d0/db/d65/d11/d5a/lc0 0 2024-02-15T18:27:14.992 INFO:tasks.workunit.client.1.smithi191.stdout:0/569: dwrite d7/fb [0,4194304] 0 2024-02-15T18:27:14.995 INFO:tasks.workunit.client.1.smithi191.stdout:4/756: rename d2/fad to d2/d5/d7/d90/d52/f111 0 2024-02-15T18:27:14.997 INFO:tasks.workunit.client.1.smithi191.stdout:3/601: mknod d6/d1b/d82/ccb 0 2024-02-15T18:27:14.998 INFO:tasks.workunit.client.1.smithi191.stdout:9/777: creat d0/d7/dc/d8c/f102 x:0 0 0 2024-02-15T18:27:15.001 INFO:tasks.workunit.client.1.smithi191.stdout:7/750: getdents d0/db/d26/d5f/d77/d35/d42 0 2024-02-15T18:27:15.001 INFO:tasks.workunit.client.1.smithi191.stdout:7/751: stat d0/db/d65/d11/d88 0 2024-02-15T18:27:15.003 INFO:tasks.workunit.client.1.smithi191.stdout:0/570: unlink d7/d4a/lb6 0 2024-02-15T18:27:15.003 INFO:tasks.workunit.client.1.smithi191.stdout:0/571: dread - d7/da/d1a/d34/fa0 zero size 2024-02-15T18:27:15.005 INFO:tasks.workunit.client.1.smithi191.stdout:4/757: mknod d2/d5/d7/dee/c112 0 2024-02-15T18:27:15.005 INFO:tasks.workunit.client.1.smithi191.stdout:4/758: chown d2/f2c 7780 1 2024-02-15T18:27:15.005 INFO:tasks.workunit.client.1.smithi191.stdout:4/759: chown d2/d5/d7/fcc 18427481 1 2024-02-15T18:27:15.006 INFO:tasks.workunit.client.1.smithi191.stdout:3/602: creat d6/d1b/d60/da2/dac/fcc x:0 0 0 2024-02-15T18:27:15.006 INFO:tasks.workunit.client.1.smithi191.stdout:3/603: fdatasync d6/d3b/d42/d49/d54/f6b 0 2024-02-15T18:27:15.006 INFO:tasks.workunit.client.1.smithi191.stdout:3/604: truncate d6/d3b/d42/d49/f85 317201 0 2024-02-15T18:27:15.008 INFO:tasks.workunit.client.1.smithi191.stdout:0/572: mknod d7/d4a/cc7 0 2024-02-15T18:27:15.010 INFO:tasks.workunit.client.1.smithi191.stdout:4/760: creat d2/d5/d7/d3e/d63/d8c/dbb/f113 x:0 0 0 2024-02-15T18:27:15.010 INFO:tasks.workunit.client.1.smithi191.stdout:4/761: getdents d2/d5/d7/d90/d52/db5/dc9 0 2024-02-15T18:27:15.012 INFO:tasks.workunit.client.1.smithi191.stdout:3/605: symlink d6/d5d/daa/lcd 0 2024-02-15T18:27:15.012 INFO:tasks.workunit.client.1.smithi191.stdout:3/606: fsync d6/d1b/fa3 0 2024-02-15T18:27:15.014 INFO:tasks.workunit.client.1.smithi191.stdout:4/762: mkdir d2/d5/d7/d2f/d114 0 2024-02-15T18:27:15.014 INFO:tasks.workunit.client.1.smithi191.stdout:4/763: dread - d2/d5/d7/d3e/d63/d8c/da4/fd7 zero size 2024-02-15T18:27:15.017 INFO:tasks.workunit.client.1.smithi191.stdout:4/764: creat d2/d5/f115 x:0 0 0 2024-02-15T18:27:15.018 INFO:tasks.workunit.client.1.smithi191.stdout:4/765: symlink d2/d5/d7/l116 0 2024-02-15T18:27:15.018 INFO:tasks.workunit.client.1.smithi191.stdout:9/778: dwrite d0/d29/f86 [0,4194304] 0 2024-02-15T18:27:15.019 INFO:tasks.workunit.client.1.smithi191.stdout:9/779: write d0/d29/d48/f5b [4078062,58187] 0 2024-02-15T18:27:15.021 INFO:tasks.workunit.client.1.smithi191.stdout:9/780: stat d0/d1d/d3f/d80/ca6 0 2024-02-15T18:27:15.023 INFO:tasks.workunit.client.1.smithi191.stdout:7/752: dwrite d0/db/d26/f3c [0,4194304] 0 2024-02-15T18:27:15.023 INFO:tasks.workunit.client.1.smithi191.stdout:9/781: mknod d0/d29/d48/d66/c103 0 2024-02-15T18:27:15.025 INFO:tasks.workunit.client.1.smithi191.stdout:7/753: creat d0/db/d26/d5f/d77/d35/d42/d104/f11a x:0 0 0 2024-02-15T18:27:15.026 INFO:tasks.workunit.client.1.smithi191.stdout:9/782: chown d0/d7/dc/d17/lc4 172829 1 2024-02-15T18:27:15.027 INFO:tasks.workunit.client.1.smithi191.stdout:7/754: getdents d0 0 2024-02-15T18:27:15.027 INFO:tasks.workunit.client.1.smithi191.stdout:7/755: stat d0/db/d65/dc/d6e/c71 0 2024-02-15T18:27:15.028 INFO:tasks.workunit.client.1.smithi191.stdout:7/756: chown d0/db/d65/dc/d4b 162365308 1 2024-02-15T18:27:15.028 INFO:tasks.workunit.client.1.smithi191.stdout:7/757: write d0/db/d65/d11/fe9 [438186,71042] 0 2024-02-15T18:27:15.028 INFO:tasks.workunit.client.1.smithi191.stdout:7/758: write d0/db/d65/dc/fa8 [685158,20552] 0 2024-02-15T18:27:15.029 INFO:tasks.workunit.client.1.smithi191.stdout:9/783: mknod d0/d29/d48/d66/c104 0 2024-02-15T18:27:15.029 INFO:tasks.workunit.client.1.smithi191.stdout:7/759: symlink d0/db/d26/d5f/d77/df/l11b 0 2024-02-15T18:27:15.034 INFO:tasks.workunit.client.1.smithi191.stdout:0/573: dwrite d7/f6a [4194304,4194304] 0 2024-02-15T18:27:15.034 INFO:tasks.workunit.client.1.smithi191.stdout:3/607: dread d6/d1b/f39 [0,4194304] 0 2024-02-15T18:27:15.034 INFO:tasks.workunit.client.1.smithi191.stdout:0/574: chown d7/d3b/l77 6259 1 2024-02-15T18:27:15.042 INFO:tasks.workunit.client.1.smithi191.stdout:9/784: mknod d0/d7/dbf/df3/c105 0 2024-02-15T18:27:15.043 INFO:tasks.workunit.client.1.smithi191.stdout:9/785: write d0/d7/f9 [1397756,14600] 0 2024-02-15T18:27:15.043 INFO:tasks.workunit.client.1.smithi191.stdout:9/786: read d0/d7/dc/fa5 [519987,116055] 0 2024-02-15T18:27:15.047 INFO:tasks.workunit.client.1.smithi191.stdout:7/760: rename d0/db/d65/d11/d1c/f45 to d0/db/d65/d11/d5a/d66/d1e/df5/f11c 0 2024-02-15T18:27:15.048 INFO:tasks.workunit.client.1.smithi191.stdout:7/761: stat d0/db/d26/d5f/ffb 0 2024-02-15T18:27:15.058 INFO:tasks.workunit.client.1.smithi191.stdout:3/608: symlink d6/lce 0 2024-02-15T18:27:15.058 INFO:tasks.workunit.client.1.smithi191.stdout:0/575: getdents d7/d3b 0 2024-02-15T18:27:15.061 INFO:tasks.workunit.client.1.smithi191.stdout:9/787: rename d0/d7/dc/d30/d3b/d56/d59 to d0/d7/dc/d32/d106 0 2024-02-15T18:27:15.062 INFO:tasks.workunit.client.1.smithi191.stdout:9/788: truncate d0/d7/dc/d30/d4f/f6d 1942066 0 2024-02-15T18:27:15.062 INFO:tasks.workunit.client.1.smithi191.stdout:9/789: chown d0/c94 1 1 2024-02-15T18:27:15.071 INFO:tasks.workunit.client.1.smithi191.stdout:7/762: rmdir d0/db/d65/dc/d4b/dc1 39 2024-02-15T18:27:15.071 INFO:tasks.workunit.client.1.smithi191.stdout:0/576: creat d7/da/d17/d31/d7d/fc8 x:0 0 0 2024-02-15T18:27:15.073 INFO:tasks.workunit.client.1.smithi191.stdout:3/609: rename d6/d5d/daa/lcd to d6/d3b/d42/d49/dbf/lcf 0 2024-02-15T18:27:15.074 INFO:tasks.workunit.client.1.smithi191.stdout:3/610: write d6/d27/d4d/d5a/da4/fbc [665319,55872] 0 2024-02-15T18:27:15.074 INFO:tasks.workunit.client.1.smithi191.stdout:3/611: stat d6/d27 0 2024-02-15T18:27:15.074 INFO:tasks.workunit.client.1.smithi191.stdout:9/790: read d0/d7/dc/d17/d33/f43 [2669709,34225] 0 2024-02-15T18:27:15.076 INFO:tasks.workunit.client.1.smithi191.stdout:1/774: sync 2024-02-15T18:27:15.076 INFO:tasks.workunit.client.1.smithi191.stdout:1/775: write d0/d8/f17 [1136754,18254] 0 2024-02-15T18:27:15.077 INFO:tasks.workunit.client.1.smithi191.stdout:9/791: dread d0/d7/dc/d8c/f78 [0,4194304] 0 2024-02-15T18:27:15.078 INFO:tasks.workunit.client.1.smithi191.stdout:1/776: write d0/d2f/d51/d10/fb4 [2088636,129696] 0 2024-02-15T18:27:15.078 INFO:tasks.workunit.client.1.smithi191.stdout:7/763: creat d0/db/d65/dc/d4b/dc1/def/f11d x:0 0 0 2024-02-15T18:27:15.080 INFO:tasks.workunit.client.1.smithi191.stdout:0/577: rmdir d7/da 39 2024-02-15T18:27:15.083 INFO:tasks.workunit.client.1.smithi191.stdout:3/612: symlink d6/d5d/ld0 0 2024-02-15T18:27:15.087 INFO:tasks.workunit.client.1.smithi191.stdout:7/764: dread d0/db/d65/dc/d4b/fb1 [0,4194304] 0 2024-02-15T18:27:15.087 INFO:tasks.workunit.client.1.smithi191.stdout:7/765: read - d0/db/d26/d5f/d77/fcd zero size 2024-02-15T18:27:15.088 INFO:tasks.workunit.client.1.smithi191.stdout:9/792: unlink d0/d7/f28 0 2024-02-15T18:27:15.089 INFO:tasks.workunit.client.1.smithi191.stdout:9/793: dread d0/d7/dc/d30/d4f/fb0 [0,4194304] 0 2024-02-15T18:27:15.089 INFO:tasks.workunit.client.1.smithi191.stdout:9/794: truncate d0/d29/d48/d66/d92/ff9 958607 0 2024-02-15T18:27:15.089 INFO:tasks.workunit.client.1.smithi191.stdout:9/795: write d0/d7/dc/d30/d3b/d56/fc7 [770607,35471] 0 2024-02-15T18:27:15.089 INFO:tasks.workunit.client.1.smithi191.stdout:1/777: unlink d0/d8/c58 0 2024-02-15T18:27:15.089 INFO:tasks.workunit.client.1.smithi191.stdout:9/796: dread d0/d29/d48/d66/d92/fca [0,4194304] 0 2024-02-15T18:27:15.091 INFO:tasks.workunit.client.1.smithi191.stdout:3/613: creat d6/d27/d4d/d5a/fd1 x:0 0 0 2024-02-15T18:27:15.098 INFO:tasks.workunit.client.1.smithi191.stdout:7/766: rename d0/db/d65/d11/d1c/c7d to d0/db/d26/d5f/d77/c11e 0 2024-02-15T18:27:15.098 INFO:tasks.workunit.client.1.smithi191.stdout:7/767: chown d0/f99 611639 1 2024-02-15T18:27:15.100 INFO:tasks.workunit.client.1.smithi191.stdout:1/778: unlink d0/d8/df/d90/d94/dc8/f106 0 2024-02-15T18:27:15.100 INFO:tasks.workunit.client.1.smithi191.stdout:1/779: dread - d0/d2f/d51/d10/ff4 zero size 2024-02-15T18:27:15.100 INFO:tasks.workunit.client.1.smithi191.stdout:1/780: write d0/d8/df/d12/f3f [5222834,113008] 0 2024-02-15T18:27:15.100 INFO:tasks.workunit.client.1.smithi191.stdout:7/768: getdents d0/db/d26/d5f/d77/df/d5d 0 2024-02-15T18:27:15.115 INFO:tasks.workunit.client.1.smithi191.stdout:9/797: dwrite d0/d7/dc/d30/f5c [4194304,4194304] 0 2024-02-15T18:27:15.117 INFO:tasks.workunit.client.1.smithi191.stdout:9/798: creat d0/d7/dc/d32/f107 x:0 0 0 2024-02-15T18:27:15.119 INFO:tasks.workunit.client.1.smithi191.stdout:9/799: write d0/d7/dc/d17/d33/f69 [4891060,96136] 0 2024-02-15T18:27:15.121 INFO:tasks.workunit.client.1.smithi191.stdout:9/800: link d0/d7/dc/d8c/f35 d0/d7/dc/d30/d3b/d56/f108 0 2024-02-15T18:27:15.121 INFO:tasks.workunit.client.1.smithi191.stdout:3/614: dwrite d6/d3b/fba [8388608,4194304] 0 2024-02-15T18:27:15.122 INFO:tasks.workunit.client.1.smithi191.stdout:9/801: write d0/d7/f21 [1247682,92353] 0 2024-02-15T18:27:15.122 INFO:tasks.workunit.client.1.smithi191.stdout:9/802: write d0/d7/dc/d30/d3b/d56/f108 [9260776,109367] 0 2024-02-15T18:27:15.125 INFO:tasks.workunit.client.1.smithi191.stdout:9/803: dread d0/d7/dc/d17/d44/fcc [0,4194304] 0 2024-02-15T18:27:15.125 INFO:tasks.workunit.client.1.smithi191.stdout:9/804: fsync d0/d1d/d3f/fd9 0 2024-02-15T18:27:15.126 INFO:tasks.workunit.client.1.smithi191.stdout:9/805: dread - d0/d7/dc/d17/d33/ff2 zero size 2024-02-15T18:27:15.128 INFO:tasks.workunit.client.1.smithi191.stdout:7/769: dwrite d0/db/d26/d5f/d77/fb7 [4194304,4194304] 0 2024-02-15T18:27:15.129 INFO:tasks.workunit.client.1.smithi191.stdout:1/781: dwrite d0/d2f/d51/d54/d7b/fc6 [0,4194304] 0 2024-02-15T18:27:15.130 INFO:tasks.workunit.client.1.smithi191.stdout:9/806: mkdir d0/d7/dc/d17/d33/d109 0 2024-02-15T18:27:15.130 INFO:tasks.workunit.client.1.smithi191.stdout:9/807: getdents d0/d7/dc/d17/d33/d109 0 2024-02-15T18:27:15.137 INFO:tasks.workunit.client.1.smithi191.stdout:1/782: mkdir d0/d8/d108 0 2024-02-15T18:27:15.140 INFO:tasks.workunit.client.1.smithi191.stdout:1/783: dread d0/d8/df/d2e/f8d [0,4194304] 0 2024-02-15T18:27:15.141 INFO:tasks.workunit.client.1.smithi191.stdout:1/784: read - d0/ff1 zero size 2024-02-15T18:27:15.145 INFO:tasks.workunit.client.1.smithi191.stdout:1/785: rename d0/d8/df/d90/d94 to d0/d8/df/d12/d2a/d2b/d78/d60/d109 0 2024-02-15T18:27:15.154 INFO:tasks.workunit.client.1.smithi191.stdout:3/615: dwrite d6/d1b/fa3 [0,4194304] 0 2024-02-15T18:27:15.154 INFO:tasks.workunit.client.1.smithi191.stdout:3/616: write d6/d27/d4d/da8/fc4 [229826,44315] 0 2024-02-15T18:27:15.156 INFO:tasks.workunit.client.1.smithi191.stdout:9/808: dwrite d0/d29/d48/d66/d92/ff9 [0,4194304] 0 2024-02-15T18:27:15.156 INFO:tasks.workunit.client.1.smithi191.stdout:9/809: readlink d0/d7/dc/d30/lb7 0 2024-02-15T18:27:15.159 INFO:tasks.workunit.client.1.smithi191.stdout:7/770: dwrite d0/db/d65/dc/fee [0,4194304] 0 2024-02-15T18:27:15.159 INFO:tasks.workunit.client.1.smithi191.stdout:7/771: stat d0/db/d65/dc/d4b/dc1/def/f116 0 2024-02-15T18:27:15.159 INFO:tasks.workunit.client.1.smithi191.stdout:7/772: readlink d0/db/d65/dc/l84 0 2024-02-15T18:27:15.159 INFO:tasks.workunit.client.1.smithi191.stdout:7/773: stat d0/db/d26/d5f/d77/d35/d42 0 2024-02-15T18:27:15.159 INFO:tasks.workunit.client.1.smithi191.stdout:7/774: chown d0/db/d65/dc/d4b/dc1/f114 1212110155 1 2024-02-15T18:27:15.159 INFO:tasks.workunit.client.1.smithi191.stdout:7/775: read - d0/db/d26/d5f/d77/d35/fd4 zero size 2024-02-15T18:27:15.159 INFO:tasks.workunit.client.1.smithi191.stdout:7/776: chown d0/db/d65/d11/d57/fcf 721869 1 2024-02-15T18:27:15.159 INFO:tasks.workunit.client.1.smithi191.stdout:3/617: unlink d6/d3b/d42/f2a 0 2024-02-15T18:27:15.167 INFO:tasks.workunit.client.1.smithi191.stdout:3/618: mkdir d6/d27/d4d/d5a/d92/dd2 0 2024-02-15T18:27:15.168 INFO:tasks.workunit.client.1.smithi191.stdout:3/619: creat d6/d27/d4d/d5a/d92/dbe/fd3 x:0 0 0 2024-02-15T18:27:15.170 INFO:tasks.workunit.client.1.smithi191.stdout:3/620: link d6/d1b/d60/f68 d6/d3b/d42/dbb/fd4 0 2024-02-15T18:27:15.171 INFO:tasks.workunit.client.1.smithi191.stdout:3/621: write d6/d27/f3f [2607283,1661] 0 2024-02-15T18:27:15.174 INFO:tasks.workunit.client.1.smithi191.stdout:3/622: truncate d6/d3b/d42/f31 1828388 0 2024-02-15T18:27:15.175 INFO:tasks.workunit.client.1.smithi191.stdout:7/777: dread d0/db/d26/d5f/d77/df/f8e [0,4194304] 0 2024-02-15T18:27:15.176 INFO:tasks.workunit.client.1.smithi191.stdout:9/810: dwrite d0/d7/f21 [0,4194304] 0 2024-02-15T18:27:15.176 INFO:tasks.workunit.client.1.smithi191.stdout:9/811: dread - d0/d7/dc/d8c/f102 zero size 2024-02-15T18:27:15.179 INFO:tasks.workunit.client.1.smithi191.stdout:3/623: rename d6/d1b/d82 to d6/d5d/d72/dd5 0 2024-02-15T18:27:15.186 INFO:tasks.workunit.client.1.smithi191.stdout:9/812: creat d0/d7/dc/d30/f10a x:0 0 0 2024-02-15T18:27:15.187 INFO:tasks.workunit.client.1.smithi191.stdout:7/778: rename d0/db/d65/d11/d1c/l58 to d0/db/d26/d5f/d77/l11f 0 2024-02-15T18:27:15.188 INFO:tasks.workunit.client.1.smithi191.stdout:9/813: creat d0/d7/dc/d32/de4/dec/f10b x:0 0 0 2024-02-15T18:27:15.191 INFO:tasks.workunit.client.1.smithi191.stdout:7/779: mkdir d0/db/d65/dc/d120 0 2024-02-15T18:27:15.193 INFO:tasks.workunit.client.1.smithi191.stdout:9/814: link d0/d29/d48/fc3 d0/d1d/d3f/f10c 0 2024-02-15T18:27:15.194 INFO:tasks.workunit.client.1.smithi191.stdout:3/624: rename d6/d3b/d42/d49/f85 to d6/d8e/fd6 0 2024-02-15T18:27:15.195 INFO:tasks.workunit.client.1.smithi191.stdout:7/780: truncate d0/db/f2e 3181373 0 2024-02-15T18:27:15.196 INFO:tasks.workunit.client.1.smithi191.stdout:7/781: chown d0/db/d26/d5f/d77/d35/dc7 74028 1 2024-02-15T18:27:15.196 INFO:tasks.workunit.client.1.smithi191.stdout:9/815: mkdir d0/d29/d10d 0 2024-02-15T18:27:15.197 INFO:tasks.workunit.client.1.smithi191.stdout:3/625: mknod d6/db8/cd7 0 2024-02-15T18:27:15.197 INFO:tasks.workunit.client.1.smithi191.stdout:3/626: chown d6/d3b/d42/f51 1 1 2024-02-15T18:27:15.198 INFO:tasks.workunit.client.1.smithi191.stdout:3/627: write d6/d27/f32 [9527590,74403] 0 2024-02-15T18:27:15.199 INFO:tasks.workunit.client.1.smithi191.stdout:7/782: mknod d0/db/d26/d5f/c121 0 2024-02-15T18:27:15.199 INFO:tasks.workunit.client.1.smithi191.stdout:9/816: truncate d0/d29/f86 4087494 0 2024-02-15T18:27:15.201 INFO:tasks.workunit.client.1.smithi191.stdout:3/628: rename d6/d1b/d60/lad to d6/d1b/d60/d73/d5e/ld8 0 2024-02-15T18:27:15.202 INFO:tasks.workunit.client.1.smithi191.stdout:7/783: mkdir d0/db/d26/d5f/d77/d35/d42/d122 0 2024-02-15T18:27:15.202 INFO:tasks.workunit.client.1.smithi191.stdout:7/784: chown d0/db/d26/d5f/d77/df/d5d/c10d 23566830 1 2024-02-15T18:27:15.210 INFO:tasks.workunit.client.1.smithi191.stdout:9/817: creat d0/d7/dc/d32/d106/f10e x:0 0 0 2024-02-15T18:27:15.211 INFO:tasks.workunit.client.1.smithi191.stdout:9/818: read d0/d7/dc/d30/fee [931092,35468] 0 2024-02-15T18:27:15.211 INFO:tasks.workunit.client.1.smithi191.stdout:3/629: unlink d6/d5d/d72/l76 0 2024-02-15T18:27:15.213 INFO:tasks.workunit.client.1.smithi191.stdout:7/785: rename d0/db/d26/d5f/d77/d35/dc7/cce to d0/db/d26/d5f/d77/df/c123 0 2024-02-15T18:27:15.213 INFO:tasks.workunit.client.1.smithi191.stdout:7/786: chown d0/db/d26/d5f/d77/l11f 534205 1 2024-02-15T18:27:15.213 INFO:tasks.workunit.client.1.smithi191.stdout:7/787: truncate d0/db/d65/f75 933229 0 2024-02-15T18:27:15.217 INFO:tasks.workunit.client.1.smithi191.stdout:9/819: read d0/fdc [2683077,68868] 0 2024-02-15T18:27:15.217 INFO:tasks.workunit.client.1.smithi191.stdout:3/630: creat d6/fd9 x:0 0 0 2024-02-15T18:27:15.217 INFO:tasks.workunit.client.1.smithi191.stdout:3/631: fsync d6/d1b/f39 0 2024-02-15T18:27:15.221 INFO:tasks.workunit.client.1.smithi191.stdout:7/788: rename d0/db/d65/dc/l84 to d0/db/d65/dc/l124 0 2024-02-15T18:27:15.223 INFO:tasks.workunit.client.1.smithi191.stdout:7/789: link d0/f80 d0/db/d65/d11/d88/f125 0 2024-02-15T18:27:15.237 INFO:tasks.workunit.client.1.smithi191.stdout:3/632: dwrite d6/d27/d4d/da8/fc4 [0,4194304] 0 2024-02-15T18:27:15.251 INFO:tasks.workunit.client.1.smithi191.stdout:3/633: dwrite d6/d3b/d30/f5c [0,4194304] 0 2024-02-15T18:27:15.251 INFO:tasks.workunit.client.1.smithi191.stdout:3/634: dread - d6/d3b/fbd zero size 2024-02-15T18:27:15.263 INFO:tasks.workunit.client.1.smithi191.stdout:3/635: link d6/d3b/d42/l24 d6/d1b/d60/d73/lda 0 2024-02-15T18:27:15.265 INFO:tasks.workunit.client.1.smithi191.stdout:3/636: dread d6/d1b/d60/d73/f8d [0,4194304] 0 2024-02-15T18:27:15.278 INFO:tasks.workunit.client.1.smithi191.stdout:3/637: dwrite d6/fa0 [0,4194304] 0 2024-02-15T18:27:15.283 INFO:tasks.workunit.client.1.smithi191.stdout:3/638: dread d6/fa [0,4194304] 0 2024-02-15T18:27:15.284 INFO:tasks.workunit.client.1.smithi191.stdout:3/639: write d6/d3b/d42/dbb/fd4 [709527,85712] 0 2024-02-15T18:27:15.285 INFO:tasks.workunit.client.1.smithi191.stdout:6/717: sync 2024-02-15T18:27:15.285 INFO:tasks.workunit.client.1.smithi191.stdout:2/727: sync 2024-02-15T18:27:15.285 INFO:tasks.workunit.client.1.smithi191.stdout:3/640: mknod d6/d8e/cdb 0 2024-02-15T18:27:15.286 INFO:tasks.workunit.client.1.smithi191.stdout:3/641: write d6/d3b/fa7 [375481,4339] 0 2024-02-15T18:27:15.287 INFO:tasks.workunit.client.1.smithi191.stdout:6/718: mkdir d8/d9/d2b/d84/ddd 0 2024-02-15T18:27:15.288 INFO:tasks.workunit.client.1.smithi191.stdout:2/728: creat df/d12/d19/d1a/d52/df2/ff8 x:0 0 0 2024-02-15T18:27:15.289 INFO:tasks.workunit.client.1.smithi191.stdout:2/729: truncate df/d12/d19/d1a/dda/ff3 639548 0 2024-02-15T18:27:15.289 INFO:tasks.workunit.client.1.smithi191.stdout:2/730: dread df/d12/d19/d1a/d35/f60 [0,4194304] 0 2024-02-15T18:27:15.290 INFO:tasks.workunit.client.1.smithi191.stdout:3/642: mkdir d6/d1b/d60/ddc 0 2024-02-15T18:27:15.290 INFO:tasks.workunit.client.1.smithi191.stdout:3/643: readlink d6/d3b/l14 0 2024-02-15T18:27:15.291 INFO:tasks.workunit.client.1.smithi191.stdout:6/719: truncate d8/d9/d4a/d4c/da4/f7c 555833 0 2024-02-15T18:27:15.293 INFO:tasks.workunit.client.1.smithi191.stdout:6/720: symlink d8/d9/dac/db0/d95/d9d/d6b/d76/lde 0 2024-02-15T18:27:15.316 INFO:tasks.workunit.client.1.smithi191.stdout:2/731: dwrite df/d12/f22 [0,4194304] 0 2024-02-15T18:27:15.317 INFO:tasks.workunit.client.1.smithi191.stdout:2/732: truncate df/d12/f2b 4740919 0 2024-02-15T18:27:15.317 INFO:tasks.workunit.client.1.smithi191.stdout:2/733: fdatasync df/d12/d19/d1a/fae 0 2024-02-15T18:27:15.317 INFO:tasks.workunit.client.1.smithi191.stdout:3/644: dwrite d6/d27/f32 [8388608,4194304] 0 2024-02-15T18:27:15.318 INFO:tasks.workunit.client.1.smithi191.stdout:6/721: dwrite d8/d9/dac/db0/d95/d9d/d6b/d76/fb1 [0,4194304] 0 2024-02-15T18:27:15.319 INFO:tasks.workunit.client.1.smithi191.stdout:6/722: creat d8/d9/dac/ddb/fdf x:0 0 0 2024-02-15T18:27:15.321 INFO:tasks.workunit.client.1.smithi191.stdout:3/645: mkdir d6/d3b/d42/ddd 0 2024-02-15T18:27:15.322 INFO:tasks.workunit.client.1.smithi191.stdout:6/723: unlink d8/d9/d4a/d4c/fd3 0 2024-02-15T18:27:15.326 INFO:tasks.workunit.client.1.smithi191.stdout:3/646: unlink d6/d3b/d42/f31 0 2024-02-15T18:27:15.326 INFO:tasks.workunit.client.1.smithi191.stdout:3/647: dread - d6/d27/d4d/d5a/d92/dbe/fd3 zero size 2024-02-15T18:27:15.330 INFO:tasks.workunit.client.1.smithi191.stdout:6/724: rmdir d8/d9/d2b/d84 39 2024-02-15T18:27:15.335 INFO:tasks.workunit.client.1.smithi191.stdout:6/725: mknod d8/d9/dac/db0/ce0 0 2024-02-15T18:27:15.341 INFO:tasks.workunit.client.1.smithi191.stdout:3/648: dwrite d6/fb4 [0,4194304] 0 2024-02-15T18:27:15.341 INFO:tasks.workunit.client.1.smithi191.stdout:3/649: fdatasync d6/d1b/f81 0 2024-02-15T18:27:15.345 INFO:tasks.workunit.client.1.smithi191.stdout:3/650: mknod d6/d3b/d30/cde 0 2024-02-15T18:27:15.346 INFO:tasks.workunit.client.1.smithi191.stdout:6/726: dread d8/fa5 [0,4194304] 0 2024-02-15T18:27:15.348 INFO:tasks.workunit.client.1.smithi191.stdout:6/727: mknod d8/d9/dac/ce1 0 2024-02-15T18:27:15.348 INFO:tasks.workunit.client.1.smithi191.stdout:6/728: fdatasync d8/d9/f79 0 2024-02-15T18:27:15.349 INFO:tasks.workunit.client.1.smithi191.stdout:6/729: read d8/d9/d2b/d57/f93 [61770,76838] 0 2024-02-15T18:27:15.350 INFO:tasks.workunit.client.1.smithi191.stdout:6/730: mkdir d8/d9/d4a/de2 0 2024-02-15T18:27:15.359 INFO:tasks.workunit.client.1.smithi191.stdout:3/651: dwrite d6/d1b/d60/d73/f8d [0,4194304] 0 2024-02-15T18:27:15.373 INFO:tasks.workunit.client.1.smithi191.stdout:6/731: dwrite d8/d9/dac/db0/d95/d9d/f7a [0,4194304] 0 2024-02-15T18:27:15.373 INFO:tasks.workunit.client.1.smithi191.stdout:6/732: stat d8/d9/dac/db0/d95/d9d/d6b/d98/lc7 0 2024-02-15T18:27:15.376 INFO:tasks.workunit.client.1.smithi191.stdout:3/652: dwrite d6/d3b/f6f [0,4194304] 0 2024-02-15T18:27:15.376 INFO:tasks.workunit.client.1.smithi191.stdout:3/653: chown d6/d1b/d60/l87 225796946 1 2024-02-15T18:27:15.376 INFO:tasks.workunit.client.1.smithi191.stdout:3/654: chown d6/d27/c4b 84031 1 2024-02-15T18:27:15.376 INFO:tasks.workunit.client.1.smithi191.stdout:3/655: readlink d6/d3b/d42/l24 0 2024-02-15T18:27:15.384 INFO:tasks.workunit.client.1.smithi191.stdout:3/656: dread d6/d3b/d42/d49/d54/fa6 [0,4194304] 0 2024-02-15T18:27:15.386 INFO:tasks.workunit.client.1.smithi191.stdout:3/657: truncate d6/d1b/d60/d73/f74 2761066 0 2024-02-15T18:27:15.389 INFO:tasks.workunit.client.1.smithi191.stdout:3/658: creat d6/d5d/d72/dd5/db2/fdf x:0 0 0 2024-02-15T18:27:15.389 INFO:tasks.workunit.client.1.smithi191.stdout:3/659: write d6/d5d/d72/dd5/db2/fdf [353232,75177] 0 2024-02-15T18:27:15.389 INFO:tasks.workunit.client.1.smithi191.stdout:3/660: chown d6/d1b/d60/d73/d5e/fc5 18322 1 2024-02-15T18:27:15.393 INFO:tasks.workunit.client.1.smithi191.stdout:3/661: unlink d6/d3b/d42/l24 0 2024-02-15T18:27:15.395 INFO:tasks.workunit.client.1.smithi191.stdout:3/662: mknod d6/db8/ce0 0 2024-02-15T18:27:15.396 INFO:tasks.workunit.client.1.smithi191.stdout:3/663: rmdir d6/d5d 39 2024-02-15T18:27:15.398 INFO:tasks.workunit.client.1.smithi191.stdout:3/664: creat d6/d27/d4d/d5a/fe1 x:0 0 0 2024-02-15T18:27:15.398 INFO:tasks.workunit.client.1.smithi191.stdout:3/665: read - d6/d27/d4d/fc9 zero size 2024-02-15T18:27:15.399 INFO:tasks.workunit.client.1.smithi191.stdout:6/733: dwrite d8/d9/d2b/d5b/f5e [0,4194304] 0 2024-02-15T18:27:15.399 INFO:tasks.workunit.client.1.smithi191.stdout:6/734: stat d8/l3f 0 2024-02-15T18:27:15.401 INFO:tasks.workunit.client.1.smithi191.stdout:3/666: mkdir d6/d1b/d60/da2/dac/de2 0 2024-02-15T18:27:15.401 INFO:tasks.workunit.client.1.smithi191.stdout:3/667: write d6/d27/fa5 [897219,48412] 0 2024-02-15T18:27:15.401 INFO:tasks.workunit.client.1.smithi191.stdout:3/668: fdatasync d6/d1b/d60/da2/dac/fcc 0 2024-02-15T18:27:15.409 INFO:tasks.workunit.client.1.smithi191.stdout:6/735: creat d8/d9/d2b/fe3 x:0 0 0 2024-02-15T18:27:15.409 INFO:tasks.workunit.client.1.smithi191.stdout:3/669: dread d6/d1b/d60/d73/f75 [8388608,4194304] 0 2024-02-15T18:27:15.411 INFO:tasks.workunit.client.1.smithi191.stdout:6/736: rmdir d8/d9/d2b/d84 39 2024-02-15T18:27:15.414 INFO:tasks.workunit.client.1.smithi191.stdout:3/670: rename d6/db8 to d6/d3b/d42/d49/dc7/de3 0 2024-02-15T18:27:15.500 INFO:tasks.workunit.client.1.smithi191.stdout:3/671: rmdir d6/d3b/d42 39 2024-02-15T18:27:15.502 INFO:tasks.workunit.client.1.smithi191.stdout:3/672: fdatasync d6/d1b/d60/d73/f74 0 2024-02-15T18:27:15.504 INFO:tasks.workunit.client.1.smithi191.stdout:3/673: unlink d6/d3b/d42/d49/d54/f6b 0 2024-02-15T18:27:15.547 INFO:tasks.workunit.client.1.smithi191.stdout:3/674: creat d6/d3b/d30/fe4 x:0 0 0 2024-02-15T18:27:15.549 INFO:tasks.workunit.client.1.smithi191.stdout:3/675: mkdir d6/d1b/d60/ddc/de5 0 2024-02-15T18:27:15.580 INFO:tasks.workunit.client.1.smithi191.stdout:5/494: sync 2024-02-15T18:27:15.580 INFO:tasks.workunit.client.1.smithi191.stdout:8/810: sync 2024-02-15T18:27:15.581 INFO:tasks.workunit.client.1.smithi191.stdout:3/676: creat d6/d1b/d60/ddc/fe6 x:0 0 0 2024-02-15T18:27:15.581 INFO:tasks.workunit.client.1.smithi191.stdout:3/677: chown d6/d3b/d42/ddd 0 1 2024-02-15T18:27:15.585 INFO:tasks.workunit.client.1.smithi191.stdout:5/495: mknod d0/d18/d5c/c9e 0 2024-02-15T18:27:15.588 INFO:tasks.workunit.client.1.smithi191.stdout:8/811: mkdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d115 0 2024-02-15T18:27:15.589 INFO:tasks.workunit.client.1.smithi191.stdout:5/496: mkdir d0/d18/d5c/d7c/d49/d9f 0 2024-02-15T18:27:15.590 INFO:tasks.workunit.client.1.smithi191.stdout:5/497: creat d0/d23/d41/d28/d31/d5a/fa0 x:0 0 0 2024-02-15T18:27:15.600 INFO:tasks.workunit.client.1.smithi191.stdout:3/678: dwrite d6/d27/d4d/da8/fb1 [0,4194304] 0 2024-02-15T18:27:15.603 INFO:tasks.workunit.client.1.smithi191.stdout:3/679: creat d6/d1b/d60/ddc/de5/fe7 x:0 0 0 2024-02-15T18:27:15.607 INFO:tasks.workunit.client.1.smithi191.stdout:5/498: dwrite d0/d23/d41/d28/d31/f47 [0,4194304] 0 2024-02-15T18:27:15.607 INFO:tasks.workunit.client.1.smithi191.stdout:3/680: write d6/d27/d4d/d5a/da4/fc0 [1620862,34255] 0 2024-02-15T18:27:15.613 INFO:tasks.workunit.client.1.smithi191.stdout:5/499: symlink d0/d23/d41/d3c/la1 0 2024-02-15T18:27:15.613 INFO:tasks.workunit.client.1.smithi191.stdout:5/500: dread - d0/d18/d5c/d7c/f9c zero size 2024-02-15T18:27:15.614 INFO:tasks.workunit.client.1.smithi191.stdout:3/681: rmdir d6/d3b/d42/d49/dc7 39 2024-02-15T18:27:15.614 INFO:tasks.workunit.client.1.smithi191.stdout:5/501: write d0/d23/d41/d28/d31/d5a/f68 [1590601,76531] 0 2024-02-15T18:27:15.614 INFO:tasks.workunit.client.1.smithi191.stdout:5/502: chown d0/d23/d41/d28 1504586004 1 2024-02-15T18:27:15.615 INFO:tasks.workunit.client.1.smithi191.stdout:5/503: write d0/f79 [4900586,113472] 0 2024-02-15T18:27:15.615 INFO:tasks.workunit.client.1.smithi191.stdout:5/504: dread - d0/d18/d5c/f63 zero size 2024-02-15T18:27:15.622 INFO:tasks.workunit.client.1.smithi191.stdout:0/578: sync 2024-02-15T18:27:15.622 INFO:tasks.workunit.client.1.smithi191.stdout:4/766: sync 2024-02-15T18:27:15.624 INFO:tasks.workunit.client.1.smithi191.stdout:0/579: write d7/d4a/f59 [5044034,74023] 0 2024-02-15T18:27:15.624 INFO:tasks.workunit.client.1.smithi191.stdout:0/580: chown d7/da/d6c/lba 215762082 1 2024-02-15T18:27:15.629 INFO:tasks.workunit.client.1.smithi191.stdout:0/581: creat d7/da/d17/fc9 x:0 0 0 2024-02-15T18:27:15.634 INFO:tasks.workunit.client.1.smithi191.stdout:0/582: link d7/d3b/l3f d7/d94/lca 0 2024-02-15T18:27:15.640 INFO:tasks.workunit.client.1.smithi191.stdout:0/583: symlink d7/da/d17/d31/d5f/dbb/lcb 0 2024-02-15T18:27:15.642 INFO:tasks.workunit.client.1.smithi191.stdout:5/505: dwrite d0/d23/d41/d28/d31/d5a/f8c [0,4194304] 0 2024-02-15T18:27:15.646 INFO:tasks.workunit.client.1.smithi191.stdout:5/506: creat d0/d23/d41/d28/d31/d5a/fa2 x:0 0 0 2024-02-15T18:27:15.646 INFO:tasks.workunit.client.1.smithi191.stdout:5/507: fsync d0/d23/d41/d28/d65/d85/d52/f40 0 2024-02-15T18:27:15.646 INFO:tasks.workunit.client.1.smithi191.stdout:4/767: dwrite d2/d5/d7/d2f/d41/f57 [0,4194304] 0 2024-02-15T18:27:15.652 INFO:tasks.workunit.client.1.smithi191.stdout:5/508: unlink d0/d23/d41/d28/f3e 0 2024-02-15T18:27:15.657 INFO:tasks.workunit.client.1.smithi191.stdout:5/509: symlink d0/d18/d5c/la3 0 2024-02-15T18:27:15.659 INFO:tasks.workunit.client.1.smithi191.stdout:4/768: rename d2/d5/d7/d90/d52/f7b to d2/d5/d7/d90/d52/db5/dc9/f117 0 2024-02-15T18:27:15.662 INFO:tasks.workunit.client.1.smithi191.stdout:0/584: dwrite d7/da/d1a/d2c/f4f [0,4194304] 0 2024-02-15T18:27:15.683 INFO:tasks.workunit.client.1.smithi191.stdout:4/769: dwrite d2/d5/d7/d3e/d63/d8c/dbb/f113 [0,4194304] 0 2024-02-15T18:27:15.685 INFO:tasks.workunit.client.1.smithi191.stdout:5/510: dwrite d0/d23/d41/d28/f4a [0,4194304] 0 2024-02-15T18:27:15.697 INFO:tasks.workunit.client.1.smithi191.stdout:0/585: rename d7/da/d17/d31/d5f/dbb to d7/d4a/dc0/dcc 0 2024-02-15T18:27:15.710 INFO:tasks.workunit.client.1.smithi191.stdout:4/770: dwrite d2/d5/d14/d47/d78/f38 [0,4194304] 0 2024-02-15T18:27:15.711 INFO:tasks.workunit.client.1.smithi191.stdout:5/511: link d0/d18/d5c/d7c/d49/l5e d0/d18/d5c/d7c/d49/d9f/la4 0 2024-02-15T18:27:15.717 INFO:tasks.workunit.client.1.smithi191.stdout:4/771: write d2/d5/d14/d71/fac [1614478,6502] 0 2024-02-15T18:27:15.717 INFO:tasks.workunit.client.1.smithi191.stdout:4/772: chown d2/d5/d7/d3e/d63/d8c/da4/fa2 1577253 1 2024-02-15T18:27:15.717 INFO:tasks.workunit.client.1.smithi191.stdout:4/773: read - d2/d5/d7/d106/f109 zero size 2024-02-15T18:27:15.718 INFO:tasks.workunit.client.1.smithi191.stdout:0/586: link d7/da/d1a/d34/f99 d7/d4a/dc0/fcd 0 2024-02-15T18:27:15.721 INFO:tasks.workunit.client.1.smithi191.stdout:5/512: symlink d0/d18/d5c/d73/la5 0 2024-02-15T18:27:15.721 INFO:tasks.workunit.client.1.smithi191.stdout:5/513: chown d0/d18/d5c/l6e 66 1 2024-02-15T18:27:15.727 INFO:tasks.workunit.client.1.smithi191.stdout:0/587: creat d7/da/fce x:0 0 0 2024-02-15T18:27:15.729 INFO:tasks.workunit.client.1.smithi191.stdout:5/514: creat d0/d23/d41/d28/d31/d5a/fa6 x:0 0 0 2024-02-15T18:27:15.731 INFO:tasks.workunit.client.1.smithi191.stdout:5/515: creat d0/fa7 x:0 0 0 2024-02-15T18:27:15.731 INFO:tasks.workunit.client.1.smithi191.stdout:5/516: chown d0/d23/d41/d28/d65/d85/d52 164847327 1 2024-02-15T18:27:15.733 INFO:tasks.workunit.client.1.smithi191.stdout:0/588: write d7/da/d6c/fbf [570034,36876] 0 2024-02-15T18:27:15.734 INFO:tasks.workunit.client.1.smithi191.stdout:0/589: stat l4 0 2024-02-15T18:27:15.736 INFO:tasks.workunit.client.1.smithi191.stdout:0/590: creat d7/da/d17/d31/d7d/fcf x:0 0 0 2024-02-15T18:27:15.736 INFO:tasks.workunit.client.1.smithi191.stdout:0/591: stat d7/d4a/c70 0 2024-02-15T18:27:15.743 INFO:tasks.workunit.client.1.smithi191.stdout:0/592: rename d7/da/d1a/d2c/f8c to d7/da/d17/d31/d5f/fd0 0 2024-02-15T18:27:15.757 INFO:tasks.workunit.client.1.smithi191.stdout:4/774: dwrite d2/d5/d7/d2f/d41/f99 [0,4194304] 0 2024-02-15T18:27:15.763 INFO:tasks.workunit.client.1.smithi191.stdout:4/775: getdents d2/d5/d8b 0 2024-02-15T18:27:15.765 INFO:tasks.workunit.client.1.smithi191.stdout:4/776: write d2/d5/d14/f58 [2536312,1095] 0 2024-02-15T18:27:15.765 INFO:tasks.workunit.client.1.smithi191.stdout:1/786: sync 2024-02-15T18:27:15.769 INFO:tasks.workunit.client.1.smithi191.stdout:4/777: unlink d2/d5/d14/f2e 0 2024-02-15T18:27:15.769 INFO:tasks.workunit.client.1.smithi191.stdout:4/778: chown d2/d5/d7/l1b 3054632 1 2024-02-15T18:27:15.769 INFO:tasks.workunit.client.1.smithi191.stdout:4/779: chown d2/d5/d14/d47/d78/f6a 10886344 1 2024-02-15T18:27:15.770 INFO:tasks.workunit.client.1.smithi191.stdout:1/787: truncate d0/d2f/d5c/f71 3325502 0 2024-02-15T18:27:15.771 INFO:tasks.workunit.client.1.smithi191.stdout:4/780: mknod d2/d5/d7/d90/d52/dcb/c118 0 2024-02-15T18:27:15.772 INFO:tasks.workunit.client.1.smithi191.stdout:1/788: mknod d0/d2f/d51/d54/db7/c10a 0 2024-02-15T18:27:15.778 INFO:tasks.workunit.client.1.smithi191.stdout:5/517: dwrite d0/d18/d5c/f63 [0,4194304] 0 2024-02-15T18:27:15.780 INFO:tasks.workunit.client.1.smithi191.stdout:4/781: dread d2/d5/d7/d90/d52/f54 [0,4194304] 0 2024-02-15T18:27:15.782 INFO:tasks.workunit.client.1.smithi191.stdout:5/518: read d0/d23/d41/d28/d65/d85/f54 [1483992,3462] 0 2024-02-15T18:27:15.782 INFO:tasks.workunit.client.1.smithi191.stdout:1/789: rename d0/d8/df/d12/d33/ca5 to d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/dc9/c10b 0 2024-02-15T18:27:15.782 INFO:tasks.workunit.client.1.smithi191.stdout:4/782: rmdir d2/d5/d7/d2f 39 2024-02-15T18:27:15.785 INFO:tasks.workunit.client.1.smithi191.stdout:1/790: mkdir d0/d8/df/d10c 0 2024-02-15T18:27:15.785 INFO:tasks.workunit.client.1.smithi191.stdout:1/791: chown d0/d8/df/d12/d2a/fe3 31 1 2024-02-15T18:27:15.786 INFO:tasks.workunit.client.1.smithi191.stdout:0/593: dwrite d7/d1b/f6e [4194304,4194304] 0 2024-02-15T18:27:15.786 INFO:tasks.workunit.client.1.smithi191.stdout:4/783: mkdir d2/d5/d14/d47/d78/d119 0 2024-02-15T18:27:15.791 INFO:tasks.workunit.client.1.smithi191.stdout:0/594: mkdir d7/da/d1a/d2c/d4c/dd1 0 2024-02-15T18:27:15.791 INFO:tasks.workunit.client.1.smithi191.stdout:1/792: write d0/d2f/d51/d10/fb4 [3851652,107778] 0 2024-02-15T18:27:15.791 INFO:tasks.workunit.client.1.smithi191.stdout:1/793: chown d0/d2f/d51/d10/f59 8 1 2024-02-15T18:27:15.791 INFO:tasks.workunit.client.1.smithi191.stdout:1/794: fsync d0/d8/df/d12/d33/d49/f4a 0 2024-02-15T18:27:15.791 INFO:tasks.workunit.client.1.smithi191.stdout:1/795: truncate d0/d8/df/d12/d2a/fb8 1028354 0 2024-02-15T18:27:15.791 INFO:tasks.workunit.client.1.smithi191.stdout:1/796: chown d0/d2f/d51/fe 2 1 2024-02-15T18:27:15.792 INFO:tasks.workunit.client.1.smithi191.stdout:4/784: creat d2/d5/d7/d3e/d63/f11a x:0 0 0 2024-02-15T18:27:15.800 INFO:tasks.workunit.client.1.smithi191.stdout:1/797: link d0/d2f/d5c/c6a d0/d2f/c10d 0 2024-02-15T18:27:15.807 INFO:tasks.workunit.client.1.smithi191.stdout:4/785: mknod d2/d5/d7/d106/c11b 0 2024-02-15T18:27:15.808 INFO:tasks.workunit.client.1.smithi191.stdout:1/798: creat d0/d2f/f10e x:0 0 0 2024-02-15T18:27:15.808 INFO:tasks.workunit.client.1.smithi191.stdout:0/595: dwrite d7/da/fce [0,4194304] 0 2024-02-15T18:27:15.812 INFO:tasks.workunit.client.1.smithi191.stdout:1/799: creat d0/d8/df/d12/d33/f10f x:0 0 0 2024-02-15T18:27:15.813 INFO:tasks.workunit.client.1.smithi191.stdout:1/800: write d0/d2f/d51/fe [591805,15685] 0 2024-02-15T18:27:15.813 INFO:tasks.workunit.client.1.smithi191.stdout:1/801: truncate d0/d8/df/d12/d2a/d2b/f5a 1392730 0 2024-02-15T18:27:15.817 INFO:tasks.workunit.client.1.smithi191.stdout:0/596: dread d7/d3b/f69 [0,4194304] 0 2024-02-15T18:27:15.817 INFO:tasks.workunit.client.1.smithi191.stdout:0/597: write d7/da/d17/d31/f87 [1037608,37467] 0 2024-02-15T18:27:15.820 INFO:tasks.workunit.client.1.smithi191.stdout:1/802: rmdir d0/d8 39 2024-02-15T18:27:15.822 INFO:tasks.workunit.client.1.smithi191.stdout:0/598: getdents d7/d4a 0 2024-02-15T18:27:15.825 INFO:tasks.workunit.client.1.smithi191.stdout:1/803: mknod d0/d8/d48/c110 0 2024-02-15T18:27:15.842 INFO:tasks.workunit.client.1.smithi191.stdout:2/734: sync 2024-02-15T18:27:15.842 INFO:tasks.workunit.client.1.smithi191.stdout:6/737: sync 2024-02-15T18:27:15.842 INFO:tasks.workunit.client.1.smithi191.stdout:7/790: sync 2024-02-15T18:27:15.842 INFO:tasks.workunit.client.1.smithi191.stdout:9/820: sync 2024-02-15T18:27:15.843 INFO:tasks.workunit.client.1.smithi191.stdout:1/804: dwrite d0/d8/df/d12/d33/f10f [0,4194304] 0 2024-02-15T18:27:15.843 INFO:tasks.workunit.client.1.smithi191.stdout:6/738: dread d8/d9/dac/ddb/fa8 [0,4194304] 0 2024-02-15T18:27:15.846 INFO:tasks.workunit.client.1.smithi191.stdout:1/805: write d0/d8/df/d12/f8f [831104,122633] 0 2024-02-15T18:27:15.846 INFO:tasks.workunit.client.1.smithi191.stdout:1/806: dread - d0/fa8 zero size 2024-02-15T18:27:15.846 INFO:tasks.workunit.client.1.smithi191.stdout:1/807: dread - d0/d2f/d51/d10/ffc zero size 2024-02-15T18:27:15.847 INFO:tasks.workunit.client.1.smithi191.stdout:2/735: rename df/d12/d32/d4d/d89/f95 to df/d12/d19/d1a/dda/ff9 0 2024-02-15T18:27:15.848 INFO:tasks.workunit.client.1.smithi191.stdout:7/791: mkdir d0/db/d26/d5f/d77/db9/d126 0 2024-02-15T18:27:15.848 INFO:tasks.workunit.client.1.smithi191.stdout:7/792: fsync d0/db/f2d 0 2024-02-15T18:27:15.854 INFO:tasks.workunit.client.1.smithi191.stdout:6/739: creat d8/d9/dac/ddb/dd7/fe4 x:0 0 0 2024-02-15T18:27:15.857 INFO:tasks.workunit.client.1.smithi191.stdout:9/821: dwrite d0/d7/dc/d30/d3b/f55 [0,4194304] 0 2024-02-15T18:27:15.863 INFO:tasks.workunit.client.1.smithi191.stdout:9/822: dread d0/d29/d48/fdd [0,4194304] 0 2024-02-15T18:27:15.863 INFO:tasks.workunit.client.1.smithi191.stdout:9/823: chown d0/d29/l7a 158 1 2024-02-15T18:27:15.962 INFO:tasks.workunit.client.1.smithi191.stdout:1/808: symlink d0/l111 0 2024-02-15T18:27:15.962 INFO:tasks.workunit.client.1.smithi191.stdout:2/736: unlink df/d61/fdc 0 2024-02-15T18:27:15.989 INFO:tasks.workunit.client.1.smithi191.stdout:7/793: creat d0/db/d26/d5f/d77/db9/f127 x:0 0 0 2024-02-15T18:27:15.989 INFO:tasks.workunit.client.1.smithi191.stdout:7/794: fdatasync d0/db/d26/d5f/d77/d35/d42/fa7 0 2024-02-15T18:27:15.989 INFO:tasks.workunit.client.1.smithi191.stdout:7/795: write d0/db/d65/d11/d88/d9d/dd2/f95 [2950674,75348] 0 2024-02-15T18:27:15.989 INFO:tasks.workunit.client.1.smithi191.stdout:7/796: write d0/db/d65/dc/d4b/dc1/fc9 [80185,51860] 0 2024-02-15T18:27:15.992 INFO:tasks.workunit.client.1.smithi191.stdout:1/809: creat d0/d2f/f112 x:0 0 0 2024-02-15T18:27:15.996 INFO:tasks.workunit.client.1.smithi191.stdout:7/797: mknod d0/db/d26/d5f/d77/d35/dc7/c128 0 2024-02-15T18:27:15.997 INFO:tasks.workunit.client.1.smithi191.stdout:9/824: link d0/d7/dc/d17/d44/de6/le9 d0/d7/dc/d17/d44/l10f 0 2024-02-15T18:27:16.003 INFO:tasks.workunit.client.1.smithi191.stdout:1/810: creat d0/d8/df/d12/d33/d49/f113 x:0 0 0 2024-02-15T18:27:16.004 INFO:tasks.workunit.client.1.smithi191.stdout:8/812: sync 2024-02-15T18:27:16.004 INFO:tasks.workunit.client.1.smithi191.stdout:3/682: sync 2024-02-15T18:27:16.004 INFO:tasks.workunit.client.1.smithi191.stdout:8/813: truncate d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/f96 836270 0 2024-02-15T18:27:16.007 INFO:tasks.workunit.client.1.smithi191.stdout:7/798: creat d0/db/d26/d5f/d77/d35/d42/f129 x:0 0 0 2024-02-15T18:27:16.009 INFO:tasks.workunit.client.1.smithi191.stdout:1/811: creat d0/d2f/d51/d54/dd3/f114 x:0 0 0 2024-02-15T18:27:16.011 INFO:tasks.workunit.client.1.smithi191.stdout:3/683: symlink d6/d5d/le8 0 2024-02-15T18:27:16.012 INFO:tasks.workunit.client.1.smithi191.stdout:8/814: symlink d3/d2c/l116 0 2024-02-15T18:27:16.019 INFO:tasks.workunit.client.1.smithi191.stdout:7/799: unlink d0/db/d65/d11/d5a/d66/d1e/f8a 0 2024-02-15T18:27:16.021 INFO:tasks.workunit.client.1.smithi191.stdout:1/812: unlink d0/d8/df/d12/d2a/d2b/d78/d60/d109/fd5 0 2024-02-15T18:27:16.021 INFO:tasks.workunit.client.1.smithi191.stdout:1/813: readlink d0/d8/d48/la2 0 2024-02-15T18:27:16.021 INFO:tasks.workunit.client.1.smithi191.stdout:1/814: fdatasync d0/d2f/d51/fd1 0 2024-02-15T18:27:16.023 INFO:tasks.workunit.client.1.smithi191.stdout:8/815: truncate d3/d2c/d1a/d23/d2a/f6e 1618734 0 2024-02-15T18:27:16.023 INFO:tasks.workunit.client.1.smithi191.stdout:8/816: write d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/ff7 [678919,52493] 0 2024-02-15T18:27:16.023 INFO:tasks.workunit.client.1.smithi191.stdout:7/800: truncate d0/db/d26/d5f/d77/d35/f48 713563 0 2024-02-15T18:27:16.024 INFO:tasks.workunit.client.1.smithi191.stdout:7/801: truncate d0/db/d65/d11/d5a/d66/d1e/f63 159992 0 2024-02-15T18:27:16.025 INFO:tasks.workunit.client.1.smithi191.stdout:8/817: creat d3/d2c/d1a/d23/d2e/de9/f117 x:0 0 0 2024-02-15T18:27:16.026 INFO:tasks.workunit.client.1.smithi191.stdout:8/818: write d3/d2c/d1a/d23/d2e/d47/f63 [1027414,125255] 0 2024-02-15T18:27:16.026 INFO:tasks.workunit.client.1.smithi191.stdout:8/819: dread - d3/d2c/d1a/d23/d2e/d5b/d69/d6d/fd2 zero size 2024-02-15T18:27:16.037 INFO:tasks.workunit.client.1.smithi191.stdout:9/825: dwrite d0/d7/dc/d32/f107 [0,4194304] 0 2024-02-15T18:27:16.037 INFO:tasks.workunit.client.1.smithi191.stdout:9/826: dread - d0/d29/fb1 zero size 2024-02-15T18:27:16.038 INFO:tasks.workunit.client.1.smithi191.stdout:1/815: write d0/d8/df/d2e/f7f [1369198,53984] 0 2024-02-15T18:27:16.038 INFO:tasks.workunit.client.1.smithi191.stdout:1/816: fdatasync d0/d2f/d5c/f71 0 2024-02-15T18:27:16.038 INFO:tasks.workunit.client.1.smithi191.stdout:1/817: fdatasync d0/d2f/d51/d5e/ffe 0 2024-02-15T18:27:16.040 INFO:tasks.workunit.client.1.smithi191.stdout:2/737: dwrite df/d12/d19/d1a/d35/f80 [0,4194304] 0 2024-02-15T18:27:16.040 INFO:tasks.workunit.client.1.smithi191.stdout:2/738: write df/d12/d1e/fbb [856098,125022] 0 2024-02-15T18:27:16.041 INFO:tasks.workunit.client.1.smithi191.stdout:3/684: dwrite d6/d5d/d72/f97 [0,4194304] 0 2024-02-15T18:27:16.043 INFO:tasks.workunit.client.1.smithi191.stdout:1/818: symlink d0/d8/df/d12/d33/l115 0 2024-02-15T18:27:16.048 INFO:tasks.workunit.client.1.smithi191.stdout:2/739: unlink df/d12/d1f/d42/fba 0 2024-02-15T18:27:16.049 INFO:tasks.workunit.client.1.smithi191.stdout:7/802: dwrite d0/db/d26/d5f/d8c/f106 [0,4194304] 0 2024-02-15T18:27:16.051 INFO:tasks.workunit.client.1.smithi191.stdout:8/820: dwrite d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/fad [0,4194304] 0 2024-02-15T18:27:16.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:14.684701+0000 mgr.smithi042.kfoihb (mgr.24435) 52 : cluster [DBG] pgmap v36: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 116 MiB/s rd, 233 MiB/s wr, 392 op/s 2024-02-15T18:27:16.069 INFO:tasks.workunit.client.1.smithi191.stdout:7/803: dwrite d0/db/d65/dc/d4b/dc1/fc9 [0,4194304] 0 2024-02-15T18:27:16.069 INFO:tasks.workunit.client.1.smithi191.stdout:7/804: dread - d0/db/d65/d11/d5a/d66/d1e/f9b zero size 2024-02-15T18:27:16.071 INFO:tasks.workunit.client.1.smithi191.stdout:9/827: link d0/f18 d0/d7/f110 0 2024-02-15T18:27:16.072 INFO:tasks.workunit.client.1.smithi191.stdout:1/819: symlink d0/d2f/d51/d54/l116 0 2024-02-15T18:27:16.072 INFO:tasks.workunit.client.1.smithi191.stdout:1/820: readlink d0/d2f/d51/lc 0 2024-02-15T18:27:16.073 INFO:tasks.workunit.client.1.smithi191.stdout:2/740: mkdir df/d12/d19/d1a/dfa 0 2024-02-15T18:27:16.083 INFO:tasks.workunit.client.1.smithi191.stdout:7/805: symlink d0/db/d26/d5f/d77/df/l12a 0 2024-02-15T18:27:16.084 INFO:tasks.workunit.client.1.smithi191.stdout:9/828: dread d0/d7/f21 [0,4194304] 0 2024-02-15T18:27:16.084 INFO:tasks.workunit.client.1.smithi191.stdout:8/821: truncate d3/d2c/d1a/d23/d3e/fb4 81867 0 2024-02-15T18:27:16.085 INFO:tasks.workunit.client.1.smithi191.stdout:3/685: dread d6/d3b/ff [0,4194304] 0 2024-02-15T18:27:16.086 INFO:tasks.workunit.client.1.smithi191.stdout:7/806: write d0/db/d26/d5f/d77/d35/d42/fa9 [1077099,120666] 0 2024-02-15T18:27:16.087 INFO:tasks.workunit.client.1.smithi191.stdout:1/821: unlink d0/d8/f57 0 2024-02-15T18:27:16.087 INFO:tasks.workunit.client.1.smithi191.stdout:1/822: chown d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/f100 205541 1 2024-02-15T18:27:16.087 INFO:tasks.workunit.client.1.smithi191.stdout:1/823: chown d0/d8/df/ld2 628 1 2024-02-15T18:27:16.088 INFO:tasks.workunit.client.1.smithi191.stdout:3/686: dread d6/d27/d4d/d5a/da4/fbc [0,4194304] 0 2024-02-15T18:27:16.094 INFO:tasks.workunit.client.1.smithi191.stdout:2/741: dwrite df/d61/d9f/da3/fb3 [0,4194304] 0 2024-02-15T18:27:16.094 INFO:tasks.workunit.client.1.smithi191.stdout:2/742: chown df/d12/cd1 29870 1 2024-02-15T18:27:16.095 INFO:tasks.workunit.client.1.smithi191.stdout:1/824: rename d0/d2f/d51/fd1 to d0/d8/d48/f117 0 2024-02-15T18:27:16.096 INFO:tasks.workunit.client.1.smithi191.stdout:7/807: write d0/db/d26/d5f/d77/db9/ffc [1943262,71521] 0 2024-02-15T18:27:16.100 INFO:tasks.workunit.client.1.smithi191.stdout:4/786: sync 2024-02-15T18:27:16.100 INFO:tasks.workunit.client.1.smithi191.stdout:5/519: sync 2024-02-15T18:27:16.101 INFO:tasks.workunit.client.1.smithi191.stdout:1/825: creat d0/d8/df/d12/d2a/d2b/f118 x:0 0 0 2024-02-15T18:27:16.101 INFO:tasks.workunit.client.1.smithi191.stdout:1/826: chown d0/d8/cb1 177788978 1 2024-02-15T18:27:16.102 INFO:tasks.workunit.client.1.smithi191.stdout:6/740: sync 2024-02-15T18:27:16.102 INFO:tasks.workunit.client.1.smithi191.stdout:0/599: sync 2024-02-15T18:27:16.102 INFO:tasks.workunit.client.1.smithi191.stdout:7/808: symlink d0/db/d65/d11/d5a/d66/dd6/l12b 0 2024-02-15T18:27:16.107 INFO:tasks.workunit.client.1.smithi191.stdout:0/600: write d7/da/d1a/d34/f52 [1980470,759] 0 2024-02-15T18:27:16.111 INFO:tasks.workunit.client.1.smithi191.stdout:4/787: symlink d2/d5/d7/d3e/d63/d8c/l11c 0 2024-02-15T18:27:16.112 INFO:tasks.workunit.client.1.smithi191.stdout:1/827: symlink d0/d8/df/d41/l119 0 2024-02-15T18:27:16.112 INFO:tasks.workunit.client.1.smithi191.stdout:8/822: dwrite d3/d2c/f114 [0,4194304] 0 2024-02-15T18:27:16.112 INFO:tasks.workunit.client.1.smithi191.stdout:8/823: chown d3/d2c/d1a/d23/d2e/d47/d5f/d4f/l85 2473 1 2024-02-15T18:27:16.114 INFO:tasks.workunit.client.1.smithi191.stdout:2/743: unlink df/d12/d19/d1a/d35/d47/d4b/f7e 0 2024-02-15T18:27:16.116 INFO:tasks.workunit.client.1.smithi191.stdout:2/744: read df/d12/d19/d1a/d35/d47/d4b/d7a/fe1 [898930,18613] 0 2024-02-15T18:27:16.117 INFO:tasks.workunit.client.1.smithi191.stdout:6/741: mknod d8/d9/dac/db0/d95/d9d/d6b/d98/ce5 0 2024-02-15T18:27:16.117 INFO:tasks.workunit.client.1.smithi191.stdout:6/742: dread - d8/d9/dac/ddb/dd7/fe4 zero size 2024-02-15T18:27:16.120 INFO:tasks.workunit.client.1.smithi191.stdout:9/829: dwrite d0/d7/dc/fbb [0,4194304] 0 2024-02-15T18:27:16.122 INFO:tasks.workunit.client.1.smithi191.stdout:0/601: rename d7/d4a/l95 to d7/da/d17/d31/ld2 0 2024-02-15T18:27:16.123 INFO:tasks.workunit.client.1.smithi191.stdout:5/520: dwrite d0/d62/f70 [0,4194304] 0 2024-02-15T18:27:16.125 INFO:tasks.workunit.client.1.smithi191.stdout:4/788: rename d2/d5/d7/d90/fef to d2/d5/d7/d90/f11d 0 2024-02-15T18:27:16.125 INFO:tasks.workunit.client.1.smithi191.stdout:4/789: fdatasync d2/d5/d7/d90/d52/f54 0 2024-02-15T18:27:16.125 INFO:tasks.workunit.client.1.smithi191.stdout:4/790: fsync d2/d5/d7/d3e/d63/d8c/f107 0 2024-02-15T18:27:16.127 INFO:tasks.workunit.client.1.smithi191.stdout:7/809: dwrite d0/db/d65/df1/f117 [0,4194304] 0 2024-02-15T18:27:16.127 INFO:tasks.workunit.client.1.smithi191.stdout:7/810: fdatasync d0/db/d65/df1/f117 0 2024-02-15T18:27:16.130 INFO:tasks.workunit.client.1.smithi191.stdout:8/824: mkdir d3/d2c/d1a/d118 0 2024-02-15T18:27:16.131 INFO:tasks.workunit.client.1.smithi191.stdout:1/828: dwrite d0/d8/df/d12/f32 [0,4194304] 0 2024-02-15T18:27:16.133 INFO:tasks.workunit.client.1.smithi191.stdout:1/829: dread d0/d8/df/d2e/f8d [0,4194304] 0 2024-02-15T18:27:16.133 INFO:tasks.workunit.client.1.smithi191.stdout:1/830: read - d0/d8/d48/f117 zero size 2024-02-15T18:27:16.133 INFO:tasks.workunit.client.1.smithi191.stdout:1/831: write d0/d2f/d51/d5e/d68/f86 [406089,9158] 0 2024-02-15T18:27:16.143 INFO:tasks.workunit.client.1.smithi191.stdout:2/745: mkdir df/d12/d19/d1a/d35/d47/d4b/d7a/dce/dfb 0 2024-02-15T18:27:16.143 INFO:tasks.workunit.client.1.smithi191.stdout:2/746: fsync df/d12/f7c 0 2024-02-15T18:27:16.143 INFO:tasks.workunit.client.1.smithi191.stdout:2/747: write df/d12/f55 [1219933,98536] 0 2024-02-15T18:27:16.143 INFO:tasks.workunit.client.1.smithi191.stdout:2/748: dread - df/d12/d19/d1a/feb zero size 2024-02-15T18:27:16.143 INFO:tasks.workunit.client.1.smithi191.stdout:2/749: fdatasync df/d61/d9f/da3/dab/dd6/fe5 0 2024-02-15T18:27:16.147 INFO:tasks.workunit.client.1.smithi191.stdout:9/830: truncate d0/d29/d48/f96 479882 0 2024-02-15T18:27:16.147 INFO:tasks.workunit.client.1.smithi191.stdout:9/831: stat d0/d7/l47 0 2024-02-15T18:27:16.147 INFO:tasks.workunit.client.1.smithi191.stdout:9/832: truncate d0/d29/d48/d66/fff 758638 0 2024-02-15T18:27:16.149 INFO:tasks.workunit.client.1.smithi191.stdout:6/743: rename d8/d9/dac/ce1 to d8/d9/d4a/d4c/ce6 0 2024-02-15T18:27:16.151 INFO:tasks.workunit.client.1.smithi191.stdout:0/602: mkdir d7/da/d17/d31/d5f/dae/dd3 0 2024-02-15T18:27:16.157 INFO:tasks.workunit.client.1.smithi191.stdout:5/521: unlink d0/d18/f2e 0 2024-02-15T18:27:16.158 INFO:tasks.workunit.client.1.smithi191.stdout:4/791: link d2/d5/d7/d2f/cea d2/d5/d8b/c11e 0 2024-02-15T18:27:16.158 INFO:tasks.workunit.client.1.smithi191.stdout:4/792: truncate d2/d5/d7/d2f/d41/f57 4956068 0 2024-02-15T18:27:16.159 INFO:tasks.workunit.client.1.smithi191.stdout:4/793: read d2/d5/d7/d90/d52/f111 [3766812,112035] 0 2024-02-15T18:27:16.165 INFO:tasks.workunit.client.1.smithi191.stdout:8/825: mkdir d3/d2c/d1a/d118/d119 0 2024-02-15T18:27:16.166 INFO:tasks.workunit.client.1.smithi191.stdout:1/832: creat d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/dc9/f11a x:0 0 0 2024-02-15T18:27:16.166 INFO:tasks.workunit.client.1.smithi191.stdout:1/833: stat d0/d2f/d51/d54/db7 0 2024-02-15T18:27:16.166 INFO:tasks.workunit.client.1.smithi191.stdout:1/834: stat d0/d8/df/d12/f3f 0 2024-02-15T18:27:16.167 INFO:tasks.workunit.client.1.smithi191.stdout:9/833: creat d0/d1d/f111 x:0 0 0 2024-02-15T18:27:16.170 INFO:tasks.workunit.client.1.smithi191.stdout:7/811: rename d0/db/d65/d11/d5a/d66/f32 to d0/db/d26/d5f/d77/db9/d126/f12c 0 2024-02-15T18:27:16.170 INFO:tasks.workunit.client.1.smithi191.stdout:6/744: creat d8/d9/d2b/d57/fe7 x:0 0 0 2024-02-15T18:27:16.173 INFO:tasks.workunit.client.1.smithi191.stdout:7/812: read d0/db/ff3 [3209358,55901] 0 2024-02-15T18:27:16.174 INFO:tasks.workunit.client.1.smithi191.stdout:5/522: symlink d0/d18/d5c/d7c/d49/d9f/la8 0 2024-02-15T18:27:16.175 INFO:tasks.workunit.client.1.smithi191.stdout:8/826: mkdir d3/d2c/d1a/d23/d2e/d5b/d69/d11a 0 2024-02-15T18:27:16.175 INFO:tasks.workunit.client.1.smithi191.stdout:8/827: readlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/l85 0 2024-02-15T18:27:16.175 INFO:tasks.workunit.client.1.smithi191.stdout:8/828: readlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/le7 0 2024-02-15T18:27:16.176 INFO:tasks.workunit.client.1.smithi191.stdout:8/829: write d3/d2c/d1a/d23/d2e/d64/fc4 [1083867,51921] 0 2024-02-15T18:27:16.177 INFO:tasks.workunit.client.1.smithi191.stdout:2/750: rename df/d12/d19/d1a/d52/c7d to df/d61/d9f/da3/dab/dd6/cfc 0 2024-02-15T18:27:16.177 INFO:tasks.workunit.client.1.smithi191.stdout:2/751: write df/d12/d19/d1a/fae [190248,89159] 0 2024-02-15T18:27:16.178 INFO:tasks.workunit.client.1.smithi191.stdout:6/745: symlink d8/d9/dac/db0/d95/le8 0 2024-02-15T18:27:16.179 INFO:tasks.workunit.client.1.smithi191.stdout:7/813: unlink d0/db/d26/d5f/d77/d35/d42/f129 0 2024-02-15T18:27:16.180 INFO:tasks.workunit.client.1.smithi191.stdout:4/794: dwrite d2/d5/d7/dee/ffc [0,4194304] 0 2024-02-15T18:27:16.180 INFO:tasks.workunit.client.1.smithi191.stdout:4/795: getdents d2/d5/d14/d71/dcd/ded 0 2024-02-15T18:27:16.180 INFO:tasks.workunit.client.1.smithi191.stdout:5/523: mkdir d0/d18/d5c/d7c/d49/da9 0 2024-02-15T18:27:16.180 INFO:tasks.workunit.client.1.smithi191.stdout:9/834: dread d0/d7/dc/d17/f2d [0,4194304] 0 2024-02-15T18:27:16.181 INFO:tasks.workunit.client.1.smithi191.stdout:1/835: dwrite d0/d8/df/d12/d33/d49/f4a [0,4194304] 0 2024-02-15T18:27:16.182 INFO:tasks.workunit.client.1.smithi191.stdout:8/830: creat d3/d2c/d1a/d23/d2e/d64/dbc/d109/f11b x:0 0 0 2024-02-15T18:27:16.183 INFO:tasks.workunit.client.1.smithi191.stdout:8/831: write d3/d2c/d1a/d23/d2e/d47/f63 [1263120,100294] 0 2024-02-15T18:27:16.183 INFO:tasks.workunit.client.1.smithi191.stdout:8/832: dread - d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d8e/f105 zero size 2024-02-15T18:27:16.183 INFO:tasks.workunit.client.1.smithi191.stdout:2/752: getdents df/d12/d19/d1a/d35/d47 0 2024-02-15T18:27:16.185 INFO:tasks.workunit.client.1.smithi191.stdout:6/746: symlink d8/d9/dac/db0/d95/d9d/d6b/le9 0 2024-02-15T18:27:16.186 INFO:tasks.workunit.client.1.smithi191.stdout:7/814: rename d0/db/d65/d11/d5a/d66/d1e/faf to d0/db/f12d 0 2024-02-15T18:27:16.188 INFO:tasks.workunit.client.1.smithi191.stdout:7/815: dread d0/db/d26/d5f/d77/f97 [0,4194304] 0 2024-02-15T18:27:16.188 INFO:tasks.workunit.client.1.smithi191.stdout:7/816: chown d0/f3a 1493 1 2024-02-15T18:27:16.194 INFO:tasks.workunit.client.1.smithi191.stdout:4/796: link d2/d5/d7/d90/d52/l91 d2/d5/d14/d47/d78/dd0/dd3/de4/l11f 0 2024-02-15T18:27:16.195 INFO:tasks.workunit.client.1.smithi191.stdout:4/797: stat d2/d5/d7/dee 0 2024-02-15T18:27:16.195 INFO:tasks.workunit.client.1.smithi191.stdout:5/524: stat d0/d18/d5c/l6e 0 2024-02-15T18:27:16.195 INFO:tasks.workunit.client.1.smithi191.stdout:5/525: write d0/d18/d5c/d7c/d49/f78 [1209899,89402] 0 2024-02-15T18:27:16.195 INFO:tasks.workunit.client.1.smithi191.stdout:5/526: truncate d0/d18/d5c/d7c/f9c 860125 0 2024-02-15T18:27:16.196 INFO:tasks.workunit.client.1.smithi191.stdout:9/835: creat d0/d7/dc/d32/de4/f112 x:0 0 0 2024-02-15T18:27:16.196 INFO:tasks.workunit.client.1.smithi191.stdout:9/836: stat d0/d7/dc/d30/d4f 0 2024-02-15T18:27:16.196 INFO:tasks.workunit.client.1.smithi191.stdout:9/837: dread - d0/d1d/f111 zero size 2024-02-15T18:27:16.196 INFO:tasks.workunit.client.1.smithi191.stdout:9/838: write d0/d7/dc/d32/d106/f10e [868501,65983] 0 2024-02-15T18:27:16.196 INFO:tasks.workunit.client.1.smithi191.stdout:9/839: chown d0/d7/cfa 3424675 1 2024-02-15T18:27:16.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:27:14.684701+0000 mgr.smithi042.kfoihb (mgr.24435) 52 : cluster [DBG] pgmap v36: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 116 MiB/s rd, 233 MiB/s wr, 392 op/s 2024-02-15T18:27:16.197 INFO:tasks.workunit.client.1.smithi191.stdout:1/836: symlink d0/d2f/d51/d5e/d68/l11b 0 2024-02-15T18:27:16.198 INFO:tasks.workunit.client.1.smithi191.stdout:8/833: link d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/ff4 d3/d2c/d1a/d10b/f11c 0 2024-02-15T18:27:16.202 INFO:tasks.workunit.client.1.smithi191.stdout:7/817: rename d0/db/d26/d5f/d77/d35/dc7/fe5 to d0/db/d65/dc/f12e 0 2024-02-15T18:27:16.202 INFO:tasks.workunit.client.1.smithi191.stdout:7/818: truncate d0/db/d65/dc/d4b/de8/ff8 403735 0 2024-02-15T18:27:16.205 INFO:tasks.workunit.client.1.smithi191.stdout:5/527: rmdir d0/d23/d41 39 2024-02-15T18:27:16.206 INFO:tasks.workunit.client.1.smithi191.stdout:6/747: dwrite d8/fcd [0,4194304] 0 2024-02-15T18:27:16.206 INFO:tasks.workunit.client.1.smithi191.stdout:6/748: write d8/d9/f86 [873236,25766] 0 2024-02-15T18:27:16.206 INFO:tasks.workunit.client.1.smithi191.stdout:2/753: dwrite df/d61/d9f/da3/dab/fdd [0,4194304] 0 2024-02-15T18:27:16.206 INFO:tasks.workunit.client.1.smithi191.stdout:2/754: write df/d12/f36 [4810075,38097] 0 2024-02-15T18:27:16.208 INFO:tasks.workunit.client.1.smithi191.stdout:1/837: creat d0/d2f/d51/d5e/f11c x:0 0 0 2024-02-15T18:27:16.209 INFO:tasks.workunit.client.1.smithi191.stdout:5/528: dread d0/d18/d5c/f6d [0,4194304] 0 2024-02-15T18:27:16.211 INFO:tasks.workunit.client.1.smithi191.stdout:8/834: dread d3/d2c/d1a/fdb [0,4194304] 0 2024-02-15T18:27:16.218 INFO:tasks.workunit.client.1.smithi191.stdout:4/798: rename d2/d5/d7/l116 to d2/d5/d14/d47/l120 0 2024-02-15T18:27:16.218 INFO:tasks.workunit.client.1.smithi191.stdout:4/799: chown d2/d5/d7/d3e/d63/f11a 743245855 1 2024-02-15T18:27:16.218 INFO:tasks.workunit.client.1.smithi191.stdout:4/800: truncate d2/d5/d7/d106/f109 682630 0 2024-02-15T18:27:16.218 INFO:tasks.workunit.client.1.smithi191.stdout:6/749: truncate d8/f33 1817652 0 2024-02-15T18:27:16.221 INFO:tasks.workunit.client.1.smithi191.stdout:2/755: mknod df/d12/d32/dee/cfd 0 2024-02-15T18:27:16.224 INFO:tasks.workunit.client.1.smithi191.stdout:9/840: symlink d0/d7/dc/d8c/l113 0 2024-02-15T18:27:16.232 INFO:tasks.workunit.client.1.smithi191.stdout:1/838: creat d0/d2f/d51/d10/f11d x:0 0 0 2024-02-15T18:27:16.232 INFO:tasks.workunit.client.1.smithi191.stdout:9/841: dread d0/d7/dc/d30/d3b/fa1 [0,4194304] 0 2024-02-15T18:27:16.235 INFO:tasks.workunit.client.1.smithi191.stdout:5/529: symlink d0/d18/d5c/d7c/d49/laa 0 2024-02-15T18:27:16.236 INFO:tasks.workunit.client.1.smithi191.stdout:8/835: creat d3/d2c/d1a/d23/d2a/f11d x:0 0 0 2024-02-15T18:27:16.240 INFO:tasks.workunit.client.1.smithi191.stdout:4/801: symlink d2/d5/d8b/l121 0 2024-02-15T18:27:16.242 INFO:tasks.workunit.client.1.smithi191.stdout:2/756: truncate df/d12/f55 993542 0 2024-02-15T18:27:16.242 INFO:tasks.workunit.client.1.smithi191.stdout:2/757: chown df/d12/d1e/l24 2672 1 2024-02-15T18:27:16.242 INFO:tasks.workunit.client.1.smithi191.stdout:2/758: readlink df/d12/d19/d1a/d35/lc1 0 2024-02-15T18:27:16.243 INFO:tasks.workunit.client.1.smithi191.stdout:6/750: dwrite d8/d9/dac/db0/d95/d90/fbe [0,4194304] 0 2024-02-15T18:27:16.243 INFO:tasks.workunit.client.1.smithi191.stdout:6/751: readlink d8/l25 0 2024-02-15T18:27:16.243 INFO:tasks.workunit.client.1.smithi191.stdout:6/752: write d8/d9/dac/ddb/fdf [451664,115411] 0 2024-02-15T18:27:16.243 INFO:tasks.workunit.client.1.smithi191.stdout:1/839: creat d0/d8/df/d12/d2a/d2b/d78/d60/d109/dc8/f11e x:0 0 0 2024-02-15T18:27:16.244 INFO:tasks.workunit.client.1.smithi191.stdout:9/842: creat d0/d7/dc/d30/d3b/d8b/f114 x:0 0 0 2024-02-15T18:27:16.249 INFO:tasks.workunit.client.1.smithi191.stdout:5/530: link d0/d23/d41/d28/d31/d5a/d61/c88 d0/d23/d41/d3c/cab 0 2024-02-15T18:27:16.249 INFO:tasks.workunit.client.1.smithi191.stdout:5/531: chown d0/d23/d41/d28/d31/d5a/d61/c88 509431 1 2024-02-15T18:27:16.250 INFO:tasks.workunit.client.1.smithi191.stdout:8/836: truncate d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/fc8 3305033 0 2024-02-15T18:27:16.254 INFO:tasks.workunit.client.1.smithi191.stdout:4/802: creat d2/d5/d7/d3e/d63/d8c/dbb/f122 x:0 0 0 2024-02-15T18:27:16.254 INFO:tasks.workunit.client.1.smithi191.stdout:9/843: dread d0/d7/dc/d17/f61 [4194304,4194304] 0 2024-02-15T18:27:16.258 INFO:tasks.workunit.client.1.smithi191.stdout:2/759: getdents df/d12/d1f 0 2024-02-15T18:27:16.261 INFO:tasks.workunit.client.1.smithi191.stdout:6/753: rename d8/d9/d4a/d4c/ce6 to d8/d9/dac/db0/d95/d9d/d6b/d98/cea 0 2024-02-15T18:27:16.261 INFO:tasks.workunit.client.1.smithi191.stdout:6/754: truncate d8/d9/d4a/d4c/f9e 742727 0 2024-02-15T18:27:16.264 INFO:tasks.workunit.client.1.smithi191.stdout:1/840: symlink d0/d8/df/d12/d33/l11f 0 2024-02-15T18:27:16.275 INFO:tasks.workunit.client.1.smithi191.stdout:4/803: creat d2/d5/d7/d3e/de8/d108/f123 x:0 0 0 2024-02-15T18:27:16.275 INFO:tasks.workunit.client.1.smithi191.stdout:4/804: truncate d2/f5c 4433390 0 2024-02-15T18:27:16.277 INFO:tasks.workunit.client.1.smithi191.stdout:9/844: mkdir d0/d7/dc/d30/d3b/d56/d115 0 2024-02-15T18:27:16.279 INFO:tasks.workunit.client.1.smithi191.stdout:6/755: dwrite d8/d9/dac/db0/d95/d9d/f1d [0,4194304] 0 2024-02-15T18:27:16.281 INFO:tasks.workunit.client.1.smithi191.stdout:9/845: dread d0/d29/d48/fc3 [0,4194304] 0 2024-02-15T18:27:16.281 INFO:tasks.workunit.client.1.smithi191.stdout:9/846: dread - d0/d7/dc/d30/d3b/d8b/f114 zero size 2024-02-15T18:27:16.281 INFO:tasks.workunit.client.1.smithi191.stdout:6/756: dread d8/d9/d4a/d4c/d7f/fd8 [0,4194304] 0 2024-02-15T18:27:16.283 INFO:tasks.workunit.client.1.smithi191.stdout:6/757: write d8/f33 [718061,127335] 0 2024-02-15T18:27:16.283 INFO:tasks.workunit.client.1.smithi191.stdout:2/760: rmdir df/d12/d19/d1a/d35/d47/d4b/d7a/dce/dfb 0 2024-02-15T18:27:16.283 INFO:tasks.workunit.client.1.smithi191.stdout:9/847: write d0/d7/dc/d8c/f35 [2813121,81462] 0 2024-02-15T18:27:16.283 INFO:tasks.workunit.client.1.smithi191.stdout:9/848: stat d0/d1d/f111 0 2024-02-15T18:27:16.285 INFO:tasks.workunit.client.1.smithi191.stdout:1/841: rmdir d0/d2f/d5c 39 2024-02-15T18:27:16.285 INFO:tasks.workunit.client.1.smithi191.stdout:1/842: stat d0/d8/df/d12/d33/l11f 0 2024-02-15T18:27:16.285 INFO:tasks.workunit.client.1.smithi191.stdout:1/843: chown d0/d2f/d77/faf 60699 1 2024-02-15T18:27:16.292 INFO:tasks.workunit.client.1.smithi191.stdout:5/532: dwrite d0/d23/d41/d28/d31/f33 [0,4194304] 0 2024-02-15T18:27:16.293 INFO:tasks.workunit.client.1.smithi191.stdout:2/761: getdents df/d61/d9f/da3/dab/dd6 0 2024-02-15T18:27:16.294 INFO:tasks.workunit.client.1.smithi191.stdout:2/762: stat df/d12 0 2024-02-15T18:27:16.294 INFO:tasks.workunit.client.1.smithi191.stdout:5/533: write d0/d23/f27 [666586,116392] 0 2024-02-15T18:27:16.295 INFO:tasks.workunit.client.1.smithi191.stdout:4/805: rename d2/d5/d86 to d2/d5/d124 0 2024-02-15T18:27:16.302 INFO:tasks.workunit.client.1.smithi191.stdout:5/534: creat d0/d62/fac x:0 0 0 2024-02-15T18:27:16.303 INFO:tasks.workunit.client.1.smithi191.stdout:4/806: creat d2/d5/d7/d2f/f125 x:0 0 0 2024-02-15T18:27:16.306 INFO:tasks.workunit.client.1.smithi191.stdout:2/763: dread df/d12/d1f/f34 [0,4194304] 0 2024-02-15T18:27:16.307 INFO:tasks.workunit.client.1.smithi191.stdout:2/764: write df/d12/d19/d1a/dda/ff3 [884488,20000] 0 2024-02-15T18:27:16.307 INFO:tasks.workunit.client.1.smithi191.stdout:2/765: dread - df/d12/d19/d1a/d35/d47/d4b/d7a/ddb/fe0 zero size 2024-02-15T18:27:16.307 INFO:tasks.workunit.client.1.smithi191.stdout:6/758: dwrite d8/d9/d4a/d4c/fc0 [0,4194304] 0 2024-02-15T18:27:16.307 INFO:tasks.workunit.client.1.smithi191.stdout:4/807: mkdir d2/d5/d7/d3e/d63/d8c/d126 0 2024-02-15T18:27:16.309 INFO:tasks.workunit.client.1.smithi191.stdout:5/535: mknod d0/cad 0 2024-02-15T18:27:16.313 INFO:tasks.workunit.client.1.smithi191.stdout:6/759: creat d8/d9/d2b/feb x:0 0 0 2024-02-15T18:27:16.314 INFO:tasks.workunit.client.1.smithi191.stdout:4/808: creat d2/d5/d7/d3e/f127 x:0 0 0 2024-02-15T18:27:16.314 INFO:tasks.workunit.client.1.smithi191.stdout:1/844: dwrite d0/ff1 [0,4194304] 0 2024-02-15T18:27:16.314 INFO:tasks.workunit.client.1.smithi191.stdout:1/845: dread - d0/d2f/d51/d10/ffc zero size 2024-02-15T18:27:16.319 INFO:tasks.workunit.client.1.smithi191.stdout:6/760: symlink d8/d9/dac/db0/lec 0 2024-02-15T18:27:16.319 INFO:tasks.workunit.client.1.smithi191.stdout:6/761: dread - d8/d9/d2b/d57/d8e/faf zero size 2024-02-15T18:27:16.319 INFO:tasks.workunit.client.1.smithi191.stdout:6/762: write d8/d9/dac/ddb/fa8 [223542,112069] 0 2024-02-15T18:27:16.319 INFO:tasks.workunit.client.1.smithi191.stdout:6/763: stat d8/d9/d4a 0 2024-02-15T18:27:16.319 INFO:tasks.workunit.client.1.smithi191.stdout:6/764: read - d8/d9/dac/db0/d95/d9d/d6b/d98/fbf zero size 2024-02-15T18:27:16.319 INFO:tasks.workunit.client.1.smithi191.stdout:6/765: truncate d8/d9/d2b/d57/fda 865911 0 2024-02-15T18:27:16.319 INFO:tasks.workunit.client.1.smithi191.stdout:5/536: dread d0/d23/d41/d28/d31/d5a/d61/f82 [0,4194304] 0 2024-02-15T18:27:16.322 INFO:tasks.workunit.client.1.smithi191.stdout:4/809: creat d2/d5/d7/d90/d52/dcb/f128 x:0 0 0 2024-02-15T18:27:16.322 INFO:tasks.workunit.client.1.smithi191.stdout:2/766: dwrite df/d12/d19/d1a/d35/d47/d4b/fd9 [0,4194304] 0 2024-02-15T18:27:16.326 INFO:tasks.workunit.client.1.smithi191.stdout:1/846: rename d0/d2f/d5c/lef to d0/de7/l120 0 2024-02-15T18:27:16.327 INFO:tasks.workunit.client.1.smithi191.stdout:6/766: dread d8/d9/d2b/f51 [0,4194304] 0 2024-02-15T18:27:16.328 INFO:tasks.workunit.client.1.smithi191.stdout:5/537: symlink d0/d23/d41/d28/d65/d85/d52/lae 0 2024-02-15T18:27:16.328 INFO:tasks.workunit.client.1.smithi191.stdout:5/538: chown d0/d18/d5c/d7c/d49/c5d 25852775 1 2024-02-15T18:27:16.351 INFO:tasks.workunit.client.1.smithi191.stdout:4/810: mkdir d2/d5/d7/d3e/de8/d108/d129 0 2024-02-15T18:27:16.351 INFO:tasks.workunit.client.1.smithi191.stdout:4/811: truncate d2/d5/d7/d90/fd4 890657 0 2024-02-15T18:27:16.351 INFO:tasks.workunit.client.1.smithi191.stdout:4/812: stat d2/d5/d8b/l121 0 2024-02-15T18:27:16.364 INFO:tasks.workunit.client.1.smithi191.stdout:2/767: unlink df/lc5 0 2024-02-15T18:27:16.366 INFO:tasks.workunit.client.1.smithi191.stdout:1/847: rename d0/d8/df/d12/d2a/d2b/l76 to d0/d2f/d51/d5e/l121 0 2024-02-15T18:27:16.368 INFO:tasks.workunit.client.1.smithi191.stdout:4/813: dwrite d2/f5c [0,4194304] 0 2024-02-15T18:27:16.368 INFO:tasks.workunit.client.1.smithi191.stdout:4/814: write d2/d5/d7/d3e/d63/f7d [540550,66945] 0 2024-02-15T18:27:16.368 INFO:tasks.workunit.client.1.smithi191.stdout:5/539: mknod d0/d23/d41/d28/d31/d5a/d61/caf 0 2024-02-15T18:27:16.368 INFO:tasks.workunit.client.1.smithi191.stdout:5/540: chown d0/d23/d41/d28/f4a 26 1 2024-02-15T18:27:16.369 INFO:tasks.workunit.client.1.smithi191.stdout:4/815: read d2/d5/d14/d47/d78/f61 [2574812,69941] 0 2024-02-15T18:27:16.370 INFO:tasks.workunit.client.1.smithi191.stdout:2/768: truncate df/d12/d19/d1a/f21 3781466 0 2024-02-15T18:27:16.371 INFO:tasks.workunit.client.1.smithi191.stdout:6/767: rename d8/d9/dac/db0/d95/d9d/d6b/d72/lb4 to d8/d9/dac/db0/d95/d90/led 0 2024-02-15T18:27:16.377 INFO:tasks.workunit.client.1.smithi191.stdout:2/769: dread df/d12/d19/d1a/d35/d47/d4b/d7a/fe1 [0,4194304] 0 2024-02-15T18:27:16.382 INFO:tasks.workunit.client.1.smithi191.stdout:4/816: dwrite d2/d5/d7/d3e/faf [0,4194304] 0 2024-02-15T18:27:16.383 INFO:tasks.workunit.client.1.smithi191.stdout:4/817: write d2/d5/d7/d2f/d41/f57 [537937,66272] 0 2024-02-15T18:27:16.383 INFO:tasks.workunit.client.1.smithi191.stdout:4/818: write d2/d5/d14/fb8 [237192,49513] 0 2024-02-15T18:27:16.384 INFO:tasks.workunit.client.1.smithi191.stdout:5/541: unlink d0/d23/d41/d28/d31/d5a/f8c 0 2024-02-15T18:27:16.384 INFO:tasks.workunit.client.1.smithi191.stdout:5/542: truncate d0/d18/d5c/f91 884132 0 2024-02-15T18:27:16.384 INFO:tasks.workunit.client.1.smithi191.stdout:5/543: write d0/d23/d41/d28/d65/d85/d52/f40 [5374514,1313] 0 2024-02-15T18:27:16.384 INFO:tasks.workunit.client.1.smithi191.stdout:5/544: chown d0/d23/d41/d28/f3f 22347 1 2024-02-15T18:27:16.387 INFO:tasks.workunit.client.1.smithi191.stdout:1/848: rename d0/d2f/d51/c4c to d0/d2f/c122 0 2024-02-15T18:27:16.388 INFO:tasks.workunit.client.1.smithi191.stdout:6/768: rmdir d8/d9/dac/db0/d95/d9d/d6b/d76 39 2024-02-15T18:27:16.388 INFO:tasks.workunit.client.1.smithi191.stdout:6/769: chown d8/d9/d2b/fb2 16108 1 2024-02-15T18:27:16.388 INFO:tasks.workunit.client.1.smithi191.stdout:6/770: write d8/d9/d2b/d57/f93 [1219253,102539] 0 2024-02-15T18:27:16.393 INFO:tasks.workunit.client.1.smithi191.stdout:2/770: creat df/d12/d19/d1a/d35/d47/d4b/d7a/ffe x:0 0 0 2024-02-15T18:27:16.393 INFO:tasks.workunit.client.1.smithi191.stdout:2/771: fdatasync df/d12/d19/d1a/d35/d5b/dbf/f9d 0 2024-02-15T18:27:16.393 INFO:tasks.workunit.client.1.smithi191.stdout:2/772: write df/f40 [319842,109930] 0 2024-02-15T18:27:16.396 INFO:tasks.workunit.client.1.smithi191.stdout:2/773: dread df/d12/d19/d1a/f5e [0,4194304] 0 2024-02-15T18:27:16.396 INFO:tasks.workunit.client.1.smithi191.stdout:2/774: write df/d12/d19/d1a/d35/d47/d4b/d7a/f86 [965708,106616] 0 2024-02-15T18:27:16.396 INFO:tasks.workunit.client.1.smithi191.stdout:2/775: write df/d12/d19/d1a/f2c [993979,124323] 0 2024-02-15T18:27:16.402 INFO:tasks.workunit.client.1.smithi191.stdout:4/819: mknod d2/d5/d7/d3e/de8/d108/d129/c12a 0 2024-02-15T18:27:16.402 INFO:tasks.workunit.client.1.smithi191.stdout:4/820: chown d2/d5/d7/d2f/d41/f8f 11265 1 2024-02-15T18:27:16.403 INFO:tasks.workunit.client.1.smithi191.stdout:1/849: mknod d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/dc9/c123 0 2024-02-15T18:27:16.403 INFO:tasks.workunit.client.1.smithi191.stdout:5/545: dwrite d0/d18/d5c/d7c/f9c [0,4194304] 0 2024-02-15T18:27:16.403 INFO:tasks.workunit.client.1.smithi191.stdout:6/771: creat d8/d9/d4a/d4c/fee x:0 0 0 2024-02-15T18:27:16.405 INFO:tasks.workunit.client.1.smithi191.stdout:5/546: write d0/d23/d41/d28/f4a [1055950,38397] 0 2024-02-15T18:27:16.405 INFO:tasks.workunit.client.1.smithi191.stdout:5/547: stat d0/d23/d41/d28/d65 0 2024-02-15T18:27:16.408 INFO:tasks.workunit.client.1.smithi191.stdout:2/776: creat df/d61/fff x:0 0 0 2024-02-15T18:27:16.409 INFO:tasks.workunit.client.1.smithi191.stdout:4/821: creat d2/d5/d7/d2f/f12b x:0 0 0 2024-02-15T18:27:16.412 INFO:tasks.workunit.client.1.smithi191.stdout:5/548: creat d0/d23/d41/d28/d65/d85/d52/fb0 x:0 0 0 2024-02-15T18:27:16.413 INFO:tasks.workunit.client.1.smithi191.stdout:6/772: dread d8/f97 [0,4194304] 0 2024-02-15T18:27:16.415 INFO:tasks.workunit.client.1.smithi191.stdout:2/777: symlink df/d12/d19/d1a/d35/d47/d4b/d7a/dce/l100 0 2024-02-15T18:27:16.415 INFO:tasks.workunit.client.1.smithi191.stdout:4/822: rmdir d2/d5/d7/d90/d52/db5 39 2024-02-15T18:27:16.417 INFO:tasks.workunit.client.1.smithi191.stdout:6/773: creat d8/d9/dac/db0/d95/d9d/d6b/d98/fef x:0 0 0 2024-02-15T18:27:16.422 INFO:tasks.workunit.client.1.smithi191.stdout:1/850: dwrite d0/d2f/f10e [0,4194304] 0 2024-02-15T18:27:16.425 INFO:tasks.workunit.client.1.smithi191.stdout:2/778: dread df/d12/d1f/f26 [0,4194304] 0 2024-02-15T18:27:16.425 INFO:tasks.workunit.client.1.smithi191.stdout:6/774: rename d8/d9/d4a/d4c/d7f/dcf to d8/d9/d2b/d57/df0 0 2024-02-15T18:27:16.425 INFO:tasks.workunit.client.1.smithi191.stdout:6/775: write d8/d9/d2b/fe3 [646627,122309] 0 2024-02-15T18:27:16.425 INFO:tasks.workunit.client.1.smithi191.stdout:6/776: read d8/fab [4148544,17734] 0 2024-02-15T18:27:16.426 INFO:tasks.workunit.client.1.smithi191.stdout:2/779: write df/d12/d1f/f31 [111592,108944] 0 2024-02-15T18:27:16.426 INFO:tasks.workunit.client.1.smithi191.stdout:6/777: read d8/d9/dac/db0/d95/d9d/f7a [2343333,71400] 0 2024-02-15T18:27:16.426 INFO:tasks.workunit.client.1.smithi191.stdout:6/778: chown d8/d9/d2b/d81/l8f 9236929 1 2024-02-15T18:27:16.430 INFO:tasks.workunit.client.1.smithi191.stdout:4/823: mkdir d2/d5/d12c 0 2024-02-15T18:27:16.431 INFO:tasks.workunit.client.1.smithi191.stdout:4/824: read d2/d5/d7/d2f/fc7 [2060285,119363] 0 2024-02-15T18:27:16.434 INFO:tasks.workunit.client.1.smithi191.stdout:2/780: dread f7 [0,4194304] 0 2024-02-15T18:27:16.434 INFO:tasks.workunit.client.1.smithi191.stdout:2/781: chown df/d12/d19/d1a/d35/d47/d4b/d7a/ddb/fe0 1210295 1 2024-02-15T18:27:16.435 INFO:tasks.workunit.client.1.smithi191.stdout:6/779: write d8/d9/d2b/fb2 [525499,71874] 0 2024-02-15T18:27:16.443 INFO:tasks.workunit.client.1.smithi191.stdout:1/851: dwrite d0/d8/df/d12/d2a/f52 [0,4194304] 0 2024-02-15T18:27:16.443 INFO:tasks.workunit.client.1.smithi191.stdout:2/782: truncate df/d12/d19/d1a/d35/d47/d4b/d7a/fe1 3031847 0 2024-02-15T18:27:16.443 INFO:tasks.workunit.client.1.smithi191.stdout:2/783: truncate df/d12/d19/d1a/d35/d47/d4b/d7a/ddb/fe0 566732 0 2024-02-15T18:27:16.446 INFO:tasks.workunit.client.1.smithi191.stdout:6/780: rename d8/d9/d2b/d5b to d8/d9/d2b/d81/df1 0 2024-02-15T18:27:16.447 INFO:tasks.workunit.client.1.smithi191.stdout:6/781: fsync d8/d9/d2b/d57/d8e/d91/fbb 0 2024-02-15T18:27:16.447 INFO:tasks.workunit.client.1.smithi191.stdout:6/782: chown d8/d9/d2b 9429 1 2024-02-15T18:27:16.453 INFO:tasks.workunit.client.1.smithi191.stdout:1/852: rename d0/d8/df/d12/d2a/d2b/d78/d60/d109/dc8/f11e to d0/f124 0 2024-02-15T18:27:16.453 INFO:tasks.workunit.client.1.smithi191.stdout:4/825: dwrite d2/d5/d14/d47/d78/dd0/dd3/de4/ff6 [4194304,4194304] 0 2024-02-15T18:27:16.460 INFO:tasks.workunit.client.1.smithi191.stdout:6/783: getdents d8/d9 0 2024-02-15T18:27:16.462 INFO:tasks.workunit.client.1.smithi191.stdout:2/784: dwrite df/d61/f67 [0,4194304] 0 2024-02-15T18:27:16.462 INFO:tasks.workunit.client.1.smithi191.stdout:1/853: symlink d0/d2f/l125 0 2024-02-15T18:27:16.464 INFO:tasks.workunit.client.1.smithi191.stdout:4/826: rmdir d2/d5/d7/d106 39 2024-02-15T18:27:16.464 INFO:tasks.workunit.client.1.smithi191.stdout:4/827: write d2/f28 [1888681,6749] 0 2024-02-15T18:27:16.465 INFO:tasks.workunit.client.1.smithi191.stdout:6/784: write d8/fab [3685581,4863] 0 2024-02-15T18:27:16.465 INFO:tasks.workunit.client.1.smithi191.stdout:6/785: write d8/d9/dac/db0/fc6 [599420,127082] 0 2024-02-15T18:27:16.475 INFO:tasks.workunit.client.1.smithi191.stdout:2/785: mknod df/d12/d19/d1a/d35/d47/d4b/d7a/c101 0 2024-02-15T18:27:16.475 INFO:tasks.workunit.client.1.smithi191.stdout:1/854: getdents d0/d8/df/d12/d2a/d2b/d78/d60/d74 0 2024-02-15T18:27:16.476 INFO:tasks.workunit.client.1.smithi191.stdout:1/855: write d0/d2f/d51/d5e/d68/fce [941046,95000] 0 2024-02-15T18:27:16.478 INFO:tasks.workunit.client.1.smithi191.stdout:4/828: truncate d2/f2c 1918691 0 2024-02-15T18:27:16.481 INFO:tasks.workunit.client.1.smithi191.stdout:6/786: creat d8/d9/d2b/d84/ddd/ff2 x:0 0 0 2024-02-15T18:27:16.482 INFO:tasks.workunit.client.1.smithi191.stdout:2/786: symlink df/d12/d19/d1a/d52/df2/l102 0 2024-02-15T18:27:16.485 INFO:tasks.workunit.client.1.smithi191.stdout:1/856: dread d0/d2f/d51/d54/d7b/fc6 [0,4194304] 0 2024-02-15T18:27:16.485 INFO:tasks.workunit.client.1.smithi191.stdout:4/829: creat d2/d5/d7/d3e/d63/f12d x:0 0 0 2024-02-15T18:27:16.486 INFO:tasks.workunit.client.1.smithi191.stdout:6/787: creat d8/d9/dac/db0/d95/d9d/d6b/d72/ff3 x:0 0 0 2024-02-15T18:27:16.487 INFO:tasks.workunit.client.1.smithi191.stdout:2/787: symlink df/d61/d9f/da3/l103 0 2024-02-15T18:27:16.489 INFO:tasks.workunit.client.1.smithi191.stdout:1/857: symlink d0/d2f/d5c/l126 0 2024-02-15T18:27:16.489 INFO:tasks.workunit.client.1.smithi191.stdout:1/858: truncate d0/f9 1869770 0 2024-02-15T18:27:16.490 INFO:tasks.workunit.client.1.smithi191.stdout:6/788: mkdir d8/d9/dac/ddb/dd7/df4 0 2024-02-15T18:27:16.492 INFO:tasks.workunit.client.1.smithi191.stdout:1/859: rename d0/d8/df/d12/d2a/d2b/d78/l80 to d0/d2f/d51/d54/db7/l127 0 2024-02-15T18:27:16.496 INFO:tasks.workunit.client.1.smithi191.stdout:4/830: dread d2/d5/d8b/f8d [0,4194304] 0 2024-02-15T18:27:16.498 INFO:tasks.workunit.client.1.smithi191.stdout:6/789: mknod d8/d9/d2b/d57/d8e/cf5 0 2024-02-15T18:27:16.498 INFO:tasks.workunit.client.1.smithi191.stdout:1/860: creat d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/dc9/f128 x:0 0 0 2024-02-15T18:27:16.500 INFO:tasks.workunit.client.1.smithi191.stdout:4/831: creat d2/d5/d7/d90/d52/f12e x:0 0 0 2024-02-15T18:27:16.500 INFO:tasks.workunit.client.1.smithi191.stdout:6/790: mkdir d8/d9/d4a/df6 0 2024-02-15T18:27:16.502 INFO:tasks.workunit.client.1.smithi191.stdout:1/861: mkdir d0/d8/df/d10c/d129 0 2024-02-15T18:27:16.503 INFO:tasks.workunit.client.1.smithi191.stdout:6/791: rmdir d8/d9/d4a/de2 0 2024-02-15T18:27:16.504 INFO:tasks.workunit.client.1.smithi191.stdout:1/862: creat d0/d2f/d51/d54/dd3/f12a x:0 0 0 2024-02-15T18:27:16.505 INFO:tasks.workunit.client.1.smithi191.stdout:6/792: rmdir d8/d9/d2b/d57/df0 39 2024-02-15T18:27:16.506 INFO:tasks.workunit.client.1.smithi191.stdout:1/863: dread d0/d2f/d51/d5e/d68/f86 [0,4194304] 0 2024-02-15T18:27:16.506 INFO:tasks.workunit.client.1.smithi191.stdout:1/864: chown d0/d8/df/d2e/c105 1720175 1 2024-02-15T18:27:16.506 INFO:tasks.workunit.client.1.smithi191.stdout:6/793: dread d8/d9/f86 [0,4194304] 0 2024-02-15T18:27:16.506 INFO:tasks.workunit.client.1.smithi191.stdout:6/794: chown d8/d9/d2b/la2 32616838 1 2024-02-15T18:27:16.511 INFO:tasks.workunit.client.1.smithi191.stdout:2/788: dwrite df/d12/d19/d1a/d35/d47/d4b/d7a/fe1 [0,4194304] 0 2024-02-15T18:27:16.511 INFO:tasks.workunit.client.1.smithi191.stdout:4/832: dread d2/d5/d7/d3e/f49 [0,4194304] 0 2024-02-15T18:27:16.511 INFO:tasks.workunit.client.1.smithi191.stdout:2/789: dread - df/d61/fff zero size 2024-02-15T18:27:16.517 INFO:tasks.workunit.client.1.smithi191.stdout:2/790: dread df/d12/d19/d1a/d35/d47/d4b/f99 [0,4194304] 0 2024-02-15T18:27:16.624 INFO:tasks.workunit.client.1.smithi191.stdout:6/795: truncate d8/d9/dac/ddb/fdf 551935 0 2024-02-15T18:27:16.624 INFO:tasks.workunit.client.1.smithi191.stdout:6/796: fdatasync d8/d9/d4a/d4c/d7f/fd8 0 2024-02-15T18:27:16.625 INFO:tasks.workunit.client.1.smithi191.stdout:4/833: creat d2/d5/d14/d47/d78/dd0/f12f x:0 0 0 2024-02-15T18:27:16.625 INFO:tasks.workunit.client.1.smithi191.stdout:4/834: stat d2/d5/d14/d71/fac 0 2024-02-15T18:27:16.626 INFO:tasks.workunit.client.1.smithi191.stdout:2/791: symlink df/d12/d19/d1a/d52/db0/l104 0 2024-02-15T18:27:16.626 INFO:tasks.workunit.client.1.smithi191.stdout:2/792: fdatasync df/f1b 0 2024-02-15T18:27:16.626 INFO:tasks.workunit.client.1.smithi191.stdout:2/793: fsync df/d12/d1e/d6f/fad 0 2024-02-15T18:27:16.627 INFO:tasks.workunit.client.1.smithi191.stdout:2/794: fdatasync df/d12/d1f/d42/db5/fe8 0 2024-02-15T18:27:16.627 INFO:tasks.workunit.client.1.smithi191.stdout:2/795: readlink df/d61/d9f/lcd 0 2024-02-15T18:27:16.627 INFO:tasks.workunit.client.1.smithi191.stdout:2/796: chown df/d12/d32/lcf 523 1 2024-02-15T18:27:16.627 INFO:tasks.workunit.client.1.smithi191.stdout:2/797: chown df/d12/d1e/c46 1937 1 2024-02-15T18:27:16.628 INFO:tasks.workunit.client.1.smithi191.stdout:1/865: write d0/d8/df/d41/f9e [1337626,51402] 0 2024-02-15T18:27:16.633 INFO:tasks.workunit.client.1.smithi191.stdout:6/797: creat d8/d9/d4a/d4c/dc4/ff7 x:0 0 0 2024-02-15T18:27:16.634 INFO:tasks.workunit.client.1.smithi191.stdout:4/835: symlink d2/l130 0 2024-02-15T18:27:16.638 INFO:tasks.workunit.client.1.smithi191.stdout:2/798: rmdir df/d12/d19/d1a/d35/d47/d4b/d7a/dce 39 2024-02-15T18:27:16.640 INFO:tasks.workunit.client.1.smithi191.stdout:1/866: truncate d0/d8/f96 1856380 0 2024-02-15T18:27:16.642 INFO:tasks.workunit.client.1.smithi191.stdout:6/798: creat d8/d9/d2b/d81/ff8 x:0 0 0 2024-02-15T18:27:16.642 INFO:tasks.workunit.client.1.smithi191.stdout:6/799: fsync d8/d9/d4a/d4c/fc0 0 2024-02-15T18:27:16.643 INFO:tasks.workunit.client.1.smithi191.stdout:4/836: link d2/d5/d7/d106/f109 d2/d5/d7/d90/d52/db5/dc9/f131 0 2024-02-15T18:27:16.644 INFO:tasks.workunit.client.1.smithi191.stdout:2/799: stat df/l38 0 2024-02-15T18:27:16.648 INFO:tasks.workunit.client.1.smithi191.stdout:1/867: creat d0/d8/df/d12/d2a/d2b/d78/d60/d109/f12b x:0 0 0 2024-02-15T18:27:16.650 INFO:tasks.workunit.client.1.smithi191.stdout:2/800: rename df/d12/d19/d1a/d35/d5b/f9b to df/d12/d19/d1a/d52/f105 0 2024-02-15T18:27:16.652 INFO:tasks.workunit.client.1.smithi191.stdout:2/801: read df/d12/d32/fc4 [403586,124764] 0 2024-02-15T18:27:16.652 INFO:tasks.workunit.client.1.smithi191.stdout:2/802: truncate df/d12/d19/d1a/dda/ff3 1580648 0 2024-02-15T18:27:16.653 INFO:tasks.workunit.client.1.smithi191.stdout:1/868: truncate d0/d2f/d66/f95 423526 0 2024-02-15T18:27:16.653 INFO:tasks.workunit.client.1.smithi191.stdout:1/869: chown d0/d2f/f10e 602301 1 2024-02-15T18:27:16.653 INFO:tasks.workunit.client.1.smithi191.stdout:1/870: chown d0/d8/df/d2e/f7f 221744282 1 2024-02-15T18:27:16.653 INFO:tasks.workunit.client.1.smithi191.stdout:1/871: chown d0/d2f/d66/lfa 1424413058 1 2024-02-15T18:27:16.653 INFO:tasks.workunit.client.1.smithi191.stdout:1/872: write d0/d8/df/d12/d2a/f3a [4709600,120561] 0 2024-02-15T18:27:16.654 INFO:tasks.workunit.client.1.smithi191.stdout:2/803: creat df/d12/d19/f106 x:0 0 0 2024-02-15T18:27:16.654 INFO:tasks.workunit.client.1.smithi191.stdout:2/804: fsync df/d61/d9f/da3/dab/fdd 0 2024-02-15T18:27:16.654 INFO:tasks.workunit.client.1.smithi191.stdout:2/805: fsync df/d12/d19/d1a/f2c 0 2024-02-15T18:27:16.654 INFO:tasks.workunit.client.1.smithi191.stdout:2/806: readlink df/d61/d9f/da3/lc9 0 2024-02-15T18:27:16.654 INFO:tasks.workunit.client.1.smithi191.stdout:2/807: read df/d12/d32/f92 [56532,19947] 0 2024-02-15T18:27:16.654 INFO:tasks.workunit.client.1.smithi191.stdout:2/808: fsync df/d12/d19/d1a/d35/d47/d4b/fd9 0 2024-02-15T18:27:16.655 INFO:tasks.workunit.client.1.smithi191.stdout:2/809: read df/d12/d1f/f34 [57873,26703] 0 2024-02-15T18:27:16.655 INFO:tasks.workunit.client.1.smithi191.stdout:2/810: dread - df/d12/d19/d1a/d35/f74 zero size 2024-02-15T18:27:16.655 INFO:tasks.workunit.client.1.smithi191.stdout:1/873: mknod d0/d2f/d51/d54/db7/c12c 0 2024-02-15T18:27:16.656 INFO:tasks.workunit.client.1.smithi191.stdout:3/687: sync 2024-02-15T18:27:16.656 INFO:tasks.workunit.client.1.smithi191.stdout:6/800: dwrite d8/d9/f79 [0,4194304] 0 2024-02-15T18:27:16.656 INFO:tasks.workunit.client.1.smithi191.stdout:6/801: write d8/d9/d4a/d4c/da4/fce [221858,112172] 0 2024-02-15T18:27:16.656 INFO:tasks.workunit.client.1.smithi191.stdout:6/802: chown d8/d9/dac/db0/d95/d9d/lb8 52158 1 2024-02-15T18:27:16.659 INFO:tasks.workunit.client.1.smithi191.stdout:6/803: dread d8/d9/d2b/d57/fa9 [4194304,4194304] 0 2024-02-15T18:27:16.659 INFO:tasks.workunit.client.1.smithi191.stdout:6/804: fsync d8/d9/d2b/d57/fe7 0 2024-02-15T18:27:16.663 INFO:tasks.workunit.client.1.smithi191.stdout:2/811: truncate df/d12/d19/d1a/d35/d47/d4b/d7a/ddb/fe0 202041 0 2024-02-15T18:27:16.665 INFO:tasks.workunit.client.1.smithi191.stdout:1/874: creat d0/d8/f12d x:0 0 0 2024-02-15T18:27:16.680 INFO:tasks.workunit.client.1.smithi191.stdout:2/812: rename df/d12/f2b to df/d12/d19/d1a/d35/d47/f107 0 2024-02-15T18:27:16.680 INFO:tasks.workunit.client.1.smithi191.stdout:1/875: unlink d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/dc9/f128 0 2024-02-15T18:27:16.686 INFO:tasks.workunit.client.1.smithi191.stdout:3/688: link d6/d1b/d60/d73/f8d d6/d5d/d72/dd5/fe9 0 2024-02-15T18:27:16.689 INFO:tasks.workunit.client.1.smithi191.stdout:2/813: creat df/d12/d19/f108 x:0 0 0 2024-02-15T18:27:16.689 INFO:tasks.workunit.client.1.smithi191.stdout:2/814: truncate df/d12/d32/d4d/f50 590998 0 2024-02-15T18:27:16.689 INFO:tasks.workunit.client.1.smithi191.stdout:2/815: write df/d12/d32/f92 [1015704,67093] 0 2024-02-15T18:27:16.691 INFO:tasks.workunit.client.1.smithi191.stdout:1/876: mkdir d0/de7/d12e 0 2024-02-15T18:27:16.691 INFO:tasks.workunit.client.1.smithi191.stdout:1/877: read - d0/d2f/d51/d54/dd3/f114 zero size 2024-02-15T18:27:16.692 INFO:tasks.workunit.client.1.smithi191.stdout:3/689: unlink d6/d27/l2b 0 2024-02-15T18:27:16.693 INFO:tasks.workunit.client.1.smithi191.stdout:2/816: unlink df/d12/d19/f106 0 2024-02-15T18:27:16.694 INFO:tasks.workunit.client.1.smithi191.stdout:2/817: truncate df/f40 490356 0 2024-02-15T18:27:16.694 INFO:tasks.workunit.client.1.smithi191.stdout:1/878: getdents d0/d2f/d51/d54/db7 0 2024-02-15T18:27:16.694 INFO:tasks.workunit.client.1.smithi191.stdout:1/879: write d0/d2f/d51/f93 [823763,87042] 0 2024-02-15T18:27:16.694 INFO:tasks.workunit.client.1.smithi191.stdout:1/880: write d0/d2f/d51/d5e/f11c [826130,58262] 0 2024-02-15T18:27:16.695 INFO:tasks.workunit.client.1.smithi191.stdout:3/690: readlink d6/d1b/d60/d73/l35 0 2024-02-15T18:27:16.697 INFO:tasks.workunit.client.1.smithi191.stdout:1/881: creat d0/d8/d108/f12f x:0 0 0 2024-02-15T18:27:16.698 INFO:tasks.workunit.client.1.smithi191.stdout:1/882: mkdir d0/d8/d48/d130 0 2024-02-15T18:27:16.715 INFO:tasks.workunit.client.1.smithi191.stdout:6/805: dwrite d8/d9/d2b/d57/f59 [0,4194304] 0 2024-02-15T18:27:16.715 INFO:tasks.workunit.client.1.smithi191.stdout:6/806: stat d8/d9/d4a/d4c/dc4/ff7 0 2024-02-15T18:27:16.721 INFO:tasks.workunit.client.1.smithi191.stdout:6/807: write d8/d9/dac/db0/d95/d9d/f1d [2453689,28995] 0 2024-02-15T18:27:16.724 INFO:tasks.workunit.client.1.smithi191.stdout:6/808: creat d8/d9/dac/ff9 x:0 0 0 2024-02-15T18:27:16.730 INFO:tasks.workunit.client.1.smithi191.stdout:6/809: creat d8/d9/ffa x:0 0 0 2024-02-15T18:27:16.730 INFO:tasks.workunit.client.1.smithi191.stdout:6/810: write d8/d9/d2b/d57/d8e/faf [934295,66108] 0 2024-02-15T18:27:16.730 INFO:tasks.workunit.client.1.smithi191.stdout:2/818: dwrite df/f1b [4194304,4194304] 0 2024-02-15T18:27:16.730 INFO:tasks.workunit.client.1.smithi191.stdout:2/819: chown df/d12/d19/d1a/d35/d5b/dbf/cd0 0 1 2024-02-15T18:27:16.731 INFO:tasks.workunit.client.1.smithi191.stdout:6/811: truncate d8/d9/dac/db0/d95/d9d/f1f 1811883 0 2024-02-15T18:27:16.731 INFO:tasks.workunit.client.1.smithi191.stdout:6/812: chown d8/d9 1 1 2024-02-15T18:27:16.732 INFO:tasks.workunit.client.1.smithi191.stdout:3/691: dwrite d6/d1b/d60/d73/d5e/fc5 [0,4194304] 0 2024-02-15T18:27:16.732 INFO:tasks.workunit.client.1.smithi191.stdout:2/820: creat df/d61/d94/f109 x:0 0 0 2024-02-15T18:27:16.734 INFO:tasks.workunit.client.1.smithi191.stdout:6/813: creat d8/d9/d2b/d57/ffb x:0 0 0 2024-02-15T18:27:16.740 INFO:tasks.workunit.client.1.smithi191.stdout:1/883: dwrite d0/d2f/d66/f95 [0,4194304] 0 2024-02-15T18:27:16.740 INFO:tasks.workunit.client.1.smithi191.stdout:1/884: chown d0/d8/df/d12/d33/d49/f107 3 1 2024-02-15T18:27:16.740 INFO:tasks.workunit.client.1.smithi191.stdout:1/885: dread - d0/d8/df/d12/d33/d49/f107 zero size 2024-02-15T18:27:16.751 INFO:tasks.workunit.client.1.smithi191.stdout:3/692: creat d6/d27/d4d/d5a/fea x:0 0 0 2024-02-15T18:27:16.751 INFO:tasks.workunit.client.1.smithi191.stdout:2/821: dwrite df/d12/f36 [0,4194304] 0 2024-02-15T18:27:16.751 INFO:tasks.workunit.client.1.smithi191.stdout:3/693: write d6/d3b/d42/d49/d54/fc2 [1000517,117616] 0 2024-02-15T18:27:16.755 INFO:tasks.workunit.client.1.smithi191.stdout:6/814: symlink d8/d9/d4a/df6/lfc 0 2024-02-15T18:27:16.755 INFO:tasks.workunit.client.1.smithi191.stdout:6/815: stat d8/d9/d2b/d57/d8e/d91/fbb 0 2024-02-15T18:27:16.758 INFO:tasks.workunit.client.1.smithi191.stdout:1/886: creat d0/d8/df/d12/d2a/d2b/d78/f131 x:0 0 0 2024-02-15T18:27:16.758 INFO:tasks.workunit.client.1.smithi191.stdout:2/822: getdents df/d12/d19/d1a/d35/d47/d4b 0 2024-02-15T18:27:16.763 INFO:tasks.workunit.client.1.smithi191.stdout:3/694: creat d6/d3b/d42/dbb/feb x:0 0 0 2024-02-15T18:27:16.763 INFO:tasks.workunit.client.1.smithi191.stdout:3/695: readlink d6/d1b/d60/d73/d5e/l8c 0 2024-02-15T18:27:16.763 INFO:tasks.workunit.client.1.smithi191.stdout:6/816: creat d8/d9/dac/db0/d95/d9d/d6b/d72/ffd x:0 0 0 2024-02-15T18:27:16.764 INFO:tasks.workunit.client.1.smithi191.stdout:1/887: rmdir d0/d2f/d51/d10/d9c 39 2024-02-15T18:27:16.764 INFO:tasks.workunit.client.1.smithi191.stdout:1/888: chown d0/d8/df/d12/d2a/d2b/f61 102672 1 2024-02-15T18:27:16.764 INFO:tasks.workunit.client.1.smithi191.stdout:1/889: write d0/d8/df/d12/d33/f8e [5175771,105053] 0 2024-02-15T18:27:16.764 INFO:tasks.workunit.client.1.smithi191.stdout:1/890: readlink d0/d8/df/d12/d2a/d2b/d78/d60/le1 0 2024-02-15T18:27:16.766 INFO:tasks.workunit.client.1.smithi191.stdout:2/823: truncate df/d12/d32/fc4 1220844 0 2024-02-15T18:27:16.766 INFO:tasks.workunit.client.1.smithi191.stdout:2/824: truncate df/d12/d1e/fbb 1715027 0 2024-02-15T18:27:16.767 INFO:tasks.workunit.client.1.smithi191.stdout:3/696: mkdir d6/d1b/d60/da2/dac/de2/dec 0 2024-02-15T18:27:16.769 INFO:tasks.workunit.client.1.smithi191.stdout:0/603: sync 2024-02-15T18:27:16.769 INFO:tasks.workunit.client.1.smithi191.stdout:0/604: write d7/d3b/f9a [351342,61610] 0 2024-02-15T18:27:16.769 INFO:tasks.workunit.client.1.smithi191.stdout:0/605: truncate d7/da/d1a/d34/fa0 786271 0 2024-02-15T18:27:16.769 INFO:tasks.workunit.client.1.smithi191.stdout:0/606: write d7/da/d1a/d2c/d4c/fac [305230,48990] 0 2024-02-15T18:27:16.769 INFO:tasks.workunit.client.1.smithi191.stdout:0/607: write d7/da/d17/d31/d5f/dae/fbe [652120,40122] 0 2024-02-15T18:27:16.769 INFO:tasks.workunit.client.1.smithi191.stdout:0/608: dread - d7/da/d17/d31/d7d/fc8 zero size 2024-02-15T18:27:16.771 INFO:tasks.workunit.client.1.smithi191.stdout:0/609: dread d7/da/d1a/d34/d27/f7c [0,4194304] 0 2024-02-15T18:27:16.771 INFO:tasks.workunit.client.1.smithi191.stdout:6/817: mkdir d8/d9/dac/db0/d95/d9d/d6b/d72/dfe 0 2024-02-15T18:27:16.775 INFO:tasks.workunit.client.1.smithi191.stdout:2/825: truncate df/f1b 1632012 0 2024-02-15T18:27:16.775 INFO:tasks.workunit.client.1.smithi191.stdout:2/826: rename df/d12/d32 to df/d12/d32/d4d/d10a 22 2024-02-15T18:27:16.778 INFO:tasks.workunit.client.1.smithi191.stdout:3/697: mknod d6/d5d/d72/dd5/db2/ced 0 2024-02-15T18:27:16.779 INFO:tasks.workunit.client.1.smithi191.stdout:2/827: dread df/d12/d19/d1a/d52/fc0 [0,4194304] 0 2024-02-15T18:27:16.779 INFO:tasks.workunit.client.1.smithi191.stdout:2/828: chown df/d12/cd1 199075 1 2024-02-15T18:27:16.779 INFO:tasks.workunit.client.1.smithi191.stdout:6/818: rmdir d8/d9/d4a/d4c/d7f 39 2024-02-15T18:27:16.779 INFO:tasks.workunit.client.1.smithi191.stdout:6/819: getdents d8/d9/dac/ddb/dd7/df4 0 2024-02-15T18:27:16.781 INFO:tasks.workunit.client.1.smithi191.stdout:1/891: dwrite d0/d8/d48/f4d [4194304,4194304] 0 2024-02-15T18:27:16.782 INFO:tasks.workunit.client.1.smithi191.stdout:2/829: unlink df/l6d 0 2024-02-15T18:27:16.784 INFO:tasks.workunit.client.1.smithi191.stdout:3/698: dread d6/d3b/d42/d49/d54/fc2 [0,4194304] 0 2024-02-15T18:27:16.789 INFO:tasks.workunit.client.1.smithi191.stdout:6/820: creat d8/d9/dac/db0/d95/d9d/d6b/d72/fff x:0 0 0 2024-02-15T18:27:16.791 INFO:tasks.workunit.client.1.smithi191.stdout:1/892: rename d0/d2f/d51/d54/cec to d0/d2f/d51/d54/c132 0 2024-02-15T18:27:16.804 INFO:tasks.workunit.client.1.smithi191.stdout:6/821: mknod d8/d9/dac/ddb/dd7/c100 0 2024-02-15T18:27:16.806 INFO:tasks.workunit.client.1.smithi191.stdout:3/699: link d6/d27/l4f d6/d3b/lee 0 2024-02-15T18:27:16.806 INFO:tasks.workunit.client.1.smithi191.stdout:3/700: fdatasync d6/d3b/fa7 0 2024-02-15T18:27:16.809 INFO:tasks.workunit.client.1.smithi191.stdout:2/830: dwrite df/d61/d94/f98 [0,4194304] 0 2024-02-15T18:27:16.809 INFO:tasks.workunit.client.1.smithi191.stdout:3/701: chown d6/d3b/c29 210620 1 2024-02-15T18:27:16.812 INFO:tasks.workunit.client.1.smithi191.stdout:0/610: dwrite d7/da/d1a/d34/f5e [0,4194304] 0 2024-02-15T18:27:16.812 INFO:tasks.workunit.client.1.smithi191.stdout:0/611: read - d7/d4a/f97 zero size 2024-02-15T18:27:16.812 INFO:tasks.workunit.client.1.smithi191.stdout:0/612: fdatasync d7/da/f8e 0 2024-02-15T18:27:16.817 INFO:tasks.workunit.client.1.smithi191.stdout:2/831: mknod df/d12/d19/d1a/dda/c10b 0 2024-02-15T18:27:16.817 INFO:tasks.workunit.client.1.smithi191.stdout:2/832: dread - df/d12/d1e/d6f/fc3 zero size 2024-02-15T18:27:16.819 INFO:tasks.workunit.client.1.smithi191.stdout:1/893: dwrite d0/d8/df/d12/d2a/fe3 [0,4194304] 0 2024-02-15T18:27:16.820 INFO:tasks.workunit.client.1.smithi191.stdout:1/894: read d0/d8/f30 [399934,54476] 0 2024-02-15T18:27:16.820 INFO:tasks.workunit.client.1.smithi191.stdout:1/895: fdatasync d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/f100 0 2024-02-15T18:27:16.821 INFO:tasks.workunit.client.1.smithi191.stdout:3/702: unlink d6/d3b/fa7 0 2024-02-15T18:27:16.822 INFO:tasks.workunit.client.1.smithi191.stdout:1/896: dread d0/d8/df/f85 [0,4194304] 0 2024-02-15T18:27:16.822 INFO:tasks.workunit.client.1.smithi191.stdout:6/822: dwrite d8/d9/dac/db0/d95/d9d/d6b/d76/fb1 [0,4194304] 0 2024-02-15T18:27:16.823 INFO:tasks.workunit.client.1.smithi191.stdout:0/613: creat d7/d4a/fd4 x:0 0 0 2024-02-15T18:27:16.823 INFO:tasks.workunit.client.1.smithi191.stdout:2/833: creat df/d12/d19/d1a/d35/d5b/dbf/f10c x:0 0 0 2024-02-15T18:27:16.829 INFO:tasks.workunit.client.1.smithi191.stdout:3/703: getdents d6/d27/d4d/d5a/d92 0 2024-02-15T18:27:16.830 INFO:tasks.workunit.client.1.smithi191.stdout:6/823: creat d8/f101 x:0 0 0 2024-02-15T18:27:16.830 INFO:tasks.workunit.client.1.smithi191.stdout:6/824: chown d8/d9/dac/db0/d95/lad 186 1 2024-02-15T18:27:16.832 INFO:tasks.workunit.client.1.smithi191.stdout:1/897: dread d0/f1b [0,4194304] 0 2024-02-15T18:27:16.833 INFO:tasks.workunit.client.1.smithi191.stdout:3/704: write d6/d27/d4d/da8/fb3 [10788319,107795] 0 2024-02-15T18:27:16.834 INFO:tasks.workunit.client.1.smithi191.stdout:0/614: creat d7/da/d1a/d2c/d4c/fd5 x:0 0 0 2024-02-15T18:27:16.837 INFO:tasks.workunit.client.1.smithi191.stdout:2/834: creat df/d12/d1f/f10d x:0 0 0 2024-02-15T18:27:16.839 INFO:tasks.workunit.client.1.smithi191.stdout:1/898: link d0/fa8 d0/d2f/d51/d54/dd3/f133 0 2024-02-15T18:27:16.840 INFO:tasks.workunit.client.1.smithi191.stdout:3/705: getdents d6/d5d 0 2024-02-15T18:27:16.840 INFO:tasks.workunit.client.1.smithi191.stdout:3/706: write d6/d3b/f84 [1988744,57604] 0 2024-02-15T18:27:16.842 INFO:tasks.workunit.client.1.smithi191.stdout:1/899: mknod d0/d8/c134 0 2024-02-15T18:27:16.842 INFO:tasks.workunit.client.1.smithi191.stdout:1/900: write d0/d2f/d51/d54/dd3/f133 [393352,70624] 0 2024-02-15T18:27:16.846 INFO:tasks.workunit.client.1.smithi191.stdout:1/901: mknod d0/d8/df/d12/d33/c135 0 2024-02-15T18:27:16.847 INFO:tasks.workunit.client.1.smithi191.stdout:6/825: dwrite d8/f33 [0,4194304] 0 2024-02-15T18:27:16.848 INFO:tasks.workunit.client.1.smithi191.stdout:7/819: sync 2024-02-15T18:27:16.848 INFO:tasks.workunit.client.1.smithi191.stdout:8/837: sync 2024-02-15T18:27:16.849 INFO:tasks.workunit.client.1.smithi191.stdout:7/820: mkdir d0/db/d65/dc/d6e/d12f 0 2024-02-15T18:27:16.849 INFO:tasks.workunit.client.1.smithi191.stdout:1/902: write d0/d2f/d51/d5e/d68/fce [467133,14566] 0 2024-02-15T18:27:16.850 INFO:tasks.workunit.client.1.smithi191.stdout:1/903: write d0/d2f/f10e [4933316,95190] 0 2024-02-15T18:27:16.850 INFO:tasks.workunit.client.1.smithi191.stdout:8/838: mkdir d3/d2c/d1a/d23/d2e/d47/d5f/d11e 0 2024-02-15T18:27:16.852 INFO:tasks.workunit.client.1.smithi191.stdout:7/821: unlink d0/db/d26/d5f/d77/df/lca 0 2024-02-15T18:27:16.853 INFO:tasks.workunit.client.1.smithi191.stdout:1/904: mknod d0/d2f/d66/c136 0 2024-02-15T18:27:16.854 INFO:tasks.workunit.client.1.smithi191.stdout:1/905: read d0/d2f/d51/d5e/d68/fce [163541,25381] 0 2024-02-15T18:27:16.855 INFO:tasks.workunit.client.1.smithi191.stdout:1/906: dread - d0/d8/df/d2e/fe5 zero size 2024-02-15T18:27:16.855 INFO:tasks.workunit.client.1.smithi191.stdout:1/907: fsync d0/d8/df/d41/f99 0 2024-02-15T18:27:16.855 INFO:tasks.workunit.client.1.smithi191.stdout:1/908: write d0/d8/df/d12/f32 [5116598,43945] 0 2024-02-15T18:27:16.855 INFO:tasks.workunit.client.1.smithi191.stdout:8/839: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/f11f x:0 0 0 2024-02-15T18:27:16.857 INFO:tasks.workunit.client.1.smithi191.stdout:7/822: rmdir d0/db/d26/d5f/d77/d35/d42 39 2024-02-15T18:27:16.858 INFO:tasks.workunit.client.1.smithi191.stdout:6/826: dread d8/d9/f79 [0,4194304] 0 2024-02-15T18:27:16.859 INFO:tasks.workunit.client.1.smithi191.stdout:1/909: rmdir d0/d2f/d51/d54 39 2024-02-15T18:27:16.859 INFO:tasks.workunit.client.1.smithi191.stdout:1/910: truncate d0/f124 752614 0 2024-02-15T18:27:16.861 INFO:tasks.workunit.client.1.smithi191.stdout:8/840: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d115/c120 0 2024-02-15T18:27:16.861 INFO:tasks.workunit.client.1.smithi191.stdout:8/841: truncate d3/d2c/d1a/d23/d3e/fa9 3408540 0 2024-02-15T18:27:16.869 INFO:tasks.workunit.client.1.smithi191.stdout:2/835: dwrite df/d12/d32/d4d/d89/ff6 [4194304,4194304] 0 2024-02-15T18:27:16.871 INFO:tasks.workunit.client.1.smithi191.stdout:1/911: getdents d0/d8/df/d12/d2a 0 2024-02-15T18:27:16.872 INFO:tasks.workunit.client.1.smithi191.stdout:8/842: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/f121 x:0 0 0 2024-02-15T18:27:16.873 INFO:tasks.workunit.client.1.smithi191.stdout:8/843: write d3/d2c/d1a/d23/d2e/d5b/d69/d6d/fab [4472472,47388] 0 2024-02-15T18:27:16.874 INFO:tasks.workunit.client.1.smithi191.stdout:8/844: readlink d3/d2c/d1a/d23/l25 0 2024-02-15T18:27:16.874 INFO:tasks.workunit.client.1.smithi191.stdout:8/845: fsync d3/d2c/d1a/d23/d2e/de9/ffc 0 2024-02-15T18:27:16.874 INFO:tasks.workunit.client.1.smithi191.stdout:2/836: mknod df/d12/d19/d1a/d52/c10e 0 2024-02-15T18:27:16.875 INFO:tasks.workunit.client.1.smithi191.stdout:1/912: symlink d0/d8/d48/d130/l137 0 2024-02-15T18:27:16.878 INFO:tasks.workunit.client.1.smithi191.stdout:2/837: symlink df/d12/d19/d1a/d35/d47/da2/l10f 0 2024-02-15T18:27:16.878 INFO:tasks.workunit.client.1.smithi191.stdout:2/838: stat df/d12/d19/d1a/d35/f74 0 2024-02-15T18:27:16.878 INFO:tasks.workunit.client.1.smithi191.stdout:2/839: write df/d12/d19/d1a/d35/d5b/dbf/f10c [355572,78486] 0 2024-02-15T18:27:16.878 INFO:tasks.workunit.client.1.smithi191.stdout:1/913: mknod d0/d2f/d51/d5e/d68/c138 0 2024-02-15T18:27:16.881 INFO:tasks.workunit.client.1.smithi191.stdout:2/840: rename df/d61/d9f/da3/dab/dd6 to df/d12/d32/d4d/d89/d110 0 2024-02-15T18:27:16.882 INFO:tasks.workunit.client.1.smithi191.stdout:3/707: dwrite f4 [4194304,4194304] 0 2024-02-15T18:27:16.882 INFO:tasks.workunit.client.1.smithi191.stdout:3/708: truncate d6/d3b/fba 12908404 0 2024-02-15T18:27:16.882 INFO:tasks.workunit.client.1.smithi191.stdout:1/914: link d0/d8/f17 d0/d2f/d51/d5e/f139 0 2024-02-15T18:27:16.882 INFO:tasks.workunit.client.1.smithi191.stdout:1/915: write d0/d8/df/d12/d33/d49/f113 [769898,69188] 0 2024-02-15T18:27:16.886 INFO:tasks.workunit.client.1.smithi191.stdout:2/841: rmdir df/d12/d19/d1a/d35/d47/d4b 39 2024-02-15T18:27:16.887 INFO:tasks.workunit.client.1.smithi191.stdout:3/709: mkdir d6/d3b/d42/d49/def 0 2024-02-15T18:27:16.888 INFO:tasks.workunit.client.1.smithi191.stdout:7/823: dwrite d0/db/f12d [0,4194304] 0 2024-02-15T18:27:16.889 INFO:tasks.workunit.client.1.smithi191.stdout:7/824: fdatasync d0/db/d65/d11/d5a/fb6 0 2024-02-15T18:27:16.889 INFO:tasks.workunit.client.1.smithi191.stdout:1/916: rename d0/f124 to d0/d2f/d5c/f13a 0 2024-02-15T18:27:16.889 INFO:tasks.workunit.client.1.smithi191.stdout:1/917: chown d0/d2f/d66/c7d 10 1 2024-02-15T18:27:16.889 INFO:tasks.workunit.client.1.smithi191.stdout:1/918: chown d0/d2f/c122 51561356 1 2024-02-15T18:27:16.890 INFO:tasks.workunit.client.1.smithi191.stdout:2/842: mkdir df/d12/d19/d1a/d35/d47/da2/d111 0 2024-02-15T18:27:16.890 INFO:tasks.workunit.client.1.smithi191.stdout:2/843: fsync df/d12/d19/d1a/dda/ff9 0 2024-02-15T18:27:16.890 INFO:tasks.workunit.client.1.smithi191.stdout:8/846: dwrite d3/f18 [4194304,4194304] 0 2024-02-15T18:27:16.892 INFO:tasks.workunit.client.1.smithi191.stdout:7/825: unlink d0/db/d26/d5f/d77/df/l11b 0 2024-02-15T18:27:16.892 INFO:tasks.workunit.client.1.smithi191.stdout:1/919: creat d0/d8/d48/f13b x:0 0 0 2024-02-15T18:27:16.893 INFO:tasks.workunit.client.1.smithi191.stdout:1/920: stat d0 0 2024-02-15T18:27:16.895 INFO:tasks.workunit.client.1.smithi191.stdout:2/844: read df/d12/d19/d1a/d35/d47/d4b/fd9 [2280052,94585] 0 2024-02-15T18:27:16.895 INFO:tasks.workunit.client.1.smithi191.stdout:2/845: chown df/d12/fbd 1 1 2024-02-15T18:27:16.895 INFO:tasks.workunit.client.1.smithi191.stdout:2/846: write df/d12/d1e/d6f/fad [425838,39218] 0 2024-02-15T18:27:16.895 INFO:tasks.workunit.client.1.smithi191.stdout:6/827: dwrite d8/fab [0,4194304] 0 2024-02-15T18:27:16.895 INFO:tasks.workunit.client.1.smithi191.stdout:3/710: dread d6/d27/d4d/f58 [4194304,4194304] 0 2024-02-15T18:27:16.901 INFO:tasks.workunit.client.1.smithi191.stdout:7/826: dread d0/db/d65/f82 [0,4194304] 0 2024-02-15T18:27:16.908 INFO:tasks.workunit.client.1.smithi191.stdout:1/921: dwrite d0/d2f/d51/d5e/ffe [0,4194304] 0 2024-02-15T18:27:16.916 INFO:tasks.workunit.client.1.smithi191.stdout:7/827: dwrite d0/fcb [0,4194304] 0 2024-02-15T18:27:16.916 INFO:tasks.workunit.client.1.smithi191.stdout:7/828: write d0/db/d65/d11/d5a/d66/d1e/fa5 [4496759,60692] 0 2024-02-15T18:27:16.916 INFO:tasks.workunit.client.1.smithi191.stdout:7/829: dread - d0/db/d65/d11/d57/fd9 zero size 2024-02-15T18:27:16.957 INFO:tasks.workunit.client.1.smithi191.stdout:8/847: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d115/f122 x:0 0 0 2024-02-15T18:27:16.965 INFO:tasks.workunit.client.1.smithi191.stdout:2/847: truncate df/d12/d1f/d42/f43 2112550 0 2024-02-15T18:27:16.966 INFO:tasks.workunit.client.1.smithi191.stdout:2/848: write df/d12/d19/d1a/d35/d47/f107 [4064893,4214] 0 2024-02-15T18:27:16.974 INFO:tasks.workunit.client.1.smithi191.stdout:2/849: dread df/d12/f22 [0,4194304] 0 2024-02-15T18:27:16.976 INFO:tasks.workunit.client.1.smithi191.stdout:8/848: dwrite d3/d2c/d1a/d23/d2e/d64/f76 [0,4194304] 0 2024-02-15T18:27:16.976 INFO:tasks.workunit.client.1.smithi191.stdout:8/849: fsync d3/d2c/d1a/d23/f4c 0 2024-02-15T18:27:16.982 INFO:tasks.workunit.client.1.smithi191.stdout:6/828: mkdir d8/d9/d4a/df6/d102 0 2024-02-15T18:27:16.982 INFO:tasks.workunit.client.1.smithi191.stdout:3/711: rmdir d6/d3b/d42/d49/dc7/de3 39 2024-02-15T18:27:16.985 INFO:tasks.workunit.client.1.smithi191.stdout:1/922: getdents d0/d2f 0 2024-02-15T18:27:16.992 INFO:tasks.workunit.client.1.smithi191.stdout:7/830: creat d0/db/d65/df1/f130 x:0 0 0 2024-02-15T18:27:16.994 INFO:tasks.workunit.client.1.smithi191.stdout:2/850: creat df/d12/d19/d1a/d52/f112 x:0 0 0 2024-02-15T18:27:16.995 INFO:tasks.workunit.client.1.smithi191.stdout:8/850: link d3/d2c/d1a/d23/d3e/db2/l108 d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/l123 0 2024-02-15T18:27:16.999 INFO:tasks.workunit.client.1.smithi191.stdout:6/829: mkdir d8/d9/d2b/d57/d103 0 2024-02-15T18:27:17.000 INFO:tasks.workunit.client.1.smithi191.stdout:3/712: mknod d6/d5d/d72/cf0 0 2024-02-15T18:27:17.002 INFO:tasks.workunit.client.1.smithi191.stdout:1/923: mkdir d0/d2f/d51/d5e/d13c 0 2024-02-15T18:27:17.002 INFO:tasks.workunit.client.1.smithi191.stdout:1/924: readlink d0/d2f/d51/l43 0 2024-02-15T18:27:17.002 INFO:tasks.workunit.client.1.smithi191.stdout:1/925: truncate d0/d2f/d51/d10/f5b 679407 0 2024-02-15T18:27:17.002 INFO:tasks.workunit.client.1.smithi191.stdout:7/831: dread d0/db/d65/d11/d1c/f6f [0,4194304] 0 2024-02-15T18:27:17.002 INFO:tasks.workunit.client.1.smithi191.stdout:7/832: chown d0/db/d65/cd1 137 1 2024-02-15T18:27:17.004 INFO:tasks.workunit.client.1.smithi191.stdout:2/851: truncate df/d12/d32/fc4 692019 0 2024-02-15T18:27:17.006 INFO:tasks.workunit.client.1.smithi191.stdout:8/851: symlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/da0/d112/l124 0 2024-02-15T18:27:17.006 INFO:tasks.workunit.client.1.smithi191.stdout:8/852: write d3/d2c/d1a/d23/d2e/d47/d5f/ff3 [658375,88263] 0 2024-02-15T18:27:17.006 INFO:tasks.workunit.client.1.smithi191.stdout:8/853: chown d3/d2c/d1a/d23/d2a/l87 0 1 2024-02-15T18:27:17.006 INFO:tasks.workunit.client.1.smithi191.stdout:8/854: write d3/d2c/d1a/d23/d2a/f6e [2514781,112298] 0 2024-02-15T18:27:17.007 INFO:tasks.workunit.client.1.smithi191.stdout:3/713: getdents d6/d3b/d30 0 2024-02-15T18:27:17.007 INFO:tasks.workunit.client.1.smithi191.stdout:3/714: fdatasync d6/d27/d4d/da8/fb1 0 2024-02-15T18:27:17.007 INFO:tasks.workunit.client.1.smithi191.stdout:3/715: chown d6/fa 32998022 1 2024-02-15T18:27:17.007 INFO:tasks.workunit.client.1.smithi191.stdout:3/716: fdatasync d6/d27/fa5 0 2024-02-15T18:27:17.009 INFO:tasks.workunit.client.1.smithi191.stdout:6/830: dread d8/d9/f36 [0,4194304] 0 2024-02-15T18:27:17.009 INFO:tasks.workunit.client.1.smithi191.stdout:2/852: rename df/d12/d19/d1a/d52/db0 to df/d12/d1f/de6/d113 0 2024-02-15T18:27:17.009 INFO:tasks.workunit.client.1.smithi191.stdout:2/853: stat df/d12/f7c 0 2024-02-15T18:27:17.010 INFO:tasks.workunit.client.1.smithi191.stdout:2/854: truncate df/d12/d1f/f10d 575813 0 2024-02-15T18:27:17.015 INFO:tasks.workunit.client.1.smithi191.stdout:6/831: write d8/d9/d4a/d4c/f5d [73042,104416] 0 2024-02-15T18:27:17.016 INFO:tasks.workunit.client.1.smithi191.stdout:8/855: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/c125 0 2024-02-15T18:27:17.017 INFO:tasks.workunit.client.1.smithi191.stdout:3/717: symlink d6/d1b/lf1 0 2024-02-15T18:27:17.019 INFO:tasks.workunit.client.1.smithi191.stdout:2/855: creat df/d12/d19/d1a/d35/d47/d4b/d7a/dcc/f114 x:0 0 0 2024-02-15T18:27:17.020 INFO:tasks.workunit.client.1.smithi191.stdout:6/832: unlink d8/d9/dac/db0/d95/d9d/d6b/d76/lde 0 2024-02-15T18:27:17.021 INFO:tasks.workunit.client.1.smithi191.stdout:7/833: dwrite d0/db/d26/d5f/d77/d35/f48 [0,4194304] 0 2024-02-15T18:27:17.022 INFO:tasks.workunit.client.1.smithi191.stdout:2/856: mknod df/d61/d94/c115 0 2024-02-15T18:27:17.023 INFO:tasks.workunit.client.1.smithi191.stdout:2/857: dread df/d12/d1f/f10d [0,4194304] 0 2024-02-15T18:27:17.025 INFO:tasks.workunit.client.1.smithi191.stdout:6/833: creat d8/d9/d4a/d4c/dc4/f104 x:0 0 0 2024-02-15T18:27:17.025 INFO:tasks.workunit.client.1.smithi191.stdout:6/834: truncate d8/d9/dac/db0/d95/d90/fd0 635846 0 2024-02-15T18:27:17.025 INFO:tasks.workunit.client.1.smithi191.stdout:6/835: truncate d8/d9/d2b/feb 32044 0 2024-02-15T18:27:17.025 INFO:tasks.workunit.client.1.smithi191.stdout:6/836: chown d8/d9/d2b/l7d 3573283 1 2024-02-15T18:27:17.030 INFO:tasks.workunit.client.1.smithi191.stdout:7/834: link d0/db/f2d d0/db/d65/d11/d5a/d66/f131 0 2024-02-15T18:27:17.030 INFO:tasks.workunit.client.1.smithi191.stdout:7/835: truncate d0/db/f12d 4771847 0 2024-02-15T18:27:17.031 INFO:tasks.workunit.client.1.smithi191.stdout:7/836: dread d0/db/d65/dc/d4b/de8/ff8 [0,4194304] 0 2024-02-15T18:27:17.031 INFO:tasks.workunit.client.1.smithi191.stdout:7/837: fdatasync d0/db/d26/d5f/d77/d35/f48 0 2024-02-15T18:27:17.035 INFO:tasks.workunit.client.1.smithi191.stdout:2/858: creat df/d12/d1e/d6f/f116 x:0 0 0 2024-02-15T18:27:17.036 INFO:tasks.workunit.client.1.smithi191.stdout:2/859: write df/d12/d19/d1a/d35/f74 [698970,116763] 0 2024-02-15T18:27:17.040 INFO:tasks.workunit.client.1.smithi191.stdout:6/837: truncate d8/d9/d2b/f42 832984 0 2024-02-15T18:27:17.043 INFO:tasks.workunit.client.1.smithi191.stdout:7/838: unlink d0/db/d26/d5f/c121 0 2024-02-15T18:27:17.045 INFO:tasks.workunit.client.1.smithi191.stdout:2/860: symlink df/d12/d19/d1a/d35/d47/d4b/d7a/dce/l117 0 2024-02-15T18:27:17.046 INFO:tasks.workunit.client.1.smithi191.stdout:2/861: write df/d12/d19/d1a/d35/d47/f51 [354840,52201] 0 2024-02-15T18:27:17.047 INFO:tasks.workunit.client.1.smithi191.stdout:6/838: creat d8/d9/dac/db0/d95/d90/f105 x:0 0 0 2024-02-15T18:27:17.050 INFO:tasks.workunit.client.1.smithi191.stdout:7/839: mknod d0/db/d65/dc/d4b/de8/df0/c132 0 2024-02-15T18:27:17.050 INFO:tasks.workunit.client.1.smithi191.stdout:7/840: stat d0/db/d65/d11/d5a/cbc 0 2024-02-15T18:27:17.050 INFO:tasks.workunit.client.1.smithi191.stdout:7/841: write d0/db/d65/d11/d88/f101 [328512,56034] 0 2024-02-15T18:27:17.052 INFO:tasks.workunit.client.1.smithi191.stdout:7/842: dread d0/db/d65/d11/d5a/d66/f53 [0,4194304] 0 2024-02-15T18:27:17.056 INFO:tasks.workunit.client.1.smithi191.stdout:2/862: unlink df/d12/d19/d1a/d35/d47/d4b/d7a/ddb/fe0 0 2024-02-15T18:27:17.057 INFO:tasks.workunit.client.1.smithi191.stdout:6/839: truncate d8/fab 2235986 0 2024-02-15T18:27:17.059 INFO:tasks.workunit.client.1.smithi191.stdout:7/843: rmdir d0/db/d65/d11/d88/d9d/dd2 39 2024-02-15T18:27:17.063 INFO:tasks.workunit.client.1.smithi191.stdout:2/863: mkdir df/d12/d19/d1a/d35/d47/d4b/d7a/dce/d118 0 2024-02-15T18:27:17.063 INFO:tasks.workunit.client.1.smithi191.stdout:2/864: fsync df/d12/d19/d1a/d35/d47/d4b/d7a/fe1 0 2024-02-15T18:27:17.064 INFO:tasks.workunit.client.1.smithi191.stdout:2/865: dread df/d12/f55 [0,4194304] 0 2024-02-15T18:27:17.064 INFO:tasks.workunit.client.1.smithi191.stdout:2/866: truncate df/d12/d1f/f10d 746665 0 2024-02-15T18:27:17.065 INFO:tasks.workunit.client.1.smithi191.stdout:6/840: mknod d8/d9/c106 0 2024-02-15T18:27:17.067 INFO:tasks.workunit.client.1.smithi191.stdout:7/844: rename d0/db/d26/d5f/d77/f21 to d0/db/d65/d11/f133 0 2024-02-15T18:27:17.067 INFO:tasks.workunit.client.1.smithi191.stdout:2/867: dread df/d12/d1f/f26 [0,4194304] 0 2024-02-15T18:27:17.072 INFO:tasks.workunit.client.1.smithi191.stdout:2/868: dread df/d12/d19/d1a/d35/d5b/dbf/f10c [0,4194304] 0 2024-02-15T18:27:17.073 INFO:tasks.workunit.client.1.smithi191.stdout:7/845: dread d0/db/d26/d5f/d77/df/f1a [0,4194304] 0 2024-02-15T18:27:17.074 INFO:tasks.workunit.client.1.smithi191.stdout:7/846: chown d0/db/d65/d11/fc4 7013098 1 2024-02-15T18:27:17.076 INFO:tasks.workunit.client.1.smithi191.stdout:2/869: creat df/d12/d19/d1a/dfa/f119 x:0 0 0 2024-02-15T18:27:17.077 INFO:tasks.workunit.client.1.smithi191.stdout:2/870: write df/d12/d19/d1a/d35/f80 [755056,72813] 0 2024-02-15T18:27:17.079 INFO:tasks.workunit.client.1.smithi191.stdout:2/871: getdents df/d61 0 2024-02-15T18:27:17.084 INFO:tasks.workunit.client.1.smithi191.stdout:9/849: sync 2024-02-15T18:27:17.084 INFO:tasks.workunit.client.1.smithi191.stdout:5/549: sync 2024-02-15T18:27:17.084 INFO:tasks.workunit.client.1.smithi191.stdout:4/837: sync 2024-02-15T18:27:17.085 INFO:tasks.workunit.client.1.smithi191.stdout:4/838: write d2/d5/d7/f15 [265148,93477] 0 2024-02-15T18:27:17.086 INFO:tasks.workunit.client.1.smithi191.stdout:4/839: write d2/d5/d7/d2f/d41/f99 [4987127,95410] 0 2024-02-15T18:27:17.086 INFO:tasks.workunit.client.1.smithi191.stdout:4/840: dread - d2/d5/d7/d3e/de8/d108/f123 zero size 2024-02-15T18:27:17.086 INFO:tasks.workunit.client.1.smithi191.stdout:4/841: chown d2/d5/d7/d3e/d63/d8c/da4 19801 1 2024-02-15T18:27:17.086 INFO:tasks.workunit.client.1.smithi191.stdout:6/841: dwrite d8/d9/dac/db0/d95/d90/f96 [0,4194304] 0 2024-02-15T18:27:17.086 INFO:tasks.workunit.client.1.smithi191.stdout:2/872: mkdir df/d12/d32/d4d/d89/d11a 0 2024-02-15T18:27:17.086 INFO:tasks.workunit.client.1.smithi191.stdout:2/873: fdatasync df/d12/d19/d1a/f21 0 2024-02-15T18:27:17.088 INFO:tasks.workunit.client.1.smithi191.stdout:5/550: creat d0/d23/d41/d28/fb1 x:0 0 0 2024-02-15T18:27:17.090 INFO:tasks.workunit.client.1.smithi191.stdout:9/850: creat d0/d7/dc/d30/d4f/f116 x:0 0 0 2024-02-15T18:27:17.091 INFO:tasks.workunit.client.1.smithi191.stdout:4/842: creat d2/d5/d7/d90/f132 x:0 0 0 2024-02-15T18:27:17.091 INFO:tasks.workunit.client.1.smithi191.stdout:4/843: chown d2/d5/d7/l5d 6226040 1 2024-02-15T18:27:17.093 INFO:tasks.workunit.client.1.smithi191.stdout:6/842: symlink d8/d9/d4a/l107 0 2024-02-15T18:27:17.096 INFO:tasks.workunit.client.1.smithi191.stdout:2/874: symlink df/d12/d32/d4d/d89/d11a/l11b 0 2024-02-15T18:27:17.098 INFO:tasks.workunit.client.1.smithi191.stdout:9/851: symlink d0/d29/d48/d66/l117 0 2024-02-15T18:27:17.098 INFO:tasks.workunit.client.1.smithi191.stdout:9/852: chown d0/d29/f95 131260799 1 2024-02-15T18:27:17.099 INFO:tasks.workunit.client.1.smithi191.stdout:4/844: getdents d2/d5/d14/d47/d48 0 2024-02-15T18:27:17.191 INFO:tasks.workunit.client.1.smithi191.stdout:6/843: creat d8/d9/f108 x:0 0 0 2024-02-15T18:27:17.193 INFO:tasks.workunit.client.1.smithi191.stdout:2/875: rename df/d61/d94/ce4 to df/d12/d32/d4d/d89/c11c 0 2024-02-15T18:27:17.195 INFO:tasks.workunit.client.1.smithi191.stdout:9/853: mkdir d0/d7/dc/d30/d3b/d56/d115/d118 0 2024-02-15T18:27:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: cluster 2024-02-15T18:27:16.685067+0000 mgr.smithi042.kfoihb (mgr.24435) 53 : cluster [DBG] pgmap v37: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 116 MiB/s rd, 233 MiB/s wr, 392 op/s 2024-02-15T18:27:17.196 INFO:tasks.workunit.client.1.smithi191.stdout:4/845: getdents d2/d5/d7/d90/d52/db5 0 2024-02-15T18:27:17.198 INFO:tasks.workunit.client.1.smithi191.stdout:6/844: mkdir d8/d9/d2b/d57/d8e/d109 0 2024-02-15T18:27:17.199 INFO:tasks.workunit.client.1.smithi191.stdout:2/876: unlink df/d12/d19/d1a/dda/ff3 0 2024-02-15T18:27:17.199 INFO:tasks.workunit.client.1.smithi191.stdout:2/877: fdatasync df/d12/d1f/f34 0 2024-02-15T18:27:17.201 INFO:tasks.workunit.client.1.smithi191.stdout:9/854: truncate d0/d7/dc/f9b 429289 0 2024-02-15T18:27:17.204 INFO:tasks.workunit.client.1.smithi191.stdout:4/846: rmdir d2/d5/d7/d90/d52 39 2024-02-15T18:27:17.205 INFO:tasks.workunit.client.1.smithi191.stdout:6/845: symlink d8/d9/d4a/df6/d102/l10a 0 2024-02-15T18:27:17.208 INFO:tasks.workunit.client.1.smithi191.stdout:4/847: write d2/d5/d14/d47/d78/f6a [548657,37961] 0 2024-02-15T18:27:17.208 INFO:tasks.workunit.client.1.smithi191.stdout:4/848: readlink d2/d5/l11 0 2024-02-15T18:27:17.208 INFO:tasks.workunit.client.1.smithi191.stdout:4/849: fdatasync d2/d5/d14/fb8 0 2024-02-15T18:27:17.208 INFO:tasks.workunit.client.1.smithi191.stdout:4/850: fsync d2/d5/d7/d2f/d41/f73 0 2024-02-15T18:27:17.208 INFO:tasks.workunit.client.1.smithi191.stdout:4/851: fsync d2/d5/d7/d2f/d41/f98 0 2024-02-15T18:27:17.208 INFO:tasks.workunit.client.1.smithi191.stdout:4/852: fsync d2/d5/d7/d106/f109 0 2024-02-15T18:27:17.208 INFO:tasks.workunit.client.1.smithi191.stdout:4/853: stat d2/d5/d14/d44 0 2024-02-15T18:27:17.212 INFO:tasks.workunit.client.1.smithi191.stdout:2/878: mkdir df/d61/d9f/d11d 0 2024-02-15T18:27:17.213 INFO:tasks.workunit.client.1.smithi191.stdout:6/846: link d8/l3f d8/d9/dac/db0/d95/d9d/d6b/l10b 0 2024-02-15T18:27:17.215 INFO:tasks.workunit.client.1.smithi191.stdout:4/854: mknod d2/d5/d7/d3e/d63/d8c/dbb/ddc/c133 0 2024-02-15T18:27:17.216 INFO:tasks.workunit.client.1.smithi191.stdout:2/879: dread df/d12/d32/fc4 [0,4194304] 0 2024-02-15T18:27:17.218 INFO:tasks.workunit.client.1.smithi191.stdout:6/847: creat d8/d9/d2b/d84/f10c x:0 0 0 2024-02-15T18:27:17.220 INFO:tasks.workunit.client.1.smithi191.stdout:9/855: dwrite d0/d7/f19 [0,4194304] 0 2024-02-15T18:27:17.220 INFO:tasks.workunit.client.1.smithi191.stdout:4/855: creat d2/d5/d7/d2f/f134 x:0 0 0 2024-02-15T18:27:17.226 INFO:tasks.workunit.client.1.smithi191.stdout:2/880: rmdir df/d12/d19 39 2024-02-15T18:27:17.226 INFO:tasks.workunit.client.1.smithi191.stdout:2/881: stat df/d12/d1f/l69 0 2024-02-15T18:27:17.228 INFO:tasks.workunit.client.1.smithi191.stdout:6/848: creat d8/d9/d4a/d4c/d7f/f10d x:0 0 0 2024-02-15T18:27:17.228 INFO:tasks.workunit.client.1.smithi191.stdout:6/849: write d8/d9/dac/db0/d95/d9d/f1f [2450430,120657] 0 2024-02-15T18:27:17.230 INFO:tasks.workunit.client.1.smithi191.stdout:9/856: creat d0/d1d/d3f/d80/f119 x:0 0 0 2024-02-15T18:27:17.231 INFO:tasks.workunit.client.1.smithi191.stdout:4/856: rename d2/d5/d7/d90/d52/db5/ld2 to d2/d5/d14/d47/d78/d119/l135 0 2024-02-15T18:27:17.232 INFO:tasks.workunit.client.1.smithi191.stdout:2/882: symlink df/d12/d32/d4d/d89/l11e 0 2024-02-15T18:27:17.234 INFO:tasks.workunit.client.1.smithi191.stdout:6/850: creat d8/d9/f10e x:0 0 0 2024-02-15T18:27:17.234 INFO:tasks.workunit.client.1.smithi191.stdout:6/851: dread - d8/f101 zero size 2024-02-15T18:27:17.235 INFO:tasks.workunit.client.1.smithi191.stdout:9/857: creat d0/d7/dc/d32/d106/dce/f11a x:0 0 0 2024-02-15T18:27:17.236 INFO:tasks.workunit.client.1.smithi191.stdout:4/857: creat d2/d5/d7/d2f/d114/f136 x:0 0 0 2024-02-15T18:27:17.237 INFO:tasks.workunit.client.1.smithi191.stdout:2/883: mknod df/d12/d1f/c11f 0 2024-02-15T18:27:17.239 INFO:tasks.workunit.client.1.smithi191.stdout:6/852: creat d8/d9/dac/db0/d95/d9d/d6b/f10f x:0 0 0 2024-02-15T18:27:17.244 INFO:tasks.workunit.client.1.smithi191.stdout:9/858: creat d0/d7/dc/d32/de4/dec/f11b x:0 0 0 2024-02-15T18:27:17.244 INFO:tasks.workunit.client.1.smithi191.stdout:9/859: fdatasync d0/d7/dc/d8c/f1c 0 2024-02-15T18:27:17.245 INFO:tasks.workunit.client.1.smithi191.stdout:4/858: creat d2/d5/d7/f137 x:0 0 0 2024-02-15T18:27:17.246 INFO:tasks.workunit.client.1.smithi191.stdout:2/884: mknod df/d12/d19/d1a/d35/d47/da2/c120 0 2024-02-15T18:27:17.247 INFO:tasks.workunit.client.1.smithi191.stdout:6/853: mkdir d8/d9/dac/db0/d95/d9d/d6b/d76/d110 0 2024-02-15T18:27:17.249 INFO:tasks.workunit.client.1.smithi191.stdout:9/860: creat d0/d7/dc/d32/de4/dec/f11c x:0 0 0 2024-02-15T18:27:17.256 INFO:tasks.workunit.client.1.smithi191.stdout:2/885: creat df/d61/d9f/da3/dab/f121 x:0 0 0 2024-02-15T18:27:17.256 INFO:tasks.workunit.client.1.smithi191.stdout:2/886: dread - df/d12/d19/d1a/feb zero size 2024-02-15T18:27:17.260 INFO:tasks.workunit.client.1.smithi191.stdout:9/861: mknod d0/d7/dc/d30/d3b/d56/d115/d118/c11d 0 2024-02-15T18:27:17.263 INFO:tasks.workunit.client.1.smithi191.stdout:2/887: rename df/d12/d19/d1a/d35/d47/l96 to df/d12/d19/d1a/d35/d47/d4b/d7a/dce/l122 0 2024-02-15T18:27:17.265 INFO:tasks.workunit.client.1.smithi191.stdout:2/888: getdents df 0 2024-02-15T18:27:17.267 INFO:tasks.workunit.client.1.smithi191.stdout:2/889: creat df/d12/d1f/f123 x:0 0 0 2024-02-15T18:27:17.272 INFO:tasks.workunit.client.1.smithi191.stdout:2/890: rmdir df/d12/d19/d1a 39 2024-02-15T18:27:17.272 INFO:tasks.workunit.client.1.smithi191.stdout:2/891: dread - df/d12/d19/d1a/d35/d47/d4b/d7a/dcc/f114 zero size 2024-02-15T18:27:17.273 INFO:tasks.workunit.client.1.smithi191.stdout:6/854: dwrite d8/d9/dac/db0/d95/d9d/f1f [0,4194304] 0 2024-02-15T18:27:17.273 INFO:tasks.workunit.client.1.smithi191.stdout:9/862: dread d0/d29/f86 [0,4194304] 0 2024-02-15T18:27:17.275 INFO:tasks.workunit.client.1.smithi191.stdout:6/855: creat d8/d9/d4a/d4c/dc4/f111 x:0 0 0 2024-02-15T18:27:17.277 INFO:tasks.workunit.client.1.smithi191.stdout:6/856: symlink d8/d9/dac/db0/d95/d9d/d6b/d72/dfe/l112 0 2024-02-15T18:27:17.279 INFO:tasks.workunit.client.1.smithi191.stdout:4/859: dwrite d2/d5/d7/f6c [0,4194304] 0 2024-02-15T18:27:17.280 INFO:tasks.workunit.client.1.smithi191.stdout:4/860: mkdir d2/d5/d7/d90/da5/d138 0 2024-02-15T18:27:17.281 INFO:tasks.workunit.client.1.smithi191.stdout:4/861: rename d2/d5/d14/lf9 to d2/d5/d7/d90/d52/dcb/l139 0 2024-02-15T18:27:17.281 INFO:tasks.workunit.client.1.smithi191.stdout:4/862: chown d2/d5/d14/l3c 23416153 1 2024-02-15T18:27:17.285 INFO:tasks.workunit.client.1.smithi191.stdout:9/863: dread d0/f4 [0,4194304] 0 2024-02-15T18:27:17.287 INFO:tasks.workunit.client.1.smithi191.stdout:2/892: dread df/d61/d9f/da3/dab/fdd [0,4194304] 0 2024-02-15T18:27:17.289 INFO:tasks.workunit.client.1.smithi191.stdout:9/864: mknod d0/d7/dc/d30/d3b/d56/c11e 0 2024-02-15T18:27:17.294 INFO:tasks.workunit.client.1.smithi191.stdout:6/857: dread d8/d9/d4a/d4c/fc0 [0,4194304] 0 2024-02-15T18:27:17.302 INFO:tasks.workunit.client.1.smithi191.stdout:4/863: dwrite d2/d5/d7/f29 [0,4194304] 0 2024-02-15T18:27:17.304 INFO:tasks.workunit.client.1.smithi191.stdout:4/864: creat d2/d5/d14/d71/f13a x:0 0 0 2024-02-15T18:27:17.305 INFO:tasks.workunit.client.1.smithi191.stdout:4/865: fsync d2/d5/d7/d2f/d41/f99 0 2024-02-15T18:27:17.305 INFO:tasks.workunit.client.1.smithi191.stdout:4/866: dread d2/d5/d7/d106/f109 [0,4194304] 0 2024-02-15T18:27:17.305 INFO:tasks.workunit.client.1.smithi191.stdout:4/867: chown d2/d5/d14/d47/cc6 234 1 2024-02-15T18:27:17.307 INFO:tasks.workunit.client.1.smithi191.stdout:4/868: mknod d2/d5/d7/d2f/d41/c13b 0 2024-02-15T18:27:17.307 INFO:tasks.workunit.client.1.smithi191.stdout:4/869: truncate d2/d5/d7/d2f/f31 8950250 0 2024-02-15T18:27:17.307 INFO:tasks.workunit.client.1.smithi191.stdout:4/870: chown d2/d5/d7/d90/f11d 6 1 2024-02-15T18:27:17.307 INFO:tasks.workunit.client.1.smithi191.stdout:4/871: write d2/d5/d7/d90/f6f [4327430,126633] 0 2024-02-15T18:27:17.308 INFO:tasks.workunit.client.1.smithi191.stdout:4/872: truncate d2/d5/d14/d47/d78/f4f 764180 0 2024-02-15T18:27:17.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:16.685067+0000 mgr.smithi042.kfoihb (mgr.24435) 53 : cluster [DBG] pgmap v37: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 116 MiB/s rd, 233 MiB/s wr, 392 op/s 2024-02-15T18:27:17.309 INFO:tasks.workunit.client.1.smithi191.stdout:4/873: mkdir d2/d5/d7/d90/d52/dcb/d10c/d13c 0 2024-02-15T18:27:17.309 INFO:tasks.workunit.client.1.smithi191.stdout:4/874: write d2/d5/d7/d3e/d63/f7d [824215,86488] 0 2024-02-15T18:27:17.315 INFO:tasks.workunit.client.1.smithi191.stdout:2/893: dwrite df/d12/d32/fc4 [0,4194304] 0 2024-02-15T18:27:17.316 INFO:tasks.workunit.client.1.smithi191.stdout:6/858: dwrite d8/d9/d2b/d57/f60 [0,4194304] 0 2024-02-15T18:27:17.319 INFO:tasks.workunit.client.1.smithi191.stdout:9/865: dwrite d0/d7/dc/d17/d44/feb [0,4194304] 0 2024-02-15T18:27:17.325 INFO:tasks.workunit.client.1.smithi191.stdout:9/866: unlink d0/d1d/d3f/d80/f119 0 2024-02-15T18:27:17.330 INFO:tasks.workunit.client.1.smithi191.stdout:4/875: dwrite d2/d5/d14/d47/d78/f61 [4194304,4194304] 0 2024-02-15T18:27:17.330 INFO:tasks.workunit.client.1.smithi191.stdout:4/876: write d2/d5/d7/f80 [1373454,62534] 0 2024-02-15T18:27:17.330 INFO:tasks.workunit.client.1.smithi191.stdout:4/877: write d2/d5/d7/d2f/f125 [824382,125441] 0 2024-02-15T18:27:17.330 INFO:tasks.workunit.client.1.smithi191.stdout:9/867: mknod d0/d7/dc/d17/d33/c11f 0 2024-02-15T18:27:17.336 INFO:tasks.workunit.client.1.smithi191.stdout:4/878: creat d2/d5/d7/d2f/d41/f13d x:0 0 0 2024-02-15T18:27:17.353 INFO:tasks.workunit.client.1.smithi191.stdout:6/859: dwrite d8/d9/d2b/f2e [0,4194304] 0 2024-02-15T18:27:17.354 INFO:tasks.workunit.client.1.smithi191.stdout:9/868: dwrite d0/d29/d48/fc3 [0,4194304] 0 2024-02-15T18:27:17.355 INFO:tasks.workunit.client.1.smithi191.stdout:6/860: write d8/d9/d2b/d57/fa9 [161439,62711] 0 2024-02-15T18:27:17.357 INFO:tasks.workunit.client.1.smithi191.stdout:4/879: dread d2/d5/d7/f80 [0,4194304] 0 2024-02-15T18:27:17.358 INFO:tasks.workunit.client.1.smithi191.stdout:2/894: dwrite df/d12/f39 [0,4194304] 0 2024-02-15T18:27:17.358 INFO:tasks.workunit.client.1.smithi191.stdout:2/895: chown df/d12/d19/d1a/d35/d47/c63 6012 1 2024-02-15T18:27:17.369 INFO:tasks.workunit.client.1.smithi191.stdout:2/896: mknod df/d61/d9f/da3/dab/c124 0 2024-02-15T18:27:17.371 INFO:tasks.workunit.client.1.smithi191.stdout:2/897: creat df/d61/d9f/f125 x:0 0 0 2024-02-15T18:27:17.380 INFO:tasks.workunit.client.1.smithi191.stdout:9/869: dwrite d0/d7/dc/d17/d33/f43 [4194304,4194304] 0 2024-02-15T18:27:17.388 INFO:tasks.workunit.client.1.smithi191.stdout:4/880: dwrite d2/d5/d7/d2f/fc2 [0,4194304] 0 2024-02-15T18:27:17.390 INFO:tasks.workunit.client.1.smithi191.stdout:2/898: dwrite df/d12/d19/d1a/d35/d47/d4b/d7a/fe1 [0,4194304] 0 2024-02-15T18:27:17.390 INFO:tasks.workunit.client.1.smithi191.stdout:6/861: dwrite d8/d9/d2b/d57/d8e/d91/fa3 [0,4194304] 0 2024-02-15T18:27:17.392 INFO:tasks.workunit.client.1.smithi191.stdout:4/881: creat d2/d5/d14/d47/d78/f13e x:0 0 0 2024-02-15T18:27:17.393 INFO:tasks.workunit.client.1.smithi191.stdout:2/899: creat df/d61/d9f/d11d/f126 x:0 0 0 2024-02-15T18:27:17.394 INFO:tasks.workunit.client.1.smithi191.stdout:2/900: chown df/d12/d1f/lf0 28845971 1 2024-02-15T18:27:17.397 INFO:tasks.workunit.client.1.smithi191.stdout:6/862: creat d8/d9/d2b/d57/d8e/d109/f113 x:0 0 0 2024-02-15T18:27:17.397 INFO:tasks.workunit.client.1.smithi191.stdout:6/863: chown d8/d9/l39 8 1 2024-02-15T18:27:17.397 INFO:tasks.workunit.client.1.smithi191.stdout:4/882: creat d2/d5/d14/d71/dcd/ded/f13f x:0 0 0 2024-02-15T18:27:17.425 INFO:tasks.workunit.client.1.smithi191.stdout:4/883: creat d2/d5/d12c/f140 x:0 0 0 2024-02-15T18:27:17.426 INFO:tasks.workunit.client.1.smithi191.stdout:4/884: dread d2/d5/d7/d90/d52/db5/dc9/f131 [0,4194304] 0 2024-02-15T18:27:17.431 INFO:tasks.workunit.client.1.smithi191.stdout:0/615: sync 2024-02-15T18:27:17.433 INFO:tasks.workunit.client.1.smithi191.stdout:4/885: dread d2/d5/d7/d2f/fc7 [0,4194304] 0 2024-02-15T18:27:17.433 INFO:tasks.workunit.client.1.smithi191.stdout:4/886: stat d2/d5/d7/d90/d52/db5/dc9/f131 0 2024-02-15T18:27:17.433 INFO:tasks.workunit.client.1.smithi191.stdout:0/616: mknod d7/d4a/dc0/cd6 0 2024-02-15T18:27:17.436 INFO:tasks.workunit.client.1.smithi191.stdout:4/887: creat d2/d5/d7/d90/d10b/f141 x:0 0 0 2024-02-15T18:27:17.437 INFO:tasks.workunit.client.1.smithi191.stdout:4/888: write d2/d5/d7/d90/f6f [3017960,54965] 0 2024-02-15T18:27:17.437 INFO:tasks.workunit.client.1.smithi191.stdout:4/889: dread - d2/d5/fc1 zero size 2024-02-15T18:27:17.438 INFO:tasks.workunit.client.1.smithi191.stdout:4/890: write d2/d5/d7/d2f/fc7 [3215511,21864] 0 2024-02-15T18:27:17.440 INFO:tasks.workunit.client.1.smithi191.stdout:0/617: dread d7/da/d1a/d34/f52 [0,4194304] 0 2024-02-15T18:27:17.446 INFO:tasks.workunit.client.1.smithi191.stdout:4/891: dread d2/d5/d7/d3e/f77 [0,4194304] 0 2024-02-15T18:27:17.448 INFO:tasks.workunit.client.1.smithi191.stdout:4/892: write d2/d5/d7/f6c [2437028,63340] 0 2024-02-15T18:27:17.452 INFO:tasks.workunit.client.1.smithi191.stdout:4/893: dread d2/d5/d7/f67 [0,4194304] 0 2024-02-15T18:27:17.471 INFO:tasks.workunit.client.1.smithi191.stdout:4/894: dwrite d2/d5/d7/d90/d52/dcb/f128 [0,4194304] 0 2024-02-15T18:27:17.472 INFO:tasks.workunit.client.1.smithi191.stdout:4/895: fsync d2/f28 0 2024-02-15T18:27:17.472 INFO:tasks.workunit.client.1.smithi191.stdout:4/896: chown d2/d5/d14/l74 1 1 2024-02-15T18:27:17.474 INFO:tasks.workunit.client.1.smithi191.stdout:4/897: dread d2/f3 [0,4194304] 0 2024-02-15T18:27:17.474 INFO:tasks.workunit.client.1.smithi191.stdout:4/898: write d2/d5/d14/d47/d78/f68 [1822164,74495] 0 2024-02-15T18:27:17.474 INFO:tasks.workunit.client.1.smithi191.stdout:4/899: dread - d2/d5/fc1 zero size 2024-02-15T18:27:17.475 INFO:tasks.workunit.client.1.smithi191.stdout:4/900: rmdir d2/d5/d7/d90/d52 39 2024-02-15T18:27:17.475 INFO:tasks.workunit.client.1.smithi191.stdout:4/901: dread - d2/d5/f115 zero size 2024-02-15T18:27:17.481 INFO:tasks.workunit.client.1.smithi191.stdout:4/902: dread d2/d5/d14/d47/d78/f68 [0,4194304] 0 2024-02-15T18:27:17.483 INFO:tasks.workunit.client.1.smithi191.stdout:4/903: mknod d2/d5/d7/d90/d52/dcb/d10c/c142 0 2024-02-15T18:27:17.863 INFO:tasks.workunit.client.1.smithi191.stdout:1/926: sync 2024-02-15T18:27:17.863 INFO:tasks.workunit.client.1.smithi191.stdout:8/856: sync 2024-02-15T18:27:17.863 INFO:tasks.workunit.client.1.smithi191.stdout:1/927: read d0/d2f/d51/d10/fb4 [1196020,88199] 0 2024-02-15T18:27:17.864 INFO:tasks.workunit.client.1.smithi191.stdout:8/857: creat d3/d2c/d1a/d23/d2e/d64/dca/f126 x:0 0 0 2024-02-15T18:27:17.864 INFO:tasks.workunit.client.1.smithi191.stdout:1/928: mknod d0/d2f/c13d 0 2024-02-15T18:27:17.867 INFO:tasks.workunit.client.1.smithi191.stdout:8/858: unlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/ccb 0 2024-02-15T18:27:17.867 INFO:tasks.workunit.client.1.smithi191.stdout:8/859: fdatasync d3/d2c/d1a/f1b 0 2024-02-15T18:27:17.870 INFO:tasks.workunit.client.1.smithi191.stdout:1/929: rename d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/dc9/c123 to d0/d8/df/d10c/d129/c13e 0 2024-02-15T18:27:17.872 INFO:tasks.workunit.client.1.smithi191.stdout:8/860: creat d3/d2c/d1a/d23/d2e/d5b/d69/d6d/f127 x:0 0 0 2024-02-15T18:27:17.872 INFO:tasks.workunit.client.1.smithi191.stdout:8/861: chown d3/d2c/d1a/l6c 0 1 2024-02-15T18:27:17.876 INFO:tasks.workunit.client.1.smithi191.stdout:1/930: symlink d0/d2f/d5c/l13f 0 2024-02-15T18:27:17.878 INFO:tasks.workunit.client.1.smithi191.stdout:8/862: creat d3/d2c/d1a/d118/d119/f128 x:0 0 0 2024-02-15T18:27:17.881 INFO:tasks.workunit.client.1.smithi191.stdout:8/863: rename d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/fc7 to d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d115/f129 0 2024-02-15T18:27:17.883 INFO:tasks.workunit.client.1.smithi191.stdout:8/864: mknod d3/c12a 0 2024-02-15T18:27:17.883 INFO:tasks.workunit.client.1.smithi191.stdout:8/865: stat d3/d2c/d1a/d23/d2e/d64/le5 0 2024-02-15T18:27:17.889 INFO:tasks.workunit.client.1.smithi191.stdout:8/866: write d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/de8/f107 [835720,87620] 0 2024-02-15T18:27:17.904 INFO:tasks.workunit.client.1.smithi191.stdout:1/931: dwrite d0/d8/df/d12/f32 [4194304,4194304] 0 2024-02-15T18:27:17.931 INFO:tasks.workunit.client.1.smithi191.stdout:1/932: dwrite d0/d2f/d51/d5e/fd6 [0,4194304] 0 2024-02-15T18:27:17.931 INFO:tasks.workunit.client.1.smithi191.stdout:1/933: truncate d0/d8/df/d12/d33/d49/f107 98791 0 2024-02-15T18:27:17.931 INFO:tasks.workunit.client.1.smithi191.stdout:1/934: write d0/d8/df/d12/fda [747615,105201] 0 2024-02-15T18:27:17.938 INFO:tasks.workunit.client.1.smithi191.stdout:1/935: truncate d0/d2f/d51/d10/fb4 2412189 0 2024-02-15T18:27:17.946 INFO:tasks.workunit.client.1.smithi191.stdout:1/936: symlink d0/d2f/l140 0 2024-02-15T18:27:17.946 INFO:tasks.workunit.client.1.smithi191.stdout:1/937: fdatasync d0/d2f/d51/fe 0 2024-02-15T18:27:17.947 INFO:tasks.workunit.client.1.smithi191.stdout:1/938: getdents d0/d2f/d77 0 2024-02-15T18:27:17.948 INFO:tasks.workunit.client.1.smithi191.stdout:1/939: chown d0/d8/df/d12/d2a/d2b/l64 283749469 1 2024-02-15T18:27:17.955 INFO:tasks.workunit.client.1.smithi191.stdout:1/940: dread d0/d8/df/d12/d2a/f52 [0,4194304] 0 2024-02-15T18:27:17.956 INFO:tasks.workunit.client.1.smithi191.stdout:1/941: mknod d0/d8/df/d12/d2a/d2b/d78/d60/c141 0 2024-02-15T18:27:17.957 INFO:tasks.workunit.client.1.smithi191.stdout:1/942: truncate d0/d8/df/f85 1290737 0 2024-02-15T18:27:17.958 INFO:tasks.workunit.client.1.smithi191.stdout:1/943: symlink d0/d8/df/d41/l142 0 2024-02-15T18:27:17.960 INFO:tasks.workunit.client.1.smithi191.stdout:1/944: truncate d0/d8/df/d12/f23 1121940 0 2024-02-15T18:27:17.978 INFO:tasks.workunit.client.1.smithi191.stdout:1/945: dwrite d0/d8/df/d12/d2a/d2b/d78/d60/d109/f12b [0,4194304] 0 2024-02-15T18:27:17.978 INFO:tasks.workunit.client.1.smithi191.stdout:1/946: read d0/d8/df/d2e/f8d [727110,121822] 0 2024-02-15T18:27:17.981 INFO:tasks.workunit.client.1.smithi191.stdout:1/947: creat d0/d8/df/d90/f143 x:0 0 0 2024-02-15T18:27:17.984 INFO:tasks.workunit.client.1.smithi191.stdout:1/948: creat d0/d8/df/d12/d2a/d2b/d78/d60/d74/f144 x:0 0 0 2024-02-15T18:27:17.984 INFO:tasks.workunit.client.1.smithi191.stdout:1/949: readlink d0/d8/ldb 0 2024-02-15T18:27:17.987 INFO:tasks.workunit.client.1.smithi191.stdout:1/950: rename d0/d2f/c10d to d0/d8/df/d12/d2a/d2b/d78/d60/d109/d9a/c145 0 2024-02-15T18:27:18.137 INFO:tasks.workunit.client.1.smithi191.stdout:3/718: sync 2024-02-15T18:27:18.137 INFO:tasks.workunit.client.1.smithi191.stdout:7/847: sync 2024-02-15T18:27:18.138 INFO:tasks.workunit.client.1.smithi191.stdout:7/848: chown d0/db/d65/df1/f117 12298 1 2024-02-15T18:27:18.138 INFO:tasks.workunit.client.1.smithi191.stdout:3/719: write d6/d1b/d60/da2/dac/fcc [156290,33705] 0 2024-02-15T18:27:18.140 INFO:tasks.workunit.client.1.smithi191.stdout:7/849: link d0/db/d65/d11/fe9 d0/db/d26/d5f/d77/d35/d42/d104/d110/f134 0 2024-02-15T18:27:18.164 INFO:tasks.workunit.client.1.smithi191.stdout:3/720: dwrite d6/d3b/d42/f51 [0,4194304] 0 2024-02-15T18:27:18.249 INFO:tasks.workunit.client.1.smithi191.stdout:7/850: mknod d0/db/d65/d11/d5a/d66/d1e/c135 0 2024-02-15T18:27:18.270 INFO:tasks.workunit.client.1.smithi191.stdout:7/851: dwrite d0/db/d65/d11/d5a/d66/d1e/fa5 [4194304,4194304] 0 2024-02-15T18:27:18.349 INFO:tasks.workunit.client.1.smithi191.stdout:3/721: symlink d6/d1b/d60/da2/lf2 0 2024-02-15T18:27:18.349 INFO:tasks.workunit.client.1.smithi191.stdout:3/722: getdents d6/d27/d4d/d5a/d92 0 2024-02-15T18:27:18.349 INFO:tasks.workunit.client.1.smithi191.stdout:3/723: chown d6/c71 39669154 1 2024-02-15T18:27:18.349 INFO:tasks.workunit.client.1.smithi191.stdout:3/724: chown d6/d3b/d42/d49/dc7 7430 1 2024-02-15T18:27:18.365 INFO:tasks.workunit.client.1.smithi191.stdout:7/852: rename d0/db/d65/d11/fac to d0/db/d65/d11/f136 0 2024-02-15T18:27:18.366 INFO:tasks.workunit.client.1.smithi191.stdout:3/725: mkdir d6/d5d/df3 0 2024-02-15T18:27:18.386 INFO:tasks.workunit.client.1.smithi191.stdout:0/618: sync 2024-02-15T18:27:18.386 INFO:tasks.workunit.client.1.smithi191.stdout:2/901: sync 2024-02-15T18:27:18.387 INFO:tasks.workunit.client.1.smithi191.stdout:5/551: sync 2024-02-15T18:27:18.387 INFO:tasks.workunit.client.1.smithi191.stdout:2/902: symlink df/d12/d19/d1a/dfa/l127 0 2024-02-15T18:27:18.388 INFO:tasks.workunit.client.1.smithi191.stdout:9/870: sync 2024-02-15T18:27:18.388 INFO:tasks.workunit.client.1.smithi191.stdout:8/867: sync 2024-02-15T18:27:18.389 INFO:tasks.workunit.client.1.smithi191.stdout:4/904: sync 2024-02-15T18:27:18.389 INFO:tasks.workunit.client.1.smithi191.stdout:6/864: sync 2024-02-15T18:27:18.389 INFO:tasks.workunit.client.1.smithi191.stdout:1/951: sync 2024-02-15T18:27:18.390 INFO:tasks.workunit.client.1.smithi191.stdout:2/903: getdents df/d12/d32/d4d/d89/d11a 0 2024-02-15T18:27:18.390 INFO:tasks.workunit.client.1.smithi191.stdout:7/853: dwrite d0/db/d65/dc/d4b/f5b [0,4194304] 0 2024-02-15T18:27:18.392 INFO:tasks.workunit.client.1.smithi191.stdout:5/552: mkdir d0/d23/d41/d28/d31/d5a/db2 0 2024-02-15T18:27:18.392 INFO:tasks.workunit.client.1.smithi191.stdout:3/726: dwrite d6/d3b/d30/f5c [0,4194304] 0 2024-02-15T18:27:18.394 INFO:tasks.workunit.client.1.smithi191.stdout:0/619: dread d7/da/d1a/f90 [0,4194304] 0 2024-02-15T18:27:18.394 INFO:tasks.workunit.client.1.smithi191.stdout:0/620: write d7/da/d17/d31/f89 [1174192,42347] 0 2024-02-15T18:27:18.394 INFO:tasks.workunit.client.1.smithi191.stdout:0/621: fsync d7/da/fce 0 2024-02-15T18:27:18.394 INFO:tasks.workunit.client.1.smithi191.stdout:3/727: write d6/d27/d4d/da8/fc4 [2405636,23705] 0 2024-02-15T18:27:18.395 INFO:tasks.workunit.client.1.smithi191.stdout:8/868: write d3/d2c/d1a/d23/d2e/d5b/d69/d6d/fab [3439910,110038] 0 2024-02-15T18:27:18.397 INFO:tasks.workunit.client.1.smithi191.stdout:9/871: rename d0/d7/dc/d30/d3b/d56/fb5 to d0/d29/d48/d66/f120 0 2024-02-15T18:27:18.397 INFO:tasks.workunit.client.1.smithi191.stdout:6/865: truncate d8/fab 255923 0 2024-02-15T18:27:18.397 INFO:tasks.workunit.client.1.smithi191.stdout:1/952: mknod d0/d2f/d51/d5e/c146 0 2024-02-15T18:27:18.400 INFO:tasks.workunit.client.1.smithi191.stdout:2/904: creat df/d12/d19/d1a/d35/d47/d4b/f128 x:0 0 0 2024-02-15T18:27:18.401 INFO:tasks.workunit.client.1.smithi191.stdout:3/728: creat d6/d27/d4d/d5a/da4/ff4 x:0 0 0 2024-02-15T18:27:18.406 INFO:tasks.workunit.client.1.smithi191.stdout:3/729: unlink d6/d1b/d60/c6c 0 2024-02-15T18:27:18.408 INFO:tasks.workunit.client.1.smithi191.stdout:9/872: rename d0/d29/d48/d66/cd6 to d0/d7/dc/d17/d44/c121 0 2024-02-15T18:27:18.410 INFO:tasks.workunit.client.1.smithi191.stdout:3/730: mknod d6/d27/d4d/da8/cf5 0 2024-02-15T18:27:18.411 INFO:tasks.workunit.client.1.smithi191.stdout:9/873: truncate d0/d7/f110 2153958 0 2024-02-15T18:27:18.411 INFO:tasks.workunit.client.1.smithi191.stdout:6/866: dread d8/d9/d2b/f42 [0,4194304] 0 2024-02-15T18:27:18.413 INFO:tasks.workunit.client.1.smithi191.stdout:6/867: dread d8/d9/d4a/d4c/d7f/fd8 [0,4194304] 0 2024-02-15T18:27:18.415 INFO:tasks.workunit.client.1.smithi191.stdout:4/905: rename d2/d5/d7/d2f/d41/f8f to d2/d5/d14/d71/dcd/ded/f143 0 2024-02-15T18:27:18.416 INFO:tasks.workunit.client.1.smithi191.stdout:4/906: write d2/d5/d7/d3e/f49 [3776936,31582] 0 2024-02-15T18:27:18.421 INFO:tasks.workunit.client.1.smithi191.stdout:5/553: dwrite d0/d18/f42 [4194304,4194304] 0 2024-02-15T18:27:18.427 INFO:tasks.workunit.client.1.smithi191.stdout:6/868: dread d8/f30 [0,4194304] 0 2024-02-15T18:27:18.427 INFO:tasks.workunit.client.1.smithi191.stdout:6/869: chown d8/d9/dac/db0/fb7 1355 1 2024-02-15T18:27:18.427 INFO:tasks.workunit.client.1.smithi191.stdout:6/870: write d8/d9/d2b/f7b [496355,59154] 0 2024-02-15T18:27:18.429 INFO:tasks.workunit.client.1.smithi191.stdout:8/869: rename d3/d2c/d1a/d23/d3e/f68 to d3/d2c/d1a/d23/d2e/d47/f12b 0 2024-02-15T18:27:18.429 INFO:tasks.workunit.client.1.smithi191.stdout:8/870: fdatasync d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d115/f129 0 2024-02-15T18:27:18.429 INFO:tasks.workunit.client.1.smithi191.stdout:8/871: write d3/d2c/d1a/d23/d2e/d47/d5f/fe2 [165867,110181] 0 2024-02-15T18:27:18.429 INFO:tasks.workunit.client.1.smithi191.stdout:6/871: write d8/d9/d2b/f51 [3384906,123220] 0 2024-02-15T18:27:18.430 INFO:tasks.workunit.client.1.smithi191.stdout:7/854: write d0/db/f2e [960102,69094] 0 2024-02-15T18:27:18.430 INFO:tasks.workunit.client.1.smithi191.stdout:0/622: dwrite d7/da/d1a/d34/f11 [0,4194304] 0 2024-02-15T18:27:18.431 INFO:tasks.workunit.client.1.smithi191.stdout:2/905: dwrite df/d12/d19/d1a/f21 [0,4194304] 0 2024-02-15T18:27:18.432 INFO:tasks.workunit.client.1.smithi191.stdout:4/907: mkdir d2/d5/d7/d3e/de8/d144 0 2024-02-15T18:27:18.433 INFO:tasks.workunit.client.1.smithi191.stdout:3/731: dwrite d6/d3b/d42/d49/d54/f83 [0,4194304] 0 2024-02-15T18:27:18.435 INFO:tasks.workunit.client.1.smithi191.stdout:5/554: creat d0/d18/d5c/fb3 x:0 0 0 2024-02-15T18:27:18.438 INFO:tasks.workunit.client.1.smithi191.stdout:8/872: rmdir d3/d2c/d1a/d23/d2e/d5b 39 2024-02-15T18:27:18.438 INFO:tasks.workunit.client.1.smithi191.stdout:0/623: write d7/da/d1a/d2c/f4b [3918918,107456] 0 2024-02-15T18:27:18.439 INFO:tasks.workunit.client.1.smithi191.stdout:6/872: creat d8/d9/dac/f114 x:0 0 0 2024-02-15T18:27:18.439 INFO:tasks.workunit.client.1.smithi191.stdout:6/873: chown d8/d9/dac/db0/d95/d9d/f38 22357 1 2024-02-15T18:27:18.440 INFO:tasks.workunit.client.1.smithi191.stdout:6/874: write d8/d9/d4a/d4c/fc0 [2156426,5414] 0 2024-02-15T18:27:18.446 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:18 smithi191 systemd[1]: Stopping Ceph mon.smithi191 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:27:18.446 INFO:tasks.workunit.client.1.smithi191.stdout:4/908: rmdir d2/d5/d124 39 2024-02-15T18:27:18.450 INFO:tasks.workunit.client.1.smithi191.stdout:3/732: rmdir d6/d3b/d42/d49/dc7 39 2024-02-15T18:27:18.452 INFO:tasks.workunit.client.1.smithi191.stdout:5/555: creat d0/d18/d5c/d7c/d49/d9f/fb4 x:0 0 0 2024-02-15T18:27:18.453 INFO:tasks.workunit.client.1.smithi191.stdout:8/873: mknod d3/d2c/d1a/d23/d3e/c12c 0 2024-02-15T18:27:18.453 INFO:tasks.workunit.client.1.smithi191.stdout:8/874: write d3/d2c/d1a/f1b [8844463,122111] 0 2024-02-15T18:27:18.454 INFO:tasks.workunit.client.1.smithi191.stdout:7/855: dwrite d0/db/d26/d5f/d77/db9/f127 [0,4194304] 0 2024-02-15T18:27:18.454 INFO:tasks.workunit.client.1.smithi191.stdout:0/624: getdents d7/d4a/dc0 0 2024-02-15T18:27:18.455 INFO:tasks.workunit.client.1.smithi191.stdout:0/625: dread d7/d1b/f81 [0,4194304] 0 2024-02-15T18:27:18.455 INFO:tasks.workunit.client.1.smithi191.stdout:0/626: getdents d7/da/d17/d31/d5f/da9 0 2024-02-15T18:27:18.456 INFO:tasks.workunit.client.1.smithi191.stdout:6/875: mknod d8/d9/dac/db0/d95/d9d/c115 0 2024-02-15T18:27:18.456 INFO:tasks.workunit.client.1.smithi191.stdout:6/876: fdatasync d8/d9/d4a/d4c/d7f/fb3 0 2024-02-15T18:27:18.457 INFO:tasks.workunit.client.1.smithi191.stdout:0/627: write d7/da/d1a/d2c/f32 [106427,46087] 0 2024-02-15T18:27:18.458 INFO:tasks.workunit.client.1.smithi191.stdout:2/906: dwrite df/d12/d19/d1a/d35/d5b/dbf/f10c [0,4194304] 0 2024-02-15T18:27:18.458 INFO:tasks.workunit.client.1.smithi191.stdout:3/733: mkdir d6/d3b/d42/ddd/df6 0 2024-02-15T18:27:18.462 INFO:tasks.workunit.client.1.smithi191.stdout:9/874: dwrite d0/f4 [0,4194304] 0 2024-02-15T18:27:18.463 INFO:tasks.workunit.client.1.smithi191.stdout:8/875: truncate d3/d2c/d1a/d23/d2a/f35 1194479 0 2024-02-15T18:27:18.463 INFO:tasks.workunit.client.1.smithi191.stdout:8/876: readlink d3/d2c/d1a/l27 0 2024-02-15T18:27:18.466 INFO:tasks.workunit.client.1.smithi191.stdout:7/856: truncate d0/db/d65/f82 1843894 0 2024-02-15T18:27:18.474 INFO:tasks.workunit.client.1.smithi191.stdout:6/877: rmdir d8/d9/dac 39 2024-02-15T18:27:18.474 INFO:tasks.workunit.client.1.smithi191.stdout:6/878: dread - d8/d9/d4a/d4c/dc4/f111 zero size 2024-02-15T18:27:18.481 INFO:tasks.workunit.client.1.smithi191.stdout:7/857: read d0/db/ff3 [1551919,82387] 0 2024-02-15T18:27:18.481 INFO:tasks.workunit.client.1.smithi191.stdout:0/628: link d7/da/d17/l76 d7/da/d1a/d34/d27/ld7 0 2024-02-15T18:27:18.491 INFO:tasks.workunit.client.1.smithi191.stdout:0/629: dread d7/da/d1a/d2c/d4c/f9f [0,4194304] 0 2024-02-15T18:27:18.498 INFO:tasks.workunit.client.1.smithi191.stdout:2/907: mknod df/d61/d9f/da3/c129 0 2024-02-15T18:27:18.503 INFO:tasks.workunit.client.1.smithi191.stdout:3/734: truncate d6/d5d/d72/f97 3763952 0 2024-02-15T18:27:18.504 INFO:tasks.workunit.client.1.smithi191.stdout:5/556: mkdir d0/db5 0 2024-02-15T18:27:18.513 INFO:tasks.workunit.client.1.smithi191.stdout:9/875: getdents d0/d29/d48/d66/d92 0 2024-02-15T18:27:18.513 INFO:tasks.workunit.client.1.smithi191.stdout:9/876: truncate d0/d7/dc/d17/d44/de6/fea 427496 0 2024-02-15T18:27:18.513 INFO:tasks.workunit.client.1.smithi191.stdout:9/877: truncate d0/fdc 4335040 0 2024-02-15T18:27:18.513 INFO:tasks.workunit.client.1.smithi191.stdout:9/878: truncate d0/d1d/d3f/d80/f100 1047424 0 2024-02-15T18:27:18.514 INFO:tasks.workunit.client.1.smithi191.stdout:8/877: creat d3/d2c/d1a/d23/d2e/d47/f12d x:0 0 0 2024-02-15T18:27:18.515 INFO:tasks.workunit.client.1.smithi191.stdout:6/879: creat d8/d9/d2b/d57/d8e/d91/f116 x:0 0 0 2024-02-15T18:27:18.517 INFO:tasks.workunit.client.1.smithi191.stdout:3/735: dwrite d6/d27/d4d/d5a/fea [0,4194304] 0 2024-02-15T18:27:18.517 INFO:tasks.workunit.client.1.smithi191.stdout:3/736: read d6/d3b/d42/f4a [3503038,28834] 0 2024-02-15T18:27:18.518 INFO:tasks.workunit.client.1.smithi191.stdout:3/737: fdatasync d6/d3b/d42/dbb/fd4 0 2024-02-15T18:27:18.518 INFO:tasks.workunit.client.1.smithi191.stdout:3/738: fsync d6/d27/d4d/d5a/fe1 0 2024-02-15T18:27:18.519 INFO:tasks.workunit.client.1.smithi191.stdout:7/858: creat d0/db/d65/f137 x:0 0 0 2024-02-15T18:27:18.525 INFO:tasks.workunit.client.1.smithi191.stdout:6/880: dread d8/d9/d2b/d57/fa9 [0,4194304] 0 2024-02-15T18:27:18.525 INFO:tasks.workunit.client.1.smithi191.stdout:6/881: read d8/d9/f86 [708731,38925] 0 2024-02-15T18:27:18.525 INFO:tasks.workunit.client.1.smithi191.stdout:6/882: read d8/d9/f46 [876829,46603] 0 2024-02-15T18:27:18.526 INFO:tasks.workunit.client.1.smithi191.stdout:6/883: read d8/d9/dac/db0/d95/d9d/f1d [3585975,73363] 0 2024-02-15T18:27:18.528 INFO:tasks.workunit.client.1.smithi191.stdout:2/908: rename df/d12/d1f to df/d12/d32/d4d/d89/d11a/d12a 0 2024-02-15T18:27:18.532 INFO:tasks.workunit.client.1.smithi191.stdout:8/878: truncate d3/d5e/fd1 229753 0 2024-02-15T18:27:18.533 INFO:tasks.workunit.client.1.smithi191.stdout:8/879: dread - d3/d2c/d1a/d23/d2e/d47/fe6 zero size 2024-02-15T18:27:18.535 INFO:tasks.workunit.client.1.smithi191.stdout:7/859: dwrite d0/f99 [0,4194304] 0 2024-02-15T18:27:18.554 INFO:tasks.workunit.client.1.smithi191.stdout:6/884: rmdir d8 39 2024-02-15T18:27:18.556 INFO:tasks.workunit.client.1.smithi191.stdout:2/909: symlink df/l12b 0 2024-02-15T18:27:18.560 INFO:tasks.workunit.client.1.smithi191.stdout:5/557: mknod d0/d23/d41/d28/d31/cb6 0 2024-02-15T18:27:18.563 INFO:tasks.workunit.client.1.smithi191.stdout:8/880: link d3/d2c/d1a/d23/d2e/de9/ffc d3/d2c/d1a/d36/d9b/f12e 0 2024-02-15T18:27:18.569 INFO:tasks.workunit.client.1.smithi191.stdout:7/860: truncate d0/db/d65/d11/d88/f101 42415 0 2024-02-15T18:27:18.569 INFO:tasks.workunit.client.1.smithi191.stdout:7/861: getdents d0/db/d26/d5f/d77/d35/d113 0 2024-02-15T18:27:18.574 INFO:tasks.workunit.client.1.smithi191.stdout:6/885: creat d8/d9/d2b/d84/ddd/f117 x:0 0 0 2024-02-15T18:27:18.576 INFO:tasks.workunit.client.1.smithi191.stdout:2/910: mknod df/d12/d19/d1a/d35/d47/d4b/d7a/dcc/c12c 0 2024-02-15T18:27:18.576 INFO:tasks.workunit.client.1.smithi191.stdout:2/911: read df/d12/d19/d1a/d35/d5b/f66 [36664,45962] 0 2024-02-15T18:27:18.577 INFO:tasks.workunit.client.1.smithi191.stdout:5/558: unlink d0/d18/d5c/d7c/f86 0 2024-02-15T18:27:18.579 INFO:tasks.workunit.client.1.smithi191.stdout:8/881: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/f12f x:0 0 0 2024-02-15T18:27:18.582 INFO:tasks.workunit.client.1.smithi191.stdout:7/862: rename d0/db/d65/d11/d5a/d66/d1e/ce7 to d0/db/d65/c138 0 2024-02-15T18:27:18.582 INFO:tasks.workunit.client.1.smithi191.stdout:7/863: readlink d0/db/d26/lab 0 2024-02-15T18:27:18.583 INFO:tasks.workunit.client.1.smithi191.stdout:6/886: creat d8/d9/d4a/d4c/f118 x:0 0 0 2024-02-15T18:27:18.587 INFO:tasks.workunit.client.1.smithi191.stdout:2/912: symlink df/d12/d32/d4d/d89/d110/l12d 0 2024-02-15T18:27:18.588 INFO:tasks.workunit.client.1.smithi191.stdout:5/559: creat d0/d23/d41/d28/d31/fb7 x:0 0 0 2024-02-15T18:27:18.589 INFO:tasks.workunit.client.1.smithi191.stdout:5/560: chown d0/d23/d41/f9a 494668 1 2024-02-15T18:27:18.589 INFO:tasks.workunit.client.1.smithi191.stdout:5/561: fsync d0/d23/d41/d28/d31/d5a/d61/f95 0 2024-02-15T18:27:18.589 INFO:tasks.workunit.client.1.smithi191.stdout:8/882: unlink d3/d2c/d1a/f81 0 2024-02-15T18:27:18.593 INFO:tasks.workunit.client.1.smithi191.stdout:7/864: rename d0/db/d26/d5f/d77/d35/d42/cad to d0/db/d65/d11/c139 0 2024-02-15T18:27:18.593 INFO:tasks.workunit.client.1.smithi191.stdout:7/865: write d0/db/d26/d5f/d77/d35/dc7/ff7 [815004,38177] 0 2024-02-15T18:27:18.594 INFO:tasks.workunit.client.1.smithi191.stdout:2/913: mknod df/d12/d19/d1a/d52/df2/c12e 0 2024-02-15T18:27:18.604 INFO:tasks.workunit.client.1.smithi191.stdout:5/562: rename d0/d23/d41/d28/d31/d5a/fa2 to d0/d23/d41/d28/d31/fb8 0 2024-02-15T18:27:18.606 INFO:tasks.workunit.client.1.smithi191.stdout:8/883: rename d3/d2c/d1a/la6 to d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/l130 0 2024-02-15T18:27:18.606 INFO:tasks.workunit.client.1.smithi191.stdout:8/884: write d3/d5e/f79 [2221372,113160] 0 2024-02-15T18:27:18.606 INFO:tasks.workunit.client.1.smithi191.stdout:8/885: readlink d3/d2c/d1a/d23/d2e/d47/lea 0 2024-02-15T18:27:18.606 INFO:tasks.workunit.client.1.smithi191.stdout:8/886: stat d3/faa 0 2024-02-15T18:27:18.607 INFO:tasks.workunit.client.1.smithi191.stdout:6/887: dwrite d8/d9/dac/db0/d95/d9d/f7a [0,4194304] 0 2024-02-15T18:27:18.608 INFO:tasks.workunit.client.1.smithi191.stdout:5/563: link d0/d23/d41/d28/d65/d85/d52/f40 d0/d23/d41/d28/d65/d85/d52/fb9 0 2024-02-15T18:27:18.609 INFO:tasks.workunit.client.1.smithi191.stdout:8/887: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/c131 0 2024-02-15T18:27:18.612 INFO:tasks.workunit.client.1.smithi191.stdout:5/564: rename d0/d23/d41/d28/d31/d5a/d61/c8d to d0/d23/d41/d28/d31/d5a/d61/cba 0 2024-02-15T18:27:18.612 INFO:tasks.workunit.client.1.smithi191.stdout:5/565: dread - d0/d23/d41/d28/d31/d5a/fa6 zero size 2024-02-15T18:27:18.612 INFO:tasks.workunit.client.1.smithi191.stdout:5/566: chown d0/d23/d41/d28/d31/f33 82 1 2024-02-15T18:27:18.612 INFO:tasks.workunit.client.1.smithi191.stdout:5/567: dread - d0/d23/d41/d28/d31/d5a/fa0 zero size 2024-02-15T18:27:18.614 INFO:tasks.workunit.client.1.smithi191.stdout:8/888: rename d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/da0/d112 to d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d132 0 2024-02-15T18:27:18.618 INFO:tasks.workunit.client.1.smithi191.stdout:2/914: dwrite df/d12/d32/d4d/d89/ff6 [0,4194304] 0 2024-02-15T18:27:18.618 INFO:tasks.workunit.client.1.smithi191.stdout:8/889: write d3/d2c/d1a/d23/d3e/fd6 [6267681,102207] 0 2024-02-15T18:27:18.619 INFO:tasks.workunit.client.1.smithi191.stdout:7/866: dwrite d0/db/d26/d5f/d77/df/f8e [0,4194304] 0 2024-02-15T18:27:18.621 INFO:tasks.workunit.client.1.smithi191.stdout:2/915: mknod df/d12/d19/d1a/d35/d5b/c12f 0 2024-02-15T18:27:18.621 INFO:tasks.workunit.client.1.smithi191.stdout:2/916: fdatasync df/d12/d19/d1a/d35/d47/d4b/d7a/dcc/f114 0 2024-02-15T18:27:18.621 INFO:tasks.workunit.client.1.smithi191.stdout:2/917: readlink df/d12/d19/d1a/d35/d47/d4b/d7a/dce/l122 0 2024-02-15T18:27:18.622 INFO:tasks.workunit.client.1.smithi191.stdout:8/890: creat d3/d2c/d1a/d23/d2a/f133 x:0 0 0 2024-02-15T18:27:18.622 INFO:tasks.workunit.client.1.smithi191.stdout:8/891: read - d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/fda zero size 2024-02-15T18:27:18.629 INFO:tasks.workunit.client.1.smithi191.stdout:2/918: mkdir df/d12/db4/d130 0 2024-02-15T18:27:18.632 INFO:tasks.workunit.client.1.smithi191.stdout:8/892: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d115/f134 x:0 0 0 2024-02-15T18:27:18.633 INFO:tasks.workunit.client.1.smithi191.stdout:5/568: rmdir d0 39 2024-02-15T18:27:18.636 INFO:tasks.workunit.client.1.smithi191.stdout:2/919: mknod df/d12/d32/d4d/d89/d11a/d12a/de6/d113/c131 0 2024-02-15T18:27:18.637 INFO:tasks.workunit.client.1.smithi191.stdout:2/920: read df/d12/d19/d1a/d52/fc0 [422300,77358] 0 2024-02-15T18:27:18.637 INFO:tasks.workunit.client.1.smithi191.stdout:6/888: dwrite d8/d9/d2b/d81/df1/f5e [0,4194304] 0 2024-02-15T18:27:18.638 INFO:tasks.workunit.client.1.smithi191.stdout:7/867: dwrite d0/db/d65/d11/d88/fec [0,4194304] 0 2024-02-15T18:27:18.639 INFO:tasks.workunit.client.1.smithi191.stdout:7/868: write d0/db/d65/d11/d57/fd9 [1042088,97576] 0 2024-02-15T18:27:18.639 INFO:tasks.workunit.client.1.smithi191.stdout:5/569: rename d0/d18/d5c/f91 to d0/d18/fbb 0 2024-02-15T18:27:18.639 INFO:tasks.workunit.client.1.smithi191.stdout:5/570: readlink d0/d18/l1c 0 2024-02-15T18:27:18.639 INFO:tasks.workunit.client.1.smithi191.stdout:5/571: stat d0/d18/d5c/f63 0 2024-02-15T18:27:18.640 INFO:tasks.workunit.client.1.smithi191.stdout:2/921: symlink df/d12/d19/d1a/d35/l132 0 2024-02-15T18:27:18.644 INFO:tasks.workunit.client.1.smithi191.stdout:6/889: mknod d8/d9/d4a/d4c/c119 0 2024-02-15T18:27:18.647 INFO:tasks.workunit.client.1.smithi191.stdout:5/572: creat d0/d23/d41/d28/d31/d5a/d61/fbc x:0 0 0 2024-02-15T18:27:18.648 INFO:tasks.workunit.client.1.smithi191.stdout:2/922: link df/c8b df/d12/d1e/c133 0 2024-02-15T18:27:18.648 INFO:tasks.workunit.client.1.smithi191.stdout:2/923: write df/d12/d19/d1a/d52/f112 [980655,23152] 0 2024-02-15T18:27:18.648 INFO:tasks.workunit.client.1.smithi191.stdout:2/924: write df/d12/d32/d4d/d89/d11a/d12a/f10d [859704,72373] 0 2024-02-15T18:27:18.653 INFO:tasks.workunit.client.1.smithi191.stdout:8/893: dwrite d3/f18 [0,4194304] 0 2024-02-15T18:27:18.653 INFO:tasks.workunit.client.1.smithi191.stdout:8/894: write d3/d2c/d1a/d23/d2e/d64/dca/f126 [290007,115010] 0 2024-02-15T18:27:18.654 INFO:tasks.workunit.client.1.smithi191.stdout:8/895: write d3/d2c/f75 [572272,87139] 0 2024-02-15T18:27:18.654 INFO:tasks.workunit.client.1.smithi191.stdout:8/896: chown d3/fc 1002049 1 2024-02-15T18:27:18.656 INFO:tasks.workunit.client.1.smithi191.stdout:5/573: mkdir d0/d18/d5c/d7c/d49/d9f/dbd 0 2024-02-15T18:27:18.661 INFO:tasks.workunit.client.1.smithi191.stdout:6/890: dwrite d8/d9/d2b/d57/d8e/d109/f113 [0,4194304] 0 2024-02-15T18:27:18.663 INFO:tasks.workunit.client.1.smithi191.stdout:8/897: creat d3/d2c/d1a/d23/d3e/db2/f135 x:0 0 0 2024-02-15T18:27:18.664 INFO:tasks.workunit.client.1.smithi191.stdout:5/574: creat d0/d23/d41/d28/d65/d85/fbe x:0 0 0 2024-02-15T18:27:18.666 INFO:tasks.workunit.client.1.smithi191.stdout:2/925: dwrite df/d12/d19/d1a/fdf [0,4194304] 0 2024-02-15T18:27:18.666 INFO:tasks.workunit.client.1.smithi191.stdout:2/926: fsync df/d12/d32/d4d/d89/d11a/d12a/d42/db5/fe8 0 2024-02-15T18:27:18.666 INFO:tasks.workunit.client.1.smithi191.stdout:2/927: stat df/d12/d19/d1a/d35/d5b/lf5 0 2024-02-15T18:27:18.667 INFO:tasks.workunit.client.1.smithi191.stdout:2/928: write df/d12/d32/d4d/d89/d11a/d12a/f10d [1742833,88112] 0 2024-02-15T18:27:18.667 INFO:tasks.workunit.client.1.smithi191.stdout:2/929: chown df/d12/d32/d4d/f50 153445 1 2024-02-15T18:27:18.667 INFO:tasks.workunit.client.1.smithi191.stdout:6/891: rename d8/l16 to d8/d9/d2b/d57/d8e/l11a 0 2024-02-15T18:27:18.668 INFO:tasks.workunit.client.1.smithi191.stdout:8/898: creat d3/d2c/d1a/d23/d2e/d64/f136 x:0 0 0 2024-02-15T18:27:18.669 INFO:tasks.workunit.client.1.smithi191.stdout:6/892: dread d8/d9/d2b/d57/d8e/faf [0,4194304] 0 2024-02-15T18:27:18.669 INFO:tasks.workunit.client.1.smithi191.stdout:6/893: write d8/d9/dac/db0/d95/d9d/d6b/d98/fbf [783683,123126] 0 2024-02-15T18:27:18.677 INFO:tasks.workunit.client.1.smithi191.stdout:8/899: mkdir d3/d2c/d1a/d118/d137 0 2024-02-15T18:27:18.680 INFO:tasks.workunit.client.1.smithi191.stdout:6/894: rmdir d8/d9/dac/db0 39 2024-02-15T18:27:18.680 INFO:tasks.workunit.client.1.smithi191.stdout:6/895: truncate d8/d9/d2b/fe3 1688058 0 2024-02-15T18:27:18.681 INFO:tasks.workunit.client.1.smithi191.stdout:6/896: truncate d8/d9/d2b/d84/f10c 401464 0 2024-02-15T18:27:18.683 INFO:tasks.workunit.client.1.smithi191.stdout:6/897: dread d8/d9/d2b/d57/d8e/d91/fcb [0,4194304] 0 2024-02-15T18:27:18.683 INFO:tasks.workunit.client.1.smithi191.stdout:6/898: fsync d8/d9/f86 0 2024-02-15T18:27:18.688 INFO:tasks.workunit.client.1.smithi191.stdout:8/900: dread d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/de8/f107 [0,4194304] 0 2024-02-15T18:27:18.692 INFO:tasks.workunit.client.1.smithi191.stdout:5/575: dwrite d0/d18/d5c/f6d [0,4194304] 0 2024-02-15T18:27:18.692 INFO:tasks.workunit.client.1.smithi191.stdout:5/576: chown d0/d18/d5c/d7c/d49/d9f/la4 97377 1 2024-02-15T18:27:18.696 INFO:tasks.workunit.client.1.smithi191.stdout:8/901: dread d3/d2c/d1a/d23/d2e/d64/f8d [0,4194304] 0 2024-02-15T18:27:18.704 INFO:tasks.workunit.client.1.smithi191.stdout:6/899: symlink d8/d9/d4a/d4c/dc4/l11b 0 2024-02-15T18:27:18.704 INFO:tasks.workunit.client.1.smithi191.stdout:6/900: dread - d8/d9/dac/db0/d95/d90/f105 zero size 2024-02-15T18:27:18.704 INFO:tasks.workunit.client.1.smithi191.stdout:2/930: truncate df/d12/d32/d4d/d89/d11a/d12a/d42/f43 2209 0 2024-02-15T18:27:18.708 INFO:tasks.workunit.client.1.smithi191.stdout:5/577: link d0/d23/d41/d28/l5b d0/d18/d5c/d7c/d49/d9f/lbf 0 2024-02-15T18:27:18.708 INFO:tasks.workunit.client.1.smithi191.stdout:5/578: chown d0/d23/d41/d28/d65/d85/c84 23474559 1 2024-02-15T18:27:18.710 INFO:tasks.workunit.client.1.smithi191.stdout:6/901: symlink d8/d9/dac/db0/d95/d9d/d6b/d76/l11c 0 2024-02-15T18:27:18.710 INFO:tasks.workunit.client.1.smithi191.stdout:2/931: unlink df/d12/d19/d1a/d35/d47/da2/l10f 0 2024-02-15T18:27:18.717 INFO:tasks.workunit.client.1.smithi191.stdout:5/579: rename d0/d23/d41/d28/d65 to d0/d18/d5c/d7c/d49/d9f/dc0 0 2024-02-15T18:27:18.721 INFO:tasks.workunit.client.1.smithi191.stdout:6/902: symlink d8/d9/d2b/d57/d8e/l11d 0 2024-02-15T18:27:18.723 INFO:tasks.workunit.client.1.smithi191.stdout:6/903: mknod d8/d9/dac/ddb/dd7/df4/c11e 0 2024-02-15T18:27:18.725 INFO:tasks.workunit.client.1.smithi191.stdout:8/902: dwrite d3/d2c/d1a/d23/d2e/d47/f88 [0,4194304] 0 2024-02-15T18:27:18.725 INFO:tasks.workunit.client.1.smithi191.stdout:6/904: getdents d8/d9/d2b/d57 0 2024-02-15T18:27:18.725 INFO:tasks.workunit.client.1.smithi191.stdout:8/903: read d3/d2c/f75 [2188810,27020] 0 2024-02-15T18:27:18.731 INFO:tasks.workunit.client.1.smithi191.stdout:6/905: dread d8/d9/dac/db0/fb6 [0,4194304] 0 2024-02-15T18:27:18.731 INFO:tasks.workunit.client.1.smithi191.stdout:6/906: chown d8/d9/f108 0 1 2024-02-15T18:27:18.734 INFO:tasks.workunit.client.1.smithi191.stdout:8/904: rmdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/dec 0 2024-02-15T18:27:18.734 INFO:tasks.workunit.client.1.smithi191.stdout:8/905: stat d3/d2c/d1a/d23/d3e/db2/dff/f110 0 2024-02-15T18:27:18.736 INFO:tasks.workunit.client.1.smithi191.stdout:6/907: truncate d8/d9/f79 1038072 0 2024-02-15T18:27:18.737 INFO:tasks.workunit.client.1.smithi191.stdout:8/906: truncate d3/d5e/f79 1474195 0 2024-02-15T18:27:18.737 INFO:tasks.workunit.client.1.smithi191.stdout:8/907: truncate d3/d2c/d1a/d23/d2a/f100 989332 0 2024-02-15T18:27:18.737 INFO:tasks.workunit.client.1.smithi191.stdout:8/908: read - d3/d2c/d1a/d23/d2a/f133 zero size 2024-02-15T18:27:18.737 INFO:tasks.workunit.client.1.smithi191.stdout:2/932: dwrite df/d12/d32/d4d/d89/d11a/d12a/f123 [0,4194304] 0 2024-02-15T18:27:18.737 INFO:tasks.workunit.client.1.smithi191.stdout:6/908: mknod d8/d9/d2b/d81/c11f 0 2024-02-15T18:27:18.742 INFO:tasks.workunit.client.1.smithi191.stdout:5/580: dwrite d0/d18/d5c/f63 [0,4194304] 0 2024-02-15T18:27:18.742 INFO:tasks.workunit.client.1.smithi191.stdout:5/581: chown d0/fa7 0 1 2024-02-15T18:27:18.754 INFO:tasks.workunit.client.1.smithi191.stdout:8/909: getdents d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/da0 0 2024-02-15T18:27:18.755 INFO:tasks.workunit.client.1.smithi191.stdout:5/582: dread d0/d23/d41/d28/d31/f47 [0,4194304] 0 2024-02-15T18:27:18.769 INFO:tasks.workunit.client.1.smithi191.stdout:2/933: getdents df/d12/d19/d1a/d52/df2 0 2024-02-15T18:27:18.769 INFO:tasks.workunit.client.1.smithi191.stdout:2/934: stat df/d12/d32/d4d/f50 0 2024-02-15T18:27:18.769 INFO:tasks.workunit.client.1.smithi191.stdout:2/935: chown df/d61/d94/cef 289316 1 2024-02-15T18:27:18.770 INFO:tasks.workunit.client.1.smithi191.stdout:6/909: creat d8/d9/d4a/d4c/da4/f120 x:0 0 0 2024-02-15T18:27:18.770 INFO:tasks.workunit.client.1.smithi191.stdout:6/910: fdatasync d8/d9/d4a/d4c/dc4/f111 0 2024-02-15T18:27:18.776 INFO:tasks.workunit.client.1.smithi191.stdout:5/583: mkdir d0/d23/d41/d28/dc1 0 2024-02-15T18:27:18.779 INFO:tasks.workunit.client.1.smithi191.stdout:8/910: dwrite d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/ff4 [0,4194304] 0 2024-02-15T18:27:18.781 INFO:tasks.workunit.client.1.smithi191.stdout:2/936: dread df/d12/fca [0,4194304] 0 2024-02-15T18:27:18.786 INFO:tasks.workunit.client.1.smithi191.stdout:5/584: mkdir d0/d23/d41/d28/d31/dc2 0 2024-02-15T18:27:18.786 INFO:tasks.workunit.client.1.smithi191.stdout:5/585: write d0/d18/d5c/d7c/f58 [4864255,45656] 0 2024-02-15T18:27:18.786 INFO:tasks.workunit.client.1.smithi191.stdout:5/586: fdatasync d0/d18/d5c/f71 0 2024-02-15T18:27:18.786 INFO:tasks.workunit.client.1.smithi191.stdout:5/587: fsync d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/fb0 0 2024-02-15T18:27:18.788 INFO:tasks.workunit.client.1.smithi191.stdout:5/588: rename d0/d18/l1c to d0/d23/d41/d28/d31/d5a/db2/lc3 0 2024-02-15T18:27:18.792 INFO:tasks.workunit.client.1.smithi191.stdout:5/589: dread d0/d23/d41/f87 [0,4194304] 0 2024-02-15T18:27:18.792 INFO:tasks.workunit.client.1.smithi191.stdout:5/590: truncate d0/d18/d5c/d7c/d49/d9f/dc0/d85/fbe 917028 0 2024-02-15T18:27:18.800 INFO:tasks.workunit.client.1.smithi191.stdout:5/591: rename d0/d23/d41/d28/d31/l57 to d0/d18/lc4 0 2024-02-15T18:27:18.814 INFO:tasks.workunit.client.1.smithi191.stdout:8/911: dwrite d3/d2c/d1a/f2d [8388608,4194304] 0 2024-02-15T18:27:18.814 INFO:tasks.workunit.client.1.smithi191.stdout:8/912: getdents d3/d2c/d1a/d23/d2e/d64/dca/d106 0 2024-02-15T18:27:18.816 INFO:tasks.workunit.client.1.smithi191.stdout:2/937: dwrite df/d12/d19/d1a/d35/d47/f51 [0,4194304] 0 2024-02-15T18:27:18.816 INFO:tasks.workunit.client.1.smithi191.stdout:2/938: write df/d12/d19/d1a/f5e [1885884,56317] 0 2024-02-15T18:27:18.816 INFO:tasks.workunit.client.1.smithi191.stdout:6/911: dwrite d8/d9/dac/db0/d95/d90/fbe [0,4194304] 0 2024-02-15T18:27:18.817 INFO:tasks.workunit.client.1.smithi191.stdout:8/913: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/de8/f138 x:0 0 0 2024-02-15T18:27:18.833 INFO:tasks.workunit.client.1.smithi191.stdout:6/912: creat d8/d9/dac/ddb/dd7/df4/f121 x:0 0 0 2024-02-15T18:27:18.836 INFO:tasks.workunit.client.1.smithi191.stdout:8/914: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/c139 0 2024-02-15T18:27:18.839 INFO:tasks.workunit.client.1.smithi191.stdout:6/913: creat d8/d9/dac/ddb/dd7/df4/f122 x:0 0 0 2024-02-15T18:27:18.840 INFO:tasks.workunit.client.1.smithi191.stdout:8/915: mknod d3/d5e/c13a 0 2024-02-15T18:27:18.844 INFO:tasks.workunit.client.1.smithi191.stdout:6/914: creat d8/d9/d2b/d57/f123 x:0 0 0 2024-02-15T18:27:18.846 INFO:tasks.workunit.client.1.smithi191.stdout:6/915: creat d8/d9/dac/ddb/dd7/df4/f124 x:0 0 0 2024-02-15T18:27:18.848 INFO:tasks.workunit.client.1.smithi191.stdout:6/916: mknod d8/d9/d2b/d57/c125 0 2024-02-15T18:27:18.848 INFO:tasks.workunit.client.1.smithi191.stdout:6/917: truncate d8/fab 407871 0 2024-02-15T18:27:18.849 INFO:tasks.workunit.client.1.smithi191.stdout:6/918: rmdir d8 39 2024-02-15T18:27:18.851 INFO:tasks.workunit.client.1.smithi191.stdout:8/916: dread d3/d2c/d1a/fb1 [0,4194304] 0 2024-02-15T18:27:18.852 INFO:tasks.workunit.client.1.smithi191.stdout:6/919: write d8/d9/d2b/d57/f93 [1145350,124204] 0 2024-02-15T18:27:18.858 INFO:tasks.workunit.client.1.smithi191.stdout:6/920: rename d8/l2a to d8/d9/d4a/d4c/da4/l126 0 2024-02-15T18:27:18.860 INFO:tasks.workunit.client.1.smithi191.stdout:6/921: symlink d8/d9/dac/db0/d95/d9d/d6b/d72/dfe/l127 0 2024-02-15T18:27:18.863 INFO:tasks.workunit.client.1.smithi191.stdout:6/922: unlink d8/d9/dac/db0/d95/d90/fbe 0 2024-02-15T18:27:18.863 INFO:tasks.workunit.client.1.smithi191.stdout:6/923: chown d8/d9/c5a 15694 1 2024-02-15T18:27:18.863 INFO:tasks.workunit.client.1.smithi191.stdout:6/924: chown d8/d9/dac/db0/d95/d9d/d6b/d72/c78 267337 1 2024-02-15T18:27:18.866 INFO:tasks.workunit.client.1.smithi191.stdout:6/925: symlink d8/d9/dac/db0/d95/d90/l128 0 2024-02-15T18:27:18.871 INFO:tasks.workunit.client.1.smithi191.stdout:6/926: creat d8/d9/d4a/d4c/da4/f129 x:0 0 0 2024-02-15T18:27:18.873 INFO:tasks.workunit.client.1.smithi191.stdout:6/927: creat d8/d9/d4a/d4c/dc4/f12a x:0 0 0 2024-02-15T18:27:18.873 INFO:tasks.workunit.client.1.smithi191.stdout:6/928: write d8/d9/d4a/d4c/da4/f120 [615449,99241] 0 2024-02-15T18:27:18.874 INFO:tasks.workunit.client.1.smithi191.stdout:8/917: dwrite d3/d2c/d1a/d118/d119/f128 [0,4194304] 0 2024-02-15T18:27:18.889 INFO:tasks.workunit.client.1.smithi191.stdout:8/918: write d3/d2c/f17 [4071728,117232] 0 2024-02-15T18:27:18.890 INFO:tasks.workunit.client.1.smithi191.stdout:8/919: rmdir d3/d2c/d1a/d23/d3e 39 2024-02-15T18:27:18.890 INFO:tasks.workunit.client.1.smithi191.stdout:8/920: fsync d3/d2c/d1a/d23/d2e/d47/d5f/fc5 0 2024-02-15T18:27:18.893 INFO:tasks.workunit.client.1.smithi191.stdout:8/921: creat d3/d2c/d1a/d23/d2e/d64/dca/d106/f13b x:0 0 0 2024-02-15T18:27:18.895 INFO:tasks.workunit.client.1.smithi191.stdout:8/922: creat d3/d2c/d1a/d23/d2e/d64/dca/d106/f13c x:0 0 0 2024-02-15T18:27:18.898 INFO:tasks.workunit.client.1.smithi191.stdout:8/923: rmdir d3/d2c/d1a/d23/d2e/d64/dbc/d109 39 2024-02-15T18:27:18.898 INFO:tasks.workunit.client.1.smithi191.stdout:8/924: chown d3/d2c/d1a/d118/d119 22243803 1 2024-02-15T18:27:18.898 INFO:tasks.workunit.client.1.smithi191.stdout:6/929: dwrite d8/d9/dac/db0/d95/d9d/d6b/d72/ffd [0,4194304] 0 2024-02-15T18:27:18.902 INFO:tasks.workunit.client.1.smithi191.stdout:8/925: link d3/d2c/d1a/d23/d2e/d47/d5f/d4f/l85 d3/d2c/d1a/d23/d2e/d64/dbc/d109/l13d 0 2024-02-15T18:27:18.906 INFO:tasks.workunit.client.1.smithi191.stdout:6/930: getdents d8/d9/dac/ddb/dd7/df4 0 2024-02-15T18:27:18.906 INFO:tasks.workunit.client.1.smithi191.stdout:6/931: fsync d8/d9/dac/db0/d95/d90/f105 0 2024-02-15T18:27:18.907 INFO:tasks.workunit.client.1.smithi191.stdout:8/926: mknod d3/d2c/d1a/d23/d2e/de9/c13e 0 2024-02-15T18:27:18.908 INFO:tasks.workunit.client.1.smithi191.stdout:8/927: stat d3/d2c/d1a/d23/d2e/d5b/l8b 0 2024-02-15T18:27:18.911 INFO:tasks.workunit.client.1.smithi191.stdout:6/932: stat d8/c48 0 2024-02-15T18:27:18.913 INFO:tasks.workunit.client.1.smithi191.stdout:8/928: mknod d3/d2c/d1a/d23/d2e/d5b/d69/c13f 0 2024-02-15T18:27:18.916 INFO:tasks.workunit.client.1.smithi191.stdout:6/933: rename d8/d9/f65 to d8/d9/dac/db0/d95/d9d/d6b/d76/f12b 0 2024-02-15T18:27:18.918 INFO:tasks.workunit.client.1.smithi191.stdout:8/929: truncate d3/d2c/d1a/d23/d2e/d64/f8d 3461227 0 2024-02-15T18:27:18.919 INFO:tasks.workunit.client.1.smithi191.stdout:8/930: write d3/d5e/f73 [4610086,57231] 0 2024-02-15T18:27:18.919 INFO:tasks.workunit.client.1.smithi191.stdout:6/934: mkdir d8/d9/d4a/d4c/dc4/d12c 0 2024-02-15T18:27:18.925 INFO:tasks.workunit.client.1.smithi191.stdout:6/935: rename d8/d9/dac/db0/d95/d9d/l1b to d8/d9/d4a/d4c/dc4/d12c/l12d 0 2024-02-15T18:27:18.930 INFO:tasks.workunit.client.1.smithi191.stdout:6/936: creat d8/f12e x:0 0 0 2024-02-15T18:27:18.930 INFO:tasks.workunit.client.1.smithi191.stdout:6/937: dread - d8/d9/d4a/d4c/dc4/ff7 zero size 2024-02-15T18:27:18.930 INFO:tasks.workunit.client.1.smithi191.stdout:6/938: rmdir d8/d9/dac/db0/d95 39 2024-02-15T18:27:18.931 INFO:tasks.workunit.client.1.smithi191.stdout:6/939: write d8/d9/d2b/d57/d8e/d91/f116 [406319,65464] 0 2024-02-15T18:27:18.933 INFO:tasks.workunit.client.1.smithi191.stdout:6/940: symlink d8/d9/dac/ddb/dd7/df4/l12f 0 2024-02-15T18:27:18.933 INFO:tasks.workunit.client.1.smithi191.stdout:6/941: chown d8/d9/dac/db0/fca 15703 1 2024-02-15T18:27:18.933 INFO:tasks.workunit.client.1.smithi191.stdout:6/942: write d8/d9/d2b/d57/f93 [1492096,15617] 0 2024-02-15T18:27:18.934 INFO:tasks.workunit.client.1.smithi191.stdout:6/943: mkdir d8/d130 0 2024-02-15T18:27:18.935 INFO:tasks.workunit.client.1.smithi191.stdout:6/944: mkdir d8/d9/d2b/d57/d103/d131 0 2024-02-15T18:27:18.937 INFO:tasks.workunit.client.1.smithi191.stdout:8/931: dwrite d3/fc [0,4194304] 0 2024-02-15T18:27:18.942 INFO:tasks.workunit.client.1.smithi191.stdout:8/932: creat d3/d2c/d1a/d10d/f140 x:0 0 0 2024-02-15T18:27:18.956 INFO:tasks.workunit.client.1.smithi191.stdout:8/933: dread d3/d2c/d1a/d10b/f11c [0,4194304] 0 2024-02-15T18:27:18.959 INFO:tasks.workunit.client.1.smithi191.stdout:6/945: dwrite d8/d9/dac/db0/d95/d9d/d6b/f10f [0,4194304] 0 2024-02-15T18:27:18.959 INFO:tasks.workunit.client.1.smithi191.stdout:6/946: stat d8/d9/d4a/d4c/d7f/f10d 0 2024-02-15T18:27:18.959 INFO:tasks.workunit.client.1.smithi191.stdout:6/947: chown d8/d9/d4a/lba 4 1 2024-02-15T18:27:18.971 INFO:tasks.workunit.client.1.smithi191.stdout:8/934: dwrite d3/d2c/d1a/fce [0,4194304] 0 2024-02-15T18:27:18.995 INFO:tasks.workunit.client.1.smithi191.stdout:8/935: dwrite d3/d2c/d1a/d23/d2e/d47/d5f/fc5 [0,4194304] 0 2024-02-15T18:27:18.995 INFO:tasks.workunit.client.1.smithi191.stdout:8/936: stat d3/d2c/d1a/d23/d2e/d64/fc4 0 2024-02-15T18:27:18.996 INFO:tasks.workunit.client.1.smithi191.stdout:8/937: read d3/d2c/d1a/d118/d119/f128 [2414511,94451] 0 2024-02-15T18:27:19.002 INFO:tasks.workunit.client.1.smithi191.stdout:8/938: symlink d3/d2c/d1a/d23/d3e/db2/l141 0 2024-02-15T18:27:19.002 INFO:tasks.workunit.client.1.smithi191.stdout:8/939: chown d3/f18 4971236 1 2024-02-15T18:27:19.002 INFO:tasks.workunit.client.1.smithi191.stdout:8/940: write d3/d2c/d1a/d23/d2e/d64/dca/fed [844389,59633] 0 2024-02-15T18:27:19.003 INFO:tasks.workunit.client.1.smithi191.stdout:8/941: mkdir d3/d2c/d1a/d23/d2e/d47/d142 0 2024-02-15T18:27:19.003 INFO:tasks.workunit.client.1.smithi191.stdout:8/942: dread - d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/f11f zero size 2024-02-15T18:27:19.008 INFO:tasks.workunit.client.1.smithi191.stdout:8/943: symlink d3/d2c/d1a/d23/d2e/d64/dbc/d109/l143 0 2024-02-15T18:27:19.012 INFO:tasks.workunit.client.1.smithi191.stdout:8/944: truncate d3/d2c/d1a/d23/d2e/d47/f63 1228463 0 2024-02-15T18:27:19.015 INFO:tasks.workunit.client.1.smithi191.stdout:8/945: creat d3/d2c/d1a/d23/d2e/d47/d5f/d11e/f144 x:0 0 0 2024-02-15T18:27:19.015 INFO:tasks.workunit.client.1.smithi191.stdout:8/946: write d3/d2c/d1a/d23/d2e/d64/f76 [22382,4756] 0 2024-02-15T18:27:19.018 INFO:tasks.workunit.client.1.smithi191.stdout:8/947: rename d3/d2c/d1a/d23/d2e/d5b/l8b to d3/d2c/d1a/d23/d3e/db2/l145 0 2024-02-15T18:27:19.018 INFO:tasks.workunit.client.1.smithi191.stdout:8/948: fsync d3/d2c/d1a/d10d/f140 0 2024-02-15T18:27:19.020 INFO:tasks.workunit.client.1.smithi191.stdout:8/949: mknod d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/de8/c146 0 2024-02-15T18:27:19.034 INFO:tasks.workunit.client.1.smithi191.stdout:4/909: sync 2024-02-15T18:27:19.034 INFO:tasks.workunit.client.1.smithi191.stdout:1/953: sync 2024-02-15T18:27:19.034 INFO:tasks.workunit.client.1.smithi191.stdout:8/950: dwrite d3/d2c/d1a/d23/d2a/f113 [0,4194304] 0 2024-02-15T18:27:19.037 INFO:tasks.workunit.client.1.smithi191.stdout:4/910: write d2/d5/d7/dee/ffc [1144801,57602] 0 2024-02-15T18:27:19.037 INFO:tasks.workunit.client.1.smithi191.stdout:1/954: creat d0/d8/df/d90/f147 x:0 0 0 2024-02-15T18:27:19.040 INFO:tasks.workunit.client.1.smithi191.stdout:4/911: mkdir d2/d5/d7/d2f/d145 0 2024-02-15T18:27:19.041 INFO:tasks.workunit.client.1.smithi191.stdout:1/955: mknod d0/c148 0 2024-02-15T18:27:19.050 INFO:tasks.workunit.client.1.smithi191.stdout:4/912: dread d2/d5/d8b/feb [0,4194304] 0 2024-02-15T18:27:19.052 INFO:tasks.workunit.client.1.smithi191.stdout:8/951: dwrite d3/d2c/d1a/d10d/f140 [0,4194304] 0 2024-02-15T18:27:19.052 INFO:tasks.workunit.client.1.smithi191.stdout:8/952: chown d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/da0 3154 1 2024-02-15T18:27:19.052 INFO:tasks.workunit.client.1.smithi191.stdout:8/953: chown d3/d2c/d1a/d36 16320614 1 2024-02-15T18:27:19.055 INFO:tasks.workunit.client.1.smithi191.stdout:8/954: write d3/d2c/d1a/d23/d2e/d47/d5f/fc5 [3641465,101022] 0 2024-02-15T18:27:19.056 INFO:tasks.workunit.client.1.smithi191.stdout:8/955: link d3/d2c/d1a/d23/d2e/cee d3/d2c/d1a/d23/d2e/d47/d142/c147 0 2024-02-15T18:27:19.057 INFO:tasks.workunit.client.1.smithi191.stdout:1/956: dwrite d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/dc9/fcc [0,4194304] 0 2024-02-15T18:27:19.058 INFO:tasks.workunit.client.1.smithi191.stdout:8/956: unlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/ff4 0 2024-02-15T18:27:19.061 INFO:tasks.workunit.client.1.smithi191.stdout:8/957: write d3/d2c/f114 [231405,119615] 0 2024-02-15T18:27:19.062 INFO:tasks.workunit.client.1.smithi191.stdout:8/958: write d3/d2c/f114 [3247994,83139] 0 2024-02-15T18:27:19.066 INFO:tasks.workunit.client.1.smithi191.stdout:1/957: mknod d0/d8/df/d12/d2a/d2b/d78/d60/d74/c149 0 2024-02-15T18:27:19.069 INFO:tasks.workunit.client.1.smithi191.stdout:4/913: dwrite d2/d5/d7/d90/d52/db5/dc9/f117 [0,4194304] 0 2024-02-15T18:27:19.069 INFO:tasks.workunit.client.1.smithi191.stdout:4/914: truncate d2/d5/d14/d47/d78/f4f 964456 0 2024-02-15T18:27:19.071 INFO:tasks.workunit.client.1.smithi191.stdout:8/959: mkdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d115/d148 0 2024-02-15T18:27:19.078 INFO:tasks.workunit.client.1.smithi191.stdout:1/958: truncate d0/d2f/d5c/f71 364516 0 2024-02-15T18:27:19.081 INFO:tasks.workunit.client.1.smithi191.stdout:4/915: link d2/d5/d7/d3e/d63/df0/cfa d2/d5/d14/d71/dcd/c146 0 2024-02-15T18:27:19.089 INFO:tasks.workunit.client.1.smithi191.stdout:8/960: rmdir d3/d2c/d1a/d23/d2e/d47/d5f/d11e 39 2024-02-15T18:27:19.100 INFO:tasks.workunit.client.1.smithi191.stdout:4/916: dwrite d2/d5/d7/d2f/fc7 [0,4194304] 0 2024-02-15T18:27:19.104 INFO:tasks.workunit.client.1.smithi191.stdout:4/917: mkdir d2/d5/d7/d90/d10b/d147 0 2024-02-15T18:27:19.109 INFO:tasks.workunit.client.1.smithi191.stdout:4/918: creat d2/d5/d14/d47/d78/dd0/f148 x:0 0 0 2024-02-15T18:27:19.110 INFO:tasks.workunit.client.1.smithi191.stdout:4/919: creat d2/d5/d7/d90/d52/db5/f149 x:0 0 0 2024-02-15T18:27:19.117 INFO:tasks.workunit.client.1.smithi191.stdout:4/920: dread d2/d5/d7/f6c [0,4194304] 0 2024-02-15T18:27:19.117 INFO:tasks.workunit.client.1.smithi191.stdout:4/921: read - d2/d5/d7/d3e/d63/d8c/dbb/ddc/fdf zero size 2024-02-15T18:27:19.117 INFO:tasks.workunit.client.1.smithi191.stdout:4/922: chown d2/d5/d7/d90/d52/dcb/d10c/d13c 250102562 1 2024-02-15T18:27:19.118 INFO:tasks.workunit.client.1.smithi191.stdout:4/923: rename d2/d5/d7/d3e/d63/df0 to d2/d5/d7/d3e/de8/d108/d14a 0 2024-02-15T18:27:19.131 INFO:tasks.workunit.client.1.smithi191.stdout:4/924: dwrite d2/d5/d7/d2f/d41/f73 [0,4194304] 0 2024-02-15T18:27:19.133 INFO:tasks.workunit.client.1.smithi191.stdout:4/925: rename d2/d5/d7/d3e/d63/d8c/d126 to d2/d5/d14/d71/d14b 0 2024-02-15T18:27:19.148 INFO:tasks.workunit.client.1.smithi191.stdout:4/926: dwrite d2/d5/d14/d71/fac [0,4194304] 0 2024-02-15T18:27:19.149 INFO:tasks.workunit.client.1.smithi191.stdout:4/927: rename d2/d5/d8b/de3/l7e to d2/d5/d14/d71/d14b/l14c 0 2024-02-15T18:27:19.168 INFO:tasks.workunit.client.1.smithi191.stdout:4/928: dwrite d2/d5/d124/faa [0,4194304] 0 2024-02-15T18:27:19.176 INFO:tasks.workunit.client.1.smithi191.stdout:4/929: mkdir d2/d5/d7/dee/d14d 0 2024-02-15T18:27:19.177 INFO:tasks.workunit.client.1.smithi191.stdout:4/930: mknod d2/d5/d7/dee/c14e 0 2024-02-15T18:27:19.179 INFO:tasks.workunit.client.1.smithi191.stdout:4/931: creat d2/d5/d7/d90/d52/dcb/d10c/d13c/f14f x:0 0 0 2024-02-15T18:27:19.180 INFO:tasks.workunit.client.1.smithi191.stdout:4/932: fdatasync d2/d5/d7/f3b 0 2024-02-15T18:27:19.181 INFO:tasks.workunit.client.1.smithi191.stdout:4/933: mkdir d2/d5/d14/d71/dcd/ded/d150 0 2024-02-15T18:27:19.182 INFO:tasks.workunit.client.1.smithi191.stdout:4/934: write d2/d5/d7/d3e/d63/d8c/dbb/f113 [1481392,113136] 0 2024-02-15T18:27:19.184 INFO:tasks.workunit.client.1.smithi191.stdout:4/935: getdents d2/d5/d14/d47/d78 0 2024-02-15T18:27:19.185 INFO:tasks.workunit.client.1.smithi191.stdout:4/936: truncate d2/d5/d8b/fe5 2730403 0 2024-02-15T18:27:19.187 INFO:tasks.workunit.client.1.smithi191.stdout:4/937: creat d2/d5/d7/d3e/d63/d8c/da4/f151 x:0 0 0 2024-02-15T18:27:19.189 INFO:tasks.workunit.client.1.smithi191.stdout:4/938: truncate d2/d5/d7/d90/d52/dcb/f128 316376 0 2024-02-15T18:27:19.189 INFO:tasks.workunit.client.1.smithi191.stdout:4/939: write d2/d5/f115 [948361,19005] 0 2024-02-15T18:27:19.189 INFO:tasks.workunit.client.1.smithi191.stdout:4/940: readlink d2/d5/d7/dee/lf3 0 2024-02-15T18:27:19.190 INFO:tasks.workunit.client.1.smithi191.stdout:4/941: dread d2/d5/d7/d90/d52/db5/dc9/f131 [0,4194304] 0 2024-02-15T18:27:19.190 INFO:tasks.workunit.client.1.smithi191.stdout:4/942: dread - d2/d5/fc1 zero size 2024-02-15T18:27:19.190 INFO:tasks.workunit.client.1.smithi191.stdout:4/943: chown d2/d5/d7/dee/lf3 3658884 1 2024-02-15T18:27:19.210 INFO:tasks.workunit.client.1.smithi191.stdout:4/944: dwrite d2/d5/d7/d90/d10b/f141 [0,4194304] 0 2024-02-15T18:27:19.222 INFO:tasks.workunit.client.1.smithi191.stdout:4/945: link d2/d5/d7/d3e/f94 d2/d5/d7/d3e/d63/d8c/da4/f152 0 2024-02-15T18:27:19.223 INFO:tasks.workunit.client.1.smithi191.stdout:4/946: readlink d2/d5/d7/d3e/d63/d8c/l11c 0 2024-02-15T18:27:19.227 INFO:tasks.workunit.client.1.smithi191.stdout:4/947: unlink d2/d5/d7/d90/d52/dcb/l139 0 2024-02-15T18:27:19.244 INFO:tasks.workunit.client.1.smithi191.stdout:4/948: dwrite d2/d5/d7/d90/d10b/f141 [0,4194304] 0 2024-02-15T18:27:19.246 INFO:tasks.workunit.client.1.smithi191.stdout:4/949: truncate d2/d5/d7/f67 2490620 0 2024-02-15T18:27:19.246 INFO:tasks.workunit.client.1.smithi191.stdout:4/950: dread - d2/d5/d14/d71/f13a zero size 2024-02-15T18:27:19.246 INFO:tasks.workunit.client.1.smithi191.stdout:4/951: stat d2/d5/d14/d47/d78/d119 0 2024-02-15T18:27:19.247 INFO:tasks.workunit.client.1.smithi191.stdout:4/952: rename d2/d5/d7/d90/d52/db5/dc9 to d2/d5/d7/dee/d14d/d153 0 2024-02-15T18:27:19.263 INFO:tasks.workunit.client.1.smithi191.stdout:4/953: dwrite d2/d5/d14/d47/d78/dd0/dd3/de4/ff6 [0,4194304] 0 2024-02-15T18:27:19.265 INFO:tasks.workunit.client.1.smithi191.stdout:4/954: rmdir d2/d5/d7/d2f/d145 0 2024-02-15T18:27:19.271 INFO:tasks.workunit.client.1.smithi191.stdout:4/955: link d2/d5/d14/d47/f8e d2/d5/d7/dee/d14d/d153/f154 0 2024-02-15T18:27:19.271 INFO:tasks.workunit.client.1.smithi191.stdout:4/956: write d2/d5/d7/f15 [674131,108318] 0 2024-02-15T18:27:19.273 INFO:tasks.workunit.client.1.smithi191.stdout:4/957: creat d2/d5/d124/f155 x:0 0 0 2024-02-15T18:27:19.288 INFO:tasks.workunit.client.1.smithi191.stdout:4/958: dwrite d2/d5/d7/f29 [0,4194304] 0 2024-02-15T18:27:19.304 INFO:tasks.workunit.client.1.smithi191.stdout:4/959: dwrite d2/d5/d7/d2f/d41/f73 [0,4194304] 0 2024-02-15T18:27:19.305 INFO:tasks.workunit.client.1.smithi191.stdout:4/960: creat d2/d5/d7/d90/d10b/d147/f156 x:0 0 0 2024-02-15T18:27:19.305 INFO:tasks.workunit.client.1.smithi191.stdout:4/961: chown d2/d5/d14/l4e 2 1 2024-02-15T18:27:19.307 INFO:tasks.workunit.client.1.smithi191.stdout:4/962: creat d2/d5/d7/d90/d52/dcb/f157 x:0 0 0 2024-02-15T18:27:19.308 INFO:tasks.workunit.client.1.smithi191.stdout:4/963: symlink d2/d5/d14/d47/d48/l158 0 2024-02-15T18:27:19.315 INFO:tasks.workunit.client.1.smithi191.stdout:4/964: dread d2/d5/d7/f93 [0,4194304] 0 2024-02-15T18:27:19.315 INFO:tasks.workunit.client.1.smithi191.stdout:4/965: truncate d2/d5/d8b/de3/f9a 226936 0 2024-02-15T18:27:19.316 INFO:tasks.workunit.client.1.smithi191.stdout:4/966: dread d2/d5/d7/d90/fd4 [0,4194304] 0 2024-02-15T18:27:19.318 INFO:tasks.workunit.client.1.smithi191.stdout:4/967: link d2/d5/d8b/feb d2/d5/d14/d71/dcd/ded/d150/f159 0 2024-02-15T18:27:19.318 INFO:tasks.workunit.client.1.smithi191.stdout:4/968: dread - d2/d5/d124/f155 zero size 2024-02-15T18:27:19.318 INFO:tasks.workunit.client.1.smithi191.stdout:4/969: truncate d2/d5/d7/d2f/f134 596113 0 2024-02-15T18:27:19.321 INFO:tasks.workunit.client.1.smithi191.stdout:4/970: rmdir d2/d5/d14/d47/d78/dd0 39 2024-02-15T18:27:19.322 INFO:tasks.workunit.client.1.smithi191.stdout:4/971: mkdir d2/d5/d7/d15a 0 2024-02-15T18:27:19.337 INFO:tasks.workunit.client.1.smithi191.stdout:4/972: dwrite d2/d5/d7/f29 [0,4194304] 0 2024-02-15T18:27:19.338 INFO:tasks.workunit.client.1.smithi191.stdout:4/973: creat d2/d5/d7/d3e/de8/d108/d14a/f15b x:0 0 0 2024-02-15T18:27:19.338 INFO:tasks.workunit.client.1.smithi191.stdout:0/630: sync 2024-02-15T18:27:19.342 INFO:tasks.workunit.client.1.smithi191.stdout:4/974: creat d2/d5/d7/dee/f15c x:0 0 0 2024-02-15T18:27:19.348 INFO:tasks.workunit.client.1.smithi191.stdout:0/631: unlink d7/d4a/cc7 0 2024-02-15T18:27:19.348 INFO:tasks.workunit.client.1.smithi191.stdout:0/632: fsync d7/d1b/d30/f43 0 2024-02-15T18:27:19.348 INFO:tasks.workunit.client.1.smithi191.stdout:0/633: write d7/da/f8e [272951,35884] 0 2024-02-15T18:27:19.374 INFO:tasks.workunit.client.1.smithi191.stdout:4/975: dwrite d2/d5/d124/f9d [0,4194304] 0 2024-02-15T18:27:19.375 INFO:tasks.workunit.client.1.smithi191.stdout:0/634: dwrite d7/da/d1a/d2c/f86 [0,4194304] 0 2024-02-15T18:27:19.376 INFO:tasks.workunit.client.1.smithi191.stdout:4/976: truncate d2/d5/d14/d71/dcd/ded/d150/f159 2813405 0 2024-02-15T18:27:19.376 INFO:tasks.workunit.client.1.smithi191.stdout:4/977: chown d2/d5/d14/l2a 1394 1 2024-02-15T18:27:19.377 INFO:tasks.workunit.client.1.smithi191.stdout:4/978: write d2/d5/d7/d2f/d41/f98 [3714742,74604] 0 2024-02-15T18:27:19.383 INFO:tasks.workunit.client.1.smithi191.stdout:0/635: rename d7/da/d17/d31/d5f/da9 to d7/da/d17/d31/d5f/d9e/dc3/dd8 0 2024-02-15T18:27:19.383 INFO:tasks.workunit.client.1.smithi191.stdout:0/636: readlink d7/d94/laf 0 2024-02-15T18:27:19.384 INFO:tasks.workunit.client.1.smithi191.stdout:4/979: symlink d2/d5/d7/d3e/de8/d108/l15d 0 2024-02-15T18:27:19.384 INFO:tasks.workunit.client.1.smithi191.stdout:4/980: chown d2/d5/f4d 16591 1 2024-02-15T18:27:19.390 INFO:tasks.workunit.client.1.smithi191.stdout:0/637: symlink d7/da/d6c/ld9 0 2024-02-15T18:27:19.391 INFO:tasks.workunit.client.1.smithi191.stdout:0/638: read d7/d1b/f23 [4581693,76234] 0 2024-02-15T18:27:19.393 INFO:tasks.workunit.client.1.smithi191.stdout:3/739: sync 2024-02-15T18:27:19.393 INFO:tasks.workunit.client.1.smithi191.stdout:9/879: sync 2024-02-15T18:27:19.393 INFO:tasks.workunit.client.1.smithi191.stdout:4/981: rename d2/d5/d124/lb7 to d2/d5/d124/l15e 0 2024-02-15T18:27:19.394 INFO:tasks.workunit.client.1.smithi191.stdout:0/639: truncate d7/d3b/f69 1357268 0 2024-02-15T18:27:19.396 INFO:tasks.workunit.client.1.smithi191.stdout:9/880: truncate d0/d7/dc/d30/d4f/f6d 923067 0 2024-02-15T18:27:19.396 INFO:tasks.workunit.client.1.smithi191.stdout:9/881: fdatasync d0/d7/dc/d17/d44/fcc 0 2024-02-15T18:27:19.396 INFO:tasks.workunit.client.1.smithi191.stdout:9/882: chown d0/d1d/d3f/f10c 49 1 2024-02-15T18:27:19.396 INFO:tasks.workunit.client.1.smithi191.stdout:3/740: mkdir d6/d3b/d42/d49/dbf/df7 0 2024-02-15T18:27:19.397 INFO:tasks.workunit.client.1.smithi191.stdout:3/741: chown d6/d5d/c7e 27036956 1 2024-02-15T18:27:19.397 INFO:tasks.workunit.client.1.smithi191.stdout:4/982: creat d2/d5/d7/d90/d52/f15f x:0 0 0 2024-02-15T18:27:19.402 INFO:tasks.workunit.client.1.smithi191.stdout:3/742: getdents d6/d27 0 2024-02-15T18:27:19.402 INFO:tasks.workunit.client.1.smithi191.stdout:3/743: chown d6/d27/f32 962509 1 2024-02-15T18:27:19.403 INFO:tasks.workunit.client.1.smithi191.stdout:5/592: sync 2024-02-15T18:27:19.403 INFO:tasks.workunit.client.1.smithi191.stdout:7/869: sync 2024-02-15T18:27:19.403 INFO:tasks.workunit.client.1.smithi191.stdout:2/939: sync 2024-02-15T18:27:19.403 INFO:tasks.workunit.client.1.smithi191.stdout:8/961: sync 2024-02-15T18:27:19.403 INFO:tasks.workunit.client.1.smithi191.stdout:5/593: dread - d0/d18/d5c/d7c/f67 zero size 2024-02-15T18:27:19.403 INFO:tasks.workunit.client.1.smithi191.stdout:8/962: chown d3/d5e/fd1 12474898 1 2024-02-15T18:27:19.403 INFO:tasks.workunit.client.1.smithi191.stdout:7/870: truncate d0/db/d26/d5f/d77/d35/d42/f93 524964 0 2024-02-15T18:27:19.403 INFO:tasks.workunit.client.1.smithi191.stdout:8/963: fsync d3/d2c/d1a/f20 0 2024-02-15T18:27:19.403 INFO:tasks.workunit.client.1.smithi191.stdout:1/959: sync 2024-02-15T18:27:19.403 INFO:tasks.workunit.client.1.smithi191.stdout:6/948: sync 2024-02-15T18:27:19.403 INFO:tasks.workunit.client.1.smithi191.stdout:4/983: mknod d2/d5/d7/d3e/de8/d108/d14a/c160 0 2024-02-15T18:27:19.405 INFO:tasks.workunit.client.1.smithi191.stdout:9/883: rename d0/d29/d48/d66/ce5 to d0/d7/dc/d8c/c122 0 2024-02-15T18:27:19.405 INFO:tasks.workunit.client.1.smithi191.stdout:3/744: symlink d6/d3b/d42/dbb/lf8 0 2024-02-15T18:27:19.407 INFO:tasks.workunit.client.1.smithi191.stdout:2/940: link df/d12/d19/d1a/d35/d47/da2/c120 df/d61/c134 0 2024-02-15T18:27:19.411 INFO:tasks.workunit.client.1.smithi191.stdout:4/984: dread d2/f5c [0,4194304] 0 2024-02-15T18:27:19.412 INFO:tasks.workunit.client.1.smithi191.stdout:7/871: symlink d0/db/d65/d11/d5a/d66/d1e/l13a 0 2024-02-15T18:27:19.418 INFO:tasks.workunit.client.1.smithi191.stdout:8/964: creat d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d115/f149 x:0 0 0 2024-02-15T18:27:19.420 INFO:tasks.workunit.client.1.smithi191.stdout:6/949: mknod d8/d9/dac/ddb/dd7/c132 0 2024-02-15T18:27:19.423 INFO:tasks.workunit.client.1.smithi191.stdout:1/960: rename d0/d2f/d51/d5e/f139 to d0/d8/df/d90/f14a 0 2024-02-15T18:27:19.423 INFO:tasks.workunit.client.1.smithi191.stdout:1/961: write d0/d8/df/d90/f143 [313853,41190] 0 2024-02-15T18:27:19.424 INFO:tasks.workunit.client.1.smithi191.stdout:9/884: rmdir d0/d7/dc/d17 39 2024-02-15T18:27:19.426 INFO:tasks.workunit.client.1.smithi191.stdout:2/941: symlink df/d12/d1e/d6f/l135 0 2024-02-15T18:27:19.426 INFO:tasks.workunit.client.1.smithi191.stdout:2/942: dread - df/d12/d19/f108 zero size 2024-02-15T18:27:19.426 INFO:tasks.workunit.client.1.smithi191.stdout:2/943: readlink df/d61/d9f/lcd 0 2024-02-15T18:27:19.426 INFO:tasks.workunit.client.1.smithi191.stdout:2/944: write df/d12/d19/f108 [335784,65101] 0 2024-02-15T18:27:19.429 INFO:tasks.workunit.client.1.smithi191.stdout:7/872: unlink d0/db/d65/d11/d5a/d66/c34 0 2024-02-15T18:27:19.429 INFO:tasks.workunit.client.1.smithi191.stdout:7/873: chown d0/db/d26/d5f/d77/d35/c105 2 1 2024-02-15T18:27:19.431 INFO:tasks.workunit.client.1.smithi191.stdout:6/950: mkdir d8/d9/d4a/df6/d133 0 2024-02-15T18:27:19.431 INFO:tasks.workunit.client.1.smithi191.stdout:8/965: dread d3/d2c/d1a/f1b [4194304,4194304] 0 2024-02-15T18:27:19.432 INFO:tasks.workunit.client.1.smithi191.stdout:4/985: dwrite d2/d5/d7/d2f/d41/f57 [0,4194304] 0 2024-02-15T18:27:19.432 INFO:tasks.workunit.client.1.smithi191.stdout:5/594: dwrite d0/d23/d41/d28/f36 [0,4194304] 0 2024-02-15T18:27:19.433 INFO:tasks.workunit.client.1.smithi191.stdout:1/962: rename d0/d2f/d5c/l126 to d0/d8/df/d12/d2a/d2b/d78/d60/d74/l14b 0 2024-02-15T18:27:19.433 INFO:tasks.workunit.client.1.smithi191.stdout:1/963: stat d0/d8/df/d12/d2a/fd0 0 2024-02-15T18:27:19.433 INFO:tasks.workunit.client.1.smithi191.stdout:1/964: truncate d0/d8/df/d2e/fe5 575498 0 2024-02-15T18:27:19.433 INFO:tasks.workunit.client.1.smithi191.stdout:1/965: chown d0/d8/df/d12/d2a/d2b/d78/d60/fbe 23 1 2024-02-15T18:27:19.433 INFO:tasks.workunit.client.1.smithi191.stdout:1/966: write d0/d8/d48/fa1 [541483,35173] 0 2024-02-15T18:27:19.434 INFO:tasks.workunit.client.1.smithi191.stdout:9/885: mknod d0/d7/dc/d32/c123 0 2024-02-15T18:27:19.434 INFO:tasks.workunit.client.1.smithi191.stdout:9/886: chown d0/d7/dc/d17/d44/fd8 337586276 1 2024-02-15T18:27:19.434 INFO:tasks.workunit.client.1.smithi191.stdout:3/745: symlink d6/d3b/d42/lf9 0 2024-02-15T18:27:19.434 INFO:tasks.workunit.client.1.smithi191.stdout:3/746: chown d6/d1b/d60/da2/dac/de2/dec 3 1 2024-02-15T18:27:19.435 INFO:tasks.workunit.client.1.smithi191.stdout:3/747: write d6/d27/d4d/d5a/da4/fc0 [4725493,50633] 0 2024-02-15T18:27:19.435 INFO:tasks.workunit.client.1.smithi191.stdout:3/748: dread - d6/d3b/fbd zero size 2024-02-15T18:27:19.435 INFO:tasks.workunit.client.1.smithi191.stdout:3/749: fsync d6/d27/f78 0 2024-02-15T18:27:19.435 INFO:tasks.workunit.client.1.smithi191.stdout:0/640: dwrite d7/da/d17/d31/d7d/fc8 [0,4194304] 0 2024-02-15T18:27:19.435 INFO:tasks.workunit.client.1.smithi191.stdout:7/874: creat d0/db/d26/d5f/d77/f13b x:0 0 0 2024-02-15T18:27:19.435 INFO:tasks.workunit.client.1.smithi191.stdout:0/641: chown d7/da/d1a/d34/f6d 10255 1 2024-02-15T18:27:19.436 INFO:tasks.workunit.client.1.smithi191.stdout:2/945: read df/d12/d19/d1a/d35/d5b/dbf/f10c [2568721,101700] 0 2024-02-15T18:27:19.436 INFO:tasks.workunit.client.1.smithi191.stdout:6/951: truncate d8/d9/dac/db0/d95/d90/f96 512417 0 2024-02-15T18:27:19.436 INFO:tasks.workunit.client.1.smithi191.stdout:6/952: fsync d8/d9/dac/f114 0 2024-02-15T18:27:19.436 INFO:tasks.workunit.client.1.smithi191.stdout:2/946: write df/d12/f36 [3845555,110528] 0 2024-02-15T18:27:19.437 INFO:tasks.workunit.client.1.smithi191.stdout:8/966: symlink d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d95/l14a 0 2024-02-15T18:27:19.440 INFO:tasks.workunit.client.1.smithi191.stdout:4/986: mkdir d2/d5/d7/d3e/d63/d8c/d161 0 2024-02-15T18:27:19.443 INFO:tasks.workunit.client.1.smithi191.stdout:5/595: symlink d0/d18/d5c/d7c/lc5 0 2024-02-15T18:27:19.445 INFO:tasks.workunit.client.1.smithi191.stdout:1/967: mkdir d0/d8/df/d12/d14c 0 2024-02-15T18:27:19.446 INFO:tasks.workunit.client.1.smithi191.stdout:9/887: dread d0/d7/dc/d32/da3/fb6 [0,4194304] 0 2024-02-15T18:27:19.446 INFO:tasks.workunit.client.1.smithi191.stdout:9/888: readlink d0/d7/dc/d17/d44/de6/le9 0 2024-02-15T18:27:19.446 INFO:tasks.workunit.client.1.smithi191.stdout:8/967: write d3/d2c/d1a/d23/d2e/d5b/d69/d6d/fbd [1019719,130678] 0 2024-02-15T18:27:19.450 INFO:tasks.workunit.client.1.smithi191.stdout:3/750: mknod d6/d27/d4d/cfa 0 2024-02-15T18:27:19.452 INFO:tasks.workunit.client.1.smithi191.stdout:9/889: dread d0/d29/d48/d66/f120 [0,4194304] 0 2024-02-15T18:27:19.456 INFO:tasks.workunit.client.1.smithi191.stdout:7/875: dwrite d0/db/d26/d5f/d77/d35/d42/f7b [0,4194304] 0 2024-02-15T18:27:19.457 INFO:tasks.workunit.client.1.smithi191.stdout:7/876: fdatasync d0/db/d65/d11/d5a/d66/f131 0 2024-02-15T18:27:19.457 INFO:tasks.workunit.client.1.smithi191.stdout:2/947: dwrite df/d12/d19/d1a/d35/d47/d4b/d7a/fe1 [0,4194304] 0 2024-02-15T18:27:19.459 INFO:tasks.workunit.client.1.smithi191.stdout:2/948: dread df/d12/d1e/d6f/fad [0,4194304] 0 2024-02-15T18:27:19.465 INFO:tasks.workunit.client.1.smithi191.stdout:0/642: rmdir d7/d1b/d30 39 2024-02-15T18:27:19.466 INFO:tasks.workunit.client.1.smithi191.stdout:0/643: dread d7/fb1 [0,4194304] 0 2024-02-15T18:27:19.466 INFO:tasks.workunit.client.1.smithi191.stdout:0/644: chown d7/da/d1a/d2c/d4c/fac 7535 1 2024-02-15T18:27:19.466 INFO:tasks.workunit.client.1.smithi191.stdout:0/645: stat d7/d1b/d30/f43 0 2024-02-15T18:27:19.540 INFO:tasks.workunit.client.1.smithi191.stdout:6/953: getdents d8/d9/dac/ddb/dd7 0 2024-02-15T18:27:19.544 INFO:tasks.workunit.client.1.smithi191.stdout:4/987: rename d2/d5/d14/d44/la6 to d2/d5/d7/d90/d52/dcb/d10c/d13c/l162 0 2024-02-15T18:27:19.547 INFO:tasks.workunit.client.1.smithi191.stdout:5/596: truncate d0/f1a 3010490 0 2024-02-15T18:27:19.548 INFO:tasks.workunit.client.1.smithi191.stdout:1/968: truncate d0/d2f/d51/f1d 1545447 0 2024-02-15T18:27:19.548 INFO:tasks.workunit.client.1.smithi191.stdout:1/969: write d0/d8/df/d12/f8f [1192559,89679] 0 2024-02-15T18:27:19.550 INFO:tasks.workunit.client.1.smithi191.stdout:8/968: creat d3/d2c/d1a/d23/f14b x:0 0 0 2024-02-15T18:27:19.551 INFO:tasks.workunit.client.1.smithi191.stdout:8/969: write d3/faa [1113874,56001] 0 2024-02-15T18:27:19.551 INFO:tasks.workunit.client.1.smithi191.stdout:8/970: truncate d3/d2c/f48 4957556 0 2024-02-15T18:27:19.551 INFO:tasks.workunit.client.1.smithi191.stdout:8/971: fsync d3/d2c/d1a/d23/d2e/d64/f136 0 2024-02-15T18:27:19.554 INFO:tasks.workunit.client.1.smithi191.stdout:3/751: symlink d6/d1b/d60/da2/lfb 0 2024-02-15T18:27:19.554 INFO:tasks.workunit.client.1.smithi191.stdout:3/752: write d6/d3b/f84 [261058,24916] 0 2024-02-15T18:27:19.555 INFO:tasks.workunit.client.1.smithi191.stdout:3/753: write d6/d3b/d30/f45 [1577402,19958] 0 2024-02-15T18:27:19.555 INFO:tasks.workunit.client.1.smithi191.stdout:9/890: mknod d0/d7/c124 0 2024-02-15T18:27:19.556 INFO:tasks.workunit.client.1.smithi191.stdout:9/891: read d0/d7/dc/d8c/f78 [794633,127237] 0 2024-02-15T18:27:19.573 INFO:tasks.workunit.client.1.smithi191.stdout:0/646: creat d7/d4a/dc0/fda x:0 0 0 2024-02-15T18:27:19.573 INFO:tasks.workunit.client.1.smithi191.stdout:0/647: dread - d7/da/d1a/d34/f6d zero size 2024-02-15T18:27:19.574 INFO:tasks.workunit.client.1.smithi191.stdout:0/648: dread d7/da/d1a/d2c/d4c/fac [0,4194304] 0 2024-02-15T18:27:19.575 INFO:tasks.workunit.client.1.smithi191.stdout:9/892: dwrite d0/fdc [0,4194304] 0 2024-02-15T18:27:19.577 INFO:tasks.workunit.client.1.smithi191.stdout:6/954: creat d8/d9/d2b/d57/d8e/d109/f134 x:0 0 0 2024-02-15T18:27:19.577 INFO:tasks.workunit.client.1.smithi191.stdout:6/955: stat d8/d9/dac/db0/d95/lad 0 2024-02-15T18:27:19.577 INFO:tasks.workunit.client.1.smithi191.stdout:6/956: chown d8/d9/d4a/d4c/c119 518339536 1 2024-02-15T18:27:19.577 INFO:tasks.workunit.client.1.smithi191.stdout:6/957: write d8/d9/d4a/d4c/fee [878417,128684] 0 2024-02-15T18:27:19.582 INFO:tasks.workunit.client.1.smithi191.stdout:7/877: rename d0/db/d26/d5f/fd5 to d0/db/d26/d5f/d77/df/f13c 0 2024-02-15T18:27:19.583 INFO:tasks.workunit.client.1.smithi191.stdout:7/878: read d0/db/d26/d5f/d77/f51 [2760801,45539] 0 2024-02-15T18:27:19.584 INFO:tasks.workunit.client.1.smithi191.stdout:4/988: creat d2/d5/d7/d90/d10b/f163 x:0 0 0 2024-02-15T18:27:19.584 INFO:tasks.workunit.client.1.smithi191.stdout:4/989: readlink d2/d5/d7/d90/d52/l91 0 2024-02-15T18:27:19.587 INFO:tasks.workunit.client.1.smithi191.stdout:5/597: creat d0/d23/d41/d3c/fc6 x:0 0 0 2024-02-15T18:27:19.591 INFO:tasks.workunit.client.1.smithi191.stdout:1/970: symlink d0/d8/df/d12/d33/d49/l14d 0 2024-02-15T18:27:19.592 INFO:tasks.workunit.client.1.smithi191.stdout:8/972: rmdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/de8 39 2024-02-15T18:27:19.592 INFO:tasks.workunit.client.1.smithi191.stdout:8/973: write d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d115/f122 [54115,62552] 0 2024-02-15T18:27:19.594 INFO:tasks.workunit.client.1.smithi191.stdout:1/971: dread d0/d8/df/d12/d2a/d2b/f5a [0,4194304] 0 2024-02-15T18:27:19.598 INFO:tasks.workunit.client.1.smithi191.stdout:3/754: mknod d6/d1b/d60/da2/dac/de2/dec/cfc 0 2024-02-15T18:27:19.599 INFO:tasks.workunit.client.1.smithi191.stdout:3/755: write d6/d3b/ff [6439487,20372] 0 2024-02-15T18:27:19.603 INFO:tasks.workunit.client.1.smithi191.stdout:4/990: dwrite d2/d5/d7/d90/d52/dcb/f157 [0,4194304] 0 2024-02-15T18:27:19.603 INFO:tasks.workunit.client.1.smithi191.stdout:4/991: stat d2/d5/d7/d3e/f127 0 2024-02-15T18:27:19.604 INFO:tasks.workunit.client.1.smithi191.stdout:0/649: symlink d7/da/d17/d31/d7d/ldb 0 2024-02-15T18:27:19.606 INFO:tasks.workunit.client.1.smithi191.stdout:9/893: creat d0/d1d/d3f/d80/f125 x:0 0 0 2024-02-15T18:27:19.606 INFO:tasks.workunit.client.1.smithi191.stdout:9/894: write d0/d7/dc/f15 [8988725,60464] 0 2024-02-15T18:27:19.608 INFO:tasks.workunit.client.1.smithi191.stdout:6/958: rmdir d8/d9/d2b/d57/d103 39 2024-02-15T18:27:19.611 INFO:tasks.workunit.client.1.smithi191.stdout:7/879: mknod d0/db/d65/dc/c13d 0 2024-02-15T18:27:19.612 INFO:tasks.workunit.client.1.smithi191.stdout:9/895: dread d0/d7/dc/d8c/f1c [0,4194304] 0 2024-02-15T18:27:19.614 INFO:tasks.workunit.client.1.smithi191.stdout:5/598: creat d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/fc7 x:0 0 0 2024-02-15T18:27:19.615 INFO:tasks.workunit.client.1.smithi191.stdout:8/974: creat d3/d2c/d1a/d36/f14c x:0 0 0 2024-02-15T18:27:19.619 INFO:tasks.workunit.client.1.smithi191.stdout:1/972: rename d0/d8/d48/fa1 to d0/d8/df/f14e 0 2024-02-15T18:27:19.620 INFO:tasks.workunit.client.1.smithi191.stdout:3/756: getdents d6/d1b/db0 0 2024-02-15T18:27:19.627 INFO:tasks.workunit.client.1.smithi191.stdout:4/992: creat d2/d5/d14/d47/d78/f164 x:0 0 0 2024-02-15T18:27:19.628 INFO:tasks.workunit.client.1.smithi191.stdout:0/650: truncate d7/da/d1a/d2c/f4b 3546958 0 2024-02-15T18:27:19.629 INFO:tasks.workunit.client.1.smithi191.stdout:6/959: creat d8/d9/d4a/d4c/d7f/f135 x:0 0 0 2024-02-15T18:27:19.630 INFO:tasks.workunit.client.1.smithi191.stdout:7/880: truncate d0/db/d26/d5f/d77/fcd 1132801 0 2024-02-15T18:27:19.630 INFO:tasks.workunit.client.1.smithi191.stdout:7/881: write d0/db/d65/dc/d4b/de8/ff8 [696780,10756] 0 2024-02-15T18:27:19.632 INFO:tasks.workunit.client.1.smithi191.stdout:5/599: link d0/d23/d41/d28/l5b d0/db5/lc8 0 2024-02-15T18:27:19.633 INFO:tasks.workunit.client.1.smithi191.stdout:8/975: creat d3/d2c/d1a/d23/d2e/d47/f14d x:0 0 0 2024-02-15T18:27:19.635 INFO:tasks.workunit.client.1.smithi191.stdout:9/896: rename d0/d7/dc/d17/d33 to d0/d7/dc/d32/de4/dec/d126 0 2024-02-15T18:27:19.636 INFO:tasks.workunit.client.1.smithi191.stdout:3/757: dwrite d6/d3b/d42/fa1 [0,4194304] 0 2024-02-15T18:27:19.638 INFO:tasks.workunit.client.1.smithi191.stdout:3/758: dread d6/d8e/f99 [0,4194304] 0 2024-02-15T18:27:19.643 INFO:tasks.workunit.client.1.smithi191.stdout:4/993: unlink d2/d5/d7/d2f/d41/f13d 0 2024-02-15T18:27:19.643 INFO:tasks.workunit.client.1.smithi191.stdout:4/994: readlink d2/d5/l11 0 2024-02-15T18:27:19.643 INFO:tasks.workunit.client.1.smithi191.stdout:4/995: truncate d2/d5/d7/d3e/d63/f12d 490452 0 2024-02-15T18:27:19.643 INFO:tasks.workunit.client.1.smithi191.stdout:4/996: chown d2/d5/d14/d47/d48/fce 182224465 1 2024-02-15T18:27:19.644 INFO:tasks.workunit.client.1.smithi191.stdout:0/651: creat d7/da/d17/d31/d7d/fdc x:0 0 0 2024-02-15T18:27:19.644 INFO:tasks.workunit.client.1.smithi191.stdout:0/652: fsync d7/da/d1a/d2c/f32 0 2024-02-15T18:27:19.644 INFO:tasks.workunit.client.1.smithi191.stdout:0/653: write d7/d1b/d30/f43 [1724611,48847] 0 2024-02-15T18:27:19.644 INFO:tasks.workunit.client.1.smithi191.stdout:0/654: readlink d7/da/d1a/d34/l19 0 2024-02-15T18:27:19.648 INFO:tasks.workunit.client.1.smithi191.stdout:7/882: creat d0/db/d65/d11/d5a/d66/d1e/df5/f13e x:0 0 0 2024-02-15T18:27:19.648 INFO:tasks.workunit.client.1.smithi191.stdout:7/883: chown d0/db/d65/d11/dd7 10 1 2024-02-15T18:27:19.648 INFO:tasks.workunit.client.1.smithi191.stdout:7/884: write d0/db/d26/d5f/ffb [416136,99052] 0 2024-02-15T18:27:19.648 INFO:tasks.workunit.client.1.smithi191.stdout:7/885: write d0/f80 [558884,46025] 0 2024-02-15T18:27:19.650 INFO:tasks.workunit.client.1.smithi191.stdout:5/600: truncate d0/d23/d41/d28/d31/d5a/d61/f82 2121479 0 2024-02-15T18:27:19.651 INFO:tasks.workunit.client.1.smithi191.stdout:8/976: symlink d3/d2c/d1a/d23/d2e/d5b/l14e 0 2024-02-15T18:27:19.651 INFO:tasks.workunit.client.1.smithi191.stdout:8/977: write d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/fda [903704,31198] 0 2024-02-15T18:27:19.654 INFO:tasks.workunit.client.1.smithi191.stdout:1/973: rename d0/d8/df/d2e/lc2 to d0/d2f/d77/l14f 0 2024-02-15T18:27:19.661 INFO:tasks.workunit.client.1.smithi191.stdout:3/759: dwrite d6/d27/d4d/d5a/fe1 [0,4194304] 0 2024-02-15T18:27:19.661 INFO:tasks.workunit.client.1.smithi191.stdout:3/760: write d6/d3b/d30/f45 [1819871,119255] 0 2024-02-15T18:27:19.664 INFO:tasks.workunit.client.1.smithi191.stdout:5/601: dwrite d0/d23/d41/d28/f98 [0,4194304] 0 2024-02-15T18:27:19.664 INFO:tasks.workunit.client.1.smithi191.stdout:5/602: chown d0/d23/l2c 1507477 1 2024-02-15T18:27:19.668 INFO:tasks.workunit.client.1.smithi191.stdout:4/997: mkdir d2/d5/d14/d71/dcd/ded/d165 0 2024-02-15T18:27:19.668 INFO:tasks.workunit.client.1.smithi191.stdout:4/998: write d2/d5/d14/d47/d78/f13e [1007551,61734] 0 2024-02-15T18:27:19.671 INFO:tasks.workunit.client.1.smithi191.stdout:0/655: link d7/da/d1a/d34/d27/l64 d7/da/d17/d31/d5f/dae/ldd 0 2024-02-15T18:27:19.674 INFO:tasks.workunit.client.1.smithi191.stdout:7/886: creat d0/db/d26/d5f/d8c/f13f x:0 0 0 2024-02-15T18:27:19.676 INFO:tasks.workunit.client.1.smithi191.stdout:8/978: getdents d3 0 2024-02-15T18:27:19.677 INFO:tasks.workunit.client.1.smithi191.stdout:1/974: mknod d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/dc9/c150 0 2024-02-15T18:27:19.679 INFO:tasks.workunit.client.1.smithi191.stdout:6/960: rename d8/f97 to d8/d9/d4a/d4c/dc4/d12c/f136 0 2024-02-15T18:27:19.682 INFO:tasks.workunit.client.1.smithi191.stdout:8/979: dread d3/d2c/d1a/d23/d2e/d47/d5f/f50 [0,4194304] 0 2024-02-15T18:27:19.684 INFO:tasks.workunit.client.1.smithi191.stdout:3/761: write d6/d3b/d42/f70 [121735,70025] 0 2024-02-15T18:27:19.686 INFO:tasks.workunit.client.1.smithi191.stdout:5/603: creat d0/d23/d41/d3c/fc9 x:0 0 0 2024-02-15T18:27:19.687 INFO:tasks.workunit.client.1.smithi191.stdout:4/999: symlink d2/d5/d7/d90/d52/db5/l166 0 2024-02-15T18:27:19.688 INFO:tasks.workunit.client.1.smithi191.stdout:0/656: creat d7/d1b/fde x:0 0 0 2024-02-15T18:27:19.689 INFO:tasks.workunit.client.1.smithi191.stdout:7/887: creat d0/db/d65/f140 x:0 0 0 2024-02-15T18:27:19.689 INFO:tasks.workunit.client.1.smithi191.stdout:7/888: chown d0/db/d26 18082 1 2024-02-15T18:27:19.689 INFO:tasks.workunit.client.1.smithi191.stdout:7/889: write d0/db/d65/dc/fa8 [1658740,28885] 0 2024-02-15T18:27:19.690 INFO:tasks.workunit.client.1.smithi191.stdout:6/961: link d8/d9/dac/db0/d95/d9d/d6b/d72/l74 d8/d9/d2b/d57/d8e/l137 0 2024-02-15T18:27:19.691 INFO:tasks.workunit.client.1.smithi191.stdout:8/980: stat d3/d2c/d1a/d23/d3e/db2/l145 0 2024-02-15T18:27:19.696 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: debug 2024-02-15T18:27:19.418+0000 7ff991ed9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi191 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-15T18:27:19.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191[40629]: debug 2024-02-15T18:27:19.418+0000 7ff991ed9700 -1 mon.smithi191@1(peon) e2 *** Got Signal Terminated *** 2024-02-15T18:27:19.697 INFO:tasks.workunit.client.1.smithi191.stdout:5/604: dread d0/d23/f27 [0,4194304] 0 2024-02-15T18:27:19.699 INFO:tasks.workunit.client.1.smithi191.stdout:3/762: mknod d6/d27/d4d/d5a/d92/dd2/cfd 0 2024-02-15T18:27:19.699 INFO:tasks.workunit.client.1.smithi191.stdout:1/975: dwrite d0/d2f/d51/d10/ffc [0,4194304] 0 2024-02-15T18:27:19.709 INFO:tasks.workunit.client.1.smithi191.stdout:8/981: mknod d3/d5e/c14f 0 2024-02-15T18:27:19.710 INFO:tasks.workunit.client.1.smithi191.stdout:7/890: dwrite d0/db/d26/d5f/d77/d35/d42/fc3 [0,4194304] 0 2024-02-15T18:27:19.715 INFO:tasks.workunit.client.1.smithi191.stdout:3/763: symlink d6/d1b/d60/da2/lfe 0 2024-02-15T18:27:19.715 INFO:tasks.workunit.client.1.smithi191.stdout:5/605: rename d0/d18/d5c/d73/la5 to d0/d18/d5c/d7c/d49/lca 0 2024-02-15T18:27:19.717 INFO:tasks.workunit.client.1.smithi191.stdout:1/976: rmdir d0/d8/df/d12/d33/d49/dea 39 2024-02-15T18:27:19.718 INFO:tasks.workunit.client.1.smithi191.stdout:3/764: write d6/d3b/d42/f51 [675112,5766] 0 2024-02-15T18:27:19.718 INFO:tasks.workunit.client.1.smithi191.stdout:6/962: dwrite d8/d9/d4a/d4c/fee [0,4194304] 0 2024-02-15T18:27:19.722 INFO:tasks.workunit.client.1.smithi191.stdout:6/963: write d8/d9/d4a/d4c/fc0 [3790684,11548] 0 2024-02-15T18:27:19.728 INFO:tasks.workunit.client.1.smithi191.stdout:7/891: creat d0/db/d26/d5f/d8c/f141 x:0 0 0 2024-02-15T18:27:19.731 INFO:tasks.workunit.client.1.smithi191.stdout:5/606: mknod d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/ccb 0 2024-02-15T18:27:19.732 INFO:tasks.workunit.client.1.smithi191.stdout:5/607: dread - d0/fa7 zero size 2024-02-15T18:27:19.732 INFO:tasks.workunit.client.1.smithi191.stdout:5/608: write d0/d18/d5c/f63 [4937444,101501] 0 2024-02-15T18:27:19.732 INFO:tasks.workunit.client.1.smithi191.stdout:5/609: fdatasync d0/d18/f42 0 2024-02-15T18:27:19.735 INFO:tasks.workunit.client.1.smithi191.stdout:8/982: rename d3/d2c/d1a/d23/d2a/f113 to d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d115/f150 0 2024-02-15T18:27:19.735 INFO:tasks.workunit.client.1.smithi191.stdout:8/983: read d3/d2c/d1a/d23/d2e/d64/f8d [154624,126560] 0 2024-02-15T18:27:19.735 INFO:tasks.workunit.client.1.smithi191.stdout:8/984: dread - d3/d2c/d1a/d23/d2e/d47/f12d zero size 2024-02-15T18:27:19.736 INFO:tasks.workunit.client.1.smithi191.stdout:8/985: readlink d3/d2c/d1a/d23/d3e/le1 0 2024-02-15T18:27:19.739 INFO:tasks.workunit.client.1.smithi191.stdout:1/977: mknod d0/d2f/d51/d5e/d68/c151 0 2024-02-15T18:27:19.756 INFO:tasks.workunit.client.1.smithi191.stdout:0/657: truncate d7/d1b/d30/f43 41170 0 2024-02-15T18:27:19.756 INFO:tasks.workunit.client.1.smithi191.stdout:0/658: truncate d7/d3b/f71 1554567 0 2024-02-15T18:27:19.757 INFO:tasks.workunit.client.1.smithi191.stdout:6/964: getdents d8/d9/dac/db0/d95/d9d 0 2024-02-15T18:27:19.760 INFO:tasks.workunit.client.1.smithi191.stdout:3/765: dwrite d6/d3b/f84 [0,4194304] 0 2024-02-15T18:27:19.760 INFO:tasks.workunit.client.1.smithi191.stdout:7/892: dwrite d0/db/d26/d5f/d77/df/f8e [0,4194304] 0 2024-02-15T18:27:19.760 INFO:tasks.workunit.client.1.smithi191.stdout:3/766: fdatasync d6/d1b/fa3 0 2024-02-15T18:27:19.760 INFO:tasks.workunit.client.1.smithi191.stdout:3/767: dread - d6/d3b/d30/fe4 zero size 2024-02-15T18:27:19.761 INFO:tasks.workunit.client.1.smithi191.stdout:5/610: unlink d0/d23/d41/d28/f4a 0 2024-02-15T18:27:19.762 INFO:tasks.workunit.client.1.smithi191.stdout:5/611: getdents d0/d18/d5c/d73/d7f 0 2024-02-15T18:27:19.763 INFO:tasks.workunit.client.1.smithi191.stdout:1/978: mknod d0/d8/df/d12/d2a/d2b/d78/c152 0 2024-02-15T18:27:19.763 INFO:tasks.workunit.client.1.smithi191.stdout:6/965: mkdir d8/d9/d4a/d4c/da4/d138 0 2024-02-15T18:27:19.768 INFO:tasks.workunit.client.1.smithi191.stdout:8/986: dwrite d3/d2c/d1a/d23/d2e/d5b/d69/d6d/fab [4194304,4194304] 0 2024-02-15T18:27:19.769 INFO:tasks.workunit.client.1.smithi191.stdout:7/893: rename d0/db/ddd/fe2 to d0/db/f142 0 2024-02-15T18:27:19.769 INFO:tasks.workunit.client.1.smithi191.stdout:3/768: creat d6/d27/d4d/d5a/d92/fff x:0 0 0 2024-02-15T18:27:19.771 INFO:tasks.workunit.client.1.smithi191.stdout:5/612: creat d0/d18/d5c/d7c/d49/d9f/dc0/fcc x:0 0 0 2024-02-15T18:27:19.772 INFO:tasks.workunit.client.1.smithi191.stdout:7/894: write d0/db/d65/dc/d4b/dc1/fc9 [1316640,21227] 0 2024-02-15T18:27:19.773 INFO:tasks.workunit.client.1.smithi191.stdout:7/895: fdatasync d0/db/d26/d5f/d77/db9/f127 0 2024-02-15T18:27:19.773 INFO:tasks.workunit.client.1.smithi191.stdout:7/896: chown d0/db/f12d 106012 1 2024-02-15T18:27:19.775 INFO:tasks.workunit.client.1.smithi191.stdout:1/979: dread d0/d8/df/d12/d2a/d2b/d78/d60/d109/f12b [0,4194304] 0 2024-02-15T18:27:19.779 INFO:tasks.workunit.client.1.smithi191.stdout:6/966: creat d8/d9/dac/f139 x:0 0 0 2024-02-15T18:27:19.782 INFO:tasks.workunit.client.1.smithi191.stdout:8/987: getdents d3/d2c/d1a/d23/d2e/d5b/d69/d6d 0 2024-02-15T18:27:19.782 INFO:tasks.workunit.client.1.smithi191.stdout:8/988: write d3/d2c/d1a/d23/d2e/d47/f63 [2193150,76040] 0 2024-02-15T18:27:19.783 INFO:tasks.workunit.client.1.smithi191.stdout:3/769: mkdir d6/d3b/d42/ddd/df6/d100 0 2024-02-15T18:27:19.783 INFO:tasks.workunit.client.1.smithi191.stdout:3/770: write d6/d27/f78 [260360,46602] 0 2024-02-15T18:27:19.785 INFO:tasks.workunit.client.1.smithi191.stdout:5/613: creat d0/d23/d41/d28/fcd x:0 0 0 2024-02-15T18:27:19.785 INFO:tasks.workunit.client.1.smithi191.stdout:5/614: fdatasync d0/d23/f43 0 2024-02-15T18:27:19.786 INFO:tasks.workunit.client.1.smithi191.stdout:0/659: dwrite d7/da/d1a/d2c/f32 [0,4194304] 0 2024-02-15T18:27:19.786 INFO:tasks.workunit.client.1.smithi191.stdout:0/660: chown d7/da/d17/d31/d7d/fdc 7 1 2024-02-15T18:27:19.788 INFO:tasks.workunit.client.1.smithi191.stdout:7/897: mkdir d0/db/d65/d11/d5a/d66/d1e/df5/d143 0 2024-02-15T18:27:19.788 INFO:tasks.workunit.client.1.smithi191.stdout:5/615: write d0/d23/d41/d28/d31/d5a/d61/f82 [1193952,74002] 0 2024-02-15T18:27:19.789 INFO:tasks.workunit.client.1.smithi191.stdout:5/616: read d0/d23/d41/d28/f98 [1071445,101211] 0 2024-02-15T18:27:19.789 INFO:tasks.workunit.client.1.smithi191.stdout:5/617: write d0/d23/d41/f9a [383957,70594] 0 2024-02-15T18:27:19.790 INFO:tasks.workunit.client.1.smithi191.stdout:1/980: creat d0/d8/df/d2e/f153 x:0 0 0 2024-02-15T18:27:19.790 INFO:tasks.workunit.client.1.smithi191.stdout:1/981: dread - d0/d8/df/ff8 zero size 2024-02-15T18:27:19.790 INFO:tasks.workunit.client.1.smithi191.stdout:1/982: stat d0/d8/df/d12/d2a/d2b/d78/d60/c92 0 2024-02-15T18:27:19.791 INFO:tasks.workunit.client.1.smithi191.stdout:6/967: mknod d8/d9/d4a/d4c/c13a 0 2024-02-15T18:27:19.795 INFO:tasks.workunit.client.1.smithi191.stdout:8/989: mkdir d3/d2c/d1a/d23/d151 0 2024-02-15T18:27:19.797 INFO:tasks.workunit.client.1.smithi191.stdout:0/661: dread d7/da/d17/d31/d5f/fd0 [0,4194304] 0 2024-02-15T18:27:19.798 INFO:tasks.workunit.client.1.smithi191.stdout:3/771: mknod d6/d3b/d42/d49/def/c101 0 2024-02-15T18:27:19.800 INFO:tasks.workunit.client.1.smithi191.stdout:7/898: creat d0/db/d65/d11/d1c/f144 x:0 0 0 2024-02-15T18:27:19.800 INFO:tasks.workunit.client.1.smithi191.stdout:7/899: stat d0/fc5 0 2024-02-15T18:27:19.800 INFO:tasks.workunit.client.1.smithi191.stdout:7/900: readlink d0/db/d65/d11/d5a/d66/dd6/l12b 0 2024-02-15T18:27:19.802 INFO:tasks.workunit.client.1.smithi191.stdout:1/983: rename d0/d8/df/d90/f143 to d0/d8/df/d2e/f154 0 2024-02-15T18:27:19.803 INFO:tasks.workunit.client.1.smithi191.stdout:6/968: mknod d8/d9/dac/ddb/c13b 0 2024-02-15T18:27:19.804 INFO:tasks.workunit.client.1.smithi191.stdout:1/984: read d0/ff1 [3178795,40157] 0 2024-02-15T18:27:19.808 INFO:tasks.workunit.client.1.smithi191.stdout:7/901: link d0/db/d65/dc/c7e d0/db/d65/d11/d88/d9d/c145 0 2024-02-15T18:27:19.808 INFO:tasks.workunit.client.1.smithi191.stdout:7/902: fsync d0/db/d65/d11/d5a/d66/d1e/f63 0 2024-02-15T18:27:19.810 INFO:tasks.workunit.client.1.smithi191.stdout:0/662: rename d7/d3b/c8d to d7/da/d1a/d34/d27/dab/cdf 0 2024-02-15T18:27:19.813 INFO:tasks.workunit.client.1.smithi191.stdout:6/969: symlink d8/d9/dac/db0/d95/d9d/d6b/d76/d110/l13c 0 2024-02-15T18:27:19.815 INFO:tasks.workunit.client.1.smithi191.stdout:0/663: mkdir d7/da/d1a/d2c/d4c/d98/de0 0 2024-02-15T18:27:19.818 INFO:tasks.workunit.client.1.smithi191.stdout:6/970: rename d8/d9/dac/db0/d95/d9d/d6b/d76/l11c to d8/d9/d4a/d4c/dc4/d12c/l13d 0 2024-02-15T18:27:19.819 INFO:tasks.workunit.client.1.smithi191.stdout:0/664: truncate d7/da/d1a/d34/f11 1231928 0 2024-02-15T18:27:19.819 INFO:tasks.workunit.client.1.smithi191.stdout:0/665: fdatasync d7/da/d17/d31/d5f/dae/fbe 0 2024-02-15T18:27:19.820 INFO:tasks.workunit.client.1.smithi191.stdout:7/903: write d0/db/d65/dc/d4b/dc1/fc9 [524768,50643] 0 2024-02-15T18:27:19.826 INFO:tasks.workunit.client.1.smithi191.stdout:8/990: dwrite d3/d2c/d1a/d23/d2e/d47/d5f/fe2 [0,4194304] 0 2024-02-15T18:27:19.826 INFO:tasks.workunit.client.1.smithi191.stdout:5/618: write d0/f1a [1496747,63008] 0 2024-02-15T18:27:19.827 INFO:tasks.workunit.client.1.smithi191.stdout:5/619: chown d0/d23/d41/d28/d31/d5a/d61/fbc 6 1 2024-02-15T18:27:19.827 INFO:tasks.workunit.client.1.smithi191.stdout:1/985: dwrite d0/f1b [0,4194304] 0 2024-02-15T18:27:19.827 INFO:tasks.workunit.client.1.smithi191.stdout:5/620: write d0/d18/d5c/d7c/f9c [293072,92149] 0 2024-02-15T18:27:19.829 INFO:tasks.workunit.client.1.smithi191.stdout:3/772: truncate d6/d3b/d42/fa1 203609 0 2024-02-15T18:27:19.830 INFO:tasks.workunit.client.1.smithi191.stdout:3/773: read d6/d27/d4d/d5a/da4/fbc [30303,42039] 0 2024-02-15T18:27:19.840 INFO:tasks.workunit.client.1.smithi191.stdout:6/971: mknod d8/d9/dac/db0/d95/d9d/d6b/c13e 0 2024-02-15T18:27:19.840 INFO:tasks.workunit.client.1.smithi191.stdout:6/972: readlink d8/d9/d4a/df6/lfc 0 2024-02-15T18:27:19.840 INFO:tasks.workunit.client.1.smithi191.stdout:6/973: fdatasync d8/d9/dac/db0/d95/d9d/d6b/d98/fef 0 2024-02-15T18:27:19.846 INFO:tasks.workunit.client.1.smithi191.stdout:0/666: truncate d7/d1b/f96 4050434 0 2024-02-15T18:27:19.847 INFO:tasks.workunit.client.1.smithi191.stdout:7/904: truncate d0/db/d26/d5f/d77/d35/d42/fc3 3241067 0 2024-02-15T18:27:19.849 INFO:tasks.workunit.client.1.smithi191.stdout:0/667: write d7/da/d1a/d2c/d4c/f9f [47239,31818] 0 2024-02-15T18:27:19.858 INFO:tasks.workunit.client.1.smithi191.stdout:7/905: dread d0/db/d26/d5f/d77/db9/f127 [0,4194304] 0 2024-02-15T18:27:19.862 INFO:tasks.workunit.client.1.smithi191.stdout:8/991: mkdir d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d9a/d152 0 2024-02-15T18:27:19.866 INFO:tasks.workunit.client.1.smithi191.stdout:1/986: mkdir d0/d2f/d155 0 2024-02-15T18:27:19.868 INFO:tasks.workunit.client.1.smithi191.stdout:5/621: mknod d0/d23/d41/d28/d31/dc2/cce 0 2024-02-15T18:27:19.870 INFO:tasks.workunit.client.1.smithi191.stdout:3/774: rename d6/d27/d4d/da8/fc4 to d6/d5d/daa/f102 0 2024-02-15T18:27:19.873 INFO:tasks.workunit.client.1.smithi191.stdout:6/974: mkdir d8/d9/d4a/d13f 0 2024-02-15T18:27:19.873 INFO:tasks.workunit.client.1.smithi191.stdout:6/975: chown d8/d9/dac/db0/d95/d9d/l28 0 1 2024-02-15T18:27:19.875 INFO:tasks.workunit.client.1.smithi191.stdout:0/668: mknod d7/da/d17/d31/d7d/ce1 0 2024-02-15T18:27:19.878 INFO:tasks.workunit.client.1.smithi191.stdout:8/992: dread d3/d2c/d1a/d10d/f140 [0,4194304] 0 2024-02-15T18:27:19.879 INFO:tasks.workunit.client.1.smithi191.stdout:1/987: creat d0/d8/df/d90/f156 x:0 0 0 2024-02-15T18:27:19.881 INFO:tasks.workunit.client.1.smithi191.stdout:7/906: dwrite d0/db/d26/d5f/d77/f13b [0,4194304] 0 2024-02-15T18:27:19.881 INFO:tasks.workunit.client.1.smithi191.stdout:3/775: write d6/d1b/d60/d73/f34 [119588,25367] 0 2024-02-15T18:27:19.883 INFO:tasks.workunit.client.1.smithi191.stdout:6/976: symlink d8/d9/d4a/l140 0 2024-02-15T18:27:19.883 INFO:tasks.workunit.client.1.smithi191.stdout:6/977: write d8/d9/d4a/d4c/da4/f120 [889646,114693] 0 2024-02-15T18:27:19.884 INFO:tasks.workunit.client.1.smithi191.stdout:0/669: mkdir d7/da/d17/d31/d5f/dae/de2 0 2024-02-15T18:27:19.886 INFO:tasks.workunit.client.1.smithi191.stdout:8/993: getdents d3/d2c/d1a/d23/d2e/d5b/d69 0 2024-02-15T18:27:19.888 INFO:tasks.workunit.client.1.smithi191.stdout:1/988: unlink d0/d8/df/d12/d2a/d2b/d78/d60/d109/dc8/f101 0 2024-02-15T18:27:19.888 INFO:tasks.workunit.client.1.smithi191.stdout:1/989: write d0/d2f/d5c/f71 [1263238,62616] 0 2024-02-15T18:27:19.888 INFO:tasks.workunit.client.1.smithi191.stdout:1/990: truncate d0/d2f/d51/d5e/fee 944041 0 2024-02-15T18:27:19.888 INFO:tasks.workunit.client.1.smithi191.stdout:1/991: dread - d0/d8/df/d90/f156 zero size 2024-02-15T18:27:19.888 INFO:tasks.workunit.client.1.smithi191.stdout:1/992: fsync d0/d8/df/d12/f28 0 2024-02-15T18:27:19.888 INFO:tasks.workunit.client.1.smithi191.stdout:5/622: dwrite d0/fa7 [0,4194304] 0 2024-02-15T18:27:19.891 INFO:tasks.workunit.client.1.smithi191.stdout:7/907: getdents d0/db/d65/dc/d4b/dc1 0 2024-02-15T18:27:19.891 INFO:tasks.workunit.client.1.smithi191.stdout:6/978: symlink d8/d9/d4a/d4c/dc4/d12c/l141 0 2024-02-15T18:27:19.892 INFO:tasks.workunit.client.1.smithi191.stdout:0/670: unlink d7/da/d17/d31/cb4 0 2024-02-15T18:27:19.895 INFO:tasks.workunit.client.1.smithi191.stdout:8/994: rename d3/d2c/d1a/d23/d2e/d5b/d69/cfa to d3/d2c/d1a/d23/d2e/d5b/d69/d11a/c153 0 2024-02-15T18:27:19.907 INFO:tasks.workunit.client.1.smithi191.stdout:3/776: dwrite d6/d3b/d42/dbb/feb [0,4194304] 0 2024-02-15T18:27:19.910 INFO:tasks.workunit.client.1.smithi191.stdout:5/623: getdents d0/d18/d5c/d7c/d49/d9f/dc0/d85 0 2024-02-15T18:27:19.912 INFO:tasks.workunit.client.1.smithi191.stdout:7/908: truncate d0/db/d26/d5f/d77/d35/d42/fa7 3761013 0 2024-02-15T18:27:19.912 INFO:tasks.workunit.client.1.smithi191.stdout:7/909: read d0/db/ff3 [542243,60674] 0 2024-02-15T18:27:19.913 INFO:tasks.workunit.client.1.smithi191.stdout:7/910: write d0/db/d65/dc/f12e [4576568,97327] 0 2024-02-15T18:27:19.915 INFO:tasks.workunit.client.1.smithi191.stdout:1/993: dwrite d0/ff1 [0,4194304] 0 2024-02-15T18:27:19.915 INFO:tasks.workunit.client.1.smithi191.stdout:0/671: rename d7/da/d1a/d34/d27/l2e to d7/le3 0 2024-02-15T18:27:19.916 INFO:tasks.workunit.client.1.smithi191.stdout:0/672: fsync d7/da/d17/d31/d5f/f61 0 2024-02-15T18:27:19.916 INFO:tasks.workunit.client.1.smithi191.stdout:0/673: chown d7/da/d17/d31/d7d 3 1 2024-02-15T18:27:19.916 INFO:tasks.workunit.client.1.smithi191.stdout:0/674: dread - d7/da/d17/d31/d7d/fcf zero size 2024-02-15T18:27:19.916 INFO:tasks.workunit.client.1.smithi191.stdout:8/995: symlink d3/d2c/d1a/db3/l154 0 2024-02-15T18:27:19.917 INFO:tasks.workunit.client.1.smithi191.stdout:6/979: dwrite d8/d9/d4a/d4c/dc4/ff7 [0,4194304] 0 2024-02-15T18:27:19.922 INFO:tasks.workunit.client.1.smithi191.stdout:5/624: rmdir d0/d18/d5c/d7c/d49/d9f/dc0 39 2024-02-15T18:27:19.922 INFO:tasks.workunit.client.1.smithi191.stdout:5/625: stat d0/d18/d5c/d7c/d49 0 2024-02-15T18:27:19.923 INFO:tasks.workunit.client.1.smithi191.stdout:5/626: fdatasync d0/d23/d41/d28/d31/d5a/d61/f95 0 2024-02-15T18:27:19.923 INFO:tasks.workunit.client.1.smithi191.stdout:5/627: readlink d0/d23/d41/d28/d31/d5a/d61/l6f 0 2024-02-15T18:27:19.948 INFO:tasks.workunit.client.1.smithi191.stdout:7/911: getdents d0/db/d65/dc/d6e 0 2024-02-15T18:27:19.969 INFO:tasks.workunit.client.1.smithi191.stdout:2/949: sync 2024-02-15T18:27:19.969 INFO:tasks.workunit.client.1.smithi191.stdout:9/897: sync 2024-02-15T18:27:19.970 INFO:tasks.workunit.client.1.smithi191.stdout:0/675: creat d7/d1b/fe4 x:0 0 0 2024-02-15T18:27:19.972 INFO:tasks.workunit.client.1.smithi191.stdout:8/996: symlink d3/d2c/l155 0 2024-02-15T18:27:19.973 INFO:tasks.workunit.client.1.smithi191.stdout:0/676: write d7/da/d1a/d34/f5e [2302439,116211] 0 2024-02-15T18:27:19.973 INFO:tasks.workunit.client.1.smithi191.stdout:0/677: chown d7/da/d17/l76 0 1 2024-02-15T18:27:19.977 INFO:tasks.workunit.client.1.smithi191.stdout:6/980: symlink d8/d9/dac/db0/d95/l142 0 2024-02-15T18:27:19.978 INFO:tasks.workunit.client.1.smithi191.stdout:6/981: chown d8/d9/d4a/df6/d102 38455456 1 2024-02-15T18:27:19.980 INFO:tasks.workunit.client.1.smithi191.stdout:7/912: creat d0/db/d26/d5f/d77/db9/d126/f146 x:0 0 0 2024-02-15T18:27:19.980 INFO:tasks.workunit.client.1.smithi191.stdout:7/913: read d0/db/d65/d11/d5a/d66/d1e/f63 [1425,95238] 0 2024-02-15T18:27:19.982 INFO:tasks.workunit.client.1.smithi191.stdout:1/994: truncate d0/d2f/d5c/f71 1255960 0 2024-02-15T18:27:19.982 INFO:tasks.workunit.client.1.smithi191.stdout:1/995: dread - d0/d8/d48/f117 zero size 2024-02-15T18:27:19.982 INFO:tasks.workunit.client.1.smithi191.stdout:1/996: chown d0/d8/df/d2e/c39 26 1 2024-02-15T18:27:19.982 INFO:tasks.workunit.client.1.smithi191.stdout:9/898: mkdir d0/d7/dc/d32/de4/d127 0 2024-02-15T18:27:19.984 INFO:tasks.workunit.client.1.smithi191.stdout:8/997: rename d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/d82/d115/f149 to d3/f156 0 2024-02-15T18:27:19.985 INFO:tasks.workunit.client.1.smithi191.stdout:8/998: chown d3/d2c/d1a/d23/d2e/d47/d5f/ff3 49658 1 2024-02-15T18:27:19.985 INFO:tasks.workunit.client.1.smithi191.stdout:8/999: read - d3/d2c/d1a/d23/d2e/d47/d5f/d4f/d67/f121 zero size 2024-02-15T18:27:19.986 INFO:tasks.workunit.client.1.smithi191.stdout:2/950: dwrite df/d12/f7c [0,4194304] 0 2024-02-15T18:27:19.987 INFO:tasks.workunit.client.1.smithi191.stdout:2/951: write df/d12/d19/d1a/f5e [2195254,70308] 0 2024-02-15T18:27:19.987 INFO:tasks.workunit.client.1.smithi191.stdout:0/678: mknod d7/d94/ce5 0 2024-02-15T18:27:19.988 INFO:tasks.workunit.client.1.smithi191.stdout:0/679: write d7/da/d6c/f93 [48764,119095] 0 2024-02-15T18:27:19.989 INFO:tasks.workunit.client.1.smithi191.stdout:5/628: dread d0/d23/d41/d28/d31/f53 [0,4194304] 0 2024-02-15T18:27:19.994 INFO:tasks.workunit.client.1.smithi191.stdout:7/914: creat d0/db/d26/d5f/d77/d35/d42/d104/f147 x:0 0 0 2024-02-15T18:27:19.995 INFO:tasks.workunit.client.1.smithi191.stdout:1/997: unlink d0/l1e 0 2024-02-15T18:27:19.997 INFO:tasks.workunit.client.1.smithi191.stdout:9/899: creat d0/d29/d48/f128 x:0 0 0 2024-02-15T18:27:20.001 INFO:tasks.workunit.client.1.smithi191.stdout:6/982: mknod d8/d9/d2b/c143 0 2024-02-15T18:27:20.002 INFO:tasks.workunit.client.1.smithi191.stdout:6/983: stat d8/d9/dac/db0/fb7 0 2024-02-15T18:27:20.002 INFO:tasks.workunit.client.1.smithi191.stdout:0/680: creat d7/da/d17/d31/d5f/dae/dd3/fe6 x:0 0 0 2024-02-15T18:27:20.003 INFO:tasks.workunit.client.1.smithi191.stdout:5/629: rmdir d0/d18/d5c/d7c/d49 39 2024-02-15T18:27:20.005 INFO:tasks.workunit.client.1.smithi191.stdout:2/952: dwrite df/d12/d19/d1a/d35/f80 [0,4194304] 0 2024-02-15T18:27:20.005 INFO:tasks.workunit.client.1.smithi191.stdout:2/953: dread - df/d12/d19/d1a/d35/d5b/dbf/f9d zero size 2024-02-15T18:27:20.005 INFO:tasks.workunit.client.1.smithi191.stdout:7/915: rename d0/db/d65/ca3 to d0/db/d65/dc/d4b/c148 0 2024-02-15T18:27:20.006 INFO:tasks.workunit.client.1.smithi191.stdout:7/916: chown d0/db/d65/dc/d4b/f7f 477584488 1 2024-02-15T18:27:20.012 INFO:tasks.workunit.client.1.smithi191.stdout:6/984: creat d8/d9/dac/db0/d95/f144 x:0 0 0 2024-02-15T18:27:20.012 INFO:tasks.workunit.client.1.smithi191.stdout:6/985: dread - d8/d9/d4a/d4c/dc4/f104 zero size 2024-02-15T18:27:20.013 INFO:tasks.workunit.client.1.smithi191.stdout:9/900: dwrite d0/d7/dc/d17/d44/fd8 [0,4194304] 0 2024-02-15T18:27:20.013 INFO:tasks.workunit.client.1.smithi191.stdout:0/681: unlink d7/da/d1a/d34/f6d 0 2024-02-15T18:27:20.017 INFO:tasks.workunit.client.1.smithi191.stdout:1/998: rename d0/d8/df/d12/d2a/d2b/d78/d60/d74/daa/f100 to d0/d8/df/d12/d2a/f157 0 2024-02-15T18:27:20.017 INFO:tasks.workunit.client.1.smithi191.stdout:1/999: read d0/d8/df/d2e/f8d [819088,6244] 0 2024-02-15T18:27:20.019 INFO:tasks.workunit.client.1.smithi191.stdout:0/682: dread d7/da/d17/d31/d5f/fd0 [0,4194304] 0 2024-02-15T18:27:20.019 INFO:tasks.workunit.client.1.smithi191.stdout:0/683: chown d7/d1b/c5a 33 1 2024-02-15T18:27:20.020 INFO:tasks.workunit.client.1.smithi191.stdout:7/917: creat d0/db/d26/d5f/d77/db9/d126/f149 x:0 0 0 2024-02-15T18:27:20.020 INFO:tasks.workunit.client.1.smithi191.stdout:7/918: chown d0/db/cdc 408326 1 2024-02-15T18:27:20.023 INFO:tasks.workunit.client.1.smithi191.stdout:9/901: symlink d0/d7/dc/d32/d106/dce/l129 0 2024-02-15T18:27:20.023 INFO:tasks.workunit.client.1.smithi191.stdout:5/630: dwrite d0/d23/d41/d28/fcd [0,4194304] 0 2024-02-15T18:27:20.024 INFO:tasks.workunit.client.1.smithi191.stdout:5/631: write d0/d23/d41/d28/d31/d5a/d61/f82 [1833245,99223] 0 2024-02-15T18:27:20.026 INFO:tasks.workunit.client.1.smithi191.stdout:2/954: rename df/d12/d19/d1a/d35/d47/c63 to df/d12/d32/d4d/d89/d11a/c136 0 2024-02-15T18:27:20.035 INFO:tasks.workunit.client.1.smithi191.stdout:6/986: write d8/d9/d2b/f42 [106953,6621] 0 2024-02-15T18:27:20.035 INFO:tasks.workunit.client.1.smithi191.stdout:7/919: truncate d0/db/d26/d5f/d77/f6a 874175 0 2024-02-15T18:27:20.036 INFO:tasks.workunit.client.1.smithi191.stdout:0/684: dwrite d7/d4a/fd4 [0,4194304] 0 2024-02-15T18:27:20.036 INFO:tasks.workunit.client.1.smithi191.stdout:0/685: read - d7/d94/fad zero size 2024-02-15T18:27:20.040 INFO:tasks.workunit.client.1.smithi191.stdout:2/955: link df/d12/d32/d4d/d89/d11a/d12a/l56 df/d12/d1e/d6f/l137 0 2024-02-15T18:27:20.044 INFO:tasks.workunit.client.1.smithi191.stdout:0/686: dread d7/f6a [0,4194304] 0 2024-02-15T18:27:20.049 INFO:tasks.workunit.client.1.smithi191.stdout:5/632: rename d0/d18/d5c/d7c/d49/d9f/dc0/fcc to d0/d23/d41/d28/fcf 0 2024-02-15T18:27:20.050 INFO:tasks.workunit.client.1.smithi191.stdout:2/956: mknod df/d61/d94/c138 0 2024-02-15T18:27:20.052 INFO:tasks.workunit.client.1.smithi191.stdout:7/920: dwrite d0/db/d65/dc/fee [0,4194304] 0 2024-02-15T18:27:20.052 INFO:tasks.workunit.client.1.smithi191.stdout:6/987: rename d8/d9/d4a/c94 to d8/d9/dac/db0/d95/d9d/d6b/d76/d110/c145 0 2024-02-15T18:27:20.054 INFO:tasks.workunit.client.1.smithi191.stdout:5/633: rmdir d0/d23/d41/d28/d31/d5a/d61 39 2024-02-15T18:27:20.055 INFO:tasks.workunit.client.1.smithi191.stdout:9/902: write d0/d7/dc/d30/d4f/f6d [1628418,7004] 0 2024-02-15T18:27:20.059 INFO:tasks.workunit.client.1.smithi191.stdout:7/921: truncate d0/db/d65/f75 3969541 0 2024-02-15T18:27:20.059 INFO:tasks.workunit.client.1.smithi191.stdout:7/922: fsync d0/db/d26/d5f/d77/db9/ffc 0 2024-02-15T18:27:20.059 INFO:tasks.workunit.client.1.smithi191.stdout:7/923: fdatasync d0/db/d26/d5f/d77/d35/dc7/ff7 0 2024-02-15T18:27:20.059 INFO:tasks.workunit.client.1.smithi191.stdout:7/924: fdatasync d0/db/d65/d11/d88/d9d/dd2/f95 0 2024-02-15T18:27:20.060 INFO:tasks.workunit.client.1.smithi191.stdout:6/988: creat d8/d9/d2b/d84/f146 x:0 0 0 2024-02-15T18:27:20.061 INFO:tasks.workunit.client.1.smithi191.stdout:6/989: dread - d8/d9/ffa zero size 2024-02-15T18:27:20.062 INFO:tasks.workunit.client.1.smithi191.stdout:7/925: rename d0/db/d65/d11/d57/cc6 to d0/db/c14a 0 2024-02-15T18:27:20.063 INFO:tasks.workunit.client.1.smithi191.stdout:7/926: truncate d0/db/d65/dc/d4b/f7f 1253250 0 2024-02-15T18:27:20.067 INFO:tasks.workunit.client.1.smithi191.stdout:7/927: mknod d0/db/d65/d11/d5a/d66/d1e/df5/d143/c14b 0 2024-02-15T18:27:20.067 INFO:tasks.workunit.client.1.smithi191.stdout:0/687: dwrite d7/da/d17/d31/f36 [0,4194304] 0 2024-02-15T18:27:20.073 INFO:tasks.workunit.client.1.smithi191.stdout:0/688: symlink d7/da/d1a/d34/d27/le7 0 2024-02-15T18:27:20.077 INFO:tasks.workunit.client.1.smithi191.stdout:0/689: rename d7/da/d1a/d2c/d4c/d98/de0 to d7/da/d1a/d2c/d4c/de8 0 2024-02-15T18:27:20.080 INFO:tasks.workunit.client.1.smithi191.stdout:5/634: dwrite d0/d23/d41/d28/d31/f53 [0,4194304] 0 2024-02-15T18:27:20.082 INFO:tasks.workunit.client.1.smithi191.stdout:0/690: write d7/f6a [7300362,69897] 0 2024-02-15T18:27:20.082 INFO:tasks.workunit.client.1.smithi191.stdout:0/691: truncate d7/da/d17/d31/d5f/f61 718371 0 2024-02-15T18:27:20.085 INFO:tasks.workunit.client.1.smithi191.stdout:6/990: dwrite d8/d9/d2b/d84/f146 [0,4194304] 0 2024-02-15T18:27:20.086 INFO:tasks.workunit.client.1.smithi191.stdout:6/991: stat d8/d9/dac/db0/d95/d9d/f7a 0 2024-02-15T18:27:20.092 INFO:tasks.workunit.client.1.smithi191.stdout:0/692: symlink d7/da/d1a/d34/d45/le9 0 2024-02-15T18:27:20.092 INFO:tasks.workunit.client.1.smithi191.stdout:0/693: write d7/da/d17/d31/d7d/fdc [214850,30592] 0 2024-02-15T18:27:20.093 INFO:tasks.workunit.client.1.smithi191.stdout:0/694: truncate d7/da/d17/d31/d7d/f80 414117 0 2024-02-15T18:27:20.093 INFO:tasks.workunit.client.1.smithi191.stdout:0/695: read d7/da/d6c/fbf [212672,18154] 0 2024-02-15T18:27:20.093 INFO:tasks.workunit.client.1.smithi191.stdout:0/696: chown d7/d3b 1 1 2024-02-15T18:27:20.095 INFO:tasks.workunit.client.1.smithi191.stdout:6/992: mknod d8/d9/dac/ddb/c147 0 2024-02-15T18:27:20.098 INFO:tasks.workunit.client.1.smithi191.stdout:6/993: truncate d8/d9/d2b/d57/f60 1360510 0 2024-02-15T18:27:20.100 INFO:tasks.workunit.client.1.smithi191.stdout:0/697: dread d7/da/d1a/d2c/f4b [0,4194304] 0 2024-02-15T18:27:20.100 INFO:tasks.workunit.client.1.smithi191.stdout:6/994: link d8/d9/ffa d8/d9/d4a/df6/d133/f148 0 2024-02-15T18:27:20.102 INFO:tasks.workunit.client.1.smithi191.stdout:9/903: dwrite d0/d1d/d3f/d80/f75 [0,4194304] 0 2024-02-15T18:27:20.105 INFO:tasks.workunit.client.1.smithi191.stdout:5/635: dwrite d0/d23/d41/d28/f98 [0,4194304] 0 2024-02-15T18:27:20.107 INFO:tasks.workunit.client.1.smithi191.stdout:6/995: write d8/d9/dac/db0/d95/d9d/f38 [2464245,12726] 0 2024-02-15T18:27:20.107 INFO:tasks.workunit.client.1.smithi191.stdout:6/996: write d8/d9/dac/db0/d95/d9d/f62 [1295747,27144] 0 2024-02-15T18:27:20.109 INFO:tasks.workunit.client.1.smithi191.stdout:7/928: dwrite d0/db/d26/d5f/d77/d35/dc7/ff7 [0,4194304] 0 2024-02-15T18:27:20.109 INFO:tasks.workunit.client.1.smithi191.stdout:7/929: stat d0/db/d65/dc 0 2024-02-15T18:27:20.109 INFO:tasks.workunit.client.1.smithi191.stdout:6/997: mknod d8/d9/d2b/d81/c149 0 2024-02-15T18:27:20.109 INFO:tasks.workunit.client.1.smithi191.stdout:0/698: dread d7/fb [4194304,4194304] 0 2024-02-15T18:27:20.120 INFO:tasks.workunit.client.1.smithi191.stdout:7/930: truncate d0/db/d65/d11/d88/f125 56980 0 2024-02-15T18:27:20.120 INFO:tasks.workunit.client.1.smithi191.stdout:7/931: chown d0 10 1 2024-02-15T18:27:20.121 INFO:tasks.workunit.client.1.smithi191.stdout:5/636: dwrite d0/d18/d5c/d7c/f67 [0,4194304] 0 2024-02-15T18:27:20.122 INFO:tasks.workunit.client.1.smithi191.stdout:6/998: unlink d8/c85 0 2024-02-15T18:27:20.123 INFO:tasks.workunit.client.1.smithi191.stdout:6/999: write d8/d9/d4a/d4c/dc4/d12c/f136 [4838879,15625] 0 2024-02-15T18:27:20.123 INFO:tasks.workunit.client.1.smithi191.stdout:0/699: getdents d7/da/d17/daa 0 2024-02-15T18:27:20.129 INFO:tasks.workunit.client.1.smithi191.stdout:5/637: dread d0/d18/d5c/f71 [0,4194304] 0 2024-02-15T18:27:20.129 INFO:tasks.workunit.client.1.smithi191.stdout:0/700: rmdir d7 39 2024-02-15T18:27:20.134 INFO:tasks.workunit.client.1.smithi191.stdout:0/701: dread - d7/da/d1a/d34/fc6 zero size 2024-02-15T18:27:20.134 INFO:tasks.workunit.client.1.smithi191.stdout:5/638: write d0/d23/f43 [1408246,56538] 0 2024-02-15T18:27:20.135 INFO:tasks.workunit.client.1.smithi191.stdout:0/702: write d7/d4a/f85 [818599,113894] 0 2024-02-15T18:27:20.136 INFO:tasks.workunit.client.1.smithi191.stdout:7/932: dwrite d0/f8 [0,4194304] 0 2024-02-15T18:27:20.137 INFO:tasks.workunit.client.1.smithi191.stdout:5/639: mknod d0/d18/d5c/d7c/d49/d9f/dc0/cd0 0 2024-02-15T18:27:20.137 INFO:tasks.workunit.client.1.smithi191.stdout:5/640: chown d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/f40 246404 1 2024-02-15T18:27:20.137 INFO:tasks.workunit.client.1.smithi191.stdout:5/641: chown d0/d23/d41/d3c 1524 1 2024-02-15T18:27:20.137 INFO:tasks.workunit.client.1.smithi191.stdout:0/703: rmdir d7/da/d1a/d34/d27/dab 39 2024-02-15T18:27:20.139 INFO:tasks.workunit.client.1.smithi191.stdout:7/933: mknod d0/db/d26/d5f/d77/db9/c14c 0 2024-02-15T18:27:20.142 INFO:tasks.workunit.client.1.smithi191.stdout:5/642: mknod d0/d18/d5c/d7c/cd1 0 2024-02-15T18:27:20.143 INFO:tasks.workunit.client.1.smithi191.stdout:0/704: symlink d7/da/d17/d31/d5f/dae/dd3/lea 0 2024-02-15T18:27:20.147 INFO:tasks.workunit.client.1.smithi191.stdout:5/643: link d0/f1b d0/d18/d5c/fd2 0 2024-02-15T18:27:20.151 INFO:tasks.workunit.client.1.smithi191.stdout:5/644: creat d0/d62/fd3 x:0 0 0 2024-02-15T18:27:20.152 INFO:tasks.workunit.client.1.smithi191.stdout:5/645: write d0/d23/d41/d3c/fc9 [748989,25855] 0 2024-02-15T18:27:20.155 INFO:tasks.workunit.client.1.smithi191.stdout:5/646: rename d0/d18/d5c/d7c/lc5 to d0/d18/d5c/d7c/d49/d9f/dbd/ld4 0 2024-02-15T18:27:20.169 INFO:tasks.workunit.client.1.smithi191.stdout:5/647: dread d0/d23/d41/d28/f36 [0,4194304] 0 2024-02-15T18:27:20.169 INFO:tasks.workunit.client.1.smithi191.stdout:7/934: dwrite d0/db/d26/fb2 [0,4194304] 0 2024-02-15T18:27:20.169 INFO:tasks.workunit.client.1.smithi191.stdout:0/705: dwrite d7/f6a [4194304,4194304] 0 2024-02-15T18:27:20.169 INFO:tasks.workunit.client.1.smithi191.stdout:5/648: readlink d0/d18/d5c/d7c/d49/d9f/dbd/ld4 0 2024-02-15T18:27:20.169 INFO:tasks.workunit.client.1.smithi191.stdout:0/706: truncate d7/da/d17/d31/d5f/fbc 117070 0 2024-02-15T18:27:20.175 INFO:tasks.workunit.client.1.smithi191.stdout:7/935: mknod d0/db/d65/dc/d4b/dc1/def/c14d 0 2024-02-15T18:27:20.176 INFO:tasks.workunit.client.1.smithi191.stdout:5/649: creat d0/d23/d41/d3c/fd5 x:0 0 0 2024-02-15T18:27:20.179 INFO:tasks.workunit.client.1.smithi191.stdout:0/707: creat d7/d94/feb x:0 0 0 2024-02-15T18:27:20.180 INFO:tasks.workunit.client.1.smithi191.stdout:7/936: link d0/db/d65/d11/d5a/d66/d1e/f9b d0/db/d26/d5f/d77/d35/d42/f14e 0 2024-02-15T18:27:20.181 INFO:tasks.workunit.client.1.smithi191.stdout:0/708: mkdir d7/da/d17/d31/d5f/dae/de2/dec 0 2024-02-15T18:27:20.181 INFO:tasks.workunit.client.1.smithi191.stdout:0/709: fsync d7/d4a/dc0/fda 0 2024-02-15T18:27:20.181 INFO:tasks.workunit.client.1.smithi191.stdout:0/710: write d7/da/d1a/d34/d27/f7c [1184109,75294] 0 2024-02-15T18:27:20.183 INFO:tasks.workunit.client.1.smithi191.stdout:7/937: getdents d0/db/d26/d5f/d8c 0 2024-02-15T18:27:20.187 INFO:tasks.workunit.client.1.smithi191.stdout:0/711: rename d7/da/d17/d31/d5f/d9e/dc3/lc5 to d7/da/d17/d31/d5f/dae/de2/dec/led 0 2024-02-15T18:27:20.188 INFO:tasks.workunit.client.1.smithi191.stdout:0/712: dread d7/d1b/f81 [0,4194304] 0 2024-02-15T18:27:20.188 INFO:tasks.workunit.client.1.smithi191.stdout:0/713: chown d7/d3b/l77 6362 1 2024-02-15T18:27:20.195 INFO:tasks.workunit.client.1.smithi191.stdout:0/714: mknod d7/da/d17/daa/cee 0 2024-02-15T18:27:20.195 INFO:tasks.workunit.client.1.smithi191.stdout:0/715: dread - d7/da/d1a/d34/fc6 zero size 2024-02-15T18:27:20.196 INFO:tasks.workunit.client.1.smithi191.stdout:7/938: dwrite d0/db/d26/d5f/d77/df/fd3 [0,4194304] 0 2024-02-15T18:27:20.196 INFO:tasks.workunit.client.1.smithi191.stdout:0/716: mknod d7/da/d1a/d2c/d4c/cef 0 2024-02-15T18:27:20.199 INFO:tasks.workunit.client.1.smithi191.stdout:7/939: creat d0/db/d26/d5f/f14f x:0 0 0 2024-02-15T18:27:20.204 INFO:tasks.workunit.client.1.smithi191.stdout:0/717: getdents d7/da/d17 0 2024-02-15T18:27:20.206 INFO:tasks.workunit.client.1.smithi191.stdout:7/940: creat d0/db/d65/dc/d120/f150 x:0 0 0 2024-02-15T18:27:20.207 INFO:tasks.workunit.client.1.smithi191.stdout:7/941: getdents d0/db/d65/dc/d4b/dc1/def 0 2024-02-15T18:27:20.208 INFO:tasks.workunit.client.1.smithi191.stdout:7/942: chown d0/db/d26/d5f/d77/d35/d42 0 1 2024-02-15T18:27:20.208 INFO:tasks.workunit.client.1.smithi191.stdout:7/943: write d0/db/d65/d11/d5a/d66/f53 [1404840,73485] 0 2024-02-15T18:27:20.208 INFO:tasks.workunit.client.1.smithi191.stdout:7/944: getdents d0/db/d65/dc/d6e/d12f 0 2024-02-15T18:27:20.208 INFO:tasks.workunit.client.1.smithi191.stdout:7/945: write d0/db/d65/d11/d88/d9d/dd2/fbe [95137,54749] 0 2024-02-15T18:27:20.214 INFO:tasks.workunit.client.1.smithi191.stdout:7/946: dread d0/db/d26/d5f/d77/d35/d42/fc3 [0,4194304] 0 2024-02-15T18:27:20.214 INFO:tasks.workunit.client.1.smithi191.stdout:7/947: dread - d0/fc5 zero size 2024-02-15T18:27:20.215 INFO:tasks.workunit.client.1.smithi191.stdout:7/948: write d0/db/d65/d11/d88/d9d/fda [1941452,104646] 0 2024-02-15T18:27:20.227 INFO:tasks.workunit.client.1.smithi191.stdout:7/949: dwrite d0/db/d26/d5f/d77/df/f13c [0,4194304] 0 2024-02-15T18:27:20.228 INFO:tasks.workunit.client.1.smithi191.stdout:7/950: write d0/db/d65/dc/fb0 [1396604,66448] 0 2024-02-15T18:27:20.235 INFO:tasks.workunit.client.1.smithi191.stdout:7/951: creat d0/db/d26/d5f/d77/d35/f151 x:0 0 0 2024-02-15T18:27:20.235 INFO:tasks.workunit.client.1.smithi191.stdout:7/952: write d0/db/d65/dc/d120/f150 [938313,106551] 0 2024-02-15T18:27:20.235 INFO:tasks.workunit.client.1.smithi191.stdout:7/953: truncate d0/db/d26/d5f/d8c/f13f 432006 0 2024-02-15T18:27:20.236 INFO:tasks.workunit.client.1.smithi191.stdout:7/954: truncate d0/db/d65/d11/d5a/d66/d1e/df5/f4c 4067560 0 2024-02-15T18:27:20.236 INFO:tasks.workunit.client.1.smithi191.stdout:7/955: dread - d0/db/d65/d11/d57/fde zero size 2024-02-15T18:27:20.542 INFO:tasks.workunit.client.1.smithi191.stdout:3/777: sync 2024-02-15T18:27:20.542 INFO:tasks.workunit.client.1.smithi191.stdout:3/778: chown d6/d5d/d72/c7d 18951 1 2024-02-15T18:27:20.542 INFO:tasks.workunit.client.1.smithi191.stdout:3/779: chown d6/d1b/l7b 1 1 2024-02-15T18:27:20.557 INFO:tasks.workunit.client.1.smithi191.stdout:3/780: dwrite d6/d5d/daa/fb9 [0,4194304] 0 2024-02-15T18:27:20.559 INFO:tasks.workunit.client.1.smithi191.stdout:3/781: creat d6/d5d/d72/f103 x:0 0 0 2024-02-15T18:27:20.567 INFO:tasks.workunit.client.1.smithi191.stdout:3/782: dread d6/d3b/d42/d49/d54/f83 [0,4194304] 0 2024-02-15T18:27:20.570 INFO:tasks.workunit.client.1.smithi191.stdout:3/783: write d6/d27/fa5 [11942,8562] 0 2024-02-15T18:27:20.570 INFO:tasks.workunit.client.1.smithi191.stdout:3/784: stat d6/d3b/d42/l8a 0 2024-02-15T18:27:20.570 INFO:tasks.workunit.client.1.smithi191.stdout:3/785: write d6/d3b/d42/f51 [4221496,71487] 0 2024-02-15T18:27:20.572 INFO:tasks.workunit.client.1.smithi191.stdout:3/786: unlink d6/d3b/d42/d49/d54/c5b 0 2024-02-15T18:27:20.572 INFO:tasks.workunit.client.1.smithi191.stdout:3/787: stat d6/d3b/d30/fe4 0 2024-02-15T18:27:20.573 INFO:tasks.workunit.client.1.smithi191.stdout:3/788: rmdir d6/d27/d4d/d5a/d92/dd2 39 2024-02-15T18:27:20.574 INFO:tasks.workunit.client.1.smithi191.stdout:3/789: dread - d6/d27/d4d/d5a/fd1 zero size 2024-02-15T18:27:20.574 INFO:tasks.workunit.client.1.smithi191.stdout:3/790: stat d6/d1b/lf1 0 2024-02-15T18:27:20.591 INFO:tasks.workunit.client.1.smithi191.stdout:3/791: dwrite d6/d3b/d42/dbb/feb [0,4194304] 0 2024-02-15T18:27:20.595 INFO:tasks.workunit.client.1.smithi191.stdout:3/792: link d6/d5d/ld0 d6/d1b/d60/ddc/l104 0 2024-02-15T18:27:20.597 INFO:tasks.workunit.client.1.smithi191.stdout:3/793: rename d6/d27/d4d/d5a/d92/fff to d6/d3b/d42/d49/dbf/df7/f105 0 2024-02-15T18:27:20.599 INFO:tasks.workunit.client.1.smithi191.stdout:3/794: mkdir d6/d27/d4d/da8/d106 0 2024-02-15T18:27:20.600 INFO:tasks.workunit.client.1.smithi191.stdout:3/795: getdents d6/d27/d4d/d5a/da4 0 2024-02-15T18:27:20.600 INFO:tasks.workunit.client.1.smithi191.stdout:3/796: dread - d6/d3b/d42/d49/dbf/df7/f105 zero size 2024-02-15T18:27:20.603 INFO:tasks.workunit.client.1.smithi191.stdout:3/797: chown d6/d1b/d60/d73/c28 3118143 1 2024-02-15T18:27:20.606 INFO:tasks.workunit.client.1.smithi191.stdout:3/798: mkdir d6/d5d/df3/d107 0 2024-02-15T18:27:20.625 INFO:tasks.workunit.client.1.smithi191.stdout:3/799: dwrite d6/d1b/d60/d73/f34 [0,4194304] 0 2024-02-15T18:27:20.625 INFO:tasks.workunit.client.1.smithi191.stdout:3/800: chown d6/d3b/d42/f51 156700 1 2024-02-15T18:27:20.643 INFO:tasks.workunit.client.1.smithi191.stdout:3/801: dwrite d6/d3b/f84 [0,4194304] 0 2024-02-15T18:27:20.649 INFO:tasks.workunit.client.1.smithi191.stdout:3/802: dread d6/d1b/d60/d73/f74 [0,4194304] 0 2024-02-15T18:27:20.651 INFO:tasks.workunit.client.1.smithi191.stdout:3/803: truncate d6/d1b/d60/d73/f41 2309721 0 2024-02-15T18:27:20.653 INFO:tasks.workunit.client.1.smithi191.stdout:3/804: mknod d6/d3b/d42/d49/c108 0 2024-02-15T18:27:20.654 INFO:tasks.workunit.client.1.smithi191.stdout:3/805: read d6/d3b/d42/fa1 [121133,100997] 0 2024-02-15T18:27:20.654 INFO:tasks.workunit.client.1.smithi191.stdout:3/806: fsync d6/d3b/f84 0 2024-02-15T18:27:20.662 INFO:tasks.workunit.client.1.smithi191.stdout:7/956: sync 2024-02-15T18:27:20.662 INFO:tasks.workunit.client.1.smithi191.stdout:2/957: sync 2024-02-15T18:27:20.662 INFO:tasks.workunit.client.1.smithi191.stdout:9/904: sync 2024-02-15T18:27:20.662 INFO:tasks.workunit.client.1.smithi191.stdout:0/718: sync 2024-02-15T18:27:20.662 INFO:tasks.workunit.client.1.smithi191.stdout:5/650: sync 2024-02-15T18:27:20.663 INFO:tasks.workunit.client.1.smithi191.stdout:3/807: dread f4 [4194304,4194304] 0 2024-02-15T18:27:20.670 INFO:tasks.workunit.client.1.smithi191.stdout:2/958: dread df/d12/d32/d4d/d89/d11a/d12a/f123 [0,4194304] 0 2024-02-15T18:27:20.670 INFO:tasks.workunit.client.1.smithi191.stdout:0/719: creat d7/da/d17/d31/d5f/dae/de2/ff0 x:0 0 0 2024-02-15T18:27:20.670 INFO:tasks.workunit.client.1.smithi191.stdout:9/905: creat d0/d1d/f12a x:0 0 0 2024-02-15T18:27:20.670 INFO:tasks.workunit.client.1.smithi191.stdout:2/959: dread - df/d12/d19/d1a/d52/df2/ff8 zero size 2024-02-15T18:27:20.670 INFO:tasks.workunit.client.1.smithi191.stdout:2/960: truncate df/d61/d9f/da3/fb3 4285361 0 2024-02-15T18:27:20.674 INFO:tasks.workunit.client.1.smithi191.stdout:3/808: mknod d6/d27/d4d/d5a/d92/c109 0 2024-02-15T18:27:20.674 INFO:tasks.workunit.client.1.smithi191.stdout:5/651: symlink d0/d18/d5c/d7c/d49/ld6 0 2024-02-15T18:27:20.676 INFO:tasks.workunit.client.1.smithi191.stdout:9/906: creat d0/d7/dc/d30/d3b/d8b/f12b x:0 0 0 2024-02-15T18:27:20.678 INFO:tasks.workunit.client.1.smithi191.stdout:2/961: mkdir df/d12/db4/d130/d139 0 2024-02-15T18:27:20.679 INFO:tasks.workunit.client.1.smithi191.stdout:3/809: creat d6/d1b/d60/d73/d5e/f10a x:0 0 0 2024-02-15T18:27:20.679 INFO:tasks.workunit.client.1.smithi191.stdout:5/652: creat d0/d23/d41/d3c/fd7 x:0 0 0 2024-02-15T18:27:20.681 INFO:tasks.workunit.client.1.smithi191.stdout:9/907: creat d0/f12c x:0 0 0 2024-02-15T18:27:20.685 INFO:tasks.workunit.client.1.smithi191.stdout:3/810: rename d6/fb4 to d6/d27/d4d/d5a/d92/dd2/f10b 0 2024-02-15T18:27:20.686 INFO:tasks.workunit.client.1.smithi191.stdout:5/653: creat d0/d18/d5c/fd8 x:0 0 0 2024-02-15T18:27:20.693 INFO:tasks.workunit.client.1.smithi191.stdout:0/720: dwrite d7/d1b/f23 [0,4194304] 0 2024-02-15T18:27:20.693 INFO:tasks.workunit.client.1.smithi191.stdout:0/721: chown d7/d1b/f46 96 1 2024-02-15T18:27:20.696 INFO:tasks.workunit.client.1.smithi191.stdout:0/722: rename d7/da/d17/l76 to d7/da/d1a/d2c/d4c/de8/lf1 0 2024-02-15T18:27:20.696 INFO:tasks.workunit.client.1.smithi191.stdout:0/723: write d7/da/d17/d31/d5f/f61 [1279090,15147] 0 2024-02-15T18:27:20.699 INFO:tasks.workunit.client.1.smithi191.stdout:5/654: dread d0/d23/d41/d28/d31/f53 [0,4194304] 0 2024-02-15T18:27:20.700 INFO:tasks.workunit.client.1.smithi191.stdout:5/655: write d0/d18/d5c/d7c/d49/f78 [795685,101955] 0 2024-02-15T18:27:20.700 INFO:tasks.workunit.client.1.smithi191.stdout:5/656: truncate d0/f79 5250589 0 2024-02-15T18:27:20.700 INFO:tasks.workunit.client.1.smithi191.stdout:5/657: readlink d0/d18/d5c/l6e 0 2024-02-15T18:27:20.701 INFO:tasks.workunit.client.1.smithi191.stdout:5/658: read d0/d18/d5c/d7c/d49/f78 [205138,77983] 0 2024-02-15T18:27:20.704 INFO:tasks.workunit.client.1.smithi191.stdout:2/962: dwrite df/d61/fff [0,4194304] 0 2024-02-15T18:27:20.706 INFO:tasks.workunit.client.1.smithi191.stdout:0/724: getdents d7 0 2024-02-15T18:27:20.708 INFO:tasks.workunit.client.1.smithi191.stdout:9/908: dwrite d0/d7/dc/d17/f25 [4194304,4194304] 0 2024-02-15T18:27:20.708 INFO:tasks.workunit.client.1.smithi191.stdout:9/909: stat d0/d1d/d3f/lac 0 2024-02-15T18:27:20.710 INFO:tasks.workunit.client.1.smithi191.stdout:3/811: dwrite d6/d3b/d42/f51 [4194304,4194304] 0 2024-02-15T18:27:20.720 INFO:tasks.workunit.client.1.smithi191.stdout:2/963: truncate df/d61/f67 336700 0 2024-02-15T18:27:20.720 INFO:tasks.workunit.client.1.smithi191.stdout:2/964: readlink df/d12/d19/d1a/d52/l5c 0 2024-02-15T18:27:20.721 INFO:tasks.workunit.client.1.smithi191.stdout:2/965: fdatasync df/d12/d19/d1a/dda/ff9 0 2024-02-15T18:27:20.723 INFO:tasks.workunit.client.1.smithi191.stdout:5/659: symlink d0/d23/d41/d28/d31/ld9 0 2024-02-15T18:27:20.723 INFO:tasks.workunit.client.1.smithi191.stdout:0/725: link d7/d4a/f85 d7/da/d1a/ff2 0 2024-02-15T18:27:20.725 INFO:tasks.workunit.client.1.smithi191.stdout:2/966: mknod df/d12/d32/d4d/d89/d11a/d12a/c13a 0 2024-02-15T18:27:20.725 INFO:tasks.workunit.client.1.smithi191.stdout:3/812: creat d6/d1b/d60/f10c x:0 0 0 2024-02-15T18:27:20.725 INFO:tasks.workunit.client.1.smithi191.stdout:2/967: write df/d12/f55 [2000086,62436] 0 2024-02-15T18:27:20.728 INFO:tasks.workunit.client.1.smithi191.stdout:5/660: mkdir d0/d23/d41/d28/d31/d5a/d61/dda 0 2024-02-15T18:27:20.730 INFO:tasks.workunit.client.1.smithi191.stdout:3/813: creat d6/d1b/d60/da2/dac/de2/f10d x:0 0 0 2024-02-15T18:27:20.730 INFO:tasks.workunit.client.1.smithi191.stdout:2/968: mknod df/d12/d1e/c13b 0 2024-02-15T18:27:20.730 INFO:tasks.workunit.client.1.smithi191.stdout:9/910: dwrite d0/d29/d48/d66/d92/fd4 [0,4194304] 0 2024-02-15T18:27:20.730 INFO:tasks.workunit.client.1.smithi191.stdout:9/911: fsync d0/d7/dc/d32/f107 0 2024-02-15T18:27:20.733 INFO:tasks.workunit.client.1.smithi191.stdout:3/814: symlink d6/d27/d4d/da8/l10e 0 2024-02-15T18:27:20.733 INFO:tasks.workunit.client.1.smithi191.stdout:3/815: chown d6/d3b/d30/cde 487 1 2024-02-15T18:27:20.733 INFO:tasks.workunit.client.1.smithi191.stdout:9/912: dread d0/d7/dc/d30/d4f/fb0 [0,4194304] 0 2024-02-15T18:27:20.755 INFO:tasks.workunit.client.1.smithi191.stdout:0/726: dwrite d7/da/d1a/d34/d27/f7c [0,4194304] 0 2024-02-15T18:27:20.773 INFO:tasks.workunit.client.1.smithi191.stdout:2/969: mknod df/d12/d19/d1a/d35/d47/d4b/d7a/ddb/c13c 0 2024-02-15T18:27:20.775 INFO:tasks.workunit.client.1.smithi191.stdout:3/816: rmdir d6/d5d/d72/dd5/db2 39 2024-02-15T18:27:20.779 INFO:tasks.workunit.client.1.smithi191.stdout:9/913: symlink d0/d7/dc/d30/d3b/d56/d115/l12d 0 2024-02-15T18:27:20.780 INFO:tasks.workunit.client.1.smithi191.stdout:3/817: dread d6/d1b/d60/d73/f74 [0,4194304] 0 2024-02-15T18:27:20.780 INFO:tasks.workunit.client.1.smithi191.stdout:3/818: dread - d6/d27/d4d/d5a/d92/dbe/fd3 zero size 2024-02-15T18:27:20.792 INFO:tasks.workunit.client.1.smithi191.stdout:0/727: mkdir d7/da/d1a/d2c/d4c/df3 0 2024-02-15T18:27:20.792 INFO:tasks.workunit.client.1.smithi191.stdout:2/970: creat df/d12/d19/d1a/dfa/f13d x:0 0 0 2024-02-15T18:27:20.794 INFO:tasks.workunit.client.1.smithi191.stdout:9/914: readlink d0/d7/dc/d32/de4/dec/d126/lf0 0 2024-02-15T18:27:20.794 INFO:tasks.workunit.client.1.smithi191.stdout:9/915: truncate d0/d7/dc/d17/d44/de6/fea 1200006 0 2024-02-15T18:27:20.801 INFO:tasks.workunit.client.1.smithi191.stdout:2/971: creat df/d12/d19/d1a/d35/d47/d4b/d7a/dcc/f13e x:0 0 0 2024-02-15T18:27:20.802 INFO:tasks.workunit.client.1.smithi191.stdout:0/728: dread d7/da/d1a/d34/f52 [0,4194304] 0 2024-02-15T18:27:20.802 INFO:tasks.workunit.client.1.smithi191.stdout:3/819: mkdir d6/d8e/d10f 0 2024-02-15T18:27:20.860 INFO:tasks.workunit.client.1.smithi191.stdout:2/972: creat df/d12/d1e/d6f/f13f x:0 0 0 2024-02-15T18:27:20.862 INFO:tasks.workunit.client.1.smithi191.stdout:9/916: dwrite d0/d29/f70 [0,4194304] 0 2024-02-15T18:27:20.862 INFO:tasks.workunit.client.1.smithi191.stdout:0/729: rename d7/da/d17/l65 to d7/da/d6c/lf4 0 2024-02-15T18:27:20.863 INFO:tasks.workunit.client.1.smithi191.stdout:3/820: truncate d6/d27/fa5 748209 0 2024-02-15T18:27:20.868 INFO:tasks.workunit.client.1.smithi191.stdout:0/730: rename d7/da/d17/d31/d5f/dae/de2/dec/led to d7/d1b/d30/lf5 0 2024-02-15T18:27:20.868 INFO:tasks.workunit.client.1.smithi191.stdout:0/731: chown d7/d1b/d30/c3e 1806 1 2024-02-15T18:27:20.868 INFO:tasks.workunit.client.1.smithi191.stdout:3/821: mkdir d6/d3b/d42/d49/dc7/d110 0 2024-02-15T18:27:20.870 INFO:tasks.workunit.client.1.smithi191.stdout:9/917: link d0/d7/dc/d30/d3b/d56/cfc d0/d7/dc/d8c/c12e 0 2024-02-15T18:27:20.870 INFO:tasks.workunit.client.1.smithi191.stdout:9/918: write d0/d1d/d3f/d80/f100 [1981599,128130] 0 2024-02-15T18:27:20.881 INFO:tasks.workunit.client.1.smithi191.stdout:0/732: unlink d7/d1b/fe4 0 2024-02-15T18:27:20.885 INFO:tasks.workunit.client.1.smithi191.stdout:0/733: getdents d7/d3b 0 2024-02-15T18:27:20.886 INFO:tasks.workunit.client.1.smithi191.stdout:3/822: mkdir d6/d3b/d111 0 2024-02-15T18:27:20.886 INFO:tasks.workunit.client.1.smithi191.stdout:9/919: creat d0/d7/dc/d32/d106/dce/f12f x:0 0 0 2024-02-15T18:27:20.886 INFO:tasks.workunit.client.1.smithi191.stdout:9/920: chown d0/d7/dc/l93 1525363369 1 2024-02-15T18:27:20.887 INFO:tasks.workunit.client.1.smithi191.stdout:0/734: dread d7/d1b/f81 [0,4194304] 0 2024-02-15T18:27:20.887 INFO:tasks.workunit.client.1.smithi191.stdout:2/973: dwrite df/d12/d19/d1a/fdf [0,4194304] 0 2024-02-15T18:27:20.888 INFO:tasks.workunit.client.1.smithi191.stdout:2/974: write df/d12/d32/f8f [3681030,113014] 0 2024-02-15T18:27:20.889 INFO:tasks.workunit.client.1.smithi191.stdout:3/823: link d6/d5d/cc8 d6/d1b/d60/ddc/c112 0 2024-02-15T18:27:20.889 INFO:tasks.workunit.client.1.smithi191.stdout:3/824: readlink d6/d3b/d42/d49/dbf/lcf 0 2024-02-15T18:27:20.890 INFO:tasks.workunit.client.1.smithi191.stdout:2/975: write df/d12/d19/d1a/f29 [3386851,81285] 0 2024-02-15T18:27:20.898 INFO:tasks.workunit.client.1.smithi191.stdout:9/921: mknod d0/d7/dc/d32/de4/dec/d126/c130 0 2024-02-15T18:27:20.898 INFO:tasks.workunit.client.1.smithi191.stdout:9/922: write d0/d7/dc/d32/d106/dce/f12f [77458,91769] 0 2024-02-15T18:27:20.907 INFO:tasks.workunit.client.1.smithi191.stdout:2/976: unlink df/d12/d19/cc2 0 2024-02-15T18:27:20.909 INFO:tasks.workunit.client.1.smithi191.stdout:9/923: link d0/d7/dc/c23 d0/d29/d10d/c131 0 2024-02-15T18:27:20.909 INFO:tasks.workunit.client.1.smithi191.stdout:2/977: dread df/d12/d19/d1a/d52/f105 [0,4194304] 0 2024-02-15T18:27:20.911 INFO:tasks.workunit.client.1.smithi191.stdout:3/825: dwrite d6/d1b/f81 [0,4194304] 0 2024-02-15T18:27:20.911 INFO:tasks.workunit.client.1.smithi191.stdout:3/826: dread - d6/d1b/d60/ddc/de5/fe7 zero size 2024-02-15T18:27:20.911 INFO:tasks.workunit.client.1.smithi191.stdout:9/924: rename d0/d7/dc/fbb to d0/d7/dc/d17/d44/de6/f132 0 2024-02-15T18:27:20.911 INFO:tasks.workunit.client.1.smithi191.stdout:9/925: stat d0/d7/dc/d8c/l113 0 2024-02-15T18:27:20.911 INFO:tasks.workunit.client.1.smithi191.stdout:9/926: truncate d0/d7/dc/f9b 741215 0 2024-02-15T18:27:20.911 INFO:tasks.workunit.client.1.smithi191.stdout:9/927: fsync d0/d7/dc/d17/f25 0 2024-02-15T18:27:20.913 INFO:tasks.workunit.client.1.smithi191.stdout:9/928: read d0/d7/dc/d30/f5c [3616071,118458] 0 2024-02-15T18:27:20.913 INFO:tasks.workunit.client.1.smithi191.stdout:9/929: write d0/d7/dc/d17/d44/de6/fe1 [1770886,44454] 0 2024-02-15T18:27:20.914 INFO:tasks.workunit.client.1.smithi191.stdout:3/827: mkdir d6/d1b/d60/da2/dac/de2/d113 0 2024-02-15T18:27:20.916 INFO:tasks.workunit.client.1.smithi191.stdout:9/930: read d0/d7/dc/d17/d44/fd8 [3571,79529] 0 2024-02-15T18:27:20.916 INFO:tasks.workunit.client.1.smithi191.stdout:2/978: rename df/d12/d19/d1a/d52/df2 to df/d12/d140 0 2024-02-15T18:27:20.919 INFO:tasks.workunit.client.1.smithi191.stdout:0/735: dwrite d7/da/d1a/f58 [4194304,4194304] 0 2024-02-15T18:27:20.919 INFO:tasks.workunit.client.1.smithi191.stdout:9/931: dread d0/d7/dc/d17/d44/de6/fea [0,4194304] 0 2024-02-15T18:27:20.925 INFO:tasks.workunit.client.1.smithi191.stdout:9/932: mkdir d0/d7/dc/d32/d106/dce/d133 0 2024-02-15T18:27:20.926 INFO:tasks.workunit.client.1.smithi191.stdout:9/933: rmdir d0/d7/dc/d30/d3b/d56/d115/d118 39 2024-02-15T18:27:20.927 INFO:tasks.workunit.client.1.smithi191.stdout:9/934: rmdir d0/d7/dbf 39 2024-02-15T18:27:20.954 INFO:tasks.workunit.client.1.smithi191.stdout:3/828: dwrite d6/d27/d4d/d5a/fea [0,4194304] 0 2024-02-15T18:27:20.954 INFO:tasks.workunit.client.1.smithi191.stdout:3/829: chown d6/d3b/d42/d49/dc7/de3 1 1 2024-02-15T18:27:20.957 INFO:tasks.workunit.client.1.smithi191.stdout:3/830: rmdir d6/d5d/d72/dd5/db2 39 2024-02-15T18:27:20.958 INFO:tasks.workunit.client.1.smithi191.stdout:0/736: dwrite d7/da/d17/d31/d5f/f61 [0,4194304] 0 2024-02-15T18:27:20.960 INFO:tasks.workunit.client.1.smithi191.stdout:9/935: dwrite d0/d7/dc/d30/f76 [0,4194304] 0 2024-02-15T18:27:20.967 INFO:tasks.workunit.client.1.smithi191.stdout:0/737: creat d7/da/d1a/d2c/d4c/ff6 x:0 0 0 2024-02-15T18:27:20.967 INFO:tasks.workunit.client.1.smithi191.stdout:0/738: write d7/da/f8e [637289,57515] 0 2024-02-15T18:27:20.967 INFO:tasks.workunit.client.1.smithi191.stdout:0/739: readlink d7/d94/l9c 0 2024-02-15T18:27:20.968 INFO:tasks.workunit.client.1.smithi191.stdout:0/740: fdatasync d7/da/d17/d31/f89 0 2024-02-15T18:27:20.968 INFO:tasks.workunit.client.1.smithi191.stdout:0/741: chown d7/da/d1a/ff2 22 1 2024-02-15T18:27:20.970 INFO:tasks.workunit.client.1.smithi191.stdout:9/936: dread d0/d7/dc/d17/d44/feb [0,4194304] 0 2024-02-15T18:27:20.970 INFO:tasks.workunit.client.1.smithi191.stdout:9/937: dread - d0/d7/dc/d30/d4f/f116 zero size 2024-02-15T18:27:20.972 INFO:tasks.workunit.client.1.smithi191.stdout:0/742: symlink d7/da/d17/d31/d5f/d9e/dc3/lf7 0 2024-02-15T18:27:20.972 INFO:tasks.workunit.client.1.smithi191.stdout:3/831: dread d6/d5d/daa/fb9 [0,4194304] 0 2024-02-15T18:27:20.974 INFO:tasks.workunit.client.1.smithi191.stdout:9/938: rename d0/d7/dc/d30/d4f/la8 to d0/d7/dc/d32/l134 0 2024-02-15T18:27:20.975 INFO:tasks.workunit.client.1.smithi191.stdout:0/743: link d7/da/d17/d31/d5f/f5b d7/d1b/ff8 0 2024-02-15T18:27:20.975 INFO:tasks.workunit.client.1.smithi191.stdout:9/939: write d0/fdc [1242194,91342] 0 2024-02-15T18:27:20.975 INFO:tasks.workunit.client.1.smithi191.stdout:9/940: dread - d0/f12c zero size 2024-02-15T18:27:20.976 INFO:tasks.workunit.client.1.smithi191.stdout:9/941: read d0/d7/dc/d8c/f1c [2415001,71938] 0 2024-02-15T18:27:20.980 INFO:tasks.workunit.client.1.smithi191.stdout:0/744: mkdir d7/da/d17/d31/d7d/df9 0 2024-02-15T18:27:20.980 INFO:tasks.workunit.client.1.smithi191.stdout:3/832: mknod d6/d1b/d60/d73/c114 0 2024-02-15T18:27:20.982 INFO:tasks.workunit.client.1.smithi191.stdout:0/745: mkdir d7/da/d1a/d34/dfa 0 2024-02-15T18:27:20.986 INFO:tasks.workunit.client.1.smithi191.stdout:0/746: creat d7/d94/ffb x:0 0 0 2024-02-15T18:27:20.988 INFO:tasks.workunit.client.1.smithi191.stdout:3/833: mknod d6/d3b/d42/c115 0 2024-02-15T18:27:20.989 INFO:tasks.workunit.client.1.smithi191.stdout:0/747: getdents d7/da/d1a/d34/d27 0 2024-02-15T18:27:20.992 INFO:tasks.workunit.client.1.smithi191.stdout:3/834: rename d6/d3b/d42/lf9 to d6/d27/l116 0 2024-02-15T18:27:20.992 INFO:tasks.workunit.client.1.smithi191.stdout:3/835: fsync d6/d27/d4d/fb6 0 2024-02-15T18:27:20.994 INFO:tasks.workunit.client.1.smithi191.stdout:3/836: link d6/d1b/f81 d6/d3b/d42/d49/def/f117 0 2024-02-15T18:27:20.995 INFO:tasks.workunit.client.1.smithi191.stdout:3/837: mknod d6/d3b/d42/ddd/c118 0 2024-02-15T18:27:20.996 INFO:tasks.workunit.client.1.smithi191.stdout:3/838: mknod d6/d3b/d42/d49/def/c119 0 2024-02-15T18:27:21.000 INFO:tasks.workunit.client.1.smithi191.stdout:9/942: dwrite d0/d7/f9 [0,4194304] 0 2024-02-15T18:27:21.000 INFO:tasks.workunit.client.1.smithi191.stdout:9/943: dread - d0/d1d/d3f/fe3 zero size 2024-02-15T18:27:21.000 INFO:tasks.workunit.client.1.smithi191.stdout:9/944: readlink d0/d7/dc/d32/l62 0 2024-02-15T18:27:21.004 INFO:tasks.workunit.client.1.smithi191.stdout:0/748: dwrite d7/d3b/fb2 [0,4194304] 0 2024-02-15T18:27:21.006 INFO:tasks.workunit.client.1.smithi191.stdout:9/945: mkdir d0/d7/dc/d30/d3b/d8b/d135 0 2024-02-15T18:27:21.006 INFO:tasks.workunit.client.1.smithi191.stdout:9/946: fsync d0/d1d/f79 0 2024-02-15T18:27:21.008 INFO:tasks.workunit.client.1.smithi191.stdout:9/947: write d0/d1d/d3f/d80/f75 [3442311,33184] 0 2024-02-15T18:27:21.010 INFO:tasks.workunit.client.1.smithi191.stdout:0/749: creat d7/da/d1a/d34/ffc x:0 0 0 2024-02-15T18:27:21.010 INFO:tasks.workunit.client.1.smithi191.stdout:0/750: readlink d7/d3b/l77 0 2024-02-15T18:27:21.019 INFO:tasks.workunit.client.1.smithi191.stdout:9/948: rename d0/l16 to d0/d7/dc/l136 0 2024-02-15T18:27:21.020 INFO:tasks.workunit.client.1.smithi191.stdout:9/949: write d0/d29/d48/d66/f120 [140333,45624] 0 2024-02-15T18:27:21.020 INFO:tasks.workunit.client.1.smithi191.stdout:9/950: chown d0/d7/dc/d32/de4/f112 0 1 2024-02-15T18:27:21.023 INFO:tasks.workunit.client.1.smithi191.stdout:0/751: write d7/d3b/f71 [1666198,74011] 0 2024-02-15T18:27:21.025 INFO:tasks.workunit.client.1.smithi191.stdout:0/752: mkdir d7/d3b/dfd 0 2024-02-15T18:27:21.055 INFO:tasks.workunit.client.1.smithi191.stdout:0/753: creat d7/da/d17/d31/d5f/d9e/dc3/dd8/ffe x:0 0 0 2024-02-15T18:27:21.058 INFO:tasks.workunit.client.1.smithi191.stdout:0/754: mkdir d7/da/d1a/d34/d27/dff 0 2024-02-15T18:27:21.062 INFO:tasks.workunit.client.1.smithi191.stdout:0/755: mkdir d7/da/d1a/d2c/d4c/dd1/d100 0 2024-02-15T18:27:21.065 INFO:tasks.workunit.client.1.smithi191.stdout:0/756: rename d7/da/d1a/d34/fc6 to d7/da/d1a/d2c/d4c/dd1/f101 0 2024-02-15T18:27:21.067 INFO:tasks.workunit.client.1.smithi191.stdout:0/757: creat d7/f102 x:0 0 0 2024-02-15T18:27:21.069 INFO:tasks.workunit.client.1.smithi191.stdout:0/758: mkdir d7/da/d17/d31/d5f/d103 0 2024-02-15T18:27:21.069 INFO:tasks.workunit.client.1.smithi191.stdout:0/759: fsync d7/da/d17/d31/d7d/fdc 0 2024-02-15T18:27:21.070 INFO:tasks.workunit.client.1.smithi191.stdout:0/760: symlink d7/da/d1a/d2c/d4c/df3/l104 0 2024-02-15T18:27:21.071 INFO:tasks.workunit.client.1.smithi191.stdout:0/761: getdents d7/d1b/d30 0 2024-02-15T18:27:21.073 INFO:tasks.workunit.client.1.smithi191.stdout:0/762: unlink d7/da/d17/d31/d5f/dae/dd3/lea 0 2024-02-15T18:27:21.074 INFO:tasks.workunit.client.1.smithi191.stdout:0/763: mkdir d7/da/d17/d31/d5f/d9e/dc3/dd8/d105 0 2024-02-15T18:27:21.076 INFO:tasks.workunit.client.1.smithi191.stdout:0/764: getdents d7/d4a/dc0 0 2024-02-15T18:27:21.079 INFO:tasks.workunit.client.1.smithi191.stdout:0/765: rmdir d7/da/d1a/d34/dfa 0 2024-02-15T18:27:21.086 INFO:tasks.workunit.client.1.smithi191.stdout:0/766: creat d7/da/d17/d31/f106 x:0 0 0 2024-02-15T18:27:21.101 INFO:tasks.workunit.client.1.smithi191.stdout:0/767: dwrite d7/d94/fad [0,4194304] 0 2024-02-15T18:27:21.104 INFO:tasks.workunit.client.1.smithi191.stdout:0/768: link d7/da/d1a/f58 d7/da/d1a/d34/d45/f107 0 2024-02-15T18:27:21.105 INFO:tasks.workunit.client.1.smithi191.stdout:0/769: dread d7/d1b/f81 [0,4194304] 0 2024-02-15T18:27:21.107 INFO:tasks.workunit.client.1.smithi191.stdout:0/770: unlink d7/da/d17/d31/d7d/ldb 0 2024-02-15T18:27:21.108 INFO:tasks.workunit.client.1.smithi191.stdout:0/771: read d7/da/d17/d31/f89 [706963,81770] 0 2024-02-15T18:27:21.110 INFO:tasks.workunit.client.1.smithi191.stdout:0/772: rmdir d7/da/d1a/d2c/d4c/dd1/d100 0 2024-02-15T18:27:21.111 INFO:tasks.workunit.client.1.smithi191.stdout:0/773: write d7/da/d1a/d34/d45/f107 [7231360,102969] 0 2024-02-15T18:27:21.113 INFO:tasks.workunit.client.1.smithi191.stdout:0/774: creat d7/da/d17/daa/f108 x:0 0 0 2024-02-15T18:27:21.114 INFO:tasks.workunit.client.1.smithi191.stdout:0/775: mknod d7/da/d17/d31/d5f/d9e/dc3/dd8/c109 0 2024-02-15T18:27:21.115 INFO:tasks.workunit.client.1.smithi191.stdout:0/776: write d7/da/d1a/d2c/d4c/f9f [356679,129811] 0 2024-02-15T18:27:21.115 INFO:tasks.workunit.client.1.smithi191.stdout:0/777: dread d7/d1b/d30/f43 [0,4194304] 0 2024-02-15T18:27:21.117 INFO:tasks.workunit.client.1.smithi191.stdout:0/778: symlink d7/da/d17/d31/d5f/l10a 0 2024-02-15T18:27:21.144 INFO:tasks.workunit.client.1.smithi191.stdout:0/779: dwrite d7/d94/feb [0,4194304] 0 2024-02-15T18:27:21.149 INFO:tasks.workunit.client.1.smithi191.stdout:0/780: unlink d7/d3b/l3f 0 2024-02-15T18:27:21.149 INFO:tasks.workunit.client.1.smithi191.stdout:0/781: dread - d7/da/d17/d31/d7d/fcf zero size 2024-02-15T18:27:21.167 INFO:tasks.workunit.client.1.smithi191.stdout:0/782: creat d7/da/d1a/d2c/f10b x:0 0 0 2024-02-15T18:27:21.168 INFO:tasks.workunit.client.1.smithi191.stdout:0/783: dread d7/da/d1a/d34/d27/f6f [0,4194304] 0 2024-02-15T18:27:21.173 INFO:tasks.workunit.client.1.smithi191.stdout:0/784: creat d7/da/d17/d31/d5f/d9e/dc3/f10c x:0 0 0 2024-02-15T18:27:21.175 INFO:tasks.workunit.client.1.smithi191.stdout:0/785: write d7/d4a/f7b [234571,29999] 0 2024-02-15T18:27:21.196 INFO:tasks.workunit.client.1.smithi191.stdout:0/786: dwrite d7/d3b/f69 [0,4194304] 0 2024-02-15T18:27:21.196 INFO:tasks.workunit.client.1.smithi191.stdout:0/787: write d7/da/d17/d31/d7d/fdc [529030,47856] 0 2024-02-15T18:27:21.196 INFO:tasks.workunit.client.1.smithi191.stdout:0/788: readlink d7/d3b/l8b 0 2024-02-15T18:27:21.196 INFO:tasks.workunit.client.1.smithi191.stdout:0/789: write d7/d3b/f71 [1823073,30301] 0 2024-02-15T18:27:21.198 INFO:tasks.workunit.client.1.smithi191.stdout:0/790: rmdir d7/da/d1a/d2c/d4c/dd1 39 2024-02-15T18:27:21.198 INFO:tasks.workunit.client.1.smithi191.stdout:0/791: chown d7/d4a/f59 0 1 2024-02-15T18:27:21.198 INFO:tasks.workunit.client.1.smithi191.stdout:0/792: write d7/da/d17/daa/f108 [509536,51865] 0 2024-02-15T18:27:21.208 INFO:tasks.workunit.client.1.smithi191.stdout:5/661: sync 2024-02-15T18:27:21.208 INFO:tasks.workunit.client.1.smithi191.stdout:9/951: sync 2024-02-15T18:27:21.208 INFO:tasks.workunit.client.1.smithi191.stdout:2/979: sync 2024-02-15T18:27:21.208 INFO:tasks.workunit.client.1.smithi191.stdout:7/957: sync 2024-02-15T18:27:21.208 INFO:tasks.workunit.client.1.smithi191.stdout:3/839: sync 2024-02-15T18:27:21.208 INFO:tasks.workunit.client.1.smithi191.stdout:5/662: chown d0/d23/d41/d28 536099 1 2024-02-15T18:27:21.208 INFO:tasks.workunit.client.1.smithi191.stdout:2/980: truncate df/d12/d19/d1a/dfa/f13d 470164 0 2024-02-15T18:27:21.208 INFO:tasks.workunit.client.1.smithi191.stdout:5/663: fdatasync d0/d23/d41/d28/d31/f53 0 2024-02-15T18:27:21.208 INFO:tasks.workunit.client.1.smithi191.stdout:9/952: write d0/d29/d48/d66/f120 [1841056,17405] 0 2024-02-15T18:27:21.212 INFO:tasks.workunit.client.1.smithi191.stdout:2/981: write df/d12/d32/fc4 [860164,8561] 0 2024-02-15T18:27:21.214 INFO:tasks.workunit.client.1.smithi191.stdout:5/664: write d0/d18/d5c/d7c/d49/d9f/dc0/d85/f54 [1382461,81860] 0 2024-02-15T18:27:21.214 INFO:tasks.workunit.client.1.smithi191.stdout:2/982: creat df/d61/d9f/da3/f141 x:0 0 0 2024-02-15T18:27:21.217 INFO:tasks.workunit.client.1.smithi191.stdout:5/665: creat d0/d18/d5c/d7c/d49/d9f/fdb x:0 0 0 2024-02-15T18:27:21.219 INFO:tasks.workunit.client.1.smithi191.stdout:2/983: creat df/d61/d94/f142 x:0 0 0 2024-02-15T18:27:21.222 INFO:tasks.workunit.client.1.smithi191.stdout:2/984: link df/d12/d19/d1a/d35/d47/f51 df/d12/d19/d1a/d35/d47/d4b/d7a/dce/f143 0 2024-02-15T18:27:21.222 INFO:tasks.workunit.client.1.smithi191.stdout:2/985: fsync df/d12/d19/d1a/d35/d47/d4b/f6c 0 2024-02-15T18:27:21.223 INFO:tasks.workunit.client.1.smithi191.stdout:0/793: dwrite d7/da/d17/daa/f108 [0,4194304] 0 2024-02-15T18:27:21.224 INFO:tasks.workunit.client.1.smithi191.stdout:2/986: mkdir df/d12/d32/d4d/d89/d11a/d12a/d144 0 2024-02-15T18:27:21.226 INFO:tasks.workunit.client.1.smithi191.stdout:0/794: creat d7/d4a/dc0/dcc/f10d x:0 0 0 2024-02-15T18:27:21.227 INFO:tasks.workunit.client.1.smithi191.stdout:2/987: rename df/d12/d19/d1a/d35/d47/d4b/d7a/dcc/f114 to df/d12/d19/d1a/d35/d47/d4b/d7a/dce/f145 0 2024-02-15T18:27:21.227 INFO:tasks.workunit.client.1.smithi191.stdout:2/988: fsync df/d61/d9f/da3/dab/fdd 0 2024-02-15T18:27:21.228 INFO:tasks.workunit.client.1.smithi191.stdout:0/795: creat d7/d4a/f10e x:0 0 0 2024-02-15T18:27:21.229 INFO:tasks.workunit.client.1.smithi191.stdout:2/989: mkdir df/d12/d32/d4d/d89/d146 0 2024-02-15T18:27:21.232 INFO:tasks.workunit.client.1.smithi191.stdout:7/958: dwrite d0/db/d65/d11/d5a/d66/d1e/f9b [0,4194304] 0 2024-02-15T18:27:21.235 INFO:tasks.workunit.client.1.smithi191.stdout:7/959: rmdir d0/db/d65/dc/d120 39 2024-02-15T18:27:21.237 INFO:tasks.workunit.client.1.smithi191.stdout:7/960: creat d0/db/d65/dc/f152 x:0 0 0 2024-02-15T18:27:21.238 INFO:tasks.workunit.client.1.smithi191.stdout:7/961: write d0/db/f8b [206303,60906] 0 2024-02-15T18:27:21.239 INFO:tasks.workunit.client.1.smithi191.stdout:3/840: dwrite d6/d1b/d60/f68 [0,4194304] 0 2024-02-15T18:27:21.239 INFO:tasks.workunit.client.1.smithi191.stdout:5/666: dwrite d0/d23/d41/d28/f36 [4194304,4194304] 0 2024-02-15T18:27:21.239 INFO:tasks.workunit.client.1.smithi191.stdout:3/841: dread - d6/d3b/fbd zero size 2024-02-15T18:27:21.240 INFO:tasks.workunit.client.1.smithi191.stdout:7/962: rename d0/db/d65/dc/d6e/c71 to d0/db/ddd/dea/c153 0 2024-02-15T18:27:21.247 INFO:tasks.workunit.client.1.smithi191.stdout:2/990: dwrite df/d61/d94/f109 [0,4194304] 0 2024-02-15T18:27:21.249 INFO:tasks.workunit.client.1.smithi191.stdout:5/667: creat d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/fdc x:0 0 0 2024-02-15T18:27:21.253 INFO:tasks.workunit.client.1.smithi191.stdout:0/796: write d7/da/d17/d31/d5f/fbc [688831,23350] 0 2024-02-15T18:27:21.254 INFO:tasks.workunit.client.1.smithi191.stdout:7/963: mkdir d0/db/d65/d11/d88/d9d/dd2/d154 0 2024-02-15T18:27:21.259 INFO:tasks.workunit.client.1.smithi191.stdout:5/668: mknod d0/d23/d41/d28/d31/d5a/d61/dda/cdd 0 2024-02-15T18:27:21.259 INFO:tasks.workunit.client.1.smithi191.stdout:5/669: fdatasync d0/d18/d5c/d7c/f9c 0 2024-02-15T18:27:21.260 INFO:tasks.workunit.client.1.smithi191.stdout:3/842: dwrite d6/d1b/d60/d73/f8d [4194304,4194304] 0 2024-02-15T18:27:21.261 INFO:tasks.workunit.client.1.smithi191.stdout:2/991: dread df/d12/f7c [0,4194304] 0 2024-02-15T18:27:21.261 INFO:tasks.workunit.client.1.smithi191.stdout:2/992: fsync df/d12/d19/d1a/d52/f112 0 2024-02-15T18:27:21.261 INFO:tasks.workunit.client.1.smithi191.stdout:0/797: creat d7/d4a/f10f x:0 0 0 2024-02-15T18:27:21.266 INFO:tasks.workunit.client.1.smithi191.stdout:3/843: chown d6/d3b/d42/l33 457019699 1 2024-02-15T18:27:21.268 INFO:tasks.workunit.client.1.smithi191.stdout:5/670: dread d0/d23/d41/d28/d31/d5a/f68 [0,4194304] 0 2024-02-15T18:27:21.268 INFO:tasks.workunit.client.1.smithi191.stdout:5/671: dread - d0/d23/d41/d28/f94 zero size 2024-02-15T18:27:21.268 INFO:tasks.workunit.client.1.smithi191.stdout:5/672: read d0/d18/fbb [428273,67083] 0 2024-02-15T18:27:21.268 INFO:tasks.workunit.client.1.smithi191.stdout:5/673: write d0/d23/d41/d28/fcf [1037535,104] 0 2024-02-15T18:27:21.268 INFO:tasks.workunit.client.1.smithi191.stdout:2/993: mknod df/d12/d32/d4d/c147 0 2024-02-15T18:27:21.268 INFO:tasks.workunit.client.1.smithi191.stdout:5/674: fdatasync d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/fb0 0 2024-02-15T18:27:21.272 INFO:tasks.workunit.client.1.smithi191.stdout:3/844: rename d6/d3b/f6f to d6/d5d/daa/f11a 0 2024-02-15T18:27:21.274 INFO:tasks.workunit.client.1.smithi191.stdout:2/994: mknod df/d61/d9f/c148 0 2024-02-15T18:27:21.275 INFO:tasks.workunit.client.1.smithi191.stdout:2/995: read df/d12/d32/d4d/d89/d11a/d12a/d42/f49 [237028,118288] 0 2024-02-15T18:27:21.275 INFO:tasks.workunit.client.1.smithi191.stdout:2/996: write df/d12/d19/d1a/dfa/f13d [1375619,130620] 0 2024-02-15T18:27:21.276 INFO:tasks.workunit.client.1.smithi191.stdout:3/845: creat d6/d1b/d60/d73/d5e/f11b x:0 0 0 2024-02-15T18:27:21.276 INFO:tasks.workunit.client.1.smithi191.stdout:5/675: mknod d0/cde 0 2024-02-15T18:27:21.279 INFO:tasks.workunit.client.1.smithi191.stdout:2/997: link df/d12/d32/d4d/d89/d11a/d12a/cde df/d12/d19/d1a/d35/d47/d4b/d7a/dce/d118/c149 0 2024-02-15T18:27:21.279 INFO:tasks.workunit.client.1.smithi191.stdout:2/998: write df/d12/d19/d1a/d35/d47/d4b/d7a/dce/f145 [196313,32544] 0 2024-02-15T18:27:21.279 INFO:tasks.workunit.client.1.smithi191.stdout:2/999: chown df/d12/d32/dee 1718 1 2024-02-15T18:27:21.280 INFO:tasks.workunit.client.1.smithi191.stdout:3/846: symlink d6/d1b/d60/da2/dac/de2/l11c 0 2024-02-15T18:27:21.281 INFO:tasks.workunit.client.1.smithi191.stdout:5/676: symlink d0/d18/d5c/d7c/d49/ldf 0 2024-02-15T18:27:21.285 INFO:tasks.workunit.client.1.smithi191.stdout:3/847: symlink d6/d5d/df3/l11d 0 2024-02-15T18:27:21.285 INFO:tasks.workunit.client.1.smithi191.stdout:3/848: chown d6/d5d/d72/f98 5 1 2024-02-15T18:27:21.287 INFO:tasks.workunit.client.1.smithi191.stdout:5/677: mknod d0/d23/ce0 0 2024-02-15T18:27:21.290 INFO:tasks.workunit.client.1.smithi191.stdout:3/849: link d6/c1e d6/d3b/d42/d49/dbf/df7/c11e 0 2024-02-15T18:27:21.293 INFO:tasks.workunit.client.1.smithi191.stdout:3/850: rename d6/d3b/d42/d49/f6d to d6/d1b/d60/da2/dac/de2/d113/f11f 0 2024-02-15T18:27:21.298 INFO:tasks.workunit.client.1.smithi191.stdout:3/851: unlink d6/d27/d4d/d5a/da4/fbc 0 2024-02-15T18:27:21.301 INFO:tasks.workunit.client.1.smithi191.stdout:3/852: rename d6/d3b/d42/d49/d54/c88 to d6/d3b/d42/d49/c120 0 2024-02-15T18:27:21.302 INFO:tasks.workunit.client.1.smithi191.stdout:3/853: mknod d6/d1b/d60/da2/dac/de2/c121 0 2024-02-15T18:27:21.302 INFO:tasks.workunit.client.1.smithi191.stdout:3/854: chown d6/d27/d4d/d5a/d92/c109 0 1 2024-02-15T18:27:21.302 INFO:tasks.workunit.client.1.smithi191.stdout:3/855: truncate d6/d8e/fc6 4234279 0 2024-02-15T18:27:21.305 INFO:tasks.workunit.client.1.smithi191.stdout:3/856: write d6/d3b/f13 [1331974,17150] 0 2024-02-15T18:27:21.305 INFO:tasks.workunit.client.1.smithi191.stdout:3/857: fdatasync d6/d3b/d42/d49/dbf/df7/f105 0 2024-02-15T18:27:21.305 INFO:tasks.workunit.client.1.smithi191.stdout:5/678: dwrite d0/d18/f7a [0,4194304] 0 2024-02-15T18:27:21.306 INFO:tasks.workunit.client.1.smithi191.stdout:3/858: creat d6/d1b/d60/d73/f122 x:0 0 0 2024-02-15T18:27:21.314 INFO:tasks.workunit.client.1.smithi191.stdout:5/679: rmdir d0/d23/d41/d28/d31/d5a/d61/dda 39 2024-02-15T18:27:21.315 INFO:tasks.workunit.client.1.smithi191.stdout:5/680: chown d0/c8e 61419713 1 2024-02-15T18:27:21.316 INFO:tasks.workunit.client.1.smithi191.stdout:3/859: mkdir d6/d1b/d60/d73/d5e/d123 0 2024-02-15T18:27:21.316 INFO:tasks.workunit.client.1.smithi191.stdout:5/681: mknod d0/d23/ce1 0 2024-02-15T18:27:21.316 INFO:tasks.workunit.client.1.smithi191.stdout:5/682: dread - d0/d18/d5c/fd8 zero size 2024-02-15T18:27:21.316 INFO:tasks.workunit.client.1.smithi191.stdout:5/683: chown d0/d23/d41/d28/d31/d5a/db2 3470654 1 2024-02-15T18:27:21.318 INFO:tasks.workunit.client.1.smithi191.stdout:3/860: rename d6/d27/d4d/da8/fb3 to d6/d3b/d42/ddd/f124 0 2024-02-15T18:27:21.319 INFO:tasks.workunit.client.1.smithi191.stdout:5/684: symlink d0/d23/d41/le2 0 2024-02-15T18:27:21.319 INFO:tasks.workunit.client.1.smithi191.stdout:5/685: truncate d0/d23/d41/d28/d31/fb8 1003671 0 2024-02-15T18:27:21.320 INFO:tasks.workunit.client.1.smithi191.stdout:3/861: creat d6/d3b/d42/d49/dc7/de3/f125 x:0 0 0 2024-02-15T18:27:21.321 INFO:tasks.workunit.client.1.smithi191.stdout:3/862: read d6/fa0 [6631569,8835] 0 2024-02-15T18:27:21.322 INFO:tasks.workunit.client.1.smithi191.stdout:5/686: creat d0/d23/d41/d28/d31/d5a/fe3 x:0 0 0 2024-02-15T18:27:21.323 INFO:tasks.workunit.client.1.smithi191.stdout:5/687: write d0/d18/d5c/f6d [1923959,41733] 0 2024-02-15T18:27:21.325 INFO:tasks.workunit.client.1.smithi191.stdout:5/688: mknod d0/d18/d5c/d7c/d49/d76/ce4 0 2024-02-15T18:27:21.326 INFO:tasks.workunit.client.1.smithi191.stdout:5/689: mkdir d0/d23/d41/d28/d31/d5a/d61/de5 0 2024-02-15T18:27:21.326 INFO:tasks.workunit.client.1.smithi191.stdout:5/690: chown d0/d18/d5c/d7c/d49/d9f/dbd/ld4 14 1 2024-02-15T18:27:21.328 INFO:tasks.workunit.client.1.smithi191.stdout:5/691: truncate d0/fa7 3775647 0 2024-02-15T18:27:21.330 INFO:tasks.workunit.client.1.smithi191.stdout:5/692: getdents d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52 0 2024-02-15T18:27:21.330 INFO:tasks.workunit.client.1.smithi191.stdout:5/693: write d0/d18/d5c/d7c/d49/d9f/fdb [971088,115313] 0 2024-02-15T18:27:21.330 INFO:tasks.workunit.client.1.smithi191.stdout:5/694: dread - d0/d23/d41/d28/fb1 zero size 2024-02-15T18:27:21.331 INFO:tasks.workunit.client.1.smithi191.stdout:5/695: dread d0/d23/d41/d28/fcf [0,4194304] 0 2024-02-15T18:27:21.332 INFO:tasks.workunit.client.1.smithi191.stdout:5/696: write d0/d18/d5c/d7c/f9c [2735617,31621] 0 2024-02-15T18:27:21.356 INFO:tasks.workunit.client.1.smithi191.stdout:5/697: dwrite d0/fa7 [0,4194304] 0 2024-02-15T18:27:21.361 INFO:tasks.workunit.client.1.smithi191.stdout:5/698: creat d0/d23/d41/d28/d31/d5a/db2/fe6 x:0 0 0 2024-02-15T18:27:21.361 INFO:tasks.workunit.client.1.smithi191.stdout:5/699: write d0/d23/d41/d3c/fd5 [934135,46750] 0 2024-02-15T18:27:21.367 INFO:tasks.workunit.client.1.smithi191.stdout:5/700: creat d0/d23/d41/fe7 x:0 0 0 2024-02-15T18:27:21.384 INFO:tasks.workunit.client.1.smithi191.stdout:5/701: dwrite d0/d23/d41/d28/d31/d5a/fa0 [0,4194304] 0 2024-02-15T18:27:21.388 INFO:tasks.workunit.client.1.smithi191.stdout:5/702: creat d0/d62/fe8 x:0 0 0 2024-02-15T18:27:21.392 INFO:tasks.workunit.client.1.smithi191.stdout:5/703: symlink d0/d23/d41/d28/d31/d5a/d61/de5/le9 0 2024-02-15T18:27:21.547 INFO:tasks.workunit.client.1.smithi191.stdout:5/704: truncate d0/d23/d41/d28/d31/fb8 1705660 0 2024-02-15T18:27:21.547 INFO:tasks.workunit.client.1.smithi191.stdout:5/705: dread - d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/fdc zero size 2024-02-15T18:27:21.564 INFO:tasks.workunit.client.1.smithi191.stdout:5/706: dwrite d0/d23/d41/d28/f3f [0,4194304] 0 2024-02-15T18:27:21.564 INFO:tasks.workunit.client.1.smithi191.stdout:5/707: chown d0/d62 27201177 1 2024-02-15T18:27:21.583 INFO:tasks.workunit.client.1.smithi191.stdout:5/708: dwrite d0/d23/d41/d28/d31/d5a/fe3 [0,4194304] 0 2024-02-15T18:27:21.589 INFO:tasks.workunit.client.1.smithi191.stdout:5/709: mkdir d0/d18/d5c/dea 0 2024-02-15T18:27:21.589 INFO:tasks.workunit.client.1.smithi191.stdout:5/710: dread - d0/d18/d5c/d7c/d49/d9f/fb4 zero size 2024-02-15T18:27:21.595 INFO:tasks.workunit.client.1.smithi191.stdout:5/711: dread d0/d18/d5c/d7c/f58 [0,4194304] 0 2024-02-15T18:27:21.598 INFO:tasks.workunit.client.1.smithi191.stdout:7/964: sync 2024-02-15T18:27:21.598 INFO:tasks.workunit.client.1.smithi191.stdout:9/953: sync 2024-02-15T18:27:21.598 INFO:tasks.workunit.client.1.smithi191.stdout:5/712: write d0/d23/d41/d28/fcd [1934470,70323] 0 2024-02-15T18:27:21.601 INFO:tasks.workunit.client.1.smithi191.stdout:7/965: mknod d0/db/d26/d5f/d77/d35/d42/c155 0 2024-02-15T18:27:21.601 INFO:tasks.workunit.client.1.smithi191.stdout:7/966: truncate d0/db/d65/d11/fc4 4528537 0 2024-02-15T18:27:21.603 INFO:tasks.workunit.client.1.smithi191.stdout:0/798: sync 2024-02-15T18:27:21.603 INFO:tasks.workunit.client.1.smithi191.stdout:3/863: sync 2024-02-15T18:27:21.603 INFO:tasks.workunit.client.1.smithi191.stdout:9/954: symlink d0/d29/d48/d66/d92/dfd/l137 0 2024-02-15T18:27:21.606 INFO:tasks.workunit.client.1.smithi191.stdout:5/713: symlink d0/db5/leb 0 2024-02-15T18:27:21.606 INFO:tasks.workunit.client.1.smithi191.stdout:5/714: chown d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52 3409844 1 2024-02-15T18:27:21.607 INFO:tasks.workunit.client.1.smithi191.stdout:5/715: write d0/d23/d41/d28/f55 [897424,20405] 0 2024-02-15T18:27:21.608 INFO:tasks.workunit.client.1.smithi191.stdout:7/967: creat d0/db/d65/d11/d88/d9d/dd2/d154/f156 x:0 0 0 2024-02-15T18:27:21.613 INFO:tasks.workunit.client.1.smithi191.stdout:3/864: truncate d6/d27/d4d/f58 7531380 0 2024-02-15T18:27:21.615 INFO:tasks.workunit.client.1.smithi191.stdout:9/955: creat d0/d29/de2/f138 x:0 0 0 2024-02-15T18:27:21.617 INFO:tasks.workunit.client.1.smithi191.stdout:7/968: rename d0/db/d65/d11/d57/l72 to d0/db/d26/d5f/d77/db9/l157 0 2024-02-15T18:27:21.619 INFO:tasks.workunit.client.1.smithi191.stdout:3/865: creat d6/d27/d4d/d5a/f126 x:0 0 0 2024-02-15T18:27:21.621 INFO:tasks.workunit.client.1.smithi191.stdout:9/956: rename d0/d29/f86 to d0/d29/d48/d66/f139 0 2024-02-15T18:27:21.621 INFO:tasks.workunit.client.1.smithi191.stdout:9/957: write d0/d29/d48/fdd [2196167,5862] 0 2024-02-15T18:27:21.623 INFO:tasks.workunit.client.1.smithi191.stdout:7/969: creat d0/db/d65/d11/d5a/d66/dd6/f158 x:0 0 0 2024-02-15T18:27:21.623 INFO:tasks.workunit.client.1.smithi191.stdout:7/970: read d0/f80 [30325,41753] 0 2024-02-15T18:27:21.625 INFO:tasks.workunit.client.1.smithi191.stdout:5/716: dwrite d0/fa7 [0,4194304] 0 2024-02-15T18:27:21.625 INFO:tasks.workunit.client.1.smithi191.stdout:3/866: mknod d6/d1b/c127 0 2024-02-15T18:27:21.630 INFO:tasks.workunit.client.1.smithi191.stdout:5/717: truncate d0/d23/d41/d28/fcf 853165 0 2024-02-15T18:27:21.630 INFO:tasks.workunit.client.1.smithi191.stdout:3/867: rmdir d6/d8e 39 2024-02-15T18:27:21.632 INFO:tasks.workunit.client.1.smithi191.stdout:5/718: rmdir d0/d23/d41/d28/d31/d5a/d61/dda 39 2024-02-15T18:27:21.632 INFO:tasks.workunit.client.1.smithi191.stdout:5/719: readlink d0/d23/d41/d28/l5b 0 2024-02-15T18:27:21.634 INFO:tasks.workunit.client.1.smithi191.stdout:3/868: unlink d6/d3b/d42/d49/c47 0 2024-02-15T18:27:21.634 INFO:tasks.workunit.client.1.smithi191.stdout:3/869: truncate d6/d27/d4d/fc9 643317 0 2024-02-15T18:27:21.635 INFO:tasks.workunit.client.1.smithi191.stdout:3/870: chown d6/d1b/d60/d73/c114 0 1 2024-02-15T18:27:21.638 INFO:tasks.workunit.client.1.smithi191.stdout:3/871: symlink d6/d1b/d60/ddc/de5/l128 0 2024-02-15T18:27:21.638 INFO:tasks.workunit.client.1.smithi191.stdout:3/872: fdatasync d6/d1b/f81 0 2024-02-15T18:27:21.640 INFO:tasks.workunit.client.1.smithi191.stdout:9/958: dwrite d0/f12c [0,4194304] 0 2024-02-15T18:27:21.641 INFO:tasks.workunit.client.1.smithi191.stdout:5/720: dread d0/d23/d41/d28/d31/d5a/fe3 [0,4194304] 0 2024-02-15T18:27:21.643 INFO:tasks.workunit.client.1.smithi191.stdout:7/971: rmdir d0/db/d65/d11/d5a/d66/dd6 39 2024-02-15T18:27:21.652 INFO:tasks.workunit.client.1.smithi191.stdout:3/873: mknod d6/d3b/d42/d49/dbf/df7/c129 0 2024-02-15T18:27:21.652 INFO:tasks.workunit.client.1.smithi191.stdout:3/874: truncate d6/d1b/d60/d73/f74 3639962 0 2024-02-15T18:27:21.653 INFO:tasks.workunit.client.1.smithi191.stdout:3/875: write d6/d3b/d42/f51 [2811961,61726] 0 2024-02-15T18:27:21.658 INFO:tasks.workunit.client.1.smithi191.stdout:9/959: creat d0/d29/d10d/f13a x:0 0 0 2024-02-15T18:27:21.661 INFO:tasks.workunit.client.1.smithi191.stdout:7/972: truncate d0/db/d65/dc/fee 3821130 0 2024-02-15T18:27:21.671 INFO:tasks.workunit.client.1.smithi191.stdout:7/973: dread d0/db/d26/d5f/d77/db9/ffc [0,4194304] 0 2024-02-15T18:27:21.672 INFO:tasks.workunit.client.1.smithi191.stdout:7/974: mkdir d0/d159 0 2024-02-15T18:27:21.674 INFO:tasks.workunit.client.1.smithi191.stdout:7/975: getdents d0/db/d26/d5f/d77/d35/dc7 0 2024-02-15T18:27:21.677 INFO:tasks.workunit.client.1.smithi191.stdout:7/976: symlink d0/db/d26/d5f/l15a 0 2024-02-15T18:27:21.677 INFO:tasks.workunit.client.1.smithi191.stdout:3/876: dwrite d6/d3b/d30/f5c [0,4194304] 0 2024-02-15T18:27:21.678 INFO:tasks.workunit.client.1.smithi191.stdout:3/877: symlink d6/d5d/d72/dd5/l12a 0 2024-02-15T18:27:21.679 INFO:tasks.workunit.client.1.smithi191.stdout:9/960: dwrite d0/d7/dc/d30/d3b/d56/fd3 [0,4194304] 0 2024-02-15T18:27:21.679 INFO:tasks.workunit.client.1.smithi191.stdout:3/878: symlink d6/d5d/d72/dd5/db2/l12b 0 2024-02-15T18:27:21.679 INFO:tasks.workunit.client.1.smithi191.stdout:9/961: chown d0/d29/f70 61 1 2024-02-15T18:27:21.687 INFO:tasks.workunit.client.1.smithi191.stdout:3/879: mknod d6/c12c 0 2024-02-15T18:27:21.688 INFO:tasks.workunit.client.1.smithi191.stdout:9/962: dread d0/d29/d48/f96 [0,4194304] 0 2024-02-15T18:27:21.689 INFO:tasks.workunit.client.1.smithi191.stdout:3/880: creat d6/d27/d4d/d5a/d92/dd2/f12d x:0 0 0 2024-02-15T18:27:21.692 INFO:tasks.workunit.client.1.smithi191.stdout:9/963: creat d0/d7/f13b x:0 0 0 2024-02-15T18:27:21.693 INFO:tasks.workunit.client.1.smithi191.stdout:7/977: dwrite d0/db/d65/d11/d88/d9d/fda [4194304,4194304] 0 2024-02-15T18:27:21.693 INFO:tasks.workunit.client.1.smithi191.stdout:7/978: stat d0/db/d26/d5f/d77/df/d5d/ce3 0 2024-02-15T18:27:21.693 INFO:tasks.workunit.client.1.smithi191.stdout:7/979: chown d0/db/d65/dc/d4b 1093 1 2024-02-15T18:27:21.693 INFO:tasks.workunit.client.1.smithi191.stdout:3/881: creat d6/d27/d4d/da8/d106/f12e x:0 0 0 2024-02-15T18:27:21.695 INFO:tasks.workunit.client.1.smithi191.stdout:9/964: rename d0/d29/d48/d66/c103 to d0/d1d/d3f/c13c 0 2024-02-15T18:27:21.702 INFO:tasks.workunit.client.1.smithi191.stdout:7/980: mknod d0/db/d65/dc/c15b 0 2024-02-15T18:27:21.705 INFO:tasks.workunit.client.1.smithi191.stdout:7/981: link d0/db/d26/d5f/d77/df/d5d/l62 d0/db/d26/d5f/d77/d35/dc7/l15c 0 2024-02-15T18:27:21.705 INFO:tasks.workunit.client.1.smithi191.stdout:7/982: stat d0/db/d65/d11/d57/lfa 0 2024-02-15T18:27:21.705 INFO:tasks.workunit.client.1.smithi191.stdout:7/983: truncate d0/db/d26/f103 376878 0 2024-02-15T18:27:21.710 INFO:tasks.workunit.client.1.smithi191.stdout:7/984: symlink d0/db/d65/d11/d88/d9d/dd2/l15d 0 2024-02-15T18:27:21.713 INFO:tasks.workunit.client.1.smithi191.stdout:3/882: dwrite d6/d27/d4d/da8/d106/f12e [0,4194304] 0 2024-02-15T18:27:21.713 INFO:tasks.workunit.client.1.smithi191.stdout:7/985: rmdir d0/d159 0 2024-02-15T18:27:21.718 INFO:tasks.workunit.client.1.smithi191.stdout:9/965: dwrite d0/d29/d48/fc3 [4194304,4194304] 0 2024-02-15T18:27:21.718 INFO:tasks.workunit.client.1.smithi191.stdout:9/966: truncate d0/d29/d48/d66/d92/fca 215074 0 2024-02-15T18:27:21.718 INFO:tasks.workunit.client.1.smithi191.stdout:9/967: chown d0/d7/dc/d17/d44/fcc 3472 1 2024-02-15T18:27:21.733 INFO:tasks.workunit.client.1.smithi191.stdout:3/883: link d6/d1b/d60/l87 d6/d27/d4d/d5a/l12f 0 2024-02-15T18:27:21.733 INFO:tasks.workunit.client.1.smithi191.stdout:3/884: write d6/d27/d4d/da8/d106/f12e [1969420,116965] 0 2024-02-15T18:27:21.735 INFO:tasks.workunit.client.1.smithi191.stdout:7/986: mknod d0/db/d65/d11/d5a/d66/d1e/c15e 0 2024-02-15T18:27:21.735 INFO:tasks.workunit.client.1.smithi191.stdout:7/987: write d0/fcb [1569056,112] 0 2024-02-15T18:27:21.738 INFO:tasks.workunit.client.1.smithi191.stdout:7/988: dread d0/db/d65/d11/d5a/d66/f53 [0,4194304] 0 2024-02-15T18:27:21.746 INFO:tasks.workunit.client.1.smithi191.stdout:9/968: read d0/d7/dc/d32/de4/dec/d126/f4d [1693110,98731] 0 2024-02-15T18:27:21.746 INFO:tasks.workunit.client.1.smithi191.stdout:9/969: readlink d0/d7/dc/d8c/l113 0 2024-02-15T18:27:21.746 INFO:tasks.workunit.client.1.smithi191.stdout:9/970: dread - d0/d7/dc/d17/d44/de6/ff8 zero size 2024-02-15T18:27:21.755 INFO:tasks.workunit.client.1.smithi191.stdout:3/885: truncate d6/d27/d4d/d5a/fea 1725785 0 2024-02-15T18:27:21.755 INFO:tasks.workunit.client.1.smithi191.stdout:7/989: creat d0/db/d65/dc/d4b/de8/df0/f15f x:0 0 0 2024-02-15T18:27:21.756 INFO:tasks.workunit.client.1.smithi191.stdout:7/990: dread - d0/fc5 zero size 2024-02-15T18:27:21.757 INFO:tasks.workunit.client.1.smithi191.stdout:9/971: creat d0/d7/dc/d32/de4/dec/d126/d109/f13d x:0 0 0 2024-02-15T18:27:21.762 INFO:tasks.workunit.client.1.smithi191.stdout:3/886: link d6/d1b/d60/d73/l35 d6/d3b/d42/dbb/l130 0 2024-02-15T18:27:21.763 INFO:tasks.workunit.client.1.smithi191.stdout:9/972: symlink d0/d7/dc/d32/l13e 0 2024-02-15T18:27:21.773 INFO:tasks.workunit.client.1.smithi191.stdout:3/887: rename d6/d1b/d60/d73/d5e/f11b to d6/f131 0 2024-02-15T18:27:21.773 INFO:tasks.workunit.client.1.smithi191.stdout:3/888: dread - d6/d3b/d42/d49/d54/faf zero size 2024-02-15T18:27:21.776 INFO:tasks.workunit.client.1.smithi191.stdout:9/973: link d0/d1d/d3f/l53 d0/d7/dc/d32/de4/dec/l13f 0 2024-02-15T18:27:21.778 INFO:tasks.workunit.client.1.smithi191.stdout:3/889: unlink f4 0 2024-02-15T18:27:21.788 INFO:tasks.workunit.client.1.smithi191.stdout:7/991: dwrite d0/db/d65/d11/d5a/d66/d1e/faa [0,4194304] 0 2024-02-15T18:27:21.788 INFO:tasks.workunit.client.1.smithi191.stdout:7/992: readlink d0/db/d26/d5f/d77/la4 0 2024-02-15T18:27:21.790 INFO:tasks.workunit.client.1.smithi191.stdout:9/974: dread d0/d1d/d3f/d80/fa2 [0,4194304] 0 2024-02-15T18:27:21.791 INFO:tasks.workunit.client.1.smithi191.stdout:9/975: write d0/d7/dc/d8c/f35 [5776574,1105] 0 2024-02-15T18:27:21.798 INFO:tasks.workunit.client.1.smithi191.stdout:3/890: getdents d6/d1b/d60/da2/dac/de2/d113 0 2024-02-15T18:27:21.798 INFO:tasks.workunit.client.1.smithi191.stdout:3/891: chown d6/d27/d4d/d5a/d92/dd2/f10b 11 1 2024-02-15T18:27:21.798 INFO:tasks.workunit.client.1.smithi191.stdout:3/892: readlink d6/lce 0 2024-02-15T18:27:21.801 INFO:tasks.workunit.client.1.smithi191.stdout:7/993: link d0/f3a d0/db/d65/dc/d4b/f160 0 2024-02-15T18:27:21.804 INFO:tasks.workunit.client.1.smithi191.stdout:7/994: dread d0/db/d65/d11/d1c/fb3 [0,4194304] 0 2024-02-15T18:27:21.806 INFO:tasks.workunit.client.1.smithi191.stdout:7/995: creat d0/db/d26/d5f/d77/d35/d42/f161 x:0 0 0 2024-02-15T18:27:21.810 INFO:tasks.workunit.client.1.smithi191.stdout:7/996: mknod d0/db/d65/d11/d88/d9d/dd2/c162 0 2024-02-15T18:27:21.811 INFO:tasks.workunit.client.1.smithi191.stdout:3/893: dread d6/d27/d4d/d5a/d92/dd2/f10b [0,4194304] 0 2024-02-15T18:27:21.811 INFO:tasks.workunit.client.1.smithi191.stdout:3/894: readlink d6/d1b/d60/da2/lf2 0 2024-02-15T18:27:21.813 INFO:tasks.workunit.client.1.smithi191.stdout:3/895: mknod d6/d1b/db0/dc1/c132 0 2024-02-15T18:27:21.816 INFO:tasks.workunit.client.1.smithi191.stdout:3/896: link d6/d3b/d42/l33 d6/d27/d4d/d5a/d92/dd2/l133 0 2024-02-15T18:27:21.827 INFO:tasks.workunit.client.1.smithi191.stdout:3/897: dread d6/d3b/d30/f46 [0,4194304] 0 2024-02-15T18:27:21.836 INFO:tasks.workunit.client.1.smithi191.stdout:3/898: dread d6/d5d/d72/f97 [0,4194304] 0 2024-02-15T18:27:21.836 INFO:tasks.workunit.client.1.smithi191.stdout:3/899: dread - d6/d3b/d42/d49/dbf/df7/f105 zero size 2024-02-15T18:27:21.837 INFO:tasks.workunit.client.1.smithi191.stdout:3/900: mkdir d6/d27/d4d/d5a/d92/d134 0 2024-02-15T18:27:21.844 INFO:tasks.workunit.client.1.smithi191.stdout:3/901: link d6/d1b/d60/da2/lf2 d6/d1b/d60/d73/l135 0 2024-02-15T18:27:21.846 INFO:tasks.workunit.client.1.smithi191.stdout:3/902: creat d6/d5d/df3/d107/f136 x:0 0 0 2024-02-15T18:27:21.854 INFO:tasks.workunit.client.1.smithi191.stdout:3/903: dread d6/d3b/d42/d49/d54/f83 [0,4194304] 0 2024-02-15T18:27:21.857 INFO:tasks.workunit.client.1.smithi191.stdout:3/904: rmdir d6/d1b/d60/d73/d5e/d123 0 2024-02-15T18:27:21.860 INFO:tasks.workunit.client.1.smithi191.stdout:3/905: rmdir d6/d3b/d42/d49/dc7/de3 39 2024-02-15T18:27:21.885 INFO:tasks.workunit.client.1.smithi191.stdout:3/906: dwrite d6/d5d/df3/d107/f136 [0,4194304] 0 2024-02-15T18:27:21.885 INFO:tasks.workunit.client.1.smithi191.stdout:0/799: sync 2024-02-15T18:27:21.885 INFO:tasks.workunit.client.1.smithi191.stdout:0/800: chown d7/da/d1a/d34/d27/f7c 2508 1 2024-02-15T18:27:21.885 INFO:tasks.workunit.client.1.smithi191.stdout:0/801: truncate d7/da/d17/d31/d7d/fc8 4379455 0 2024-02-15T18:27:21.885 INFO:tasks.workunit.client.1.smithi191.stdout:5/721: sync 2024-02-15T18:27:21.885 INFO:tasks.workunit.client.1.smithi191.stdout:5/722: dread - d0/d23/d41/d28/d31/d5a/fa6 zero size 2024-02-15T18:27:21.885 INFO:tasks.workunit.client.1.smithi191.stdout:5/723: readlink d0/d23/d41/d28/l5b 0 2024-02-15T18:27:21.885 INFO:tasks.workunit.client.1.smithi191.stdout:3/907: write d6/d3b/fba [1544111,9307] 0 2024-02-15T18:27:21.887 INFO:tasks.workunit.client.1.smithi191.stdout:0/802: mknod d7/d3b/dfd/c110 0 2024-02-15T18:27:21.891 INFO:tasks.workunit.client.1.smithi191.stdout:0/803: write d7/da/d6c/fbf [1522542,88825] 0 2024-02-15T18:27:21.893 INFO:tasks.workunit.client.1.smithi191.stdout:5/724: write d0/d18/f42 [4456366,56594] 0 2024-02-15T18:27:21.895 INFO:tasks.workunit.client.1.smithi191.stdout:3/908: symlink d6/d27/d4d/da8/l137 0 2024-02-15T18:27:21.895 INFO:tasks.workunit.client.1.smithi191.stdout:3/909: chown d6/d27/d4d/d5a/fe1 529 1 2024-02-15T18:27:21.896 INFO:tasks.workunit.client.1.smithi191.stdout:0/804: rmdir d7/da/d17/d31/d5f/dae/de2 39 2024-02-15T18:27:21.899 INFO:tasks.workunit.client.1.smithi191.stdout:5/725: creat d0/d23/d41/d28/fec x:0 0 0 2024-02-15T18:27:21.899 INFO:tasks.workunit.client.1.smithi191.stdout:3/910: dread d6/d1b/d60/d73/f74 [0,4194304] 0 2024-02-15T18:27:21.899 INFO:tasks.workunit.client.1.smithi191.stdout:0/805: link d7/da/d17/l37 d7/da/d17/daa/l111 0 2024-02-15T18:27:21.904 INFO:tasks.workunit.client.1.smithi191.stdout:0/806: creat d7/d94/f112 x:0 0 0 2024-02-15T18:27:21.910 INFO:tasks.workunit.client.1.smithi191.stdout:3/911: dread d6/d5d/d72/dd5/db2/fdf [0,4194304] 0 2024-02-15T18:27:21.911 INFO:tasks.workunit.client.1.smithi191.stdout:3/912: creat d6/d1b/d60/d73/d5e/f138 x:0 0 0 2024-02-15T18:27:21.911 INFO:tasks.workunit.client.1.smithi191.stdout:3/913: write d6/d27/d4d/da8/fb1 [4627465,61252] 0 2024-02-15T18:27:21.913 INFO:tasks.workunit.client.1.smithi191.stdout:3/914: creat d6/d27/d4d/da8/f139 x:0 0 0 2024-02-15T18:27:21.920 INFO:tasks.workunit.client.1.smithi191.stdout:3/915: rename d6/d5d/df3/d107 to d6/d27/d4d/d5a/da4/d13a 0 2024-02-15T18:27:21.924 INFO:tasks.workunit.client.1.smithi191.stdout:5/726: dwrite d0/d18/d5c/fd2 [4194304,4194304] 0 2024-02-15T18:27:21.926 INFO:tasks.workunit.client.1.smithi191.stdout:0/807: dwrite d7/da/d1a/fa5 [0,4194304] 0 2024-02-15T18:27:21.934 INFO:tasks.workunit.client.1.smithi191.stdout:5/727: symlink d0/d23/d41/d28/d31/d5a/d61/dda/led 0 2024-02-15T18:27:21.936 INFO:tasks.workunit.client.1.smithi191.stdout:5/728: symlink d0/d18/d5c/d7c/lee 0 2024-02-15T18:27:21.939 INFO:tasks.workunit.client.1.smithi191.stdout:5/729: creat d0/d23/d41/d28/d31/dc2/fef x:0 0 0 2024-02-15T18:27:21.940 INFO:tasks.workunit.client.1.smithi191.stdout:5/730: symlink d0/d23/d41/d28/d31/lf0 0 2024-02-15T18:27:21.941 INFO:tasks.workunit.client.1.smithi191.stdout:5/731: read d0/d23/d41/f87 [307392,102016] 0 2024-02-15T18:27:21.950 INFO:tasks.workunit.client.1.smithi191.stdout:0/808: dwrite d7/d1b/f46 [0,4194304] 0 2024-02-15T18:27:21.950 INFO:tasks.workunit.client.1.smithi191.stdout:3/916: dwrite d6/d1b/f39 [0,4194304] 0 2024-02-15T18:27:21.954 INFO:tasks.workunit.client.1.smithi191.stdout:0/809: mkdir d7/da/d1a/d2c/d4c/d113 0 2024-02-15T18:27:21.954 INFO:tasks.workunit.client.1.smithi191.stdout:0/810: fdatasync d7/da/d17/d31/f87 0 2024-02-15T18:27:21.954 INFO:tasks.workunit.client.1.smithi191.stdout:0/811: readlink d7/d1b/l75 0 2024-02-15T18:27:21.955 INFO:tasks.workunit.client.1.smithi191.stdout:5/732: dread d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/f60 [0,4194304] 0 2024-02-15T18:27:21.958 INFO:tasks.workunit.client.1.smithi191.stdout:3/917: creat d6/d1b/db0/dc1/f13b x:0 0 0 2024-02-15T18:27:21.958 INFO:tasks.workunit.client.1.smithi191.stdout:3/918: stat d6/d1b/d60/d73/c28 0 2024-02-15T18:27:21.958 INFO:tasks.workunit.client.1.smithi191.stdout:0/812: symlink d7/d1b/d30/l114 0 2024-02-15T18:27:21.960 INFO:tasks.workunit.client.1.smithi191.stdout:5/733: rename d0/d18/d5c/d7c/d49/d9f/la8 to d0/d23/lf1 0 2024-02-15T18:27:21.961 INFO:tasks.workunit.client.1.smithi191.stdout:0/813: link d7/d4a/dc0/cd6 d7/da/d17/d31/c115 0 2024-02-15T18:27:21.963 INFO:tasks.workunit.client.1.smithi191.stdout:5/734: mknod d0/d18/d5c/d7c/d49/d9f/dc0/d85/cf2 0 2024-02-15T18:27:21.964 INFO:tasks.workunit.client.1.smithi191.stdout:0/814: creat d7/d4a/dc0/f116 x:0 0 0 2024-02-15T18:27:21.967 INFO:tasks.workunit.client.1.smithi191.stdout:0/815: rename d7/d3b/f69 to d7/da/d1a/f117 0 2024-02-15T18:27:21.968 INFO:tasks.workunit.client.1.smithi191.stdout:3/919: dread d6/d1b/d60/d73/f34 [0,4194304] 0 2024-02-15T18:27:21.968 INFO:tasks.workunit.client.1.smithi191.stdout:3/920: chown d6/d5d/daa/f11a 696250 1 2024-02-15T18:27:21.968 INFO:tasks.workunit.client.1.smithi191.stdout:0/816: dread d7/d1b/f81 [0,4194304] 0 2024-02-15T18:27:21.971 INFO:tasks.workunit.client.1.smithi191.stdout:3/921: creat d6/f13c x:0 0 0 2024-02-15T18:27:21.971 INFO:tasks.workunit.client.1.smithi191.stdout:3/922: stat d6/d1b/d60/da2/lfb 0 2024-02-15T18:27:21.971 INFO:tasks.workunit.client.1.smithi191.stdout:3/923: dread - d6/d27/d4d/da8/f139 zero size 2024-02-15T18:27:21.977 INFO:tasks.workunit.client.1.smithi191.stdout:0/817: dread d7/d4a/fd4 [0,4194304] 0 2024-02-15T18:27:21.977 INFO:tasks.workunit.client.1.smithi191.stdout:0/818: write d7/d4a/f85 [1253033,80418] 0 2024-02-15T18:27:21.977 INFO:tasks.workunit.client.1.smithi191.stdout:0/819: truncate d7/da/d1a/ff2 1634389 0 2024-02-15T18:27:21.978 INFO:tasks.workunit.client.1.smithi191.stdout:3/924: dread d6/d27/d4d/d5a/da4/fc0 [0,4194304] 0 2024-02-15T18:27:21.978 INFO:tasks.workunit.client.1.smithi191.stdout:0/820: symlink d7/da/d17/d31/d5f/dae/de2/l118 0 2024-02-15T18:27:21.980 INFO:tasks.workunit.client.1.smithi191.stdout:3/925: rename d6/d1b/d60/da2/dac/de2/l11c to d6/d27/d4d/d5a/da4/l13d 0 2024-02-15T18:27:21.981 INFO:tasks.workunit.client.1.smithi191.stdout:0/821: mknod d7/da/d17/d31/d5f/d103/c119 0 2024-02-15T18:27:21.981 INFO:tasks.workunit.client.1.smithi191.stdout:0/822: chown d7/da/d17 1718 1 2024-02-15T18:27:21.982 INFO:tasks.workunit.client.1.smithi191.stdout:3/926: creat d6/d1b/d60/ddc/de5/f13e x:0 0 0 2024-02-15T18:27:21.982 INFO:tasks.workunit.client.1.smithi191.stdout:0/823: creat d7/da/d17/d31/d5f/d103/f11a x:0 0 0 2024-02-15T18:27:21.985 INFO:tasks.workunit.client.1.smithi191.stdout:0/824: mknod d7/da/d1a/d2c/c11b 0 2024-02-15T18:27:21.986 INFO:tasks.workunit.client.1.smithi191.stdout:0/825: creat d7/da/d17/daa/f11c x:0 0 0 2024-02-15T18:27:21.988 INFO:tasks.workunit.client.1.smithi191.stdout:0/826: mkdir d7/d3b/d11d 0 2024-02-15T18:27:21.990 INFO:tasks.workunit.client.1.smithi191.stdout:0/827: symlink d7/d3b/l11e 0 2024-02-15T18:27:21.991 INFO:tasks.workunit.client.1.smithi191.stdout:0/828: rename d7/da/d17/d31/d5f/d9e/dc3/dd8/ffe to d7/da/d17/d31/d5f/dae/de2/dec/f11f 0 2024-02-15T18:27:21.992 INFO:tasks.workunit.client.1.smithi191.stdout:0/829: chown d7/d1b/l21 10019 1 2024-02-15T18:27:21.992 INFO:tasks.workunit.client.1.smithi191.stdout:0/830: dread - d7/da/d1a/d2c/f10b zero size 2024-02-15T18:27:21.992 INFO:tasks.workunit.client.1.smithi191.stdout:0/831: readlink d7/da/d1a/l84 0 2024-02-15T18:27:21.992 INFO:tasks.workunit.client.1.smithi191.stdout:0/832: creat d7/da/d1a/d34/d27/f120 x:0 0 0 2024-02-15T18:27:21.994 INFO:tasks.workunit.client.1.smithi191.stdout:0/833: symlink d7/da/d1a/d2c/d4c/d113/l121 0 2024-02-15T18:27:21.994 INFO:tasks.workunit.client.1.smithi191.stdout:0/834: truncate d7/da/d17/d31/d5f/f5b 1284207 0 2024-02-15T18:27:21.994 INFO:tasks.workunit.client.1.smithi191.stdout:0/835: dread - d7/da/d17/d31/d5f/dae/dd3/fe6 zero size 2024-02-15T18:27:22.006 INFO:tasks.workunit.client.1.smithi191.stdout:0/836: truncate d7/d94/fad 433120 0 2024-02-15T18:27:22.008 INFO:tasks.workunit.client.1.smithi191.stdout:0/837: mkdir d7/da/d1a/d2c/d4c/dd1/d122 0 2024-02-15T18:27:22.008 INFO:tasks.workunit.client.1.smithi191.stdout:0/838: truncate d7/d1b/d30/f43 691508 0 2024-02-15T18:27:22.010 INFO:tasks.workunit.client.1.smithi191.stdout:0/839: mkdir d7/da/d1a/d123 0 2024-02-15T18:27:22.018 INFO:tasks.workunit.client.1.smithi191.stdout:0/840: dread d7/d1b/f23 [0,4194304] 0 2024-02-15T18:27:22.019 INFO:tasks.workunit.client.1.smithi191.stdout:0/841: getdents d7/da/d17/d31/d5f/dae/de2/dec 0 2024-02-15T18:27:22.021 INFO:tasks.workunit.client.1.smithi191.stdout:0/842: creat d7/da/d17/d31/d5f/dae/f124 x:0 0 0 2024-02-15T18:27:22.041 INFO:tasks.workunit.client.1.smithi191.stdout:0/843: dwrite d7/d4a/f7b [0,4194304] 0 2024-02-15T18:27:22.041 INFO:tasks.workunit.client.1.smithi191.stdout:0/844: write d7/da/d17/d31/f106 [386136,16014] 0 2024-02-15T18:27:22.041 INFO:tasks.workunit.client.1.smithi191.stdout:0/845: write d7/d4a/dc0/fda [712358,122701] 0 2024-02-15T18:27:22.041 INFO:tasks.workunit.client.1.smithi191.stdout:0/846: write d7/da/d17/d31/d5f/dae/dd3/fe6 [885035,72425] 0 2024-02-15T18:27:22.042 INFO:tasks.workunit.client.1.smithi191.stdout:0/847: creat d7/da/d17/d31/d5f/d9e/dc3/f125 x:0 0 0 2024-02-15T18:27:22.056 INFO:tasks.workunit.client.1.smithi191.stdout:0/848: dwrite d7/da/d1a/d34/d27/f60 [0,4194304] 0 2024-02-15T18:27:22.070 INFO:tasks.workunit.client.1.smithi191.stdout:0/849: dwrite d7/d94/fad [0,4194304] 0 2024-02-15T18:27:22.072 INFO:tasks.workunit.client.1.smithi191.stdout:0/850: symlink d7/da/d17/l126 0 2024-02-15T18:27:22.072 INFO:tasks.workunit.client.1.smithi191.stdout:0/851: write d7/da/d1a/d2c/f86 [1434470,78256] 0 2024-02-15T18:27:22.075 INFO:tasks.workunit.client.1.smithi191.stdout:0/852: dread d7/da/d1a/ff2 [0,4194304] 0 2024-02-15T18:27:22.077 INFO:tasks.workunit.client.1.smithi191.stdout:0/853: rename d7/d4a/dc0/dcc to d7/da/d17/d31/d5f/d9e/dc3/dd8/d105/d127 0 2024-02-15T18:27:22.095 INFO:tasks.workunit.client.1.smithi191.stdout:0/854: dwrite d7/d94/f112 [0,4194304] 0 2024-02-15T18:27:22.095 INFO:tasks.workunit.client.1.smithi191.stdout:0/855: readlink d7/da/d17/d31/d5f/d9e/dc3/lf7 0 2024-02-15T18:27:22.097 INFO:tasks.workunit.client.1.smithi191.stdout:0/856: creat d7/d3b/d11d/f128 x:0 0 0 2024-02-15T18:27:22.111 INFO:tasks.workunit.client.1.smithi191.stdout:0/857: dwrite d7/da/d6c/fbf [0,4194304] 0 2024-02-15T18:27:22.112 INFO:tasks.workunit.client.1.smithi191.stdout:0/858: dread - d7/d94/ffb zero size 2024-02-15T18:27:22.112 INFO:tasks.workunit.client.1.smithi191.stdout:0/859: read d7/d4a/dc0/fcd [1496311,72561] 0 2024-02-15T18:27:22.112 INFO:tasks.workunit.client.1.smithi191.stdout:0/860: write d7/d3b/d11d/f128 [646976,80341] 0 2024-02-15T18:27:22.114 INFO:tasks.workunit.client.1.smithi191.stdout:0/861: symlink d7/da/d17/d31/d5f/d9e/dc3/l129 0 2024-02-15T18:27:22.115 INFO:tasks.workunit.client.1.smithi191.stdout:0/862: creat d7/da/d17/d31/d7d/f12a x:0 0 0 2024-02-15T18:27:22.116 INFO:tasks.workunit.client.1.smithi191.stdout:0/863: symlink d7/da/d17/d31/d7d/df9/l12b 0 2024-02-15T18:27:22.120 INFO:tasks.workunit.client.1.smithi191.stdout:0/864: unlink d7/d94/laf 0 2024-02-15T18:27:22.123 INFO:tasks.workunit.client.1.smithi191.stdout:0/865: rename d7/da/d1a/d34/d27 to d7/da/d17/d31/d5f/d9e/dc3/dd8/d12c 0 2024-02-15T18:27:22.124 INFO:tasks.workunit.client.1.smithi191.stdout:0/866: write d7/d4a/fd4 [116797,109660] 0 2024-02-15T18:27:22.147 INFO:tasks.workunit.client.1.smithi191.stdout:0/867: dwrite d7/d4a/dc0/fda [0,4194304] 0 2024-02-15T18:27:22.147 INFO:tasks.workunit.client.1.smithi191.stdout:0/868: dread - d7/da/d17/d31/d5f/d9e/dc3/dd8/d12c/f120 zero size 2024-02-15T18:27:22.171 INFO:tasks.workunit.client.1.smithi191.stdout:0/869: dwrite d7/d4a/dc0/fda [0,4194304] 0 2024-02-15T18:27:22.171 INFO:tasks.workunit.client.1.smithi191.stdout:0/870: fsync d7/da/d1a/d34/f29 0 2024-02-15T18:27:22.172 INFO:tasks.workunit.client.1.smithi191.stdout:0/871: rmdir d7/da/d17/d31/d5f/d9e 39 2024-02-15T18:27:22.173 INFO:tasks.workunit.client.1.smithi191.stdout:0/872: symlink d7/da/d17/d31/d5f/dae/l12d 0 2024-02-15T18:27:22.241 INFO:tasks.workunit.client.1.smithi191.stdout:0/873: dwrite d7/da/d1a/f90 [0,4194304] 0 2024-02-15T18:27:22.245 INFO:tasks.workunit.client.1.smithi191.stdout:0/874: symlink d7/d1b/l12e 0 2024-02-15T18:27:22.245 INFO:tasks.workunit.client.1.smithi191.stdout:0/875: chown d7/da/d1a/l2d 137 1 2024-02-15T18:27:22.246 INFO:tasks.workunit.client.1.smithi191.stdout:0/876: symlink d7/d3b/dfd/l12f 0 2024-02-15T18:27:22.248 INFO:tasks.workunit.client.1.smithi191.stdout:0/877: creat d7/d3b/d11d/f130 x:0 0 0 2024-02-15T18:27:22.262 INFO:tasks.workunit.client.1.smithi191.stdout:0/878: dwrite d7/da/d17/d31/d5f/dae/de2/dec/f11f [0,4194304] 0 2024-02-15T18:27:22.265 INFO:tasks.workunit.client.1.smithi191.stdout:0/879: truncate d7/da/d17/d31/d7d/fc8 1433127 0 2024-02-15T18:27:22.283 INFO:tasks.workunit.client.1.smithi191.stdout:0/880: dwrite d7/fb [0,4194304] 0 2024-02-15T18:27:22.294 INFO:tasks.workunit.client.1.smithi191.stdout:9/976: sync 2024-02-15T18:27:22.295 INFO:tasks.workunit.client.1.smithi191.stdout:9/977: chown d0/d7/f13b 0 1 2024-02-15T18:27:22.295 INFO:tasks.workunit.client.1.smithi191.stdout:9/978: chown d0/d29/d48/d66/f68 1041835 1 2024-02-15T18:27:22.295 INFO:tasks.workunit.client.1.smithi191.stdout:9/979: write d0/d7/dc/d17/d44/de6/ff8 [377078,102726] 0 2024-02-15T18:27:22.296 INFO:tasks.workunit.client.1.smithi191.stdout:9/980: symlink d0/d7/dbf/df3/l140 0 2024-02-15T18:27:22.298 INFO:tasks.workunit.client.1.smithi191.stdout:9/981: write d0/d29/d48/f96 [1478093,74152] 0 2024-02-15T18:27:22.298 INFO:tasks.workunit.client.1.smithi191.stdout:9/982: chown d0/d7/dc/d30/d3b/ffb 9869 1 2024-02-15T18:27:22.298 INFO:tasks.workunit.client.1.smithi191.stdout:9/983: chown d0/d7/dc/d32/d106/dce 2158 1 2024-02-15T18:27:22.300 INFO:tasks.workunit.client.1.smithi191.stdout:9/984: rename d0/d7/dc/d30/d4f/f6d to d0/d7/dbf/df3/f141 0 2024-02-15T18:27:22.301 INFO:tasks.workunit.client.1.smithi191.stdout:9/985: symlink d0/d7/dc/d32/de4/dec/l142 0 2024-02-15T18:27:22.303 INFO:tasks.workunit.client.1.smithi191.stdout:9/986: mknod d0/d7/dc/d30/d3b/c143 0 2024-02-15T18:27:22.303 INFO:tasks.workunit.client.1.smithi191.stdout:9/987: chown d0/d7/dc/d17/d44/fd8 6029149 1 2024-02-15T18:27:22.306 INFO:tasks.workunit.client.1.smithi191.stdout:0/881: dwrite d7/da/d17/d31/f87 [0,4194304] 0 2024-02-15T18:27:22.306 INFO:tasks.workunit.client.1.smithi191.stdout:0/882: fdatasync d7/d4a/fd4 0 2024-02-15T18:27:22.309 INFO:tasks.workunit.client.1.smithi191.stdout:0/883: dread d7/da/d17/d31/f89 [0,4194304] 0 2024-02-15T18:27:22.312 INFO:tasks.workunit.client.1.smithi191.stdout:9/988: dread d0/d7/dc/d8c/f72 [0,4194304] 0 2024-02-15T18:27:22.313 INFO:tasks.workunit.client.1.smithi191.stdout:9/989: mknod d0/d1d/d3f/c144 0 2024-02-15T18:27:22.314 INFO:tasks.workunit.client.1.smithi191.stdout:9/990: chown d0/d7/dc/d17/f25 35 1 2024-02-15T18:27:22.314 INFO:tasks.workunit.client.1.smithi191.stdout:9/991: write d0/d7/dc/d17/d44/de6/fe1 [579320,50312] 0 2024-02-15T18:27:22.315 INFO:tasks.workunit.client.1.smithi191.stdout:9/992: write d0/d29/d48/d66/d92/fd4 [1482192,109086] 0 2024-02-15T18:27:22.316 INFO:tasks.workunit.client.1.smithi191.stdout:9/993: read d0/d7/dc/d17/f99 [15411,62199] 0 2024-02-15T18:27:22.325 INFO:tasks.workunit.client.1.smithi191.stdout:0/884: dwrite d7/da/d17/fc9 [0,4194304] 0 2024-02-15T18:27:22.328 INFO:tasks.workunit.client.1.smithi191.stdout:0/885: mknod d7/d3b/dfd/c131 0 2024-02-15T18:27:22.328 INFO:tasks.workunit.client.1.smithi191.stdout:0/886: stat d7/d1b/d30 0 2024-02-15T18:27:22.328 INFO:tasks.workunit.client.1.smithi191.stdout:0/887: truncate d7/d94/fb9 332299 0 2024-02-15T18:27:22.329 INFO:tasks.workunit.client.1.smithi191.stdout:0/888: mkdir d7/d4a/dc0/d132 0 2024-02-15T18:27:22.331 INFO:tasks.workunit.client.1.smithi191.stdout:0/889: getdents d7 0 2024-02-15T18:27:22.332 INFO:tasks.workunit.client.1.smithi191.stdout:0/890: mkdir d7/da/d17/d31/d5f/d133 0 2024-02-15T18:27:22.334 INFO:tasks.workunit.client.1.smithi191.stdout:9/994: dwrite d0/d7/dc/d30/f5c [8388608,4194304] 0 2024-02-15T18:27:22.334 INFO:tasks.workunit.client.1.smithi191.stdout:9/995: read d0/d7/dc/fa5 [24062,11689] 0 2024-02-15T18:27:22.337 INFO:tasks.workunit.client.1.smithi191.stdout:9/996: link d0/d7/dc/d32/de4/f112 d0/d29/d48/d66/f145 0 2024-02-15T18:27:22.337 INFO:tasks.workunit.client.1.smithi191.stdout:9/997: fdatasync d0/d7/dc/d8c/f1c 0 2024-02-15T18:27:22.341 INFO:tasks.workunit.client.1.smithi191.stdout:9/998: symlink d0/d7/dc/d32/d106/dce/l146 0 2024-02-15T18:27:22.343 INFO:tasks.workunit.client.1.smithi191.stdout:9/999: unlink d0/d7/dc/d32/d106/dce/cf5 0 2024-02-15T18:27:22.406 INFO:tasks.workunit.client.1.smithi191.stdout:7/997: sync 2024-02-15T18:27:22.410 INFO:tasks.workunit.client.1.smithi191.stdout:7/998: unlink d0/db/d26/d5f/d77/db9/f127 0 2024-02-15T18:27:22.410 INFO:tasks.workunit.client.1.smithi191.stdout:7/999: stat d0/db/d65/dc/d120 0 2024-02-15T18:27:22.445 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:22 smithi191 bash[59457]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191 2024-02-15T18:27:22.557 INFO:tasks.workunit.client.1.smithi191.stdout:5/735: sync 2024-02-15T18:27:22.558 INFO:tasks.workunit.client.1.smithi191.stdout:5/736: read d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/f60 [244452,87465] 0 2024-02-15T18:27:22.564 INFO:tasks.workunit.client.1.smithi191.stdout:5/737: dread d0/f1a [0,4194304] 0 2024-02-15T18:27:22.566 INFO:tasks.workunit.client.1.smithi191.stdout:5/738: mknod d0/d18/d5c/dea/cf3 0 2024-02-15T18:27:22.582 INFO:tasks.workunit.client.1.smithi191.stdout:5/739: dwrite d0/d23/d41/d28/fec [0,4194304] 0 2024-02-15T18:27:22.582 INFO:tasks.workunit.client.1.smithi191.stdout:5/740: write d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/fc7 [767423,122778] 0 2024-02-15T18:27:22.582 INFO:tasks.workunit.client.1.smithi191.stdout:5/741: chown d0/d23/d41/d28/d31/d5a/db2/fe6 401598 1 2024-02-15T18:27:22.584 INFO:tasks.workunit.client.1.smithi191.stdout:5/742: truncate d0/d18/d5c/d7c/d49/d9f/fdb 284918 0 2024-02-15T18:27:22.585 INFO:tasks.workunit.client.1.smithi191.stdout:5/743: getdents d0/d18/d5c/d7c/d49/d9f/dc0 0 2024-02-15T18:27:22.604 INFO:tasks.workunit.client.1.smithi191.stdout:5/744: dwrite d0/d23/f43 [0,4194304] 0 2024-02-15T18:27:22.606 INFO:tasks.workunit.client.1.smithi191.stdout:5/745: write d0/d18/f7a [539250,98550] 0 2024-02-15T18:27:22.606 INFO:tasks.workunit.client.1.smithi191.stdout:5/746: write d0/d18/d5c/fd8 [602272,19244] 0 2024-02-15T18:27:22.611 INFO:tasks.workunit.client.1.smithi191.stdout:5/747: unlink d0/d23/d41/d28/d31/d5a/d61/c88 0 2024-02-15T18:27:22.612 INFO:tasks.workunit.client.1.smithi191.stdout:5/748: creat d0/ff4 x:0 0 0 2024-02-15T18:27:22.612 INFO:tasks.workunit.client.1.smithi191.stdout:5/749: fsync d0/d23/d41/d28/d31/fb8 0 2024-02-15T18:27:22.616 INFO:tasks.workunit.client.1.smithi191.stdout:5/750: rmdir d0/d23/d41/d28/d31/d5a/d61/dda 39 2024-02-15T18:27:22.618 INFO:tasks.workunit.client.1.smithi191.stdout:5/751: symlink d0/d23/d41/d28/d31/d5a/d61/de5/lf5 0 2024-02-15T18:27:22.622 INFO:tasks.workunit.client.1.smithi191.stdout:5/752: truncate d0/d23/d41/d28/f3f 1086040 0 2024-02-15T18:27:22.623 INFO:tasks.workunit.client.1.smithi191.stdout:5/753: read d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/f60 [3912318,50019] 0 2024-02-15T18:27:22.624 INFO:tasks.workunit.client.1.smithi191.stdout:5/754: mkdir d0/d18/d5c/d7c/d49/df6 0 2024-02-15T18:27:22.624 INFO:tasks.workunit.client.1.smithi191.stdout:5/755: stat d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/fdc 0 2024-02-15T18:27:22.624 INFO:tasks.workunit.client.1.smithi191.stdout:5/756: fsync d0/d18/fbb 0 2024-02-15T18:27:22.625 INFO:tasks.workunit.client.1.smithi191.stdout:5/757: write d0/d18/d5c/fd2 [5875229,111058] 0 2024-02-15T18:27:22.625 INFO:tasks.workunit.client.1.smithi191.stdout:5/758: chown d0/d23/d41/l6b 149055 1 2024-02-15T18:27:22.628 INFO:tasks.workunit.client.1.smithi191.stdout:0/891: sync 2024-02-15T18:27:22.628 INFO:tasks.workunit.client.1.smithi191.stdout:3/927: sync 2024-02-15T18:27:22.633 INFO:tasks.workunit.client.1.smithi191.stdout:3/928: readlink d6/d3b/d42/l9a 0 2024-02-15T18:27:22.710 INFO:tasks.workunit.client.1.smithi191.stdout:3/929: creat d6/d27/d4d/da8/f13f x:0 0 0 2024-02-15T18:27:22.710 INFO:tasks.workunit.client.1.smithi191.stdout:3/930: chown d6/d3b/d42/c115 1345 1 2024-02-15T18:27:22.712 INFO:tasks.workunit.client.1.smithi191.stdout:3/931: mknod d6/d3b/d42/d49/dc7/de3/c140 0 2024-02-15T18:27:22.714 INFO:tasks.workunit.client.1.smithi191.stdout:3/932: mknod d6/d1b/d60/ddc/de5/c141 0 2024-02-15T18:27:22.730 INFO:tasks.workunit.client.1.smithi191.stdout:3/933: dwrite d6/d1b/d60/d73/f8d [0,4194304] 0 2024-02-15T18:27:22.730 INFO:tasks.workunit.client.1.smithi191.stdout:3/934: readlink d6/l44 0 2024-02-15T18:27:22.740 INFO:tasks.workunit.client.1.smithi191.stdout:3/935: rename d6/d1b/d60/d73/d5e/f138 to d6/d3b/d42/d49/dc7/de3/f142 0 2024-02-15T18:27:22.743 INFO:tasks.workunit.client.1.smithi191.stdout:3/936: creat d6/f143 x:0 0 0 2024-02-15T18:27:22.744 INFO:tasks.workunit.client.1.smithi191.stdout:3/937: write d6/d5d/d72/f103 [131441,109965] 0 2024-02-15T18:27:22.849 INFO:tasks.workunit.client.1.smithi191.stdout:3/938: mknod d6/d3b/d30/c144 0 2024-02-15T18:27:22.851 INFO:tasks.workunit.client.1.smithi191.stdout:3/939: creat d6/d1b/d60/da2/dac/de2/d113/f145 x:0 0 0 2024-02-15T18:27:22.872 INFO:tasks.workunit.client.1.smithi191.stdout:3/940: dwrite d6/d27/d4d/d5a/da4/fc0 [0,4194304] 0 2024-02-15T18:27:22.894 INFO:tasks.workunit.client.1.smithi191.stdout:3/941: dwrite d6/d1b/d60/ddc/de5/fe7 [0,4194304] 0 2024-02-15T18:27:22.895 INFO:tasks.workunit.client.1.smithi191.stdout:3/942: write d6/d27/d4d/d5a/d92/dbe/fd3 [694884,110338] 0 2024-02-15T18:27:22.921 INFO:tasks.workunit.client.1.smithi191.stdout:3/943: dwrite d6/d27/f32 [0,4194304] 0 2024-02-15T18:27:22.923 INFO:tasks.workunit.client.1.smithi191.stdout:3/944: link d6/d1b/d60/d73/d5e/f10a d6/d1b/d60/da2/dac/de2/d113/f146 0 2024-02-15T18:27:22.925 INFO:tasks.workunit.client.1.smithi191.stdout:3/945: symlink d6/d3b/d42/ddd/df6/l147 0 2024-02-15T18:27:22.962 INFO:tasks.workunit.client.1.smithi191.stdout:3/946: dwrite d6/d3b/d42/d49/d54/fa6 [0,4194304] 0 2024-02-15T18:27:22.962 INFO:tasks.workunit.client.1.smithi191.stdout:3/947: dread - d6/f143 zero size 2024-02-15T18:27:22.962 INFO:tasks.workunit.client.1.smithi191.stdout:3/948: truncate d6/d3b/d42/d49/dbf/df7/f105 226132 0 2024-02-15T18:27:22.962 INFO:tasks.workunit.client.1.smithi191.stdout:3/949: truncate d6/d3b/d42/f70 432815 0 2024-02-15T18:27:22.969 INFO:tasks.workunit.client.1.smithi191.stdout:3/950: creat d6/d3b/d42/f148 x:0 0 0 2024-02-15T18:27:22.971 INFO:tasks.workunit.client.1.smithi191.stdout:3/951: creat d6/d27/d4d/d5a/d92/dd2/f149 x:0 0 0 2024-02-15T18:27:22.972 INFO:tasks.workunit.client.1.smithi191.stdout:5/759: sync 2024-02-15T18:27:22.972 INFO:tasks.workunit.client.1.smithi191.stdout:3/952: link d6/d3b/d42/ddd/f124 d6/d5d/d72/dd5/f14a 0 2024-02-15T18:27:22.974 INFO:tasks.workunit.client.1.smithi191.stdout:5/760: creat d0/d18/d5c/dea/ff7 x:0 0 0 2024-02-15T18:27:22.975 INFO:tasks.workunit.client.1.smithi191.stdout:3/953: symlink d6/d3b/d42/d49/l14b 0 2024-02-15T18:27:22.976 INFO:tasks.workunit.client.1.smithi191.stdout:5/761: creat d0/d18/ff8 x:0 0 0 2024-02-15T18:27:22.977 INFO:tasks.workunit.client.1.smithi191.stdout:5/762: write d0/d23/d41/d28/d31/d5a/d61/f82 [2285516,78955] 0 2024-02-15T18:27:22.977 INFO:tasks.workunit.client.1.smithi191.stdout:3/954: write d6/d1b/f39 [1968855,62105] 0 2024-02-15T18:27:22.977 INFO:tasks.workunit.client.1.smithi191.stdout:5/763: dread d0/d18/d5c/d7c/d49/d9f/dc0/d85/fbe [0,4194304] 0 2024-02-15T18:27:22.977 INFO:tasks.workunit.client.1.smithi191.stdout:5/764: readlink d0/d23/d41/d28/d31/d5a/d61/l7e 0 2024-02-15T18:27:22.983 INFO:tasks.workunit.client.1.smithi191.stdout:5/765: fdatasync d0/d23/d41/d3c/fd7 0 2024-02-15T18:27:22.983 INFO:tasks.workunit.client.1.smithi191.stdout:5/766: readlink d0/d18/d5c/d7c/d49/d9f/dc0/d85/l59 0 2024-02-15T18:27:22.986 INFO:tasks.workunit.client.1.smithi191.stdout:3/955: truncate d6/d1b/d60/d73/f41 2509182 0 2024-02-15T18:27:22.988 INFO:tasks.workunit.client.1.smithi191.stdout:3/956: read d6/d5d/d72/f103 [2038,98437] 0 2024-02-15T18:27:22.989 INFO:tasks.workunit.client.1.smithi191.stdout:5/767: mknod d0/d23/cf9 0 2024-02-15T18:27:22.989 INFO:tasks.workunit.client.1.smithi191.stdout:5/768: fsync d0/d18/d5c/fd2 0 2024-02-15T18:27:22.990 INFO:tasks.workunit.client.1.smithi191.stdout:3/957: symlink d6/d3b/d30/l14c 0 2024-02-15T18:27:23.001 INFO:tasks.workunit.client.1.smithi191.stdout:5/769: dread d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/fb9 [0,4194304] 0 2024-02-15T18:27:23.005 INFO:tasks.workunit.client.1.smithi191.stdout:5/770: rename d0/d23/d41/d28/d31/fb7 to d0/d23/d41/d28/d31/d5a/d61/dda/ffa 0 2024-02-15T18:27:23.005 INFO:tasks.workunit.client.1.smithi191.stdout:5/771: stat d0/fa7 0 2024-02-15T18:27:23.011 INFO:tasks.workunit.client.1.smithi191.stdout:5/772: symlink d0/d18/d5c/d7c/d49/d9f/dc0/d85/lfb 0 2024-02-15T18:27:23.011 INFO:tasks.workunit.client.1.smithi191.stdout:5/773: fdatasync d0/d18/d5c/d7c/d49/d9f/dc0/d85/fbe 0 2024-02-15T18:27:23.011 INFO:tasks.workunit.client.1.smithi191.stdout:5/774: chown d0/d23/d41/d28/d31/d5a/d61/c8f 0 1 2024-02-15T18:27:23.011 INFO:tasks.workunit.client.1.smithi191.stdout:3/958: dwrite d6/fa [0,4194304] 0 2024-02-15T18:27:23.015 INFO:tasks.workunit.client.1.smithi191.stdout:3/959: write d6/d3b/d42/d49/f59 [2375351,120802] 0 2024-02-15T18:27:23.022 INFO:tasks.workunit.client.1.smithi191.stdout:3/960: mkdir d6/d27/d14d 0 2024-02-15T18:27:23.025 INFO:tasks.workunit.client.1.smithi191.stdout:3/961: write d6/d1b/d60/f68 [3287202,130619] 0 2024-02-15T18:27:23.032 INFO:tasks.workunit.client.1.smithi191.stdout:5/775: rmdir d0/d23/d41/d28/d31/d5a 39 2024-02-15T18:27:23.039 INFO:tasks.workunit.client.1.smithi191.stdout:5/776: mkdir d0/d18/d5c/d7c/d49/da9/dfc 0 2024-02-15T18:27:23.043 INFO:tasks.workunit.client.1.smithi191.stdout:3/962: dread d6/d1b/d60/da2/dac/de2/d113/f11f [0,4194304] 0 2024-02-15T18:27:23.045 INFO:tasks.workunit.client.1.smithi191.stdout:0/892: sync 2024-02-15T18:27:23.046 INFO:tasks.workunit.client.1.smithi191.stdout:5/777: creat d0/d18/d5c/d7c/d49/da9/dfc/ffd x:0 0 0 2024-02-15T18:27:23.048 INFO:tasks.workunit.client.1.smithi191.stdout:5/778: write d0/d18/fbb [141534,76324] 0 2024-02-15T18:27:23.048 INFO:tasks.workunit.client.1.smithi191.stdout:0/893: rename d7/d3b/d11d to d7/d4a/dc0/d132/d134 0 2024-02-15T18:27:23.050 INFO:tasks.workunit.client.1.smithi191.stdout:5/779: truncate d0/d23/d41/d28/d31/d5a/f68 1598470 0 2024-02-15T18:27:23.051 INFO:tasks.workunit.client.1.smithi191.stdout:5/780: creat d0/d18/d5c/d7c/d49/da9/dfc/ffe x:0 0 0 2024-02-15T18:27:23.053 INFO:tasks.workunit.client.1.smithi191.stdout:5/781: creat d0/d18/d5c/d7c/d49/fff x:0 0 0 2024-02-15T18:27:23.054 INFO:tasks.workunit.client.1.smithi191.stdout:0/894: write d7/da/d6c/fbf [4053382,20013] 0 2024-02-15T18:27:23.075 INFO:tasks.workunit.client.1.smithi191.stdout:5/782: dwrite d0/d23/f27 [0,4194304] 0 2024-02-15T18:27:23.076 INFO:tasks.workunit.client.1.smithi191.stdout:0/895: dwrite d7/da/d17/d31/d5f/d9e/dc3/dd8/d12c/f6f [0,4194304] 0 2024-02-15T18:27:23.077 INFO:tasks.workunit.client.1.smithi191.stdout:0/896: dread - d7/f102 zero size 2024-02-15T18:27:23.080 INFO:tasks.workunit.client.1.smithi191.stdout:5/783: mknod d0/d18/d5c/d73/c100 0 2024-02-15T18:27:23.088 INFO:tasks.workunit.client.1.smithi191.stdout:5/784: dread d0/d18/d5c/fd2 [4194304,4194304] 0 2024-02-15T18:27:23.089 INFO:tasks.workunit.client.1.smithi191.stdout:0/897: link d7/da/d1a/d2c/d4c/d113/l121 d7/d4a/dc0/d132/d134/l135 0 2024-02-15T18:27:23.089 INFO:tasks.workunit.client.1.smithi191.stdout:5/785: write d0/d23/f27 [894531,18930] 0 2024-02-15T18:27:23.092 INFO:tasks.workunit.client.1.smithi191.stdout:0/898: symlink d7/da/d17/d31/d5f/dae/de2/dec/l136 0 2024-02-15T18:27:23.093 INFO:tasks.workunit.client.1.smithi191.stdout:0/899: write d7/d3b/f9a [2838963,39527] 0 2024-02-15T18:27:23.094 INFO:tasks.workunit.client.1.smithi191.stdout:5/786: creat d0/d18/d5c/f101 x:0 0 0 2024-02-15T18:27:23.096 INFO:tasks.workunit.client.1.smithi191.stdout:0/900: symlink d7/d1b/l137 0 2024-02-15T18:27:23.097 INFO:tasks.workunit.client.1.smithi191.stdout:5/787: mkdir d0/d18/d5c/d102 0 2024-02-15T18:27:23.098 INFO:tasks.workunit.client.1.smithi191.stdout:0/901: read d7/d1b/ff8 [1153442,119830] 0 2024-02-15T18:27:23.099 INFO:tasks.workunit.client.1.smithi191.stdout:5/788: symlink d0/d23/d41/d3c/l103 0 2024-02-15T18:27:23.104 INFO:tasks.workunit.client.1.smithi191.stdout:0/902: creat d7/d4a/f138 x:0 0 0 2024-02-15T18:27:23.105 INFO:tasks.workunit.client.1.smithi191.stdout:0/903: symlink d7/da/d1a/d2c/d4c/df3/l139 0 2024-02-15T18:27:23.106 INFO:tasks.workunit.client.1.smithi191.stdout:0/904: symlink d7/da/d17/daa/l13a 0 2024-02-15T18:27:23.119 INFO:tasks.workunit.client.1.smithi191.stdout:0/905: dread d7/da/d1a/d2c/f86 [0,4194304] 0 2024-02-15T18:27:23.119 INFO:tasks.workunit.client.1.smithi191.stdout:5/789: dwrite d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/fdc [0,4194304] 0 2024-02-15T18:27:23.119 INFO:tasks.workunit.client.1.smithi191.stdout:5/790: readlink d0/d18/l9b 0 2024-02-15T18:27:23.121 INFO:tasks.workunit.client.1.smithi191.stdout:0/906: truncate d7/fb 3685986 0 2024-02-15T18:27:23.121 INFO:tasks.workunit.client.1.smithi191.stdout:0/907: stat d7/da/d17/d31/l24 0 2024-02-15T18:27:23.123 INFO:tasks.workunit.client.1.smithi191.stdout:5/791: creat d0/d23/d41/d28/d31/d5a/d61/f104 x:0 0 0 2024-02-15T18:27:23.123 INFO:tasks.workunit.client.1.smithi191.stdout:5/792: fsync d0/d23/d41/f9a 0 2024-02-15T18:27:23.123 INFO:tasks.workunit.client.1.smithi191.stdout:0/908: rmdir d7/da/d17/d31/d5f 39 2024-02-15T18:27:23.125 INFO:tasks.workunit.client.1.smithi191.stdout:5/793: rename d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/ccb to d0/d23/d41/d28/d31/dc2/c105 0 2024-02-15T18:27:23.125 INFO:tasks.workunit.client.1.smithi191.stdout:5/794: read d0/f79 [4216317,90248] 0 2024-02-15T18:27:23.126 INFO:tasks.workunit.client.1.smithi191.stdout:5/795: write d0/f1a [971951,116154] 0 2024-02-15T18:27:23.132 INFO:tasks.workunit.client.1.smithi191.stdout:0/909: mkdir d7/da/d1a/d2c/d4c/d13b 0 2024-02-15T18:27:23.135 INFO:tasks.workunit.client.1.smithi191.stdout:0/910: creat d7/d3b/f13c x:0 0 0 2024-02-15T18:27:23.137 INFO:tasks.workunit.client.1.smithi191.stdout:0/911: mknod d7/da/d1a/d2c/d4c/df3/c13d 0 2024-02-15T18:27:23.137 INFO:tasks.workunit.client.1.smithi191.stdout:0/912: truncate d7/d4a/dc0/fcd 4659249 0 2024-02-15T18:27:23.152 INFO:tasks.workunit.client.1.smithi191.stdout:5/796: dwrite d0/d18/d5c/f6d [0,4194304] 0 2024-02-15T18:27:23.152 INFO:tasks.workunit.client.1.smithi191.stdout:0/913: dread d7/da/d17/f47 [4194304,4194304] 0 2024-02-15T18:27:23.152 INFO:tasks.workunit.client.1.smithi191.stdout:0/914: readlink d7/da/d1a/d34/d45/le9 0 2024-02-15T18:27:23.152 INFO:tasks.workunit.client.1.smithi191.stdout:5/797: chown d0/d23/d41/d28/d31/d5a/d61/f82 33324278 1 2024-02-15T18:27:23.152 INFO:tasks.workunit.client.1.smithi191.stdout:0/915: fdatasync d7/d1b/ff8 0 2024-02-15T18:27:23.152 INFO:tasks.workunit.client.1.smithi191.stdout:5/798: write d0/d62/fd3 [1013508,60635] 0 2024-02-15T18:27:23.173 INFO:tasks.workunit.client.1.smithi191.stdout:5/799: dwrite d0/d18/d5c/d7c/d49/d9f/fdb [0,4194304] 0 2024-02-15T18:27:23.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:22 smithi191 bash[59510]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191 2024-02-15T18:27:23.237 INFO:tasks.workunit.client.1.smithi191.stdout:0/916: mknod d7/da/d17/d31/d5f/dae/de2/dec/c13e 0 2024-02-15T18:27:23.255 INFO:tasks.workunit.client.1.smithi191.stdout:0/917: dwrite d7/d4a/f7b [0,4194304] 0 2024-02-15T18:27:23.255 INFO:tasks.workunit.client.1.smithi191.stdout:0/918: chown d7/da/d1a/d34/c3d 32190407 1 2024-02-15T18:27:23.328 INFO:tasks.workunit.client.1.smithi191.stdout:0/919: creat d7/da/d1a/d2c/f13f x:0 0 0 2024-02-15T18:27:23.328 INFO:tasks.workunit.client.1.smithi191.stdout:0/920: truncate d7/da/d17/d31/f106 521128 0 2024-02-15T18:27:23.329 INFO:tasks.workunit.client.1.smithi191.stdout:5/800: rmdir d0/d23 39 2024-02-15T18:27:23.329 INFO:tasks.workunit.client.1.smithi191.stdout:5/801: chown d0/d18/d5c/d7c/d49/d9f/dc0/d85/lfb 4 1 2024-02-15T18:27:23.334 INFO:tasks.workunit.client.1.smithi191.stdout:0/921: rename d7/da/d17/d31/d5f/d9e/dc3/dd8/d12c/f7c to d7/d4a/dc0/d132/f140 0 2024-02-15T18:27:23.334 INFO:tasks.workunit.client.1.smithi191.stdout:5/802: creat d0/d18/d5c/d7c/d49/da9/dfc/f106 x:0 0 0 2024-02-15T18:27:23.334 INFO:tasks.workunit.client.1.smithi191.stdout:5/803: write d0/d18/d5c/fb3 [126023,105488] 0 2024-02-15T18:27:23.334 INFO:tasks.workunit.client.1.smithi191.stdout:5/804: chown d0/d18/d5c/d7c/f96 14 1 2024-02-15T18:27:23.336 INFO:tasks.workunit.client.1.smithi191.stdout:5/805: mkdir d0/d18/d5c/d7c/d49/da9/d107 0 2024-02-15T18:27:23.338 INFO:tasks.workunit.client.1.smithi191.stdout:5/806: rename d0/d23/d41/d3c/f92 to d0/d18/d5c/d73/d7f/f108 0 2024-02-15T18:27:23.341 INFO:tasks.workunit.client.1.smithi191.stdout:5/807: chown d0/d23/d41/d3c/c93 1409724 1 2024-02-15T18:27:23.342 INFO:tasks.workunit.client.1.smithi191.stdout:5/808: getdents d0/d23/d41/d28/d31/dc2 0 2024-02-15T18:27:23.342 INFO:tasks.workunit.client.1.smithi191.stdout:5/809: chown d0/d18/d5c/d7c/d49/laa 17433 1 2024-02-15T18:27:23.342 INFO:tasks.workunit.client.1.smithi191.stdout:5/810: truncate d0/d62/fac 520876 0 2024-02-15T18:27:23.344 INFO:tasks.workunit.client.1.smithi191.stdout:5/811: truncate d0/d18/d5c/fd2 7234196 0 2024-02-15T18:27:23.347 INFO:tasks.workunit.client.1.smithi191.stdout:5/812: write d0/d23/d41/d28/d31/f53 [5047804,21382] 0 2024-02-15T18:27:23.352 INFO:tasks.workunit.client.1.smithi191.stdout:5/813: mknod d0/d18/d5c/d7c/d49/da9/dfc/c109 0 2024-02-15T18:27:23.352 INFO:tasks.workunit.client.1.smithi191.stdout:5/814: chown d0/d23/d41/d28/d31/d5a/fa6 9 1 2024-02-15T18:27:23.354 INFO:tasks.workunit.client.1.smithi191.stdout:5/815: truncate d0/d23/d41/d28/d31/d5a/fa0 966466 0 2024-02-15T18:27:23.354 INFO:tasks.workunit.client.1.smithi191.stdout:5/816: write d0/d18/fbb [1781963,3660] 0 2024-02-15T18:27:23.355 INFO:tasks.workunit.client.1.smithi191.stdout:5/817: write d0/d18/d5c/d7c/d49/d9f/dc0/d85/f81 [1543758,31079] 0 2024-02-15T18:27:23.356 INFO:tasks.workunit.client.1.smithi191.stdout:5/818: write d0/d18/d5c/d7c/d49/d9f/fdb [1825706,108447] 0 2024-02-15T18:27:23.357 INFO:tasks.workunit.client.1.smithi191.stdout:5/819: write d0/d23/d41/d28/fec [878301,59849] 0 2024-02-15T18:27:23.358 INFO:tasks.workunit.client.1.smithi191.stdout:0/922: dwrite d7/d4a/dc0/d132/d134/f130 [0,4194304] 0 2024-02-15T18:27:23.364 INFO:tasks.workunit.client.1.smithi191.stdout:5/820: fsync d0/d18/fbb 0 2024-02-15T18:27:23.380 INFO:tasks.workunit.client.1.smithi191.stdout:5/821: dwrite d0/d23/d41/fe7 [0,4194304] 0 2024-02-15T18:27:23.380 INFO:tasks.workunit.client.1.smithi191.stdout:5/822: readlink d0/d18/d5c/d7c/d49/d9f/dc0/d85/l97 0 2024-02-15T18:27:23.384 INFO:tasks.workunit.client.1.smithi191.stdout:5/823: symlink d0/d23/d41/d28/l10a 0 2024-02-15T18:27:23.384 INFO:tasks.workunit.client.1.smithi191.stdout:5/824: write d0/d62/fac [912019,102182] 0 2024-02-15T18:27:23.388 INFO:tasks.workunit.client.1.smithi191.stdout:5/825: dread - d0/d23/d41/d3c/fc6 zero size 2024-02-15T18:27:23.389 INFO:tasks.workunit.client.1.smithi191.stdout:5/826: getdents d0/db5 0 2024-02-15T18:27:23.392 INFO:tasks.workunit.client.1.smithi191.stdout:5/827: mknod d0/d23/d41/d28/d31/c10b 0 2024-02-15T18:27:23.392 INFO:tasks.workunit.client.1.smithi191.stdout:5/828: write d0/d62/fe8 [860747,61675] 0 2024-02-15T18:27:23.392 INFO:tasks.workunit.client.1.smithi191.stdout:5/829: fdatasync d0/d18/d5c/d7c/f58 0 2024-02-15T18:27:23.393 INFO:tasks.workunit.client.1.smithi191.stdout:5/830: creat d0/f10c x:0 0 0 2024-02-15T18:27:23.412 INFO:tasks.workunit.client.1.smithi191.stdout:5/831: dwrite d0/d18/d5c/f63 [0,4194304] 0 2024-02-15T18:27:23.434 INFO:tasks.workunit.client.1.smithi191.stdout:5/832: dwrite d0/d23/d41/d28/d31/dc2/fef [0,4194304] 0 2024-02-15T18:27:23.434 INFO:tasks.workunit.client.1.smithi191.stdout:5/833: chown d0/d18/d5c/d7c/d49/da9/dfc/ffd 233211 1 2024-02-15T18:27:23.440 INFO:tasks.workunit.client.1.smithi191.stdout:5/834: rename d0/d18/d5c/d7c/d49/da9/dfc/ffe to d0/d23/d41/f10d 0 2024-02-15T18:27:23.459 INFO:tasks.workunit.client.1.smithi191.stdout:5/835: dwrite d0/f1b [0,4194304] 0 2024-02-15T18:27:23.459 INFO:tasks.workunit.client.1.smithi191.stdout:5/836: fsync d0/d18/d5c/d7c/f96 0 2024-02-15T18:27:23.459 INFO:tasks.workunit.client.1.smithi191.stdout:5/837: write d0/d23/d41/d28/d31/d5a/d61/f104 [220333,34084] 0 2024-02-15T18:27:23.460 INFO:tasks.workunit.client.1.smithi191.stdout:5/838: creat d0/d23/f10e x:0 0 0 2024-02-15T18:27:23.460 INFO:tasks.workunit.client.1.smithi191.stdout:5/839: chown d0/d18/d5c/d7c/cd1 120 1 2024-02-15T18:27:23.460 INFO:tasks.workunit.client.1.smithi191.stdout:5/840: readlink d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/l50 0 2024-02-15T18:27:23.460 INFO:tasks.workunit.client.1.smithi191.stdout:5/841: mknod d0/db5/c10f 0 2024-02-15T18:27:23.466 INFO:tasks.workunit.client.1.smithi191.stdout:5/842: creat d0/d23/d41/d28/d31/d5a/f110 x:0 0 0 2024-02-15T18:27:23.467 INFO:tasks.workunit.client.1.smithi191.stdout:5/843: mknod d0/d18/d5c/c111 0 2024-02-15T18:27:23.910 INFO:tasks.workunit.client.1.smithi191.stdout:0/923: sync 2024-02-15T18:27:23.910 INFO:tasks.workunit.client.1.smithi191.stdout:3/963: sync 2024-02-15T18:27:23.910 INFO:tasks.workunit.client.1.smithi191.stdout:3/964: write d6/d5d/d72/dd5/fe9 [931441,22923] 0 2024-02-15T18:27:23.912 INFO:tasks.workunit.client.1.smithi191.stdout:0/924: creat d7/da/d1a/d2c/d4c/d13b/f141 x:0 0 0 2024-02-15T18:27:23.914 INFO:tasks.workunit.client.1.smithi191.stdout:3/965: truncate d6/d27/d4d/d5a/d92/dd2/f10b 3926985 0 2024-02-15T18:27:23.916 INFO:tasks.workunit.client.1.smithi191.stdout:3/966: dread d6/d5d/d72/dd5/db2/fdf [0,4194304] 0 2024-02-15T18:27:23.917 INFO:tasks.workunit.client.1.smithi191.stdout:0/925: rename d7/da/d1a/d34/ffc to d7/da/d17/d31/d5f/f142 0 2024-02-15T18:27:23.917 INFO:tasks.workunit.client.1.smithi191.stdout:0/926: write d7/da/d17/d31/d5f/d9e/dc3/dd8/d105/d127/f10d [1022012,104502] 0 2024-02-15T18:27:23.924 INFO:tasks.workunit.client.1.smithi191.stdout:3/967: rename d6/c18 to d6/d1b/d60/ddc/de5/c14e 0 2024-02-15T18:27:23.924 INFO:tasks.workunit.client.1.smithi191.stdout:3/968: write d6/f13c [67911,23356] 0 2024-02-15T18:27:23.925 INFO:tasks.workunit.client.1.smithi191.stdout:3/969: write d6/d27/d4d/da8/fb1 [1043587,55118] 0 2024-02-15T18:27:23.927 INFO:tasks.workunit.client.1.smithi191.stdout:3/970: write d6/d27/d4d/d5a/da4/d13a/f136 [3439792,117674] 0 2024-02-15T18:27:23.927 INFO:tasks.workunit.client.1.smithi191.stdout:3/971: dread - d6/d27/d4d/d5a/fd1 zero size 2024-02-15T18:27:23.928 INFO:tasks.workunit.client.1.smithi191.stdout:3/972: mknod d6/d27/d4d/da8/d106/c14f 0 2024-02-15T18:27:23.930 INFO:tasks.workunit.client.1.smithi191.stdout:3/973: rename d6/d1b/d60/l87 to d6/d27/d14d/l150 0 2024-02-15T18:27:23.936 INFO:tasks.workunit.client.1.smithi191.stdout:0/927: dwrite d7/d4a/dc0/fcd [0,4194304] 0 2024-02-15T18:27:23.936 INFO:tasks.workunit.client.1.smithi191.stdout:0/928: chown d7/d4a/cb7 3917 1 2024-02-15T18:27:23.937 INFO:tasks.workunit.client.1.smithi191.stdout:0/929: symlink d7/da/d1a/d2c/d4c/d13b/l143 0 2024-02-15T18:27:23.937 INFO:tasks.workunit.client.1.smithi191.stdout:0/930: fsync d7/da/f16 0 2024-02-15T18:27:23.937 INFO:tasks.workunit.client.1.smithi191.stdout:0/931: stat d7/da/d17/d31/d7d/df9/l12b 0 2024-02-15T18:27:23.938 INFO:tasks.workunit.client.1.smithi191.stdout:0/932: rename d7/da/d1a/d34/cc2 to d7/d3b/dfd/c144 0 2024-02-15T18:27:23.942 INFO:tasks.workunit.client.1.smithi191.stdout:0/933: dread d7/da/d1a/d34/f11 [0,4194304] 0 2024-02-15T18:27:23.946 INFO:tasks.workunit.client.1.smithi191.stdout:3/974: dread d6/d1b/d60/f68 [0,4194304] 0 2024-02-15T18:27:23.955 INFO:tasks.workunit.client.1.smithi191.stdout:3/975: dread d6/d1b/d60/d73/d5e/fc5 [0,4194304] 0 2024-02-15T18:27:23.955 INFO:tasks.workunit.client.1.smithi191.stdout:3/976: read - d6/d27/d4d/fb6 zero size 2024-02-15T18:27:23.955 INFO:tasks.workunit.client.1.smithi191.stdout:3/977: chown c1 88587 1 2024-02-15T18:27:23.959 INFO:tasks.workunit.client.1.smithi191.stdout:3/978: creat d6/d1b/d60/d73/d5e/f151 x:0 0 0 2024-02-15T18:27:23.961 INFO:tasks.workunit.client.1.smithi191.stdout:3/979: symlink d6/d1b/d60/da2/dac/l152 0 2024-02-15T18:27:23.978 INFO:tasks.workunit.client.1.smithi191.stdout:3/980: dwrite d6/d27/d4d/d5a/d92/dbe/fd3 [0,4194304] 0 2024-02-15T18:27:23.979 INFO:tasks.workunit.client.1.smithi191.stdout:3/981: stat d6/d27/d4d/d5a/d92/dd2/cfd 0 2024-02-15T18:27:23.979 INFO:tasks.workunit.client.1.smithi191.stdout:3/982: chown d6/d3b/d42/d49/d54/fa6 2934282 1 2024-02-15T18:27:23.986 INFO:tasks.workunit.client.1.smithi191.stdout:3/983: creat d6/d5d/f153 x:0 0 0 2024-02-15T18:27:23.993 INFO:tasks.workunit.client.1.smithi191.stdout:3/984: dread d6/fa0 [0,4194304] 0 2024-02-15T18:27:23.995 INFO:tasks.workunit.client.1.smithi191.stdout:3/985: link d6/fd9 d6/d5d/d72/f154 0 2024-02-15T18:27:23.998 INFO:tasks.workunit.client.1.smithi191.stdout:3/986: link d6/d1b/d60/d73/d5e/l96 d6/d3b/d42/ddd/df6/l155 0 2024-02-15T18:27:23.999 INFO:tasks.workunit.client.1.smithi191.stdout:3/987: creat d6/d1b/d60/da2/f156 x:0 0 0 2024-02-15T18:27:24.001 INFO:tasks.workunit.client.1.smithi191.stdout:3/988: mkdir d6/d27/d157 0 2024-02-15T18:27:24.002 INFO:tasks.workunit.client.1.smithi191.stdout:3/989: stat d6/d1b/d60/d73/c2f 0 2024-02-15T18:27:24.005 INFO:tasks.workunit.client.1.smithi191.stdout:3/990: unlink d6/d1b/d60/da2/lf2 0 2024-02-15T18:27:24.005 INFO:tasks.workunit.client.1.smithi191.stdout:3/991: write d6/f131 [295967,22488] 0 2024-02-15T18:27:24.006 INFO:tasks.workunit.client.1.smithi191.stdout:3/992: write d6/d3b/d42/f51 [6363572,26735] 0 2024-02-15T18:27:24.008 INFO:tasks.workunit.client.1.smithi191.stdout:3/993: mkdir d6/d1b/d60/d73/d158 0 2024-02-15T18:27:24.015 INFO:tasks.workunit.client.1.smithi191.stdout:3/994: dread d6/d27/f3f [0,4194304] 0 2024-02-15T18:27:24.017 INFO:tasks.workunit.client.1.smithi191.stdout:3/995: rename d6/lce to d6/d1b/d60/da2/dac/de2/d113/l159 0 2024-02-15T18:27:24.019 INFO:tasks.workunit.client.1.smithi191.stdout:3/996: truncate d6/d3b/d42/dbb/fd4 1835172 0 2024-02-15T18:27:24.023 INFO:tasks.workunit.client.1.smithi191.stdout:3/997: dread d6/d5d/daa/fb9 [0,4194304] 0 2024-02-15T18:27:24.026 INFO:tasks.workunit.client.1.smithi191.stdout:3/998: dread d6/d1b/f81 [0,4194304] 0 2024-02-15T18:27:24.037 INFO:tasks.workunit.client.1.smithi191.stdout:3/999: symlink d6/d27/l15a 0 2024-02-15T18:27:24.089 INFO:tasks.workunit.client.1.smithi191.stdout:5/844: sync 2024-02-15T18:27:24.093 INFO:tasks.workunit.client.1.smithi191.stdout:5/845: mkdir d0/d18/d5c/d7c/d49/d76/d112 0 2024-02-15T18:27:24.093 INFO:tasks.workunit.client.1.smithi191.stdout:5/846: write d0/d18/d5c/f101 [511421,30056] 0 2024-02-15T18:27:24.109 INFO:tasks.workunit.client.1.smithi191.stdout:5/847: dwrite d0/d23/d41/d3c/fc9 [0,4194304] 0 2024-02-15T18:27:24.110 INFO:tasks.workunit.client.1.smithi191.stdout:5/848: dread - d0/d23/d41/d28/f94 zero size 2024-02-15T18:27:24.116 INFO:tasks.workunit.client.1.smithi191.stdout:5/849: mknod d0/d18/d5c/d7c/d49/d9f/dc0/d85/c113 0 2024-02-15T18:27:24.116 INFO:tasks.workunit.client.1.smithi191.stdout:5/850: truncate d0/d18/d5c/fd8 969625 0 2024-02-15T18:27:24.117 INFO:tasks.workunit.client.1.smithi191.stdout:5/851: creat d0/d18/d5c/d7c/d49/d9f/f114 x:0 0 0 2024-02-15T18:27:24.119 INFO:tasks.workunit.client.1.smithi191.stdout:5/852: getdents d0/d62 0 2024-02-15T18:27:24.120 INFO:tasks.workunit.client.1.smithi191.stdout:5/853: mknod d0/d18/d5c/d7c/d49/df6/c115 0 2024-02-15T18:27:24.121 INFO:tasks.workunit.client.1.smithi191.stdout:5/854: symlink d0/d18/d5c/d7c/d49/da9/l116 0 2024-02-15T18:27:24.137 INFO:tasks.workunit.client.1.smithi191.stdout:5/855: dwrite d0/d18/d5c/fd8 [0,4194304] 0 2024-02-15T18:27:24.140 INFO:tasks.workunit.client.1.smithi191.stdout:5/856: creat d0/d23/d41/d28/d31/d5a/f117 x:0 0 0 2024-02-15T18:27:24.145 INFO:tasks.workunit.client.1.smithi191.stdout:5/857: dread d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/fb9 [0,4194304] 0 2024-02-15T18:27:24.148 INFO:tasks.workunit.client.1.smithi191.stdout:5/858: rmdir d0/d18/d5c/dea 39 2024-02-15T18:27:24.166 INFO:tasks.workunit.client.1.smithi191.stdout:5/859: dwrite d0/d23/d41/d28/d31/d5a/fa6 [0,4194304] 0 2024-02-15T18:27:24.170 INFO:tasks.workunit.client.1.smithi191.stdout:5/860: dread d0/d23/d41/d28/d31/f47 [0,4194304] 0 2024-02-15T18:27:24.170 INFO:tasks.workunit.client.1.smithi191.stdout:5/861: truncate d0/d23/d41/f87 1890501 0 2024-02-15T18:27:24.173 INFO:tasks.workunit.client.1.smithi191.stdout:5/862: dread d0/d62/fe8 [0,4194304] 0 2024-02-15T18:27:24.174 INFO:tasks.workunit.client.1.smithi191.stdout:5/863: write d0/d23/d41/fe7 [2462544,4629] 0 2024-02-15T18:27:24.178 INFO:tasks.workunit.client.1.smithi191.stdout:5/864: mkdir d0/d18/d5c/d118 0 2024-02-15T18:27:24.178 INFO:tasks.workunit.client.1.smithi191.stdout:5/865: truncate d0/d18/d5c/d7c/d49/da9/dfc/f106 24023 0 2024-02-15T18:27:24.178 INFO:tasks.workunit.client.1.smithi191.stdout:5/866: write d0/d23/d41/d28/d31/d5a/f117 [731069,92712] 0 2024-02-15T18:27:24.186 INFO:tasks.workunit.client.1.smithi191.stdout:5/867: dread d0/d23/f43 [0,4194304] 0 2024-02-15T18:27:24.186 INFO:tasks.workunit.client.1.smithi191.stdout:5/868: dread - d0/d23/f10e zero size 2024-02-15T18:27:24.186 INFO:tasks.workunit.client.1.smithi191.stdout:5/869: write d0/d18/d5c/d7c/d49/fff [443274,87030] 0 2024-02-15T18:27:24.187 INFO:tasks.workunit.client.1.smithi191.stdout:5/870: dread d0/d23/d41/f9a [0,4194304] 0 2024-02-15T18:27:24.196 INFO:tasks.workunit.client.1.smithi191.stdout:5/871: dread d0/d18/d5c/f6d [0,4194304] 0 2024-02-15T18:27:24.196 INFO:tasks.workunit.client.1.smithi191.stdout:5/872: write d0/d23/f43 [3672623,24516] 0 2024-02-15T18:27:24.199 INFO:tasks.workunit.client.1.smithi191.stdout:5/873: symlink d0/d23/d41/d28/d31/dc2/l119 0 2024-02-15T18:27:24.203 INFO:tasks.workunit.client.1.smithi191.stdout:5/874: rename d0/d23/d41/d28/d31/d5a/d61/l6f to d0/d23/d41/d28/d31/l11a 0 2024-02-15T18:27:24.220 INFO:tasks.workunit.client.1.smithi191.stdout:5/875: dwrite d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/fb9 [4194304,4194304] 0 2024-02-15T18:27:24.444 INFO:tasks.workunit.client.1.smithi191.stdout:0/934: sync 2024-02-15T18:27:24.444 INFO:tasks.workunit.client.1.smithi191.stdout:0/935: chown d7/d1b/l75 10389 1 2024-02-15T18:27:24.445 INFO:tasks.workunit.client.1.smithi191.stdout:0/936: getdents d7/da/d17/d31/d5f 0 2024-02-15T18:27:24.459 INFO:tasks.workunit.client.1.smithi191.stdout:0/937: mknod d7/da/d17/d31/d5f/dae/dd3/c145 0 2024-02-15T18:27:24.459 INFO:tasks.workunit.client.1.smithi191.stdout:0/938: chown d7/da/d1a/d34/f29 24023056 1 2024-02-15T18:27:24.482 INFO:tasks.workunit.client.1.smithi191.stdout:5/876: sync 2024-02-15T18:27:24.482 INFO:tasks.workunit.client.1.smithi191.stdout:5/877: stat d0/d18/fbb 0 2024-02-15T18:27:24.482 INFO:tasks.workunit.client.1.smithi191.stdout:0/939: dwrite d7/da/d1a/d2c/f4f [0,4194304] 0 2024-02-15T18:27:24.482 INFO:tasks.workunit.client.1.smithi191.stdout:5/878: truncate d0/ff4 121470 0 2024-02-15T18:27:24.484 INFO:tasks.workunit.client.1.smithi191.stdout:0/940: write d7/da/d17/d31/d5f/d9e/dc3/dd8/d12c/f6f [298757,82872] 0 2024-02-15T18:27:24.484 INFO:tasks.workunit.client.1.smithi191.stdout:0/941: write d7/d4a/dc0/d132/d134/f128 [943402,71792] 0 2024-02-15T18:27:24.485 INFO:tasks.workunit.client.1.smithi191.stdout:5/879: mkdir d0/d23/d41/d28/d31/d5a/d61/de5/d11b 0 2024-02-15T18:27:24.485 INFO:tasks.workunit.client.1.smithi191.stdout:5/880: write d0/d23/d41/d28/f36 [1435179,998] 0 2024-02-15T18:27:24.485 INFO:tasks.workunit.client.1.smithi191.stdout:0/942: write d7/d94/f112 [3463518,47019] 0 2024-02-15T18:27:24.487 INFO:tasks.workunit.client.1.smithi191.stdout:5/881: creat d0/d23/d41/d28/dc1/f11c x:0 0 0 2024-02-15T18:27:24.487 INFO:tasks.workunit.client.1.smithi191.stdout:5/882: truncate d0/d23/d41/d28/d31/f47 4607107 0 2024-02-15T18:27:24.489 INFO:tasks.workunit.client.1.smithi191.stdout:0/943: mkdir d7/da/d1a/d2c/d4c/d98/d146 0 2024-02-15T18:27:24.493 INFO:tasks.workunit.client.1.smithi191.stdout:5/883: stat d0/d23/d41/l34 0 2024-02-15T18:27:24.495 INFO:tasks.workunit.client.1.smithi191.stdout:0/944: unlink d7/da/d1a/d2c/d4c/dd1/f101 0 2024-02-15T18:27:24.495 INFO:tasks.workunit.client.1.smithi191.stdout:0/945: truncate d7/da/d17/d31/d7d/fcf 888862 0 2024-02-15T18:27:24.495 INFO:tasks.workunit.client.1.smithi191.stdout:0/946: fdatasync d7/d94/ffb 0 2024-02-15T18:27:24.496 INFO:tasks.workunit.client.1.smithi191.stdout:5/884: dread d0/d18/fbb [0,4194304] 0 2024-02-15T18:27:24.497 INFO:tasks.workunit.client.1.smithi191.stdout:5/885: dread d0/d23/d41/d28/d31/fb8 [0,4194304] 0 2024-02-15T18:27:24.500 INFO:tasks.workunit.client.1.smithi191.stdout:0/947: rename d7/d94/ce5 to d7/d4a/dc0/d132/c147 0 2024-02-15T18:27:24.500 INFO:tasks.workunit.client.1.smithi191.stdout:0/948: dread - d7/da/d1a/d2c/f10b zero size 2024-02-15T18:27:24.500 INFO:tasks.workunit.client.1.smithi191.stdout:0/949: dread - d7/d94/ffb zero size 2024-02-15T18:27:24.520 INFO:tasks.workunit.client.1.smithi191.stdout:5/886: dwrite d0/d18/fbb [0,4194304] 0 2024-02-15T18:27:24.522 INFO:tasks.workunit.client.1.smithi191.stdout:5/887: mknod d0/d18/d5c/d7c/d49/d76/c11d 0 2024-02-15T18:27:24.524 INFO:tasks.workunit.client.1.smithi191.stdout:5/888: creat d0/d18/d5c/d73/f11e x:0 0 0 2024-02-15T18:27:24.525 INFO:tasks.workunit.client.1.smithi191.stdout:0/950: dwrite d7/da/fce [0,4194304] 0 2024-02-15T18:27:24.529 INFO:tasks.workunit.client.1.smithi191.stdout:5/889: dread d0/d23/d41/d28/d31/d5a/d61/f82 [0,4194304] 0 2024-02-15T18:27:24.535 INFO:tasks.workunit.client.1.smithi191.stdout:0/951: truncate d7/d3b/fb2 2211642 0 2024-02-15T18:27:24.537 INFO:tasks.workunit.client.1.smithi191.stdout:5/890: creat d0/db5/f11f x:0 0 0 2024-02-15T18:27:24.542 INFO:tasks.workunit.client.1.smithi191.stdout:0/952: mknod d7/da/d1a/d2c/d4c/de8/c148 0 2024-02-15T18:27:24.545 INFO:tasks.workunit.client.1.smithi191.stdout:0/953: getdents d7/da/d17/d31/d5f/dae 0 2024-02-15T18:27:24.545 INFO:tasks.workunit.client.1.smithi191.stdout:0/954: stat d7/da/d1a/d2c/f32 0 2024-02-15T18:27:24.547 INFO:tasks.workunit.client.1.smithi191.stdout:0/955: rename d7/da/d1a/d2c/d4c/ff6 to d7/da/d17/d31/d5f/dae/dd3/f149 0 2024-02-15T18:27:24.551 INFO:tasks.workunit.client.1.smithi191.stdout:0/956: mkdir d7/da/d17/d31/d5f/d133/d14a 0 2024-02-15T18:27:24.551 INFO:tasks.workunit.client.1.smithi191.stdout:0/957: read - d7/da/d17/d31/d5f/d9e/dc3/f10c zero size 2024-02-15T18:27:24.552 INFO:tasks.workunit.client.1.smithi191.stdout:0/958: read d7/d4a/dc0/d132/d134/f130 [1183845,94009] 0 2024-02-15T18:27:24.552 INFO:tasks.workunit.client.1.smithi191.stdout:0/959: readlink d7/d1b/l21 0 2024-02-15T18:27:24.553 INFO:tasks.workunit.client.1.smithi191.stdout:0/960: truncate d7/da/d1a/fa5 3584784 0 2024-02-15T18:27:24.553 INFO:tasks.workunit.client.1.smithi191.stdout:0/961: chown d7/da/d17/d31/d5f/dae/cb5 1400995 1 2024-02-15T18:27:24.559 INFO:tasks.workunit.client.1.smithi191.stdout:5/891: dwrite d0/d18/d5c/fd2 [0,4194304] 0 2024-02-15T18:27:24.575 INFO:tasks.workunit.client.1.smithi191.stdout:5/892: dwrite d0/d23/d41/fe7 [4194304,4194304] 0 2024-02-15T18:27:24.575 INFO:tasks.workunit.client.1.smithi191.stdout:5/893: truncate d0/d23/f10e 988550 0 2024-02-15T18:27:24.592 INFO:tasks.workunit.client.1.smithi191.stdout:5/894: dwrite d0/d23/d41/d28/f3f [0,4194304] 0 2024-02-15T18:27:24.594 INFO:tasks.workunit.client.1.smithi191.stdout:5/895: mkdir d0/d18/d5c/d7c/d120 0 2024-02-15T18:27:24.595 INFO:tasks.workunit.client.1.smithi191.stdout:5/896: creat d0/d18/d5c/dea/f121 x:0 0 0 2024-02-15T18:27:24.595 INFO:tasks.workunit.client.1.smithi191.stdout:5/897: readlink d0/d23/d41/d28/d31/d5a/d61/de5/lf5 0 2024-02-15T18:27:24.597 INFO:tasks.workunit.client.1.smithi191.stdout:5/898: truncate d0/d23/f27 2767229 0 2024-02-15T18:27:24.599 INFO:tasks.workunit.client.1.smithi191.stdout:5/899: write d0/d18/f42 [4653746,81590] 0 2024-02-15T18:27:24.600 INFO:tasks.workunit.client.1.smithi191.stdout:5/900: symlink d0/d18/d5c/d7c/d49/d76/l122 0 2024-02-15T18:27:24.601 INFO:tasks.workunit.client.1.smithi191.stdout:5/901: write d0/d18/d5c/f71 [2306604,70752] 0 2024-02-15T18:27:24.606 INFO:tasks.workunit.client.1.smithi191.stdout:5/902: mknod d0/d18/d5c/d7c/d49/d9f/dc0/c123 0 2024-02-15T18:27:24.608 INFO:tasks.workunit.client.1.smithi191.stdout:5/903: creat d0/d18/f124 x:0 0 0 2024-02-15T18:27:24.609 INFO:tasks.workunit.client.1.smithi191.stdout:5/904: write d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/fdc [1214351,124109] 0 2024-02-15T18:27:24.609 INFO:tasks.workunit.client.1.smithi191.stdout:5/905: write d0/d23/d41/d28/d31/d5a/d61/dda/ffa [403784,19786] 0 2024-02-15T18:27:24.610 INFO:tasks.workunit.client.1.smithi191.stdout:5/906: mknod d0/d23/d41/d28/d31/d5a/d61/c125 0 2024-02-15T18:27:24.821 INFO:tasks.workunit.client.1.smithi191.stdout:0/962: sync 2024-02-15T18:27:24.821 INFO:tasks.workunit.client.1.smithi191.stdout:0/963: write d7/da/d6c/f93 [973407,6709] 0 2024-02-15T18:27:24.821 INFO:tasks.workunit.client.1.smithi191.stdout:5/907: sync 2024-02-15T18:27:24.852 INFO:tasks.workunit.client.1.smithi191.stdout:0/964: mkdir d7/da/d17/d31/d5f/d9e/dc3/dd8/d105/d127/d14b 0 2024-02-15T18:27:24.852 INFO:tasks.workunit.client.1.smithi191.stdout:5/908: link d0/d18/d5c/d7c/d49/lca d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/l126 0 2024-02-15T18:27:24.865 INFO:tasks.workunit.client.1.smithi191.stdout:0/965: unlink d7/da/d17/l9b 0 2024-02-15T18:27:24.865 INFO:tasks.workunit.client.1.smithi191.stdout:5/909: unlink d0/d18/d5c/d7c/d49/d9f/dc0/d85/l59 0 2024-02-15T18:27:24.866 INFO:tasks.workunit.client.1.smithi191.stdout:5/910: chown d0/d18/d5c/d7c/d49/d76/ce4 529452 1 2024-02-15T18:27:24.866 INFO:tasks.workunit.client.1.smithi191.stdout:5/911: chown d0/d23/d41/d3c/fc6 1472770 1 2024-02-15T18:27:24.874 INFO:tasks.workunit.client.1.smithi191.stdout:5/912: mknod d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/c127 0 2024-02-15T18:27:24.874 INFO:tasks.workunit.client.1.smithi191.stdout:5/913: write d0/d18/fbb [4900097,109424] 0 2024-02-15T18:27:24.902 INFO:tasks.workunit.client.1.smithi191.stdout:0/966: dwrite d7/da/d1a/d2c/f7e [0,4194304] 0 2024-02-15T18:27:24.905 INFO:tasks.workunit.client.1.smithi191.stdout:5/914: dwrite d0/d23/d41/d28/f36 [4194304,4194304] 0 2024-02-15T18:27:24.905 INFO:tasks.workunit.client.1.smithi191.stdout:5/915: chown d0/d18/d5c/d7c/d49/d9f/dc0/d85/l97 180658 1 2024-02-15T18:27:24.906 INFO:tasks.workunit.client.1.smithi191.stdout:5/916: creat d0/d23/f128 x:0 0 0 2024-02-15T18:27:24.906 INFO:tasks.workunit.client.1.smithi191.stdout:5/917: write d0/d18/d5c/d7c/d49/da9/dfc/ffd [742966,67627] 0 2024-02-15T18:27:24.930 INFO:tasks.workunit.client.1.smithi191.stdout:0/967: dwrite d7/d4a/dc0/d132/f140 [0,4194304] 0 2024-02-15T18:27:24.935 INFO:tasks.workunit.client.1.smithi191.stdout:0/968: write d7/da/d17/d31/d5f/fbc [219215,126716] 0 2024-02-15T18:27:24.941 INFO:tasks.workunit.client.1.smithi191.stdout:5/918: dwrite d0/d23/d41/d28/f94 [0,4194304] 0 2024-02-15T18:27:24.941 INFO:tasks.workunit.client.1.smithi191.stdout:5/919: dread - d0/d23/d41/d28/d31/d5a/db2/fe6 zero size 2024-02-15T18:27:24.945 INFO:tasks.workunit.client.1.smithi191.stdout:5/920: creat d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/f129 x:0 0 0 2024-02-15T18:27:24.945 INFO:tasks.workunit.client.1.smithi191.stdout:5/921: readlink d0/d23/d41/d3c/l103 0 2024-02-15T18:27:24.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:24 smithi191 systemd[1]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi191.service: Succeeded. 2024-02-15T18:27:24.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:24 smithi191 systemd[1]: Stopped Ceph mon.smithi191 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:27:24.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:24 smithi191 systemd[1]: Starting Ceph mon.smithi191 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:27:24.948 INFO:tasks.workunit.client.1.smithi191.stdout:5/922: rename d0/db5/c10f to d0/d18/d5c/d7c/d49/da9/dfc/c12a 0 2024-02-15T18:27:24.950 INFO:tasks.workunit.client.1.smithi191.stdout:5/923: creat d0/d18/d5c/d7c/d49/da9/d107/f12b x:0 0 0 2024-02-15T18:27:24.964 INFO:tasks.workunit.client.1.smithi191.stdout:0/969: dwrite d7/da/d17/d31/d5f/dae/dd3/f149 [0,4194304] 0 2024-02-15T18:27:24.964 INFO:tasks.workunit.client.1.smithi191.stdout:0/970: fdatasync d7/da/d1a/fa5 0 2024-02-15T18:27:24.978 INFO:tasks.workunit.client.1.smithi191.stdout:0/971: getdents d7/d1b 0 2024-02-15T18:27:24.978 INFO:tasks.workunit.client.1.smithi191.stdout:0/972: truncate d7/da/d1a/d34/d45/f107 8420508 0 2024-02-15T18:27:24.981 INFO:tasks.workunit.client.1.smithi191.stdout:0/973: write d7/da/d1a/d2c/d4c/f9f [616989,130830] 0 2024-02-15T18:27:24.981 INFO:tasks.workunit.client.1.smithi191.stdout:0/974: fdatasync d7/da/d6c/fbf 0 2024-02-15T18:27:24.981 INFO:tasks.workunit.client.1.smithi191.stdout:5/924: dwrite d0/d23/d41/d3c/fd7 [0,4194304] 0 2024-02-15T18:27:24.990 INFO:tasks.workunit.client.1.smithi191.stdout:5/925: unlink d0/d18/d5c/d7c/d49/lca 0 2024-02-15T18:27:24.992 INFO:tasks.workunit.client.1.smithi191.stdout:5/926: symlink d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/l12c 0 2024-02-15T18:27:25.000 INFO:tasks.workunit.client.1.smithi191.stdout:5/927: getdents d0/d23/d41/d28/d31/d5a 0 2024-02-15T18:27:25.009 INFO:tasks.workunit.client.1.smithi191.stdout:5/928: rename d0/d23/d41/d28/d31/c5f to d0/d18/d5c/d7c/d49/d76/c12d 0 2024-02-15T18:27:25.013 INFO:tasks.workunit.client.1.smithi191.stdout:5/929: symlink d0/d18/d5c/d7c/d49/d9f/dc0/l12e 0 2024-02-15T18:27:25.021 INFO:tasks.workunit.client.1.smithi191.stdout:5/930: symlink d0/d62/l12f 0 2024-02-15T18:27:25.048 INFO:tasks.workunit.client.1.smithi191.stdout:5/931: dwrite d0/f10c [0,4194304] 0 2024-02-15T18:27:25.057 INFO:tasks.workunit.client.1.smithi191.stdout:5/932: getdents d0/d62 0 2024-02-15T18:27:25.069 INFO:tasks.workunit.client.1.smithi191.stdout:5/933: dread d0/d23/d41/d3c/fd7 [0,4194304] 0 2024-02-15T18:27:25.078 INFO:tasks.workunit.client.1.smithi191.stdout:5/934: write d0/d23/f27 [1038855,12491] 0 2024-02-15T18:27:25.085 INFO:tasks.workunit.client.1.smithi191.stdout:5/935: link d0/d18/d5c/d7c/d49/l5e d0/d18/d5c/d7c/d49/da9/l130 0 2024-02-15T18:27:25.089 INFO:tasks.workunit.client.1.smithi191.stdout:5/936: readlink d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/lae 0 2024-02-15T18:27:25.103 INFO:tasks.workunit.client.1.smithi191.stdout:5/937: dread d0/d18/f7a [0,4194304] 0 2024-02-15T18:27:25.115 INFO:tasks.workunit.client.1.smithi191.stdout:5/938: rmdir d0/d23/d41/d28/d31/dc2 39 2024-02-15T18:27:25.121 INFO:tasks.workunit.client.1.smithi191.stdout:5/939: read d0/d23/d41/d3c/fd5 [315627,100234] 0 2024-02-15T18:27:25.121 INFO:tasks.workunit.client.1.smithi191.stdout:5/940: chown d0/d18/d5c/d7c/f9c 3 1 2024-02-15T18:27:25.121 INFO:tasks.workunit.client.1.smithi191.stdout:5/941: read - d0/d18/d5c/d73/d7f/f108 zero size 2024-02-15T18:27:25.121 INFO:tasks.workunit.client.1.smithi191.stdout:5/942: getdents d0/d18/d5c/d118 0 2024-02-15T18:27:25.121 INFO:tasks.workunit.client.1.smithi191.stdout:5/943: write d0/d18/d5c/d7c/f67 [4827181,106310] 0 2024-02-15T18:27:25.121 INFO:tasks.workunit.client.1.smithi191.stdout:5/944: chown d0/d18/d5c/dea 0 1 2024-02-15T18:27:25.131 INFO:tasks.workunit.client.1.smithi191.stdout:5/945: dread d0/d18/d5c/fd2 [0,4194304] 0 2024-02-15T18:27:25.180 INFO:tasks.workunit.client.1.smithi191.stdout:5/946: dread d0/fa7 [0,4194304] 0 2024-02-15T18:27:25.206 INFO:tasks.workunit.client.1.smithi191.stdout:5/947: dwrite d0/d23/d41/d28/d31/d5a/f68 [0,4194304] 0 2024-02-15T18:27:25.220 INFO:tasks.workunit.client.1.smithi191.stdout:5/948: dwrite d0/d23/d41/d28/d31/d5a/fe3 [4194304,4194304] 0 2024-02-15T18:27:25.223 INFO:tasks.workunit.client.1.smithi191.stdout:5/949: chown d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/fdc 5652587 1 2024-02-15T18:27:25.223 INFO:tasks.workunit.client.1.smithi191.stdout:5/950: fsync d0/d23/d41/d28/d31/d5a/db2/fe6 0 2024-02-15T18:27:25.480 INFO:tasks.workunit.client.1.smithi191.stdout:0/975: sync 2024-02-15T18:27:25.480 INFO:tasks.workunit.client.1.smithi191.stdout:0/976: write d7/da/fce [563261,28195] 0 2024-02-15T18:27:25.480 INFO:tasks.workunit.client.1.smithi191.stdout:0/977: chown d7/da/f8e 529144324 1 2024-02-15T18:27:25.486 INFO:tasks.workunit.client.1.smithi191.stdout:0/978: dread d7/d4a/fd4 [0,4194304] 0 2024-02-15T18:27:25.502 INFO:tasks.workunit.client.1.smithi191.stdout:0/979: dwrite d7/d4a/dc0/d132/d134/f128 [0,4194304] 0 2024-02-15T18:27:25.507 INFO:tasks.workunit.client.1.smithi191.stdout:0/980: rename d7/da/d17/d31/d7d/fcf to d7/da/d17/d31/d5f/dae/de2/f14c 0 2024-02-15T18:27:25.512 INFO:tasks.workunit.client.1.smithi191.stdout:0/981: rename d7/da/fce to d7/d4a/dc0/d132/f14d 0 2024-02-15T18:27:25.513 INFO:tasks.workunit.client.1.smithi191.stdout:0/982: write d7/d1b/f46 [3811528,11121] 0 2024-02-15T18:27:25.514 INFO:tasks.workunit.client.1.smithi191.stdout:0/983: mknod d7/da/d17/d31/d5f/d103/c14e 0 2024-02-15T18:27:25.540 INFO:tasks.workunit.client.1.smithi191.stdout:0/984: dwrite d7/da/d17/d31/f106 [0,4194304] 0 2024-02-15T18:27:25.559 INFO:tasks.workunit.client.1.smithi191.stdout:0/985: dwrite d7/da/d1a/f90 [0,4194304] 0 2024-02-15T18:27:25.559 INFO:tasks.workunit.client.1.smithi191.stdout:0/986: read d7/da/d17/d31/f106 [56752,101748] 0 2024-02-15T18:27:25.561 INFO:tasks.workunit.client.1.smithi191.stdout:0/987: getdents d7/da/d17/d31/d5f/d9e/dc3/dd8/d12c 0 2024-02-15T18:27:25.610 INFO:tasks.workunit.client.1.smithi191.stdout:5/951: sync 2024-02-15T18:27:25.612 INFO:tasks.workunit.client.1.smithi191.stdout:5/952: unlink d0/c3 0 2024-02-15T18:27:25.614 INFO:tasks.workunit.client.1.smithi191.stdout:5/953: mkdir d0/d23/d41/d28/d31/dc2/d131 0 2024-02-15T18:27:25.614 INFO:tasks.workunit.client.1.smithi191.stdout:5/954: chown d0/d23/d41/d28/d31/dc2/c105 7707 1 2024-02-15T18:27:25.795 INFO:tasks.workunit.client.1.smithi191.stdout:5/955: unlink d0/d23/d41/d28/d31/dc2/cce 0 2024-02-15T18:27:25.795 INFO:tasks.workunit.client.1.smithi191.stdout:5/956: stat d0/d23/d41/d28/d31/d5a/d61/de5/le9 0 2024-02-15T18:27:25.805 INFO:tasks.workunit.client.1.smithi191.stdout:5/957: dread d0/d23/d41/fe7 [0,4194304] 0 2024-02-15T18:27:25.810 INFO:tasks.workunit.client.1.smithi191.stdout:5/958: unlink d0/d18/f124 0 2024-02-15T18:27:25.819 INFO:tasks.workunit.client.1.smithi191.stdout:5/959: rename d0/d23/d41/d28/d31/c6c to d0/d18/d5c/dea/c132 0 2024-02-15T18:27:25.819 INFO:tasks.workunit.client.1.smithi191.stdout:5/960: read d0/d18/d5c/f63 [2511680,29509] 0 2024-02-15T18:27:25.822 INFO:tasks.workunit.client.1.smithi191.stdout:5/961: rename d0/d23/d41/d28/d31/fb8 to d0/d18/d5c/d102/f133 0 2024-02-15T18:27:25.823 INFO:tasks.workunit.client.1.smithi191.stdout:5/962: mknod d0/d23/d41/d28/d31/dc2/c134 0 2024-02-15T18:27:25.823 INFO:tasks.workunit.client.1.smithi191.stdout:5/963: write d0/d18/d5c/f63 [2527881,48078] 0 2024-02-15T18:27:25.824 INFO:tasks.workunit.client.1.smithi191.stdout:5/964: truncate d0/d23/d41/d28/d31/d5a/d61/fbc 931710 0 2024-02-15T18:27:25.826 INFO:tasks.workunit.client.1.smithi191.stdout:5/965: symlink d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/l135 0 2024-02-15T18:27:25.827 INFO:tasks.workunit.client.1.smithi191.stdout:5/966: mknod d0/d18/d5c/d7c/d49/da9/dfc/c136 0 2024-02-15T18:27:25.830 INFO:tasks.workunit.client.1.smithi191.stdout:5/967: read d0/d18/d5c/d7c/d49/fff [242566,110240] 0 2024-02-15T18:27:25.831 INFO:tasks.workunit.client.1.smithi191.stdout:5/968: getdents d0/d23/d41/d28/d31/d5a/d61/dda 0 2024-02-15T18:27:25.833 INFO:tasks.workunit.client.1.smithi191.stdout:5/969: creat d0/d23/d41/d28/d31/dc2/f137 x:0 0 0 2024-02-15T18:27:25.833 INFO:tasks.workunit.client.1.smithi191.stdout:5/970: chown d0/d18/d5c/d7c/d49/df6/c115 7085 1 2024-02-15T18:27:25.833 INFO:tasks.workunit.client.1.smithi191.stdout:5/971: chown d0/d18/d5c/d102 57 1 2024-02-15T18:27:25.833 INFO:tasks.workunit.client.1.smithi191.stdout:5/972: readlink d0/d18/d5c/d7c/d49/d9f/dc0/d85/lfb 0 2024-02-15T18:27:25.835 INFO:tasks.workunit.client.1.smithi191.stdout:5/973: mknod d0/d23/d41/d28/d31/d5a/d61/de5/c138 0 2024-02-15T18:27:25.980 INFO:tasks.workunit.client.1.smithi191.stdout:5/974: sync 2024-02-15T18:27:25.980 INFO:tasks.workunit.client.1.smithi191.stdout:0/988: sync 2024-02-15T18:27:25.980 INFO:tasks.workunit.client.1.smithi191.stdout:5/975: chown d0/d23/d41/d28/d31/f53 586219694 1 2024-02-15T18:27:25.981 INFO:tasks.workunit.client.1.smithi191.stdout:0/989: link d7/da/d1a/d2c/d4c/df3/l104 d7/da/d17/d31/d5f/d9e/l14f 0 2024-02-15T18:27:25.988 INFO:tasks.workunit.client.1.smithi191.stdout:5/976: link d0/d23/d41/d28/f3f d0/d18/d5c/f139 0 2024-02-15T18:27:25.990 INFO:tasks.workunit.client.1.smithi191.stdout:0/990: truncate d7/da/d1a/d2c/f4f 2092235 0 2024-02-15T18:27:25.992 INFO:tasks.workunit.client.1.smithi191.stdout:5/977: mkdir d0/d23/d41/d28/dc1/d13a 0 2024-02-15T18:27:25.998 INFO:tasks.workunit.client.1.smithi191.stdout:0/991: rename d7/d4a/l7a to d7/da/d1a/d2c/d4c/d13b/l150 0 2024-02-15T18:27:25.999 INFO:tasks.workunit.client.1.smithi191.stdout:0/992: write d7/d4a/dc0/d132/d134/f128 [2781586,15877] 0 2024-02-15T18:27:25.999 INFO:tasks.workunit.client.1.smithi191.stdout:0/993: read d7/da/d17/f47 [3101434,58822] 0 2024-02-15T18:27:26.000 INFO:tasks.workunit.client.1.smithi191.stdout:5/978: mkdir d0/d18/d5c/d7c/d49/d9f/dc0/d85/d13b 0 2024-02-15T18:27:26.001 INFO:tasks.workunit.client.1.smithi191.stdout:0/994: mkdir d7/d4a/dc0/d132/d134/d151 0 2024-02-15T18:27:26.004 INFO:tasks.workunit.client.1.smithi191.stdout:5/979: getdents d0/d18/d5c/d7c/d49 0 2024-02-15T18:27:26.005 INFO:tasks.workunit.client.1.smithi191.stdout:0/995: creat d7/da/d1a/d2c/d4c/d98/d146/f152 x:0 0 0 2024-02-15T18:27:26.009 INFO:tasks.workunit.client.1.smithi191.stdout:5/980: write d0/d62/fac [1094142,97597] 0 2024-02-15T18:27:26.011 INFO:tasks.workunit.client.1.smithi191.stdout:5/981: getdents d0 0 2024-02-15T18:27:26.014 INFO:tasks.workunit.client.1.smithi191.stdout:0/996: mknod d7/d94/c153 0 2024-02-15T18:27:26.015 INFO:tasks.workunit.client.1.smithi191.stdout:0/997: fsync d7/da/d6c/f93 0 2024-02-15T18:27:26.015 INFO:tasks.workunit.client.1.smithi191.stdout:0/998: fdatasync d7/da/d17/d31/d5f/d9e/dc3/f10c 0 2024-02-15T18:27:26.017 INFO:tasks.workunit.client.1.smithi191.stdout:0/999: unlink d7/da/d17/la6 0 2024-02-15T18:27:26.019 INFO:tasks.workunit.client.1.smithi191.stdout:5/982: symlink d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/l13c 0 2024-02-15T18:27:26.023 INFO:tasks.workunit.client.1.smithi191.stdout:5/983: symlink d0/d23/d41/d3c/l13d 0 2024-02-15T18:27:26.026 INFO:tasks.workunit.client.1.smithi191.stdout:5/984: truncate d0/d18/d5c/f63 2736212 0 2024-02-15T18:27:26.026 INFO:tasks.workunit.client.1.smithi191.stdout:5/985: chown d0/d23/d41/d28/d31/f33 403395183 1 2024-02-15T18:27:26.026 INFO:tasks.workunit.client.1.smithi191.stdout:5/986: write d0/d18/d5c/d7c/d49/da9/dfc/f106 [748530,52377] 0 2024-02-15T18:27:26.026 INFO:tasks.workunit.client.1.smithi191.stdout:5/987: fdatasync d0/d23/d41/f10d 0 2024-02-15T18:27:26.028 INFO:tasks.workunit.client.1.smithi191.stdout:5/988: creat d0/d18/d5c/d7c/d49/f13e x:0 0 0 2024-02-15T18:27:26.034 INFO:tasks.workunit.client.1.smithi191.stdout:5/989: chown d0/d18/d5c/d7c/d49/d9f/dc0/d85/d52/f40 29111 1 2024-02-15T18:27:26.052 INFO:tasks.workunit.client.1.smithi191.stdout:5/990: dwrite d0/d23/d41/fe7 [0,4194304] 0 2024-02-15T18:27:26.060 INFO:tasks.workunit.client.1.smithi191.stdout:5/991: dread d0/d62/fac [0,4194304] 0 2024-02-15T18:27:26.062 INFO:tasks.workunit.client.1.smithi191.stdout:5/992: link d0/d18/d5c/d7c/f96 d0/d18/d5c/d7c/d49/d76/d112/f13f 0 2024-02-15T18:27:26.062 INFO:tasks.workunit.client.1.smithi191.stdout:5/993: truncate d0/d18/d5c/d7c/d49/d9f/f114 241091 0 2024-02-15T18:27:26.062 INFO:tasks.workunit.client.1.smithi191.stdout:5/994: read - d0/d18/d5c/dea/f121 zero size 2024-02-15T18:27:26.064 INFO:tasks.workunit.client.1.smithi191.stdout:5/995: mkdir d0/d23/d41/d28/d31/d140 0 2024-02-15T18:27:26.080 INFO:tasks.workunit.client.1.smithi191.stdout:5/996: dwrite d0/d18/d5c/f139 [0,4194304] 0 2024-02-15T18:27:26.082 INFO:tasks.workunit.client.1.smithi191.stdout:5/997: creat d0/d23/d41/d28/d31/d5a/d61/de5/f141 x:0 0 0 2024-02-15T18:27:26.083 INFO:tasks.workunit.client.1.smithi191.stdout:5/998: creat d0/d18/d5c/dea/f142 x:0 0 0 2024-02-15T18:27:26.093 INFO:tasks.workunit.client.1.smithi191.stdout:5/999: fsync d0/d18/d5c/dea/f142 0 2024-02-15T18:27:26.096 INFO:tasks.workunit.client.1.smithi191.stderr:+ rm -rf -- ./tmp.ca64ueUWIM 2024-02-15T18:27:26.696 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.335+0000 7f637a584880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-15T18:27:26.696 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.335+0000 7f637a584880 0 ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable), process ceph-mon, pid 7 2024-02-15T18:27:26.696 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.335+0000 7f637a584880 0 pidfile_write: ignore empty --pid-file 2024-02-15T18:27:26.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 0 load: jerasure load: lrc load: isa 2024-02-15T18:27:26.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: RocksDB version: 6.8.1 2024-02-15T18:27:26.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-15T18:27:26.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Compile date Feb 8 2024 2024-02-15T18:27:26.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: DB SUMMARY 2024-02-15T18:27:26.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: CURRENT file: CURRENT 2024-02-15T18:27:26.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-15T18:27:26.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 689 Bytes 2024-02-15T18:27:26.697 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi191/store.db dir, Total Num: 1, files: 000017.sst 2024-02-15T18:27:26.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi191/store.db: 000015.log size: 1625319 ; 2024-02-15T18:27:26.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.error_if_exists: 0 2024-02-15T18:27:26.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.create_if_missing: 0 2024-02-15T18:27:26.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.paranoid_checks: 1 2024-02-15T18:27:26.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.env: 0x55d553b5e080 2024-02-15T18:27:26.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.fs: Posix File System 2024-02-15T18:27:26.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.info_log: 0x55d55526e560 2024-02-15T18:27:26.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-15T18:27:26.698 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.statistics: (nil) 2024-02-15T18:27:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.use_fsync: 0 2024-02-15T18:27:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.max_log_file_size: 0 2024-02-15T18:27:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-15T18:27:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-15T18:27:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-15T18:27:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-15T18:27:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.allow_fallocate: 1 2024-02-15T18:27:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-15T18:27:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-15T18:27:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.use_direct_reads: 0 2024-02-15T18:27:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-15T18:27:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-15T18:27:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.db_log_dir: 2024-02-15T18:27:26.699 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi191/store.db 2024-02-15T18:27:26.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-15T18:27:26.700 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.max_subcompactions: 1 2024-02-15T18:27:26.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.max_background_flushes: -1 2024-02-15T18:27:26.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-15T18:27:26.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-15T18:27:26.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-15T18:27:26.702 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-15T18:27:26.703 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-15T18:27:26.703 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-15T18:27:26.703 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-15T18:27:26.703 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.write_buffer_manager: 0x55d555f60420 2024-02-15T18:27:26.703 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-15T18:27:26.703 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-15T18:27:26.703 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-15T18:27:26.703 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-15T18:27:26.703 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-15T18:27:26.703 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-15T18:27:26.703 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-15T18:27:26.703 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-15T18:27:26.703 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-15T18:27:26.703 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.unordered_write: 0 2024-02-15T18:27:26.704 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-15T18:27:26.704 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-15T18:27:26.704 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-15T18:27:26.704 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-15T18:27:26.704 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.row_cache: None 2024-02-15T18:27:26.704 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.wal_filter: None 2024-02-15T18:27:26.704 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-15T18:27:26.704 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-15T18:27:26.704 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.preserve_deletes: 0 2024-02-15T18:27:26.704 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.two_write_queues: 0 2024-02-15T18:27:26.704 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-15T18:27:26.704 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.atomic_flush: 0 2024-02-15T18:27:26.704 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-15T18:27:26.705 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-15T18:27:26.705 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-15T18:27:26.705 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.log_readahead_size: 0 2024-02-15T18:27:26.705 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-15T18:27:26.705 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.max_background_jobs: 2 2024-02-15T18:27:26.705 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.max_background_compactions: -1 2024-02-15T18:27:26.705 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-15T18:27:26.705 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-15T18:27:26.705 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-15T18:27:26.705 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-15T18:27:26.705 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-15T18:27:26.705 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-15T18:27:26.706 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-15T18:27:26.706 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-15T18:27:26.706 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.max_open_files: -1 2024-02-15T18:27:26.706 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-15T18:27:26.706 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-15T18:27:26.706 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-15T18:27:26.706 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-15T18:27:26.706 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Compression algorithms supported: 2024-02-15T18:27:26.706 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-15T18:27:26.706 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: kZSTD supported: 0 2024-02-15T18:27:26.706 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: kXpressCompression supported: 0 2024-02-15T18:27:26.706 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-15T18:27:26.706 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: kLZ4Compression supported: 1 2024-02-15T18:27:26.706 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: kBZip2Compression supported: 0 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: kZlibCompression supported: 1 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: kSnappyCompression supported: 1 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.341+0000 7f637a584880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi191/store.db/MANIFEST-000005 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.merge_operator: 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compaction_filter: None 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d555213d28) 2024-02-15T18:27:26.707 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cache_index_and_filter_blocks: 1 2024-02-15T18:27:26.708 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-15T18:27:26.708 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-15T18:27:26.708 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: pin_top_level_index_and_filter: 1 2024-02-15T18:27:26.708 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: index_type: 0 2024-02-15T18:27:26.708 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: data_block_index_type: 0 2024-02-15T18:27:26.708 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: index_shortening: 1 2024-02-15T18:27:26.708 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: data_block_hash_table_util_ratio: 0.750000 2024-02-15T18:27:26.708 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: hash_index_allow_collision: 1 2024-02-15T18:27:26.708 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: checksum: 1 2024-02-15T18:27:26.708 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: no_block_cache: 0 2024-02-15T18:27:26.708 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: block_cache: 0x55d555248f10 2024-02-15T18:27:26.708 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: block_cache_name: BinnedLRUCache 2024-02-15T18:27:26.708 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: block_cache_options: 2024-02-15T18:27:26.709 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: capacity : 536870912 2024-02-15T18:27:26.709 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: num_shard_bits : 4 2024-02-15T18:27:26.709 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: strict_capacity_limit : 0 2024-02-15T18:27:26.709 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: high_pri_pool_ratio: 0.000 2024-02-15T18:27:26.709 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: block_cache_compressed: (nil) 2024-02-15T18:27:26.709 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: persistent_cache: (nil) 2024-02-15T18:27:26.709 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: block_size: 4096 2024-02-15T18:27:26.709 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: block_size_deviation: 10 2024-02-15T18:27:26.709 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: block_restart_interval: 16 2024-02-15T18:27:26.709 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: index_block_restart_interval: 1 2024-02-15T18:27:26.709 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: metadata_block_size: 4096 2024-02-15T18:27:26.709 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: partition_filters: 0 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: use_delta_encoding: 1 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: whole_key_filtering: 1 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: verify_compression: 0 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: read_amp_bytes_per_bit: 0 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: format_version: 2 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: enable_index_compression: 1 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: block_align: 0 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compression: NoCompression 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.num_levels: 7 2024-02-15T18:27:26.710 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-15T18:27:26.711 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-15T18:27:26.711 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-15T18:27:26.711 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-15T18:27:26.711 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-15T18:27:26.711 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-15T18:27:26.711 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-15T18:27:26.711 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:27:26.711 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-15T18:27:26.711 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-15T18:27:26.711 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-15T18:27:26.711 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-15T18:27:26.711 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-15T18:27:26.711 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-15T18:27:26.712 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-15T18:27:26.712 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-15T18:27:26.712 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-15T18:27:26.712 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-15T18:27:26.712 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-15T18:27:26.712 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-15T18:27:26.712 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-15T18:27:26.712 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-15T18:27:26.712 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-15T18:27:26.712 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-15T18:27:26.712 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-15T18:27:26.712 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-15T18:27:26.712 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-15T18:27:26.712 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-15T18:27:26.713 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.table_properties_collectors: 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.inplace_update_support: 0 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.bloom_locality: 0 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.max_successive_merges: 0 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.ttl: 2592000 2024-02-15T18:27:26.714 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: [version_set.cc:3291] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi191/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 19, last_sequence is 9230, log_number is 15,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 15 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708021646343732, "job": 1, "event": "recovery_started", "log_files": [15]} 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.342+0000 7f637a584880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #15 mode 2 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.344+0000 7f637a584880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.346+0000 7f637a584880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708021646347097, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 19, "file_size": 1081473, "table_properties": {"data_size": 1078570, "index_size": 1765, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 325, "raw_key_size": 1995, "raw_average_key_size": 21, "raw_value_size": 1075741, "raw_average_value_size": 11444, "num_data_blocks": 56, "num_entries": 94, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708021646, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.346+0000 7f637a584880 4 rocksdb: [version_set.cc:3826] Creating manifest 20 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.346+0000 7f637a584880 4 rocksdb: [version_set.cc:3291] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.346+0000 7f637a584880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708021646347506, "job": 1, "event": "recovery_finished"} 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.347+0000 7f637a584880 4 rocksdb: DB pointer 0x55d555311800 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.347+0000 7f6361375700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-15T18:27:26.715 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.347+0000 7f6361375700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: ** DB Stats ** 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: ** Compaction Stats [default] ** 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: L0 1/0 1.03 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 515.9 0.00 0.00 1 0.002 0 0 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: L6 1/0 22.59 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Sum 2/0 23.63 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 515.9 0.00 0.00 1 0.002 0 0 2024-02-15T18:27:26.716 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 515.9 0.00 0.00 1 0.002 0 0 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: ** Compaction Stats [default] ** 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 515.9 0.00 0.00 1 0.002 0 0 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Flush(GB): cumulative 0.001, interval 0.001 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Cumulative compaction: 0.00 GB write, 186.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Interval compaction: 0.00 GB write, 186.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.717 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: ** Compaction Stats [default] ** 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: L0 1/0 1.03 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 515.9 0.00 0.00 1 0.002 0 0 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: L6 1/0 22.59 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Sum 2/0 23.63 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 515.9 0.00 0.00 1 0.002 0 0 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: ** Compaction Stats [default] ** 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 515.9 0.00 0.00 1 0.002 0 0 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-15T18:27:26.718 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Flush(GB): cumulative 0.001, interval 0.000 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: AddFile(Total Files): cumulative 0, interval 0 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: AddFile(Keys): cumulative 0, interval 0 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Cumulative compaction: 0.00 GB write, 184.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: ** File Read Latency Histogram By Level [default] ** 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.348+0000 7f637a584880 0 starting mon.smithi191 rank 1 at public addrs [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] at bind addrs [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi191 fsid 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.349+0000 7f637a584880 1 mon.smithi191@-1(???) e2 preinit fsid 51458a44-cc2e-11ee-95ba-87774f69a715 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.349+0000 7f637a584880 0 mon.smithi191@-1(???).mds e13 new map 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.349+0000 7f637a584880 0 mon.smithi191@-1(???).mds e13 print_map 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: e13 2024-02-15T18:27:26.719 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: legacy client fscid: 1 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Filesystem 'cephfs' (1) 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: fs_name cephfs 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: epoch 11 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: flags 32 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: tableserver 0 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: root 0 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: session_timeout 60 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: session_autoclose 300 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: max_file_size 1099511627776 2024-02-15T18:27:26.720 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: required_client_features {} 2024-02-15T18:27:26.721 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: last_failure 0 2024-02-15T18:27:26.721 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: last_failure_osd_epoch 0 2024-02-15T18:27:26.721 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:27:26.721 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: max_mds 1 2024-02-15T18:27:26.721 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: in 0 2024-02-15T18:27:26.721 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: up {0=24263} 2024-02-15T18:27:26.721 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: failed 2024-02-15T18:27:26.721 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: damaged 2024-02-15T18:27:26.721 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: stopped 2024-02-15T18:27:26.721 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: data_pools [3] 2024-02-15T18:27:26.721 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: metadata_pool 2 2024-02-15T18:27:26.721 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: inline_data disabled 2024-02-15T18:27:26.721 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: balancer 2024-02-15T18:27:26.721 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: standby_count_wanted 1 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: [mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: [mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: Standby daemons: 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: [mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: [mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.349+0000 7f637a584880 0 mon.smithi191@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.349+0000 7f637a584880 0 mon.smithi191@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.349+0000 7f637a584880 0 mon.smithi191@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.349+0000 7f637a584880 0 mon.smithi191@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:16.685067+0000 mgr.smithi042.kfoihb (mgr.24435) 53 : cluster [DBG] pgmap v37: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 116 MiB/s rd, 233 MiB/s wr, 392 op/s 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:27:26.349+0000 7f637a584880 1 mon.smithi191@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 bash[59638]: 7abbf59144ba7362c3c473f6e1df48c8351a2dec9e39ebe2a3ebf7988e3261b5 2024-02-15T18:27:26.722 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:26 smithi191 systemd[1]: Started Ceph mon.smithi191 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:27:28.355 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-15T18:27:28.355 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-15T18:27:34.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:18.685614+0000 mgr.smithi042.kfoihb (mgr.24435) 54 : cluster [DBG] pgmap v38: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 149 MiB/s rd, 307 MiB/s wr, 540 op/s 2024-02-15T18:27:34.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:19.430742+0000 mon.smithi042 (mon.0) 15 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-02-15T18:27:34.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:19.432085+0000 mon.smithi042 (mon.0) 16 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:27:34.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:18.685614+0000 mgr.smithi042.kfoihb (mgr.24435) 54 : cluster [DBG] pgmap v38: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 149 MiB/s rd, 307 MiB/s wr, 540 op/s 2024-02-15T18:27:34.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:19.430742+0000 mon.smithi042 (mon.0) 15 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-02-15T18:27:34.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:19.432085+0000 mon.smithi042 (mon.0) 16 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mon metadata", "id": "smithi191"}]: dispatch 2024-02-15T18:27:35.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:20.686077+0000 mgr.smithi042.kfoihb (mgr.24435) 55 : cluster [DBG] pgmap v39: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 93 MiB/s rd, 189 MiB/s wr, 332 op/s 2024-02-15T18:27:35.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:22.686552+0000 mgr.smithi042.kfoihb (mgr.24435) 56 : cluster [DBG] pgmap v40: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 116 MiB/s rd, 247 MiB/s wr, 468 op/s 2024-02-15T18:27:35.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:24.686837+0000 mgr.smithi042.kfoihb (mgr.24435) 57 : cluster [DBG] pgmap v41: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 56 MiB/s rd, 132 MiB/s wr, 284 op/s 2024-02-15T18:27:35.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:26.687186+0000 mgr.smithi042.kfoihb (mgr.24435) 58 : cluster [DBG] pgmap v42: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 56 MiB/s rd, 132 MiB/s wr, 284 op/s 2024-02-15T18:27:35.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:28.687659+0000 mgr.smithi042.kfoihb (mgr.24435) 59 : cluster [DBG] pgmap v43: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 69 MiB/s rd, 155 MiB/s wr, 376 op/s 2024-02-15T18:27:35.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:30.688237+0000 mgr.smithi042.kfoihb (mgr.24435) 60 : cluster [DBG] pgmap v44: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 36 MiB/s rd, 81 MiB/s wr, 228 op/s 2024-02-15T18:27:35.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:32.351504+0000 mon.smithi191 (mon.1) 1 : cluster [INF] mon.smithi191 calling monitor election 2024-02-15T18:27:35.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:32.688845+0000 mgr.smithi042.kfoihb (mgr.24435) 61 : cluster [DBG] pgmap v45: 65 pgs: 65 active+clean; 2.2 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 36 MiB/s rd, 81 MiB/s wr, 275 op/s 2024-02-15T18:27:35.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:33.746256+0000 mon.smithi042 (mon.0) 17 : cluster [INF] mon.smithi042 calling monitor election 2024-02-15T18:27:35.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:33.759959+0000 mon.smithi042 (mon.0) 18 : cluster [INF] mon.smithi042 is new leader, mons smithi042,smithi191 in quorum (ranks 0,1) 2024-02-15T18:27:35.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:33.764719+0000 mon.smithi042 (mon.0) 19 : cluster [DBG] monmap e2: 2 mons at {smithi042=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],smithi191=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} 2024-02-15T18:27:35.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:33.776674+0000 mon.smithi042 (mon.0) 20 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 2 up:standby 2024-02-15T18:27:35.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:33.776707+0000 mon.smithi042 (mon.0) 21 : cluster [DBG] osdmap e48: 6 total, 6 up, 6 in 2024-02-15T18:27:35.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:33.777084+0000 mon.smithi042 (mon.0) 22 : cluster [DBG] mgrmap e30: smithi042.kfoihb(active, since 84s), standbys: smithi191.gftdzo 2024-02-15T18:27:35.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:33.783315+0000 mon.smithi042 (mon.0) 23 : cluster [INF] overall HEALTH_OK 2024-02-15T18:27:35.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:33.786450+0000 mon.smithi042 (mon.0) 24 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:35.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:33.787498+0000 mon.smithi042 (mon.0) 25 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:27:35.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:20.686077+0000 mgr.smithi042.kfoihb (mgr.24435) 55 : cluster [DBG] pgmap v39: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 93 MiB/s rd, 189 MiB/s wr, 332 op/s 2024-02-15T18:27:35.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:22.686552+0000 mgr.smithi042.kfoihb (mgr.24435) 56 : cluster [DBG] pgmap v40: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 116 MiB/s rd, 247 MiB/s wr, 468 op/s 2024-02-15T18:27:35.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:24.686837+0000 mgr.smithi042.kfoihb (mgr.24435) 57 : cluster [DBG] pgmap v41: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 56 MiB/s rd, 132 MiB/s wr, 284 op/s 2024-02-15T18:27:35.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:26.687186+0000 mgr.smithi042.kfoihb (mgr.24435) 58 : cluster [DBG] pgmap v42: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 56 MiB/s rd, 132 MiB/s wr, 284 op/s 2024-02-15T18:27:35.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:28.687659+0000 mgr.smithi042.kfoihb (mgr.24435) 59 : cluster [DBG] pgmap v43: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 69 MiB/s rd, 155 MiB/s wr, 376 op/s 2024-02-15T18:27:35.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:30.688237+0000 mgr.smithi042.kfoihb (mgr.24435) 60 : cluster [DBG] pgmap v44: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 36 MiB/s rd, 81 MiB/s wr, 228 op/s 2024-02-15T18:27:35.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:32.351504+0000 mon.smithi191 (mon.1) 1 : cluster [INF] mon.smithi191 calling monitor election 2024-02-15T18:27:35.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:32.688845+0000 mgr.smithi042.kfoihb (mgr.24435) 61 : cluster [DBG] pgmap v45: 65 pgs: 65 active+clean; 2.2 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 36 MiB/s rd, 81 MiB/s wr, 275 op/s 2024-02-15T18:27:35.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:33.746256+0000 mon.smithi042 (mon.0) 17 : cluster [INF] mon.smithi042 calling monitor election 2024-02-15T18:27:35.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:33.759959+0000 mon.smithi042 (mon.0) 18 : cluster [INF] mon.smithi042 is new leader, mons smithi042,smithi191 in quorum (ranks 0,1) 2024-02-15T18:27:35.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:33.764719+0000 mon.smithi042 (mon.0) 19 : cluster [DBG] monmap e2: 2 mons at {smithi042=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],smithi191=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} 2024-02-15T18:27:35.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:33.776674+0000 mon.smithi042 (mon.0) 20 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 1 up:standby-replay 2 up:standby 2024-02-15T18:27:35.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:33.776707+0000 mon.smithi042 (mon.0) 21 : cluster [DBG] osdmap e48: 6 total, 6 up, 6 in 2024-02-15T18:27:35.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:33.777084+0000 mon.smithi042 (mon.0) 22 : cluster [DBG] mgrmap e30: smithi042.kfoihb(active, since 84s), standbys: smithi191.gftdzo 2024-02-15T18:27:35.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:33.783315+0000 mon.smithi042 (mon.0) 23 : cluster [INF] overall HEALTH_OK 2024-02-15T18:27:35.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:33.786450+0000 mon.smithi042 (mon.0) 24 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:35.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:33.787498+0000 mon.smithi042 (mon.0) 25 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:27:35.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:34.689229+0000 mgr.smithi042.kfoihb (mgr.24435) 62 : cluster [DBG] pgmap v46: 65 pgs: 65 active+clean; 2.2 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 13 MiB/s rd, 23 MiB/s wr, 139 op/s 2024-02-15T18:27:36.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:34.689229+0000 mgr.smithi042.kfoihb (mgr.24435) 62 : cluster [DBG] pgmap v46: 65 pgs: 65 active+clean; 2.2 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 13 MiB/s rd, 23 MiB/s wr, 139 op/s 2024-02-15T18:27:37.696 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:36.689548+0000 mgr.smithi042.kfoihb (mgr.24435) 63 : cluster [DBG] pgmap v47: 65 pgs: 65 active+clean; 2.2 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 13 MiB/s rd, 23 MiB/s wr, 139 op/s 2024-02-15T18:27:37.808 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:36.689548+0000 mgr.smithi042.kfoihb (mgr.24435) 63 : cluster [DBG] pgmap v47: 65 pgs: 65 active+clean; 2.2 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 13 MiB/s rd, 23 MiB/s wr, 139 op/s 2024-02-15T18:27:39.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:37.860418+0000 mon.smithi042 (mon.0) 26 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:39.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:38.253029+0000 mon.smithi042 (mon.0) 27 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:27:39.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:38.254922+0000 mon.smithi042 (mon.0) 28 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:27:39.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:38.261199+0000 mon.smithi042 (mon.0) 29 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:39.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:38.266950+0000 mon.smithi042 (mon.0) 30 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:27:39.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:38.269604+0000 mon.smithi042 (mon.0) 31 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:39.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:38.270746+0000 mon.smithi042 (mon.0) 32 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:39.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:38.274624+0000 mon.smithi042 (mon.0) 33 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:39.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:38.275313+0000 mon.smithi042 (mon.0) 34 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi042"}]: dispatch 2024-02-15T18:27:39.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:38.278432+0000 mon.smithi042 (mon.0) 35 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi042"}]': finished 2024-02-15T18:27:39.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:38.278955+0000 mon.smithi042 (mon.0) 36 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi191"}]: dispatch 2024-02-15T18:27:39.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:38.282108+0000 mon.smithi042 (mon.0) 37 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi191"}]': finished 2024-02-15T18:27:39.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:37.860418+0000 mon.smithi042 (mon.0) 26 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:39.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:38.253029+0000 mon.smithi042 (mon.0) 27 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:27:39.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:38.254922+0000 mon.smithi042 (mon.0) 28 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:27:39.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:38.261199+0000 mon.smithi042 (mon.0) 29 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:39.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:38.266950+0000 mon.smithi042 (mon.0) 30 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:27:39.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:38.269604+0000 mon.smithi042 (mon.0) 31 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:39.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:38.270746+0000 mon.smithi042 (mon.0) 32 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:39.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:38.274624+0000 mon.smithi042 (mon.0) 33 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:39.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:38.275313+0000 mon.smithi042 (mon.0) 34 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi042"}]: dispatch 2024-02-15T18:27:39.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:38.278432+0000 mon.smithi042 (mon.0) 35 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi042"}]': finished 2024-02-15T18:27:39.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:38.278955+0000 mon.smithi042 (mon.0) 36 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi191"}]: dispatch 2024-02-15T18:27:39.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:38.282108+0000 mon.smithi042 (mon.0) 37 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi191"}]': finished 2024-02-15T18:27:40.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:27:38.256359+0000 mgr.smithi042.kfoihb (mgr.24435) 64 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:27:40.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:27:38.271278+0000 mgr.smithi042.kfoihb (mgr.24435) 65 : cephadm [INF] Upgrade: Setting container_image for all mon 2024-02-15T18:27:40.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:38.690170+0000 mgr.smithi042.kfoihb (mgr.24435) 66 : cluster [DBG] pgmap v48: 65 pgs: 65 active+clean; 2.0 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 13 MiB/s rd, 24 MiB/s wr, 195 op/s 2024-02-15T18:27:40.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:27:38.256359+0000 mgr.smithi042.kfoihb (mgr.24435) 64 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:27:40.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:27:38.271278+0000 mgr.smithi042.kfoihb (mgr.24435) 65 : cephadm [INF] Upgrade: Setting container_image for all mon 2024-02-15T18:27:40.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:38.690170+0000 mgr.smithi042.kfoihb (mgr.24435) 66 : cluster [DBG] pgmap v48: 65 pgs: 65 active+clean; 2.0 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 13 MiB/s rd, 24 MiB/s wr, 195 op/s 2024-02-15T18:27:41.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:27:40.186005+0000 mgr.smithi042.kfoihb (mgr.24435) 67 : cephadm [INF] Upgrade: Updating crash.smithi042 (1/2) 2024-02-15T18:27:41.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:40.189793+0000 mon.smithi042 (mon.0) 38 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:41.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:40.190594+0000 mon.smithi042 (mon.0) 39 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi042", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:27:41.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:40.191239+0000 mon.smithi042 (mon.0) 40 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:27:41.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:27:40.191736+0000 mgr.smithi042.kfoihb (mgr.24435) 68 : cephadm [INF] Deploying daemon crash.smithi042 on smithi042 2024-02-15T18:27:41.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:40.690485+0000 mgr.smithi042.kfoihb (mgr.24435) 69 : cluster [DBG] pgmap v49: 65 pgs: 65 active+clean; 2.0 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 610 KiB/s rd, 587 KiB/s wr, 102 op/s 2024-02-15T18:27:41.446 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:27:40.186005+0000 mgr.smithi042.kfoihb (mgr.24435) 67 : cephadm [INF] Upgrade: Updating crash.smithi042 (1/2) 2024-02-15T18:27:41.446 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:40.189793+0000 mon.smithi042 (mon.0) 38 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:41.446 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:40.190594+0000 mon.smithi042 (mon.0) 39 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi042", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:27:41.446 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:40.191239+0000 mon.smithi042 (mon.0) 40 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:27:41.446 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:27:40.191736+0000 mgr.smithi042.kfoihb (mgr.24435) 68 : cephadm [INF] Deploying daemon crash.smithi042 on smithi042 2024-02-15T18:27:41.446 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:40.690485+0000 mgr.smithi042.kfoihb (mgr.24435) 69 : cluster [DBG] pgmap v49: 65 pgs: 65 active+clean; 2.0 GiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 610 KiB/s rd, 587 KiB/s wr, 102 op/s 2024-02-15T18:27:42.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.339+0000 7fb4d3552700 1 -- 172.21.15.42:0/1621646826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4cc0f55c0 msgr2=0x7fb4cc0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:42.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.339+0000 7fb4d3552700 1 --2- 172.21.15.42:0/1621646826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4cc0f55c0 0x7fb4cc0f59e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fb4c8004660 tx=0x7fb4c8009b30).stop 2024-02-15T18:27:42.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.339+0000 7fb4d3552700 1 -- 172.21.15.42:0/1621646826 shutdown_connections 2024-02-15T18:27:42.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.339+0000 7fb4d3552700 1 --2- 172.21.15.42:0/1621646826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4cc0f6860 0x7fb4cc0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:42.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.339+0000 7fb4d3552700 1 --2- 172.21.15.42:0/1621646826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4cc0f55c0 0x7fb4cc0f59e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:42.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.339+0000 7fb4d3552700 1 -- 172.21.15.42:0/1621646826 >> 172.21.15.42:0/1621646826 conn(0x7fb4cc0f0d40 msgr2=0x7fb4cc0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.339+0000 7fb4d3552700 1 -- 172.21.15.42:0/1621646826 shutdown_connections 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.339+0000 7fb4d3552700 1 -- 172.21.15.42:0/1621646826 wait complete. 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.340+0000 7fb4d3552700 1 Processor -- start 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.340+0000 7fb4d3552700 1 -- start start 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.340+0000 7fb4d3552700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4cc0f55c0 0x7fb4cc1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.340+0000 7fb4d3552700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4cc0f6860 0x7fb4cc1096c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.340+0000 7fb4d3552700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc109c70 con 0x7fb4cc0f55c0 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.340+0000 7fb4d3552700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc109db0 con 0x7fb4cc0f6860 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.340+0000 7fb4d12ee700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4cc0f55c0 0x7fb4cc1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.340+0000 7fb4d12ee700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4cc0f55c0 0x7fb4cc1091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56366/0 (socket says 172.21.15.42:56366) 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.340+0000 7fb4d12ee700 1 -- 172.21.15.42:0/3534403948 learned_addr learned my addr 172.21.15.42:0/3534403948 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.340+0000 7fb4d0aed700 1 --2- 172.21.15.42:0/3534403948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4cc0f6860 0x7fb4cc1096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.340+0000 7fb4d12ee700 1 -- 172.21.15.42:0/3534403948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4cc0f6860 msgr2=0x7fb4cc1096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.340+0000 7fb4d12ee700 1 --2- 172.21.15.42:0/3534403948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4cc0f6860 0x7fb4cc1096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:42.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.340+0000 7fb4d12ee700 1 -- 172.21.15.42:0/3534403948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c8005040 con 0x7fb4cc0f55c0 2024-02-15T18:27:42.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.341+0000 7fb4d12ee700 1 --2- 172.21.15.42:0/3534403948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4cc0f55c0 0x7fb4cc1091d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb4c8002040 tx=0x7fb4c800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:42.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.341+0000 7fb4c1ffb700 1 -- 172.21.15.42:0/3534403948 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4c8019070 con 0x7fb4cc0f55c0 2024-02-15T18:27:42.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.341+0000 7fb4d3552700 1 -- 172.21.15.42:0/3534403948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc10a6f0 con 0x7fb4cc0f55c0 2024-02-15T18:27:42.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.341+0000 7fb4c1ffb700 1 -- 172.21.15.42:0/3534403948 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fb4c8007e80 con 0x7fb4cc0f55c0 2024-02-15T18:27:42.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.341+0000 7fb4c1ffb700 1 -- 172.21.15.42:0/3534403948 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4c8014db0 con 0x7fb4cc0f55c0 2024-02-15T18:27:42.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.341+0000 7fb4d3552700 1 -- 172.21.15.42:0/3534403948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc10ac50 con 0x7fb4cc0f55c0 2024-02-15T18:27:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.342+0000 7fb4d3552700 1 -- 172.21.15.42:0/3534403948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc045ad0 con 0x7fb4cc0f55c0 2024-02-15T18:27:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.343+0000 7fb4c1ffb700 1 -- 172.21.15.42:0/3534403948 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fb4c8025070 con 0x7fb4cc0f55c0 2024-02-15T18:27:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.343+0000 7fb4c1ffb700 1 --2- 172.21.15.42:0/3534403948 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb4b8060ec0 0x7fb4b8063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.343+0000 7fb4c1ffb700 1 -- 172.21.15.42:0/3534403948 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fb4c807d020 con 0x7fb4cc0f55c0 2024-02-15T18:27:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.343+0000 7fb4d0aed700 1 --2- 172.21.15.42:0/3534403948 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb4b8060ec0 0x7fb4b8063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:42.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.344+0000 7fb4d0aed700 1 --2- 172.21.15.42:0/3534403948 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb4b8060ec0 0x7fb4b8063380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fb4bc009fd0 tx=0x7fb4bc00b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:42.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.348+0000 7fb4c1ffb700 1 -- 172.21.15.42:0/3534403948 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb4c8050e70 con 0x7fb4cc0f55c0 2024-02-15T18:27:42.537 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:41.203142+0000 mon.smithi042 (mon.0) 41 : cluster [DBG] mgrmap e31: smithi042.kfoihb(active, since 91s), standbys: smithi191.gftdzo 2024-02-15T18:27:42.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.552+0000 7fb4d3552700 1 -- 172.21.15.42:0/3534403948 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4cc0fb2c0 con 0x7fb4b8060ec0 2024-02-15T18:27:42.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.554+0000 7fb4c1ffb700 1 -- 172.21.15.42:0/3534403948 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fb4cc0fb2c0 con 0x7fb4b8060ec0 2024-02-15T18:27:42.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.555+0000 7fb4d3552700 1 -- 172.21.15.42:0/3534403948 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb4b8060ec0 msgr2=0x7fb4b8063380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:42.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.555+0000 7fb4d3552700 1 --2- 172.21.15.42:0/3534403948 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb4b8060ec0 0x7fb4b8063380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fb4bc009fd0 tx=0x7fb4bc00b040).stop 2024-02-15T18:27:42.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.555+0000 7fb4d3552700 1 -- 172.21.15.42:0/3534403948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4cc0f55c0 msgr2=0x7fb4cc1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:42.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.555+0000 7fb4d3552700 1 --2- 172.21.15.42:0/3534403948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4cc0f55c0 0x7fb4cc1091d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb4c8002040 tx=0x7fb4c800b040).stop 2024-02-15T18:27:42.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.555+0000 7fb4d3552700 1 -- 172.21.15.42:0/3534403948 shutdown_connections 2024-02-15T18:27:42.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.555+0000 7fb4d3552700 1 --2- 172.21.15.42:0/3534403948 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb4b8060ec0 0x7fb4b8063380 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:42.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.555+0000 7fb4d3552700 1 --2- 172.21.15.42:0/3534403948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4cc0f55c0 0x7fb4cc1091d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:42.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.555+0000 7fb4d3552700 1 --2- 172.21.15.42:0/3534403948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4cc0f6860 0x7fb4cc1096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:42.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.555+0000 7fb4d3552700 1 -- 172.21.15.42:0/3534403948 >> 172.21.15.42:0/3534403948 conn(0x7fb4cc0f0d40 msgr2=0x7fb4cc05f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:42.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.556+0000 7fb4d3552700 1 -- 172.21.15.42:0/3534403948 shutdown_connections 2024-02-15T18:27:42.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.556+0000 7fb4d3552700 1 -- 172.21.15.42:0/3534403948 wait complete. 2024-02-15T18:27:42.561 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:27:42.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.666+0000 7f41c8ab3700 1 -- 172.21.15.42:0/4207167498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41c40f49a0 msgr2=0x7f41c40f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:42.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.666+0000 7f41c8ab3700 1 --2- 172.21.15.42:0/4207167498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41c40f49a0 0x7f41c40f4dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f41b4004660 tx=0x7f41b4009b30).stop 2024-02-15T18:27:42.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.666+0000 7f41c8ab3700 1 -- 172.21.15.42:0/4207167498 shutdown_connections 2024-02-15T18:27:42.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.666+0000 7f41c8ab3700 1 --2- 172.21.15.42:0/4207167498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c40f5340 0x7f41c40f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:42.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.666+0000 7f41c8ab3700 1 --2- 172.21.15.42:0/4207167498 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41c40f49a0 0x7f41c40f4dc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:42.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.666+0000 7f41c8ab3700 1 -- 172.21.15.42:0/4207167498 >> 172.21.15.42:0/4207167498 conn(0x7f41c40f04c0 msgr2=0x7f41c40f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:42.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.666+0000 7f41c8ab3700 1 -- 172.21.15.42:0/4207167498 shutdown_connections 2024-02-15T18:27:42.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.666+0000 7f41c8ab3700 1 -- 172.21.15.42:0/4207167498 wait complete. 2024-02-15T18:27:42.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.666+0000 7f41c8ab3700 1 Processor -- start 2024-02-15T18:27:42.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.666+0000 7f41c8ab3700 1 -- start start 2024-02-15T18:27:42.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.667+0000 7f41c8ab3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41c40f49a0 0x7f41c4107020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:42.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.667+0000 7f41c8ab3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c40f5340 0x7f41c4107510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:42.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.667+0000 7f41c8ab3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c4107ac0 con 0x7f41c40f49a0 2024-02-15T18:27:42.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.667+0000 7f41c8ab3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c4107c00 con 0x7f41c40f5340 2024-02-15T18:27:42.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.667+0000 7f41c259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41c40f49a0 0x7f41c4107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:42.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.667+0000 7f41c259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41c40f49a0 0x7f41c4107020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56384/0 (socket says 172.21.15.42:56384) 2024-02-15T18:27:42.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.667+0000 7f41c259c700 1 -- 172.21.15.42:0/1691138964 learned_addr learned my addr 172.21.15.42:0/1691138964 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:27:42.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.667+0000 7f41c1d9b700 1 --2- 172.21.15.42:0/1691138964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c40f5340 0x7f41c4107510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:42.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.667+0000 7f41c259c700 1 -- 172.21.15.42:0/1691138964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c40f5340 msgr2=0x7f41c4107510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:42.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.667+0000 7f41c259c700 1 --2- 172.21.15.42:0/1691138964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c40f5340 0x7f41c4107510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:42.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.667+0000 7f41c259c700 1 -- 172.21.15.42:0/1691138964 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b4005040 con 0x7f41c40f49a0 2024-02-15T18:27:42.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.667+0000 7f41c259c700 1 --2- 172.21.15.42:0/1691138964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41c40f49a0 0x7f41c4107020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f41b4002040 tx=0x7f41b400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:42.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.668+0000 7f41b2ffd700 1 -- 172.21.15.42:0/1691138964 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b4019070 con 0x7f41c40f49a0 2024-02-15T18:27:42.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.668+0000 7f41c8ab3700 1 -- 172.21.15.42:0/1691138964 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c4108540 con 0x7f41c40f49a0 2024-02-15T18:27:42.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.668+0000 7f41c8ab3700 1 -- 172.21.15.42:0/1691138964 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c4108a20 con 0x7f41c40f49a0 2024-02-15T18:27:42.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.668+0000 7f41b2ffd700 1 -- 172.21.15.42:0/1691138964 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f41b4007e80 con 0x7f41c40f49a0 2024-02-15T18:27:42.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.668+0000 7f41b2ffd700 1 -- 172.21.15.42:0/1691138964 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b4014db0 con 0x7f41c40f49a0 2024-02-15T18:27:42.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.669+0000 7f41c8ab3700 1 -- 172.21.15.42:0/1691138964 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a8004fa0 con 0x7f41c40f49a0 2024-02-15T18:27:42.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.669+0000 7f41b2ffd700 1 -- 172.21.15.42:0/1691138964 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f41b4025070 con 0x7f41c40f49a0 2024-02-15T18:27:42.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.670+0000 7f41b2ffd700 1 --2- 172.21.15.42:0/1691138964 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f41ac060f70 0x7f41ac063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:42.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.670+0000 7f41b2ffd700 1 -- 172.21.15.42:0/1691138964 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f41b407cf50 con 0x7f41c40f49a0 2024-02-15T18:27:42.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.670+0000 7f41c1d9b700 1 --2- 172.21.15.42:0/1691138964 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f41ac060f70 0x7f41ac063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.670+0000 7f41c1d9b700 1 --2- 172.21.15.42:0/1691138964 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f41ac060f70 0x7f41ac063430 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f41b8009fd0 tx=0x7f41b800b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:42.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.675+0000 7f41b2ffd700 1 -- 172.21.15.42:0/1691138964 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f41b4051eb0 con 0x7f41c40f49a0 2024-02-15T18:27:42.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:41.203142+0000 mon.smithi042 (mon.0) 41 : cluster [DBG] mgrmap e31: smithi042.kfoihb(active, since 91s), standbys: smithi191.gftdzo 2024-02-15T18:27:42.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.870+0000 7f41c8ab3700 1 -- 172.21.15.42:0/1691138964 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41a8000bc0 con 0x7f41ac060f70 2024-02-15T18:27:42.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.872+0000 7f41b2ffd700 1 -- 172.21.15.42:0/1691138964 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f41a8000bc0 con 0x7f41ac060f70 2024-02-15T18:27:42.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.873+0000 7f41c8ab3700 1 -- 172.21.15.42:0/1691138964 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f41ac060f70 msgr2=0x7f41ac063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:42.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.873+0000 7f41c8ab3700 1 --2- 172.21.15.42:0/1691138964 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f41ac060f70 0x7f41ac063430 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f41b8009fd0 tx=0x7f41b800b040).stop 2024-02-15T18:27:42.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.873+0000 7f41c8ab3700 1 -- 172.21.15.42:0/1691138964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41c40f49a0 msgr2=0x7f41c4107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:42.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.873+0000 7f41c8ab3700 1 --2- 172.21.15.42:0/1691138964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41c40f49a0 0x7f41c4107020 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f41b4002040 tx=0x7f41b400b040).stop 2024-02-15T18:27:42.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.873+0000 7f41c8ab3700 1 -- 172.21.15.42:0/1691138964 shutdown_connections 2024-02-15T18:27:42.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.873+0000 7f41c8ab3700 1 --2- 172.21.15.42:0/1691138964 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f41ac060f70 0x7f41ac063430 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:42.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.873+0000 7f41c8ab3700 1 --2- 172.21.15.42:0/1691138964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41c40f49a0 0x7f41c4107020 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:42.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.873+0000 7f41c8ab3700 1 --2- 172.21.15.42:0/1691138964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41c40f5340 0x7f41c4107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:42.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.873+0000 7f41c8ab3700 1 -- 172.21.15.42:0/1691138964 >> 172.21.15.42:0/1691138964 conn(0x7f41c40f04c0 msgr2=0x7f41c40fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:42.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.874+0000 7f41c8ab3700 1 -- 172.21.15.42:0/1691138964 shutdown_connections 2024-02-15T18:27:42.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:42.874+0000 7f41c8ab3700 1 -- 172.21.15.42:0/1691138964 wait complete. 2024-02-15T18:27:42.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.000+0000 7f3e2332c700 1 -- 172.21.15.42:0/3101293119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1c0f49a0 msgr2=0x7f3e1c0f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:42.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.000+0000 7f3e2332c700 1 --2- 172.21.15.42:0/3101293119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1c0f49a0 0x7f3e1c0f4dc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f3e18004660 tx=0x7f3e18009b30).stop 2024-02-15T18:27:42.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.004+0000 7f3e2332c700 1 -- 172.21.15.42:0/3101293119 shutdown_connections 2024-02-15T18:27:42.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.004+0000 7f3e2332c700 1 --2- 172.21.15.42:0/3101293119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e1c0f5340 0x7f3e1c0f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:43.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.004+0000 7f3e2332c700 1 --2- 172.21.15.42:0/3101293119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1c0f49a0 0x7f3e1c0f4dc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:43.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.004+0000 7f3e2332c700 1 -- 172.21.15.42:0/3101293119 >> 172.21.15.42:0/3101293119 conn(0x7f3e1c0f04c0 msgr2=0x7f3e1c0f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:43.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.006+0000 7f3e2332c700 1 -- 172.21.15.42:0/3101293119 shutdown_connections 2024-02-15T18:27:43.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.007+0000 7f3e2332c700 1 -- 172.21.15.42:0/3101293119 wait complete. 2024-02-15T18:27:43.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.008+0000 7f3e2332c700 1 Processor -- start 2024-02-15T18:27:43.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.009+0000 7f3e2332c700 1 -- start start 2024-02-15T18:27:43.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.009+0000 7f3e2332c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1c0f49a0 0x7f3e1c104ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:43.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.009+0000 7f3e2332c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e1c0f5340 0x7f3e1c1053d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:43.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.009+0000 7f3e2332c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c105980 con 0x7f3e1c0f49a0 2024-02-15T18:27:43.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.009+0000 7f3e2332c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1c105ac0 con 0x7f3e1c0f5340 2024-02-15T18:27:43.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.009+0000 7f3e210c8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1c0f49a0 0x7f3e1c104ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:43.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.009+0000 7f3e210c8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1c0f49a0 0x7f3e1c104ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56408/0 (socket says 172.21.15.42:56408) 2024-02-15T18:27:43.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.009+0000 7f3e210c8700 1 -- 172.21.15.42:0/244807271 learned_addr learned my addr 172.21.15.42:0/244807271 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:27:43.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.010+0000 7f3e210c8700 1 -- 172.21.15.42:0/244807271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e1c0f5340 msgr2=0x7f3e1c1053d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-15T18:27:43.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.010+0000 7f3e210c8700 1 --2- 172.21.15.42:0/244807271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e1c0f5340 0x7f3e1c1053d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:27:43.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.010+0000 7f3e210c8700 1 -- 172.21.15.42:0/244807271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e18005040 con 0x7f3e1c0f49a0 2024-02-15T18:27:43.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.010+0000 7f3e210c8700 1 --2- 172.21.15.42:0/244807271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1c0f49a0 0x7f3e1c104ee0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f3e18002040 tx=0x7f3e1800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:43.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.012+0000 7f3e11ffb700 1 -- 172.21.15.42:0/244807271 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e18019070 con 0x7f3e1c0f49a0 2024-02-15T18:27:43.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.012+0000 7f3e11ffb700 1 -- 172.21.15.42:0/244807271 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f3e18007e80 con 0x7f3e1c0f49a0 2024-02-15T18:27:43.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.012+0000 7f3e11ffb700 1 -- 172.21.15.42:0/244807271 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e18014e10 con 0x7f3e1c0f49a0 2024-02-15T18:27:43.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.013+0000 7f3e2332c700 1 -- 172.21.15.42:0/244807271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1c106400 con 0x7f3e1c0f49a0 2024-02-15T18:27:43.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.013+0000 7f3e2332c700 1 -- 172.21.15.42:0/244807271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1c106910 con 0x7f3e1c0f49a0 2024-02-15T18:27:43.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.017+0000 7f3e2332c700 1 -- 172.21.15.42:0/244807271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1c045ad0 con 0x7f3e1c0f49a0 2024-02-15T18:27:43.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.017+0000 7f3e11ffb700 1 -- 172.21.15.42:0/244807271 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f3e18025070 con 0x7f3e1c0f49a0 2024-02-15T18:27:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.018+0000 7f3e11ffb700 1 --2- 172.21.15.42:0/244807271 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3e08060fc0 0x7f3e08063480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:43.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.018+0000 7f3e11ffb700 1 -- 172.21.15.42:0/244807271 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f3e1807d350 con 0x7f3e1c0f49a0 2024-02-15T18:27:43.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.019+0000 7f3e208c7700 1 --2- 172.21.15.42:0/244807271 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3e08060fc0 0x7f3e08063480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:43.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.020+0000 7f3e208c7700 1 --2- 172.21.15.42:0/244807271 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3e08060fc0 0x7f3e08063480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3e1c0f5780 tx=0x7f3e0c013040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:43.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.027+0000 7f3e11ffb700 1 -- 172.21.15.42:0/244807271 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3e180511c0 con 0x7f3e1c0f49a0 2024-02-15T18:27:43.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.273+0000 7f3e2332c700 1 -- 172.21.15.42:0/244807271 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e1c106bc0 con 0x7f3e08060fc0 2024-02-15T18:27:43.280 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:27:43.280 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (7m) 44s ago 9m 20.6M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:27:43.280 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (9m) 44s ago 9m 6400k - 16.2.4 8d91d370c2b8 94d74c33c758 2024-02-15T18:27:43.280 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (8m) 5s ago 8m 6405k - 16.2.4 8d91d370c2b8 ff0f56f9475a 2024-02-15T18:27:43.280 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (7m) 44s ago 8m 33.7M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:27:43.280 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (3m) 44s ago 3m 13.9M - 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:27:43.280 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (3m) 44s ago 3m 16.6M - 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:27:43.281 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (3m) 5s ago 3m 148M - 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:27:43.281 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (4m) 5s ago 4m 227M - 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:27:43.281 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (101s) 44s ago 10m 425M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:27:43.281 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (79s) 5s ago 8m 387M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:27:43.281 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (57s) 44s ago 10m 23.5M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:27:43.281 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (18s) 5s ago 8m 18.9M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:27:43.281 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (8m) 44s ago 8m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:27:43.281 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (7m) 5s ago 7m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:27:43.281 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (7m) 44s ago 7m 146M 4096M 16.2.4 8d91d370c2b8 973745b31d5a 2024-02-15T18:27:43.281 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (7m) 44s ago 7m 161M 4096M 16.2.4 8d91d370c2b8 2b3ae86111e8 2024-02-15T18:27:43.281 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (6m) 44s ago 6m 144M 4096M 16.2.4 8d91d370c2b8 3706071e4698 2024-02-15T18:27:43.281 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (6m) 5s ago 6m 438M 4096M 16.2.4 8d91d370c2b8 e6fe8e77c6fe 2024-02-15T18:27:43.281 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (5m) 5s ago 5m 389M 4096M 16.2.4 8d91d370c2b8 24582012e234 2024-02-15T18:27:43.282 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (5m) 5s ago 5m 348M 4096M 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:27:43.282 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (108s) 44s ago 8m 48.2M - 2.18.1 de242295e225 63b6dfb825ed 2024-02-15T18:27:43.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.284+0000 7f3e11ffb700 1 -- 172.21.15.42:0/244807271 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f3e1c106bc0 con 0x7f3e08060fc0 2024-02-15T18:27:43.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.286+0000 7f3e2332c700 1 -- 172.21.15.42:0/244807271 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3e08060fc0 msgr2=0x7f3e08063480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:43.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.286+0000 7f3e2332c700 1 --2- 172.21.15.42:0/244807271 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3e08060fc0 0x7f3e08063480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3e1c0f5780 tx=0x7f3e0c013040).stop 2024-02-15T18:27:43.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.286+0000 7f3e2332c700 1 -- 172.21.15.42:0/244807271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1c0f49a0 msgr2=0x7f3e1c104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:43.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.286+0000 7f3e2332c700 1 --2- 172.21.15.42:0/244807271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1c0f49a0 0x7f3e1c104ee0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f3e18002040 tx=0x7f3e1800b040).stop 2024-02-15T18:27:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.286+0000 7f3e2332c700 1 -- 172.21.15.42:0/244807271 shutdown_connections 2024-02-15T18:27:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.286+0000 7f3e2332c700 1 --2- 172.21.15.42:0/244807271 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3e08060fc0 0x7f3e08063480 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.286+0000 7f3e2332c700 1 --2- 172.21.15.42:0/244807271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e1c0f49a0 0x7f3e1c104ee0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.286+0000 7f3e2332c700 1 --2- 172.21.15.42:0/244807271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e1c0f5340 0x7f3e1c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:27:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.286+0000 7f3e2332c700 1 -- 172.21.15.42:0/244807271 >> 172.21.15.42:0/244807271 conn(0x7f3e1c0f04c0 msgr2=0x7f3e1c0fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.286+0000 7f3e2332c700 1 -- 172.21.15.42:0/244807271 shutdown_connections 2024-02-15T18:27:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.286+0000 7f3e2332c700 1 -- 172.21.15.42:0/244807271 wait complete. 2024-02-15T18:27:43.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.415+0000 7fa915fda700 1 -- 172.21.15.42:0/1921200234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9100f7980 msgr2=0x7fa9100f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:43.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.415+0000 7fa915fda700 1 --2- 172.21.15.42:0/1921200234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9100f7980 0x7fa9100f7de0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fa904004660 tx=0x7fa904009b30).stop 2024-02-15T18:27:43.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.415+0000 7fa915fda700 1 -- 172.21.15.42:0/1921200234 shutdown_connections 2024-02-15T18:27:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.415+0000 7fa915fda700 1 --2- 172.21.15.42:0/1921200234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9100f7980 0x7fa9100f7de0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.415+0000 7fa915fda700 1 --2- 172.21.15.42:0/1921200234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9100f66e0 0x7fa9100f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.415+0000 7fa915fda700 1 -- 172.21.15.42:0/1921200234 >> 172.21.15.42:0/1921200234 conn(0x7fa9100f1e40 msgr2=0x7fa9100f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.415+0000 7fa915fda700 1 -- 172.21.15.42:0/1921200234 shutdown_connections 2024-02-15T18:27:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.415+0000 7fa915fda700 1 -- 172.21.15.42:0/1921200234 wait complete. 2024-02-15T18:27:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.415+0000 7fa915fda700 1 Processor -- start 2024-02-15T18:27:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.415+0000 7fa915fda700 1 -- start start 2024-02-15T18:27:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa915fda700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9100f66e0 0x7fa910107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa915fda700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9100f7980 0x7fa9101074f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa915fda700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa910107aa0 con 0x7fa9100f7980 2024-02-15T18:27:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa915fda700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa910107be0 con 0x7fa9100f66e0 2024-02-15T18:27:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa90effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9100f7980 0x7fa9101074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa90f7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9100f66e0 0x7fa910107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:43.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa90effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9100f7980 0x7fa9101074f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56426/0 (socket says 172.21.15.42:56426) 2024-02-15T18:27:43.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa90effd700 1 -- 172.21.15.42:0/3223554822 learned_addr learned my addr 172.21.15.42:0/3223554822 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:27:43.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa90f7fe700 1 -- 172.21.15.42:0/3223554822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9100f7980 msgr2=0x7fa9101074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:43.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa90f7fe700 1 --2- 172.21.15.42:0/3223554822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9100f7980 0x7fa9101074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:43.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa90f7fe700 1 -- 172.21.15.42:0/3223554822 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa904005040 con 0x7fa9100f66e0 2024-02-15T18:27:43.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa90effd700 1 --2- 172.21.15.42:0/3223554822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9100f7980 0x7fa9101074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:27:43.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa90f7fe700 1 --2- 172.21.15.42:0/3223554822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9100f66e0 0x7fa910107000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fa9100f77e0 tx=0x7fa90000b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:43.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa8f7fff700 1 -- 172.21.15.42:0/3223554822 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa900015070 con 0x7fa9100f66e0 2024-02-15T18:27:43.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa8f7fff700 1 -- 172.21.15.42:0/3223554822 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fa900007ca0 con 0x7fa9100f66e0 2024-02-15T18:27:43.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa915fda700 1 -- 172.21.15.42:0/3223554822 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa910108580 con 0x7fa9100f66e0 2024-02-15T18:27:43.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa8f7fff700 1 -- 172.21.15.42:0/3223554822 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa90000bba0 con 0x7fa9100f66e0 2024-02-15T18:27:43.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.416+0000 7fa915fda700 1 -- 172.21.15.42:0/3223554822 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa910108b40 con 0x7fa9100f66e0 2024-02-15T18:27:43.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.418+0000 7fa915fda700 1 -- 172.21.15.42:0/3223554822 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa910045ad0 con 0x7fa9100f66e0 2024-02-15T18:27:43.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.418+0000 7fa8f7fff700 1 -- 172.21.15.42:0/3223554822 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fa90000bd00 con 0x7fa9100f66e0 2024-02-15T18:27:43.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.418+0000 7fa8f7fff700 1 --2- 172.21.15.42:0/3223554822 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fa8f8060f20 0x7fa8f80633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:43.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.418+0000 7fa8f7fff700 1 -- 172.21.15.42:0/3223554822 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fa9000804b0 con 0x7fa9100f66e0 2024-02-15T18:27:43.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.418+0000 7fa90effd700 1 --2- 172.21.15.42:0/3223554822 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fa8f8060f20 0x7fa8f80633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:43.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.419+0000 7fa90effd700 1 --2- 172.21.15.42:0/3223554822 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fa8f8060f20 0x7fa8f80633e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa904009720 tx=0x7fa904016040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:43.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.423+0000 7fa8f7fff700 1 -- 172.21.15.42:0/3223554822 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa9000544b0 con 0x7fa9100f66e0 2024-02-15T18:27:43.508 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:42.554473+0000 mgr.smithi042.kfoihb (mgr.24435) 70 : audit [DBG] from='client.34122 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:43.508 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:42.691089+0000 mgr.smithi042.kfoihb (mgr.24435) 71 : cluster [DBG] pgmap v50: 65 pgs: 65 active+clean; 1.5 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 227 op/s 2024-02-15T18:27:43.508 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:42.872438+0000 mgr.smithi042.kfoihb (mgr.24435) 72 : audit [DBG] from='client.34126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:43.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.692+0000 7fa915fda700 1 -- 172.21.15.42:0/3223554822 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa9100ff130 con 0x7fa9100f66e0 2024-02-15T18:27:43.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.693+0000 7fa8f7fff700 1 -- 172.21.15.42:0/3223554822 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+712 (secure 0 0 0) 0x7fa9000540a0 con 0x7fa9100f66e0 2024-02-15T18:27:43.689 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:27:43.689 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:27:43.689 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:27:43.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:27:43.689 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:27:43.689 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:27:43.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:27:43.689 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:27:43.689 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:27:43.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:27:43.689 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:27:43.690 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:27:43.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:27:43.690 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:27:43.690 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4, 2024-02-15T18:27:43.690 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 10 2024-02-15T18:27:43.690 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:27:43.690 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:27:43.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.694+0000 7fa915fda700 1 -- 172.21.15.42:0/3223554822 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fa8f8060f20 msgr2=0x7fa8f80633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:43.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.694+0000 7fa915fda700 1 --2- 172.21.15.42:0/3223554822 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fa8f8060f20 0x7fa8f80633e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fa904009720 tx=0x7fa904016040).stop 2024-02-15T18:27:43.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.694+0000 7fa915fda700 1 -- 172.21.15.42:0/3223554822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9100f66e0 msgr2=0x7fa910107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:43.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.694+0000 7fa915fda700 1 --2- 172.21.15.42:0/3223554822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9100f66e0 0x7fa910107000 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fa9100f77e0 tx=0x7fa90000b040).stop 2024-02-15T18:27:43.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.694+0000 7fa915fda700 1 -- 172.21.15.42:0/3223554822 shutdown_connections 2024-02-15T18:27:43.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.694+0000 7fa915fda700 1 --2- 172.21.15.42:0/3223554822 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fa8f8060f20 0x7fa8f80633e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:43.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.694+0000 7fa915fda700 1 --2- 172.21.15.42:0/3223554822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9100f66e0 0x7fa910107000 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:43.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.694+0000 7fa915fda700 1 --2- 172.21.15.42:0/3223554822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9100f7980 0x7fa9101074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:43.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.694+0000 7fa915fda700 1 -- 172.21.15.42:0/3223554822 >> 172.21.15.42:0/3223554822 conn(0x7fa9100f1e40 msgr2=0x7fa9100fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:43.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.695+0000 7fa915fda700 1 -- 172.21.15.42:0/3223554822 shutdown_connections 2024-02-15T18:27:43.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.695+0000 7fa915fda700 1 -- 172.21.15.42:0/3223554822 wait complete. 2024-02-15T18:27:43.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:42.554473+0000 mgr.smithi042.kfoihb (mgr.24435) 70 : audit [DBG] from='client.34122 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:43.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:42.691089+0000 mgr.smithi042.kfoihb (mgr.24435) 71 : cluster [DBG] pgmap v50: 65 pgs: 65 active+clean; 1.5 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 227 op/s 2024-02-15T18:27:43.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:42.872438+0000 mgr.smithi042.kfoihb (mgr.24435) 72 : audit [DBG] from='client.34126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:43.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.799+0000 7fddc5736700 1 -- 172.21.15.42:0/939935004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddc00f6860 msgr2=0x7fddc00f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:43.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.799+0000 7fddc5736700 1 --2- 172.21.15.42:0/939935004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddc00f6860 0x7fddc00f6cc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fdda8004660 tx=0x7fdda8009b30).stop 2024-02-15T18:27:43.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.799+0000 7fddc5736700 1 -- 172.21.15.42:0/939935004 shutdown_connections 2024-02-15T18:27:43.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.799+0000 7fddc5736700 1 --2- 172.21.15.42:0/939935004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddc00f6860 0x7fddc00f6cc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:43.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.799+0000 7fddc5736700 1 --2- 172.21.15.42:0/939935004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddc00f55c0 0x7fddc00f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:43.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.799+0000 7fddc5736700 1 -- 172.21.15.42:0/939935004 >> 172.21.15.42:0/939935004 conn(0x7fddc00f0d40 msgr2=0x7fddc00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:43.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.799+0000 7fddc5736700 1 -- 172.21.15.42:0/939935004 shutdown_connections 2024-02-15T18:27:43.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.799+0000 7fddc5736700 1 -- 172.21.15.42:0/939935004 wait complete. 2024-02-15T18:27:43.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.800+0000 7fddc5736700 1 Processor -- start 2024-02-15T18:27:43.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.800+0000 7fddc5736700 1 -- start start 2024-02-15T18:27:43.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.800+0000 7fddc5736700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddc00f55c0 0x7fddc0109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:43.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.800+0000 7fddc5736700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddc00f6860 0x7fddc010c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:43.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.800+0000 7fddc5736700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc00603c0 con 0x7fddc00f6860 2024-02-15T18:27:43.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.800+0000 7fddc5736700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc0060500 con 0x7fddc00f55c0 2024-02-15T18:27:43.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.800+0000 7fddbe7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddc00f6860 0x7fddc010c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:43.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.800+0000 7fddbeffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddc00f55c0 0x7fddc0109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:43.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.800+0000 7fddbe7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddc00f6860 0x7fddc010c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56438/0 (socket says 172.21.15.42:56438) 2024-02-15T18:27:43.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.800+0000 7fddbe7fc700 1 -- 172.21.15.42:0/4211935573 learned_addr learned my addr 172.21.15.42:0/4211935573 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:27:43.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.800+0000 7fddbe7fc700 1 -- 172.21.15.42:0/4211935573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddc00f55c0 msgr2=0x7fddc0109cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:43.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.800+0000 7fddbe7fc700 1 --2- 172.21.15.42:0/4211935573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddc00f55c0 0x7fddc0109cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:43.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.800+0000 7fddbe7fc700 1 -- 172.21.15.42:0/4211935573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda8005040 con 0x7fddc00f6860 2024-02-15T18:27:43.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.800+0000 7fddbe7fc700 1 --2- 172.21.15.42:0/4211935573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddc00f6860 0x7fddc010c1d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fdda8000c00 tx=0x7fdda800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:43.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.801+0000 7fddb77fe700 1 -- 172.21.15.42:0/4211935573 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdda8019070 con 0x7fddc00f6860 2024-02-15T18:27:43.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.801+0000 7fddb77fe700 1 -- 172.21.15.42:0/4211935573 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fdda8007e80 con 0x7fddc00f6860 2024-02-15T18:27:43.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.801+0000 7fddb77fe700 1 -- 172.21.15.42:0/4211935573 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdda8014db0 con 0x7fddc00f6860 2024-02-15T18:27:43.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.801+0000 7fddc5736700 1 -- 172.21.15.42:0/4211935573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc010c720 con 0x7fddc00f6860 2024-02-15T18:27:43.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.801+0000 7fddc5736700 1 -- 172.21.15.42:0/4211935573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc010cc50 con 0x7fddc00f6860 2024-02-15T18:27:43.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.802+0000 7fddc5736700 1 -- 172.21.15.42:0/4211935573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc00f49b0 con 0x7fddc00f6860 2024-02-15T18:27:43.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.802+0000 7fddb77fe700 1 -- 172.21.15.42:0/4211935573 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fdda8025070 con 0x7fddc00f6860 2024-02-15T18:27:43.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.803+0000 7fddb77fe700 1 --2- 172.21.15.42:0/4211935573 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fddac060ec0 0x7fddac063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:43.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.803+0000 7fddb77fe700 1 -- 172.21.15.42:0/4211935573 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fdda807d000 con 0x7fddc00f6860 2024-02-15T18:27:43.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.803+0000 7fddbeffd700 1 --2- 172.21.15.42:0/4211935573 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fddac060ec0 0x7fddac063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:43.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.803+0000 7fddbeffd700 1 --2- 172.21.15.42:0/4211935573 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fddac060ec0 0x7fddac063380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fddc0060040 tx=0x7fddb000b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:43.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:43.807+0000 7fddb77fe700 1 -- 172.21.15.42:0/4211935573 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdda804d480 con 0x7fddc00f6860 2024-02-15T18:27:44.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.049+0000 7fddc5736700 1 -- 172.21.15.42:0/4211935573 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fddc0045ad0 con 0x7fddc00f6860 2024-02-15T18:27:44.051 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:27:44.051 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:27:44.052 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:27:44.052 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:27:44.052 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:27:44.052 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:27:44.052 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:27:44.052 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:27:44.052 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:27:44.052 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:27:44.052 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:27:44.052 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:27:44.052 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:27:44.052 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:27:44.052 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:27:44.053 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:27:44.053 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:27:44.053 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:27:44.053 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:27:44.053 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:27:44.053 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:27:44.053 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:27:44.053 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:27:44.053 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:27:44.053 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:27:44.053 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:27:44.053 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:27:44.053 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:27:44.054 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:27:44.054 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:27:44.054 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:27:44.054 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:27:44.054 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:27:44.054 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:27:44.054 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:27:44.054 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:27:44.054 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:27:44.054 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:27:44.054 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:27:44.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.050+0000 7fddb77fe700 1 -- 172.21.15.42:0/4211935573 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1773 (secure 0 0 0) 0x7fdda8023070 con 0x7fddc00f6860 2024-02-15T18:27:44.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.053+0000 7fddc5736700 1 -- 172.21.15.42:0/4211935573 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fddac060ec0 msgr2=0x7fddac063380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:44.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.053+0000 7fddc5736700 1 --2- 172.21.15.42:0/4211935573 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fddac060ec0 0x7fddac063380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fddc0060040 tx=0x7fddb000b040).stop 2024-02-15T18:27:44.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.053+0000 7fddc5736700 1 -- 172.21.15.42:0/4211935573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddc00f6860 msgr2=0x7fddc010c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:44.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.053+0000 7fddc5736700 1 --2- 172.21.15.42:0/4211935573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddc00f6860 0x7fddc010c1d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fdda8000c00 tx=0x7fdda800b040).stop 2024-02-15T18:27:44.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.056+0000 7fddc5736700 1 -- 172.21.15.42:0/4211935573 shutdown_connections 2024-02-15T18:27:44.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.056+0000 7fddc5736700 1 --2- 172.21.15.42:0/4211935573 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fddac060ec0 0x7fddac063380 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:44.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.056+0000 7fddc5736700 1 --2- 172.21.15.42:0/4211935573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddc00f55c0 0x7fddc0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:44.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.056+0000 7fddc5736700 1 --2- 172.21.15.42:0/4211935573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddc00f6860 0x7fddc010c1d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:44.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.056+0000 7fddc5736700 1 -- 172.21.15.42:0/4211935573 >> 172.21.15.42:0/4211935573 conn(0x7fddc00f0d40 msgr2=0x7fddc00fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:44.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.056+0000 7fddc5736700 1 -- 172.21.15.42:0/4211935573 shutdown_connections 2024-02-15T18:27:44.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.056+0000 7fddc5736700 1 -- 172.21.15.42:0/4211935573 wait complete. 2024-02-15T18:27:44.056 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:27:44.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.208+0000 7fe61ce66700 1 -- 172.21.15.42:0/3629006438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6180f6860 msgr2=0x7fe6180f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:44.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.208+0000 7fe61ce66700 1 --2- 172.21.15.42:0/3629006438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6180f6860 0x7fe6180f6cc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe60c004660 tx=0x7fe60c009b30).stop 2024-02-15T18:27:44.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.208+0000 7fe61ce66700 1 -- 172.21.15.42:0/3629006438 shutdown_connections 2024-02-15T18:27:44.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.208+0000 7fe61ce66700 1 --2- 172.21.15.42:0/3629006438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6180f6860 0x7fe6180f6cc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:44.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.208+0000 7fe61ce66700 1 --2- 172.21.15.42:0/3629006438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6180f55c0 0x7fe6180f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:44.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.208+0000 7fe61ce66700 1 -- 172.21.15.42:0/3629006438 >> 172.21.15.42:0/3629006438 conn(0x7fe6180f0d40 msgr2=0x7fe6180f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:44.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.208+0000 7fe61ce66700 1 -- 172.21.15.42:0/3629006438 shutdown_connections 2024-02-15T18:27:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.208+0000 7fe61ce66700 1 -- 172.21.15.42:0/3629006438 wait complete. 2024-02-15T18:27:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.209+0000 7fe61ce66700 1 Processor -- start 2024-02-15T18:27:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.209+0000 7fe61ce66700 1 -- start start 2024-02-15T18:27:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.209+0000 7fe61ce66700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6180f55c0 0x7fe6180f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.209+0000 7fe61ce66700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6180f6860 0x7fe6180f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.209+0000 7fe61ce66700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180f8100 con 0x7fe6180f55c0 2024-02-15T18:27:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.209+0000 7fe61ce66700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6180f8240 con 0x7fe6180f6860 2024-02-15T18:27:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.209+0000 7fe61659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6180f55c0 0x7fe6180f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.209+0000 7fe61659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6180f55c0 0x7fe6180f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56460/0 (socket says 172.21.15.42:56460) 2024-02-15T18:27:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.209+0000 7fe61659c700 1 -- 172.21.15.42:0/3887844307 learned_addr learned my addr 172.21.15.42:0/3887844307 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:27:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.209+0000 7fe615d9b700 1 --2- 172.21.15.42:0/3887844307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6180f6860 0x7fe6180f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:44.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.210+0000 7fe615d9b700 1 -- 172.21.15.42:0/3887844307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6180f55c0 msgr2=0x7fe6180f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:44.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.210+0000 7fe615d9b700 1 --2- 172.21.15.42:0/3887844307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6180f55c0 0x7fe6180f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:44.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.210+0000 7fe615d9b700 1 -- 172.21.15.42:0/3887844307 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60c005040 con 0x7fe6180f6860 2024-02-15T18:27:44.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.210+0000 7fe61659c700 1 --2- 172.21.15.42:0/3887844307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6180f55c0 0x7fe6180f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:27:44.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.210+0000 7fe615d9b700 1 --2- 172.21.15.42:0/3887844307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6180f6860 0x7fe6180f7ba0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe60c004660 tx=0x7fe60c00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:44.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.210+0000 7fe606ffd700 1 -- 172.21.15.42:0/3887844307 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe60c019070 con 0x7fe6180f6860 2024-02-15T18:27:44.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.210+0000 7fe61ce66700 1 -- 172.21.15.42:0/3887844307 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6180f84c0 con 0x7fe6180f6860 2024-02-15T18:27:44.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.210+0000 7fe606ffd700 1 -- 172.21.15.42:0/3887844307 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fe60c007e80 con 0x7fe6180f6860 2024-02-15T18:27:44.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.210+0000 7fe606ffd700 1 -- 172.21.15.42:0/3887844307 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe60c014e30 con 0x7fe6180f6860 2024-02-15T18:27:44.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.210+0000 7fe61ce66700 1 -- 172.21.15.42:0/3887844307 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6180f8a20 con 0x7fe6180f6860 2024-02-15T18:27:44.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.211+0000 7fe61ce66700 1 -- 172.21.15.42:0/3887844307 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe618045ad0 con 0x7fe6180f6860 2024-02-15T18:27:44.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.212+0000 7fe606ffd700 1 -- 172.21.15.42:0/3887844307 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fe60c025070 con 0x7fe6180f6860 2024-02-15T18:27:44.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.212+0000 7fe606ffd700 1 --2- 172.21.15.42:0/3887844307 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe600060ec0 0x7fe600063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:27:44.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.212+0000 7fe606ffd700 1 -- 172.21.15.42:0/3887844307 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe60c07cef0 con 0x7fe6180f6860 2024-02-15T18:27:44.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.212+0000 7fe61659c700 1 --2- 172.21.15.42:0/3887844307 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe600060ec0 0x7fe600063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:27:44.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.212+0000 7fe61659c700 1 --2- 172.21.15.42:0/3887844307 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe600060ec0 0x7fe600063380 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fe608004fd0 tx=0x7fe60800b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:27:44.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.220+0000 7fe606ffd700 1 -- 172.21.15.42:0/3887844307 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe60c04d370 con 0x7fe6180f6860 2024-02-15T18:27:44.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.417+0000 7fe61ce66700 1 -- 172.21.15.42:0/3887844307 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6180faca0 con 0x7fe600060ec0 2024-02-15T18:27:44.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.418+0000 7fe606ffd700 1 -- 172.21.15.42:0/3887844307 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fe6180faca0 con 0x7fe600060ec0 2024-02-15T18:27:44.414 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:27:44.414 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:27:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:27:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:27:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:27:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-02-15T18:27:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-02-15T18:27:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:27:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "4/21 daemons upgraded", 2024-02-15T18:27:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading crash daemons", 2024-02-15T18:27:44.415 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:27:44.415 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:27:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.420+0000 7fe61ce66700 1 -- 172.21.15.42:0/3887844307 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe600060ec0 msgr2=0x7fe600063380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.420+0000 7fe61ce66700 1 --2- 172.21.15.42:0/3887844307 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe600060ec0 0x7fe600063380 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fe608004fd0 tx=0x7fe60800b040).stop 2024-02-15T18:27:44.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.420+0000 7fe61ce66700 1 -- 172.21.15.42:0/3887844307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6180f6860 msgr2=0x7fe6180f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:27:44.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.420+0000 7fe61ce66700 1 --2- 172.21.15.42:0/3887844307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6180f6860 0x7fe6180f7ba0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fe60c004660 tx=0x7fe60c00b040).stop 2024-02-15T18:27:44.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.420+0000 7fe61ce66700 1 -- 172.21.15.42:0/3887844307 shutdown_connections 2024-02-15T18:27:44.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.420+0000 7fe61ce66700 1 --2- 172.21.15.42:0/3887844307 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe600060ec0 0x7fe600063380 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:44.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.420+0000 7fe61ce66700 1 --2- 172.21.15.42:0/3887844307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6180f55c0 0x7fe6180f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:44.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.420+0000 7fe61ce66700 1 --2- 172.21.15.42:0/3887844307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6180f6860 0x7fe6180f7ba0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:27:44.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.420+0000 7fe61ce66700 1 -- 172.21.15.42:0/3887844307 >> 172.21.15.42:0/3887844307 conn(0x7fe6180f0d40 msgr2=0x7fe61805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:27:44.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.420+0000 7fe61ce66700 1 -- 172.21.15.42:0/3887844307 shutdown_connections 2024-02-15T18:27:44.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:27:44.420+0000 7fe61ce66700 1 -- 172.21.15.42:0/3887844307 wait complete. 2024-02-15T18:27:44.558 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:43.275621+0000 mgr.smithi042.kfoihb (mgr.24435) 73 : audit [DBG] from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:44.559 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:43.693586+0000 mon.smithi191 (mon.1) 2 : audit [DBG] from='client.? 172.21.15.42:0/3223554822' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:44.559 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:44.051171+0000 mon.smithi042 (mon.0) 42 : audit [DBG] from='client.? 172.21.15.42:0/4211935573' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:27:44.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:43.275621+0000 mgr.smithi042.kfoihb (mgr.24435) 73 : audit [DBG] from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:44.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:43.693586+0000 mon.smithi191 (mon.1) 2 : audit [DBG] from='client.? 172.21.15.42:0/3223554822' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:44.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:44.051171+0000 mon.smithi042 (mon.0) 42 : audit [DBG] from='client.? 172.21.15.42:0/4211935573' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:27:45.664 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:44.339930+0000 mon.smithi042 (mon.0) 43 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:45.664 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:44.419410+0000 mgr.smithi042.kfoihb (mgr.24435) 74 : audit [DBG] from='client.44119 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:45.664 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:44.691373+0000 mgr.smithi042.kfoihb (mgr.24435) 75 : cluster [DBG] pgmap v51: 65 pgs: 65 active+clean; 1.5 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 906 KiB/s rd, 1.0 MiB/s wr, 180 op/s 2024-02-15T18:27:45.808 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:44.339930+0000 mon.smithi042 (mon.0) 43 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:45.808 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:44.419410+0000 mgr.smithi042.kfoihb (mgr.24435) 74 : audit [DBG] from='client.44119 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:27:45.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:44.691373+0000 mgr.smithi042.kfoihb (mgr.24435) 75 : cluster [DBG] pgmap v51: 65 pgs: 65 active+clean; 1.5 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 906 KiB/s rd, 1.0 MiB/s wr, 180 op/s 2024-02-15T18:27:47.445 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:27:46.151676+0000 mgr.smithi042.kfoihb (mgr.24435) 76 : cephadm [INF] Upgrade: Updating crash.smithi191 (2/2) 2024-02-15T18:27:47.445 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:46.157935+0000 mon.smithi042 (mon.0) 44 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:47.446 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:46.159051+0000 mon.smithi042 (mon.0) 45 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi191", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:27:47.446 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:46.159978+0000 mon.smithi042 (mon.0) 46 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:27:47.446 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:27:46.160948+0000 mgr.smithi042.kfoihb (mgr.24435) 77 : cephadm [INF] Deploying daemon crash.smithi191 on smithi191 2024-02-15T18:27:47.446 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:46.691672+0000 mgr.smithi042.kfoihb (mgr.24435) 78 : cluster [DBG] pgmap v52: 65 pgs: 65 active+clean; 1.5 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 906 KiB/s rd, 1.0 MiB/s wr, 180 op/s 2024-02-15T18:27:47.558 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:27:46.151676+0000 mgr.smithi042.kfoihb (mgr.24435) 76 : cephadm [INF] Upgrade: Updating crash.smithi191 (2/2) 2024-02-15T18:27:47.558 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:46.157935+0000 mon.smithi042 (mon.0) 44 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:47.559 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:46.159051+0000 mon.smithi042 (mon.0) 45 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi191", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-15T18:27:47.559 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:46.159978+0000 mon.smithi042 (mon.0) 46 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:27:47.559 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:27:46.160948+0000 mgr.smithi042.kfoihb (mgr.24435) 77 : cephadm [INF] Deploying daemon crash.smithi191 on smithi191 2024-02-15T18:27:47.559 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:46.691672+0000 mgr.smithi042.kfoihb (mgr.24435) 78 : cluster [DBG] pgmap v52: 65 pgs: 65 active+clean; 1.5 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 906 KiB/s rd, 1.0 MiB/s wr, 180 op/s 2024-02-15T18:27:49.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:48.692252+0000 mgr.smithi042.kfoihb (mgr.24435) 79 : cluster [DBG] pgmap v53: 65 pgs: 65 active+clean; 895 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 285 op/s 2024-02-15T18:27:50.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:48.692252+0000 mgr.smithi042.kfoihb (mgr.24435) 79 : cluster [DBG] pgmap v53: 65 pgs: 65 active+clean; 895 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 285 op/s 2024-02-15T18:27:51.055 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-15T18:27:51.055 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-02-15T18:27:51.808 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:50.499725+0000 mon.smithi042 (mon.0) 47 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:51.808 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:50.500506+0000 mon.smithi042 (mon.0) 48 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:27:51.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:50.692609+0000 mgr.smithi042.kfoihb (mgr.24435) 80 : cluster [DBG] pgmap v54: 65 pgs: 65 active+clean; 895 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-02-15T18:27:51.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:50.748727+0000 mon.smithi042 (mon.0) 49 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:51.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:50.499725+0000 mon.smithi042 (mon.0) 47 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:51.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:50.500506+0000 mon.smithi042 (mon.0) 48 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:27:51.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:50.692609+0000 mgr.smithi042.kfoihb (mgr.24435) 80 : cluster [DBG] pgmap v54: 65 pgs: 65 active+clean; 895 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 229 op/s 2024-02-15T18:27:51.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:50.748727+0000 mon.smithi042 (mon.0) 49 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:54.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:52.693054+0000 mgr.smithi042.kfoihb (mgr.24435) 81 : cluster [DBG] pgmap v55: 65 pgs: 65 active+clean; 336 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 371 op/s 2024-02-15T18:27:54.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:52.693054+0000 mgr.smithi042.kfoihb (mgr.24435) 81 : cluster [DBG] pgmap v55: 65 pgs: 65 active+clean; 336 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 371 op/s 2024-02-15T18:27:56.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:54.693345+0000 mgr.smithi042.kfoihb (mgr.24435) 82 : cluster [DBG] pgmap v56: 65 pgs: 65 active+clean; 336 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-02-15T18:27:56.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:54.693345+0000 mgr.smithi042.kfoihb (mgr.24435) 82 : cluster [DBG] pgmap v56: 65 pgs: 65 active+clean; 336 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-02-15T18:27:58.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:56.693644+0000 mgr.smithi042.kfoihb (mgr.24435) 83 : cluster [DBG] pgmap v57: 65 pgs: 65 active+clean; 336 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-02-15T18:27:58.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:56.736087+0000 mon.smithi042 (mon.0) 50 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:58.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.618834+0000 mon.smithi042 (mon.0) 51 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:58.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.619756+0000 mon.smithi042 (mon.0) 52 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:27:58.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.620638+0000 mon.smithi042 (mon.0) 53 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:27:58.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.624975+0000 mon.smithi042 (mon.0) 54 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:58.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.630926+0000 mon.smithi042 (mon.0) 55 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:27:58.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.633534+0000 mon.smithi042 (mon.0) 56 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:58.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.635026+0000 mon.smithi042 (mon.0) 57 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:58.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.636349+0000 mon.smithi042 (mon.0) 58 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:58.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.640545+0000 mon.smithi042 (mon.0) 59 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:58.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.641403+0000 mon.smithi042 (mon.0) 60 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi042"}]: dispatch 2024-02-15T18:27:58.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.645382+0000 mon.smithi042 (mon.0) 61 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi042"}]': finished 2024-02-15T18:27:58.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.646059+0000 mon.smithi042 (mon.0) 62 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi191"}]: dispatch 2024-02-15T18:27:58.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.650203+0000 mon.smithi042 (mon.0) 63 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi191"}]': finished 2024-02-15T18:27:58.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.651456+0000 mon.smithi042 (mon.0) 64 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-15T18:27:58.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:56.693644+0000 mgr.smithi042.kfoihb (mgr.24435) 83 : cluster [DBG] pgmap v57: 65 pgs: 65 active+clean; 336 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 247 op/s 2024-02-15T18:27:58.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:56.736087+0000 mon.smithi042 (mon.0) 50 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:58.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.618834+0000 mon.smithi042 (mon.0) 51 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:58.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.619756+0000 mon.smithi042 (mon.0) 52 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:27:58.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.620638+0000 mon.smithi042 (mon.0) 53 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:27:58.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.624975+0000 mon.smithi042 (mon.0) 54 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:58.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.630926+0000 mon.smithi042 (mon.0) 55 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:27:58.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.633534+0000 mon.smithi042 (mon.0) 56 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:58.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.635026+0000 mon.smithi042 (mon.0) 57 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:58.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.636349+0000 mon.smithi042 (mon.0) 58 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:27:58.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.640545+0000 mon.smithi042 (mon.0) 59 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:27:58.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.641403+0000 mon.smithi042 (mon.0) 60 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi042"}]: dispatch 2024-02-15T18:27:58.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.645382+0000 mon.smithi042 (mon.0) 61 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi042"}]': finished 2024-02-15T18:27:58.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.646059+0000 mon.smithi042 (mon.0) 62 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi191"}]: dispatch 2024-02-15T18:27:58.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.650203+0000 mon.smithi042 (mon.0) 63 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi191"}]': finished 2024-02-15T18:27:58.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.651456+0000 mon.smithi042 (mon.0) 64 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-15T18:27:59.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:27:57.621359+0000 mgr.smithi042.kfoihb (mgr.24435) 84 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:27:59.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:27:57.637026+0000 mgr.smithi042.kfoihb (mgr.24435) 85 : cephadm [INF] Upgrade: Setting container_image for all crash 2024-02-15T18:27:59.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:57.651622+0000 mgr.smithi042.kfoihb (mgr.24435) 86 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-15T18:27:59.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:27:57.652479+0000 mgr.smithi042.kfoihb (mgr.24435) 87 : cephadm [INF] Upgrade: osd.0 is safe to restart 2024-02-15T18:27:59.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:27:57.621359+0000 mgr.smithi042.kfoihb (mgr.24435) 84 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:27:59.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:27:57.637026+0000 mgr.smithi042.kfoihb (mgr.24435) 85 : cephadm [INF] Upgrade: Setting container_image for all crash 2024-02-15T18:27:59.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:57.651622+0000 mgr.smithi042.kfoihb (mgr.24435) 86 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-15T18:27:59.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:27:57.652479+0000 mgr.smithi042.kfoihb (mgr.24435) 87 : cephadm [INF] Upgrade: osd.0 is safe to restart 2024-02-15T18:28:00.020 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:27:58.694220+0000 mgr.smithi042.kfoihb (mgr.24435) 88 : cluster [DBG] pgmap v58: 65 pgs: 65 active+clean; 294 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 287 op/s 2024-02-15T18:28:00.020 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:59.294313+0000 mon.smithi042 (mon.0) 65 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:28:00.020 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:59.295072+0000 mon.smithi042 (mon.0) 66 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-15T18:28:00.020 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:27:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:27:59.295692+0000 mon.smithi042 (mon.0) 67 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:00.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:27:58.694220+0000 mgr.smithi042.kfoihb (mgr.24435) 88 : cluster [DBG] pgmap v58: 65 pgs: 65 active+clean; 294 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 287 op/s 2024-02-15T18:28:00.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:59.294313+0000 mon.smithi042 (mon.0) 65 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:28:00.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:59.295072+0000 mon.smithi042 (mon.0) 66 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-15T18:28:00.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:27:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:27:59.295692+0000 mon.smithi042 (mon.0) 67 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:01.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:27:59.289393+0000 mgr.smithi042.kfoihb (mgr.24435) 89 : cephadm [INF] Upgrade: Updating osd.0 2024-02-15T18:28:01.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:27:59.296753+0000 mgr.smithi042.kfoihb (mgr.24435) 90 : cephadm [INF] Deploying daemon osd.0 on smithi042 2024-02-15T18:28:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:27:59.289393+0000 mgr.smithi042.kfoihb (mgr.24435) 89 : cephadm [INF] Upgrade: Updating osd.0 2024-02-15T18:28:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:27:59.296753+0000 mgr.smithi042.kfoihb (mgr.24435) 90 : cephadm [INF] Deploying daemon osd.0 on smithi042 2024-02-15T18:28:02.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:00.694779+0000 mgr.smithi042.kfoihb (mgr.24435) 91 : cluster [DBG] pgmap v59: 65 pgs: 65 active+clean; 294 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 182 op/s 2024-02-15T18:28:02.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:00.694779+0000 mgr.smithi042.kfoihb (mgr.24435) 91 : cluster [DBG] pgmap v59: 65 pgs: 65 active+clean; 294 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 182 op/s 2024-02-15T18:28:02.808 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:02 smithi042 systemd[1]: Stopping Ceph osd.0 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:28:03.230 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.0[48500]: debug 2024-02-15T18:28:02.845+0000 7f3e673cd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:28:03.230 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.0[48500]: debug 2024-02-15T18:28:02.845+0000 7f3e673cd700 -1 osd.0 48 *** Got signal Terminated *** 2024-02-15T18:28:03.230 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.0[48500]: debug 2024-02-15T18:28:02.845+0000 7f3e673cd700 -1 osd.0 48 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:28:03.507 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:03 smithi042 bash[80415]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.0 2024-02-15T18:28:03.808 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:03 smithi042 bash[80467]: Error: no container with name or ID "ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:02.695230+0000 mgr.smithi042.kfoihb (mgr.24435) 92 : cluster [DBG] pgmap v60: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 239 op/s 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.093342+0000 mon.smithi042 (mon.0) 68 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.093368+0000 mon.smithi042 (mon.0) 69 : cluster [INF] osd.0 failed (root=default,host=smithi042) (connection refused reported by osd.1) 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.093484+0000 mon.smithi042 (mon.0) 70 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.093582+0000 mon.smithi042 (mon.0) 71 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.093661+0000 mon.smithi042 (mon.0) 72 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.094078+0000 mon.smithi042 (mon.0) 73 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.094199+0000 mon.smithi042 (mon.0) 74 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.094242+0000 mon.smithi042 (mon.0) 75 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.094340+0000 mon.smithi042 (mon.0) 76 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.094408+0000 mon.smithi042 (mon.0) 77 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.094474+0000 mon.smithi042 (mon.0) 78 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.385109+0000 mon.smithi042 (mon.0) 79 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.494276+0000 mon.smithi042 (mon.0) 80 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:28:03.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.495290+0000 mon.smithi042 (mon.0) 81 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:28:03.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.495397+0000 mon.smithi042 (mon.0) 82 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:28:03.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.495452+0000 mon.smithi042 (mon.0) 83 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:28:03.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.495562+0000 mon.smithi042 (mon.0) 84 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:28:03.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.495613+0000 mon.smithi042 (mon.0) 85 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:28:03.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.495719+0000 mon.smithi042 (mon.0) 86 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:28:03.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.495801+0000 mon.smithi042 (mon.0) 87 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:28:03.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.495852+0000 mon.smithi042 (mon.0) 88 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:28:03.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.495954+0000 mon.smithi042 (mon.0) 89 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:28:03.810 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.585317+0000 mon.smithi042 (mon.0) 90 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:28:04.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:02.695230+0000 mgr.smithi042.kfoihb (mgr.24435) 92 : cluster [DBG] pgmap v60: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 239 op/s 2024-02-15T18:28:04.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.093342+0000 mon.smithi042 (mon.0) 68 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:28:04.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.093368+0000 mon.smithi042 (mon.0) 69 : cluster [INF] osd.0 failed (root=default,host=smithi042) (connection refused reported by osd.1) 2024-02-15T18:28:04.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.093484+0000 mon.smithi042 (mon.0) 70 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:28:04.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.093582+0000 mon.smithi042 (mon.0) 71 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:28:04.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.093661+0000 mon.smithi042 (mon.0) 72 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:28:04.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.094078+0000 mon.smithi042 (mon.0) 73 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:28:04.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.094199+0000 mon.smithi042 (mon.0) 74 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:28:04.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.094242+0000 mon.smithi042 (mon.0) 75 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:28:04.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.094340+0000 mon.smithi042 (mon.0) 76 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:28:04.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.094408+0000 mon.smithi042 (mon.0) 77 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:28:04.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.094474+0000 mon.smithi042 (mon.0) 78 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:28:04.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.385109+0000 mon.smithi042 (mon.0) 79 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:28:04.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.494276+0000 mon.smithi042 (mon.0) 80 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:28:04.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.495290+0000 mon.smithi042 (mon.0) 81 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:28:04.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.495397+0000 mon.smithi042 (mon.0) 82 : cluster [DBG] osd.0 reported immediately failed by osd.5 2024-02-15T18:28:04.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.495452+0000 mon.smithi042 (mon.0) 83 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:28:04.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.495562+0000 mon.smithi042 (mon.0) 84 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:28:04.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.495613+0000 mon.smithi042 (mon.0) 85 : cluster [DBG] osd.0 reported immediately failed by osd.4 2024-02-15T18:28:04.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.495719+0000 mon.smithi042 (mon.0) 86 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:28:04.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.495801+0000 mon.smithi042 (mon.0) 87 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:28:04.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.495852+0000 mon.smithi042 (mon.0) 88 : cluster [DBG] osd.0 reported immediately failed by osd.1 2024-02-15T18:28:04.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.495954+0000 mon.smithi042 (mon.0) 89 : cluster [DBG] osd.0 reported immediately failed by osd.3 2024-02-15T18:28:04.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.585317+0000 mon.smithi042 (mon.0) 90 : cluster [DBG] osd.0 reported immediately failed by osd.2 2024-02-15T18:28:05.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.748088+0000 mon.smithi042 (mon.0) 91 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:28:05.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:03.751833+0000 mon.smithi042 (mon.0) 92 : cluster [DBG] osdmap e49: 6 total, 5 up, 6 in 2024-02-15T18:28:05.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:03.752443+0000 mon.smithi042 (mon.0) 93 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-15T18:28:05.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:03.754184+0000 mon.smithi042 (mon.0) 94 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:28:05.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:04.752300+0000 mon.smithi042 (mon.0) 95 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-15T18:28:05.060 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:04.752443+0000 mon.smithi042 (mon.0) 96 : cluster [DBG] osdmap e50: 6 total, 5 up, 6 in 2024-02-15T18:28:05.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.748088+0000 mon.smithi042 (mon.0) 91 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:28:05.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:03.751833+0000 mon.smithi042 (mon.0) 92 : cluster [DBG] osdmap e49: 6 total, 5 up, 6 in 2024-02-15T18:28:05.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:03.752443+0000 mon.smithi042 (mon.0) 93 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-15T18:28:05.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:03.754184+0000 mon.smithi042 (mon.0) 94 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-15T18:28:05.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:04.752300+0000 mon.smithi042 (mon.0) 95 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-15T18:28:05.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:04.752443+0000 mon.smithi042 (mon.0) 96 : cluster [DBG] osdmap e50: 6 total, 5 up, 6 in 2024-02-15T18:28:06.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:04.695571+0000 mgr.smithi042.kfoihb (mgr.24435) 93 : cluster [DBG] pgmap v62: 65 pgs: 9 stale+active+clean, 56 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 117 op/s 2024-02-15T18:28:06.058 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:05 smithi042 systemd[1]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.0.service: Succeeded. 2024-02-15T18:28:06.059 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:05 smithi042 systemd[1]: Stopped Ceph osd.0 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:28:06.059 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:05 smithi042 systemd[1]: Starting Ceph osd.0 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:28:06.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:04.695571+0000 mgr.smithi042.kfoihb (mgr.24435) 93 : cluster [DBG] pgmap v62: 65 pgs: 9 stale+active+clean, 56 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 117 op/s 2024-02-15T18:28:07.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:06.695935+0000 mgr.smithi042.kfoihb (mgr.24435) 94 : cluster [DBG] pgmap v64: 65 pgs: 9 stale+active+clean, 56 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 85 op/s 2024-02-15T18:28:07.058 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:06 smithi042 bash[80727]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-02-15T18:28:07.059 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:06 smithi042 bash[80727]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-15T18:28:07.059 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:06 smithi042 bash[80727]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2024-02-15T18:28:07.059 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:06 smithi042 bash[80727]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-15T18:28:07.059 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:06 smithi042 bash[80727]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block 2024-02-15T18:28:07.059 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:06 smithi042 bash[80727]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-15T18:28:07.059 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:06 smithi042 bash[80727]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-15T18:28:07.059 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:06 smithi042 bash[80727]: --> ceph-volume lvm activate successful for osd ID: 0 2024-02-15T18:28:07.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:06.695935+0000 mgr.smithi042.kfoihb (mgr.24435) 94 : cluster [DBG] pgmap v64: 65 pgs: 9 stale+active+clean, 56 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 85 op/s 2024-02-15T18:28:08.558 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-0[80976]: debug 2024-02-15T18:28:08.064+0000 7f7bf8429700 1 -- 172.21.15.42:0/1328499718 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55769b73c680 con 0x55769b703000 2024-02-15T18:28:08.559 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:08 smithi042 bash[80962]: f226bea76cab65d489fa3c56d56107042bb700f8f813374b7e0006e35dd5fbd2 2024-02-15T18:28:08.559 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:08 smithi042 systemd[1]: Started Ceph osd.0 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:28:08.929 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-0[80976]: debug 2024-02-15T18:28:08.587+0000 7f7bffeb7380 -1 Falling back to public interface 2024-02-15T18:28:09.445 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:08.193045+0000 mon.smithi042 (mon.0) 97 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:28:09.445 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:08.193818+0000 mon.smithi042 (mon.0) 98 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:28:09.445 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:08.696661+0000 mgr.smithi042.kfoihb (mgr.24435) 95 : cluster [DBG] pgmap v65: 65 pgs: 32 active+undersized+degraded, 2 active+undersized, 31 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 333 op/s; 7133/48168 objects degraded (14.809%) 2024-02-15T18:28:09.558 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:08.193045+0000 mon.smithi042 (mon.0) 97 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:28:09.558 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:08.193818+0000 mon.smithi042 (mon.0) 98 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:28:09.558 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:08.696661+0000 mgr.smithi042.kfoihb (mgr.24435) 95 : cluster [DBG] pgmap v65: 65 pgs: 32 active+undersized+degraded, 2 active+undersized, 31 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 333 op/s; 7133/48168 objects degraded (14.809%) 2024-02-15T18:28:10.445 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:09.191480+0000 mon.smithi042 (mon.0) 99 : cluster [WRN] Health check failed: Degraded data redundancy: 7133/48168 objects degraded (14.809%), 32 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:10.558 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:09.191480+0000 mon.smithi042 (mon.0) 99 : cluster [WRN] Health check failed: Degraded data redundancy: 7133/48168 objects degraded (14.809%), 32 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:12.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:10.697192+0000 mgr.smithi042.kfoihb (mgr.24435) 96 : cluster [DBG] pgmap v66: 65 pgs: 32 active+undersized+degraded, 2 active+undersized, 31 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 446 KiB/s rd, 1.2 MiB/s wr, 247 op/s; 7133/48168 objects degraded (14.809%) 2024-02-15T18:28:12.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:10.832722+0000 mon.smithi042 (mon.0) 100 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:28:12.302 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:10.697192+0000 mgr.smithi042.kfoihb (mgr.24435) 96 : cluster [DBG] pgmap v66: 65 pgs: 32 active+undersized+degraded, 2 active+undersized, 31 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 446 KiB/s rd, 1.2 MiB/s wr, 247 op/s; 7133/48168 objects degraded (14.809%) 2024-02-15T18:28:12.302 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:10.832722+0000 mon.smithi042 (mon.0) 100 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:28:14.140 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:12.697808+0000 mgr.smithi042.kfoihb (mgr.24435) 97 : cluster [DBG] pgmap v67: 65 pgs: 32 active+undersized+degraded, 2 active+undersized, 31 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 457 op/s; 6342/42774 objects degraded (14.827%) 2024-02-15T18:28:14.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:12.697808+0000 mgr.smithi042.kfoihb (mgr.24435) 97 : cluster [DBG] pgmap v67: 65 pgs: 32 active+undersized+degraded, 2 active+undersized, 31 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 457 op/s; 6342/42774 objects degraded (14.827%) 2024-02-15T18:28:14.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.536+0000 7fb740b91700 1 -- 172.21.15.42:0/2686769509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb73c0f66d0 msgr2=0x7fb73c0f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:14.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.536+0000 7fb740b91700 1 --2- 172.21.15.42:0/2686769509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb73c0f66d0 0x7fb73c0f6b30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fb724004660 tx=0x7fb724009b30).stop 2024-02-15T18:28:14.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.536+0000 7fb740b91700 1 -- 172.21.15.42:0/2686769509 shutdown_connections 2024-02-15T18:28:14.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.536+0000 7fb740b91700 1 --2- 172.21.15.42:0/2686769509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb73c0f66d0 0x7fb73c0f6b30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:14.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.536+0000 7fb740b91700 1 --2- 172.21.15.42:0/2686769509 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb73c0f5430 0x7fb73c0f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:14.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.536+0000 7fb740b91700 1 -- 172.21.15.42:0/2686769509 >> 172.21.15.42:0/2686769509 conn(0x7fb73c0f0b90 msgr2=0x7fb73c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:14.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.536+0000 7fb740b91700 1 -- 172.21.15.42:0/2686769509 shutdown_connections 2024-02-15T18:28:14.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.536+0000 7fb740b91700 1 -- 172.21.15.42:0/2686769509 wait complete. 2024-02-15T18:28:14.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.537+0000 7fb740b91700 1 Processor -- start 2024-02-15T18:28:14.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.537+0000 7fb740b91700 1 -- start start 2024-02-15T18:28:14.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.537+0000 7fb740b91700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb73c0f5430 0x7fb73c0f9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:14.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.537+0000 7fb740b91700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb73c0f66d0 0x7fb73c0f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:14.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.537+0000 7fb740b91700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c0f7f70 con 0x7fb73c0f66d0 2024-02-15T18:28:14.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.537+0000 7fb740b91700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c0f80b0 con 0x7fb73c0f5430 2024-02-15T18:28:14.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.537+0000 7fb733fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb73c0f66d0 0x7fb73c0f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:14.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.537+0000 7fb733fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb73c0f66d0 0x7fb73c0f7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59152/0 (socket says 172.21.15.42:59152) 2024-02-15T18:28:14.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.537+0000 7fb733fff700 1 -- 172.21.15.42:0/4199797278 learned_addr learned my addr 172.21.15.42:0/4199797278 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:28:14.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.538+0000 7fb733fff700 1 -- 172.21.15.42:0/4199797278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb73c0f5430 msgr2=0x7fb73c0f9590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-15T18:28:14.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.538+0000 7fb733fff700 1 --2- 172.21.15.42:0/4199797278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb73c0f5430 0x7fb73c0f9590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:28:14.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.538+0000 7fb733fff700 1 -- 172.21.15.42:0/4199797278 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb724005040 con 0x7fb73c0f66d0 2024-02-15T18:28:14.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.538+0000 7fb733fff700 1 --2- 172.21.15.42:0/4199797278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb73c0f66d0 0x7fb73c0f7a10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb724000c00 tx=0x7fb72400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:14.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.538+0000 7fb738ff9700 1 -- 172.21.15.42:0/4199797278 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb724019070 con 0x7fb73c0f66d0 2024-02-15T18:28:14.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.538+0000 7fb740b91700 1 -- 172.21.15.42:0/4199797278 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73c0f8330 con 0x7fb73c0f66d0 2024-02-15T18:28:14.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.538+0000 7fb740b91700 1 -- 172.21.15.42:0/4199797278 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73c0f8890 con 0x7fb73c0f66d0 2024-02-15T18:28:14.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.538+0000 7fb738ff9700 1 -- 172.21.15.42:0/4199797278 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fb724014420 con 0x7fb73c0f66d0 2024-02-15T18:28:14.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.538+0000 7fb738ff9700 1 -- 172.21.15.42:0/4199797278 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb72401d600 con 0x7fb73c0f66d0 2024-02-15T18:28:14.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.540+0000 7fb740b91700 1 -- 172.21.15.42:0/4199797278 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb720004fa0 con 0x7fb73c0f66d0 2024-02-15T18:28:14.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.540+0000 7fb738ff9700 1 -- 172.21.15.42:0/4199797278 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fb724025070 con 0x7fb73c0f66d0 2024-02-15T18:28:14.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.540+0000 7fb738ff9700 1 --2- 172.21.15.42:0/4199797278 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb728060f70 0x7fb728063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:14.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.540+0000 7fb738ff9700 1 -- 172.21.15.42:0/4199797278 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fb72407cfb0 con 0x7fb73c0f66d0 2024-02-15T18:28:14.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.540+0000 7fb73b7fe700 1 --2- 172.21.15.42:0/4199797278 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb728060f70 0x7fb728063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:14.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.541+0000 7fb73b7fe700 1 --2- 172.21.15.42:0/4199797278 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb728060f70 0x7fb728063430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fb73c0f9320 tx=0x7fb72c00b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:14.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.546+0000 7fb738ff9700 1 -- 172.21.15.42:0/4199797278 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb72404d600 con 0x7fb73c0f66d0 2024-02-15T18:28:14.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.897+0000 7fb740b91700 1 -- 172.21.15.42:0/4199797278 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb720000bc0 con 0x7fb728060f70 2024-02-15T18:28:14.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.910+0000 7fb738ff9700 1 -- 172.21.15.42:0/4199797278 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb720000bc0 con 0x7fb728060f70 2024-02-15T18:28:14.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.917+0000 7fb740b91700 1 -- 172.21.15.42:0/4199797278 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb728060f70 msgr2=0x7fb728063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:14.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.917+0000 7fb740b91700 1 --2- 172.21.15.42:0/4199797278 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb728060f70 0x7fb728063430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fb73c0f9320 tx=0x7fb72c00b040).stop 2024-02-15T18:28:14.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.917+0000 7fb740b91700 1 -- 172.21.15.42:0/4199797278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb73c0f66d0 msgr2=0x7fb73c0f7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:14.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.917+0000 7fb740b91700 1 --2- 172.21.15.42:0/4199797278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb73c0f66d0 0x7fb73c0f7a10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb724000c00 tx=0x7fb72400b040).stop 2024-02-15T18:28:14.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.918+0000 7fb740b91700 1 -- 172.21.15.42:0/4199797278 shutdown_connections 2024-02-15T18:28:14.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.918+0000 7fb740b91700 1 --2- 172.21.15.42:0/4199797278 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb728060f70 0x7fb728063430 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0x7fb73c0f9320 tx=0x7fb72c00b040).stop 2024-02-15T18:28:14.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.918+0000 7fb740b91700 1 --2- 172.21.15.42:0/4199797278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb73c0f5430 0x7fb73c0f9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:28:14.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.918+0000 7fb740b91700 1 --2- 172.21.15.42:0/4199797278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb73c0f66d0 0x7fb73c0f7a10 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0x7fb724000c00 tx=0x7fb72400b040).stop 2024-02-15T18:28:14.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.918+0000 7fb740b91700 1 -- 172.21.15.42:0/4199797278 >> 172.21.15.42:0/4199797278 conn(0x7fb73c0f0b90 msgr2=0x7fb73c05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:14.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.921+0000 7fb740b91700 1 -- 172.21.15.42:0/4199797278 shutdown_connections 2024-02-15T18:28:14.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:14.921+0000 7fb740b91700 1 -- 172.21.15.42:0/4199797278 wait complete. 2024-02-15T18:28:14.928 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:28:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.029+0000 7f0e938cf700 1 -- 172.21.15.42:0/243258410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e8c0f55c0 msgr2=0x7f0e8c0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.029+0000 7f0e938cf700 1 --2- 172.21.15.42:0/243258410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e8c0f55c0 0x7f0e8c0f59e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f0e88004660 tx=0x7f0e88009b30).stop 2024-02-15T18:28:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.029+0000 7f0e938cf700 1 -- 172.21.15.42:0/243258410 shutdown_connections 2024-02-15T18:28:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.029+0000 7f0e938cf700 1 --2- 172.21.15.42:0/243258410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e8c0f6860 0x7f0e8c0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.029+0000 7f0e938cf700 1 --2- 172.21.15.42:0/243258410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e8c0f55c0 0x7f0e8c0f59e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.029+0000 7f0e938cf700 1 -- 172.21.15.42:0/243258410 >> 172.21.15.42:0/243258410 conn(0x7f0e8c0f0d40 msgr2=0x7f0e8c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.029+0000 7f0e938cf700 1 -- 172.21.15.42:0/243258410 shutdown_connections 2024-02-15T18:28:15.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.029+0000 7f0e938cf700 1 -- 172.21.15.42:0/243258410 wait complete. 2024-02-15T18:28:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.030+0000 7f0e938cf700 1 Processor -- start 2024-02-15T18:28:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.030+0000 7f0e938cf700 1 -- start start 2024-02-15T18:28:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.030+0000 7f0e938cf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e8c0f55c0 0x7f0e8c0fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.030+0000 7f0e938cf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e8c0f6860 0x7f0e8c0fca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.030+0000 7f0e938cf700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0fcfb0 con 0x7f0e8c0f6860 2024-02-15T18:28:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.030+0000 7f0e938cf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0fd0f0 con 0x7f0e8c0f55c0 2024-02-15T18:28:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.030+0000 7f0e90e6a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e8c0f6860 0x7f0e8c0fca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.030+0000 7f0e90e6a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e8c0f6860 0x7f0e8c0fca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59166/0 (socket says 172.21.15.42:59166) 2024-02-15T18:28:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.030+0000 7f0e90e6a700 1 -- 172.21.15.42:0/1521810579 learned_addr learned my addr 172.21.15.42:0/1521810579 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:28:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.030+0000 7f0e9166b700 1 --2- 172.21.15.42:0/1521810579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e8c0f55c0 0x7f0e8c0fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.031+0000 7f0e90e6a700 1 -- 172.21.15.42:0/1521810579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e8c0f55c0 msgr2=0x7f0e8c0fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.031+0000 7f0e90e6a700 1 --2- 172.21.15.42:0/1521810579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e8c0f55c0 0x7f0e8c0fe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.031+0000 7f0e90e6a700 1 -- 172.21.15.42:0/1521810579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e88005040 con 0x7f0e8c0f6860 2024-02-15T18:28:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.031+0000 7f0e90e6a700 1 --2- 172.21.15.42:0/1521810579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e8c0f6860 0x7f0e8c0fca50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f0e7c00e7f0 tx=0x7f0e7c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.031+0000 7f0e81ffb700 1 -- 172.21.15.42:0/1521810579 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e7c0092e0 con 0x7f0e8c0f6860 2024-02-15T18:28:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.031+0000 7f0e81ffb700 1 -- 172.21.15.42:0/1521810579 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f0e7c00f040 con 0x7f0e8c0f6860 2024-02-15T18:28:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.031+0000 7f0e81ffb700 1 -- 172.21.15.42:0/1521810579 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e7c014850 con 0x7f0e8c0f6860 2024-02-15T18:28:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.032+0000 7f0e938cf700 1 -- 172.21.15.42:0/1521810579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8c0fd3d0 con 0x7f0e8c0f6860 2024-02-15T18:28:15.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.032+0000 7f0e938cf700 1 -- 172.21.15.42:0/1521810579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8c10eff0 con 0x7f0e8c0f6860 2024-02-15T18:28:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.033+0000 7f0e938cf700 1 -- 172.21.15.42:0/1521810579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8c0f49b0 con 0x7f0e8c0f6860 2024-02-15T18:28:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.033+0000 7f0e81ffb700 1 -- 172.21.15.42:0/1521810579 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f0e7c0149b0 con 0x7f0e8c0f6860 2024-02-15T18:28:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.034+0000 7f0e81ffb700 1 --2- 172.21.15.42:0/1521810579 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0e78060f20 0x7f0e780633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:15.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.034+0000 7f0e81ffb700 1 -- 172.21.15.42:0/1521810579 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f0e7c07c3a0 con 0x7f0e8c0f6860 2024-02-15T18:28:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.034+0000 7f0e9166b700 1 --2- 172.21.15.42:0/1521810579 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0e78060f20 0x7f0e780633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:15.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.034+0000 7f0e9166b700 1 --2- 172.21.15.42:0/1521810579 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0e78060f20 0x7f0e780633e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f0e88004d80 tx=0x7f0e88016040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:15.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.039+0000 7f0e81ffb700 1 -- 172.21.15.42:0/1521810579 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0e7c04c7c0 con 0x7f0e8c0f6860 2024-02-15T18:28:15.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.233+0000 7f0e938cf700 1 -- 172.21.15.42:0/1521810579 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e8c0ff080 con 0x7f0e78060f20 2024-02-15T18:28:15.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.234+0000 7f0e81ffb700 1 -- 172.21.15.42:0/1521810579 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0e8c0ff080 con 0x7f0e78060f20 2024-02-15T18:28:15.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.236+0000 7f0e938cf700 1 -- 172.21.15.42:0/1521810579 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0e78060f20 msgr2=0x7f0e780633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:15.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.236+0000 7f0e938cf700 1 --2- 172.21.15.42:0/1521810579 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0e78060f20 0x7f0e780633e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f0e88004d80 tx=0x7f0e88016040).stop 2024-02-15T18:28:15.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.236+0000 7f0e938cf700 1 -- 172.21.15.42:0/1521810579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e8c0f6860 msgr2=0x7f0e8c0fca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:15.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.236+0000 7f0e938cf700 1 --2- 172.21.15.42:0/1521810579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e8c0f6860 0x7f0e8c0fca50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f0e7c00e7f0 tx=0x7f0e7c00b040).stop 2024-02-15T18:28:15.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.236+0000 7f0e938cf700 1 -- 172.21.15.42:0/1521810579 shutdown_connections 2024-02-15T18:28:15.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.236+0000 7f0e938cf700 1 --2- 172.21.15.42:0/1521810579 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0e78060f20 0x7f0e780633e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.236+0000 7f0e938cf700 1 --2- 172.21.15.42:0/1521810579 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e8c0f55c0 0x7f0e8c0fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.236+0000 7f0e938cf700 1 --2- 172.21.15.42:0/1521810579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e8c0f6860 0x7f0e8c0fca50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.236+0000 7f0e938cf700 1 -- 172.21.15.42:0/1521810579 >> 172.21.15.42:0/1521810579 conn(0x7f0e8c0f0d40 msgr2=0x7f0e8c0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:15.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.236+0000 7f0e938cf700 1 -- 172.21.15.42:0/1521810579 shutdown_connections 2024-02-15T18:28:15.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.236+0000 7f0e938cf700 1 -- 172.21.15.42:0/1521810579 wait complete. 2024-02-15T18:28:15.308 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-0[80976]: debug 2024-02-15T18:28:15.050+0000 7f7bffeb7380 -1 osd.0 48 log_to_monitors {default=true} 2024-02-15T18:28:15.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.340+0000 7f603912b700 1 -- 172.21.15.42:0/3191074962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340f6ff0 msgr2=0x7f60340f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:15.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.340+0000 7f603912b700 1 --2- 172.21.15.42:0/3191074962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340f6ff0 0x7f60340f7a20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f6028004660 tx=0x7f6028009b30).stop 2024-02-15T18:28:15.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.340+0000 7f603912b700 1 -- 172.21.15.42:0/3191074962 shutdown_connections 2024-02-15T18:28:15.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.340+0000 7f603912b700 1 --2- 172.21.15.42:0/3191074962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340f6ff0 0x7f60340f7a20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.340+0000 7f603912b700 1 --2- 172.21.15.42:0/3191074962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60340f66e0 0x7f60340f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.340+0000 7f603912b700 1 -- 172.21.15.42:0/3191074962 >> 172.21.15.42:0/3191074962 conn(0x7f60340f1f00 msgr2=0x7f60340f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:15.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.341+0000 7f603912b700 1 -- 172.21.15.42:0/3191074962 shutdown_connections 2024-02-15T18:28:15.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.341+0000 7f603912b700 1 -- 172.21.15.42:0/3191074962 wait complete. 2024-02-15T18:28:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.341+0000 7f603912b700 1 Processor -- start 2024-02-15T18:28:15.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.342+0000 7f603912b700 1 -- start start 2024-02-15T18:28:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.342+0000 7f603912b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60340f66e0 0x7f6034106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.342+0000 7f603912b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340f6ff0 0x7f60341074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.342+0000 7f603912b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6034107a60 con 0x7f60340f6ff0 2024-02-15T18:28:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.342+0000 7f603912b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6034107ba0 con 0x7f60340f66e0 2024-02-15T18:28:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.342+0000 7f603259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340f6ff0 0x7f60341074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.342+0000 7f603259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340f6ff0 0x7f60341074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59184/0 (socket says 172.21.15.42:59184) 2024-02-15T18:28:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.342+0000 7f603259c700 1 -- 172.21.15.42:0/3266915680 learned_addr learned my addr 172.21.15.42:0/3266915680 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:28:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.342+0000 7f6032d9d700 1 --2- 172.21.15.42:0/3266915680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60340f66e0 0x7f6034106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.342+0000 7f603259c700 1 -- 172.21.15.42:0/3266915680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60340f66e0 msgr2=0x7f6034106fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.342+0000 7f603259c700 1 --2- 172.21.15.42:0/3266915680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60340f66e0 0x7f6034106fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.342+0000 7f603259c700 1 -- 172.21.15.42:0/3266915680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6028005040 con 0x7f60340f6ff0 2024-02-15T18:28:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.342+0000 7f603259c700 1 --2- 172.21.15.42:0/3266915680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340f6ff0 0x7f60341074b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f6028009f50 tx=0x7f6028009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.343+0000 7f601b7fe700 1 -- 172.21.15.42:0/3266915680 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6028018070 con 0x7f60340f6ff0 2024-02-15T18:28:15.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.343+0000 7f601b7fe700 1 -- 172.21.15.42:0/3266915680 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f6028007ba0 con 0x7f60340f6ff0 2024-02-15T18:28:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.343+0000 7f603912b700 1 -- 172.21.15.42:0/3266915680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60341084e0 con 0x7f60340f6ff0 2024-02-15T18:28:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.343+0000 7f601b7fe700 1 -- 172.21.15.42:0/3266915680 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f602801c420 con 0x7f60340f6ff0 2024-02-15T18:28:15.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.343+0000 7f603912b700 1 -- 172.21.15.42:0/3266915680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6034108a40 con 0x7f60340f6ff0 2024-02-15T18:28:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.344+0000 7f603912b700 1 -- 172.21.15.42:0/3266915680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6034045ad0 con 0x7f60340f6ff0 2024-02-15T18:28:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.344+0000 7f601b7fe700 1 -- 172.21.15.42:0/3266915680 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f602801d420 con 0x7f60340f6ff0 2024-02-15T18:28:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.344+0000 7f601b7fe700 1 --2- 172.21.15.42:0/3266915680 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f601c060f20 0x7f601c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.345+0000 7f601b7fe700 1 -- 172.21.15.42:0/3266915680 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f602807ddc0 con 0x7f60340f6ff0 2024-02-15T18:28:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.345+0000 7f6032d9d700 1 --2- 172.21.15.42:0/3266915680 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f601c060f20 0x7f601c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:15.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.345+0000 7f6032d9d700 1 --2- 172.21.15.42:0/3266915680 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f601c060f20 0x7f601c0633e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f6024004c40 tx=0x7f602400b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:15.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.349+0000 7f601b7fe700 1 -- 172.21.15.42:0/3266915680 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6028051c10 con 0x7f60340f6ff0 2024-02-15T18:28:15.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.532+0000 7f603912b700 1 -- 172.21.15.42:0/3266915680 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f60340ff150 con 0x7f601c060f20 2024-02-15T18:28:15.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.542+0000 7f601b7fe700 1 -- 172.21.15.42:0/3266915680 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f60340ff150 con 0x7f601c060f20 2024-02-15T18:28:15.538 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:28:15.538 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (7m) 18s ago 9m 21.6M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:28:15.538 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (31s) 18s ago 9m 6643k - 16.2.14-521-g0e714d9a 1fbc92c859dc 2da9d875539a 2024-02-15T18:28:15.538 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (25s) 17s ago 8m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 5c46a68ff62b 2024-02-15T18:28:15.538 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (7m) 18s ago 9m 33.7M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:28:15.538 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (4m) 18s ago 4m 14.2M - 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:28:15.538 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (4m) 18s ago 4m 17.0M - 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:28:15.538 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (4m) 17s ago 4m 154M - 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:28:15.539 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (4m) 17s ago 4m 227M - 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:28:15.539 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (2m) 18s ago 11m 428M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:28:15.539 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (111s) 17s ago 8m 388M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:28:15.539 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (90s) 18s ago 11m 44.2M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:28:15.539 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (50s) 17s ago 8m 25.8M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:28:15.539 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (9m) 18s ago 9m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:28:15.539 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (8m) 17s ago 8m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:28:15.539 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 starting - - - 4096M 2024-02-15T18:28:15.539 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (7m) 18s ago 7m 385M 4096M 16.2.4 8d91d370c2b8 2b3ae86111e8 2024-02-15T18:28:15.539 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (7m) 18s ago 7m 340M 4096M 16.2.4 8d91d370c2b8 3706071e4698 2024-02-15T18:28:15.539 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (6m) 17s ago 6m 432M 4096M 16.2.4 8d91d370c2b8 e6fe8e77c6fe 2024-02-15T18:28:15.539 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (6m) 17s ago 6m 428M 4096M 16.2.4 8d91d370c2b8 24582012e234 2024-02-15T18:28:15.539 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (6m) 17s ago 6m 347M 4096M 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:28:15.539 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (2m) 18s ago 8m 57.0M - 2.18.1 de242295e225 63b6dfb825ed 2024-02-15T18:28:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.544+0000 7f603912b700 1 -- 172.21.15.42:0/3266915680 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f601c060f20 msgr2=0x7f601c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.544+0000 7f603912b700 1 --2- 172.21.15.42:0/3266915680 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f601c060f20 0x7f601c0633e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f6024004c40 tx=0x7f602400b040).stop 2024-02-15T18:28:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.544+0000 7f603912b700 1 -- 172.21.15.42:0/3266915680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340f6ff0 msgr2=0x7f60341074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.544+0000 7f603912b700 1 --2- 172.21.15.42:0/3266915680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340f6ff0 0x7f60341074b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f6028009f50 tx=0x7f6028009b30).stop 2024-02-15T18:28:15.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.544+0000 7f603912b700 1 -- 172.21.15.42:0/3266915680 shutdown_connections 2024-02-15T18:28:15.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.544+0000 7f603912b700 1 --2- 172.21.15.42:0/3266915680 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f601c060f20 0x7f601c0633e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.544+0000 7f603912b700 1 --2- 172.21.15.42:0/3266915680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60340f66e0 0x7f6034106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.544+0000 7f603912b700 1 --2- 172.21.15.42:0/3266915680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60340f6ff0 0x7f60341074b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.544+0000 7f603912b700 1 -- 172.21.15.42:0/3266915680 >> 172.21.15.42:0/3266915680 conn(0x7f60340f1f00 msgr2=0x7f60340fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:15.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.544+0000 7f603912b700 1 -- 172.21.15.42:0/3266915680 shutdown_connections 2024-02-15T18:28:15.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.544+0000 7f603912b700 1 -- 172.21.15.42:0/3266915680 wait complete. 2024-02-15T18:28:15.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.681+0000 7f349b59e700 1 -- 172.21.15.42:0/2139964896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c0f58b0 msgr2=0x7f349c0f5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:15.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.682+0000 7f349b59e700 1 --2- 172.21.15.42:0/2139964896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c0f58b0 0x7f349c0f5cd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3484004660 tx=0x7f3484009b30).stop 2024-02-15T18:28:15.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.682+0000 7f349b59e700 1 -- 172.21.15.42:0/2139964896 shutdown_connections 2024-02-15T18:28:15.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.682+0000 7f349b59e700 1 --2- 172.21.15.42:0/2139964896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f349c0f4600 0x7f349c0f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.682+0000 7f349b59e700 1 --2- 172.21.15.42:0/2139964896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c0f58b0 0x7f349c0f5cd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.682+0000 7f349b59e700 1 -- 172.21.15.42:0/2139964896 >> 172.21.15.42:0/2139964896 conn(0x7f349c0f0320 msgr2=0x7f349c0f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:15.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.682+0000 7f349b59e700 1 -- 172.21.15.42:0/2139964896 shutdown_connections 2024-02-15T18:28:15.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.682+0000 7f349b59e700 1 -- 172.21.15.42:0/2139964896 wait complete. 2024-02-15T18:28:15.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.683+0000 7f349b59e700 1 Processor -- start 2024-02-15T18:28:15.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.683+0000 7f349b59e700 1 -- start start 2024-02-15T18:28:15.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.683+0000 7f349b59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f349c0f4600 0x7f349c068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:15.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.683+0000 7f349b59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c0f58b0 0x7f349c069080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:15.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.683+0000 7f349b59e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c069630 con 0x7f349c0f58b0 2024-02-15T18:28:15.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.683+0000 7f349b59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349c069770 con 0x7f349c0f4600 2024-02-15T18:28:15.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.683+0000 7f3499d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c0f58b0 0x7f349c069080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:15.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.683+0000 7f349a59c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f349c0f4600 0x7f349c068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:15.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.683+0000 7f3499d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c0f58b0 0x7f349c069080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59198/0 (socket says 172.21.15.42:59198) 2024-02-15T18:28:15.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.683+0000 7f3499d9b700 1 -- 172.21.15.42:0/2582646100 learned_addr learned my addr 172.21.15.42:0/2582646100 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:28:15.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.684+0000 7f3499d9b700 1 -- 172.21.15.42:0/2582646100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f349c0f4600 msgr2=0x7f349c068b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:15.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.684+0000 7f3499d9b700 1 --2- 172.21.15.42:0/2582646100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f349c0f4600 0x7f349c068b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:15.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.684+0000 7f3499d9b700 1 -- 172.21.15.42:0/2582646100 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3484005040 con 0x7f349c0f58b0 2024-02-15T18:28:15.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.684+0000 7f3499d9b700 1 --2- 172.21.15.42:0/2582646100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c0f58b0 0x7f349c069080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f348c0026d0 tx=0x7f348c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:15.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.686+0000 7f3492ffd700 1 -- 172.21.15.42:0/2582646100 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f348c0079c0 con 0x7f349c0f58b0 2024-02-15T18:28:15.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.686+0000 7f3492ffd700 1 -- 172.21.15.42:0/2582646100 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f348c00f040 con 0x7f349c0f58b0 2024-02-15T18:28:15.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.686+0000 7f3492ffd700 1 -- 172.21.15.42:0/2582646100 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f348c014a00 con 0x7f349c0f58b0 2024-02-15T18:28:15.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.686+0000 7f349b59e700 1 -- 172.21.15.42:0/2582646100 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349c117120 con 0x7f349c0f58b0 2024-02-15T18:28:15.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.686+0000 7f349b59e700 1 -- 172.21.15.42:0/2582646100 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349c1176e0 con 0x7f349c0f58b0 2024-02-15T18:28:15.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.687+0000 7f349b59e700 1 -- 172.21.15.42:0/2582646100 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349c045ad0 con 0x7f349c0f58b0 2024-02-15T18:28:15.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.688+0000 7f3492ffd700 1 -- 172.21.15.42:0/2582646100 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f348c007b20 con 0x7f349c0f58b0 2024-02-15T18:28:15.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.689+0000 7f3492ffd700 1 --2- 172.21.15.42:0/2582646100 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3488060fc0 0x7f3488063480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:15.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.689+0000 7f3492ffd700 1 -- 172.21.15.42:0/2582646100 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f348c07c2f0 con 0x7f349c0f58b0 2024-02-15T18:28:15.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.689+0000 7f349a59c700 1 --2- 172.21.15.42:0/2582646100 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3488060fc0 0x7f3488063480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:15.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.690+0000 7f349a59c700 1 --2- 172.21.15.42:0/2582646100 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3488060fc0 0x7f3488063480 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f3484004630 tx=0x7f3484009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:15.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:15.696+0000 7f3492ffd700 1 -- 172.21.15.42:0/2582646100 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f348c050160 con 0x7f349c0f58b0 2024-02-15T18:28:16.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.070+0000 7f349b59e700 1 -- 172.21.15.42:0/2582646100 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f349c027a00 con 0x7f349c0f58b0 2024-02-15T18:28:16.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.070+0000 7f3492ffd700 1 -- 172.21.15.42:0/2582646100 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f348c04fd50 con 0x7f349c0f58b0 2024-02-15T18:28:16.066 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:28:16.066 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:28:16.066 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:28:16.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:28:16.067 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:28:16.067 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:28:16.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:28:16.067 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:28:16.067 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 5 2024-02-15T18:28:16.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:28:16.067 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:28:16.067 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:28:16.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:28:16.067 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:28:16.067 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4, 2024-02-15T18:28:16.067 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 9 2024-02-15T18:28:16.067 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:28:16.068 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:28:16.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.072+0000 7f349b59e700 1 -- 172.21.15.42:0/2582646100 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3488060fc0 msgr2=0x7f3488063480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:16.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.072+0000 7f349b59e700 1 --2- 172.21.15.42:0/2582646100 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3488060fc0 0x7f3488063480 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f3484004630 tx=0x7f3484009b30).stop 2024-02-15T18:28:16.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.072+0000 7f349b59e700 1 -- 172.21.15.42:0/2582646100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c0f58b0 msgr2=0x7f349c069080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:16.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.072+0000 7f349b59e700 1 --2- 172.21.15.42:0/2582646100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c0f58b0 0x7f349c069080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f348c0026d0 tx=0x7f348c009de0).stop 2024-02-15T18:28:16.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.072+0000 7f349b59e700 1 -- 172.21.15.42:0/2582646100 shutdown_connections 2024-02-15T18:28:16.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.072+0000 7f349b59e700 1 --2- 172.21.15.42:0/2582646100 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3488060fc0 0x7f3488063480 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.072+0000 7f349b59e700 1 --2- 172.21.15.42:0/2582646100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f349c0f4600 0x7f349c068b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.072+0000 7f349b59e700 1 --2- 172.21.15.42:0/2582646100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f349c0f58b0 0x7f349c069080 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.072+0000 7f349b59e700 1 -- 172.21.15.42:0/2582646100 >> 172.21.15.42:0/2582646100 conn(0x7f349c0f0320 msgr2=0x7f349c0f9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:16.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.073+0000 7f349b59e700 1 -- 172.21.15.42:0/2582646100 shutdown_connections 2024-02-15T18:28:16.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.073+0000 7f349b59e700 1 -- 172.21.15.42:0/2582646100 wait complete. 2024-02-15T18:28:16.150 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:28:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-0[80976]: debug 2024-02-15T18:28:15.873+0000 7f7bf2ba1700 -1 osd.0 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-15T18:28:16.151 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:14.700407+0000 mgr.smithi042.kfoihb (mgr.24435) 98 : cluster [DBG] pgmap v68: 65 pgs: 32 active+undersized+degraded, 2 active+undersized, 31 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 409 op/s; 6342/42774 objects degraded (14.827%) 2024-02-15T18:28:16.151 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:14.904548+0000 mgr.smithi042.kfoihb (mgr.24435) 99 : audit [DBG] from='client.34148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:16.151 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:15.054797+0000 mon.smithi191 (mon.1) 3 : audit [INF] from='osd.0 [v2:172.21.15.42:6802/4030668481,v1:172.21.15.42:6803/4030668481]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-15T18:28:16.151 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:15.056327+0000 mon.smithi042 (mon.0) 101 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-15T18:28:16.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.191+0000 7f07fff21700 1 -- 172.21.15.42:0/2354558838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07f80f5340 msgr2=0x7f07f80f7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:16.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.191+0000 7f07fff21700 1 --2- 172.21.15.42:0/2354558838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07f80f5340 0x7f07f80f7ac0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f07f4004660 tx=0x7f07f4009b30).stop 2024-02-15T18:28:16.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.191+0000 7f07fff21700 1 -- 172.21.15.42:0/2354558838 shutdown_connections 2024-02-15T18:28:16.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.191+0000 7f07fff21700 1 --2- 172.21.15.42:0/2354558838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07f80f5340 0x7f07f80f7ac0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.191+0000 7f07fff21700 1 --2- 172.21.15.42:0/2354558838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07f80f49a0 0x7f07f80f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.191+0000 7f07fff21700 1 -- 172.21.15.42:0/2354558838 >> 172.21.15.42:0/2354558838 conn(0x7f07f80f04c0 msgr2=0x7f07f80f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:16.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.191+0000 7f07fff21700 1 -- 172.21.15.42:0/2354558838 shutdown_connections 2024-02-15T18:28:16.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.191+0000 7f07fff21700 1 -- 172.21.15.42:0/2354558838 wait complete. 2024-02-15T18:28:16.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.191+0000 7f07fff21700 1 Processor -- start 2024-02-15T18:28:16.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.191+0000 7f07fff21700 1 -- start start 2024-02-15T18:28:16.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07fff21700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07f80f49a0 0x7f07f8107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:16.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07fff21700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07f80f5340 0x7f07f81074f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:16.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07fff21700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f8107aa0 con 0x7f07f80f5340 2024-02-15T18:28:16.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07fff21700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f8107be0 con 0x7f07f80f49a0 2024-02-15T18:28:16.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07fd4bc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07f80f5340 0x7f07f81074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:16.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07fd4bc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07f80f5340 0x7f07f81074f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59216/0 (socket says 172.21.15.42:59216) 2024-02-15T18:28:16.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07fd4bc700 1 -- 172.21.15.42:0/841237554 learned_addr learned my addr 172.21.15.42:0/841237554 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:28:16.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07fdcbd700 1 --2- 172.21.15.42:0/841237554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07f80f49a0 0x7f07f8107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:16.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07fd4bc700 1 -- 172.21.15.42:0/841237554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07f80f49a0 msgr2=0x7f07f8107000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:16.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07fd4bc700 1 --2- 172.21.15.42:0/841237554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07f80f49a0 0x7f07f8107000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07fd4bc700 1 -- 172.21.15.42:0/841237554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f4005040 con 0x7f07f80f5340 2024-02-15T18:28:16.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07fd4bc700 1 --2- 172.21.15.42:0/841237554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07f80f5340 0x7f07f81074f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f07f4002040 tx=0x7f07f400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:16.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07ee7fc700 1 -- 172.21.15.42:0/841237554 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07f4019070 con 0x7f07f80f5340 2024-02-15T18:28:16.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07ee7fc700 1 -- 172.21.15.42:0/841237554 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f07f4007e80 con 0x7f07f80f5340 2024-02-15T18:28:16.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07fff21700 1 -- 172.21.15.42:0/841237554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f8108520 con 0x7f07f80f5340 2024-02-15T18:28:16.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07ee7fc700 1 -- 172.21.15.42:0/841237554 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07f4014db0 con 0x7f07f80f5340 2024-02-15T18:28:16.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.192+0000 7f07fff21700 1 -- 172.21.15.42:0/841237554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f8108a80 con 0x7f07f80f5340 2024-02-15T18:28:16.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.193+0000 7f07fff21700 1 -- 172.21.15.42:0/841237554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f8045ad0 con 0x7f07f80f5340 2024-02-15T18:28:16.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.194+0000 7f07ee7fc700 1 -- 172.21.15.42:0/841237554 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f07f4025070 con 0x7f07f80f5340 2024-02-15T18:28:16.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.194+0000 7f07ee7fc700 1 --2- 172.21.15.42:0/841237554 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f07e4060f20 0x7f07e40633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:16.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.194+0000 7f07ee7fc700 1 -- 172.21.15.42:0/841237554 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f07f407cef0 con 0x7f07f80f5340 2024-02-15T18:28:16.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.194+0000 7f07fdcbd700 1 --2- 172.21.15.42:0/841237554 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f07e4060f20 0x7f07e40633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:16.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.195+0000 7f07fdcbd700 1 --2- 172.21.15.42:0/841237554 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f07e4060f20 0x7f07e40633e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f07e8009fd0 tx=0x7f07e800b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:16.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.199+0000 7f07ee7fc700 1 -- 172.21.15.42:0/841237554 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f07f4050dc0 con 0x7f07f80f5340 2024-02-15T18:28:16.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:14.700407+0000 mgr.smithi042.kfoihb (mgr.24435) 98 : cluster [DBG] pgmap v68: 65 pgs: 32 active+undersized+degraded, 2 active+undersized, 31 active+clean; 305 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 409 op/s; 6342/42774 objects degraded (14.827%) 2024-02-15T18:28:16.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:14.904548+0000 mgr.smithi042.kfoihb (mgr.24435) 99 : audit [DBG] from='client.34148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:16.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:15.054797+0000 mon.smithi191 (mon.1) 3 : audit [INF] from='osd.0 [v2:172.21.15.42:6802/4030668481,v1:172.21.15.42:6803/4030668481]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-15T18:28:16.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:15.056327+0000 mon.smithi042 (mon.0) 101 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-15T18:28:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.419+0000 7f07fff21700 1 -- 172.21.15.42:0/841237554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f07f8027a00 con 0x7f07f80f5340 2024-02-15T18:28:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.419+0000 7f07ee7fc700 1 -- 172.21.15.42:0/841237554 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1773 (secure 0 0 0) 0x7f07f4023070 con 0x7f07f80f5340 2024-02-15T18:28:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:28:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:28:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:28:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:28:16.415 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:28:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:28:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:28:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:28:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:28:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:28:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:28:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:28:16.415 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:28:16.416 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:28:16.416 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:28:16.416 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:28:16.416 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:28:16.416 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:28:16.416 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:28:16.416 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:28:16.416 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:28:16.416 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:28:16.416 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:28:16.416 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:28:16.416 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:28:16.416 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:28:16.416 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:28:16.417 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:28:16.417 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:28:16.417 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:28:16.417 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:28:16.417 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:28:16.417 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:28:16.417 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:28:16.417 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:28:16.417 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:28:16.417 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:28:16.417 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:28:16.417 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:28:16.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.421+0000 7f07fff21700 1 -- 172.21.15.42:0/841237554 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f07e4060f20 msgr2=0x7f07e40633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:16.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.421+0000 7f07fff21700 1 --2- 172.21.15.42:0/841237554 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f07e4060f20 0x7f07e40633e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f07e8009fd0 tx=0x7f07e800b040).stop 2024-02-15T18:28:16.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.421+0000 7f07fff21700 1 -- 172.21.15.42:0/841237554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07f80f5340 msgr2=0x7f07f81074f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:16.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.421+0000 7f07fff21700 1 --2- 172.21.15.42:0/841237554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07f80f5340 0x7f07f81074f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f07f4002040 tx=0x7f07f400b040).stop 2024-02-15T18:28:16.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.421+0000 7f07fff21700 1 -- 172.21.15.42:0/841237554 shutdown_connections 2024-02-15T18:28:16.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.421+0000 7f07fff21700 1 --2- 172.21.15.42:0/841237554 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f07e4060f20 0x7f07e40633e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.421+0000 7f07fff21700 1 --2- 172.21.15.42:0/841237554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07f80f49a0 0x7f07f8107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.421+0000 7f07fff21700 1 --2- 172.21.15.42:0/841237554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f07f80f5340 0x7f07f81074f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.421+0000 7f07fff21700 1 -- 172.21.15.42:0/841237554 >> 172.21.15.42:0/841237554 conn(0x7f07f80f04c0 msgr2=0x7f07f80fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:16.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.421+0000 7f07fff21700 1 -- 172.21.15.42:0/841237554 shutdown_connections 2024-02-15T18:28:16.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.421+0000 7f07fff21700 1 -- 172.21.15.42:0/841237554 wait complete. 2024-02-15T18:28:16.419 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:28:16.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.523+0000 7f713389b700 1 -- 172.21.15.42:0/3138424672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c0f55c0 msgr2=0x7f712c0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:16.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.523+0000 7f713389b700 1 --2- 172.21.15.42:0/3138424672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c0f55c0 0x7f712c0f59e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f7128004660 tx=0x7f7128009b30).stop 2024-02-15T18:28:16.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.523+0000 7f713389b700 1 -- 172.21.15.42:0/3138424672 shutdown_connections 2024-02-15T18:28:16.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.523+0000 7f713389b700 1 --2- 172.21.15.42:0/3138424672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f712c0f6860 0x7f712c0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.523+0000 7f713389b700 1 --2- 172.21.15.42:0/3138424672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c0f55c0 0x7f712c0f59e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.523+0000 7f713389b700 1 -- 172.21.15.42:0/3138424672 >> 172.21.15.42:0/3138424672 conn(0x7f712c0f0d40 msgr2=0x7f712c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:16.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.523+0000 7f713389b700 1 -- 172.21.15.42:0/3138424672 shutdown_connections 2024-02-15T18:28:16.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.523+0000 7f713389b700 1 -- 172.21.15.42:0/3138424672 wait complete. 2024-02-15T18:28:16.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.523+0000 7f713389b700 1 Processor -- start 2024-02-15T18:28:16.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.523+0000 7f713389b700 1 -- start start 2024-02-15T18:28:16.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f713389b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f712c0f55c0 0x7f712c106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:16.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f713389b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c0f6860 0x7f712c1074e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:16.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f713389b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c107a90 con 0x7f712c0f6860 2024-02-15T18:28:16.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f713389b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c107bd0 con 0x7f712c0f55c0 2024-02-15T18:28:16.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f7130e36700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c0f6860 0x7f712c1074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:16.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f7130e36700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c0f6860 0x7f712c1074e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59226/0 (socket says 172.21.15.42:59226) 2024-02-15T18:28:16.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f7130e36700 1 -- 172.21.15.42:0/3336523601 learned_addr learned my addr 172.21.15.42:0/3336523601 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:28:16.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f7131637700 1 --2- 172.21.15.42:0/3336523601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f712c0f55c0 0x7f712c106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:16.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f7130e36700 1 -- 172.21.15.42:0/3336523601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f712c0f55c0 msgr2=0x7f712c106ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:16.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f7130e36700 1 --2- 172.21.15.42:0/3336523601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f712c0f55c0 0x7f712c106ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f7130e36700 1 -- 172.21.15.42:0/3336523601 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7128005040 con 0x7f712c0f6860 2024-02-15T18:28:16.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f7130e36700 1 --2- 172.21.15.42:0/3336523601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c0f6860 0x7f712c1074e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f71200026d0 tx=0x7f7120009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:16.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f711dffb700 1 -- 172.21.15.42:0/3336523601 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71200079c0 con 0x7f712c0f6860 2024-02-15T18:28:16.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f713389b700 1 -- 172.21.15.42:0/3336523601 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c108570 con 0x7f712c0f6860 2024-02-15T18:28:16.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f711dffb700 1 -- 172.21.15.42:0/3336523601 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f712000f040 con 0x7f712c0f6860 2024-02-15T18:28:16.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.524+0000 7f711dffb700 1 -- 172.21.15.42:0/3336523601 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71200149f0 con 0x7f712c0f6860 2024-02-15T18:28:16.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.525+0000 7f713389b700 1 -- 172.21.15.42:0/3336523601 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c108b30 con 0x7f712c0f6860 2024-02-15T18:28:16.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.526+0000 7f713389b700 1 -- 172.21.15.42:0/3336523601 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c045ad0 con 0x7f712c0f6860 2024-02-15T18:28:16.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.526+0000 7f711dffb700 1 -- 172.21.15.42:0/3336523601 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f7120007b20 con 0x7f712c0f6860 2024-02-15T18:28:16.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.526+0000 7f711dffb700 1 --2- 172.21.15.42:0/3336523601 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f7118060f20 0x7f71180633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:16.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.526+0000 7f711dffb700 1 -- 172.21.15.42:0/3336523601 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f712007c0e0 con 0x7f712c0f6860 2024-02-15T18:28:16.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.526+0000 7f7131637700 1 --2- 172.21.15.42:0/3336523601 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f7118060f20 0x7f71180633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:16.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.527+0000 7f7131637700 1 --2- 172.21.15.42:0/3336523601 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f7118060f20 0x7f71180633e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f7128004630 tx=0x7f7128009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:16.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.531+0000 7f711dffb700 1 -- 172.21.15.42:0/3336523601 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f712004ff90 con 0x7f712c0f6860 2024-02-15T18:28:16.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.794+0000 7f713389b700 1 -- 172.21.15.42:0/3336523601 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f712c0faca0 con 0x7f7118060f20 2024-02-15T18:28:16.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.795+0000 7f711dffb700 1 -- 172.21.15.42:0/3336523601 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f712c0faca0 con 0x7f7118060f20 2024-02-15T18:28:16.791 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:28:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:28:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:28:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:28:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:28:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-02-15T18:28:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-02-15T18:28:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-02-15T18:28:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:28:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "7/21 daemons upgraded", 2024-02-15T18:28:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading osd daemons", 2024-02-15T18:28:16.791 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:28:16.791 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:28:16.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.797+0000 7f713389b700 1 -- 172.21.15.42:0/3336523601 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f7118060f20 msgr2=0x7f71180633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:16.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.797+0000 7f713389b700 1 --2- 172.21.15.42:0/3336523601 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f7118060f20 0x7f71180633e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f7128004630 tx=0x7f7128009b30).stop 2024-02-15T18:28:16.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.797+0000 7f713389b700 1 -- 172.21.15.42:0/3336523601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c0f6860 msgr2=0x7f712c1074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:16.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.797+0000 7f713389b700 1 --2- 172.21.15.42:0/3336523601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c0f6860 0x7f712c1074e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f71200026d0 tx=0x7f7120009de0).stop 2024-02-15T18:28:16.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.797+0000 7f713389b700 1 -- 172.21.15.42:0/3336523601 shutdown_connections 2024-02-15T18:28:16.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.797+0000 7f713389b700 1 --2- 172.21.15.42:0/3336523601 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f7118060f20 0x7f71180633e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.797+0000 7f713389b700 1 --2- 172.21.15.42:0/3336523601 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f712c0f55c0 0x7f712c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.797+0000 7f713389b700 1 --2- 172.21.15.42:0/3336523601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c0f6860 0x7f712c1074e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:16.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.797+0000 7f713389b700 1 -- 172.21.15.42:0/3336523601 >> 172.21.15.42:0/3336523601 conn(0x7f712c0f0d40 msgr2=0x7f712c05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:16.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.797+0000 7f713389b700 1 -- 172.21.15.42:0/3336523601 shutdown_connections 2024-02-15T18:28:16.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:16.797+0000 7f713389b700 1 -- 172.21.15.42:0/3336523601 wait complete. 2024-02-15T18:28:17.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:15.234854+0000 mgr.smithi042.kfoihb (mgr.24435) 100 : audit [DBG] from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:17.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:15.533955+0000 mgr.smithi042.kfoihb (mgr.24435) 101 : audit [DBG] from='client.34156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:17.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:15.855362+0000 mon.smithi042 (mon.0) 102 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-15T18:28:17.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:15.855443+0000 mon.smithi042 (mon.0) 103 : cluster [DBG] osdmap e51: 6 total, 5 up, 6 in 2024-02-15T18:28:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:15.863135+0000 mon.smithi191 (mon.1) 4 : audit [INF] from='osd.0 [v2:172.21.15.42:6802/4030668481,v1:172.21.15.42:6803/4030668481]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-02-15T18:28:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:15.863677+0000 mon.smithi042 (mon.0) 104 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-02-15T18:28:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:15.941835+0000 mon.smithi042 (mon.0) 105 : cluster [WRN] Health check update: Degraded data redundancy: 6342/42774 objects degraded (14.827%), 32 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:16.071490+0000 mon.smithi042 (mon.0) 106 : audit [DBG] from='client.? 172.21.15.42:0/2582646100' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:28:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:16.136524+0000 mon.smithi042 (mon.0) 107 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:28:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:16.137468+0000 mon.smithi042 (mon.0) 108 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:16.138202+0000 mon.smithi042 (mon.0) 109 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:28:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:16.141996+0000 mon.smithi042 (mon.0) 110 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:28:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:16.150022+0000 mon.smithi042 (mon.0) 111 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:28:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:16.152739+0000 mon.smithi042 (mon.0) 112 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:28:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:16.153859+0000 mon.smithi042 (mon.0) 113 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:28:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:16.154876+0000 mon.smithi042 (mon.0) 114 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:28:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:16.155892+0000 mon.smithi042 (mon.0) 115 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:28:17.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:16.420375+0000 mon.smithi042 (mon.0) 116 : audit [DBG] from='client.? 172.21.15.42:0/841237554' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:28:17.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:15.234854+0000 mgr.smithi042.kfoihb (mgr.24435) 100 : audit [DBG] from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:17.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:15.533955+0000 mgr.smithi042.kfoihb (mgr.24435) 101 : audit [DBG] from='client.34156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:17.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:15.855362+0000 mon.smithi042 (mon.0) 102 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-15T18:28:17.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:15.855443+0000 mon.smithi042 (mon.0) 103 : cluster [DBG] osdmap e51: 6 total, 5 up, 6 in 2024-02-15T18:28:17.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:15.863135+0000 mon.smithi191 (mon.1) 4 : audit [INF] from='osd.0 [v2:172.21.15.42:6802/4030668481,v1:172.21.15.42:6803/4030668481]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-02-15T18:28:17.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:15.863677+0000 mon.smithi042 (mon.0) 104 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-02-15T18:28:17.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:15.941835+0000 mon.smithi042 (mon.0) 105 : cluster [WRN] Health check update: Degraded data redundancy: 6342/42774 objects degraded (14.827%), 32 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:17.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:16.071490+0000 mon.smithi042 (mon.0) 106 : audit [DBG] from='client.? 172.21.15.42:0/2582646100' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:28:17.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:16.136524+0000 mon.smithi042 (mon.0) 107 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:28:17.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:16.137468+0000 mon.smithi042 (mon.0) 108 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:28:17.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:16.138202+0000 mon.smithi042 (mon.0) 109 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:28:17.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:16.141996+0000 mon.smithi042 (mon.0) 110 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:28:17.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:16.150022+0000 mon.smithi042 (mon.0) 111 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:28:17.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:16.152739+0000 mon.smithi042 (mon.0) 112 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:28:17.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:16.153859+0000 mon.smithi042 (mon.0) 113 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:28:17.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:16.154876+0000 mon.smithi042 (mon.0) 114 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:28:17.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:16.155892+0000 mon.smithi042 (mon.0) 115 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:28:17.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:16.420375+0000 mon.smithi042 (mon.0) 116 : audit [DBG] from='client.? 172.21.15.42:0/841237554' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:28:18.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:28:16.138894+0000 mgr.smithi042.kfoihb (mgr.24435) 102 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:28:18.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:16.156058+0000 mgr.smithi042.kfoihb (mgr.24435) 103 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:28:18.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:28:16.156855+0000 mgr.smithi042.kfoihb (mgr.24435) 104 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-02-15T18:28:18.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:16.700909+0000 mgr.smithi042.kfoihb (mgr.24435) 105 : cluster [DBG] pgmap v70: 65 pgs: 32 active+undersized+degraded, 2 active+undersized, 31 active+clean; 306 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 454 op/s; 6207/41712 objects degraded (14.881%) 2024-02-15T18:28:18.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:16.796100+0000 mgr.smithi042.kfoihb (mgr.24435) 106 : audit [DBG] from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:18.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:16.857839+0000 mon.smithi042 (mon.0) 117 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:28:18.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:16.861069+0000 mon.smithi042 (mon.0) 118 : cluster [INF] osd.0 [v2:172.21.15.42:6802/4030668481,v1:172.21.15.42:6803/4030668481] boot 2024-02-15T18:28:18.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:16.861109+0000 mon.smithi042 (mon.0) 119 : cluster [DBG] osdmap e52: 6 total, 6 up, 6 in 2024-02-15T18:28:18.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:16.861627+0000 mon.smithi042 (mon.0) 120 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:28:18.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:17.864277+0000 mon.smithi042 (mon.0) 121 : cluster [DBG] osdmap e53: 6 total, 6 up, 6 in 2024-02-15T18:28:18.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:28:16.138894+0000 mgr.smithi042.kfoihb (mgr.24435) 102 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:28:18.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:16.156058+0000 mgr.smithi042.kfoihb (mgr.24435) 103 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:28:18.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:28:16.156855+0000 mgr.smithi042.kfoihb (mgr.24435) 104 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-02-15T18:28:18.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:16.700909+0000 mgr.smithi042.kfoihb (mgr.24435) 105 : cluster [DBG] pgmap v70: 65 pgs: 32 active+undersized+degraded, 2 active+undersized, 31 active+clean; 306 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 454 op/s; 6207/41712 objects degraded (14.881%) 2024-02-15T18:28:18.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:16.796100+0000 mgr.smithi042.kfoihb (mgr.24435) 106 : audit [DBG] from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:18.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:16.857839+0000 mon.smithi042 (mon.0) 117 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:28:18.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:16.861069+0000 mon.smithi042 (mon.0) 118 : cluster [INF] osd.0 [v2:172.21.15.42:6802/4030668481,v1:172.21.15.42:6803/4030668481] boot 2024-02-15T18:28:18.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:16.861109+0000 mon.smithi042 (mon.0) 119 : cluster [DBG] osdmap e52: 6 total, 6 up, 6 in 2024-02-15T18:28:18.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:16.861627+0000 mon.smithi042 (mon.0) 120 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-15T18:28:18.309 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:17.864277+0000 mon.smithi042 (mon.0) 121 : cluster [DBG] osdmap e53: 6 total, 6 up, 6 in 2024-02-15T18:28:20.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:18.702297+0000 mgr.smithi042.kfoihb (mgr.24435) 107 : cluster [DBG] pgmap v73: 65 pgs: 32 active+undersized+degraded, 2 active+undersized, 31 active+clean; 308 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 362 op/s; 5483/37074 objects degraded (14.789%) 2024-02-15T18:28:20.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:18.870721+0000 mon.smithi042 (mon.0) 122 : cluster [DBG] osdmap e54: 6 total, 6 up, 6 in 2024-02-15T18:28:20.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:18.702297+0000 mgr.smithi042.kfoihb (mgr.24435) 107 : cluster [DBG] pgmap v73: 65 pgs: 32 active+undersized+degraded, 2 active+undersized, 31 active+clean; 308 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 362 op/s; 5483/37074 objects degraded (14.789%) 2024-02-15T18:28:20.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:18.870721+0000 mon.smithi042 (mon.0) 122 : cluster [DBG] osdmap e54: 6 total, 6 up, 6 in 2024-02-15T18:28:21.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:19.903016+0000 mon.smithi042 (mon.0) 123 : cluster [DBG] osdmap e55: 6 total, 6 up, 6 in 2024-02-15T18:28:21.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:19.903016+0000 mon.smithi042 (mon.0) 123 : cluster [DBG] osdmap e55: 6 total, 6 up, 6 in 2024-02-15T18:28:22.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:20.702810+0000 mgr.smithi042.kfoihb (mgr.24435) 108 : cluster [DBG] pgmap v76: 65 pgs: 32 active+undersized+degraded, 2 active+undersized, 31 active+clean; 308 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 430 op/s; 5483/37074 objects degraded (14.789%) 2024-02-15T18:28:22.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:20.906768+0000 mon.smithi042 (mon.0) 124 : cluster [DBG] osdmap e56: 6 total, 6 up, 6 in 2024-02-15T18:28:22.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:20.942815+0000 mon.smithi042 (mon.0) 125 : cluster [WRN] Health check update: Degraded data redundancy: 5483/37074 objects degraded (14.789%), 32 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:22.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:20.702810+0000 mgr.smithi042.kfoihb (mgr.24435) 108 : cluster [DBG] pgmap v76: 65 pgs: 32 active+undersized+degraded, 2 active+undersized, 31 active+clean; 308 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 430 op/s; 5483/37074 objects degraded (14.789%) 2024-02-15T18:28:22.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:20.906768+0000 mon.smithi042 (mon.0) 124 : cluster [DBG] osdmap e56: 6 total, 6 up, 6 in 2024-02-15T18:28:22.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:20.942815+0000 mon.smithi042 (mon.0) 125 : cluster [WRN] Health check update: Degraded data redundancy: 5483/37074 objects degraded (14.789%), 32 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:23.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:21.910550+0000 mon.smithi042 (mon.0) 126 : cluster [DBG] osdmap e57: 6 total, 6 up, 6 in 2024-02-15T18:28:23.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:21.910550+0000 mon.smithi042 (mon.0) 126 : cluster [DBG] osdmap e57: 6 total, 6 up, 6 in 2024-02-15T18:28:24.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:22.703218+0000 mgr.smithi042.kfoihb (mgr.24435) 109 : cluster [DBG] pgmap v79: 65 pgs: 2 active+undersized+remapped, 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 2 peering, 1 active+undersized+degraded, 51 active+clean; 311 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 2.41k op/s; 1679/35463 objects degraded (4.735%); 2.1 MiB/s, 158 objects/s recovering 2024-02-15T18:28:24.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:22.917317+0000 mon.smithi042 (mon.0) 127 : cluster [DBG] osdmap e58: 6 total, 6 up, 6 in 2024-02-15T18:28:24.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:22.703218+0000 mgr.smithi042.kfoihb (mgr.24435) 109 : cluster [DBG] pgmap v79: 65 pgs: 2 active+undersized+remapped, 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 2 peering, 1 active+undersized+degraded, 51 active+clean; 311 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 2.41k op/s; 1679/35463 objects degraded (4.735%); 2.1 MiB/s, 158 objects/s recovering 2024-02-15T18:28:24.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:22.917317+0000 mon.smithi042 (mon.0) 127 : cluster [DBG] osdmap e58: 6 total, 6 up, 6 in 2024-02-15T18:28:25.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:24.703668+0000 mgr.smithi042.kfoihb (mgr.24435) 110 : cluster [DBG] pgmap v81: 65 pgs: 2 active+undersized+remapped, 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 2 peering, 1 active+undersized+degraded, 51 active+clean; 311 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 2.01k op/s; 1679/35463 objects degraded (4.735%); 1.8 MiB/s, 132 objects/s recovering 2024-02-15T18:28:25.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:24.703668+0000 mgr.smithi042.kfoihb (mgr.24435) 110 : cluster [DBG] pgmap v81: 65 pgs: 2 active+undersized+remapped, 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 2 peering, 1 active+undersized+degraded, 51 active+clean; 311 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 2.01k op/s; 1679/35463 objects degraded (4.735%); 1.8 MiB/s, 132 objects/s recovering 2024-02-15T18:28:27.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:25.835144+0000 mon.smithi042 (mon.0) 128 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:28:27.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:25.943761+0000 mon.smithi042 (mon.0) 129 : cluster [WRN] Health check update: Degraded data redundancy: 1679/35463 objects degraded (4.735%), 9 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:27.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:25.835144+0000 mon.smithi042 (mon.0) 128 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:28:27.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:25.943761+0000 mon.smithi042 (mon.0) 129 : cluster [WRN] Health check update: Degraded data redundancy: 1679/35463 objects degraded (4.735%), 9 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:28.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:26.704165+0000 mgr.smithi042.kfoihb (mgr.24435) 111 : cluster [DBG] pgmap v82: 65 pgs: 1 active+undersized+remapped, 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 2 peering, 53 active+clean; 310 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 1.75k op/s; 1274/34572 objects degraded (3.685%); 1.4 MiB/s, 135 objects/s recovering 2024-02-15T18:28:28.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:26.704165+0000 mgr.smithi042.kfoihb (mgr.24435) 111 : cluster [DBG] pgmap v82: 65 pgs: 1 active+undersized+remapped, 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 2 peering, 53 active+clean; 310 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 1.75k op/s; 1274/34572 objects degraded (3.685%); 1.4 MiB/s, 135 objects/s recovering 2024-02-15T18:28:30.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:28.704732+0000 mgr.smithi042.kfoihb (mgr.24435) 112 : cluster [DBG] pgmap v83: 65 pgs: 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 56 active+clean; 302 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 1.55k op/s; 1274/31650 objects degraded (4.025%); 1.1 MiB/s, 111 objects/s recovering 2024-02-15T18:28:30.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:28.704732+0000 mgr.smithi042.kfoihb (mgr.24435) 112 : cluster [DBG] pgmap v83: 65 pgs: 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 56 active+clean; 302 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 1.55k op/s; 1274/31650 objects degraded (4.025%); 1.1 MiB/s, 111 objects/s recovering 2024-02-15T18:28:32.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:30.705157+0000 mgr.smithi042.kfoihb (mgr.24435) 113 : cluster [DBG] pgmap v84: 65 pgs: 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 56 active+clean; 302 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 971 KiB/s wr, 292 op/s; 1274/31650 objects degraded (4.025%); 0 B/s, 26 objects/s recovering 2024-02-15T18:28:32.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:30.944665+0000 mon.smithi042 (mon.0) 130 : cluster [WRN] Health check update: Degraded data redundancy: 1274/31650 objects degraded (4.025%), 8 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:32.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:31.167204+0000 mon.smithi042 (mon.0) 131 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:28:32.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:30.705157+0000 mgr.smithi042.kfoihb (mgr.24435) 113 : cluster [DBG] pgmap v84: 65 pgs: 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 56 active+clean; 302 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 971 KiB/s wr, 292 op/s; 1274/31650 objects degraded (4.025%); 0 B/s, 26 objects/s recovering 2024-02-15T18:28:32.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:30.944665+0000 mon.smithi042 (mon.0) 130 : cluster [WRN] Health check update: Degraded data redundancy: 1274/31650 objects degraded (4.025%), 8 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:32.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:31.167204+0000 mon.smithi042 (mon.0) 131 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:28:33.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:31.167382+0000 mgr.smithi042.kfoihb (mgr.24435) 114 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:28:33.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:28:31.168099+0000 mgr.smithi042.kfoihb (mgr.24435) 115 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-02-15T18:28:33.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:31.167382+0000 mgr.smithi042.kfoihb (mgr.24435) 114 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:28:33.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:28:31.168099+0000 mgr.smithi042.kfoihb (mgr.24435) 115 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-02-15T18:28:34.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:32.705667+0000 mgr.smithi042.kfoihb (mgr.24435) 116 : cluster [DBG] pgmap v85: 65 pgs: 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 56 active+clean; 291 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 460 op/s; 1274/26403 objects degraded (4.825%); 0 B/s, 29 objects/s recovering 2024-02-15T18:28:34.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:32.705667+0000 mgr.smithi042.kfoihb (mgr.24435) 116 : cluster [DBG] pgmap v85: 65 pgs: 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 56 active+clean; 291 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 460 op/s; 1274/26403 objects degraded (4.825%); 0 B/s, 29 objects/s recovering 2024-02-15T18:28:36.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:34.706183+0000 mgr.smithi042.kfoihb (mgr.24435) 117 : cluster [DBG] pgmap v86: 65 pgs: 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 56 active+clean; 291 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 390 op/s; 1274/26403 objects degraded (4.825%); 0 B/s, 25 objects/s recovering 2024-02-15T18:28:36.197 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:34.867923+0000 mon.smithi042 (mon.0) 132 : cluster [DBG] osdmap e59: 6 total, 6 up, 6 in 2024-02-15T18:28:36.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:34.706183+0000 mgr.smithi042.kfoihb (mgr.24435) 117 : cluster [DBG] pgmap v86: 65 pgs: 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 56 active+clean; 291 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 390 op/s; 1274/26403 objects degraded (4.825%); 0 B/s, 25 objects/s recovering 2024-02-15T18:28:36.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:34.867923+0000 mon.smithi042 (mon.0) 132 : cluster [DBG] osdmap e59: 6 total, 6 up, 6 in 2024-02-15T18:28:37.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:35.872337+0000 mon.smithi042 (mon.0) 133 : cluster [DBG] osdmap e60: 6 total, 6 up, 6 in 2024-02-15T18:28:37.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:35.945466+0000 mon.smithi042 (mon.0) 134 : cluster [WRN] Health check update: Degraded data redundancy: 1274/26403 objects degraded (4.825%), 8 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:37.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:35.872337+0000 mon.smithi042 (mon.0) 133 : cluster [DBG] osdmap e60: 6 total, 6 up, 6 in 2024-02-15T18:28:37.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:35.945466+0000 mon.smithi042 (mon.0) 134 : cluster [WRN] Health check update: Degraded data redundancy: 1274/26403 objects degraded (4.825%), 8 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:38.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:36.706653+0000 mgr.smithi042.kfoihb (mgr.24435) 118 : cluster [DBG] pgmap v89: 65 pgs: 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 56 active+clean; 292 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 518 op/s; 1274/26019 objects degraded (4.896%); 0 B/s, 7 objects/s recovering 2024-02-15T18:28:38.310 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:36.706653+0000 mgr.smithi042.kfoihb (mgr.24435) 118 : cluster [DBG] pgmap v89: 65 pgs: 1 active+recovering+undersized+remapped, 8 active+recovery_wait+undersized+degraded+remapped, 56 active+clean; 292 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 518 op/s; 1274/26019 objects degraded (4.896%); 0 B/s, 7 objects/s recovering 2024-02-15T18:28:40.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:38.707259+0000 mgr.smithi042.kfoihb (mgr.24435) 119 : cluster [DBG] pgmap v90: 65 pgs: 1 active+recovering+undersized+remapped, 7 active+recovery_wait+undersized+degraded+remapped, 57 active+clean; 280 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 1.31k op/s; 1102/23841 objects degraded (4.622%); 0 B/s, 13 objects/s recovering 2024-02-15T18:28:40.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:38.707259+0000 mgr.smithi042.kfoihb (mgr.24435) 119 : cluster [DBG] pgmap v90: 65 pgs: 1 active+recovering+undersized+remapped, 7 active+recovery_wait+undersized+degraded+remapped, 57 active+clean; 280 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 1.31k op/s; 1102/23841 objects degraded (4.622%); 0 B/s, 13 objects/s recovering 2024-02-15T18:28:42.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:40.707624+0000 mgr.smithi042.kfoihb (mgr.24435) 120 : cluster [DBG] pgmap v91: 65 pgs: 1 active+recovering+undersized+remapped, 7 active+recovery_wait+undersized+degraded+remapped, 57 active+clean; 280 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 945 KiB/s rd, 1.1 MiB/s wr, 1.06k op/s; 1102/23841 objects degraded (4.622%); 0 B/s, 6 objects/s recovering 2024-02-15T18:28:42.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:40.946561+0000 mon.smithi042 (mon.0) 135 : cluster [WRN] Health check update: Degraded data redundancy: 1102/23841 objects degraded (4.622%), 7 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:42.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:40.707624+0000 mgr.smithi042.kfoihb (mgr.24435) 120 : cluster [DBG] pgmap v91: 65 pgs: 1 active+recovering+undersized+remapped, 7 active+recovery_wait+undersized+degraded+remapped, 57 active+clean; 280 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 945 KiB/s rd, 1.1 MiB/s wr, 1.06k op/s; 1102/23841 objects degraded (4.622%); 0 B/s, 6 objects/s recovering 2024-02-15T18:28:42.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:40.946561+0000 mon.smithi042 (mon.0) 135 : cluster [WRN] Health check update: Degraded data redundancy: 1102/23841 objects degraded (4.622%), 7 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:44.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:42.708170+0000 mgr.smithi042.kfoihb (mgr.24435) 121 : cluster [DBG] pgmap v92: 65 pgs: 1 active+recovering+undersized+remapped, 7 active+recovery_wait+undersized+degraded+remapped, 57 active+clean; 266 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 1.58k op/s; 1102/19623 objects degraded (5.616%); 0 B/s, 14 objects/s recovering 2024-02-15T18:28:44.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:42.708170+0000 mgr.smithi042.kfoihb (mgr.24435) 121 : cluster [DBG] pgmap v92: 65 pgs: 1 active+recovering+undersized+remapped, 7 active+recovery_wait+undersized+degraded+remapped, 57 active+clean; 266 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 1.58k op/s; 1102/19623 objects degraded (5.616%); 0 B/s, 14 objects/s recovering 2024-02-15T18:28:46.178 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:44.708519+0000 mgr.smithi042.kfoihb (mgr.24435) 122 : cluster [DBG] pgmap v93: 65 pgs: 1 active+recovering+undersized+remapped, 7 active+recovery_wait+undersized+degraded+remapped, 57 active+clean; 266 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 1.29k op/s; 1102/19623 objects degraded (5.616%); 0 B/s, 11 objects/s recovering 2024-02-15T18:28:46.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:44.708519+0000 mgr.smithi042.kfoihb (mgr.24435) 122 : cluster [DBG] pgmap v93: 65 pgs: 1 active+recovering+undersized+remapped, 7 active+recovery_wait+undersized+degraded+remapped, 57 active+clean; 266 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 1.29k op/s; 1102/19623 objects degraded (5.616%); 0 B/s, 11 objects/s recovering 2024-02-15T18:28:46.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.903+0000 7fdc4708e700 1 -- 172.21.15.42:0/3135507984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc400f55c0 msgr2=0x7fdc400f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:46.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.903+0000 7fdc4708e700 1 --2- 172.21.15.42:0/3135507984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc400f55c0 0x7fdc400f59e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fdc34004660 tx=0x7fdc34009b30).stop 2024-02-15T18:28:46.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.903+0000 7fdc4708e700 1 -- 172.21.15.42:0/3135507984 shutdown_connections 2024-02-15T18:28:46.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.903+0000 7fdc4708e700 1 --2- 172.21.15.42:0/3135507984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc400f6860 0x7fdc400f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:46.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.903+0000 7fdc4708e700 1 --2- 172.21.15.42:0/3135507984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc400f55c0 0x7fdc400f59e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:46.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.903+0000 7fdc4708e700 1 -- 172.21.15.42:0/3135507984 >> 172.21.15.42:0/3135507984 conn(0x7fdc400f0d40 msgr2=0x7fdc400f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:46.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.903+0000 7fdc4708e700 1 -- 172.21.15.42:0/3135507984 shutdown_connections 2024-02-15T18:28:46.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.903+0000 7fdc4708e700 1 -- 172.21.15.42:0/3135507984 wait complete. 2024-02-15T18:28:46.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.904+0000 7fdc4708e700 1 Processor -- start 2024-02-15T18:28:46.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.904+0000 7fdc4708e700 1 -- start start 2024-02-15T18:28:46.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.904+0000 7fdc4708e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc400f55c0 0x7fdc40106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:46.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.904+0000 7fdc4708e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc400f6860 0x7fdc401074e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:46.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.904+0000 7fdc4708e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc40107a90 con 0x7fdc400f55c0 2024-02-15T18:28:46.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.904+0000 7fdc4708e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc40107bd0 con 0x7fdc400f6860 2024-02-15T18:28:46.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.904+0000 7fdc44e2a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc400f55c0 0x7fdc40106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:46.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.904+0000 7fdc44e2a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc400f55c0 0x7fdc40106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33972/0 (socket says 172.21.15.42:33972) 2024-02-15T18:28:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.904+0000 7fdc44e2a700 1 -- 172.21.15.42:0/1978215516 learned_addr learned my addr 172.21.15.42:0/1978215516 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:28:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.904+0000 7fdc3ffff700 1 --2- 172.21.15.42:0/1978215516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc400f6860 0x7fdc401074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.905+0000 7fdc44e2a700 1 -- 172.21.15.42:0/1978215516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc400f6860 msgr2=0x7fdc401074e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.905+0000 7fdc44e2a700 1 --2- 172.21.15.42:0/1978215516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc400f6860 0x7fdc401074e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.905+0000 7fdc44e2a700 1 -- 172.21.15.42:0/1978215516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc34005040 con 0x7fdc400f55c0 2024-02-15T18:28:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.905+0000 7fdc44e2a700 1 --2- 172.21.15.42:0/1978215516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc400f55c0 0x7fdc40106ff0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fdc34002040 tx=0x7fdc3400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.905+0000 7fdc3d7fa700 1 -- 172.21.15.42:0/1978215516 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc34019070 con 0x7fdc400f55c0 2024-02-15T18:28:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.905+0000 7fdc3d7fa700 1 -- 172.21.15.42:0/1978215516 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fdc34007e80 con 0x7fdc400f55c0 2024-02-15T18:28:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.905+0000 7fdc4708e700 1 -- 172.21.15.42:0/1978215516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc40108510 con 0x7fdc400f55c0 2024-02-15T18:28:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.905+0000 7fdc3d7fa700 1 -- 172.21.15.42:0/1978215516 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc34014db0 con 0x7fdc400f55c0 2024-02-15T18:28:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.905+0000 7fdc4708e700 1 -- 172.21.15.42:0/1978215516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc40108a70 con 0x7fdc400f55c0 2024-02-15T18:28:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.906+0000 7fdc4708e700 1 -- 172.21.15.42:0/1978215516 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc40045ad0 con 0x7fdc400f55c0 2024-02-15T18:28:46.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.906+0000 7fdc3d7fa700 1 -- 172.21.15.42:0/1978215516 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fdc34025070 con 0x7fdc400f55c0 2024-02-15T18:28:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.907+0000 7fdc3d7fa700 1 --2- 172.21.15.42:0/1978215516 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdc30060f20 0x7fdc300633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.907+0000 7fdc3d7fa700 1 -- 172.21.15.42:0/1978215516 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6044+0+0 (secure 0 0 0) 0x7fdc3407cfd0 con 0x7fdc400f55c0 2024-02-15T18:28:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.907+0000 7fdc3ffff700 1 --2- 172.21.15.42:0/1978215516 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdc30060f20 0x7fdc300633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:46.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.907+0000 7fdc3ffff700 1 --2- 172.21.15.42:0/1978215516 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdc30060f20 0x7fdc300633e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fdc2c009fd0 tx=0x7fdc2c00b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:46.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:46.911+0000 7fdc3d7fa700 1 -- 172.21.15.42:0/1978215516 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdc34050dc0 con 0x7fdc400f55c0 2024-02-15T18:28:47.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.137+0000 7fdc4708e700 1 -- 172.21.15.42:0/1978215516 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc400faca0 con 0x7fdc30060f20 2024-02-15T18:28:47.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.139+0000 7fdc3d7fa700 1 -- 172.21.15.42:0/1978215516 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdc400faca0 con 0x7fdc30060f20 2024-02-15T18:28:47.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.141+0000 7fdc4708e700 1 -- 172.21.15.42:0/1978215516 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdc30060f20 msgr2=0x7fdc300633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:47.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.141+0000 7fdc4708e700 1 --2- 172.21.15.42:0/1978215516 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdc30060f20 0x7fdc300633e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fdc2c009fd0 tx=0x7fdc2c00b040).stop 2024-02-15T18:28:47.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.141+0000 7fdc4708e700 1 -- 172.21.15.42:0/1978215516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc400f55c0 msgr2=0x7fdc40106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:47.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.141+0000 7fdc4708e700 1 --2- 172.21.15.42:0/1978215516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc400f55c0 0x7fdc40106ff0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fdc34002040 tx=0x7fdc3400b040).stop 2024-02-15T18:28:47.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.141+0000 7fdc4708e700 1 -- 172.21.15.42:0/1978215516 shutdown_connections 2024-02-15T18:28:47.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.141+0000 7fdc4708e700 1 --2- 172.21.15.42:0/1978215516 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdc30060f20 0x7fdc300633e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.141+0000 7fdc4708e700 1 --2- 172.21.15.42:0/1978215516 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdc400f55c0 0x7fdc40106ff0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.141+0000 7fdc4708e700 1 --2- 172.21.15.42:0/1978215516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc400f6860 0x7fdc401074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.141+0000 7fdc4708e700 1 -- 172.21.15.42:0/1978215516 >> 172.21.15.42:0/1978215516 conn(0x7fdc400f0d40 msgr2=0x7fdc4005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:47.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.141+0000 7fdc4708e700 1 -- 172.21.15.42:0/1978215516 shutdown_connections 2024-02-15T18:28:47.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.141+0000 7fdc4708e700 1 -- 172.21.15.42:0/1978215516 wait complete. 2024-02-15T18:28:47.151 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:28:47.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:46 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:45.947368+0000 mon.smithi042 (mon.0) 136 : cluster [WRN] Health check update: Degraded data redundancy: 1102/19623 objects degraded (5.616%), 7 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:47.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:46 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:46.183439+0000 mon.smithi042 (mon.0) 137 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:28:47.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.288+0000 7f84d643b700 1 -- 172.21.15.42:0/1398136433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d00f77d0 msgr2=0x7f84d00f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:47.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.288+0000 7f84d643b700 1 --2- 172.21.15.42:0/1398136433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d00f77d0 0x7f84d00f7c30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f84c0004660 tx=0x7f84c0009b30).stop 2024-02-15T18:28:47.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.288+0000 7f84d643b700 1 -- 172.21.15.42:0/1398136433 shutdown_connections 2024-02-15T18:28:47.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.288+0000 7f84d643b700 1 --2- 172.21.15.42:0/1398136433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d00f77d0 0x7f84d00f7c30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.288+0000 7f84d643b700 1 --2- 172.21.15.42:0/1398136433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84d00f6530 0x7f84d00f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.288+0000 7f84d643b700 1 -- 172.21.15.42:0/1398136433 >> 172.21.15.42:0/1398136433 conn(0x7f84d00f1c90 msgr2=0x7f84d00f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:47.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.288+0000 7f84d643b700 1 -- 172.21.15.42:0/1398136433 shutdown_connections 2024-02-15T18:28:47.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.288+0000 7f84d643b700 1 -- 172.21.15.42:0/1398136433 wait complete. 2024-02-15T18:28:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.289+0000 7f84d643b700 1 Processor -- start 2024-02-15T18:28:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.289+0000 7f84d643b700 1 -- start start 2024-02-15T18:28:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.289+0000 7f84d643b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d00f6530 0x7f84d0069ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.289+0000 7f84d643b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84d00f77d0 0x7f84d006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.289+0000 7f84d643b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d010aa40 con 0x7f84d00f6530 2024-02-15T18:28:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.289+0000 7f84d643b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d010ab80 con 0x7f84d00f77d0 2024-02-15T18:28:47.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.289+0000 7f84d5439700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d00f6530 0x7f84d0069ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:47.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.289+0000 7f84d5439700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d00f6530 0x7f84d0069ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33982/0 (socket says 172.21.15.42:33982) 2024-02-15T18:28:47.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.289+0000 7f84d5439700 1 -- 172.21.15.42:0/1278825774 learned_addr learned my addr 172.21.15.42:0/1278825774 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:28:47.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.289+0000 7f84d5439700 1 -- 172.21.15.42:0/1278825774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84d00f77d0 msgr2=0x7f84d006a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:47.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.290+0000 7f84d5439700 1 --2- 172.21.15.42:0/1278825774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84d00f77d0 0x7f84d006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:28:47.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.290+0000 7f84d5439700 1 -- 172.21.15.42:0/1278825774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c0005040 con 0x7f84d00f6530 2024-02-15T18:28:47.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.290+0000 7f84d5439700 1 --2- 172.21.15.42:0/1278825774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d00f6530 0x7f84d0069ec0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f84cc00e7c0 tx=0x7f84cc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:47.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.290+0000 7f84c5ffb700 1 -- 172.21.15.42:0/1278825774 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84cc0092e0 con 0x7f84d00f6530 2024-02-15T18:28:47.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.290+0000 7f84c5ffb700 1 -- 172.21.15.42:0/1278825774 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f84cc00f040 con 0x7f84d00f6530 2024-02-15T18:28:47.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.290+0000 7f84d643b700 1 -- 172.21.15.42:0/1278825774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d010ae30 con 0x7f84d00f6530 2024-02-15T18:28:47.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.290+0000 7f84c5ffb700 1 -- 172.21.15.42:0/1278825774 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84cc014880 con 0x7f84d00f6530 2024-02-15T18:28:47.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.290+0000 7f84d643b700 1 -- 172.21.15.42:0/1278825774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d010b3f0 con 0x7f84d00f6530 2024-02-15T18:28:47.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.291+0000 7f84d643b700 1 -- 172.21.15.42:0/1278825774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d0045ad0 con 0x7f84d00f6530 2024-02-15T18:28:47.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.292+0000 7f84c5ffb700 1 -- 172.21.15.42:0/1278825774 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f84cc01b030 con 0x7f84d00f6530 2024-02-15T18:28:47.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.292+0000 7f84c5ffb700 1 --2- 172.21.15.42:0/1278825774 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f84bc060f70 0x7f84bc063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:47.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.292+0000 7f84c5ffb700 1 -- 172.21.15.42:0/1278825774 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6044+0+0 (secure 0 0 0) 0x7f84cc07c010 con 0x7f84d00f6530 2024-02-15T18:28:47.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.296+0000 7f84d4c38700 1 --2- 172.21.15.42:0/1278825774 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f84bc060f70 0x7f84bc063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:47.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.296+0000 7f84d4c38700 1 --2- 172.21.15.42:0/1278825774 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f84bc060f70 0x7f84bc063430 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f84c0005010 tx=0x7f84c000b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:47.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.296+0000 7f84c5ffb700 1 -- 172.21.15.42:0/1278825774 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f84cc04c460 con 0x7f84d00f6530 2024-02-15T18:28:47.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:45.947368+0000 mon.smithi042 (mon.0) 136 : cluster [WRN] Health check update: Degraded data redundancy: 1102/19623 objects degraded (5.616%), 7 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:47.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:46.183439+0000 mon.smithi042 (mon.0) 137 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:28:47.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.491+0000 7f84d643b700 1 -- 172.21.15.42:0/1278825774 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84d006a8a0 con 0x7f84bc060f70 2024-02-15T18:28:47.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.492+0000 7f84c5ffb700 1 -- 172.21.15.42:0/1278825774 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f84d006a8a0 con 0x7f84bc060f70 2024-02-15T18:28:47.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.494+0000 7f84d643b700 1 -- 172.21.15.42:0/1278825774 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f84bc060f70 msgr2=0x7f84bc063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:47.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.494+0000 7f84d643b700 1 --2- 172.21.15.42:0/1278825774 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f84bc060f70 0x7f84bc063430 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f84c0005010 tx=0x7f84c000b040).stop 2024-02-15T18:28:47.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.494+0000 7f84d643b700 1 -- 172.21.15.42:0/1278825774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d00f6530 msgr2=0x7f84d0069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:47.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.494+0000 7f84d643b700 1 --2- 172.21.15.42:0/1278825774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d00f6530 0x7f84d0069ec0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f84cc00e7c0 tx=0x7f84cc00b040).stop 2024-02-15T18:28:47.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.494+0000 7f84d643b700 1 -- 172.21.15.42:0/1278825774 shutdown_connections 2024-02-15T18:28:47.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.494+0000 7f84d643b700 1 --2- 172.21.15.42:0/1278825774 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f84bc060f70 0x7f84bc063430 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.494+0000 7f84d643b700 1 --2- 172.21.15.42:0/1278825774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d00f6530 0x7f84d0069ec0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.494+0000 7f84d643b700 1 --2- 172.21.15.42:0/1278825774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84d00f77d0 0x7f84d006a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:28:47.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.494+0000 7f84d643b700 1 -- 172.21.15.42:0/1278825774 >> 172.21.15.42:0/1278825774 conn(0x7f84d00f1c90 msgr2=0x7f84d00fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:47.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.494+0000 7f84d643b700 1 -- 172.21.15.42:0/1278825774 shutdown_connections 2024-02-15T18:28:47.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.494+0000 7f84d643b700 1 -- 172.21.15.42:0/1278825774 wait complete. 2024-02-15T18:28:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.597+0000 7fc632515700 1 -- 172.21.15.42:0/1079165525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc62c0f66e0 msgr2=0x7fc62c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.597+0000 7fc632515700 1 --2- 172.21.15.42:0/1079165525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc62c0f66e0 0x7fc62c0f6b00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fc61c004660 tx=0x7fc61c009b30).stop 2024-02-15T18:28:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.597+0000 7fc632515700 1 -- 172.21.15.42:0/1079165525 shutdown_connections 2024-02-15T18:28:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.597+0000 7fc632515700 1 --2- 172.21.15.42:0/1079165525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc62c0f7980 0x7fc62c0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.597+0000 7fc632515700 1 --2- 172.21.15.42:0/1079165525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc62c0f66e0 0x7fc62c0f6b00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.597+0000 7fc632515700 1 -- 172.21.15.42:0/1079165525 >> 172.21.15.42:0/1079165525 conn(0x7fc62c0f1e40 msgr2=0x7fc62c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.597+0000 7fc632515700 1 -- 172.21.15.42:0/1079165525 shutdown_connections 2024-02-15T18:28:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.597+0000 7fc632515700 1 -- 172.21.15.42:0/1079165525 wait complete. 2024-02-15T18:28:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.597+0000 7fc632515700 1 Processor -- start 2024-02-15T18:28:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.598+0000 7fc632515700 1 -- start start 2024-02-15T18:28:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.598+0000 7fc632515700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc62c0f66e0 0x7fc62c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.598+0000 7fc632515700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc62c0f7980 0x7fc62c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.598+0000 7fc632515700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c06a380 con 0x7fc62c0f66e0 2024-02-15T18:28:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.598+0000 7fc632515700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c06a4c0 con 0x7fc62c0f7980 2024-02-15T18:28:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.598+0000 7fc62bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc62c0f66e0 0x7fc62c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.598+0000 7fc62bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc62c0f66e0 0x7fc62c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34004/0 (socket says 172.21.15.42:34004) 2024-02-15T18:28:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.598+0000 7fc62bfff700 1 -- 172.21.15.42:0/1424520326 learned_addr learned my addr 172.21.15.42:0/1424520326 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:28:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.598+0000 7fc62b7fe700 1 --2- 172.21.15.42:0/1424520326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc62c0f7980 0x7fc62c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.598+0000 7fc62bfff700 1 -- 172.21.15.42:0/1424520326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc62c0f7980 msgr2=0x7fc62c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.598+0000 7fc62bfff700 1 --2- 172.21.15.42:0/1424520326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc62c0f7980 0x7fc62c069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.598+0000 7fc62bfff700 1 -- 172.21.15.42:0/1424520326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61c005040 con 0x7fc62c0f66e0 2024-02-15T18:28:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.598+0000 7fc62bfff700 1 --2- 172.21.15.42:0/1424520326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc62c0f66e0 0x7fc62c06b9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fc61c002040 tx=0x7fc61c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:47.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.599+0000 7fc628ff9700 1 -- 172.21.15.42:0/1424520326 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc61c019070 con 0x7fc62c0f66e0 2024-02-15T18:28:47.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.599+0000 7fc632515700 1 -- 172.21.15.42:0/1424520326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62c06a740 con 0x7fc62c0f66e0 2024-02-15T18:28:47.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.599+0000 7fc628ff9700 1 -- 172.21.15.42:0/1424520326 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fc61c007e80 con 0x7fc62c0f66e0 2024-02-15T18:28:47.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.599+0000 7fc628ff9700 1 -- 172.21.15.42:0/1424520326 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc61c014db0 con 0x7fc62c0f66e0 2024-02-15T18:28:47.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.599+0000 7fc632515700 1 -- 172.21.15.42:0/1424520326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62c06aca0 con 0x7fc62c0f66e0 2024-02-15T18:28:47.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.600+0000 7fc632515700 1 -- 172.21.15.42:0/1424520326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62c045ad0 con 0x7fc62c0f66e0 2024-02-15T18:28:47.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.600+0000 7fc628ff9700 1 -- 172.21.15.42:0/1424520326 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fc61c025070 con 0x7fc62c0f66e0 2024-02-15T18:28:47.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.600+0000 7fc628ff9700 1 --2- 172.21.15.42:0/1424520326 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc614060ec0 0x7fc614063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:47.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.601+0000 7fc628ff9700 1 -- 172.21.15.42:0/1424520326 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6044+0+0 (secure 0 0 0) 0x7fc61c07d0d0 con 0x7fc62c0f66e0 2024-02-15T18:28:47.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.601+0000 7fc62b7fe700 1 --2- 172.21.15.42:0/1424520326 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc614060ec0 0x7fc614063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:47.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.601+0000 7fc62b7fe700 1 --2- 172.21.15.42:0/1424520326 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc614060ec0 0x7fc614063380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fc620009fd0 tx=0x7fc62000b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:47.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.605+0000 7fc628ff9700 1 -- 172.21.15.42:0/1424520326 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc61c04d410 con 0x7fc62c0f66e0 2024-02-15T18:28:47.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.789+0000 7fc632515700 1 -- 172.21.15.42:0/1424520326 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc62c057f10 con 0x7fc614060ec0 2024-02-15T18:28:47.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.799+0000 7fc628ff9700 1 -- 172.21.15.42:0/1424520326 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fc62c057f10 con 0x7fc614060ec0 2024-02-15T18:28:47.794 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:28:47.794 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (8m) 31s ago 10m 21.6M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:28:47.794 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (64s) 31s ago 10m 6643k - 16.2.14-521-g0e714d9a 1fbc92c859dc 2da9d875539a 2024-02-15T18:28:47.794 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (57s) 50s ago 9m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 5c46a68ff62b 2024-02-15T18:28:47.794 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (8m) 31s ago 10m 33.7M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:28:47.794 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (4m) 31s ago 4m 14.3M - 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (5m) 31s ago 5m 17.0M - 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (4m) 50s ago 4m 154M - 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (5m) 50s ago 5m 227M - 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (2m) 31s ago 11m 430M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (2m) 50s ago 9m 388M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (2m) 31s ago 11m 44.5M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (82s) 50s ago 9m 25.8M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (9m) 31s ago 9m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (8m) 50s ago 8m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (40s) 31s ago 8m 11.6M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc f226bea76cab 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (8m) 31s ago 8m 355M 4096M 16.2.4 8d91d370c2b8 2b3ae86111e8 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (7m) 31s ago 7m 364M 4096M 16.2.4 8d91d370c2b8 3706071e4698 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (7m) 50s ago 7m 432M 4096M 16.2.4 8d91d370c2b8 e6fe8e77c6fe 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (7m) 50s ago 7m 428M 4096M 16.2.4 8d91d370c2b8 24582012e234 2024-02-15T18:28:47.795 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (6m) 50s ago 6m 347M 4096M 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:28:47.796 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (2m) 31s ago 9m 60.5M - 2.18.1 de242295e225 63b6dfb825ed 2024-02-15T18:28:47.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.801+0000 7fc632515700 1 -- 172.21.15.42:0/1424520326 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc614060ec0 msgr2=0x7fc614063380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:47.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.801+0000 7fc632515700 1 --2- 172.21.15.42:0/1424520326 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc614060ec0 0x7fc614063380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fc620009fd0 tx=0x7fc62000b040).stop 2024-02-15T18:28:47.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.801+0000 7fc632515700 1 -- 172.21.15.42:0/1424520326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc62c0f66e0 msgr2=0x7fc62c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:47.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.801+0000 7fc632515700 1 --2- 172.21.15.42:0/1424520326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc62c0f66e0 0x7fc62c06b9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fc61c002040 tx=0x7fc61c00b040).stop 2024-02-15T18:28:47.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.801+0000 7fc632515700 1 -- 172.21.15.42:0/1424520326 shutdown_connections 2024-02-15T18:28:47.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.801+0000 7fc632515700 1 --2- 172.21.15.42:0/1424520326 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc614060ec0 0x7fc614063380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.801+0000 7fc632515700 1 --2- 172.21.15.42:0/1424520326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc62c0f66e0 0x7fc62c06b9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.801+0000 7fc632515700 1 --2- 172.21.15.42:0/1424520326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc62c0f7980 0x7fc62c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.801+0000 7fc632515700 1 -- 172.21.15.42:0/1424520326 >> 172.21.15.42:0/1424520326 conn(0x7fc62c0f1e40 msgr2=0x7fc62c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:47.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.801+0000 7fc632515700 1 -- 172.21.15.42:0/1424520326 shutdown_connections 2024-02-15T18:28:47.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.801+0000 7fc632515700 1 -- 172.21.15.42:0/1424520326 wait complete. 2024-02-15T18:28:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.904+0000 7f3278a44700 1 -- 172.21.15.42:0/3566883358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32740f77d0 msgr2=0x7f32740f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.904+0000 7f3278a44700 1 --2- 172.21.15.42:0/3566883358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32740f77d0 0x7f32740f7c30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f3268004660 tx=0x7f3268009b30).stop 2024-02-15T18:28:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.904+0000 7f3278a44700 1 -- 172.21.15.42:0/3566883358 shutdown_connections 2024-02-15T18:28:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.904+0000 7f3278a44700 1 --2- 172.21.15.42:0/3566883358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32740f77d0 0x7f32740f7c30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.904+0000 7f3278a44700 1 --2- 172.21.15.42:0/3566883358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32740f6530 0x7f32740f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.904+0000 7f3278a44700 1 -- 172.21.15.42:0/3566883358 >> 172.21.15.42:0/3566883358 conn(0x7f32740f1c90 msgr2=0x7f32740f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.904+0000 7f3278a44700 1 -- 172.21.15.42:0/3566883358 shutdown_connections 2024-02-15T18:28:47.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.904+0000 7f3278a44700 1 -- 172.21.15.42:0/3566883358 wait complete. 2024-02-15T18:28:47.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.905+0000 7f3278a44700 1 Processor -- start 2024-02-15T18:28:47.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.905+0000 7f3278a44700 1 -- start start 2024-02-15T18:28:47.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.905+0000 7f3278a44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32740f6530 0x7f3274106e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:47.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.905+0000 7f3278a44700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32740f77d0 0x7f3274107310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:47.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.905+0000 7f3278a44700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3274107910 con 0x7f32740f6530 2024-02-15T18:28:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.905+0000 7f3278a44700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3274108120 con 0x7f32740f77d0 2024-02-15T18:28:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.906+0000 7f3272d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32740f6530 0x7f3274106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.906+0000 7f3272d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32740f6530 0x7f3274106e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34012/0 (socket says 172.21.15.42:34012) 2024-02-15T18:28:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.906+0000 7f3272d9d700 1 -- 172.21.15.42:0/3478090758 learned_addr learned my addr 172.21.15.42:0/3478090758 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:28:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.906+0000 7f327259c700 1 --2- 172.21.15.42:0/3478090758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32740f77d0 0x7f3274107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.906+0000 7f3272d9d700 1 -- 172.21.15.42:0/3478090758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32740f77d0 msgr2=0x7f3274107310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.906+0000 7f3272d9d700 1 --2- 172.21.15.42:0/3478090758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32740f77d0 0x7f3274107310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.906+0000 7f3272d9d700 1 -- 172.21.15.42:0/3478090758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3268005040 con 0x7f32740f6530 2024-02-15T18:28:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.906+0000 7f3272d9d700 1 --2- 172.21.15.42:0/3478090758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32740f6530 0x7f3274106e20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f326400e7c0 tx=0x7f326400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.906+0000 7f325b7fe700 1 -- 172.21.15.42:0/3478090758 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32640092e0 con 0x7f32740f6530 2024-02-15T18:28:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.906+0000 7f325b7fe700 1 -- 172.21.15.42:0/3478090758 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f326400f040 con 0x7f32740f6530 2024-02-15T18:28:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.906+0000 7f3278a44700 1 -- 172.21.15.42:0/3478090758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32741083a0 con 0x7f32740f6530 2024-02-15T18:28:47.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.906+0000 7f325b7fe700 1 -- 172.21.15.42:0/3478090758 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3264014920 con 0x7f32740f6530 2024-02-15T18:28:47.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.906+0000 7f3278a44700 1 -- 172.21.15.42:0/3478090758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3274108960 con 0x7f32740f6530 2024-02-15T18:28:47.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.907+0000 7f3278a44700 1 -- 172.21.15.42:0/3478090758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3274059100 con 0x7f32740f6530 2024-02-15T18:28:47.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.908+0000 7f325b7fe700 1 -- 172.21.15.42:0/3478090758 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f326401b030 con 0x7f32740f6530 2024-02-15T18:28:47.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.908+0000 7f325b7fe700 1 --2- 172.21.15.42:0/3478090758 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f325c060f70 0x7f325c063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:47.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.908+0000 7f325b7fe700 1 -- 172.21.15.42:0/3478090758 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6044+0+0 (secure 0 0 0) 0x7f326407c070 con 0x7f32740f6530 2024-02-15T18:28:47.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.908+0000 7f327259c700 1 --2- 172.21.15.42:0/3478090758 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f325c060f70 0x7f325c063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:47.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.908+0000 7f327259c700 1 --2- 172.21.15.42:0/3478090758 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f325c060f70 0x7f325c063430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f3268009720 tx=0x7f3268016040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:47.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:47.913+0000 7f325b7fe700 1 -- 172.21.15.42:0/3478090758 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f326404c410 con 0x7f32740f6530 2024-02-15T18:28:48.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:46.183665+0000 mgr.smithi042.kfoihb (mgr.24435) 123 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:28:48.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:28:46.185599+0000 mgr.smithi042.kfoihb (mgr.24435) 124 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-02-15T18:28:48.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:46.709044+0000 mgr.smithi042.kfoihb (mgr.24435) 125 : cluster [DBG] pgmap v94: 65 pgs: 1 active+recovering+undersized+remapped, 7 active+recovery_wait+undersized+degraded+remapped, 57 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 1.20k op/s; 1102/18822 objects degraded (5.855%); 0 B/s, 10 objects/s recovering 2024-02-15T18:28:48.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.204+0000 7f3278a44700 1 -- 172.21.15.42:0/3478090758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f32740008d0 con 0x7f32740f6530 2024-02-15T18:28:48.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.204+0000 7f325b7fe700 1 -- 172.21.15.42:0/3478090758 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+820 (secure 0 0 0) 0x7f3264019070 con 0x7f32740f6530 2024-02-15T18:28:48.201 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:28:48.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:28:48.202 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:28:48.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:28:48.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:28:48.202 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:28:48.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:28:48.202 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:28:48.202 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 1, 2024-02-15T18:28:48.202 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 5 2024-02-15T18:28:48.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:28:48.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:28:48.202 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:28:48.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:28:48.202 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:28:48.203 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 5, 2024-02-15T18:28:48.203 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 9 2024-02-15T18:28:48.203 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:28:48.203 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:28:48.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.207+0000 7f3278a44700 1 -- 172.21.15.42:0/3478090758 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f325c060f70 msgr2=0x7f325c063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:48.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.207+0000 7f3278a44700 1 --2- 172.21.15.42:0/3478090758 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f325c060f70 0x7f325c063430 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f3268009720 tx=0x7f3268016040).stop 2024-02-15T18:28:48.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.207+0000 7f3278a44700 1 -- 172.21.15.42:0/3478090758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32740f6530 msgr2=0x7f3274106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:48.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.207+0000 7f3278a44700 1 --2- 172.21.15.42:0/3478090758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32740f6530 0x7f3274106e20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f326400e7c0 tx=0x7f326400b040).stop 2024-02-15T18:28:48.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.208+0000 7f3278a44700 1 -- 172.21.15.42:0/3478090758 shutdown_connections 2024-02-15T18:28:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.208+0000 7f3278a44700 1 --2- 172.21.15.42:0/3478090758 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f325c060f70 0x7f325c063430 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.208+0000 7f3278a44700 1 --2- 172.21.15.42:0/3478090758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32740f6530 0x7f3274106e20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.208+0000 7f3278a44700 1 --2- 172.21.15.42:0/3478090758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32740f77d0 0x7f3274107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.208+0000 7f3278a44700 1 -- 172.21.15.42:0/3478090758 >> 172.21.15.42:0/3478090758 conn(0x7f32740f1c90 msgr2=0x7f32740fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.209+0000 7f3278a44700 1 -- 172.21.15.42:0/3478090758 shutdown_connections 2024-02-15T18:28:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.209+0000 7f3278a44700 1 -- 172.21.15.42:0/3478090758 wait complete. 2024-02-15T18:28:48.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:46.183665+0000 mgr.smithi042.kfoihb (mgr.24435) 123 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:28:48.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:28:46.185599+0000 mgr.smithi042.kfoihb (mgr.24435) 124 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-02-15T18:28:48.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:46.709044+0000 mgr.smithi042.kfoihb (mgr.24435) 125 : cluster [DBG] pgmap v94: 65 pgs: 1 active+recovering+undersized+remapped, 7 active+recovery_wait+undersized+degraded+remapped, 57 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 1.20k op/s; 1102/18822 objects degraded (5.855%); 0 B/s, 10 objects/s recovering 2024-02-15T18:28:48.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.359+0000 7fec51acb700 1 -- 172.21.15.42:0/1057855005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec4c0f4bc0 msgr2=0x7fec4c0f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:48.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.359+0000 7fec51acb700 1 --2- 172.21.15.42:0/1057855005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec4c0f4bc0 0x7fec4c0f4fe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fec400046b0 tx=0x7fec4000ac40).stop 2024-02-15T18:28:48.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.360+0000 7fec51acb700 1 -- 172.21.15.42:0/1057855005 shutdown_connections 2024-02-15T18:28:48.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.360+0000 7fec51acb700 1 --2- 172.21.15.42:0/1057855005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec4c0f5e60 0x7fec4c0f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:48.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.360+0000 7fec51acb700 1 --2- 172.21.15.42:0/1057855005 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec4c0f4bc0 0x7fec4c0f4fe0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:48.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.360+0000 7fec51acb700 1 -- 172.21.15.42:0/1057855005 >> 172.21.15.42:0/1057855005 conn(0x7fec4c0f0320 msgr2=0x7fec4c0f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:48.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.362+0000 7fec51acb700 1 -- 172.21.15.42:0/1057855005 shutdown_connections 2024-02-15T18:28:48.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.362+0000 7fec51acb700 1 -- 172.21.15.42:0/1057855005 wait complete. 2024-02-15T18:28:48.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.362+0000 7fec51acb700 1 Processor -- start 2024-02-15T18:28:48.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.362+0000 7fec51acb700 1 -- start start 2024-02-15T18:28:48.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.363+0000 7fec51acb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec4c0f4bc0 0x7fec4c10d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:48.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.363+0000 7fec51acb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec4c0f5e60 0x7fec4c10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:48.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.363+0000 7fec51acb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c10df90 con 0x7fec4c0f4bc0 2024-02-15T18:28:48.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.363+0000 7fec51acb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c10e0d0 con 0x7fec4c0f5e60 2024-02-15T18:28:48.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.363+0000 7fec50ac9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec4c0f4bc0 0x7fec4c10d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:48.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.363+0000 7fec50ac9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec4c0f4bc0 0x7fec4c10d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34038/0 (socket says 172.21.15.42:34038) 2024-02-15T18:28:48.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.363+0000 7fec50ac9700 1 -- 172.21.15.42:0/3594358679 learned_addr learned my addr 172.21.15.42:0/3594358679 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:28:48.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.364+0000 7fec50ac9700 1 -- 172.21.15.42:0/3594358679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec4c0f5e60 msgr2=0x7fec4c10d9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-15T18:28:48.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.364+0000 7fec50ac9700 1 --2- 172.21.15.42:0/3594358679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec4c0f5e60 0x7fec4c10d9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:28:48.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.364+0000 7fec50ac9700 1 -- 172.21.15.42:0/3594358679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec40005040 con 0x7fec4c0f4bc0 2024-02-15T18:28:48.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.364+0000 7fec50ac9700 1 --2- 172.21.15.42:0/3594358679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec4c0f4bc0 0x7fec4c10d4f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fec40000c00 tx=0x7fec4000ac40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:48.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.365+0000 7fec497fa700 1 -- 172.21.15.42:0/3594358679 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec40014a40 con 0x7fec4c0f4bc0 2024-02-15T18:28:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.365+0000 7fec497fa700 1 -- 172.21.15.42:0/3594358679 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fec40014ba0 con 0x7fec4c0f4bc0 2024-02-15T18:28:48.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.365+0000 7fec497fa700 1 -- 172.21.15.42:0/3594358679 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec40015420 con 0x7fec4c0f4bc0 2024-02-15T18:28:48.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.368+0000 7fec51acb700 1 -- 172.21.15.42:0/3594358679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4c10ea10 con 0x7fec4c0f4bc0 2024-02-15T18:28:48.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.368+0000 7fec51acb700 1 -- 172.21.15.42:0/3594358679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4c10ef40 con 0x7fec4c0f4bc0 2024-02-15T18:28:48.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.370+0000 7fec51acb700 1 -- 172.21.15.42:0/3594358679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4c059100 con 0x7fec4c0f4bc0 2024-02-15T18:28:48.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.370+0000 7fec497fa700 1 -- 172.21.15.42:0/3594358679 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fec40014d10 con 0x7fec4c0f4bc0 2024-02-15T18:28:48.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.371+0000 7fec497fa700 1 --2- 172.21.15.42:0/3594358679 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fec3c060f70 0x7fec3c063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:48.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.371+0000 7fec497fa700 1 -- 172.21.15.42:0/3594358679 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6044+0+0 (secure 0 0 0) 0x7fec4007dc70 con 0x7fec4c0f4bc0 2024-02-15T18:28:48.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.372+0000 7fec4bfff700 1 --2- 172.21.15.42:0/3594358679 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fec3c060f70 0x7fec3c063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:48.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.376+0000 7fec4bfff700 1 --2- 172.21.15.42:0/3594358679 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fec3c060f70 0x7fec3c063430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fec38004ae0 tx=0x7fec3800b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:48.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.379+0000 7fec497fa700 1 -- 172.21.15.42:0/3594358679 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fec40084080 con 0x7fec4c0f4bc0 2024-02-15T18:28:48.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.670+0000 7fec51acb700 1 -- 172.21.15.42:0/3594358679 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fec4c0fcd40 con 0x7fec4c0f4bc0 2024-02-15T18:28:48.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.671+0000 7fec497fa700 1 -- 172.21.15.42:0/3594358679 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1773 (secure 0 0 0) 0x7fec4004dfd0 con 0x7fec4c0f4bc0 2024-02-15T18:28:48.666 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:28:48.666 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:28:48.666 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:28:48.666 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:28:48.666 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:28:48.667 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:28:48.667 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:28:48.667 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:28:48.667 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:28:48.667 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:28:48.667 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:28:48.667 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:28:48.667 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:28:48.667 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:28:48.667 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:28:48.667 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:28:48.667 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:28:48.668 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:28:48.668 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:28:48.668 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:28:48.668 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:28:48.668 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:28:48.668 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:28:48.668 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:28:48.668 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:28:48.668 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:28:48.668 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:28:48.668 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:28:48.668 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:28:48.668 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:28:48.669 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:28:48.669 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:28:48.669 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:28:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:28:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:28:48.669 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:28:48.669 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:28:48.669 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:28:48.669 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:28:48.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.673+0000 7fec51acb700 1 -- 172.21.15.42:0/3594358679 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fec3c060f70 msgr2=0x7fec3c063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:48.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.673+0000 7fec51acb700 1 --2- 172.21.15.42:0/3594358679 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fec3c060f70 0x7fec3c063430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fec38004ae0 tx=0x7fec3800b040).stop 2024-02-15T18:28:48.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.673+0000 7fec51acb700 1 -- 172.21.15.42:0/3594358679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec4c0f4bc0 msgr2=0x7fec4c10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:48.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.673+0000 7fec51acb700 1 --2- 172.21.15.42:0/3594358679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec4c0f4bc0 0x7fec4c10d4f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fec40000c00 tx=0x7fec4000ac40).stop 2024-02-15T18:28:48.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.673+0000 7fec51acb700 1 -- 172.21.15.42:0/3594358679 shutdown_connections 2024-02-15T18:28:48.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.673+0000 7fec51acb700 1 --2- 172.21.15.42:0/3594358679 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fec3c060f70 0x7fec3c063430 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:48.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.673+0000 7fec51acb700 1 --2- 172.21.15.42:0/3594358679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec4c0f4bc0 0x7fec4c10d4f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:48.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.673+0000 7fec51acb700 1 --2- 172.21.15.42:0/3594358679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec4c0f5e60 0x7fec4c10d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:28:48.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.673+0000 7fec51acb700 1 -- 172.21.15.42:0/3594358679 >> 172.21.15.42:0/3594358679 conn(0x7fec4c0f0320 msgr2=0x7fec4c0f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:48.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.673+0000 7fec51acb700 1 -- 172.21.15.42:0/3594358679 shutdown_connections 2024-02-15T18:28:48.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.673+0000 7fec51acb700 1 -- 172.21.15.42:0/3594358679 wait complete. 2024-02-15T18:28:48.671 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:28:48.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.783+0000 7f423234c700 1 -- 172.21.15.42:0/579010252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f422c0f49a0 msgr2=0x7f422c0f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:48.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.783+0000 7f423234c700 1 --2- 172.21.15.42:0/579010252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f422c0f49a0 0x7f422c0f4dc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f421c004660 tx=0x7f421c009b30).stop 2024-02-15T18:28:48.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.783+0000 7f423234c700 1 -- 172.21.15.42:0/579010252 shutdown_connections 2024-02-15T18:28:48.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.783+0000 7f423234c700 1 --2- 172.21.15.42:0/579010252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f422c0f5340 0x7f422c0f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:48.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.783+0000 7f423234c700 1 --2- 172.21.15.42:0/579010252 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f422c0f49a0 0x7f422c0f4dc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:48.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.783+0000 7f423234c700 1 -- 172.21.15.42:0/579010252 >> 172.21.15.42:0/579010252 conn(0x7f422c0f04c0 msgr2=0x7f422c0f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:48.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.783+0000 7f423234c700 1 -- 172.21.15.42:0/579010252 shutdown_connections 2024-02-15T18:28:48.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.783+0000 7f423234c700 1 -- 172.21.15.42:0/579010252 wait complete. 2024-02-15T18:28:48.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.784+0000 7f423234c700 1 Processor -- start 2024-02-15T18:28:48.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.784+0000 7f423234c700 1 -- start start 2024-02-15T18:28:48.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.784+0000 7f423234c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f422c0f49a0 0x7f422c1070f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:48.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.784+0000 7f423234c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f422c0f5340 0x7f422c1075e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:48.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.784+0000 7f423234c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c107b90 con 0x7f422c0f49a0 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.784+0000 7f423234c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c107cd0 con 0x7f422c0f5340 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.785+0000 7f422bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f422c0f49a0 0x7f422c1070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.785+0000 7f422bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f422c0f49a0 0x7f422c1070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34056/0 (socket says 172.21.15.42:34056) 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.785+0000 7f422bfff700 1 -- 172.21.15.42:0/2002791794 learned_addr learned my addr 172.21.15.42:0/2002791794 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.785+0000 7f422b7fe700 1 --2- 172.21.15.42:0/2002791794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f422c0f5340 0x7f422c1075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.785+0000 7f422bfff700 1 -- 172.21.15.42:0/2002791794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f422c0f5340 msgr2=0x7f422c1075e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.785+0000 7f422bfff700 1 --2- 172.21.15.42:0/2002791794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f422c0f5340 0x7f422c1075e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.785+0000 7f422bfff700 1 -- 172.21.15.42:0/2002791794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c005040 con 0x7f422c0f49a0 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.785+0000 7f422bfff700 1 --2- 172.21.15.42:0/2002791794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f422c0f49a0 0x7f422c1070f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f421c009830 tx=0x7f421c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.785+0000 7f4228ff9700 1 -- 172.21.15.42:0/2002791794 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f421c018070 con 0x7f422c0f49a0 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.785+0000 7f4228ff9700 1 -- 172.21.15.42:0/2002791794 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f421c007ba0 con 0x7f422c0f49a0 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.785+0000 7f423234c700 1 -- 172.21.15.42:0/2002791794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c108610 con 0x7f422c0f49a0 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.785+0000 7f4228ff9700 1 -- 172.21.15.42:0/2002791794 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f421c01c3c0 con 0x7f422c0f49a0 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.785+0000 7f423234c700 1 -- 172.21.15.42:0/2002791794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c108af0 con 0x7f422c0f49a0 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.786+0000 7f423234c700 1 -- 172.21.15.42:0/2002791794 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422c045ad0 con 0x7f422c0f49a0 2024-02-15T18:28:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.787+0000 7f4228ff9700 1 -- 172.21.15.42:0/2002791794 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f421c01d420 con 0x7f422c0f49a0 2024-02-15T18:28:48.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.787+0000 7f4228ff9700 1 --2- 172.21.15.42:0/2002791794 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4214060f20 0x7f42140633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:28:48.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.787+0000 7f4228ff9700 1 -- 172.21.15.42:0/2002791794 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6044+0+0 (secure 0 0 0) 0x7f421c07de40 con 0x7f422c0f49a0 2024-02-15T18:28:48.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.787+0000 7f422b7fe700 1 --2- 172.21.15.42:0/2002791794 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4214060f20 0x7f42140633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:28:48.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.788+0000 7f422b7fe700 1 --2- 172.21.15.42:0/2002791794 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4214060f20 0x7f42140633e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f422c0f5780 tx=0x7f422000b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:28:48.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.792+0000 7f4228ff9700 1 -- 172.21.15.42:0/2002791794 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f421c051c10 con 0x7f422c0f49a0 2024-02-15T18:28:48.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.992+0000 7f423234c700 1 -- 172.21.15.42:0/2002791794 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f422c0ff170 con 0x7f4214060f20 2024-02-15T18:28:48.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.993+0000 7f4228ff9700 1 -- 172.21.15.42:0/2002791794 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f422c0ff170 con 0x7f4214060f20 2024-02-15T18:28:48.988 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:28:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:28:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:28:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:28:48.988 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:28:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-02-15T18:28:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-02-15T18:28:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-02-15T18:28:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:28:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "7/21 daemons upgraded", 2024-02-15T18:28:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading osd daemons", 2024-02-15T18:28:48.989 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:28:48.989 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:28:48.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.995+0000 7f423234c700 1 -- 172.21.15.42:0/2002791794 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4214060f20 msgr2=0x7f42140633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:48.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.995+0000 7f423234c700 1 --2- 172.21.15.42:0/2002791794 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4214060f20 0x7f42140633e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f422c0f5780 tx=0x7f422000b040).stop 2024-02-15T18:28:48.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.995+0000 7f423234c700 1 -- 172.21.15.42:0/2002791794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f422c0f49a0 msgr2=0x7f422c1070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:28:48.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.995+0000 7f423234c700 1 --2- 172.21.15.42:0/2002791794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f422c0f49a0 0x7f422c1070f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f421c009830 tx=0x7f421c009b30).stop 2024-02-15T18:28:48.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.995+0000 7f423234c700 1 -- 172.21.15.42:0/2002791794 shutdown_connections 2024-02-15T18:28:48.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.995+0000 7f423234c700 1 --2- 172.21.15.42:0/2002791794 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4214060f20 0x7f42140633e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:48.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.995+0000 7f423234c700 1 --2- 172.21.15.42:0/2002791794 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f422c0f49a0 0x7f422c1070f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.995+0000 7f423234c700 1 --2- 172.21.15.42:0/2002791794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f422c0f5340 0x7f422c1075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:28:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.995+0000 7f423234c700 1 -- 172.21.15.42:0/2002791794 >> 172.21.15.42:0/2002791794 conn(0x7f422c0f04c0 msgr2=0x7f422c0fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:28:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.995+0000 7f423234c700 1 -- 172.21.15.42:0/2002791794 shutdown_connections 2024-02-15T18:28:48.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:28:48.995+0000 7f423234c700 1 -- 172.21.15.42:0/2002791794 wait complete. 2024-02-15T18:28:49.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:47.139493+0000 mgr.smithi042.kfoihb (mgr.24435) 126 : audit [DBG] from='client.34172 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:49.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:47.492857+0000 mgr.smithi042.kfoihb (mgr.24435) 127 : audit [DBG] from='client.34176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:49.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:47.791553+0000 mgr.smithi042.kfoihb (mgr.24435) 128 : audit [DBG] from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:49.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:48.205529+0000 mon.smithi042 (mon.0) 138 : audit [DBG] from='client.? 172.21.15.42:0/3478090758' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:28:49.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:48.672060+0000 mon.smithi042 (mon.0) 139 : audit [DBG] from='client.? 172.21.15.42:0/3594358679' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:28:49.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:47.139493+0000 mgr.smithi042.kfoihb (mgr.24435) 126 : audit [DBG] from='client.34172 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:49.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:47.492857+0000 mgr.smithi042.kfoihb (mgr.24435) 127 : audit [DBG] from='client.34176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:49.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:47.791553+0000 mgr.smithi042.kfoihb (mgr.24435) 128 : audit [DBG] from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:49.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:48.205529+0000 mon.smithi042 (mon.0) 138 : audit [DBG] from='client.? 172.21.15.42:0/3478090758' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:28:49.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:48.672060+0000 mon.smithi042 (mon.0) 139 : audit [DBG] from='client.? 172.21.15.42:0/3594358679' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:28:50.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:48.709525+0000 mgr.smithi042.kfoihb (mgr.24435) 129 : cluster [DBG] pgmap v95: 65 pgs: 1 active+recovering+undersized+remapped, 7 active+recovery_wait+undersized+degraded+remapped, 57 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 1.02k op/s; 1102/16164 objects degraded (6.818%); 0 B/s, 14 objects/s recovering 2024-02-15T18:28:50.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:48.709525+0000 mgr.smithi042.kfoihb (mgr.24435) 129 : cluster [DBG] pgmap v95: 65 pgs: 1 active+recovering+undersized+remapped, 7 active+recovery_wait+undersized+degraded+remapped, 57 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 1.02k op/s; 1102/16164 objects degraded (6.818%); 0 B/s, 14 objects/s recovering 2024-02-15T18:28:51.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:28:48.993863+0000 mgr.smithi042.kfoihb (mgr.24435) 130 : audit [DBG] from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:51.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:49.895872+0000 mon.smithi042 (mon.0) 140 : cluster [DBG] osdmap e61: 6 total, 6 up, 6 in 2024-02-15T18:28:51.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:28:48.993863+0000 mgr.smithi042.kfoihb (mgr.24435) 130 : audit [DBG] from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:28:51.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:49.895872+0000 mon.smithi042 (mon.0) 140 : cluster [DBG] osdmap e61: 6 total, 6 up, 6 in 2024-02-15T18:28:52.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:50.709863+0000 mgr.smithi042.kfoihb (mgr.24435) 131 : cluster [DBG] pgmap v97: 65 pgs: 1 active+recovering+undersized+remapped, 7 active+recovery_wait+undersized+degraded+remapped, 57 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 589 op/s; 1102/16164 objects degraded (6.818%); 0 B/s, 11 objects/s recovering 2024-02-15T18:28:52.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:50.900202+0000 mon.smithi042 (mon.0) 141 : cluster [DBG] osdmap e62: 6 total, 6 up, 6 in 2024-02-15T18:28:52.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:50.948030+0000 mon.smithi042 (mon.0) 142 : cluster [WRN] Health check update: Degraded data redundancy: 1102/16164 objects degraded (6.818%), 7 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:52.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:50.709863+0000 mgr.smithi042.kfoihb (mgr.24435) 131 : cluster [DBG] pgmap v97: 65 pgs: 1 active+recovering+undersized+remapped, 7 active+recovery_wait+undersized+degraded+remapped, 57 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 589 op/s; 1102/16164 objects degraded (6.818%); 0 B/s, 11 objects/s recovering 2024-02-15T18:28:52.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:50.900202+0000 mon.smithi042 (mon.0) 141 : cluster [DBG] osdmap e62: 6 total, 6 up, 6 in 2024-02-15T18:28:52.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:50.948030+0000 mon.smithi042 (mon.0) 142 : cluster [WRN] Health check update: Degraded data redundancy: 1102/16164 objects degraded (6.818%), 7 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:53.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:51.903059+0000 mon.smithi042 (mon.0) 143 : cluster [DBG] osdmap e63: 6 total, 6 up, 6 in 2024-02-15T18:28:53.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:51.903059+0000 mon.smithi042 (mon.0) 143 : cluster [DBG] osdmap e63: 6 total, 6 up, 6 in 2024-02-15T18:28:54.200 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:52.710423+0000 mgr.smithi042.kfoihb (mgr.24435) 132 : cluster [DBG] pgmap v100: 65 pgs: 1 peering, 1 active+recovering+undersized+remapped, 4 active+recovery_wait+undersized+degraded+remapped, 59 active+clean; 250 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 523 op/s; 630/12192 objects degraded (5.167%); 0 B/s, 70 objects/s recovering 2024-02-15T18:28:54.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:52.710423+0000 mgr.smithi042.kfoihb (mgr.24435) 132 : cluster [DBG] pgmap v100: 65 pgs: 1 peering, 1 active+recovering+undersized+remapped, 4 active+recovery_wait+undersized+degraded+remapped, 59 active+clean; 250 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 523 op/s; 630/12192 objects degraded (5.167%); 0 B/s, 70 objects/s recovering 2024-02-15T18:28:54.890 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-02-15T18:28:54.890 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-15T18:28:55.209 DEBUG:teuthology.parallel:result is None 2024-02-15T18:28:56.202 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:54.710816+0000 mgr.smithi042.kfoihb (mgr.24435) 133 : cluster [DBG] pgmap v101: 65 pgs: 1 peering, 1 active+recovering+undersized+remapped, 4 active+recovery_wait+undersized+degraded+remapped, 59 active+clean; 250 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 306 op/s; 630/12192 objects degraded (5.167%); 0 B/s, 61 objects/s recovering 2024-02-15T18:28:56.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:54.710816+0000 mgr.smithi042.kfoihb (mgr.24435) 133 : cluster [DBG] pgmap v101: 65 pgs: 1 peering, 1 active+recovering+undersized+remapped, 4 active+recovery_wait+undersized+degraded+remapped, 59 active+clean; 250 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 306 op/s; 630/12192 objects degraded (5.167%); 0 B/s, 61 objects/s recovering 2024-02-15T18:28:57.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:55.948791+0000 mon.smithi042 (mon.0) 144 : cluster [WRN] Health check update: Degraded data redundancy: 630/12192 objects degraded (5.167%), 4 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:57.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:55.948791+0000 mon.smithi042 (mon.0) 144 : cluster [WRN] Health check update: Degraded data redundancy: 630/12192 objects degraded (5.167%), 4 pgs degraded (PG_DEGRADED) 2024-02-15T18:28:58.198 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:56.711200+0000 mgr.smithi042.kfoihb (mgr.24435) 134 : cluster [DBG] pgmap v102: 65 pgs: 1 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 61 active+clean; 246 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 357 op/s; 465/10836 objects degraded (4.291%); 0 B/s, 78 objects/s recovering 2024-02-15T18:28:58.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:56.711200+0000 mgr.smithi042.kfoihb (mgr.24435) 134 : cluster [DBG] pgmap v102: 65 pgs: 1 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 61 active+clean; 246 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 357 op/s; 465/10836 objects degraded (4.291%); 0 B/s, 78 objects/s recovering 2024-02-15T18:29:00.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:28:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:28:58.711604+0000 mgr.smithi042.kfoihb (mgr.24435) 135 : cluster [DBG] pgmap v103: 65 pgs: 1 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 61 active+clean; 231 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 480 op/s; 465/7359 objects degraded (6.319%); 0 B/s, 73 objects/s recovering 2024-02-15T18:29:00.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:28:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:28:58.711604+0000 mgr.smithi042.kfoihb (mgr.24435) 135 : cluster [DBG] pgmap v103: 65 pgs: 1 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 61 active+clean; 231 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 480 op/s; 465/7359 objects degraded (6.319%); 0 B/s, 73 objects/s recovering 2024-02-15T18:29:02.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:00.711913+0000 mgr.smithi042.kfoihb (mgr.24435) 136 : cluster [DBG] pgmap v104: 65 pgs: 1 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 61 active+clean; 231 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 379 op/s; 465/7359 objects degraded (6.319%); 0 B/s, 60 objects/s recovering 2024-02-15T18:29:02.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:00.950937+0000 mon.smithi042 (mon.0) 145 : cluster [WRN] Health check update: Degraded data redundancy: 465/7359 objects degraded (6.319%), 3 pgs degraded (PG_DEGRADED) 2024-02-15T18:29:02.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:01.201112+0000 mon.smithi042 (mon.0) 146 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:29:02.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:00.711913+0000 mgr.smithi042.kfoihb (mgr.24435) 136 : cluster [DBG] pgmap v104: 65 pgs: 1 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 61 active+clean; 231 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 379 op/s; 465/7359 objects degraded (6.319%); 0 B/s, 60 objects/s recovering 2024-02-15T18:29:02.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:00.950937+0000 mon.smithi042 (mon.0) 145 : cluster [WRN] Health check update: Degraded data redundancy: 465/7359 objects degraded (6.319%), 3 pgs degraded (PG_DEGRADED) 2024-02-15T18:29:02.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:01.201112+0000 mon.smithi042 (mon.0) 146 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:29:03.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:01.201268+0000 mgr.smithi042.kfoihb (mgr.24435) 137 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:29:03.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:29:01.202063+0000 mgr.smithi042.kfoihb (mgr.24435) 138 : cephadm [INF] Upgrade: osd.1 is safe to restart 2024-02-15T18:29:03.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:01.201268+0000 mgr.smithi042.kfoihb (mgr.24435) 137 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-15T18:29:03.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:29:01.202063+0000 mgr.smithi042.kfoihb (mgr.24435) 138 : cephadm [INF] Upgrade: osd.1 is safe to restart 2024-02-15T18:29:04.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:02.712543+0000 mgr.smithi042.kfoihb (mgr.24435) 139 : cluster [DBG] pgmap v105: 65 pgs: 1 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 61 active+clean; 217 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 359 op/s; 465/3366 objects degraded (13.815%); 0 B/s, 25 objects/s recovering 2024-02-15T18:29:04.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:02.963593+0000 mon.smithi042 (mon.0) 147 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:04.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:02.964659+0000 mon.smithi042 (mon.0) 148 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-15T18:29:04.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:02.965376+0000 mon.smithi042 (mon.0) 149 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:04.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:02.712543+0000 mgr.smithi042.kfoihb (mgr.24435) 139 : cluster [DBG] pgmap v105: 65 pgs: 1 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 61 active+clean; 217 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 359 op/s; 465/3366 objects degraded (13.815%); 0 B/s, 25 objects/s recovering 2024-02-15T18:29:04.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:02.963593+0000 mon.smithi042 (mon.0) 147 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:04.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:02.964659+0000 mon.smithi042 (mon.0) 148 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-15T18:29:04.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:02.965376+0000 mon.smithi042 (mon.0) 149 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:04.542 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-02-15T18:29:04.543 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-02-15T18:29:04.814 DEBUG:teuthology.parallel:result is None 2024-02-15T18:29:04.815 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-15T18:29:04.852 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-15T18:29:04.852 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-02-15T18:29:04.892 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-02-15T18:29:04.892 DEBUG:teuthology.parallel:result is None 2024-02-15T18:29:05.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:29:02.959460+0000 mgr.smithi042.kfoihb (mgr.24435) 140 : cephadm [INF] Upgrade: Updating osd.1 2024-02-15T18:29:05.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:29:02.966358+0000 mgr.smithi042.kfoihb (mgr.24435) 141 : cephadm [INF] Deploying daemon osd.1 on smithi042 2024-02-15T18:29:05.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:03.964258+0000 mon.smithi042 (mon.0) 150 : cluster [DBG] osdmap e64: 6 total, 6 up, 6 in 2024-02-15T18:29:05.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:29:02.959460+0000 mgr.smithi042.kfoihb (mgr.24435) 140 : cephadm [INF] Upgrade: Updating osd.1 2024-02-15T18:29:05.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:29:02.966358+0000 mgr.smithi042.kfoihb (mgr.24435) 141 : cephadm [INF] Deploying daemon osd.1 on smithi042 2024-02-15T18:29:05.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:03.964258+0000 mon.smithi042 (mon.0) 150 : cluster [DBG] osdmap e64: 6 total, 6 up, 6 in 2024-02-15T18:29:06.122 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:04.712906+0000 mgr.smithi042.kfoihb (mgr.24435) 142 : cluster [DBG] pgmap v107: 65 pgs: 1 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 61 active+clean; 217 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 388 op/s; 465/3366 objects degraded (13.815%); 0 B/s, 28 objects/s recovering 2024-02-15T18:29:06.123 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:04.966737+0000 mon.smithi042 (mon.0) 151 : cluster [DBG] osdmap e65: 6 total, 6 up, 6 in 2024-02-15T18:29:06.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:04.712906+0000 mgr.smithi042.kfoihb (mgr.24435) 142 : cluster [DBG] pgmap v107: 65 pgs: 1 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 61 active+clean; 217 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 388 op/s; 465/3366 objects degraded (13.815%); 0 B/s, 28 objects/s recovering 2024-02-15T18:29:06.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:04.966737+0000 mon.smithi042 (mon.0) 151 : cluster [DBG] osdmap e65: 6 total, 6 up, 6 in 2024-02-15T18:29:06.450 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:06 smithi042 systemd[1]: Stopping Ceph osd.1 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:29:06.807 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.1[52563]: debug 2024-02-15T18:29:06.454+0000 7f0619e1b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:29:06.807 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.1[52563]: debug 2024-02-15T18:29:06.454+0000 7f0619e1b700 -1 osd.1 65 *** Got signal Terminated *** 2024-02-15T18:29:06.807 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.1[52563]: debug 2024-02-15T18:29:06.454+0000 7f0619e1b700 -1 osd.1 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:29:07.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:05.951632+0000 mon.smithi042 (mon.0) 152 : cluster [WRN] Health check update: Degraded data redundancy: 465/3366 objects degraded (13.815%), 3 pgs degraded (PG_DEGRADED) 2024-02-15T18:29:07.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:06.712287+0000 mon.smithi042 (mon.0) 153 : cluster [DBG] osd.1 reported immediately failed by osd.4 2024-02-15T18:29:07.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:06.712315+0000 mon.smithi042 (mon.0) 154 : cluster [INF] osd.1 failed (root=default,host=smithi042) (connection refused reported by osd.4) 2024-02-15T18:29:07.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:06.712431+0000 mon.smithi042 (mon.0) 155 : cluster [DBG] osd.1 reported immediately failed by osd.3 2024-02-15T18:29:07.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:06.712479+0000 mon.smithi042 (mon.0) 156 : cluster [DBG] osd.1 reported immediately failed by osd.5 2024-02-15T18:29:07.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:06.712597+0000 mon.smithi042 (mon.0) 157 : cluster [DBG] osd.1 reported immediately failed by osd.2 2024-02-15T18:29:07.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:06.712677+0000 mon.smithi042 (mon.0) 158 : cluster [DBG] osd.1 reported immediately failed by osd.0 2024-02-15T18:29:07.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:06.712787+0000 mon.smithi042 (mon.0) 159 : cluster [DBG] osd.1 reported immediately failed by osd.4 2024-02-15T18:29:07.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:06.712868+0000 mon.smithi042 (mon.0) 160 : cluster [DBG] osd.1 reported immediately failed by osd.3 2024-02-15T18:29:07.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:06.712941+0000 mon.smithi042 (mon.0) 161 : cluster [DBG] osd.1 reported immediately failed by osd.5 2024-02-15T18:29:07.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:06.713039+0000 mon.smithi042 (mon.0) 162 : cluster [DBG] osd.1 reported immediately failed by osd.2 2024-02-15T18:29:07.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:06.713137+0000 mon.smithi042 (mon.0) 163 : cluster [DBG] osd.1 reported immediately failed by osd.0 2024-02-15T18:29:07.307 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:07 smithi042 bash[83215]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.1 2024-02-15T18:29:07.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:05.951632+0000 mon.smithi042 (mon.0) 152 : cluster [WRN] Health check update: Degraded data redundancy: 465/3366 objects degraded (13.815%), 3 pgs degraded (PG_DEGRADED) 2024-02-15T18:29:07.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:06.712287+0000 mon.smithi042 (mon.0) 153 : cluster [DBG] osd.1 reported immediately failed by osd.4 2024-02-15T18:29:07.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:06.712315+0000 mon.smithi042 (mon.0) 154 : cluster [INF] osd.1 failed (root=default,host=smithi042) (connection refused reported by osd.4) 2024-02-15T18:29:07.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:06.712431+0000 mon.smithi042 (mon.0) 155 : cluster [DBG] osd.1 reported immediately failed by osd.3 2024-02-15T18:29:07.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:06.712479+0000 mon.smithi042 (mon.0) 156 : cluster [DBG] osd.1 reported immediately failed by osd.5 2024-02-15T18:29:07.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:06.712597+0000 mon.smithi042 (mon.0) 157 : cluster [DBG] osd.1 reported immediately failed by osd.2 2024-02-15T18:29:07.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:06.712677+0000 mon.smithi042 (mon.0) 158 : cluster [DBG] osd.1 reported immediately failed by osd.0 2024-02-15T18:29:07.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:06.712787+0000 mon.smithi042 (mon.0) 159 : cluster [DBG] osd.1 reported immediately failed by osd.4 2024-02-15T18:29:07.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:06.712868+0000 mon.smithi042 (mon.0) 160 : cluster [DBG] osd.1 reported immediately failed by osd.3 2024-02-15T18:29:07.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:06.712941+0000 mon.smithi042 (mon.0) 161 : cluster [DBG] osd.1 reported immediately failed by osd.5 2024-02-15T18:29:07.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:06.713039+0000 mon.smithi042 (mon.0) 162 : cluster [DBG] osd.1 reported immediately failed by osd.2 2024-02-15T18:29:07.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:06.713137+0000 mon.smithi042 (mon.0) 163 : cluster [DBG] osd.1 reported immediately failed by osd.0 2024-02-15T18:29:08.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:06.713272+0000 mgr.smithi042.kfoihb (mgr.24435) 143 : cluster [DBG] pgmap v109: 65 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 62 active+clean; 217 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 885 KiB/s rd, 820 KiB/s wr, 250 op/s; 465/3006 objects degraded (15.469%); 0 B/s, 9 objects/s recovering 2024-02-15T18:29:08.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:06.953767+0000 mon.smithi042 (mon.0) 164 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:29:08.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:06.958464+0000 mon.smithi042 (mon.0) 165 : cluster [DBG] osdmap e66: 6 total, 5 up, 6 in 2024-02-15T18:29:08.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:06.713272+0000 mgr.smithi042.kfoihb (mgr.24435) 143 : cluster [DBG] pgmap v109: 65 pgs: 3 active+recovery_wait+undersized+degraded+remapped, 62 active+clean; 217 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 885 KiB/s rd, 820 KiB/s wr, 250 op/s; 465/3006 objects degraded (15.469%); 0 B/s, 9 objects/s recovering 2024-02-15T18:29:08.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:06.953767+0000 mon.smithi042 (mon.0) 164 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:29:08.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:06.958464+0000 mon.smithi042 (mon.0) 165 : cluster [DBG] osdmap e66: 6 total, 5 up, 6 in 2024-02-15T18:29:09.246 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:07.959550+0000 mon.smithi042 (mon.0) 166 : cluster [DBG] osdmap e67: 6 total, 5 up, 6 in 2024-02-15T18:29:09.246 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:08.713712+0000 mgr.smithi042.kfoihb (mgr.24435) 144 : cluster [DBG] pgmap v112: 65 pgs: 12 stale+active+clean, 1 active+recovering+undersized+remapped, 2 active+recovery_wait+undersized+degraded+remapped, 50 active+clean; 209 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 551 KiB/s rd, 607 KiB/s wr, 214 op/s; 295/765 objects degraded (38.562%); 0 B/s, 12 objects/s recovering 2024-02-15T18:29:09.246 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:09 smithi042 systemd[1]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.1.service: Succeeded. 2024-02-15T18:29:09.246 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:09 smithi042 systemd[1]: Stopped Ceph osd.1 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:29:09.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:07.959550+0000 mon.smithi042 (mon.0) 166 : cluster [DBG] osdmap e67: 6 total, 5 up, 6 in 2024-02-15T18:29:09.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:08.713712+0000 mgr.smithi042.kfoihb (mgr.24435) 144 : cluster [DBG] pgmap v112: 65 pgs: 12 stale+active+clean, 1 active+recovering+undersized+remapped, 2 active+recovery_wait+undersized+degraded+remapped, 50 active+clean; 209 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 551 KiB/s rd, 607 KiB/s wr, 214 op/s; 295/765 objects degraded (38.562%); 0 B/s, 12 objects/s recovering 2024-02-15T18:29:09.557 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:09 smithi042 systemd[1]: Starting Ceph osd.1 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:29:11.808 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:11 smithi042 bash[83511]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-02-15T18:29:11.808 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:11 smithi042 bash[83511]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-15T18:29:11.808 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:11 smithi042 bash[83511]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-1 --no-mon-config 2024-02-15T18:29:11.808 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:11 smithi042 bash[83511]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-15T18:29:11.808 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:11 smithi042 bash[83511]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-1/block 2024-02-15T18:29:11.808 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:11 smithi042 bash[83511]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-15T18:29:11.809 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:11 smithi042 bash[83511]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-15T18:29:11.809 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:11 smithi042 bash[83511]: --> ceph-volume lvm activate successful for osd ID: 1 2024-02-15T18:29:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:10.714195+0000 mgr.smithi042.kfoihb (mgr.24435) 145 : cluster [DBG] pgmap v113: 65 pgs: 12 stale+active+clean, 1 active+recovering+undersized+remapped, 2 active+recovery_wait+undersized+degraded+remapped, 50 active+clean; 209 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 436 KiB/s rd, 481 KiB/s wr, 169 op/s; 295/765 objects degraded (38.562%); 0 B/s, 9 objects/s recovering 2024-02-15T18:29:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:10.834425+0000 mon.smithi042 (mon.0) 167 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:10.952821+0000 mon.smithi042 (mon.0) 168 : cluster [WRN] Health check update: Degraded data redundancy: 295/765 objects degraded (38.562%), 2 pgs degraded (PG_DEGRADED) 2024-02-15T18:29:12.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:10.714195+0000 mgr.smithi042.kfoihb (mgr.24435) 145 : cluster [DBG] pgmap v113: 65 pgs: 12 stale+active+clean, 1 active+recovering+undersized+remapped, 2 active+recovery_wait+undersized+degraded+remapped, 50 active+clean; 209 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 436 KiB/s rd, 481 KiB/s wr, 169 op/s; 295/765 objects degraded (38.562%); 0 B/s, 9 objects/s recovering 2024-02-15T18:29:12.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:10.834425+0000 mon.smithi042 (mon.0) 167 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:12.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:10.952821+0000 mon.smithi042 (mon.0) 168 : cluster [WRN] Health check update: Degraded data redundancy: 295/765 objects degraded (38.562%), 2 pgs degraded (PG_DEGRADED) 2024-02-15T18:29:13.057 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-1[83769]: debug 2024-02-15T18:29:12.662+0000 7f333ad3d700 1 -- 172.21.15.42:0/1849859905 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d7d589c680 con 0x55d7d5863400 2024-02-15T18:29:13.057 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:12 smithi042 bash[83754]: 5114ed0267248bf13f3344d1de68574aa7c7e49413ad0b3b62c8a03e031c904e 2024-02-15T18:29:13.057 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:12 smithi042 systemd[1]: Started Ceph osd.1 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:29:13.472 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-1[83769]: debug 2024-02-15T18:29:13.177+0000 7f33427cb380 -1 Falling back to public interface 2024-02-15T18:29:14.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:12.714864+0000 mgr.smithi042.kfoihb (mgr.24435) 146 : cluster [DBG] pgmap v114: 65 pgs: 12 active+undersized+degraded, 1 active+recovering+undersized+remapped, 2 active+recovery_wait+undersized+degraded+remapped, 22 active+undersized, 28 active+clean; 197 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 KiB/s rd, 590 KiB/s wr, 173 op/s; 324/213 objects degraded (152.113%); 0 B/s, 14 objects/s recovering 2024-02-15T18:29:14.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:12.770906+0000 mon.smithi042 (mon.0) 169 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:14.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:12.771649+0000 mon.smithi042 (mon.0) 170 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:29:14.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:12.714864+0000 mgr.smithi042.kfoihb (mgr.24435) 146 : cluster [DBG] pgmap v114: 65 pgs: 12 active+undersized+degraded, 1 active+recovering+undersized+remapped, 2 active+recovery_wait+undersized+degraded+remapped, 22 active+undersized, 28 active+clean; 197 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 KiB/s rd, 590 KiB/s wr, 173 op/s; 324/213 objects degraded (152.113%); 0 B/s, 14 objects/s recovering 2024-02-15T18:29:14.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:12.770906+0000 mon.smithi042 (mon.0) 169 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:14.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:12.771649+0000 mon.smithi042 (mon.0) 170 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:29:16.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:14.715231+0000 mgr.smithi042.kfoihb (mgr.24435) 147 : cluster [DBG] pgmap v115: 65 pgs: 12 active+undersized+degraded, 1 active+recovering+undersized+remapped, 2 active+recovery_wait+undersized+degraded+remapped, 22 active+undersized, 28 active+clean; 197 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 618 KiB/s rd, 572 KiB/s wr, 152 op/s; 324/213 objects degraded (152.113%); 0 B/s, 12 objects/s recovering 2024-02-15T18:29:16.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:14.715231+0000 mgr.smithi042.kfoihb (mgr.24435) 147 : cluster [DBG] pgmap v115: 65 pgs: 12 active+undersized+degraded, 1 active+recovering+undersized+remapped, 2 active+recovery_wait+undersized+degraded+remapped, 22 active+undersized, 28 active+clean; 197 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 618 KiB/s rd, 572 KiB/s wr, 152 op/s; 324/213 objects degraded (152.113%); 0 B/s, 12 objects/s recovering 2024-02-15T18:29:17.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:15.953939+0000 mon.smithi042 (mon.0) 171 : cluster [WRN] Health check update: Degraded data redundancy: 324/213 objects degraded (152.113%), 14 pgs degraded (PG_DEGRADED) 2024-02-15T18:29:17.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:15.953939+0000 mon.smithi042 (mon.0) 171 : cluster [WRN] Health check update: Degraded data redundancy: 324/213 objects degraded (152.113%), 14 pgs degraded (PG_DEGRADED) 2024-02-15T18:29:18.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:16.715594+0000 mgr.smithi042.kfoihb (mgr.24435) 148 : cluster [DBG] pgmap v116: 65 pgs: 12 active+undersized+degraded, 1 active+recovering+undersized+remapped, 2 active+recovery_wait+undersized+degraded+remapped, 22 active+undersized, 28 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 239 KiB/s rd, 173 KiB/s wr, 33 op/s; 324/210 objects degraded (154.286%); 0 B/s, 5 objects/s recovering 2024-02-15T18:29:18.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:16.715594+0000 mgr.smithi042.kfoihb (mgr.24435) 148 : cluster [DBG] pgmap v116: 65 pgs: 12 active+undersized+degraded, 1 active+recovering+undersized+remapped, 2 active+recovery_wait+undersized+degraded+remapped, 22 active+undersized, 28 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 239 KiB/s rd, 173 KiB/s wr, 33 op/s; 324/210 objects degraded (154.286%); 0 B/s, 5 objects/s recovering 2024-02-15T18:29:19.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.102+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2007168172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b80f9860 msgr2=0x7fd6b80fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.102+0000 7fd6bf7d1700 1 --2- 172.21.15.42:0/2007168172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b80f9860 0x7fd6b80fbc70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fd6a8004660 tx=0x7fd6a8009b30).stop 2024-02-15T18:29:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.102+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2007168172 shutdown_connections 2024-02-15T18:29:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.102+0000 7fd6bf7d1700 1 --2- 172.21.15.42:0/2007168172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b80f9860 0x7fd6b80fbc70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.102+0000 7fd6bf7d1700 1 --2- 172.21.15.42:0/2007168172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b80602d0 0x7fd6b80f92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.102+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2007168172 >> 172.21.15.42:0/2007168172 conn(0x7fd6b80f0d40 msgr2=0x7fd6b80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.102+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2007168172 shutdown_connections 2024-02-15T18:29:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.102+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2007168172 wait complete. 2024-02-15T18:29:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.103+0000 7fd6bf7d1700 1 Processor -- start 2024-02-15T18:29:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.103+0000 7fd6bf7d1700 1 -- start start 2024-02-15T18:29:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.103+0000 7fd6bf7d1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b80602d0 0x7fd6b8106f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.103+0000 7fd6bf7d1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b80f9860 0x7fd6b8107480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:19.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.103+0000 7fd6bf7d1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b8107a30 con 0x7fd6b80602d0 2024-02-15T18:29:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.103+0000 7fd6bf7d1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b8107b70 con 0x7fd6b80f9860 2024-02-15T18:29:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.103+0000 7fd6bd56d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b80602d0 0x7fd6b8106f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.103+0000 7fd6bd56d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b80602d0 0x7fd6b8106f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47284/0 (socket says 172.21.15.42:47284) 2024-02-15T18:29:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.103+0000 7fd6bd56d700 1 -- 172.21.15.42:0/2130340992 learned_addr learned my addr 172.21.15.42:0/2130340992 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:29:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.103+0000 7fd6bcd6c700 1 --2- 172.21.15.42:0/2130340992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b80f9860 0x7fd6b8107480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.104+0000 7fd6bd56d700 1 -- 172.21.15.42:0/2130340992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b80f9860 msgr2=0x7fd6b8107480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.104+0000 7fd6bd56d700 1 --2- 172.21.15.42:0/2130340992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b80f9860 0x7fd6b8107480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.104+0000 7fd6bd56d700 1 -- 172.21.15.42:0/2130340992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a8005040 con 0x7fd6b80602d0 2024-02-15T18:29:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.104+0000 7fd6bd56d700 1 --2- 172.21.15.42:0/2130340992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b80602d0 0x7fd6b8106f90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fd6b40026d0 tx=0x7fd6b4009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.104+0000 7fd6adffb700 1 -- 172.21.15.42:0/2130340992 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6b4009320 con 0x7fd6b80602d0 2024-02-15T18:29:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.104+0000 7fd6adffb700 1 -- 172.21.15.42:0/2130340992 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fd6b400f040 con 0x7fd6b80602d0 2024-02-15T18:29:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.104+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2130340992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b8108510 con 0x7fd6b80602d0 2024-02-15T18:29:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.104+0000 7fd6adffb700 1 -- 172.21.15.42:0/2130340992 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6b40149b0 con 0x7fd6b80602d0 2024-02-15T18:29:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.104+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2130340992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b8108ad0 con 0x7fd6b80602d0 2024-02-15T18:29:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.105+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2130340992 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b8045ad0 con 0x7fd6b80602d0 2024-02-15T18:29:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.106+0000 7fd6adffb700 1 -- 172.21.15.42:0/2130340992 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fd6b4009480 con 0x7fd6b80602d0 2024-02-15T18:29:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.106+0000 7fd6adffb700 1 --2- 172.21.15.42:0/2130340992 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd6a4060f20 0x7fd6a40633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.106+0000 7fd6adffb700 1 -- 172.21.15.42:0/2130340992 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5870+0+0 (secure 0 0 0) 0x7fd6b407c600 con 0x7fd6b80602d0 2024-02-15T18:29:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.106+0000 7fd6bcd6c700 1 --2- 172.21.15.42:0/2130340992 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd6a4060f20 0x7fd6a40633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.106+0000 7fd6bcd6c700 1 --2- 172.21.15.42:0/2130340992 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd6a4060f20 0x7fd6a40633e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fd6a8004e30 tx=0x7fd6a8009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.110+0000 7fd6adffb700 1 -- 172.21.15.42:0/2130340992 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd6b4050500 con 0x7fd6b80602d0 2024-02-15T18:29:19.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.304+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2130340992 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6b8108da0 con 0x7fd6a4060f20 2024-02-15T18:29:19.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.306+0000 7fd6adffb700 1 -- 172.21.15.42:0/2130340992 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd6b8108da0 con 0x7fd6a4060f20 2024-02-15T18:29:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.309+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2130340992 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd6a4060f20 msgr2=0x7fd6a40633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.309+0000 7fd6bf7d1700 1 --2- 172.21.15.42:0/2130340992 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd6a4060f20 0x7fd6a40633e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fd6a8004e30 tx=0x7fd6a8009b30).stop 2024-02-15T18:29:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.309+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2130340992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b80602d0 msgr2=0x7fd6b8106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.309+0000 7fd6bf7d1700 1 --2- 172.21.15.42:0/2130340992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b80602d0 0x7fd6b8106f90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fd6b40026d0 tx=0x7fd6b4009de0).stop 2024-02-15T18:29:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.309+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2130340992 shutdown_connections 2024-02-15T18:29:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.309+0000 7fd6bf7d1700 1 --2- 172.21.15.42:0/2130340992 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd6a4060f20 0x7fd6a40633e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.309+0000 7fd6bf7d1700 1 --2- 172.21.15.42:0/2130340992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6b80602d0 0x7fd6b8106f90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.309+0000 7fd6bf7d1700 1 --2- 172.21.15.42:0/2130340992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6b80f9860 0x7fd6b8107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.309+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2130340992 >> 172.21.15.42:0/2130340992 conn(0x7fd6b80f0d40 msgr2=0x7fd6b80f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.309+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2130340992 shutdown_connections 2024-02-15T18:29:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.309+0000 7fd6bf7d1700 1 -- 172.21.15.42:0/2130340992 wait complete. 2024-02-15T18:29:19.318 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:29:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.422+0000 7f25d9454700 1 -- 172.21.15.42:0/3084903226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d4060310 msgr2=0x7f25d40f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.422+0000 7f25d9454700 1 --2- 172.21.15.42:0/3084903226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d4060310 0x7f25d40f9110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f25c4004660 tx=0x7f25c4009b30).stop 2024-02-15T18:29:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.422+0000 7f25d9454700 1 -- 172.21.15.42:0/3084903226 shutdown_connections 2024-02-15T18:29:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.422+0000 7f25d9454700 1 --2- 172.21.15.42:0/3084903226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25d40f9690 0x7f25d40fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.422+0000 7f25d9454700 1 --2- 172.21.15.42:0/3084903226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d4060310 0x7f25d40f9110 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.422+0000 7f25d9454700 1 -- 172.21.15.42:0/3084903226 >> 172.21.15.42:0/3084903226 conn(0x7f25d40f0b90 msgr2=0x7f25d40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.422+0000 7f25d9454700 1 -- 172.21.15.42:0/3084903226 shutdown_connections 2024-02-15T18:29:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.422+0000 7f25d9454700 1 -- 172.21.15.42:0/3084903226 wait complete. 2024-02-15T18:29:19.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.423+0000 7f25d9454700 1 Processor -- start 2024-02-15T18:29:19.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.423+0000 7f25d9454700 1 -- start start 2024-02-15T18:29:19.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.423+0000 7f25d9454700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d4060310 0x7f25d40f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:19.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.423+0000 7f25d9454700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25d40f9690 0x7f25d40f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:19.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.423+0000 7f25d9454700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d40f5570 con 0x7f25d4060310 2024-02-15T18:29:19.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.423+0000 7f25d9454700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d40f56b0 con 0x7f25d40f9690 2024-02-15T18:29:19.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.423+0000 7f25d3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d4060310 0x7f25d40f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:19.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.423+0000 7f25d3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d4060310 0x7f25d40f6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47298/0 (socket says 172.21.15.42:47298) 2024-02-15T18:29:19.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.423+0000 7f25d3fff700 1 -- 172.21.15.42:0/3196734438 learned_addr learned my addr 172.21.15.42:0/3196734438 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:29:19.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.424+0000 7f25d37fe700 1 --2- 172.21.15.42:0/3196734438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25d40f9690 0x7f25d40f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:19.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.424+0000 7f25d3fff700 1 -- 172.21.15.42:0/3196734438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25d40f9690 msgr2=0x7f25d40f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:19.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.424+0000 7f25d3fff700 1 --2- 172.21.15.42:0/3196734438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25d40f9690 0x7f25d40f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.424+0000 7f25d3fff700 1 -- 172.21.15.42:0/3196734438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c4005040 con 0x7f25d4060310 2024-02-15T18:29:19.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.424+0000 7f25d3fff700 1 --2- 172.21.15.42:0/3196734438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d4060310 0x7f25d40f6b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f25c4009f50 tx=0x7f25c4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:19.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.424+0000 7f25d0ff9700 1 -- 172.21.15.42:0/3196734438 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25c4018070 con 0x7f25d4060310 2024-02-15T18:29:19.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.424+0000 7f25d0ff9700 1 -- 172.21.15.42:0/3196734438 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f25c4007ba0 con 0x7f25d4060310 2024-02-15T18:29:19.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.424+0000 7f25d9454700 1 -- 172.21.15.42:0/3196734438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d40f5930 con 0x7f25d4060310 2024-02-15T18:29:19.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.424+0000 7f25d0ff9700 1 -- 172.21.15.42:0/3196734438 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25c401c420 con 0x7f25d4060310 2024-02-15T18:29:19.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.424+0000 7f25d9454700 1 -- 172.21.15.42:0/3196734438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d40f5e90 con 0x7f25d4060310 2024-02-15T18:29:19.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.426+0000 7f25d9454700 1 -- 172.21.15.42:0/3196734438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d4045ad0 con 0x7f25d4060310 2024-02-15T18:29:19.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.426+0000 7f25d0ff9700 1 -- 172.21.15.42:0/3196734438 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f25c401d420 con 0x7f25d4060310 2024-02-15T18:29:19.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.426+0000 7f25d0ff9700 1 --2- 172.21.15.42:0/3196734438 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f25bc060f70 0x7f25bc063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:19.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.426+0000 7f25d0ff9700 1 -- 172.21.15.42:0/3196734438 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5870+0+0 (secure 0 0 0) 0x7f25c407d4b0 con 0x7f25d4060310 2024-02-15T18:29:19.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.426+0000 7f25d37fe700 1 --2- 172.21.15.42:0/3196734438 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f25bc060f70 0x7f25bc063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:19.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.427+0000 7f25d37fe700 1 --2- 172.21.15.42:0/3196734438 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f25bc060f70 0x7f25bc063430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f25d40600e0 tx=0x7f25c800b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:19.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.431+0000 7f25d0ff9700 1 -- 172.21.15.42:0/3196734438 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f25c404d890 con 0x7f25d4060310 2024-02-15T18:29:19.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.657+0000 7f25d9454700 1 -- 172.21.15.42:0/3196734438 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f25d4057f10 con 0x7f25bc060f70 2024-02-15T18:29:19.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.659+0000 7f25d0ff9700 1 -- 172.21.15.42:0/3196734438 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f25d4057f10 con 0x7f25bc060f70 2024-02-15T18:29:19.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.660+0000 7f25d9454700 1 -- 172.21.15.42:0/3196734438 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f25bc060f70 msgr2=0x7f25bc063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:19.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.660+0000 7f25d9454700 1 --2- 172.21.15.42:0/3196734438 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f25bc060f70 0x7f25bc063430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f25d40600e0 tx=0x7f25c800b040).stop 2024-02-15T18:29:19.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.660+0000 7f25d9454700 1 -- 172.21.15.42:0/3196734438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d4060310 msgr2=0x7f25d40f6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:19.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.660+0000 7f25d9454700 1 --2- 172.21.15.42:0/3196734438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d4060310 0x7f25d40f6b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f25c4009f50 tx=0x7f25c4009b30).stop 2024-02-15T18:29:19.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.660+0000 7f25d9454700 1 -- 172.21.15.42:0/3196734438 shutdown_connections 2024-02-15T18:29:19.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.660+0000 7f25d9454700 1 --2- 172.21.15.42:0/3196734438 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f25bc060f70 0x7f25bc063430 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.660+0000 7f25d9454700 1 --2- 172.21.15.42:0/3196734438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d4060310 0x7f25d40f6b90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.660+0000 7f25d9454700 1 --2- 172.21.15.42:0/3196734438 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25d40f9690 0x7f25d40f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.660+0000 7f25d9454700 1 -- 172.21.15.42:0/3196734438 >> 172.21.15.42:0/3196734438 conn(0x7f25d40f0b90 msgr2=0x7f25d40f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:19.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.661+0000 7f25d9454700 1 -- 172.21.15.42:0/3196734438 shutdown_connections 2024-02-15T18:29:19.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.661+0000 7f25d9454700 1 -- 172.21.15.42:0/3196734438 wait complete. 2024-02-15T18:29:19.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.765+0000 7f6bee300700 1 -- 172.21.15.42:0/3297399803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6be80f77d0 msgr2=0x7f6be80f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.765+0000 7f6bee300700 1 --2- 172.21.15.42:0/3297399803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6be80f77d0 0x7f6be80f7c30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f6bd8004660 tx=0x7f6bd8009b30).stop 2024-02-15T18:29:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.765+0000 7f6bee300700 1 -- 172.21.15.42:0/3297399803 shutdown_connections 2024-02-15T18:29:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.765+0000 7f6bee300700 1 --2- 172.21.15.42:0/3297399803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6be80f77d0 0x7f6be80f7c30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.765+0000 7f6bee300700 1 --2- 172.21.15.42:0/3297399803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6be80f6530 0x7f6be80f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.765+0000 7f6bee300700 1 -- 172.21.15.42:0/3297399803 >> 172.21.15.42:0/3297399803 conn(0x7f6be80f1c90 msgr2=0x7f6be80f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.765+0000 7f6bee300700 1 -- 172.21.15.42:0/3297399803 shutdown_connections 2024-02-15T18:29:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.765+0000 7f6bee300700 1 -- 172.21.15.42:0/3297399803 wait complete. 2024-02-15T18:29:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.766+0000 7f6bee300700 1 Processor -- start 2024-02-15T18:29:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.766+0000 7f6bee300700 1 -- start start 2024-02-15T18:29:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.766+0000 7f6bee300700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6be80f6530 0x7f6be806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.766+0000 7f6bee300700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6be80f77d0 0x7f6be8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.766+0000 7f6bee300700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be806a380 con 0x7f6be80f6530 2024-02-15T18:29:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.766+0000 7f6bee300700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be806a4c0 con 0x7f6be80f77d0 2024-02-15T18:29:19.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.766+0000 7f6bed2fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6be80f6530 0x7f6be806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:19.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.766+0000 7f6bed2fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6be80f6530 0x7f6be806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47310/0 (socket says 172.21.15.42:47310) 2024-02-15T18:29:19.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.766+0000 7f6bed2fe700 1 -- 172.21.15.42:0/3483662035 learned_addr learned my addr 172.21.15.42:0/3483662035 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:29:19.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.766+0000 7f6becafd700 1 --2- 172.21.15.42:0/3483662035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6be80f77d0 0x7f6be8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:19.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.767+0000 7f6bed2fe700 1 -- 172.21.15.42:0/3483662035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6be80f77d0 msgr2=0x7f6be8069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:19.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.767+0000 7f6bed2fe700 1 --2- 172.21.15.42:0/3483662035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6be80f77d0 0x7f6be8069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.767+0000 7f6bed2fe700 1 -- 172.21.15.42:0/3483662035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd8005040 con 0x7f6be80f6530 2024-02-15T18:29:19.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.767+0000 7f6bed2fe700 1 --2- 172.21.15.42:0/3483662035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6be80f6530 0x7f6be806b9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f6be4002700 tx=0x7f6be4009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:19.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.767+0000 7f6bddffb700 1 -- 172.21.15.42:0/3483662035 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6be40079c0 con 0x7f6be80f6530 2024-02-15T18:29:19.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.767+0000 7f6bddffb700 1 -- 172.21.15.42:0/3483662035 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f6be400f040 con 0x7f6be80f6530 2024-02-15T18:29:19.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.767+0000 7f6bee300700 1 -- 172.21.15.42:0/3483662035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be806a7a0 con 0x7f6be80f6530 2024-02-15T18:29:19.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.767+0000 7f6bddffb700 1 -- 172.21.15.42:0/3483662035 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6be4014940 con 0x7f6be80f6530 2024-02-15T18:29:19.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.767+0000 7f6bee300700 1 -- 172.21.15.42:0/3483662035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be810ede0 con 0x7f6be80f6530 2024-02-15T18:29:19.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.768+0000 7f6bee300700 1 -- 172.21.15.42:0/3483662035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be8045ad0 con 0x7f6be80f6530 2024-02-15T18:29:19.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.769+0000 7f6bddffb700 1 -- 172.21.15.42:0/3483662035 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f6be401b060 con 0x7f6be80f6530 2024-02-15T18:29:19.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.769+0000 7f6bddffb700 1 --2- 172.21.15.42:0/3483662035 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6bd4060f70 0x7f6bd4063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:19.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.769+0000 7f6bddffb700 1 -- 172.21.15.42:0/3483662035 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5870+0+0 (secure 0 0 0) 0x7f6be407bf90 con 0x7f6be80f6530 2024-02-15T18:29:19.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.769+0000 7f6becafd700 1 --2- 172.21.15.42:0/3483662035 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6bd4060f70 0x7f6bd4063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:19.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.769+0000 7f6becafd700 1 --2- 172.21.15.42:0/3483662035 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6bd4060f70 0x7f6bd4063430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f6bd8004630 tx=0x7f6bd8009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:19.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.773+0000 7f6bddffb700 1 -- 172.21.15.42:0/3483662035 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6be404c3e0 con 0x7f6be80f6530 2024-02-15T18:29:19.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:18.636483+0000 mon.smithi042 (mon.0) 172 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:19.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:18.716240+0000 mgr.smithi042.kfoihb (mgr.24435) 149 : cluster [DBG] pgmap v117: 65 pgs: 12 active+undersized+degraded, 1 active+recovering+undersized+remapped, 2 active+recovery_wait+undersized+degraded+remapped, 22 active+undersized, 28 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 217 KiB/s rd, 158 KiB/s wr, 31 op/s; 324/210 objects degraded (154.286%); 0 B/s, 10 objects/s recovering 2024-02-15T18:29:19.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:18.779519+0000 mon.smithi042 (mon.0) 173 : cluster [DBG] osdmap e68: 6 total, 5 up, 6 in 2024-02-15T18:29:19.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:18.958448+0000 mon.smithi042 (mon.0) 174 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:19.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:18.959199+0000 mon.smithi042 (mon.0) 175 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:29:19.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:18.963518+0000 mon.smithi042 (mon.0) 176 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:19.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:18.970350+0000 mon.smithi042 (mon.0) 177 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:29:19.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:18.973065+0000 mon.smithi042 (mon.0) 178 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:29:19.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:18.974504+0000 mon.smithi042 (mon.0) 179 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:29:19.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:18.975880+0000 mon.smithi042 (mon.0) 180 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:29:19.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:18.977237+0000 mon.smithi042 (mon.0) 181 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:29:19.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.964+0000 7f6bee300700 1 -- 172.21.15.42:0/3483662035 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6be80fefb0 con 0x7f6bd4060f70 2024-02-15T18:29:19.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.973+0000 7f6bddffb700 1 -- 172.21.15.42:0/3483662035 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f6bd8019040 con 0x7f6bd4060f70 2024-02-15T18:29:19.968 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:29:19.968 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (9m) 1s ago 10m 21.3M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:29:19.968 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (96s) 1s ago 10m 6643k - 16.2.14-521-g0e714d9a 1fbc92c859dc 2da9d875539a 2024-02-15T18:29:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (90s) 82s ago 9m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 5c46a68ff62b 2024-02-15T18:29:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (8m) 1s ago 10m 33.7M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:29:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (5m) 1s ago 5m 14.6M - 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:29:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (5m) 1s ago 5m 17.3M - 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:29:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (5m) 82s ago 5m 154M - 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:29:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (5m) 82s ago 5m 227M - 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:29:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (3m) 1s ago 12m 433M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:29:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (2m) 82s ago 9m 388M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:29:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (2m) 1s ago 12m 50.5M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:29:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (114s) 82s ago 9m 25.8M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:29:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (10m) 1s ago 10m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:29:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (9m) 82s ago 9m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:29:19.969 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (72s) 1s ago 9m 249M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc f226bea76cab 2024-02-15T18:29:19.970 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (7s) 1s ago 8m 10.6M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 5114ed026724 2024-02-15T18:29:19.970 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (8m) 1s ago 8m 395M 4096M 16.2.4 8d91d370c2b8 3706071e4698 2024-02-15T18:29:19.970 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (7m) 82s ago 7m 432M 4096M 16.2.4 8d91d370c2b8 e6fe8e77c6fe 2024-02-15T18:29:19.970 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (7m) 82s ago 7m 428M 4096M 16.2.4 8d91d370c2b8 24582012e234 2024-02-15T18:29:19.970 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (7m) 82s ago 7m 347M 4096M 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:29:19.970 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (3m) 1s ago 10m 60.9M - 2.18.1 de242295e225 63b6dfb825ed 2024-02-15T18:29:19.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.975+0000 7f6bee300700 1 -- 172.21.15.42:0/3483662035 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6bd4060f70 msgr2=0x7f6bd4063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:19.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.975+0000 7f6bee300700 1 --2- 172.21.15.42:0/3483662035 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6bd4060f70 0x7f6bd4063430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f6bd8004630 tx=0x7f6bd8009b30).stop 2024-02-15T18:29:19.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.975+0000 7f6bee300700 1 -- 172.21.15.42:0/3483662035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6be80f6530 msgr2=0x7f6be806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:19.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.975+0000 7f6bee300700 1 --2- 172.21.15.42:0/3483662035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6be80f6530 0x7f6be806b9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f6be4002700 tx=0x7f6be4009d80).stop 2024-02-15T18:29:19.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.975+0000 7f6bee300700 1 -- 172.21.15.42:0/3483662035 shutdown_connections 2024-02-15T18:29:19.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.975+0000 7f6bee300700 1 --2- 172.21.15.42:0/3483662035 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6bd4060f70 0x7f6bd4063430 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.975+0000 7f6bee300700 1 --2- 172.21.15.42:0/3483662035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6be80f6530 0x7f6be806b9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.975+0000 7f6bee300700 1 --2- 172.21.15.42:0/3483662035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6be80f77d0 0x7f6be8069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:19.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.975+0000 7f6bee300700 1 -- 172.21.15.42:0/3483662035 >> 172.21.15.42:0/3483662035 conn(0x7f6be80f1c90 msgr2=0x7f6be80fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:19.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.976+0000 7f6bee300700 1 -- 172.21.15.42:0/3483662035 shutdown_connections 2024-02-15T18:29:19.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:19.976+0000 7f6bee300700 1 -- 172.21.15.42:0/3483662035 wait complete. 2024-02-15T18:29:20.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:18.636483+0000 mon.smithi042 (mon.0) 172 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:20.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:18.716240+0000 mgr.smithi042.kfoihb (mgr.24435) 149 : cluster [DBG] pgmap v117: 65 pgs: 12 active+undersized+degraded, 1 active+recovering+undersized+remapped, 2 active+recovery_wait+undersized+degraded+remapped, 22 active+undersized, 28 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 217 KiB/s rd, 158 KiB/s wr, 31 op/s; 324/210 objects degraded (154.286%); 0 B/s, 10 objects/s recovering 2024-02-15T18:29:20.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:18.779519+0000 mon.smithi042 (mon.0) 173 : cluster [DBG] osdmap e68: 6 total, 5 up, 6 in 2024-02-15T18:29:20.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:18.958448+0000 mon.smithi042 (mon.0) 174 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:20.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:18.959199+0000 mon.smithi042 (mon.0) 175 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:29:20.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:18.963518+0000 mon.smithi042 (mon.0) 176 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:20.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:18.970350+0000 mon.smithi042 (mon.0) 177 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:29:20.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:18.973065+0000 mon.smithi042 (mon.0) 178 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:29:20.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:18.974504+0000 mon.smithi042 (mon.0) 179 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:29:20.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:18.975880+0000 mon.smithi042 (mon.0) 180 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:29:20.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:18.977237+0000 mon.smithi042 (mon.0) 181 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:29:20.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.077+0000 7f141ffff700 1 -- 172.21.15.42:0/3202746304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14200602b0 msgr2=0x7f14200fb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:20.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.077+0000 7f141ffff700 1 --2- 172.21.15.42:0/3202746304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14200602b0 0x7f14200fb570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f1410004660 tx=0x7f1410009b30).stop 2024-02-15T18:29:20.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.078+0000 7f141ffff700 1 -- 172.21.15.42:0/3202746304 shutdown_connections 2024-02-15T18:29:20.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.078+0000 7f141ffff700 1 --2- 172.21.15.42:0/3202746304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14200fba60 0x7f14200fde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.078+0000 7f141ffff700 1 --2- 172.21.15.42:0/3202746304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14200602b0 0x7f14200fb570 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.078+0000 7f141ffff700 1 -- 172.21.15.42:0/3202746304 >> 172.21.15.42:0/3202746304 conn(0x7f14200f0b90 msgr2=0x7f14200f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:20.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.078+0000 7f141ffff700 1 -- 172.21.15.42:0/3202746304 shutdown_connections 2024-02-15T18:29:20.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.078+0000 7f141ffff700 1 -- 172.21.15.42:0/3202746304 wait complete. 2024-02-15T18:29:20.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.078+0000 7f141ffff700 1 Processor -- start 2024-02-15T18:29:20.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.078+0000 7f141ffff700 1 -- start start 2024-02-15T18:29:20.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.078+0000 7f141ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14200602b0 0x7f1420108fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:20.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.078+0000 7f141ffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14200fba60 0x7f14201094b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:20.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f141ffff700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1420109a60 con 0x7f14200602b0 2024-02-15T18:29:20.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f141ffff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1420109ba0 con 0x7f14200fba60 2024-02-15T18:29:20.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f141effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14200602b0 0x7f1420108fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:20.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f141effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14200602b0 0x7f1420108fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47328/0 (socket says 172.21.15.42:47328) 2024-02-15T18:29:20.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f141effd700 1 -- 172.21.15.42:0/1307370883 learned_addr learned my addr 172.21.15.42:0/1307370883 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:29:20.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f141e7fc700 1 --2- 172.21.15.42:0/1307370883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14200fba60 0x7f14201094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:20.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f141effd700 1 -- 172.21.15.42:0/1307370883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14200fba60 msgr2=0x7f14201094b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:20.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f141effd700 1 --2- 172.21.15.42:0/1307370883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14200fba60 0x7f14201094b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f141effd700 1 -- 172.21.15.42:0/1307370883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1410005040 con 0x7f14200602b0 2024-02-15T18:29:20.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f141e7fc700 1 --2- 172.21.15.42:0/1307370883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14200fba60 0x7f14201094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:29:20.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f141effd700 1 --2- 172.21.15.42:0/1307370883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14200602b0 0x7f1420108fc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f1410009f50 tx=0x7f1410009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:20.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f14077fe700 1 -- 172.21.15.42:0/1307370883 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1410018070 con 0x7f14200602b0 2024-02-15T18:29:20.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f141ffff700 1 -- 172.21.15.42:0/1307370883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142010a4e0 con 0x7f14200602b0 2024-02-15T18:29:20.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f14077fe700 1 -- 172.21.15.42:0/1307370883 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f1410007ba0 con 0x7f14200602b0 2024-02-15T18:29:20.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.079+0000 7f14077fe700 1 -- 172.21.15.42:0/1307370883 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f141001c420 con 0x7f14200602b0 2024-02-15T18:29:20.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.080+0000 7f141ffff700 1 -- 172.21.15.42:0/1307370883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142010aa40 con 0x7f14200602b0 2024-02-15T18:29:20.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.081+0000 7f141ffff700 1 -- 172.21.15.42:0/1307370883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1420045ad0 con 0x7f14200602b0 2024-02-15T18:29:20.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.081+0000 7f14077fe700 1 -- 172.21.15.42:0/1307370883 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f141001d420 con 0x7f14200602b0 2024-02-15T18:29:20.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.081+0000 7f14077fe700 1 --2- 172.21.15.42:0/1307370883 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f1408060f70 0x7f1408063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:20.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.081+0000 7f14077fe700 1 -- 172.21.15.42:0/1307370883 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5870+0+0 (secure 0 0 0) 0x7f141007ddf0 con 0x7f14200602b0 2024-02-15T18:29:20.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.081+0000 7f141e7fc700 1 --2- 172.21.15.42:0/1307370883 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f1408060f70 0x7f1408063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:20.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.082+0000 7f141e7fc700 1 --2- 172.21.15.42:0/1307370883 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f1408060f70 0x7f1408063430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f1420060080 tx=0x7f141400b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:20.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.086+0000 7f14077fe700 1 -- 172.21.15.42:0/1307370883 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1410051c10 con 0x7f14200602b0 2024-02-15T18:29:20.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.336+0000 7f141ffff700 1 -- 172.21.15.42:0/1307370883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1420101130 con 0x7f14200602b0 2024-02-15T18:29:20.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.337+0000 7f14077fe700 1 -- 172.21.15.42:0/1307370883 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+820 (secure 0 0 0) 0x7f1410051800 con 0x7f14200602b0 2024-02-15T18:29:20.332 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:29:20.332 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:29:20.332 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:29:20.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:29:20.332 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:29:20.332 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:29:20.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:29:20.332 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:29:20.332 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 1, 2024-02-15T18:29:20.332 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:29:20.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:29:20.333 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:29:20.333 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:29:20.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:29:20.333 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:29:20.333 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 5, 2024-02-15T18:29:20.333 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 8 2024-02-15T18:29:20.333 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:29:20.333 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:29:20.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.339+0000 7f141ffff700 1 -- 172.21.15.42:0/1307370883 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f1408060f70 msgr2=0x7f1408063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:20.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.339+0000 7f141ffff700 1 --2- 172.21.15.42:0/1307370883 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f1408060f70 0x7f1408063430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f1420060080 tx=0x7f141400b040).stop 2024-02-15T18:29:20.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.339+0000 7f141ffff700 1 -- 172.21.15.42:0/1307370883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14200602b0 msgr2=0x7f1420108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:20.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.339+0000 7f141ffff700 1 --2- 172.21.15.42:0/1307370883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14200602b0 0x7f1420108fc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f1410009f50 tx=0x7f1410009b30).stop 2024-02-15T18:29:20.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.339+0000 7f141ffff700 1 -- 172.21.15.42:0/1307370883 shutdown_connections 2024-02-15T18:29:20.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.339+0000 7f141ffff700 1 --2- 172.21.15.42:0/1307370883 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f1408060f70 0x7f1408063430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.339+0000 7f141ffff700 1 --2- 172.21.15.42:0/1307370883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14200602b0 0x7f1420108fc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.339+0000 7f141ffff700 1 --2- 172.21.15.42:0/1307370883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14200fba60 0x7f14201094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.339+0000 7f141ffff700 1 -- 172.21.15.42:0/1307370883 >> 172.21.15.42:0/1307370883 conn(0x7f14200f0b90 msgr2=0x7f14200f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:20.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.339+0000 7f141ffff700 1 -- 172.21.15.42:0/1307370883 shutdown_connections 2024-02-15T18:29:20.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.339+0000 7f141ffff700 1 -- 172.21.15.42:0/1307370883 wait complete. 2024-02-15T18:29:20.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.451+0000 7f9aa0965700 1 -- 172.21.15.42:0/217121656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c0f55c0 msgr2=0x7f9a9c0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:20.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.451+0000 7f9aa0965700 1 --2- 172.21.15.42:0/217121656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c0f55c0 0x7f9a9c0f59e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f9a8c004660 tx=0x7f9a8c009b30).stop 2024-02-15T18:29:20.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.451+0000 7f9aa0965700 1 -- 172.21.15.42:0/217121656 shutdown_connections 2024-02-15T18:29:20.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.451+0000 7f9aa0965700 1 --2- 172.21.15.42:0/217121656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a9c0f6860 0x7f9a9c0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.451+0000 7f9aa0965700 1 --2- 172.21.15.42:0/217121656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c0f55c0 0x7f9a9c0f59e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.451+0000 7f9aa0965700 1 -- 172.21.15.42:0/217121656 >> 172.21.15.42:0/217121656 conn(0x7f9a9c0f0d40 msgr2=0x7f9a9c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:20.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.451+0000 7f9aa0965700 1 -- 172.21.15.42:0/217121656 shutdown_connections 2024-02-15T18:29:20.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.451+0000 7f9aa0965700 1 -- 172.21.15.42:0/217121656 wait complete. 2024-02-15T18:29:20.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.451+0000 7f9aa0965700 1 Processor -- start 2024-02-15T18:29:20.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.452+0000 7f9aa0965700 1 -- start start 2024-02-15T18:29:20.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.452+0000 7f9aa0965700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c0f55c0 0x7f9a9c105f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:20.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.452+0000 7f9aa0965700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a9c0f6860 0x7f9a9c1063f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.452+0000 7f9aa0965700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c1069a0 con 0x7f9a9c0f55c0 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.452+0000 7f9aa0965700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c106ae0 con 0x7f9a9c0f6860 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.452+0000 7f9a9a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c0f55c0 0x7f9a9c105f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.452+0000 7f9a9a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c0f55c0 0x7f9a9c105f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47356/0 (socket says 172.21.15.42:47356) 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.452+0000 7f9a9a59c700 1 -- 172.21.15.42:0/2226835909 learned_addr learned my addr 172.21.15.42:0/2226835909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.452+0000 7f9a99d9b700 1 --2- 172.21.15.42:0/2226835909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a9c0f6860 0x7f9a9c1063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.452+0000 7f9a9a59c700 1 -- 172.21.15.42:0/2226835909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a9c0f6860 msgr2=0x7f9a9c1063f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.452+0000 7f9a9a59c700 1 --2- 172.21.15.42:0/2226835909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a9c0f6860 0x7f9a9c1063f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.452+0000 7f9a9a59c700 1 -- 172.21.15.42:0/2226835909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c005040 con 0x7f9a9c0f55c0 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.452+0000 7f9a9a59c700 1 --2- 172.21.15.42:0/2226835909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c0f55c0 0x7f9a9c105f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f9a8c002040 tx=0x7f9a8c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.453+0000 7f9a8affd700 1 -- 172.21.15.42:0/2226835909 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a8c01b070 con 0x7f9a9c0f55c0 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.453+0000 7f9a8affd700 1 -- 172.21.15.42:0/2226835909 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f9a8c014420 con 0x7f9a9c0f55c0 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.453+0000 7f9aa0965700 1 -- 172.21.15.42:0/2226835909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9c107420 con 0x7f9a9c0f55c0 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.453+0000 7f9a8affd700 1 -- 172.21.15.42:0/2226835909 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a8c00f610 con 0x7f9a9c0f55c0 2024-02-15T18:29:20.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.453+0000 7f9aa0965700 1 -- 172.21.15.42:0/2226835909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c107980 con 0x7f9a9c0f55c0 2024-02-15T18:29:20.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.454+0000 7f9aa0965700 1 -- 172.21.15.42:0/2226835909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9c045ad0 con 0x7f9a9c0f55c0 2024-02-15T18:29:20.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.454+0000 7f9a8affd700 1 -- 172.21.15.42:0/2226835909 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f9a8c027070 con 0x7f9a9c0f55c0 2024-02-15T18:29:20.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.454+0000 7f9a8affd700 1 --2- 172.21.15.42:0/2226835909 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9a84060f20 0x7f9a840633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:20.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.454+0000 7f9a8affd700 1 -- 172.21.15.42:0/2226835909 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5870+0+0 (secure 0 0 0) 0x7f9a8c07ed60 con 0x7f9a9c0f55c0 2024-02-15T18:29:20.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.454+0000 7f9a99d9b700 1 --2- 172.21.15.42:0/2226835909 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9a84060f20 0x7f9a840633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:20.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.455+0000 7f9a99d9b700 1 --2- 172.21.15.42:0/2226835909 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9a84060f20 0x7f9a840633e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f9a90007f80 tx=0x7f9a9000b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:20.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.462+0000 7f9a8affd700 1 -- 172.21.15.42:0/2226835909 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9a8c052c60 con 0x7f9a9c0f55c0 2024-02-15T18:29:20.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.691+0000 7f9aa0965700 1 -- 172.21.15.42:0/2226835909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9a9c0ff8f0 con 0x7f9a9c0f55c0 2024-02-15T18:29:20.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.691+0000 7f9a8affd700 1 -- 172.21.15.42:0/2226835909 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1773 (secure 0 0 0) 0x7f9a8c025030 con 0x7f9a9c0f55c0 2024-02-15T18:29:20.687 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:29:20.687 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:29:20.687 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:29:20.687 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:29:20.688 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:29:20.688 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:29:20.688 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:29:20.688 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:29:20.688 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:29:20.688 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:29:20.688 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:29:20.688 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:29:20.688 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:29:20.688 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:29:20.689 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:29:20.690 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:29:20.690 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:29:20.690 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:29:20.690 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:29:20.690 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:29:20.690 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:29:20.690 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:29:20.690 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:29:20.690 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:29:20.690 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:29:20.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.693+0000 7f9aa0965700 1 -- 172.21.15.42:0/2226835909 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9a84060f20 msgr2=0x7f9a840633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:20.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.693+0000 7f9aa0965700 1 --2- 172.21.15.42:0/2226835909 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9a84060f20 0x7f9a840633e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f9a90007f80 tx=0x7f9a9000b040).stop 2024-02-15T18:29:20.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.693+0000 7f9aa0965700 1 -- 172.21.15.42:0/2226835909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c0f55c0 msgr2=0x7f9a9c105f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:20.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.693+0000 7f9aa0965700 1 --2- 172.21.15.42:0/2226835909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c0f55c0 0x7f9a9c105f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f9a8c002040 tx=0x7f9a8c00b040).stop 2024-02-15T18:29:20.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.693+0000 7f9aa0965700 1 -- 172.21.15.42:0/2226835909 shutdown_connections 2024-02-15T18:29:20.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.693+0000 7f9aa0965700 1 --2- 172.21.15.42:0/2226835909 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9a84060f20 0x7f9a840633e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.693+0000 7f9aa0965700 1 --2- 172.21.15.42:0/2226835909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a9c0f55c0 0x7f9a9c105f00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.693+0000 7f9aa0965700 1 --2- 172.21.15.42:0/2226835909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a9c0f6860 0x7f9a9c1063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.693+0000 7f9aa0965700 1 -- 172.21.15.42:0/2226835909 >> 172.21.15.42:0/2226835909 conn(0x7f9a9c0f0d40 msgr2=0x7f9a9c0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:20.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.694+0000 7f9aa0965700 1 -- 172.21.15.42:0/2226835909 shutdown_connections 2024-02-15T18:29:20.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.694+0000 7f9aa0965700 1 -- 172.21.15.42:0/2226835909 wait complete. 2024-02-15T18:29:20.692 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:29:20.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.804+0000 7f69ff1d0700 1 -- 172.21.15.42:0/3345593288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f8060310 msgr2=0x7f69f80f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:20.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.804+0000 7f69ff1d0700 1 --2- 172.21.15.42:0/3345593288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f8060310 0x7f69f80f9120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f69f4004660 tx=0x7f69f4009b30).stop 2024-02-15T18:29:20.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.804+0000 7f69ff1d0700 1 -- 172.21.15.42:0/3345593288 shutdown_connections 2024-02-15T18:29:20.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.804+0000 7f69ff1d0700 1 --2- 172.21.15.42:0/3345593288 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69f80f96a0 0x7f69f80fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.804+0000 7f69ff1d0700 1 --2- 172.21.15.42:0/3345593288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f8060310 0x7f69f80f9120 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.804+0000 7f69ff1d0700 1 -- 172.21.15.42:0/3345593288 >> 172.21.15.42:0/3345593288 conn(0x7f69f80f0ba0 msgr2=0x7f69f80f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:20.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.805+0000 7f69ff1d0700 1 -- 172.21.15.42:0/3345593288 shutdown_connections 2024-02-15T18:29:20.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.805+0000 7f69ff1d0700 1 -- 172.21.15.42:0/3345593288 wait complete. 2024-02-15T18:29:20.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.805+0000 7f69ff1d0700 1 Processor -- start 2024-02-15T18:29:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.806+0000 7f69ff1d0700 1 -- start start 2024-02-15T18:29:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.806+0000 7f69ff1d0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69f8060310 0x7f69f8106e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.806+0000 7f69ff1d0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f80f96a0 0x7f69f8107340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.806+0000 7f69ff1d0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8107940 con 0x7f69f80f96a0 2024-02-15T18:29:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.806+0000 7f69ff1d0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8108150 con 0x7f69f8060310 2024-02-15T18:29:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.806+0000 7f69fd9cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f80f96a0 0x7f69f8107340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.806+0000 7f69fd9cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f80f96a0 0x7f69f8107340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47376/0 (socket says 172.21.15.42:47376) 2024-02-15T18:29:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.806+0000 7f69fd9cd700 1 -- 172.21.15.42:0/2198042580 learned_addr learned my addr 172.21.15.42:0/2198042580 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:29:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.807+0000 7f69fe1ce700 1 --2- 172.21.15.42:0/2198042580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69f8060310 0x7f69f8106e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.807+0000 7f69fd9cd700 1 -- 172.21.15.42:0/2198042580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69f8060310 msgr2=0x7f69f8106e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.807+0000 7f69fd9cd700 1 --2- 172.21.15.42:0/2198042580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69f8060310 0x7f69f8106e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.807+0000 7f69fd9cd700 1 -- 172.21.15.42:0/2198042580 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f4005040 con 0x7f69f80f96a0 2024-02-15T18:29:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.807+0000 7f69fd9cd700 1 --2- 172.21.15.42:0/2198042580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f80f96a0 0x7f69f8107340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f69e800e7c0 tx=0x7f69e800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.807+0000 7f69eeffd700 1 -- 172.21.15.42:0/2198042580 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69e80092e0 con 0x7f69f80f96a0 2024-02-15T18:29:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.807+0000 7f69ff1d0700 1 -- 172.21.15.42:0/2198042580 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f81083d0 con 0x7f69f80f96a0 2024-02-15T18:29:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.807+0000 7f69eeffd700 1 -- 172.21.15.42:0/2198042580 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f69e800f040 con 0x7f69f80f96a0 2024-02-15T18:29:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.807+0000 7f69eeffd700 1 -- 172.21.15.42:0/2198042580 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69e8014920 con 0x7f69f80f96a0 2024-02-15T18:29:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.807+0000 7f69ff1d0700 1 -- 172.21.15.42:0/2198042580 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f8108990 con 0x7f69f80f96a0 2024-02-15T18:29:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.808+0000 7f69ff1d0700 1 -- 172.21.15.42:0/2198042580 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f8059100 con 0x7f69f80f96a0 2024-02-15T18:29:20.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.810+0000 7f69eeffd700 1 -- 172.21.15.42:0/2198042580 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f69e8009440 con 0x7f69f80f96a0 2024-02-15T18:29:20.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.810+0000 7f69eeffd700 1 --2- 172.21.15.42:0/2198042580 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f69e4065380 0x7f69e4067840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:20.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.810+0000 7f69eeffd700 1 -- 172.21.15.42:0/2198042580 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5870+0+0 (secure 0 0 0) 0x7f69e807caa0 con 0x7f69f80f96a0 2024-02-15T18:29:20.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.810+0000 7f69fe1ce700 1 --2- 172.21.15.42:0/2198042580 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f69e4065380 0x7f69e4067840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:20.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.811+0000 7f69fe1ce700 1 --2- 172.21.15.42:0/2198042580 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f69e4065380 0x7f69e4067840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f69f4009720 tx=0x7f69f4016040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:20.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:20.814+0000 7f69eeffd700 1 -- 172.21.15.42:0/2198042580 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f69e804cef0 con 0x7f69f80f96a0 2024-02-15T18:29:21.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:21.012+0000 7f69ff1d0700 1 -- 172.21.15.42:0/2198042580 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f69f80fefe0 con 0x7f69e4065380 2024-02-15T18:29:21.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:21.013+0000 7f69eeffd700 1 -- 172.21.15.42:0/2198042580 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f69f400f070 con 0x7f69e4065380 2024-02-15T18:29:21.008 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:29:21.009 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:29:21.009 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:29:21.009 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:29:21.009 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:29:21.009 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-02-15T18:29:21.009 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-02-15T18:29:21.009 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-02-15T18:29:21.009 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:29:21.009 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "8/21 daemons upgraded", 2024-02-15T18:29:21.009 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading osd daemons", 2024-02-15T18:29:21.009 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:29:21.010 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:29:21.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:21.015+0000 7f69ff1d0700 1 -- 172.21.15.42:0/2198042580 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f69e4065380 msgr2=0x7f69e4067840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:21.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:21.015+0000 7f69ff1d0700 1 --2- 172.21.15.42:0/2198042580 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f69e4065380 0x7f69e4067840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f69f4009720 tx=0x7f69f4016040).stop 2024-02-15T18:29:21.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:21.015+0000 7f69ff1d0700 1 -- 172.21.15.42:0/2198042580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f80f96a0 msgr2=0x7f69f8107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:21.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:21.015+0000 7f69ff1d0700 1 --2- 172.21.15.42:0/2198042580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f80f96a0 0x7f69f8107340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f69e800e7c0 tx=0x7f69e800b040).stop 2024-02-15T18:29:21.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:21.016+0000 7f69ff1d0700 1 -- 172.21.15.42:0/2198042580 shutdown_connections 2024-02-15T18:29:21.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:21.016+0000 7f69ff1d0700 1 --2- 172.21.15.42:0/2198042580 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f69e4065380 0x7f69e4067840 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:21.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:21.016+0000 7f69ff1d0700 1 --2- 172.21.15.42:0/2198042580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f69f8060310 0x7f69f8106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:21.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:21.016+0000 7f69ff1d0700 1 --2- 172.21.15.42:0/2198042580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f80f96a0 0x7f69f8107340 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:21.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:21.016+0000 7f69ff1d0700 1 -- 172.21.15.42:0/2198042580 >> 172.21.15.42:0/2198042580 conn(0x7f69f80f0ba0 msgr2=0x7f69f80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:21.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:21.016+0000 7f69ff1d0700 1 -- 172.21.15.42:0/2198042580 shutdown_connections 2024-02-15T18:29:21.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:21.016+0000 7f69ff1d0700 1 -- 172.21.15.42:0/2198042580 wait complete. 2024-02-15T18:29:21.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:29:18.959831+0000 mgr.smithi042.kfoihb (mgr.24435) 150 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:29:21.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:18.977392+0000 mgr.smithi042.kfoihb (mgr.24435) 151 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:29:21.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:29:18.978065+0000 mgr.smithi042.kfoihb (mgr.24435) 152 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (14 PGs are or would become offline) 2024-02-15T18:29:21.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:19.307003+0000 mgr.smithi042.kfoihb (mgr.24435) 153 : audit [DBG] from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:21.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:19.659530+0000 mgr.smithi042.kfoihb (mgr.24435) 154 : audit [DBG] from='client.34202 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:21.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:19.779598+0000 mon.smithi042 (mon.0) 182 : cluster [DBG] osdmap e69: 6 total, 5 up, 6 in 2024-02-15T18:29:21.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:20.338128+0000 mon.smithi042 (mon.0) 183 : audit [DBG] from='client.? 172.21.15.42:0/1307370883' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:29:21.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:20.692756+0000 mon.smithi042 (mon.0) 184 : audit [DBG] from='client.? 172.21.15.42:0/2226835909' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:29:21.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:29:18.959831+0000 mgr.smithi042.kfoihb (mgr.24435) 150 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:29:21.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:18.977392+0000 mgr.smithi042.kfoihb (mgr.24435) 151 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:29:21.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:29:18.978065+0000 mgr.smithi042.kfoihb (mgr.24435) 152 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (14 PGs are or would become offline) 2024-02-15T18:29:21.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:19.307003+0000 mgr.smithi042.kfoihb (mgr.24435) 153 : audit [DBG] from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:21.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:19.659530+0000 mgr.smithi042.kfoihb (mgr.24435) 154 : audit [DBG] from='client.34202 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:21.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:19.779598+0000 mon.smithi042 (mon.0) 182 : cluster [DBG] osdmap e69: 6 total, 5 up, 6 in 2024-02-15T18:29:21.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:20.338128+0000 mon.smithi042 (mon.0) 183 : audit [DBG] from='client.? 172.21.15.42:0/1307370883' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:29:21.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:20.692756+0000 mon.smithi042 (mon.0) 184 : audit [DBG] from='client.? 172.21.15.42:0/2226835909' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:29:21.961 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:19.965740+0000 mgr.smithi042.kfoihb (mgr.24435) 155 : audit [DBG] from='client.34206 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:21.961 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:20.716725+0000 mgr.smithi042.kfoihb (mgr.24435) 156 : cluster [DBG] pgmap v120: 65 pgs: 12 active+undersized+degraded, 1 active+recovering+undersized+remapped, 2 active+recovery_wait+undersized+degraded+remapped, 22 active+undersized, 28 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 767 B/s wr, 2 op/s; 324/210 objects degraded (154.286%); 0 B/s, 7 objects/s recovering 2024-02-15T18:29:21.961 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:20.955032+0000 mon.smithi042 (mon.0) 185 : cluster [WRN] Health check update: Degraded data redundancy: 324/210 objects degraded (154.286%), 14 pgs degraded, 3 pgs undersized (PG_DEGRADED) 2024-02-15T18:29:21.961 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:21.586407+0000 mon.smithi042 (mon.0) 186 : audit [INF] from='osd.1 [v2:172.21.15.42:6810/4294036630,v1:172.21.15.42:6811/4294036630]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-15T18:29:21.962 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-1[83769]: debug 2024-02-15T18:29:21.581+0000 7f33427cb380 -1 osd.1 65 log_to_monitors {default=true} 2024-02-15T18:29:21.962 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:29:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-1[83769]: debug 2024-02-15T18:29:21.966+0000 7f3332cb0700 -1 osd.1 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-15T18:29:22.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:19.965740+0000 mgr.smithi042.kfoihb (mgr.24435) 155 : audit [DBG] from='client.34206 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:22.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:20.716725+0000 mgr.smithi042.kfoihb (mgr.24435) 156 : cluster [DBG] pgmap v120: 65 pgs: 12 active+undersized+degraded, 1 active+recovering+undersized+remapped, 2 active+recovery_wait+undersized+degraded+remapped, 22 active+undersized, 28 active+clean; 193 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 767 B/s wr, 2 op/s; 324/210 objects degraded (154.286%); 0 B/s, 7 objects/s recovering 2024-02-15T18:29:22.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:20.955032+0000 mon.smithi042 (mon.0) 185 : cluster [WRN] Health check update: Degraded data redundancy: 324/210 objects degraded (154.286%), 14 pgs degraded, 3 pgs undersized (PG_DEGRADED) 2024-02-15T18:29:22.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:21.586407+0000 mon.smithi042 (mon.0) 186 : audit [INF] from='osd.1 [v2:172.21.15.42:6810/4294036630,v1:172.21.15.42:6811/4294036630]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-15T18:29:23.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:21.014395+0000 mgr.smithi042.kfoihb (mgr.24435) 157 : audit [DBG] from='client.34218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:23.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:21.961469+0000 mon.smithi042 (mon.0) 187 : audit [INF] from='osd.1 [v2:172.21.15.42:6810/4294036630,v1:172.21.15.42:6811/4294036630]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-15T18:29:23.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:21.961536+0000 mon.smithi042 (mon.0) 188 : cluster [DBG] osdmap e70: 6 total, 5 up, 6 in 2024-02-15T18:29:23.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:21.962187+0000 mon.smithi042 (mon.0) 189 : audit [INF] from='osd.1 [v2:172.21.15.42:6810/4294036630,v1:172.21.15.42:6811/4294036630]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-02-15T18:29:23.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:21.014395+0000 mgr.smithi042.kfoihb (mgr.24435) 157 : audit [DBG] from='client.34218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:23.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:21.961469+0000 mon.smithi042 (mon.0) 187 : audit [INF] from='osd.1 [v2:172.21.15.42:6810/4294036630,v1:172.21.15.42:6811/4294036630]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-15T18:29:23.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:21.961536+0000 mon.smithi042 (mon.0) 188 : cluster [DBG] osdmap e70: 6 total, 5 up, 6 in 2024-02-15T18:29:23.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:21.962187+0000 mon.smithi042 (mon.0) 189 : audit [INF] from='osd.1 [v2:172.21.15.42:6810/4294036630,v1:172.21.15.42:6811/4294036630]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-02-15T18:29:24.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:22.717121+0000 mgr.smithi042.kfoihb (mgr.24435) 158 : cluster [DBG] pgmap v122: 65 pgs: 12 active+undersized+degraded, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 22 active+undersized, 29 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s; 163/210 objects degraded (77.619%); 0 B/s, 18 objects/s recovering 2024-02-15T18:29:24.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:22.959814+0000 mon.smithi042 (mon.0) 190 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:29:24.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:22.964820+0000 mon.smithi042 (mon.0) 191 : cluster [INF] osd.1 [v2:172.21.15.42:6810/4294036630,v1:172.21.15.42:6811/4294036630] boot 2024-02-15T18:29:24.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:22.964868+0000 mon.smithi042 (mon.0) 192 : cluster [DBG] osdmap e71: 6 total, 6 up, 6 in 2024-02-15T18:29:24.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:22.965038+0000 mon.smithi042 (mon.0) 193 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:29:24.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:22.717121+0000 mgr.smithi042.kfoihb (mgr.24435) 158 : cluster [DBG] pgmap v122: 65 pgs: 12 active+undersized+degraded, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 22 active+undersized, 29 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s; 163/210 objects degraded (77.619%); 0 B/s, 18 objects/s recovering 2024-02-15T18:29:24.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:22.959814+0000 mon.smithi042 (mon.0) 190 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:29:24.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:22.964820+0000 mon.smithi042 (mon.0) 191 : cluster [INF] osd.1 [v2:172.21.15.42:6810/4294036630,v1:172.21.15.42:6811/4294036630] boot 2024-02-15T18:29:24.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:22.964868+0000 mon.smithi042 (mon.0) 192 : cluster [DBG] osdmap e71: 6 total, 6 up, 6 in 2024-02-15T18:29:24.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:22.965038+0000 mon.smithi042 (mon.0) 193 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-15T18:29:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:23.966780+0000 mon.smithi042 (mon.0) 194 : cluster [DBG] osdmap e72: 6 total, 6 up, 6 in 2024-02-15T18:29:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:24.717525+0000 mgr.smithi042.kfoihb (mgr.24435) 159 : cluster [DBG] pgmap v125: 65 pgs: 12 active+undersized+degraded, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 22 active+undersized, 29 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s; 163/210 objects degraded (77.619%); 0 B/s, 10 objects/s recovering 2024-02-15T18:29:25.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:23.966780+0000 mon.smithi042 (mon.0) 194 : cluster [DBG] osdmap e72: 6 total, 6 up, 6 in 2024-02-15T18:29:25.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:24.717525+0000 mgr.smithi042.kfoihb (mgr.24435) 159 : cluster [DBG] pgmap v125: 65 pgs: 12 active+undersized+degraded, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 22 active+undersized, 29 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s; 163/210 objects degraded (77.619%); 0 B/s, 10 objects/s recovering 2024-02-15T18:29:27.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:25.834261+0000 mon.smithi042 (mon.0) 195 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:27.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:25.957068+0000 mon.smithi042 (mon.0) 196 : cluster [WRN] Health check update: Degraded data redundancy: 163/210 objects degraded (77.619%), 13 pgs degraded, 2 pgs undersized (PG_DEGRADED) 2024-02-15T18:29:27.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:25.834261+0000 mon.smithi042 (mon.0) 195 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:27.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:25.957068+0000 mon.smithi042 (mon.0) 196 : cluster [WRN] Health check update: Degraded data redundancy: 163/210 objects degraded (77.619%), 13 pgs degraded, 2 pgs undersized (PG_DEGRADED) 2024-02-15T18:29:28.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:26.717915+0000 mgr.smithi042.kfoihb (mgr.24435) 160 : cluster [DBG] pgmap v126: 65 pgs: 12 active+undersized+degraded, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 20 active+undersized, 31 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 163/210 objects degraded (77.619%); 0 B/s, 8 objects/s recovering 2024-02-15T18:29:28.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:26.717915+0000 mgr.smithi042.kfoihb (mgr.24435) 160 : cluster [DBG] pgmap v126: 65 pgs: 12 active+undersized+degraded, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 20 active+undersized, 31 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 163/210 objects degraded (77.619%); 0 B/s, 8 objects/s recovering 2024-02-15T18:29:30.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:28.718369+0000 mgr.smithi042.kfoihb (mgr.24435) 161 : cluster [DBG] pgmap v127: 65 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 63 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 606 B/s rd, 1 op/s; 97/210 objects degraded (46.190%); 16 KiB/s, 0 keys/s, 8 objects/s recovering 2024-02-15T18:29:30.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:28.718369+0000 mgr.smithi042.kfoihb (mgr.24435) 161 : cluster [DBG] pgmap v127: 65 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 63 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 606 B/s rd, 1 op/s; 97/210 objects degraded (46.190%); 16 KiB/s, 0 keys/s, 8 objects/s recovering 2024-02-15T18:29:32.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:30.718701+0000 mgr.smithi042.kfoihb (mgr.24435) 162 : cluster [DBG] pgmap v128: 65 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 63 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s; 97/210 objects degraded (46.190%); 14 KiB/s, 0 keys/s, 6 objects/s recovering 2024-02-15T18:29:32.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:30.957769+0000 mon.smithi042 (mon.0) 197 : cluster [WRN] Health check update: Degraded data redundancy: 97/210 objects degraded (46.190%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-02-15T18:29:32.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:30.718701+0000 mgr.smithi042.kfoihb (mgr.24435) 162 : cluster [DBG] pgmap v128: 65 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 63 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s rd, 0 op/s; 97/210 objects degraded (46.190%); 14 KiB/s, 0 keys/s, 6 objects/s recovering 2024-02-15T18:29:32.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:30.957769+0000 mon.smithi042 (mon.0) 197 : cluster [WRN] Health check update: Degraded data redundancy: 97/210 objects degraded (46.190%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-02-15T18:29:34.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:32.719173+0000 mgr.smithi042.kfoihb (mgr.24435) 163 : cluster [DBG] pgmap v129: 65 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 63 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 944 B/s rd, 1 op/s; 97/210 objects degraded (46.190%); 11 KiB/s, 0 keys/s, 11 objects/s recovering 2024-02-15T18:29:34.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:32.719173+0000 mgr.smithi042.kfoihb (mgr.24435) 163 : cluster [DBG] pgmap v129: 65 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 63 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 944 B/s rd, 1 op/s; 97/210 objects degraded (46.190%); 11 KiB/s, 0 keys/s, 11 objects/s recovering 2024-02-15T18:29:35.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:33.992379+0000 mon.smithi042 (mon.0) 198 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:29:35.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:33.992379+0000 mon.smithi042 (mon.0) 198 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:29:36.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:33.992540+0000 mgr.smithi042.kfoihb (mgr.24435) 164 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:29:36.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:29:33.993237+0000 mgr.smithi042.kfoihb (mgr.24435) 165 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-02-15T18:29:36.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:34.719501+0000 mgr.smithi042.kfoihb (mgr.24435) 166 : cluster [DBG] pgmap v130: 65 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 63 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 856 B/s rd, 1 op/s; 97/210 objects degraded (46.190%); 10 KiB/s, 0 keys/s, 10 objects/s recovering 2024-02-15T18:29:36.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:33.992540+0000 mgr.smithi042.kfoihb (mgr.24435) 164 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:29:36.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:29:33.993237+0000 mgr.smithi042.kfoihb (mgr.24435) 165 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (2 PGs are or would become offline) 2024-02-15T18:29:36.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:34.719501+0000 mgr.smithi042.kfoihb (mgr.24435) 166 : cluster [DBG] pgmap v130: 65 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 63 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 856 B/s rd, 1 op/s; 97/210 objects degraded (46.190%); 10 KiB/s, 0 keys/s, 10 objects/s recovering 2024-02-15T18:29:37.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:36.719831+0000 mgr.smithi042.kfoihb (mgr.24435) 167 : cluster [DBG] pgmap v131: 65 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 63 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 97/210 objects degraded (46.190%); 9.3 KiB/s, 0 keys/s, 9 objects/s recovering 2024-02-15T18:29:37.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:36.719831+0000 mgr.smithi042.kfoihb (mgr.24435) 167 : cluster [DBG] pgmap v131: 65 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 63 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 97/210 objects degraded (46.190%); 9.3 KiB/s, 0 keys/s, 9 objects/s recovering 2024-02-15T18:29:38.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:36.966452+0000 mon.smithi042 (mon.0) 199 : cluster [DBG] osdmap e73: 6 total, 6 up, 6 in 2024-02-15T18:29:38.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:36.966452+0000 mon.smithi042 (mon.0) 199 : cluster [DBG] osdmap e73: 6 total, 6 up, 6 in 2024-02-15T18:29:39.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:37.967735+0000 mon.smithi042 (mon.0) 200 : cluster [DBG] osdmap e74: 6 total, 6 up, 6 in 2024-02-15T18:29:39.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:38 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:38.720157+0000 mgr.smithi042.kfoihb (mgr.24435) 168 : cluster [DBG] pgmap v134: 65 pgs: 1 active+undersized+remapped, 1 active+recovering+undersized+remapped, 63 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 14 objects/s recovering 2024-02-15T18:29:39.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:37.967735+0000 mon.smithi042 (mon.0) 200 : cluster [DBG] osdmap e74: 6 total, 6 up, 6 in 2024-02-15T18:29:39.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:38 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:38.720157+0000 mgr.smithi042.kfoihb (mgr.24435) 168 : cluster [DBG] pgmap v134: 65 pgs: 1 active+undersized+remapped, 1 active+recovering+undersized+remapped, 63 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 14 objects/s recovering 2024-02-15T18:29:40.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:38.969721+0000 mon.smithi042 (mon.0) 201 : cluster [WRN] Health check update: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-02-15T18:29:40.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:38.969721+0000 mon.smithi042 (mon.0) 201 : cluster [WRN] Health check update: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-02-15T18:29:41.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:40.720471+0000 mgr.smithi042.kfoihb (mgr.24435) 169 : cluster [DBG] pgmap v135: 65 pgs: 1 active+undersized+remapped, 1 active+recovering+undersized+remapped, 63 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 0 B/s, 7 objects/s recovering 2024-02-15T18:29:41.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:40.720471+0000 mgr.smithi042.kfoihb (mgr.24435) 169 : cluster [DBG] pgmap v135: 65 pgs: 1 active+undersized+remapped, 1 active+recovering+undersized+remapped, 63 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 0 B/s, 7 objects/s recovering 2024-02-15T18:29:44.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:42.720908+0000 mgr.smithi042.kfoihb (mgr.24435) 170 : cluster [DBG] pgmap v136: 65 pgs: 1 active+recovering+undersized+remapped, 64 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 14 objects/s recovering 2024-02-15T18:29:44.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:42.720908+0000 mgr.smithi042.kfoihb (mgr.24435) 170 : cluster [DBG] pgmap v136: 65 pgs: 1 active+recovering+undersized+remapped, 64 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 14 objects/s recovering 2024-02-15T18:29:46.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:44.721381+0000 mgr.smithi042.kfoihb (mgr.24435) 171 : cluster [DBG] pgmap v137: 65 pgs: 1 active+recovering+undersized+remapped, 64 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 14 objects/s recovering 2024-02-15T18:29:46.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:44.780271+0000 mon.smithi042 (mon.0) 202 : cluster [DBG] osdmap e75: 6 total, 6 up, 6 in 2024-02-15T18:29:46.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:44.721381+0000 mgr.smithi042.kfoihb (mgr.24435) 171 : cluster [DBG] pgmap v137: 65 pgs: 1 active+recovering+undersized+remapped, 64 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 14 objects/s recovering 2024-02-15T18:29:46.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:44.780271+0000 mon.smithi042 (mon.0) 202 : cluster [DBG] osdmap e75: 6 total, 6 up, 6 in 2024-02-15T18:29:47.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:45.783660+0000 mon.smithi042 (mon.0) 203 : cluster [DBG] osdmap e76: 6 total, 6 up, 6 in 2024-02-15T18:29:47.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:45.959788+0000 mon.smithi042 (mon.0) 204 : cluster [WRN] Health check update: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-02-15T18:29:47.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:46 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:45.783660+0000 mon.smithi042 (mon.0) 203 : cluster [DBG] osdmap e76: 6 total, 6 up, 6 in 2024-02-15T18:29:47.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:46 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:45.959788+0000 mon.smithi042 (mon.0) 204 : cluster [WRN] Health check update: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-02-15T18:29:48.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:46.721824+0000 mgr.smithi042.kfoihb (mgr.24435) 172 : cluster [DBG] pgmap v140: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 0 B/s, 11 objects/s recovering 2024-02-15T18:29:48.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:46.961861+0000 mon.smithi042 (mon.0) 205 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1 pg undersized) 2024-02-15T18:29:48.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:46.961916+0000 mon.smithi042 (mon.0) 206 : cluster [INF] Cluster is now healthy 2024-02-15T18:29:48.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:46.721824+0000 mgr.smithi042.kfoihb (mgr.24435) 172 : cluster [DBG] pgmap v140: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 0 B/s, 11 objects/s recovering 2024-02-15T18:29:48.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:46.961861+0000 mon.smithi042 (mon.0) 205 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1 pg undersized) 2024-02-15T18:29:48.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:46.961916+0000 mon.smithi042 (mon.0) 206 : cluster [INF] Cluster is now healthy 2024-02-15T18:29:50.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:48.722380+0000 mgr.smithi042.kfoihb (mgr.24435) 173 : cluster [DBG] pgmap v141: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 11 objects/s recovering 2024-02-15T18:29:50.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:49.002538+0000 mon.smithi042 (mon.0) 207 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:29:50.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:48.722380+0000 mgr.smithi042.kfoihb (mgr.24435) 173 : cluster [DBG] pgmap v141: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 11 objects/s recovering 2024-02-15T18:29:50.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:49.002538+0000 mon.smithi042 (mon.0) 207 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:29:51.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:49.002742+0000 mgr.smithi042.kfoihb (mgr.24435) 174 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:29:51.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:29:49.003824+0000 mgr.smithi042.kfoihb (mgr.24435) 175 : cephadm [INF] Upgrade: osd.2 is safe to restart 2024-02-15T18:29:51.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:50.697933+0000 mon.smithi042 (mon.0) 208 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:51.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:50.698789+0000 mon.smithi042 (mon.0) 209 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-15T18:29:51.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:50.699437+0000 mon.smithi042 (mon.0) 210 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:51.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.124+0000 7fa0bda68700 1 -- 172.21.15.42:0/2449679689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b80f66d0 msgr2=0x7fa0b80f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.124+0000 7fa0bda68700 1 --2- 172.21.15.42:0/2449679689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b80f66d0 0x7fa0b80f6b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fa0a4004660 tx=0x7fa0a4009b30).stop 2024-02-15T18:29:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.124+0000 7fa0bda68700 1 -- 172.21.15.42:0/2449679689 shutdown_connections 2024-02-15T18:29:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.124+0000 7fa0bda68700 1 --2- 172.21.15.42:0/2449679689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b80f66d0 0x7fa0b80f6b30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.124+0000 7fa0bda68700 1 --2- 172.21.15.42:0/2449679689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0b80f5430 0x7fa0b80f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:29:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.124+0000 7fa0bda68700 1 -- 172.21.15.42:0/2449679689 >> 172.21.15.42:0/2449679689 conn(0x7fa0b80f0b90 msgr2=0x7fa0b80f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.124+0000 7fa0bda68700 1 -- 172.21.15.42:0/2449679689 shutdown_connections 2024-02-15T18:29:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.124+0000 7fa0bda68700 1 -- 172.21.15.42:0/2449679689 wait complete. 2024-02-15T18:29:51.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.125+0000 7fa0bda68700 1 Processor -- start 2024-02-15T18:29:51.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.125+0000 7fa0bda68700 1 -- start start 2024-02-15T18:29:51.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.125+0000 7fa0bda68700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0b80f5430 0x7fa0b8106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:51.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.125+0000 7fa0bda68700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b80f66d0 0x7fa0b81073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:51.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.125+0000 7fa0bda68700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b8107a40 con 0x7fa0b80f66d0 2024-02-15T18:29:51.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.125+0000 7fa0bda68700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b81081c0 con 0x7fa0b80f5430 2024-02-15T18:29:51.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.125+0000 7fa0b7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b80f66d0 0x7fa0b81073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:51.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.125+0000 7fa0b7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b80f66d0 0x7fa0b81073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45914/0 (socket says 172.21.15.42:45914) 2024-02-15T18:29:51.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.125+0000 7fa0b7fff700 1 -- 172.21.15.42:0/746022855 learned_addr learned my addr 172.21.15.42:0/746022855 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:29:51.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.125+0000 7fa0bca66700 1 --2- 172.21.15.42:0/746022855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0b80f5430 0x7fa0b8106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:51.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.126+0000 7fa0b7fff700 1 -- 172.21.15.42:0/746022855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0b80f5430 msgr2=0x7fa0b8106ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:51.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.126+0000 7fa0b7fff700 1 --2- 172.21.15.42:0/746022855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0b80f5430 0x7fa0b8106ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.126+0000 7fa0b7fff700 1 -- 172.21.15.42:0/746022855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a4005040 con 0x7fa0b80f66d0 2024-02-15T18:29:51.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.126+0000 7fa0b7fff700 1 --2- 172.21.15.42:0/746022855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b80f66d0 0x7fa0b81073b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fa0a4009f50 tx=0x7fa0a4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:51.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.126+0000 7fa0b57fa700 1 -- 172.21.15.42:0/746022855 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0a4018070 con 0x7fa0b80f66d0 2024-02-15T18:29:51.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.126+0000 7fa0b57fa700 1 -- 172.21.15.42:0/746022855 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fa0a4007c70 con 0x7fa0b80f66d0 2024-02-15T18:29:51.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.126+0000 7fa0bda68700 1 -- 172.21.15.42:0/746022855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b81083e0 con 0x7fa0b80f66d0 2024-02-15T18:29:51.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.126+0000 7fa0b57fa700 1 -- 172.21.15.42:0/746022855 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0a401c620 con 0x7fa0b80f66d0 2024-02-15T18:29:51.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.126+0000 7fa0bda68700 1 -- 172.21.15.42:0/746022855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b81088c0 con 0x7fa0b80f66d0 2024-02-15T18:29:51.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.127+0000 7fa0bda68700 1 -- 172.21.15.42:0/746022855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b8045ad0 con 0x7fa0b80f66d0 2024-02-15T18:29:51.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.127+0000 7fa0b57fa700 1 -- 172.21.15.42:0/746022855 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fa0a401d420 con 0x7fa0b80f66d0 2024-02-15T18:29:51.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.128+0000 7fa0b57fa700 1 --2- 172.21.15.42:0/746022855 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fa0a8060f70 0x7fa0a8063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:51.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.128+0000 7fa0b57fa700 1 -- 172.21.15.42:0/746022855 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fa0a407e040 con 0x7fa0b80f66d0 2024-02-15T18:29:51.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.128+0000 7fa0bca66700 1 --2- 172.21.15.42:0/746022855 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fa0a8060f70 0x7fa0a8063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:51.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.128+0000 7fa0bca66700 1 --2- 172.21.15.42:0/746022855 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fa0a8060f70 0x7fa0a8063430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fa0b8106c30 tx=0x7fa0ac008040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:51.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.132+0000 7fa0b57fa700 1 -- 172.21.15.42:0/746022855 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa0a404e4c0 con 0x7fa0b80f66d0 2024-02-15T18:29:51.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:49.002742+0000 mgr.smithi042.kfoihb (mgr.24435) 174 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-15T18:29:51.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:29:49.003824+0000 mgr.smithi042.kfoihb (mgr.24435) 175 : cephadm [INF] Upgrade: osd.2 is safe to restart 2024-02-15T18:29:51.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:50.697933+0000 mon.smithi042 (mon.0) 208 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:51.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:50.698789+0000 mon.smithi042 (mon.0) 209 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-15T18:29:51.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:50.699437+0000 mon.smithi042 (mon.0) 210 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:29:51.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.329+0000 7fa0bda68700 1 -- 172.21.15.42:0/746022855 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0b80fab10 con 0x7fa0a8060f70 2024-02-15T18:29:51.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.330+0000 7fa0b57fa700 1 -- 172.21.15.42:0/746022855 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa0b80fab10 con 0x7fa0a8060f70 2024-02-15T18:29:51.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.332+0000 7fa0bda68700 1 -- 172.21.15.42:0/746022855 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fa0a8060f70 msgr2=0x7fa0a8063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:51.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.332+0000 7fa0bda68700 1 --2- 172.21.15.42:0/746022855 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fa0a8060f70 0x7fa0a8063430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fa0b8106c30 tx=0x7fa0ac008040).stop 2024-02-15T18:29:51.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.332+0000 7fa0bda68700 1 -- 172.21.15.42:0/746022855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b80f66d0 msgr2=0x7fa0b81073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:51.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.332+0000 7fa0bda68700 1 --2- 172.21.15.42:0/746022855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b80f66d0 0x7fa0b81073b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fa0a4009f50 tx=0x7fa0a4009b30).stop 2024-02-15T18:29:51.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.332+0000 7fa0bda68700 1 -- 172.21.15.42:0/746022855 shutdown_connections 2024-02-15T18:29:51.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.332+0000 7fa0bda68700 1 --2- 172.21.15.42:0/746022855 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fa0a8060f70 0x7fa0a8063430 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.332+0000 7fa0bda68700 1 --2- 172.21.15.42:0/746022855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0b80f5430 0x7fa0b8106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.332+0000 7fa0bda68700 1 --2- 172.21.15.42:0/746022855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b80f66d0 0x7fa0b81073b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.332+0000 7fa0bda68700 1 -- 172.21.15.42:0/746022855 >> 172.21.15.42:0/746022855 conn(0x7fa0b80f0b90 msgr2=0x7fa0b805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:51.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.332+0000 7fa0bda68700 1 -- 172.21.15.42:0/746022855 shutdown_connections 2024-02-15T18:29:51.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.332+0000 7fa0bda68700 1 -- 172.21.15.42:0/746022855 wait complete. 2024-02-15T18:29:51.344 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:29:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.436+0000 7fd3f56fd700 1 -- 172.21.15.42:0/2000182101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f00f4800 msgr2=0x7fd3f00f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.436+0000 7fd3f56fd700 1 --2- 172.21.15.42:0/2000182101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f00f4800 0x7fd3f00f4c20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fd3e0004660 tx=0x7fd3e0009b30).stop 2024-02-15T18:29:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.436+0000 7fd3f56fd700 1 -- 172.21.15.42:0/2000182101 shutdown_connections 2024-02-15T18:29:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.436+0000 7fd3f56fd700 1 --2- 172.21.15.42:0/2000182101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3f00f51a0 0x7fd3f00f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.436+0000 7fd3f56fd700 1 --2- 172.21.15.42:0/2000182101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f00f4800 0x7fd3f00f4c20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.436+0000 7fd3f56fd700 1 -- 172.21.15.42:0/2000182101 >> 172.21.15.42:0/2000182101 conn(0x7fd3f00f0320 msgr2=0x7fd3f00f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.436+0000 7fd3f56fd700 1 -- 172.21.15.42:0/2000182101 shutdown_connections 2024-02-15T18:29:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.437+0000 7fd3f56fd700 1 -- 172.21.15.42:0/2000182101 wait complete. 2024-02-15T18:29:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.437+0000 7fd3f56fd700 1 Processor -- start 2024-02-15T18:29:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.437+0000 7fd3f56fd700 1 -- start start 2024-02-15T18:29:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.437+0000 7fd3f56fd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3f00f4800 0x7fd3f0106ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.437+0000 7fd3f56fd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f00f51a0 0x7fd3f0107390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.437+0000 7fd3f56fd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f0107940 con 0x7fd3f00f51a0 2024-02-15T18:29:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.437+0000 7fd3f56fd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f0108140 con 0x7fd3f00f4800 2024-02-15T18:29:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.438+0000 7fd3ef7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f00f51a0 0x7fd3f0107390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.438+0000 7fd3ef7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f00f51a0 0x7fd3f0107390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45938/0 (socket says 172.21.15.42:45938) 2024-02-15T18:29:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.438+0000 7fd3ef7fe700 1 -- 172.21.15.42:0/732746253 learned_addr learned my addr 172.21.15.42:0/732746253 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:29:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.438+0000 7fd3effff700 1 --2- 172.21.15.42:0/732746253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3f00f4800 0x7fd3f0106ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.438+0000 7fd3ef7fe700 1 -- 172.21.15.42:0/732746253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3f00f4800 msgr2=0x7fd3f0106ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.438+0000 7fd3ef7fe700 1 --2- 172.21.15.42:0/732746253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3f00f4800 0x7fd3f0106ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.438+0000 7fd3ef7fe700 1 -- 172.21.15.42:0/732746253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e0005040 con 0x7fd3f00f51a0 2024-02-15T18:29:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.438+0000 7fd3ef7fe700 1 --2- 172.21.15.42:0/732746253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f00f51a0 0x7fd3f0107390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fd3e400c510 tx=0x7fd3e4007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.438+0000 7fd3ecff9700 1 -- 172.21.15.42:0/732746253 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3e400f650 con 0x7fd3f00f51a0 2024-02-15T18:29:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.438+0000 7fd3f56fd700 1 -- 172.21.15.42:0/732746253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f0108420 con 0x7fd3f00f51a0 2024-02-15T18:29:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.438+0000 7fd3ecff9700 1 -- 172.21.15.42:0/732746253 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fd3e400fca0 con 0x7fd3f00f51a0 2024-02-15T18:29:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.438+0000 7fd3ecff9700 1 -- 172.21.15.42:0/732746253 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3e4012df0 con 0x7fd3f00f51a0 2024-02-15T18:29:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.438+0000 7fd3f56fd700 1 -- 172.21.15.42:0/732746253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f01089e0 con 0x7fd3f00f51a0 2024-02-15T18:29:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.439+0000 7fd3f56fd700 1 -- 172.21.15.42:0/732746253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f0059100 con 0x7fd3f00f51a0 2024-02-15T18:29:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.440+0000 7fd3ecff9700 1 -- 172.21.15.42:0/732746253 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fd3e4010070 con 0x7fd3f00f51a0 2024-02-15T18:29:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.440+0000 7fd3ecff9700 1 --2- 172.21.15.42:0/732746253 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd3d8060f10 0x7fd3d80633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.440+0000 7fd3ecff9700 1 -- 172.21.15.42:0/732746253 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd3e407be30 con 0x7fd3f00f51a0 2024-02-15T18:29:51.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.440+0000 7fd3effff700 1 --2- 172.21.15.42:0/732746253 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd3d8060f10 0x7fd3d80633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:51.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.441+0000 7fd3effff700 1 --2- 172.21.15.42:0/732746253 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd3d8060f10 0x7fd3d80633d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fd3e0009490 tx=0x7fd3e0009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:51.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.445+0000 7fd3ecff9700 1 -- 172.21.15.42:0/732746253 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd3e4047f60 con 0x7fd3f00f51a0 2024-02-15T18:29:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.641+0000 7fd3f56fd700 1 -- 172.21.15.42:0/732746253 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3f00fef60 con 0x7fd3d8060f10 2024-02-15T18:29:51.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.642+0000 7fd3ecff9700 1 -- 172.21.15.42:0/732746253 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd3f00fef60 con 0x7fd3d8060f10 2024-02-15T18:29:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.644+0000 7fd3f56fd700 1 -- 172.21.15.42:0/732746253 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd3d8060f10 msgr2=0x7fd3d80633d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.644+0000 7fd3f56fd700 1 --2- 172.21.15.42:0/732746253 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd3d8060f10 0x7fd3d80633d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fd3e0009490 tx=0x7fd3e0009b30).stop 2024-02-15T18:29:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.644+0000 7fd3f56fd700 1 -- 172.21.15.42:0/732746253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f00f51a0 msgr2=0x7fd3f0107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.644+0000 7fd3f56fd700 1 --2- 172.21.15.42:0/732746253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f00f51a0 0x7fd3f0107390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fd3e400c510 tx=0x7fd3e4007df0).stop 2024-02-15T18:29:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.644+0000 7fd3f56fd700 1 -- 172.21.15.42:0/732746253 shutdown_connections 2024-02-15T18:29:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.644+0000 7fd3f56fd700 1 --2- 172.21.15.42:0/732746253 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd3d8060f10 0x7fd3d80633d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.644+0000 7fd3f56fd700 1 --2- 172.21.15.42:0/732746253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3f00f4800 0x7fd3f0106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.644+0000 7fd3f56fd700 1 --2- 172.21.15.42:0/732746253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3f00f51a0 0x7fd3f0107390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.644+0000 7fd3f56fd700 1 -- 172.21.15.42:0/732746253 >> 172.21.15.42:0/732746253 conn(0x7fd3f00f0320 msgr2=0x7fd3f00fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:51.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.645+0000 7fd3f56fd700 1 -- 172.21.15.42:0/732746253 shutdown_connections 2024-02-15T18:29:51.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.645+0000 7fd3f56fd700 1 -- 172.21.15.42:0/732746253 wait complete. 2024-02-15T18:29:51.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.746+0000 7f92ef353700 1 -- 172.21.15.42:0/776338409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e80f66d0 msgr2=0x7f92e80f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:51.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.746+0000 7f92ef353700 1 --2- 172.21.15.42:0/776338409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e80f66d0 0x7f92e80f6b30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f92dc004660 tx=0x7f92dc009b30).stop 2024-02-15T18:29:51.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.746+0000 7f92ef353700 1 -- 172.21.15.42:0/776338409 shutdown_connections 2024-02-15T18:29:51.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.746+0000 7f92ef353700 1 --2- 172.21.15.42:0/776338409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e80f66d0 0x7f92e80f6b30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.746+0000 7f92ef353700 1 --2- 172.21.15.42:0/776338409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92e80f5430 0x7f92e80f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.746+0000 7f92ef353700 1 -- 172.21.15.42:0/776338409 >> 172.21.15.42:0/776338409 conn(0x7f92e80f0b90 msgr2=0x7f92e80f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:51.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.746+0000 7f92ef353700 1 -- 172.21.15.42:0/776338409 shutdown_connections 2024-02-15T18:29:51.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.746+0000 7f92ef353700 1 -- 172.21.15.42:0/776338409 wait complete. 2024-02-15T18:29:51.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.747+0000 7f92ef353700 1 Processor -- start 2024-02-15T18:29:51.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.747+0000 7f92ef353700 1 -- start start 2024-02-15T18:29:51.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.747+0000 7f92ef353700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92e80f5430 0x7f92e8106e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:51.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.747+0000 7f92ef353700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e80f66d0 0x7f92e8107320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:51.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.747+0000 7f92ef353700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8107920 con 0x7f92e80f66d0 2024-02-15T18:29:51.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.747+0000 7f92ef353700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8108130 con 0x7f92e80f5430 2024-02-15T18:29:51.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.747+0000 7f92edb50700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e80f66d0 0x7f92e8107320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:51.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.747+0000 7f92edb50700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e80f66d0 0x7f92e8107320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37794/0 (socket says 172.21.15.42:37794) 2024-02-15T18:29:51.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.747+0000 7f92edb50700 1 -- 172.21.15.42:0/2332918812 learned_addr learned my addr 172.21.15.42:0/2332918812 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:29:51.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.747+0000 7f92ee351700 1 --2- 172.21.15.42:0/2332918812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92e80f5430 0x7f92e8106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:51.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.748+0000 7f92edb50700 1 -- 172.21.15.42:0/2332918812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92e80f5430 msgr2=0x7f92e8106e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:51.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.748+0000 7f92edb50700 1 --2- 172.21.15.42:0/2332918812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92e80f5430 0x7f92e8106e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.748+0000 7f92edb50700 1 -- 172.21.15.42:0/2332918812 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc005040 con 0x7f92e80f66d0 2024-02-15T18:29:51.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.748+0000 7f92edb50700 1 --2- 172.21.15.42:0/2332918812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e80f66d0 0x7f92e8107320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f92dc009f50 tx=0x7f92dc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:51.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.748+0000 7f92daffd700 1 -- 172.21.15.42:0/2332918812 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92dc018070 con 0x7f92e80f66d0 2024-02-15T18:29:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.748+0000 7f92daffd700 1 -- 172.21.15.42:0/2332918812 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f92dc007d70 con 0x7f92e80f66d0 2024-02-15T18:29:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.748+0000 7f92ef353700 1 -- 172.21.15.42:0/2332918812 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e8108350 con 0x7f92e80f66d0 2024-02-15T18:29:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.748+0000 7f92daffd700 1 -- 172.21.15.42:0/2332918812 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92dc01c620 con 0x7f92e80f66d0 2024-02-15T18:29:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.748+0000 7f92ef353700 1 -- 172.21.15.42:0/2332918812 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e81088b0 con 0x7f92e80f66d0 2024-02-15T18:29:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.749+0000 7f92ef353700 1 -- 172.21.15.42:0/2332918812 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e8059100 con 0x7f92e80f66d0 2024-02-15T18:29:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.749+0000 7f92daffd700 1 -- 172.21.15.42:0/2332918812 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f92dc01c8e0 con 0x7f92e80f66d0 2024-02-15T18:29:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.750+0000 7f92daffd700 1 --2- 172.21.15.42:0/2332918812 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f92d4060f70 0x7f92d4063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.750+0000 7f92daffd700 1 -- 172.21.15.42:0/2332918812 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f92dc07e070 con 0x7f92e80f66d0 2024-02-15T18:29:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.750+0000 7f92ee351700 1 --2- 172.21.15.42:0/2332918812 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f92d4060f70 0x7f92d4063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:51.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.750+0000 7f92ee351700 1 --2- 172.21.15.42:0/2332918812 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f92d4060f70 0x7f92d4063430 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f92e8106ba0 tx=0x7f92e4009040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:51.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.754+0000 7f92daffd700 1 -- 172.21.15.42:0/2332918812 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f92dc04e4f0 con 0x7f92e80f66d0 2024-02-15T18:29:51.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.939+0000 7f92ef353700 1 -- 172.21.15.42:0/2332918812 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f92e80fab10 con 0x7f92d4060f70 2024-02-15T18:29:51.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.948+0000 7f92daffd700 1 -- 172.21.15.42:0/2332918812 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f92e80fab10 con 0x7f92d4060f70 2024-02-15T18:29:51.943 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:29:51.943 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (9m) 33s ago 11m 21.3M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:29:51.943 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (2m) 33s ago 11m 6643k - 16.2.14-521-g0e714d9a 1fbc92c859dc 2da9d875539a 2024-02-15T18:29:51.943 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (2m) 114s ago 10m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 5c46a68ff62b 2024-02-15T18:29:51.943 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (9m) 33s ago 11m 33.7M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:29:51.943 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (5m) 33s ago 5m 14.6M - 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:29:51.943 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (6m) 33s ago 6m 17.3M - 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:29:51.943 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (6m) 114s ago 6m 154M - 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:29:51.943 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (6m) 114s ago 6m 227M - 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:29:51.944 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (3m) 33s ago 12m 433M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:29:51.944 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (3m) 114s ago 10m 388M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:29:51.944 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (3m) 33s ago 12m 50.5M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:29:51.944 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (2m) 114s ago 10m 25.8M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:29:51.944 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (10m) 33s ago 10m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:29:51.944 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (10m) 114s ago 10m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:29:51.944 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (104s) 33s ago 9m 249M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc f226bea76cab 2024-02-15T18:29:51.944 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (39s) 33s ago 9m 10.6M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 5114ed026724 2024-02-15T18:29:51.944 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (8m) 33s ago 8m 395M 4096M 16.2.4 8d91d370c2b8 3706071e4698 2024-02-15T18:29:51.944 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (8m) 114s ago 8m 432M 4096M 16.2.4 8d91d370c2b8 e6fe8e77c6fe 2024-02-15T18:29:51.944 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (8m) 114s ago 8m 428M 4096M 16.2.4 8d91d370c2b8 24582012e234 2024-02-15T18:29:51.944 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (7m) 114s ago 7m 347M 4096M 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:29:51.944 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (3m) 33s ago 10m 60.9M - 2.18.1 de242295e225 63b6dfb825ed 2024-02-15T18:29:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.950+0000 7f92ef353700 1 -- 172.21.15.42:0/2332918812 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f92d4060f70 msgr2=0x7f92d4063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.950+0000 7f92ef353700 1 --2- 172.21.15.42:0/2332918812 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f92d4060f70 0x7f92d4063430 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f92e8106ba0 tx=0x7f92e4009040).stop 2024-02-15T18:29:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.950+0000 7f92ef353700 1 -- 172.21.15.42:0/2332918812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e80f66d0 msgr2=0x7f92e8107320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.950+0000 7f92ef353700 1 --2- 172.21.15.42:0/2332918812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e80f66d0 0x7f92e8107320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f92dc009f50 tx=0x7f92dc009b30).stop 2024-02-15T18:29:51.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.950+0000 7f92ef353700 1 -- 172.21.15.42:0/2332918812 shutdown_connections 2024-02-15T18:29:51.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.950+0000 7f92ef353700 1 --2- 172.21.15.42:0/2332918812 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f92d4060f70 0x7f92d4063430 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.950+0000 7f92ef353700 1 --2- 172.21.15.42:0/2332918812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92e80f5430 0x7f92e8106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.950+0000 7f92ef353700 1 --2- 172.21.15.42:0/2332918812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92e80f66d0 0x7f92e8107320 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:51.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.950+0000 7f92ef353700 1 -- 172.21.15.42:0/2332918812 >> 172.21.15.42:0/2332918812 conn(0x7f92e80f0b90 msgr2=0x7f92e805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:51.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.950+0000 7f92ef353700 1 -- 172.21.15.42:0/2332918812 shutdown_connections 2024-02-15T18:29:51.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:51.951+0000 7f92ef353700 1 -- 172.21.15.42:0/2332918812 wait complete. 2024-02-15T18:29:52.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.052+0000 7ff760621700 1 -- 172.21.15.42:0/86075746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7580f55c0 msgr2=0x7ff7580f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:52.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.052+0000 7ff760621700 1 --2- 172.21.15.42:0/86075746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7580f55c0 0x7ff7580f59e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7ff754004660 tx=0x7ff754009b30).stop 2024-02-15T18:29:52.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.052+0000 7ff760621700 1 -- 172.21.15.42:0/86075746 shutdown_connections 2024-02-15T18:29:52.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.052+0000 7ff760621700 1 --2- 172.21.15.42:0/86075746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7580f6860 0x7ff7580f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.052+0000 7ff760621700 1 --2- 172.21.15.42:0/86075746 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7580f55c0 0x7ff7580f59e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.052+0000 7ff760621700 1 -- 172.21.15.42:0/86075746 >> 172.21.15.42:0/86075746 conn(0x7ff7580f0d40 msgr2=0x7ff7580f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:52.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.052+0000 7ff760621700 1 -- 172.21.15.42:0/86075746 shutdown_connections 2024-02-15T18:29:52.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.052+0000 7ff760621700 1 -- 172.21.15.42:0/86075746 wait complete. 2024-02-15T18:29:52.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.052+0000 7ff760621700 1 Processor -- start 2024-02-15T18:29:52.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.052+0000 7ff760621700 1 -- start start 2024-02-15T18:29:52.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff760621700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7580f55c0 0x7ff758109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:52.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff760621700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7580f6860 0x7ff75810c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff760621700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff758060420 con 0x7ff7580f55c0 2024-02-15T18:29:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff760621700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff758060560 con 0x7ff7580f6860 2024-02-15T18:29:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff75e3bd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7580f55c0 0x7ff758109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff75e3bd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7580f55c0 0x7ff758109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37810/0 (socket says 172.21.15.42:37810) 2024-02-15T18:29:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff75e3bd700 1 -- 172.21.15.42:0/4223324356 learned_addr learned my addr 172.21.15.42:0/4223324356 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:29:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff75e3bd700 1 -- 172.21.15.42:0/4223324356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7580f6860 msgr2=0x7ff75810c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff75e3bd700 1 --2- 172.21.15.42:0/4223324356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7580f6860 0x7ff75810c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:29:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff75e3bd700 1 -- 172.21.15.42:0/4223324356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff754005040 con 0x7ff7580f55c0 2024-02-15T18:29:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff75e3bd700 1 --2- 172.21.15.42:0/4223324356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7580f55c0 0x7ff758109cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7ff754004620 tx=0x7ff754009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff74affd700 1 -- 172.21.15.42:0/4223324356 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff754018070 con 0x7ff7580f55c0 2024-02-15T18:29:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff74affd700 1 -- 172.21.15.42:0/4223324356 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7ff754007df0 con 0x7ff7580f55c0 2024-02-15T18:29:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff760621700 1 -- 172.21.15.42:0/4223324356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75810c770 con 0x7ff7580f55c0 2024-02-15T18:29:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff74affd700 1 -- 172.21.15.42:0/4223324356 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff75401c7b0 con 0x7ff7580f55c0 2024-02-15T18:29:52.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.053+0000 7ff760621700 1 -- 172.21.15.42:0/4223324356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75810cca0 con 0x7ff7580f55c0 2024-02-15T18:29:52.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.054+0000 7ff760621700 1 -- 172.21.15.42:0/4223324356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7580f49b0 con 0x7ff7580f55c0 2024-02-15T18:29:52.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.055+0000 7ff74affd700 1 -- 172.21.15.42:0/4223324356 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7ff75401c910 con 0x7ff7580f55c0 2024-02-15T18:29:52.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.055+0000 7ff74affd700 1 --2- 172.21.15.42:0/4223324356 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff744060f20 0x7ff7440633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:52.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.055+0000 7ff74affd700 1 -- 172.21.15.42:0/4223324356 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff75407dfc0 con 0x7ff7580f55c0 2024-02-15T18:29:52.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.055+0000 7ff75dbbc700 1 --2- 172.21.15.42:0/4223324356 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff744060f20 0x7ff7440633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:52.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.056+0000 7ff75dbbc700 1 --2- 172.21.15.42:0/4223324356 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff744060f20 0x7ff7440633e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7ff74c0047d0 tx=0x7ff74c007a40).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:52.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.059+0000 7ff74affd700 1 -- 172.21.15.42:0/4223324356 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff75404e460 con 0x7ff7580f55c0 2024-02-15T18:29:52.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:29:50.693578+0000 mgr.smithi042.kfoihb (mgr.24435) 176 : cephadm [INF] Upgrade: Updating osd.2 2024-02-15T18:29:52.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:29:50.700484+0000 mgr.smithi042.kfoihb (mgr.24435) 177 : cephadm [INF] Deploying daemon osd.2 on smithi042 2024-02-15T18:29:52.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:50.722709+0000 mgr.smithi042.kfoihb (mgr.24435) 178 : cluster [DBG] pgmap v142: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 0 B/s, 3 objects/s recovering 2024-02-15T18:29:52.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:29:50.693578+0000 mgr.smithi042.kfoihb (mgr.24435) 176 : cephadm [INF] Upgrade: Updating osd.2 2024-02-15T18:29:52.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:29:50.700484+0000 mgr.smithi042.kfoihb (mgr.24435) 177 : cephadm [INF] Deploying daemon osd.2 on smithi042 2024-02-15T18:29:52.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:50.722709+0000 mgr.smithi042.kfoihb (mgr.24435) 178 : cluster [DBG] pgmap v142: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 0 B/s, 3 objects/s recovering 2024-02-15T18:29:52.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.308+0000 7ff760621700 1 -- 172.21.15.42:0/4223324356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff758027a00 con 0x7ff7580f55c0 2024-02-15T18:29:52.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.309+0000 7ff74affd700 1 -- 172.21.15.42:0/4223324356 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+820 (secure 0 0 0) 0x7ff754051a80 con 0x7ff7580f55c0 2024-02-15T18:29:52.303 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:29:52.303 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:29:52.303 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:29:52.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:29:52.303 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:29:52.304 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:29:52.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:29:52.304 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:29:52.304 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2, 2024-02-15T18:29:52.304 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:29:52.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:29:52.304 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:29:52.304 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:29:52.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:29:52.304 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:29:52.304 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 6, 2024-02-15T18:29:52.304 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 8 2024-02-15T18:29:52.304 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:29:52.305 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:29:52.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.310+0000 7ff760621700 1 -- 172.21.15.42:0/4223324356 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff744060f20 msgr2=0x7ff7440633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:52.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.310+0000 7ff760621700 1 --2- 172.21.15.42:0/4223324356 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff744060f20 0x7ff7440633e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7ff74c0047d0 tx=0x7ff74c007a40).stop 2024-02-15T18:29:52.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.310+0000 7ff760621700 1 -- 172.21.15.42:0/4223324356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7580f55c0 msgr2=0x7ff758109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:52.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.311+0000 7ff760621700 1 --2- 172.21.15.42:0/4223324356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7580f55c0 0x7ff758109cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7ff754004620 tx=0x7ff754009b30).stop 2024-02-15T18:29:52.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.311+0000 7ff760621700 1 -- 172.21.15.42:0/4223324356 shutdown_connections 2024-02-15T18:29:52.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.311+0000 7ff760621700 1 --2- 172.21.15.42:0/4223324356 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff744060f20 0x7ff7440633e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.311+0000 7ff760621700 1 --2- 172.21.15.42:0/4223324356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7580f55c0 0x7ff758109cc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.311+0000 7ff760621700 1 --2- 172.21.15.42:0/4223324356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7580f6860 0x7ff75810c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:29:52.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.311+0000 7ff760621700 1 -- 172.21.15.42:0/4223324356 >> 172.21.15.42:0/4223324356 conn(0x7ff7580f0d40 msgr2=0x7ff7580fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:52.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.311+0000 7ff760621700 1 -- 172.21.15.42:0/4223324356 shutdown_connections 2024-02-15T18:29:52.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.311+0000 7ff760621700 1 -- 172.21.15.42:0/4223324356 wait complete. 2024-02-15T18:29:52.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.414+0000 7f6c65c8b700 1 -- 172.21.15.42:0/1292219496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c600f66e0 msgr2=0x7f6c600f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:52.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.414+0000 7f6c65c8b700 1 --2- 172.21.15.42:0/1292219496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c600f66e0 0x7f6c600f6b00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f6c50004660 tx=0x7f6c50009b30).stop 2024-02-15T18:29:52.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.414+0000 7f6c65c8b700 1 -- 172.21.15.42:0/1292219496 shutdown_connections 2024-02-15T18:29:52.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.414+0000 7f6c65c8b700 1 --2- 172.21.15.42:0/1292219496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c600f7980 0x7f6c600f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.414+0000 7f6c65c8b700 1 --2- 172.21.15.42:0/1292219496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c600f66e0 0x7f6c600f6b00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.414+0000 7f6c65c8b700 1 -- 172.21.15.42:0/1292219496 >> 172.21.15.42:0/1292219496 conn(0x7f6c600f1e40 msgr2=0x7f6c600f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:52.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.414+0000 7f6c65c8b700 1 -- 172.21.15.42:0/1292219496 shutdown_connections 2024-02-15T18:29:52.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.414+0000 7f6c65c8b700 1 -- 172.21.15.42:0/1292219496 wait complete. 2024-02-15T18:29:52.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.415+0000 7f6c65c8b700 1 Processor -- start 2024-02-15T18:29:52.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.415+0000 7f6c65c8b700 1 -- start start 2024-02-15T18:29:52.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.415+0000 7f6c65c8b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c600f66e0 0x7f6c60107050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:52.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.415+0000 7f6c65c8b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c600f7980 0x7f6c60107540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:52.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.415+0000 7f6c65c8b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c60107af0 con 0x7f6c600f7980 2024-02-15T18:29:52.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.415+0000 7f6c65c8b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c60107c30 con 0x7f6c600f66e0 2024-02-15T18:29:52.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.416+0000 7f6c5effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c600f7980 0x7f6c60107540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:52.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.416+0000 7f6c5effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c600f7980 0x7f6c60107540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37818/0 (socket says 172.21.15.42:37818) 2024-02-15T18:29:52.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.416+0000 7f6c5effd700 1 -- 172.21.15.42:0/237453299 learned_addr learned my addr 172.21.15.42:0/237453299 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:29:52.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.416+0000 7f6c5f7fe700 1 --2- 172.21.15.42:0/237453299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c600f66e0 0x7f6c60107050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:52.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.416+0000 7f6c5effd700 1 -- 172.21.15.42:0/237453299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c600f66e0 msgr2=0x7f6c60107050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:52.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.416+0000 7f6c5effd700 1 --2- 172.21.15.42:0/237453299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c600f66e0 0x7f6c60107050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.416+0000 7f6c5effd700 1 -- 172.21.15.42:0/237453299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c50005040 con 0x7f6c600f7980 2024-02-15T18:29:52.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.416+0000 7f6c5effd700 1 --2- 172.21.15.42:0/237453299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c600f7980 0x7f6c60107540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f6c540026d0 tx=0x7f6c54009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:52.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.416+0000 7f6c47fff700 1 -- 172.21.15.42:0/237453299 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c540079c0 con 0x7f6c600f7980 2024-02-15T18:29:52.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.416+0000 7f6c47fff700 1 -- 172.21.15.42:0/237453299 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f6c5400f040 con 0x7f6c600f7980 2024-02-15T18:29:52.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.416+0000 7f6c65c8b700 1 -- 172.21.15.42:0/237453299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c601085d0 con 0x7f6c600f7980 2024-02-15T18:29:52.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.416+0000 7f6c47fff700 1 -- 172.21.15.42:0/237453299 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c540139a0 con 0x7f6c600f7980 2024-02-15T18:29:52.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.416+0000 7f6c65c8b700 1 -- 172.21.15.42:0/237453299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c60108b90 con 0x7f6c600f7980 2024-02-15T18:29:52.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.418+0000 7f6c47fff700 1 -- 172.21.15.42:0/237453299 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f6c54013b00 con 0x7f6c600f7980 2024-02-15T18:29:52.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.418+0000 7f6c47fff700 1 --2- 172.21.15.42:0/237453299 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6c48060f20 0x7f6c480633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:52.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.418+0000 7f6c47fff700 1 -- 172.21.15.42:0/237453299 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f6c5407c120 con 0x7f6c600f7980 2024-02-15T18:29:52.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.418+0000 7f6c5f7fe700 1 --2- 172.21.15.42:0/237453299 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6c48060f20 0x7f6c480633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:52.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.418+0000 7f6c65c8b700 1 -- 172.21.15.42:0/237453299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c60045ad0 con 0x7f6c600f7980 2024-02-15T18:29:52.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.419+0000 7f6c5f7fe700 1 --2- 172.21.15.42:0/237453299 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6c48060f20 0x7f6c480633e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f6c50004630 tx=0x7f6c50009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:52.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.423+0000 7f6c47fff700 1 -- 172.21.15.42:0/237453299 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6c5404ffd0 con 0x7f6c600f7980 2024-02-15T18:29:52.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.644+0000 7f6c65c8b700 1 -- 172.21.15.42:0/237453299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6c600ff0a0 con 0x7f6c600f7980 2024-02-15T18:29:52.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.644+0000 7f6c47fff700 1 -- 172.21.15.42:0/237453299 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1773 (secure 0 0 0) 0x7f6c5404fbe0 con 0x7f6c600f7980 2024-02-15T18:29:52.639 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:29:52.639 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:29:52.639 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:29:52.639 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:29:52.639 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:29:52.639 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:29:52.639 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:29:52.640 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:29:52.640 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:29:52.640 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:29:52.640 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:29:52.640 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:29:52.640 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:29:52.640 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:29:52.640 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:29:52.640 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:29:52.640 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:29:52.640 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:29:52.640 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:29:52.640 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:29:52.640 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:29:52.641 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:29:52.642 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:29:52.642 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:29:52.642 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:29:52.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.646+0000 7f6c65c8b700 1 -- 172.21.15.42:0/237453299 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6c48060f20 msgr2=0x7f6c480633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:52.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.646+0000 7f6c65c8b700 1 --2- 172.21.15.42:0/237453299 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6c48060f20 0x7f6c480633e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f6c50004630 tx=0x7f6c50009b30).stop 2024-02-15T18:29:52.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.646+0000 7f6c65c8b700 1 -- 172.21.15.42:0/237453299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c600f7980 msgr2=0x7f6c60107540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:52.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.646+0000 7f6c65c8b700 1 --2- 172.21.15.42:0/237453299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c600f7980 0x7f6c60107540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f6c540026d0 tx=0x7f6c54009de0).stop 2024-02-15T18:29:52.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.646+0000 7f6c65c8b700 1 -- 172.21.15.42:0/237453299 shutdown_connections 2024-02-15T18:29:52.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.646+0000 7f6c65c8b700 1 --2- 172.21.15.42:0/237453299 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6c48060f20 0x7f6c480633e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.646+0000 7f6c65c8b700 1 --2- 172.21.15.42:0/237453299 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c600f66e0 0x7f6c60107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.646+0000 7f6c65c8b700 1 --2- 172.21.15.42:0/237453299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c600f7980 0x7f6c60107540 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.646+0000 7f6c65c8b700 1 -- 172.21.15.42:0/237453299 >> 172.21.15.42:0/237453299 conn(0x7f6c600f1e40 msgr2=0x7f6c600fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:52.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.646+0000 7f6c65c8b700 1 -- 172.21.15.42:0/237453299 shutdown_connections 2024-02-15T18:29:52.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.646+0000 7f6c65c8b700 1 -- 172.21.15.42:0/237453299 wait complete. 2024-02-15T18:29:52.643 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:29:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.761+0000 7f0508426700 1 -- 172.21.15.42:0/1984376427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05000f55c0 msgr2=0x7f05000f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.761+0000 7f0508426700 1 --2- 172.21.15.42:0/1984376427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05000f55c0 0x7f05000f59e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f04f00099e0 tx=0x7f04f0004d10).stop 2024-02-15T18:29:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.761+0000 7f0508426700 1 -- 172.21.15.42:0/1984376427 shutdown_connections 2024-02-15T18:29:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.761+0000 7f0508426700 1 --2- 172.21.15.42:0/1984376427 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05000f6860 0x7f05000f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.761+0000 7f0508426700 1 --2- 172.21.15.42:0/1984376427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05000f55c0 0x7f05000f59e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.761+0000 7f0508426700 1 -- 172.21.15.42:0/1984376427 >> 172.21.15.42:0/1984376427 conn(0x7f05000f0d40 msgr2=0x7f05000f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.762+0000 7f0508426700 1 -- 172.21.15.42:0/1984376427 shutdown_connections 2024-02-15T18:29:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.762+0000 7f0508426700 1 -- 172.21.15.42:0/1984376427 wait complete. 2024-02-15T18:29:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.762+0000 7f0508426700 1 Processor -- start 2024-02-15T18:29:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.762+0000 7f0508426700 1 -- start start 2024-02-15T18:29:52.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f0508426700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05000f55c0 0x7f0500106070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f0508426700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05000f6860 0x7f0500106560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f0508426700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0500106b10 con 0x7f05000f6860 2024-02-15T18:29:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f0508426700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0500106c50 con 0x7f05000f55c0 2024-02-15T18:29:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f05059c1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05000f6860 0x7f0500106560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f05059c1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05000f6860 0x7f0500106560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37848/0 (socket says 172.21.15.42:37848) 2024-02-15T18:29:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f05059c1700 1 -- 172.21.15.42:0/2604356213 learned_addr learned my addr 172.21.15.42:0/2604356213 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:29:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f05061c2700 1 --2- 172.21.15.42:0/2604356213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05000f55c0 0x7f0500106070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f05059c1700 1 -- 172.21.15.42:0/2604356213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05000f55c0 msgr2=0x7f0500106070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f05059c1700 1 --2- 172.21.15.42:0/2604356213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05000f55c0 0x7f0500106070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f05059c1700 1 -- 172.21.15.42:0/2604356213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f0009640 con 0x7f05000f6860 2024-02-15T18:29:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f05059c1700 1 --2- 172.21.15.42:0/2604356213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05000f6860 0x7f0500106560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f04f4004f60 tx=0x7f04f4009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f04feffd700 1 -- 172.21.15.42:0/2604356213 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04f4007ba0 con 0x7f05000f6860 2024-02-15T18:29:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f04feffd700 1 -- 172.21.15.42:0/2604356213 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f04f40095c0 con 0x7f05000f6860 2024-02-15T18:29:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.763+0000 7f04feffd700 1 -- 172.21.15.42:0/2604356213 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04f4011d90 con 0x7f05000f6860 2024-02-15T18:29:52.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.764+0000 7f0508426700 1 -- 172.21.15.42:0/2604356213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05001075f0 con 0x7f05000f6860 2024-02-15T18:29:52.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.764+0000 7f0508426700 1 -- 172.21.15.42:0/2604356213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0500107b30 con 0x7f05000f6860 2024-02-15T18:29:52.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.765+0000 7f04feffd700 1 -- 172.21.15.42:0/2604356213 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f04f4007d00 con 0x7f05000f6860 2024-02-15T18:29:52.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.765+0000 7f0508426700 1 -- 172.21.15.42:0/2604356213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0500045ad0 con 0x7f05000f6860 2024-02-15T18:29:52.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.765+0000 7f04feffd700 1 --2- 172.21.15.42:0/2604356213 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f04e8060f20 0x7f04e80633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:29:52.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.765+0000 7f04feffd700 1 -- 172.21.15.42:0/2604356213 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f04f407e630 con 0x7f05000f6860 2024-02-15T18:29:52.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.765+0000 7f05061c2700 1 --2- 172.21.15.42:0/2604356213 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f04e8060f20 0x7f04e80633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:29:52.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.766+0000 7f05061c2700 1 --2- 172.21.15.42:0/2604356213 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f04e8060f20 0x7f04e80633e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f04f0003380 tx=0x7f04f0004d10).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:29:52.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.769+0000 7f04feffd700 1 -- 172.21.15.42:0/2604356213 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f04f4052560 con 0x7f05000f6860 2024-02-15T18:29:52.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.962+0000 7f0508426700 1 -- 172.21.15.42:0/2604356213 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f05000ff9b0 con 0x7f04e8060f20 2024-02-15T18:29:52.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.963+0000 7f04feffd700 1 -- 172.21.15.42:0/2604356213 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f05000ff9b0 con 0x7f04e8060f20 2024-02-15T18:29:52.957 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:29:52.957 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:29:52.957 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:29:52.957 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:29:52.957 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:29:52.958 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-02-15T18:29:52.958 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-02-15T18:29:52.958 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-02-15T18:29:52.958 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:29:52.958 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "8/21 daemons upgraded", 2024-02-15T18:29:52.958 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading osd daemons", 2024-02-15T18:29:52.958 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:29:52.958 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:29:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.964+0000 7f0508426700 1 -- 172.21.15.42:0/2604356213 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f04e8060f20 msgr2=0x7f04e80633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.964+0000 7f0508426700 1 --2- 172.21.15.42:0/2604356213 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f04e8060f20 0x7f04e80633e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f04f0003380 tx=0x7f04f0004d10).stop 2024-02-15T18:29:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.965+0000 7f0508426700 1 -- 172.21.15.42:0/2604356213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05000f6860 msgr2=0x7f0500106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:29:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.965+0000 7f0508426700 1 --2- 172.21.15.42:0/2604356213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05000f6860 0x7f0500106560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f04f4004f60 tx=0x7f04f4009de0).stop 2024-02-15T18:29:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.965+0000 7f0508426700 1 -- 172.21.15.42:0/2604356213 shutdown_connections 2024-02-15T18:29:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.965+0000 7f0508426700 1 --2- 172.21.15.42:0/2604356213 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f04e8060f20 0x7f04e80633e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.965+0000 7f0508426700 1 --2- 172.21.15.42:0/2604356213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05000f55c0 0x7f0500106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.965+0000 7f0508426700 1 --2- 172.21.15.42:0/2604356213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05000f6860 0x7f0500106560 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:29:52.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.965+0000 7f0508426700 1 -- 172.21.15.42:0/2604356213 >> 172.21.15.42:0/2604356213 conn(0x7f05000f0d40 msgr2=0x7f05000fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:29:52.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.965+0000 7f0508426700 1 -- 172.21.15.42:0/2604356213 shutdown_connections 2024-02-15T18:29:52.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:29:52.965+0000 7f0508426700 1 -- 172.21.15.42:0/2604356213 wait complete. 2024-02-15T18:29:53.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:51.330859+0000 mgr.smithi042.kfoihb (mgr.24435) 179 : audit [DBG] from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:53.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:51.643327+0000 mgr.smithi042.kfoihb (mgr.24435) 180 : audit [DBG] from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:53.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:52.309809+0000 mon.smithi042 (mon.0) 211 : audit [DBG] from='client.? 172.21.15.42:0/4223324356' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:29:53.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:52.645647+0000 mon.smithi042 (mon.0) 212 : audit [DBG] from='client.? 172.21.15.42:0/237453299' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:29:53.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:51.330859+0000 mgr.smithi042.kfoihb (mgr.24435) 179 : audit [DBG] from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:53.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:51.643327+0000 mgr.smithi042.kfoihb (mgr.24435) 180 : audit [DBG] from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:53.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:52.309809+0000 mon.smithi042 (mon.0) 211 : audit [DBG] from='client.? 172.21.15.42:0/4223324356' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:29:53.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:52.645647+0000 mon.smithi042 (mon.0) 212 : audit [DBG] from='client.? 172.21.15.42:0/237453299' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:29:54.033 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:53 smithi042 systemd[1]: Stopping Ceph osd.2 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:29:54.033 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:51.940857+0000 mgr.smithi042.kfoihb (mgr.24435) 181 : audit [DBG] from='client.34232 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:54.033 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:52.723086+0000 mgr.smithi042.kfoihb (mgr.24435) 182 : cluster [DBG] pgmap v143: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 3 objects/s recovering 2024-02-15T18:29:54.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:51.940857+0000 mgr.smithi042.kfoihb (mgr.24435) 181 : audit [DBG] from='client.34232 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:54.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:52.723086+0000 mgr.smithi042.kfoihb (mgr.24435) 182 : cluster [DBG] pgmap v143: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 3 objects/s recovering 2024-02-15T18:29:54.307 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.2[56906]: debug 2024-02-15T18:29:54.037+0000 7f4a80c05700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:29:54.307 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.2[56906]: debug 2024-02-15T18:29:54.037+0000 7f4a80c05700 -1 osd.2 76 *** Got signal Terminated *** 2024-02-15T18:29:54.307 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.2[56906]: debug 2024-02-15T18:29:54.037+0000 7f4a80c05700 -1 osd.2 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:29:54.779 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:54 smithi042 bash[86039]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.2 2024-02-15T18:29:55.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:52.963766+0000 mgr.smithi042.kfoihb (mgr.24435) 183 : audit [DBG] from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:55.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.283068+0000 mon.smithi042 (mon.0) 213 : cluster [DBG] osd.2 reported immediately failed by osd.5 2024-02-15T18:29:55.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.283108+0000 mon.smithi042 (mon.0) 214 : cluster [INF] osd.2 failed (root=default,host=smithi042) (connection refused reported by osd.5) 2024-02-15T18:29:55.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.283291+0000 mon.smithi042 (mon.0) 215 : cluster [DBG] osd.2 reported immediately failed by osd.3 2024-02-15T18:29:55.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.283336+0000 mon.smithi042 (mon.0) 216 : cluster [DBG] osd.2 reported immediately failed by osd.5 2024-02-15T18:29:55.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.283435+0000 mon.smithi042 (mon.0) 217 : cluster [DBG] osd.2 reported immediately failed by osd.1 2024-02-15T18:29:55.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.283518+0000 mon.smithi042 (mon.0) 218 : cluster [DBG] osd.2 reported immediately failed by osd.4 2024-02-15T18:29:55.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.283587+0000 mon.smithi042 (mon.0) 219 : cluster [DBG] osd.2 reported immediately failed by osd.3 2024-02-15T18:29:55.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.283655+0000 mon.smithi042 (mon.0) 220 : cluster [DBG] osd.2 reported immediately failed by osd.1 2024-02-15T18:29:55.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.283765+0000 mon.smithi042 (mon.0) 221 : cluster [DBG] osd.2 reported immediately failed by osd.0 2024-02-15T18:29:55.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.283835+0000 mon.smithi042 (mon.0) 222 : cluster [DBG] osd.2 reported immediately failed by osd.4 2024-02-15T18:29:55.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.283926+0000 mon.smithi042 (mon.0) 223 : cluster [DBG] osd.2 reported immediately failed by osd.0 2024-02-15T18:29:55.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.683778+0000 mon.smithi042 (mon.0) 224 : cluster [DBG] osd.2 reported immediately failed by osd.3 2024-02-15T18:29:55.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.683955+0000 mon.smithi042 (mon.0) 225 : cluster [DBG] osd.2 reported immediately failed by osd.5 2024-02-15T18:29:55.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.684107+0000 mon.smithi042 (mon.0) 226 : cluster [DBG] osd.2 reported immediately failed by osd.4 2024-02-15T18:29:55.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.684198+0000 mon.smithi042 (mon.0) 227 : cluster [DBG] osd.2 reported immediately failed by osd.3 2024-02-15T18:29:55.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.684403+0000 mon.smithi042 (mon.0) 228 : cluster [DBG] osd.2 reported immediately failed by osd.5 2024-02-15T18:29:55.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.684570+0000 mon.smithi042 (mon.0) 229 : cluster [DBG] osd.2 reported immediately failed by osd.1 2024-02-15T18:29:55.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.684713+0000 mon.smithi042 (mon.0) 230 : cluster [DBG] osd.2 reported immediately failed by osd.4 2024-02-15T18:29:55.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.684800+0000 mon.smithi042 (mon.0) 231 : cluster [DBG] osd.2 reported immediately failed by osd.1 2024-02-15T18:29:55.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.685066+0000 mon.smithi042 (mon.0) 232 : cluster [DBG] osd.2 reported immediately failed by osd.0 2024-02-15T18:29:55.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.685251+0000 mon.smithi042 (mon.0) 233 : cluster [DBG] osd.2 reported immediately failed by osd.0 2024-02-15T18:29:55.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:52.963766+0000 mgr.smithi042.kfoihb (mgr.24435) 183 : audit [DBG] from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:29:55.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.283068+0000 mon.smithi042 (mon.0) 213 : cluster [DBG] osd.2 reported immediately failed by osd.5 2024-02-15T18:29:55.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.283108+0000 mon.smithi042 (mon.0) 214 : cluster [INF] osd.2 failed (root=default,host=smithi042) (connection refused reported by osd.5) 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.283291+0000 mon.smithi042 (mon.0) 215 : cluster [DBG] osd.2 reported immediately failed by osd.3 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.283336+0000 mon.smithi042 (mon.0) 216 : cluster [DBG] osd.2 reported immediately failed by osd.5 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.283435+0000 mon.smithi042 (mon.0) 217 : cluster [DBG] osd.2 reported immediately failed by osd.1 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.283518+0000 mon.smithi042 (mon.0) 218 : cluster [DBG] osd.2 reported immediately failed by osd.4 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.283587+0000 mon.smithi042 (mon.0) 219 : cluster [DBG] osd.2 reported immediately failed by osd.3 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.283655+0000 mon.smithi042 (mon.0) 220 : cluster [DBG] osd.2 reported immediately failed by osd.1 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.283765+0000 mon.smithi042 (mon.0) 221 : cluster [DBG] osd.2 reported immediately failed by osd.0 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.283835+0000 mon.smithi042 (mon.0) 222 : cluster [DBG] osd.2 reported immediately failed by osd.4 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.283926+0000 mon.smithi042 (mon.0) 223 : cluster [DBG] osd.2 reported immediately failed by osd.0 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.683778+0000 mon.smithi042 (mon.0) 224 : cluster [DBG] osd.2 reported immediately failed by osd.3 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.683955+0000 mon.smithi042 (mon.0) 225 : cluster [DBG] osd.2 reported immediately failed by osd.5 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.684107+0000 mon.smithi042 (mon.0) 226 : cluster [DBG] osd.2 reported immediately failed by osd.4 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.684198+0000 mon.smithi042 (mon.0) 227 : cluster [DBG] osd.2 reported immediately failed by osd.3 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.684403+0000 mon.smithi042 (mon.0) 228 : cluster [DBG] osd.2 reported immediately failed by osd.5 2024-02-15T18:29:55.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.684570+0000 mon.smithi042 (mon.0) 229 : cluster [DBG] osd.2 reported immediately failed by osd.1 2024-02-15T18:29:55.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.684713+0000 mon.smithi042 (mon.0) 230 : cluster [DBG] osd.2 reported immediately failed by osd.4 2024-02-15T18:29:55.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.684800+0000 mon.smithi042 (mon.0) 231 : cluster [DBG] osd.2 reported immediately failed by osd.1 2024-02-15T18:29:55.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.685066+0000 mon.smithi042 (mon.0) 232 : cluster [DBG] osd.2 reported immediately failed by osd.0 2024-02-15T18:29:55.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.685251+0000 mon.smithi042 (mon.0) 233 : cluster [DBG] osd.2 reported immediately failed by osd.0 2024-02-15T18:29:56.017 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.723374+0000 mgr.smithi042.kfoihb (mgr.24435) 184 : cluster [DBG] pgmap v144: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s; 0 B/s, 3 objects/s recovering 2024-02-15T18:29:56.017 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.801696+0000 mon.smithi042 (mon.0) 234 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:29:56.018 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:54.810068+0000 mon.smithi042 (mon.0) 235 : cluster [DBG] osdmap e77: 6 total, 5 up, 6 in 2024-02-15T18:29:56.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.723374+0000 mgr.smithi042.kfoihb (mgr.24435) 184 : cluster [DBG] pgmap v144: 65 pgs: 65 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s; 0 B/s, 3 objects/s recovering 2024-02-15T18:29:56.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.801696+0000 mon.smithi042 (mon.0) 234 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:29:56.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:54.810068+0000 mon.smithi042 (mon.0) 235 : cluster [DBG] osdmap e77: 6 total, 5 up, 6 in 2024-02-15T18:29:56.557 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:56 smithi042 systemd[1]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.2.service: Succeeded. 2024-02-15T18:29:56.557 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:56 smithi042 systemd[1]: Stopped Ceph osd.2 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:29:56.557 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:56 smithi042 systemd[1]: Starting Ceph osd.2 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:29:57.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:55.809629+0000 mon.smithi042 (mon.0) 236 : cluster [DBG] osdmap e78: 6 total, 5 up, 6 in 2024-02-15T18:29:57.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:55.837030+0000 mon.smithi042 (mon.0) 237 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:57.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:55.916133+0000 mon.smithi042 (mon.0) 238 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:57.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:55.809629+0000 mon.smithi042 (mon.0) 236 : cluster [DBG] osdmap e78: 6 total, 5 up, 6 in 2024-02-15T18:29:57.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:55.837030+0000 mon.smithi042 (mon.0) 237 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:57.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:55.916133+0000 mon.smithi042 (mon.0) 238 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:57.556 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:57 smithi042 bash[86343]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-02-15T18:29:57.557 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:57 smithi042 bash[86343]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-15T18:29:57.557 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:57 smithi042 bash[86343]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-2 --no-mon-config 2024-02-15T18:29:57.557 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:57 smithi042 bash[86343]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-15T18:29:57.557 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:57 smithi042 bash[86343]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-2/block 2024-02-15T18:29:57.557 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:57 smithi042 bash[86343]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-15T18:29:57.557 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:57 smithi042 bash[86343]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-15T18:29:57.557 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:57 smithi042 bash[86343]: --> ceph-volume lvm activate successful for osd ID: 2 2024-02-15T18:29:58.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:56.723851+0000 mgr.smithi042.kfoihb (mgr.24435) 185 : cluster [DBG] pgmap v147: 65 pgs: 4 active+undersized+degraded, 6 stale+active+clean, 7 active+undersized, 48 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 8/210 objects degraded (3.810%) 2024-02-15T18:29:58.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:56.963879+0000 mon.smithi042 (mon.0) 239 : cluster [WRN] Health check failed: Degraded data redundancy: 8/210 objects degraded (3.810%), 4 pgs degraded (PG_DEGRADED) 2024-02-15T18:29:58.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:56.723851+0000 mgr.smithi042.kfoihb (mgr.24435) 185 : cluster [DBG] pgmap v147: 65 pgs: 4 active+undersized+degraded, 6 stale+active+clean, 7 active+undersized, 48 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 8/210 objects degraded (3.810%) 2024-02-15T18:29:58.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:56.963879+0000 mon.smithi042 (mon.0) 239 : cluster [WRN] Health check failed: Degraded data redundancy: 8/210 objects degraded (3.810%), 4 pgs degraded (PG_DEGRADED) 2024-02-15T18:29:58.651 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-2[86626]: debug 2024-02-15T18:29:58.358+0000 7fe5666af700 1 -- 172.21.15.42:0/2047345402 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563ab0924680 con 0x563ab08eb400 2024-02-15T18:29:58.651 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:58 smithi042 bash[86612]: 2e26b1ec292a080e55ff3b0f17075655cf62182651bff50ede69587de58afc4c 2024-02-15T18:29:58.651 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:58 smithi042 systemd[1]: Started Ceph osd.2 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:29:59.057 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:29:58 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-2[86626]: debug 2024-02-15T18:29:58.889+0000 7fe56e13d380 -1 Falling back to public interface 2024-02-15T18:29:59.807 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:58.483236+0000 mon.smithi042 (mon.0) 240 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:59.807 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:58.483973+0000 mon.smithi042 (mon.0) 241 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:29:59.807 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:29:58.724328+0000 mgr.smithi042.kfoihb (mgr.24435) 186 : cluster [DBG] pgmap v148: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29/210 objects degraded (13.810%) 2024-02-15T18:29:59.807 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:29:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:29:58.727413+0000 mon.smithi042 (mon.0) 242 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:59.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:58.483236+0000 mon.smithi042 (mon.0) 240 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:29:59.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:58.483973+0000 mon.smithi042 (mon.0) 241 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:29:59.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:29:58.724328+0000 mgr.smithi042.kfoihb (mgr.24435) 186 : cluster [DBG] pgmap v148: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29/210 objects degraded (13.810%) 2024-02-15T18:29:59.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:29:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:29:58.727413+0000 mon.smithi042 (mon.0) 242 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:01.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000203+0000 mon.smithi042 (mon.0) 243 : cluster [WRN] Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 29/210 objects degraded (13.810%), 11 pgs degraded 2024-02-15T18:30:01.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000241+0000 mon.smithi042 (mon.0) 244 : cluster [WRN] [WRN] OSD_DOWN: 1 osds down 2024-02-15T18:30:01.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000258+0000 mon.smithi042 (mon.0) 245 : cluster [WRN] osd.2 (root=default,host=smithi042) is down 2024-02-15T18:30:01.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000273+0000 mon.smithi042 (mon.0) 246 : cluster [WRN] [WRN] PG_DEGRADED: Degraded data redundancy: 29/210 objects degraded (13.810%), 11 pgs degraded 2024-02-15T18:30:01.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000287+0000 mon.smithi042 (mon.0) 247 : cluster [WRN] pg 2.1 is active+undersized+degraded, acting [1,0] 2024-02-15T18:30:01.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000301+0000 mon.smithi042 (mon.0) 248 : cluster [WRN] pg 2.3 is active+undersized+degraded, acting [5,1] 2024-02-15T18:30:01.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000317+0000 mon.smithi042 (mon.0) 249 : cluster [WRN] pg 2.7 is active+undersized+degraded, acting [3,4] 2024-02-15T18:30:01.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000336+0000 mon.smithi042 (mon.0) 250 : cluster [WRN] pg 2.c is active+undersized+degraded, acting [3,0] 2024-02-15T18:30:01.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000351+0000 mon.smithi042 (mon.0) 251 : cluster [WRN] pg 2.d is active+undersized+degraded, acting [1,3] 2024-02-15T18:30:01.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000366+0000 mon.smithi042 (mon.0) 252 : cluster [WRN] pg 2.e is active+undersized+degraded, acting [0,3] 2024-02-15T18:30:01.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000381+0000 mon.smithi042 (mon.0) 253 : cluster [WRN] pg 2.10 is active+undersized+degraded, acting [1,0] 2024-02-15T18:30:01.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000398+0000 mon.smithi042 (mon.0) 254 : cluster [WRN] pg 2.13 is active+undersized+degraded, acting [0,4] 2024-02-15T18:30:01.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000413+0000 mon.smithi042 (mon.0) 255 : cluster [WRN] pg 2.16 is active+undersized+degraded, acting [5,3] 2024-02-15T18:30:01.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000429+0000 mon.smithi042 (mon.0) 256 : cluster [WRN] pg 2.1c is active+undersized+degraded, acting [4,5] 2024-02-15T18:30:01.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.000445+0000 mon.smithi042 (mon.0) 257 : cluster [WRN] pg 2.1e is active+undersized+degraded, acting [0,5] 2024-02-15T18:30:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000203+0000 mon.smithi042 (mon.0) 243 : cluster [WRN] Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 29/210 objects degraded (13.810%), 11 pgs degraded 2024-02-15T18:30:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000241+0000 mon.smithi042 (mon.0) 244 : cluster [WRN] [WRN] OSD_DOWN: 1 osds down 2024-02-15T18:30:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000258+0000 mon.smithi042 (mon.0) 245 : cluster [WRN] osd.2 (root=default,host=smithi042) is down 2024-02-15T18:30:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000273+0000 mon.smithi042 (mon.0) 246 : cluster [WRN] [WRN] PG_DEGRADED: Degraded data redundancy: 29/210 objects degraded (13.810%), 11 pgs degraded 2024-02-15T18:30:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000287+0000 mon.smithi042 (mon.0) 247 : cluster [WRN] pg 2.1 is active+undersized+degraded, acting [1,0] 2024-02-15T18:30:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000301+0000 mon.smithi042 (mon.0) 248 : cluster [WRN] pg 2.3 is active+undersized+degraded, acting [5,1] 2024-02-15T18:30:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000317+0000 mon.smithi042 (mon.0) 249 : cluster [WRN] pg 2.7 is active+undersized+degraded, acting [3,4] 2024-02-15T18:30:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000336+0000 mon.smithi042 (mon.0) 250 : cluster [WRN] pg 2.c is active+undersized+degraded, acting [3,0] 2024-02-15T18:30:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000351+0000 mon.smithi042 (mon.0) 251 : cluster [WRN] pg 2.d is active+undersized+degraded, acting [1,3] 2024-02-15T18:30:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000366+0000 mon.smithi042 (mon.0) 252 : cluster [WRN] pg 2.e is active+undersized+degraded, acting [0,3] 2024-02-15T18:30:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000381+0000 mon.smithi042 (mon.0) 253 : cluster [WRN] pg 2.10 is active+undersized+degraded, acting [1,0] 2024-02-15T18:30:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000398+0000 mon.smithi042 (mon.0) 254 : cluster [WRN] pg 2.13 is active+undersized+degraded, acting [0,4] 2024-02-15T18:30:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000413+0000 mon.smithi042 (mon.0) 255 : cluster [WRN] pg 2.16 is active+undersized+degraded, acting [5,3] 2024-02-15T18:30:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000429+0000 mon.smithi042 (mon.0) 256 : cluster [WRN] pg 2.1c is active+undersized+degraded, acting [4,5] 2024-02-15T18:30:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.000445+0000 mon.smithi042 (mon.0) 257 : cluster [WRN] pg 2.1e is active+undersized+degraded, acting [0,5] 2024-02-15T18:30:02.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:00.724741+0000 mgr.smithi042.kfoihb (mgr.24435) 187 : cluster [DBG] pgmap v149: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29/210 objects degraded (13.810%) 2024-02-15T18:30:02.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:00.724741+0000 mgr.smithi042.kfoihb (mgr.24435) 187 : cluster [DBG] pgmap v149: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29/210 objects degraded (13.810%) 2024-02-15T18:30:03.807 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:02.725397+0000 mgr.smithi042.kfoihb (mgr.24435) 188 : cluster [DBG] pgmap v150: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29/210 objects degraded (13.810%) 2024-02-15T18:30:04.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:02.725397+0000 mgr.smithi042.kfoihb (mgr.24435) 188 : cluster [DBG] pgmap v150: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29/210 objects degraded (13.810%) 2024-02-15T18:30:05.557 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-2[86626]: debug 2024-02-15T18:30:05.145+0000 7fe56e13d380 -1 osd.2 76 log_to_monitors {default=true} 2024-02-15T18:30:05.557 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-2[86626]: debug 2024-02-15T18:30:05.244+0000 7fe55e622700 -1 osd.2 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-15T18:30:05.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:04.231141+0000 mon.smithi042 (mon.0) 258 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:05.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:04.231979+0000 mon.smithi042 (mon.0) 259 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:05.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:04.232638+0000 mon.smithi042 (mon.0) 260 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:30:05.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:30:04.233164+0000 mgr.smithi042.kfoihb (mgr.24435) 189 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:30:05.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:04.236182+0000 mon.smithi042 (mon.0) 261 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:05.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:04.240885+0000 mon.smithi042 (mon.0) 262 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:05.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:04.242974+0000 mon.smithi042 (mon.0) 263 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:05.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:04.244075+0000 mon.smithi042 (mon.0) 264 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:05.558 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:04.245173+0000 mon.smithi042 (mon.0) 265 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:05.558 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:04.246304+0000 mon.smithi042 (mon.0) 266 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:30:05.558 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:04.246459+0000 mgr.smithi042.kfoihb (mgr.24435) 190 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:30:05.558 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:30:04.247081+0000 mgr.smithi042.kfoihb (mgr.24435) 191 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-15T18:30:05.558 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:04.725795+0000 mgr.smithi042.kfoihb (mgr.24435) 192 : cluster [DBG] pgmap v151: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29/210 objects degraded (13.810%) 2024-02-15T18:30:05.558 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:05.149125+0000 mon.smithi042 (mon.0) 267 : audit [INF] from='osd.2 [v2:172.21.15.42:6818/1789911666,v1:172.21.15.42:6819/1789911666]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-15T18:30:05.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:04.231141+0000 mon.smithi042 (mon.0) 258 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:05.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:04.231979+0000 mon.smithi042 (mon.0) 259 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:05.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:04.232638+0000 mon.smithi042 (mon.0) 260 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:30:05.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:30:04.233164+0000 mgr.smithi042.kfoihb (mgr.24435) 189 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:30:05.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:04.236182+0000 mon.smithi042 (mon.0) 261 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:05.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:04.240885+0000 mon.smithi042 (mon.0) 262 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:05.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:04.242974+0000 mon.smithi042 (mon.0) 263 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:05.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:04.244075+0000 mon.smithi042 (mon.0) 264 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:05.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:04.245173+0000 mon.smithi042 (mon.0) 265 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:05.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:04.246304+0000 mon.smithi042 (mon.0) 266 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:30:05.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:04.246459+0000 mgr.smithi042.kfoihb (mgr.24435) 190 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:30:05.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:30:04.247081+0000 mgr.smithi042.kfoihb (mgr.24435) 191 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-15T18:30:05.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:04.725795+0000 mgr.smithi042.kfoihb (mgr.24435) 192 : cluster [DBG] pgmap v151: 65 pgs: 11 active+undersized+degraded, 16 active+undersized, 38 active+clean; 193 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29/210 objects degraded (13.810%) 2024-02-15T18:30:05.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:05.149125+0000 mon.smithi042 (mon.0) 267 : audit [INF] from='osd.2 [v2:172.21.15.42:6818/1789911666,v1:172.21.15.42:6819/1789911666]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-15T18:30:06.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:05.238950+0000 mon.smithi042 (mon.0) 268 : audit [INF] from='osd.2 [v2:172.21.15.42:6818/1789911666,v1:172.21.15.42:6819/1789911666]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-15T18:30:06.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:05.239001+0000 mon.smithi042 (mon.0) 269 : cluster [DBG] osdmap e79: 6 total, 5 up, 6 in 2024-02-15T18:30:06.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:05.239409+0000 mon.smithi042 (mon.0) 270 : audit [INF] from='osd.2 [v2:172.21.15.42:6818/1789911666,v1:172.21.15.42:6819/1789911666]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-02-15T18:30:06.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:05.963890+0000 mon.smithi042 (mon.0) 271 : cluster [WRN] Health check update: Degraded data redundancy: 29/210 objects degraded (13.810%), 11 pgs degraded (PG_DEGRADED) 2024-02-15T18:30:06.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:05.238950+0000 mon.smithi042 (mon.0) 268 : audit [INF] from='osd.2 [v2:172.21.15.42:6818/1789911666,v1:172.21.15.42:6819/1789911666]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-15T18:30:06.723 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:05.239001+0000 mon.smithi042 (mon.0) 269 : cluster [DBG] osdmap e79: 6 total, 5 up, 6 in 2024-02-15T18:30:06.723 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:05.239409+0000 mon.smithi042 (mon.0) 270 : audit [INF] from='osd.2 [v2:172.21.15.42:6818/1789911666,v1:172.21.15.42:6819/1789911666]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-02-15T18:30:06.723 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:05.963890+0000 mon.smithi042 (mon.0) 271 : cluster [WRN] Health check update: Degraded data redundancy: 29/210 objects degraded (13.810%), 11 pgs degraded (PG_DEGRADED) 2024-02-15T18:30:07.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:06.238069+0000 mon.smithi042 (mon.0) 272 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:30:07.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:06.241612+0000 mon.smithi042 (mon.0) 273 : cluster [INF] osd.2 [v2:172.21.15.42:6818/1789911666,v1:172.21.15.42:6819/1789911666] boot 2024-02-15T18:30:07.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:06.241680+0000 mon.smithi042 (mon.0) 274 : cluster [DBG] osdmap e80: 6 total, 6 up, 6 in 2024-02-15T18:30:07.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:06.244418+0000 mon.smithi042 (mon.0) 275 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:30:07.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:06.726363+0000 mgr.smithi042.kfoihb (mgr.24435) 193 : cluster [DBG] pgmap v154: 65 pgs: 3 peering, 10 active+undersized+degraded, 14 active+undersized, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 28/210 objects degraded (13.333%) 2024-02-15T18:30:07.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:07.244362+0000 mon.smithi042 (mon.0) 276 : cluster [DBG] osdmap e81: 6 total, 6 up, 6 in 2024-02-15T18:30:07.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:06.238069+0000 mon.smithi042 (mon.0) 272 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:30:07.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:06.241612+0000 mon.smithi042 (mon.0) 273 : cluster [INF] osd.2 [v2:172.21.15.42:6818/1789911666,v1:172.21.15.42:6819/1789911666] boot 2024-02-15T18:30:07.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:06.241680+0000 mon.smithi042 (mon.0) 274 : cluster [DBG] osdmap e80: 6 total, 6 up, 6 in 2024-02-15T18:30:07.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:06.244418+0000 mon.smithi042 (mon.0) 275 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-15T18:30:07.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:06.726363+0000 mgr.smithi042.kfoihb (mgr.24435) 193 : cluster [DBG] pgmap v154: 65 pgs: 3 peering, 10 active+undersized+degraded, 14 active+undersized, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 28/210 objects degraded (13.333%) 2024-02-15T18:30:07.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:07.244362+0000 mon.smithi042 (mon.0) 276 : cluster [DBG] osdmap e81: 6 total, 6 up, 6 in 2024-02-15T18:30:10.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:08.726816+0000 mgr.smithi042.kfoihb (mgr.24435) 194 : cluster [DBG] pgmap v156: 65 pgs: 13 peering, 6 active+undersized+degraded, 8 active+undersized, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 18/210 objects degraded (8.571%) 2024-02-15T18:30:10.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:08.726816+0000 mgr.smithi042.kfoihb (mgr.24435) 194 : cluster [DBG] pgmap v156: 65 pgs: 13 peering, 6 active+undersized+degraded, 8 active+undersized, 38 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 18/210 objects degraded (8.571%) 2024-02-15T18:30:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:10.727381+0000 mgr.smithi042.kfoihb (mgr.24435) 195 : cluster [DBG] pgmap v157: 65 pgs: 13 peering, 1 active+undersized+degraded, 4 active+undersized, 47 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2/210 objects degraded (0.952%) 2024-02-15T18:30:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:10.835349+0000 mon.smithi042 (mon.0) 277 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:10.964726+0000 mon.smithi042 (mon.0) 278 : cluster [WRN] Health check update: Degraded data redundancy: 2/210 objects degraded (0.952%), 1 pg degraded (PG_DEGRADED) 2024-02-15T18:30:12.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:10.727381+0000 mgr.smithi042.kfoihb (mgr.24435) 195 : cluster [DBG] pgmap v157: 65 pgs: 13 peering, 1 active+undersized+degraded, 4 active+undersized, 47 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2/210 objects degraded (0.952%) 2024-02-15T18:30:12.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:10.835349+0000 mon.smithi042 (mon.0) 277 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:12.311 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:10.964726+0000 mon.smithi042 (mon.0) 278 : cluster [WRN] Health check update: Degraded data redundancy: 2/210 objects degraded (0.952%), 1 pg degraded (PG_DEGRADED) 2024-02-15T18:30:13.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:12.835734+0000 mon.smithi042 (mon.0) 279 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/210 objects degraded (0.952%), 1 pg degraded) 2024-02-15T18:30:13.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:12.835790+0000 mon.smithi042 (mon.0) 280 : cluster [INF] Cluster is now healthy 2024-02-15T18:30:13.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:12.835734+0000 mon.smithi042 (mon.0) 279 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/210 objects degraded (0.952%), 1 pg degraded) 2024-02-15T18:30:13.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:12.835790+0000 mon.smithi042 (mon.0) 280 : cluster [INF] Cluster is now healthy 2024-02-15T18:30:14.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:12.727964+0000 mgr.smithi042.kfoihb (mgr.24435) 196 : cluster [DBG] pgmap v158: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:30:14.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:12.727964+0000 mgr.smithi042.kfoihb (mgr.24435) 196 : cluster [DBG] pgmap v158: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:30:16.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:14.728575+0000 mgr.smithi042.kfoihb (mgr.24435) 197 : cluster [DBG] pgmap v159: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:30:16.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:14.728575+0000 mgr.smithi042.kfoihb (mgr.24435) 197 : cluster [DBG] pgmap v159: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-02-15T18:30:17.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:16.729246+0000 mgr.smithi042.kfoihb (mgr.24435) 198 : cluster [DBG] pgmap v160: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 716 B/s rd, 0 op/s 2024-02-15T18:30:17.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:16.729246+0000 mgr.smithi042.kfoihb (mgr.24435) 198 : cluster [DBG] pgmap v160: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 716 B/s rd, 0 op/s 2024-02-15T18:30:20.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:18.729784+0000 mgr.smithi042.kfoihb (mgr.24435) 199 : cluster [DBG] pgmap v161: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 623 B/s rd, 0 op/s 2024-02-15T18:30:20.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:19.252357+0000 mon.smithi042 (mon.0) 281 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:30:20.100 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:18.729784+0000 mgr.smithi042.kfoihb (mgr.24435) 199 : cluster [DBG] pgmap v161: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 623 B/s rd, 0 op/s 2024-02-15T18:30:20.100 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:19.252357+0000 mon.smithi042 (mon.0) 281 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:30:21.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:19.252597+0000 mgr.smithi042.kfoihb (mgr.24435) 200 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:30:21.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:30:19.253715+0000 mgr.smithi042.kfoihb (mgr.24435) 201 : cephadm [INF] Upgrade: osd.3 is safe to restart 2024-02-15T18:30:21.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:19.252597+0000 mgr.smithi042.kfoihb (mgr.24435) 200 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-15T18:30:21.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:30:19.253715+0000 mgr.smithi042.kfoihb (mgr.24435) 201 : cephadm [INF] Upgrade: osd.3 is safe to restart 2024-02-15T18:30:22.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:20.730228+0000 mgr.smithi042.kfoihb (mgr.24435) 202 : cluster [DBG] pgmap v162: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 597 B/s rd, 0 op/s 2024-02-15T18:30:22.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:30:20.912695+0000 mgr.smithi042.kfoihb (mgr.24435) 203 : cephadm [INF] Upgrade: Updating osd.3 2024-02-15T18:30:22.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:20.917646+0000 mon.smithi042 (mon.0) 282 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:22.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:20.918533+0000 mon.smithi042 (mon.0) 283 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-15T18:30:22.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:20.919394+0000 mon.smithi042 (mon.0) 284 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:22.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:30:20.920645+0000 mgr.smithi042.kfoihb (mgr.24435) 204 : cephadm [INF] Deploying daemon osd.3 on smithi191 2024-02-15T18:30:22.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:20.928419+0000 mon.smithi042 (mon.0) 285 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:22.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:20.730228+0000 mgr.smithi042.kfoihb (mgr.24435) 202 : cluster [DBG] pgmap v162: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 597 B/s rd, 0 op/s 2024-02-15T18:30:22.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:30:20.912695+0000 mgr.smithi042.kfoihb (mgr.24435) 203 : cephadm [INF] Upgrade: Updating osd.3 2024-02-15T18:30:22.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:20.917646+0000 mon.smithi042 (mon.0) 282 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:22.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:20.918533+0000 mon.smithi042 (mon.0) 283 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-15T18:30:22.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:20.919394+0000 mon.smithi042 (mon.0) 284 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:22.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:30:20.920645+0000 mgr.smithi042.kfoihb (mgr.24435) 204 : cephadm [INF] Deploying daemon osd.3 on smithi191 2024-02-15T18:30:22.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:20.928419+0000 mon.smithi042 (mon.0) 285 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:23.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.082+0000 7f3fc414b700 1 -- 172.21.15.42:0/1793628358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fbc0f66e0 msgr2=0x7f3fbc0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.082+0000 7f3fc414b700 1 --2- 172.21.15.42:0/1793628358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fbc0f66e0 0x7f3fbc0f6b00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f3fb8004660 tx=0x7f3fb8009b30).stop 2024-02-15T18:30:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.082+0000 7f3fc414b700 1 -- 172.21.15.42:0/1793628358 shutdown_connections 2024-02-15T18:30:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.082+0000 7f3fc414b700 1 --2- 172.21.15.42:0/1793628358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fbc0f7980 0x7f3fbc0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.082+0000 7f3fc414b700 1 --2- 172.21.15.42:0/1793628358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fbc0f66e0 0x7f3fbc0f6b00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.082+0000 7f3fc414b700 1 -- 172.21.15.42:0/1793628358 >> 172.21.15.42:0/1793628358 conn(0x7f3fbc0f1e40 msgr2=0x7f3fbc0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.082+0000 7f3fc414b700 1 -- 172.21.15.42:0/1793628358 shutdown_connections 2024-02-15T18:30:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.082+0000 7f3fc414b700 1 -- 172.21.15.42:0/1793628358 wait complete. 2024-02-15T18:30:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.083+0000 7f3fc414b700 1 Processor -- start 2024-02-15T18:30:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.083+0000 7f3fc414b700 1 -- start start 2024-02-15T18:30:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.083+0000 7f3fc414b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fbc0f66e0 0x7f3fbc109250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.083+0000 7f3fc414b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fbc0f7980 0x7f3fbc109740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.083+0000 7f3fc414b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc109cf0 con 0x7f3fbc0f7980 2024-02-15T18:30:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.083+0000 7f3fc414b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc109e30 con 0x7f3fbc0f66e0 2024-02-15T18:30:23.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.083+0000 7f3fc16e6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fbc0f7980 0x7f3fbc109740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.083+0000 7f3fc16e6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fbc0f7980 0x7f3fbc109740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46118/0 (socket says 172.21.15.42:46118) 2024-02-15T18:30:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.083+0000 7f3fc16e6700 1 -- 172.21.15.42:0/2953532279 learned_addr learned my addr 172.21.15.42:0/2953532279 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:30:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.083+0000 7f3fc1ee7700 1 --2- 172.21.15.42:0/2953532279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fbc0f66e0 0x7f3fbc109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.083+0000 7f3fc16e6700 1 -- 172.21.15.42:0/2953532279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fbc0f66e0 msgr2=0x7f3fbc109250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.083+0000 7f3fc16e6700 1 --2- 172.21.15.42:0/2953532279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fbc0f66e0 0x7f3fbc109250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.083+0000 7f3fc16e6700 1 -- 172.21.15.42:0/2953532279 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb8005040 con 0x7f3fbc0f7980 2024-02-15T18:30:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.083+0000 7f3fc16e6700 1 --2- 172.21.15.42:0/2953532279 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fbc0f7980 0x7f3fbc109740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f3fac00e7f0 tx=0x7f3fac00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.084+0000 7f3fb27fc700 1 -- 172.21.15.42:0/2953532279 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fac0092e0 con 0x7f3fbc0f7980 2024-02-15T18:30:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.084+0000 7f3fb27fc700 1 -- 172.21.15.42:0/2953532279 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f3fac00f040 con 0x7f3fbc0f7980 2024-02-15T18:30:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.084+0000 7f3fc414b700 1 -- 172.21.15.42:0/2953532279 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc10a7d0 con 0x7f3fbc0f7980 2024-02-15T18:30:23.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.084+0000 7f3fb27fc700 1 -- 172.21.15.42:0/2953532279 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fac014850 con 0x7f3fbc0f7980 2024-02-15T18:30:23.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.084+0000 7f3fc414b700 1 -- 172.21.15.42:0/2953532279 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc10ad90 con 0x7f3fbc0f7980 2024-02-15T18:30:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.085+0000 7f3fc414b700 1 -- 172.21.15.42:0/2953532279 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fbc045ad0 con 0x7f3fbc0f7980 2024-02-15T18:30:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.085+0000 7f3fb27fc700 1 -- 172.21.15.42:0/2953532279 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f3fac01b030 con 0x7f3fbc0f7980 2024-02-15T18:30:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.085+0000 7f3fb27fc700 1 --2- 172.21.15.42:0/2953532279 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3fa8060f20 0x7f3fa80633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.086+0000 7f3fb27fc700 1 -- 172.21.15.42:0/2953532279 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f3fac07bd90 con 0x7f3fbc0f7980 2024-02-15T18:30:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.086+0000 7f3fc1ee7700 1 --2- 172.21.15.42:0/2953532279 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3fa8060f20 0x7f3fa80633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:23.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.086+0000 7f3fc1ee7700 1 --2- 172.21.15.42:0/2953532279 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3fa8060f20 0x7f3fa80633e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f3fb8009720 tx=0x7f3fb8016040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:23.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.090+0000 7f3fb27fc700 1 -- 172.21.15.42:0/2953532279 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3fac04fcf0 con 0x7f3fbc0f7980 2024-02-15T18:30:23.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.283+0000 7f3fc414b700 1 -- 172.21.15.42:0/2953532279 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fbc0ff0c0 con 0x7f3fa8060f20 2024-02-15T18:30:23.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.284+0000 7f3fb27fc700 1 -- 172.21.15.42:0/2953532279 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f3fbc0ff0c0 con 0x7f3fa8060f20 2024-02-15T18:30:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.286+0000 7f3fc414b700 1 -- 172.21.15.42:0/2953532279 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3fa8060f20 msgr2=0x7f3fa80633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.286+0000 7f3fc414b700 1 --2- 172.21.15.42:0/2953532279 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3fa8060f20 0x7f3fa80633e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f3fb8009720 tx=0x7f3fb8016040).stop 2024-02-15T18:30:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.286+0000 7f3fc414b700 1 -- 172.21.15.42:0/2953532279 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fbc0f7980 msgr2=0x7f3fbc109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.286+0000 7f3fc414b700 1 --2- 172.21.15.42:0/2953532279 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fbc0f7980 0x7f3fbc109740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f3fac00e7f0 tx=0x7f3fac00b040).stop 2024-02-15T18:30:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.286+0000 7f3fc414b700 1 -- 172.21.15.42:0/2953532279 shutdown_connections 2024-02-15T18:30:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.286+0000 7f3fc414b700 1 --2- 172.21.15.42:0/2953532279 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3fa8060f20 0x7f3fa80633e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.286+0000 7f3fc414b700 1 --2- 172.21.15.42:0/2953532279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fbc0f66e0 0x7f3fbc109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.286+0000 7f3fc414b700 1 --2- 172.21.15.42:0/2953532279 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fbc0f7980 0x7f3fbc109740 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.286+0000 7f3fc414b700 1 -- 172.21.15.42:0/2953532279 >> 172.21.15.42:0/2953532279 conn(0x7f3fbc0f1e40 msgr2=0x7f3fbc06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:23.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.286+0000 7f3fc414b700 1 -- 172.21.15.42:0/2953532279 shutdown_connections 2024-02-15T18:30:23.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.286+0000 7f3fc414b700 1 -- 172.21.15.42:0/2953532279 wait complete. 2024-02-15T18:30:23.290 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:30:23.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.391+0000 7f0fc72ba700 1 -- 172.21.15.42:0/3330633719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fc00f66e0 msgr2=0x7f0fc00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:23.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.391+0000 7f0fc72ba700 1 --2- 172.21.15.42:0/3330633719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fc00f66e0 0x7f0fc00f6b00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f0fbc004660 tx=0x7f0fbc009b30).stop 2024-02-15T18:30:23.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.391+0000 7f0fc72ba700 1 -- 172.21.15.42:0/3330633719 shutdown_connections 2024-02-15T18:30:23.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.391+0000 7f0fc72ba700 1 --2- 172.21.15.42:0/3330633719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fc00f7980 0x7f0fc00f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.391+0000 7f0fc72ba700 1 --2- 172.21.15.42:0/3330633719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fc00f66e0 0x7f0fc00f6b00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.391+0000 7f0fc72ba700 1 -- 172.21.15.42:0/3330633719 >> 172.21.15.42:0/3330633719 conn(0x7f0fc00f1e40 msgr2=0x7f0fc00f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:23.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.391+0000 7f0fc72ba700 1 -- 172.21.15.42:0/3330633719 shutdown_connections 2024-02-15T18:30:23.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.391+0000 7f0fc72ba700 1 -- 172.21.15.42:0/3330633719 wait complete. 2024-02-15T18:30:23.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.391+0000 7f0fc72ba700 1 Processor -- start 2024-02-15T18:30:23.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.391+0000 7f0fc72ba700 1 -- start start 2024-02-15T18:30:23.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.391+0000 7f0fc72ba700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fc00f66e0 0x7f0fc006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:23.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.391+0000 7f0fc72ba700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fc00f7980 0x7f0fc0069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:23.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.391+0000 7f0fc72ba700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc006a380 con 0x7f0fc00f66e0 2024-02-15T18:30:23.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.391+0000 7f0fc72ba700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc006a4c0 con 0x7f0fc00f7980 2024-02-15T18:30:23.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.392+0000 7f0fc5056700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fc00f66e0 0x7f0fc006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:23.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.392+0000 7f0fc5056700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fc00f66e0 0x7f0fc006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46142/0 (socket says 172.21.15.42:46142) 2024-02-15T18:30:23.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.392+0000 7f0fc5056700 1 -- 172.21.15.42:0/2248368609 learned_addr learned my addr 172.21.15.42:0/2248368609 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:30:23.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.392+0000 7f0fc4855700 1 --2- 172.21.15.42:0/2248368609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fc00f7980 0x7f0fc0069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:23.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.392+0000 7f0fc5056700 1 -- 172.21.15.42:0/2248368609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fc00f7980 msgr2=0x7f0fc0069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:23.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.392+0000 7f0fc5056700 1 --2- 172.21.15.42:0/2248368609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fc00f7980 0x7f0fc0069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.392+0000 7f0fc5056700 1 -- 172.21.15.42:0/2248368609 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc005040 con 0x7f0fc00f66e0 2024-02-15T18:30:23.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.392+0000 7f0fc5056700 1 --2- 172.21.15.42:0/2248368609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fc00f66e0 0x7f0fc006b9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f0fbc004e30 tx=0x7f0fbc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:23.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.392+0000 7f0fb5ffb700 1 -- 172.21.15.42:0/2248368609 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fbc018070 con 0x7f0fc00f66e0 2024-02-15T18:30:23.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.392+0000 7f0fb5ffb700 1 -- 172.21.15.42:0/2248368609 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f0fbc007ba0 con 0x7f0fc00f66e0 2024-02-15T18:30:23.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.392+0000 7f0fb5ffb700 1 -- 172.21.15.42:0/2248368609 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fbc01c3c0 con 0x7f0fc00f66e0 2024-02-15T18:30:23.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.392+0000 7f0fc72ba700 1 -- 172.21.15.42:0/2248368609 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc006a740 con 0x7f0fc00f66e0 2024-02-15T18:30:23.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.392+0000 7f0fc72ba700 1 -- 172.21.15.42:0/2248368609 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc006aca0 con 0x7f0fc00f66e0 2024-02-15T18:30:23.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.393+0000 7f0fc72ba700 1 -- 172.21.15.42:0/2248368609 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc0045ad0 con 0x7f0fc00f66e0 2024-02-15T18:30:23.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.394+0000 7f0fb5ffb700 1 -- 172.21.15.42:0/2248368609 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f0fbc01d420 con 0x7f0fc00f66e0 2024-02-15T18:30:23.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.394+0000 7f0fb5ffb700 1 --2- 172.21.15.42:0/2248368609 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0fac060f20 0x7f0fac0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:23.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.394+0000 7f0fb5ffb700 1 -- 172.21.15.42:0/2248368609 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f0fbc07dd10 con 0x7f0fc00f66e0 2024-02-15T18:30:23.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.394+0000 7f0fc4855700 1 --2- 172.21.15.42:0/2248368609 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0fac060f20 0x7f0fac0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:23.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.395+0000 7f0fc4855700 1 --2- 172.21.15.42:0/2248368609 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0fac060f20 0x7f0fac0633e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f0fb0004e20 tx=0x7f0fb000b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:23.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.398+0000 7f0fb5ffb700 1 -- 172.21.15.42:0/2248368609 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0fbc04e200 con 0x7f0fc00f66e0 2024-02-15T18:30:23.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.591+0000 7f0fc72ba700 1 -- 172.21.15.42:0/2248368609 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fc0057f10 con 0x7f0fac060f20 2024-02-15T18:30:23.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.592+0000 7f0fb5ffb700 1 -- 172.21.15.42:0/2248368609 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0fc0057f10 con 0x7f0fac060f20 2024-02-15T18:30:23.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.594+0000 7f0fc72ba700 1 -- 172.21.15.42:0/2248368609 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0fac060f20 msgr2=0x7f0fac0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:23.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.594+0000 7f0fc72ba700 1 --2- 172.21.15.42:0/2248368609 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0fac060f20 0x7f0fac0633e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f0fb0004e20 tx=0x7f0fb000b040).stop 2024-02-15T18:30:23.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.594+0000 7f0fc72ba700 1 -- 172.21.15.42:0/2248368609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fc00f66e0 msgr2=0x7f0fc006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:23.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.594+0000 7f0fc72ba700 1 --2- 172.21.15.42:0/2248368609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fc00f66e0 0x7f0fc006b9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f0fbc004e30 tx=0x7f0fbc009b30).stop 2024-02-15T18:30:23.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.594+0000 7f0fc72ba700 1 -- 172.21.15.42:0/2248368609 shutdown_connections 2024-02-15T18:30:23.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.594+0000 7f0fc72ba700 1 --2- 172.21.15.42:0/2248368609 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0fac060f20 0x7f0fac0633e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.594+0000 7f0fc72ba700 1 --2- 172.21.15.42:0/2248368609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0fc00f66e0 0x7f0fc006b9a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.594+0000 7f0fc72ba700 1 --2- 172.21.15.42:0/2248368609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0fc00f7980 0x7f0fc0069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.594+0000 7f0fc72ba700 1 -- 172.21.15.42:0/2248368609 >> 172.21.15.42:0/2248368609 conn(0x7f0fc00f1e40 msgr2=0x7f0fc00fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:23.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.594+0000 7f0fc72ba700 1 -- 172.21.15.42:0/2248368609 shutdown_connections 2024-02-15T18:30:23.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.594+0000 7f0fc72ba700 1 -- 172.21.15.42:0/2248368609 wait complete. 2024-02-15T18:30:23.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.695+0000 7f188c907700 1 -- 172.21.15.42:0/2424964455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18880f8540 msgr2=0x7f18880f46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:23.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.695+0000 7f188c907700 1 --2- 172.21.15.42:0/2424964455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18880f8540 0x7f18880f46c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f187c004660 tx=0x7f187c009b30).stop 2024-02-15T18:30:23.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.695+0000 7f188c907700 1 -- 172.21.15.42:0/2424964455 shutdown_connections 2024-02-15T18:30:23.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.695+0000 7f188c907700 1 --2- 172.21.15.42:0/2424964455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18880f8540 0x7f18880f46c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.695+0000 7f188c907700 1 --2- 172.21.15.42:0/2424964455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18880f7ba0 0x7f18880f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.695+0000 7f188c907700 1 -- 172.21.15.42:0/2424964455 >> 172.21.15.42:0/2424964455 conn(0x7f18880f0320 msgr2=0x7f18880f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:23.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.695+0000 7f188c907700 1 -- 172.21.15.42:0/2424964455 shutdown_connections 2024-02-15T18:30:23.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.695+0000 7f188c907700 1 -- 172.21.15.42:0/2424964455 wait complete. 2024-02-15T18:30:23.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.696+0000 7f188c907700 1 Processor -- start 2024-02-15T18:30:23.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.696+0000 7f188c907700 1 -- start start 2024-02-15T18:30:23.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.696+0000 7f188c907700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18880f7ba0 0x7f1888106e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:23.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.696+0000 7f188c907700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18880f8540 0x7f1888107340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:23.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.696+0000 7f188c907700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1888107940 con 0x7f18880f7ba0 2024-02-15T18:30:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.696+0000 7f188c907700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1888108150 con 0x7f18880f8540 2024-02-15T18:30:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.696+0000 7f1886d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18880f7ba0 0x7f1888106e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.696+0000 7f1886d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18880f7ba0 0x7f1888106e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46164/0 (socket says 172.21.15.42:46164) 2024-02-15T18:30:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.696+0000 7f1886d9d700 1 -- 172.21.15.42:0/1292258462 learned_addr learned my addr 172.21.15.42:0/1292258462 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:30:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.696+0000 7f188659c700 1 --2- 172.21.15.42:0/1292258462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18880f8540 0x7f1888107340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.697+0000 7f1886d9d700 1 -- 172.21.15.42:0/1292258462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18880f8540 msgr2=0x7f1888107340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.697+0000 7f1886d9d700 1 --2- 172.21.15.42:0/1292258462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18880f8540 0x7f1888107340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.697+0000 7f1886d9d700 1 -- 172.21.15.42:0/1292258462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c005040 con 0x7f18880f7ba0 2024-02-15T18:30:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.697+0000 7f1886d9d700 1 --2- 172.21.15.42:0/1292258462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18880f7ba0 0x7f1888106e50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f1878002700 tx=0x7f1878009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.697+0000 7f186f7fe700 1 -- 172.21.15.42:0/1292258462 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18780079c0 con 0x7f18880f7ba0 2024-02-15T18:30:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.697+0000 7f188c907700 1 -- 172.21.15.42:0/1292258462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18881083d0 con 0x7f18880f7ba0 2024-02-15T18:30:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.697+0000 7f186f7fe700 1 -- 172.21.15.42:0/1292258462 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f187800f040 con 0x7f18880f7ba0 2024-02-15T18:30:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.697+0000 7f186f7fe700 1 -- 172.21.15.42:0/1292258462 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1878014940 con 0x7f18880f7ba0 2024-02-15T18:30:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.697+0000 7f188c907700 1 -- 172.21.15.42:0/1292258462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1888108990 con 0x7f18880f7ba0 2024-02-15T18:30:23.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.698+0000 7f188c907700 1 -- 172.21.15.42:0/1292258462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1888059100 con 0x7f18880f7ba0 2024-02-15T18:30:23.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.698+0000 7f186f7fe700 1 -- 172.21.15.42:0/1292258462 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f187801b060 con 0x7f18880f7ba0 2024-02-15T18:30:23.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.699+0000 7f186f7fe700 1 --2- 172.21.15.42:0/1292258462 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f1870060f70 0x7f1870063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:23.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.699+0000 7f186f7fe700 1 -- 172.21.15.42:0/1292258462 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f187807bf60 con 0x7f18880f7ba0 2024-02-15T18:30:23.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.699+0000 7f188659c700 1 --2- 172.21.15.42:0/1292258462 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f1870060f70 0x7f1870063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:23.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.699+0000 7f188659c700 1 --2- 172.21.15.42:0/1292258462 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f1870060f70 0x7f1870063430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f187c004630 tx=0x7f187c009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:23.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.703+0000 7f186f7fe700 1 -- 172.21.15.42:0/1292258462 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f187804c3e0 con 0x7f18880f7ba0 2024-02-15T18:30:23.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.886+0000 7f188c907700 1 -- 172.21.15.42:0/1292258462 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f18880fef00 con 0x7f1870060f70 2024-02-15T18:30:23.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.895+0000 7f186f7fe700 1 -- 172.21.15.42:0/1292258462 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f18880fef00 con 0x7f1870060f70 2024-02-15T18:30:23.889 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:30:23.889 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (10m) 19s ago 11m 21.4M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:30:23.889 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (2m) 19s ago 11m 6643k - 16.2.14-521-g0e714d9a 1fbc92c859dc 2da9d875539a 2024-02-15T18:30:23.889 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (2m) 2m ago 10m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 5c46a68ff62b 2024-02-15T18:30:23.889 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (9m) 19s ago 11m 33.7M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:30:23.889 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (6m) 19s ago 6m 14.9M - 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:30:23.889 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (6m) 19s ago 6m 17.5M - 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:30:23.889 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (6m) 2m ago 6m 154M - 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:30:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (6m) 2m ago 6m 227M - 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:30:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (4m) 19s ago 13m 435M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:30:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (3m) 2m ago 10m 388M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:30:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (3m) 19s ago 13m 46.9M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:30:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (2m) 2m ago 10m 25.8M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:30:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (11m) 19s ago 11m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:30:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (10m) 2m ago 10m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:30:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (2m) 19s ago 10m 250M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc f226bea76cab 2024-02-15T18:30:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (71s) 19s ago 9m 264M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 5114ed026724 2024-02-15T18:30:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (25s) 19s ago 9m 11.6M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 2e26b1ec292a 2024-02-15T18:30:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (8m) 2m ago 8m 432M 4096M 16.2.4 8d91d370c2b8 e6fe8e77c6fe 2024-02-15T18:30:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (8m) 2m ago 8m 428M 4096M 16.2.4 8d91d370c2b8 24582012e234 2024-02-15T18:30:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (8m) 2m ago 8m 347M 4096M 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:30:23.890 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (4m) 19s ago 11m 61.0M - 2.18.1 de242295e225 63b6dfb825ed 2024-02-15T18:30:23.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.897+0000 7f188c907700 1 -- 172.21.15.42:0/1292258462 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f1870060f70 msgr2=0x7f1870063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:23.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.897+0000 7f188c907700 1 --2- 172.21.15.42:0/1292258462 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f1870060f70 0x7f1870063430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f187c004630 tx=0x7f187c009b30).stop 2024-02-15T18:30:23.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.897+0000 7f188c907700 1 -- 172.21.15.42:0/1292258462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18880f7ba0 msgr2=0x7f1888106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:23.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.897+0000 7f188c907700 1 --2- 172.21.15.42:0/1292258462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18880f7ba0 0x7f1888106e50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f1878002700 tx=0x7f1878009d80).stop 2024-02-15T18:30:23.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.897+0000 7f188c907700 1 -- 172.21.15.42:0/1292258462 shutdown_connections 2024-02-15T18:30:23.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.897+0000 7f188c907700 1 --2- 172.21.15.42:0/1292258462 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f1870060f70 0x7f1870063430 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.897+0000 7f188c907700 1 --2- 172.21.15.42:0/1292258462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18880f7ba0 0x7f1888106e50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.897+0000 7f188c907700 1 --2- 172.21.15.42:0/1292258462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18880f8540 0x7f1888107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.897+0000 7f188c907700 1 -- 172.21.15.42:0/1292258462 >> 172.21.15.42:0/1292258462 conn(0x7f18880f0320 msgr2=0x7f18880fb430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:23.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.897+0000 7f188c907700 1 -- 172.21.15.42:0/1292258462 shutdown_connections 2024-02-15T18:30:23.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.897+0000 7f188c907700 1 -- 172.21.15.42:0/1292258462 wait complete. 2024-02-15T18:30:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.998+0000 7fcd63823700 1 -- 172.21.15.42:0/1068776693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd5c0f6860 msgr2=0x7fcd5c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.998+0000 7fcd63823700 1 --2- 172.21.15.42:0/1068776693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd5c0f6860 0x7fcd5c0f6cc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fcd50004660 tx=0x7fcd50009b30).stop 2024-02-15T18:30:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.998+0000 7fcd63823700 1 -- 172.21.15.42:0/1068776693 shutdown_connections 2024-02-15T18:30:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.998+0000 7fcd63823700 1 --2- 172.21.15.42:0/1068776693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd5c0f6860 0x7fcd5c0f6cc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.998+0000 7fcd63823700 1 --2- 172.21.15.42:0/1068776693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd5c0f55c0 0x7fcd5c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.998+0000 7fcd63823700 1 -- 172.21.15.42:0/1068776693 >> 172.21.15.42:0/1068776693 conn(0x7fcd5c0f0d40 msgr2=0x7fcd5c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:23.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.998+0000 7fcd63823700 1 -- 172.21.15.42:0/1068776693 shutdown_connections 2024-02-15T18:30:23.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.998+0000 7fcd63823700 1 -- 172.21.15.42:0/1068776693 wait complete. 2024-02-15T18:30:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.999+0000 7fcd63823700 1 Processor -- start 2024-02-15T18:30:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.999+0000 7fcd63823700 1 -- start start 2024-02-15T18:30:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.999+0000 7fcd63823700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd5c0f55c0 0x7fcd5c106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.999+0000 7fcd63823700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd5c0f6860 0x7fcd5c1074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.999+0000 7fcd63823700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c107a60 con 0x7fcd5c0f55c0 2024-02-15T18:30:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.999+0000 7fcd63823700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5c107ba0 con 0x7fcd5c0f6860 2024-02-15T18:30:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.999+0000 7fcd615bf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd5c0f55c0 0x7fcd5c106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.999+0000 7fcd615bf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd5c0f55c0 0x7fcd5c106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46172/0 (socket says 172.21.15.42:46172) 2024-02-15T18:30:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:23.999+0000 7fcd615bf700 1 -- 172.21.15.42:0/433772688 learned_addr learned my addr 172.21.15.42:0/433772688 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:30:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.000+0000 7fcd60dbe700 1 --2- 172.21.15.42:0/433772688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd5c0f6860 0x7fcd5c1074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:23.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.000+0000 7fcd60dbe700 1 -- 172.21.15.42:0/433772688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd5c0f55c0 msgr2=0x7fcd5c106fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.000+0000 7fcd60dbe700 1 --2- 172.21.15.42:0/433772688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd5c0f55c0 0x7fcd5c106fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.000+0000 7fcd60dbe700 1 -- 172.21.15.42:0/433772688 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50005040 con 0x7fcd5c0f6860 2024-02-15T18:30:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.000+0000 7fcd60dbe700 1 --2- 172.21.15.42:0/433772688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd5c0f6860 0x7fcd5c1074b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fcd50009f50 tx=0x7fcd50009b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.000+0000 7fcd4dffb700 1 -- 172.21.15.42:0/433772688 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd50018070 con 0x7fcd5c0f6860 2024-02-15T18:30:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.000+0000 7fcd63823700 1 -- 172.21.15.42:0/433772688 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5c1084e0 con 0x7fcd5c0f6860 2024-02-15T18:30:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.001+0000 7fcd4dffb700 1 -- 172.21.15.42:0/433772688 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fcd50007ba0 con 0x7fcd5c0f6860 2024-02-15T18:30:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.001+0000 7fcd4dffb700 1 -- 172.21.15.42:0/433772688 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd50012400 con 0x7fcd5c0f6860 2024-02-15T18:30:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.001+0000 7fcd63823700 1 -- 172.21.15.42:0/433772688 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5c108a10 con 0x7fcd5c0f6860 2024-02-15T18:30:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.002+0000 7fcd63823700 1 -- 172.21.15.42:0/433772688 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5c0f49b0 con 0x7fcd5c0f6860 2024-02-15T18:30:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.002+0000 7fcd4dffb700 1 -- 172.21.15.42:0/433772688 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fcd5001d420 con 0x7fcd5c0f6860 2024-02-15T18:30:23.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.003+0000 7fcd4dffb700 1 --2- 172.21.15.42:0/433772688 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcd48060f20 0x7fcd480633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:23.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.003+0000 7fcd4dffb700 1 -- 172.21.15.42:0/433772688 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fcd5007dc60 con 0x7fcd5c0f6860 2024-02-15T18:30:23.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.003+0000 7fcd615bf700 1 --2- 172.21.15.42:0/433772688 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcd48060f20 0x7fcd480633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:23.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.003+0000 7fcd615bf700 1 --2- 172.21.15.42:0/433772688 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcd48060f20 0x7fcd480633e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fcd58004d00 tx=0x7fcd5800b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:24.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.007+0000 7fcd4dffb700 1 -- 172.21.15.42:0/433772688 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcd50051ab0 con 0x7fcd5c0f6860 2024-02-15T18:30:24.133 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:22.730791+0000 mgr.smithi042.kfoihb (mgr.24435) 205 : cluster [DBG] pgmap v163: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:30:24.133 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:23 smithi191 systemd[1]: Stopping Ceph osd.3 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:30:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.248+0000 7fcd63823700 1 -- 172.21.15.42:0/433772688 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcd5c045ad0 con 0x7fcd5c0f6860 2024-02-15T18:30:24.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.249+0000 7fcd4dffb700 1 -- 172.21.15.42:0/433772688 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+820 (secure 0 0 0) 0x7fcd500516a0 con 0x7fcd5c0f6860 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 3, 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 3 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:30:24.243 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:30:24.244 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 7, 2024-02-15T18:30:24.244 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 7 2024-02-15T18:30:24.244 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:30:24.244 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:30:24.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.250+0000 7fcd63823700 1 -- 172.21.15.42:0/433772688 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcd48060f20 msgr2=0x7fcd480633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:24.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.250+0000 7fcd63823700 1 --2- 172.21.15.42:0/433772688 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcd48060f20 0x7fcd480633e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fcd58004d00 tx=0x7fcd5800b040).stop 2024-02-15T18:30:24.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.250+0000 7fcd63823700 1 -- 172.21.15.42:0/433772688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd5c0f6860 msgr2=0x7fcd5c1074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:24.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.250+0000 7fcd63823700 1 --2- 172.21.15.42:0/433772688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd5c0f6860 0x7fcd5c1074b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fcd50009f50 tx=0x7fcd50009b30).stop 2024-02-15T18:30:24.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.250+0000 7fcd63823700 1 -- 172.21.15.42:0/433772688 shutdown_connections 2024-02-15T18:30:24.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.250+0000 7fcd63823700 1 --2- 172.21.15.42:0/433772688 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcd48060f20 0x7fcd480633e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.250+0000 7fcd63823700 1 --2- 172.21.15.42:0/433772688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcd5c0f55c0 0x7fcd5c106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.250+0000 7fcd63823700 1 --2- 172.21.15.42:0/433772688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd5c0f6860 0x7fcd5c1074b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.250+0000 7fcd63823700 1 -- 172.21.15.42:0/433772688 >> 172.21.15.42:0/433772688 conn(0x7fcd5c0f0d40 msgr2=0x7fcd5c05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:24.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.251+0000 7fcd63823700 1 -- 172.21.15.42:0/433772688 shutdown_connections 2024-02-15T18:30:24.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.251+0000 7fcd63823700 1 -- 172.21.15.42:0/433772688 wait complete. 2024-02-15T18:30:24.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:22.730791+0000 mgr.smithi042.kfoihb (mgr.24435) 205 : cluster [DBG] pgmap v163: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:30:24.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.352+0000 7fc40b991700 1 -- 172.21.15.42:0/362296540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4040f55c0 msgr2=0x7fc4040f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:24.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.352+0000 7fc40b991700 1 --2- 172.21.15.42:0/362296540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4040f55c0 0x7fc4040f59e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fc400004660 tx=0x7fc400009b30).stop 2024-02-15T18:30:24.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.352+0000 7fc40b991700 1 -- 172.21.15.42:0/362296540 shutdown_connections 2024-02-15T18:30:24.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.352+0000 7fc40b991700 1 --2- 172.21.15.42:0/362296540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4040f6860 0x7fc4040f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.352+0000 7fc40b991700 1 --2- 172.21.15.42:0/362296540 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4040f55c0 0x7fc4040f59e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.352+0000 7fc40b991700 1 -- 172.21.15.42:0/362296540 >> 172.21.15.42:0/362296540 conn(0x7fc4040f0d40 msgr2=0x7fc4040f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:24.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.352+0000 7fc40b991700 1 -- 172.21.15.42:0/362296540 shutdown_connections 2024-02-15T18:30:24.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.352+0000 7fc40b991700 1 -- 172.21.15.42:0/362296540 wait complete. 2024-02-15T18:30:24.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.353+0000 7fc40b991700 1 Processor -- start 2024-02-15T18:30:24.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.353+0000 7fc40b991700 1 -- start start 2024-02-15T18:30:24.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.353+0000 7fc40b991700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4040f55c0 0x7fc404105f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:24.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.353+0000 7fc40b991700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4040f6860 0x7fc4041063f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:24.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.353+0000 7fc40b991700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4041069a0 con 0x7fc4040f6860 2024-02-15T18:30:24.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.353+0000 7fc40b991700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc404106ae0 con 0x7fc4040f55c0 2024-02-15T18:30:24.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.353+0000 7fc408f2c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4040f6860 0x7fc4041063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:24.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.353+0000 7fc408f2c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4040f6860 0x7fc4041063f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46192/0 (socket says 172.21.15.42:46192) 2024-02-15T18:30:24.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.353+0000 7fc408f2c700 1 -- 172.21.15.42:0/2798630360 learned_addr learned my addr 172.21.15.42:0/2798630360 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:30:24.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.354+0000 7fc40972d700 1 --2- 172.21.15.42:0/2798630360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4040f55c0 0x7fc404105f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:24.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.354+0000 7fc408f2c700 1 -- 172.21.15.42:0/2798630360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4040f55c0 msgr2=0x7fc404105f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:24.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.354+0000 7fc408f2c700 1 --2- 172.21.15.42:0/2798630360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4040f55c0 0x7fc404105f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.354+0000 7fc408f2c700 1 -- 172.21.15.42:0/2798630360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc400005040 con 0x7fc4040f6860 2024-02-15T18:30:24.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.354+0000 7fc408f2c700 1 --2- 172.21.15.42:0/2798630360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4040f6860 0x7fc4041063f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fc3f800e7f0 tx=0x7fc3f800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:24.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.354+0000 7fc3f5ffb700 1 -- 172.21.15.42:0/2798630360 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3f80092e0 con 0x7fc4040f6860 2024-02-15T18:30:24.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.354+0000 7fc3f5ffb700 1 -- 172.21.15.42:0/2798630360 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fc3f800f040 con 0x7fc4040f6860 2024-02-15T18:30:24.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.354+0000 7fc40b991700 1 -- 172.21.15.42:0/2798630360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc404107480 con 0x7fc4040f6860 2024-02-15T18:30:24.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.354+0000 7fc3f5ffb700 1 -- 172.21.15.42:0/2798630360 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3f8014880 con 0x7fc4040f6860 2024-02-15T18:30:24.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.354+0000 7fc40b991700 1 -- 172.21.15.42:0/2798630360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc404107a40 con 0x7fc4040f6860 2024-02-15T18:30:24.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.355+0000 7fc40b991700 1 -- 172.21.15.42:0/2798630360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc404045ad0 con 0x7fc4040f6860 2024-02-15T18:30:24.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.356+0000 7fc3f5ffb700 1 -- 172.21.15.42:0/2798630360 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fc3f801b030 con 0x7fc4040f6860 2024-02-15T18:30:24.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.356+0000 7fc3f5ffb700 1 --2- 172.21.15.42:0/2798630360 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc3f0060f20 0x7fc3f00633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:24.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.356+0000 7fc3f5ffb700 1 -- 172.21.15.42:0/2798630360 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fc3f807bf40 con 0x7fc4040f6860 2024-02-15T18:30:24.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.356+0000 7fc40972d700 1 --2- 172.21.15.42:0/2798630360 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc3f0060f20 0x7fc3f00633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:24.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.357+0000 7fc40972d700 1 --2- 172.21.15.42:0/2798630360 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc3f0060f20 0x7fc3f00633e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fc400009720 tx=0x7fc40000b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:24.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.360+0000 7fc3f5ffb700 1 -- 172.21.15.42:0/2798630360 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc3f80aaf60 con 0x7fc4040f6860 2024-02-15T18:30:24.444 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.3[44935]: debug 2024-02-15T18:30:24.138+0000 7f2ac6979700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:30:24.444 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.3[44935]: debug 2024-02-15T18:30:24.138+0000 7f2ac6979700 -1 osd.3 81 *** Got signal Terminated *** 2024-02-15T18:30:24.444 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.3[44935]: debug 2024-02-15T18:30:24.138+0000 7f2ac6979700 -1 osd.3 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:30:24.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.579+0000 7fc40b991700 1 -- 172.21.15.42:0/2798630360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc4040fdf90 con 0x7fc4040f6860 2024-02-15T18:30:24.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.579+0000 7fc3f5ffb700 1 -- 172.21.15.42:0/2798630360 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1773 (secure 0 0 0) 0x7fc3f8019020 con 0x7fc4040f6860 2024-02-15T18:30:24.574 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:30:24.574 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:30:24.574 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:30:24.574 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:30:24.574 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:30:24.574 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:30:24.574 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:30:24.574 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:30:24.574 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:30:24.574 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:30:24.575 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:30:24.576 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:30:24.576 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:30:24.576 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:30:24.576 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:30:24.576 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:30:24.576 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:30:24.576 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:30:24.576 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:30:24.576 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:30:24.576 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:30:24.576 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:30:24.576 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:30:24.576 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:30:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.581+0000 7fc40b991700 1 -- 172.21.15.42:0/2798630360 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc3f0060f20 msgr2=0x7fc3f00633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.581+0000 7fc40b991700 1 --2- 172.21.15.42:0/2798630360 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc3f0060f20 0x7fc3f00633e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fc400009720 tx=0x7fc40000b040).stop 2024-02-15T18:30:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.581+0000 7fc40b991700 1 -- 172.21.15.42:0/2798630360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4040f6860 msgr2=0x7fc4041063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.581+0000 7fc40b991700 1 --2- 172.21.15.42:0/2798630360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4040f6860 0x7fc4041063f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fc3f800e7f0 tx=0x7fc3f800b040).stop 2024-02-15T18:30:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.581+0000 7fc40b991700 1 -- 172.21.15.42:0/2798630360 shutdown_connections 2024-02-15T18:30:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.581+0000 7fc40b991700 1 --2- 172.21.15.42:0/2798630360 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc3f0060f20 0x7fc3f00633e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.581+0000 7fc40b991700 1 --2- 172.21.15.42:0/2798630360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4040f55c0 0x7fc404105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.581+0000 7fc40b991700 1 --2- 172.21.15.42:0/2798630360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc4040f6860 0x7fc4041063f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.581+0000 7fc40b991700 1 -- 172.21.15.42:0/2798630360 >> 172.21.15.42:0/2798630360 conn(0x7fc4040f0d40 msgr2=0x7fc4040fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:24.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.582+0000 7fc40b991700 1 -- 172.21.15.42:0/2798630360 shutdown_connections 2024-02-15T18:30:24.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.582+0000 7fc40b991700 1 -- 172.21.15.42:0/2798630360 wait complete. 2024-02-15T18:30:24.578 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:30:24.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.683+0000 7f34af4e5700 1 -- 172.21.15.42:0/1011283881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a80f66e0 msgr2=0x7f34a80f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:24.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.683+0000 7f34af4e5700 1 --2- 172.21.15.42:0/1011283881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a80f66e0 0x7f34a80f6b40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f349c004660 tx=0x7f349c009b30).stop 2024-02-15T18:30:24.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.683+0000 7f34af4e5700 1 -- 172.21.15.42:0/1011283881 shutdown_connections 2024-02-15T18:30:24.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.683+0000 7f34af4e5700 1 --2- 172.21.15.42:0/1011283881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a80f66e0 0x7f34a80f6b40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.683+0000 7f34af4e5700 1 --2- 172.21.15.42:0/1011283881 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34a80f5440 0x7f34a80f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:30:24.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.683+0000 7f34af4e5700 1 -- 172.21.15.42:0/1011283881 >> 172.21.15.42:0/1011283881 conn(0x7f34a80f0ba0 msgr2=0x7f34a80f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:24.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.683+0000 7f34af4e5700 1 -- 172.21.15.42:0/1011283881 shutdown_connections 2024-02-15T18:30:24.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.683+0000 7f34af4e5700 1 -- 172.21.15.42:0/1011283881 wait complete. 2024-02-15T18:30:24.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.684+0000 7f34af4e5700 1 Processor -- start 2024-02-15T18:30:24.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.684+0000 7f34af4e5700 1 -- start start 2024-02-15T18:30:24.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.684+0000 7f34af4e5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a80f5440 0x7f34a8106e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:24.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.684+0000 7f34af4e5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34a80f66e0 0x7f34a8107350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:24.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.684+0000 7f34af4e5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a81079e0 con 0x7f34a80f5440 2024-02-15T18:30:24.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.684+0000 7f34af4e5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a8108160 con 0x7f34a80f66e0 2024-02-15T18:30:24.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.684+0000 7f34ae4e3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a80f5440 0x7f34a8106e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:24.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.684+0000 7f34ae4e3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a80f5440 0x7f34a8106e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46208/0 (socket says 172.21.15.42:46208) 2024-02-15T18:30:24.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.684+0000 7f34ae4e3700 1 -- 172.21.15.42:0/477681970 learned_addr learned my addr 172.21.15.42:0/477681970 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:30:24.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.684+0000 7f34adce2700 1 --2- 172.21.15.42:0/477681970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34a80f66e0 0x7f34a8107350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:24.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.685+0000 7f34ae4e3700 1 -- 172.21.15.42:0/477681970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34a80f66e0 msgr2=0x7f34a8107350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:24.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.685+0000 7f34ae4e3700 1 --2- 172.21.15.42:0/477681970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34a80f66e0 0x7f34a8107350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.685+0000 7f34ae4e3700 1 -- 172.21.15.42:0/477681970 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c005040 con 0x7f34a80f5440 2024-02-15T18:30:24.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.685+0000 7f34ae4e3700 1 --2- 172.21.15.42:0/477681970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a80f5440 0x7f34a8106e60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f34a40093f0 tx=0x7f34a4006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:24.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.685+0000 7f349affd700 1 -- 172.21.15.42:0/477681970 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34a4013640 con 0x7f34a80f5440 2024-02-15T18:30:24.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.685+0000 7f349affd700 1 -- 172.21.15.42:0/477681970 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f34a4013c90 con 0x7f34a80f5440 2024-02-15T18:30:24.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.685+0000 7f34af4e5700 1 -- 172.21.15.42:0/477681970 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a81083e0 con 0x7f34a80f5440 2024-02-15T18:30:24.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.685+0000 7f349affd700 1 -- 172.21.15.42:0/477681970 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34a4012de0 con 0x7f34a80f5440 2024-02-15T18:30:24.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.685+0000 7f34af4e5700 1 -- 172.21.15.42:0/477681970 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a8108920 con 0x7f34a80f5440 2024-02-15T18:30:24.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.686+0000 7f34af4e5700 1 -- 172.21.15.42:0/477681970 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a8045ad0 con 0x7f34a80f5440 2024-02-15T18:30:24.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.686+0000 7f349affd700 1 -- 172.21.15.42:0/477681970 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f34a401a060 con 0x7f34a80f5440 2024-02-15T18:30:24.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.687+0000 7f349affd700 1 --2- 172.21.15.42:0/477681970 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3494060f70 0x7f3494063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:24.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.687+0000 7f349affd700 1 -- 172.21.15.42:0/477681970 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f34a407c0a0 con 0x7f34a80f5440 2024-02-15T18:30:24.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.687+0000 7f34adce2700 1 --2- 172.21.15.42:0/477681970 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3494060f70 0x7f3494063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:24.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.687+0000 7f34adce2700 1 --2- 172.21.15.42:0/477681970 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3494060f70 0x7f3494063430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f349c004630 tx=0x7f349c009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:24.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.691+0000 7f349affd700 1 -- 172.21.15.42:0/477681970 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f34a404c4c0 con 0x7f34a80f5440 2024-02-15T18:30:24.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.879+0000 7f34af4e5700 1 -- 172.21.15.42:0/477681970 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f34a80fab20 con 0x7f3494060f70 2024-02-15T18:30:24.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.880+0000 7f349affd700 1 -- 172.21.15.42:0/477681970 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f34a80fab20 con 0x7f3494060f70 2024-02-15T18:30:24.874 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:30:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:30:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:30:24.874 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:30:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:30:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-02-15T18:30:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-02-15T18:30:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-02-15T18:30:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:30:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "9/21 daemons upgraded", 2024-02-15T18:30:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading osd daemons", 2024-02-15T18:30:24.875 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:30:24.875 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:30:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.882+0000 7f34af4e5700 1 -- 172.21.15.42:0/477681970 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3494060f70 msgr2=0x7f3494063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.882+0000 7f34af4e5700 1 --2- 172.21.15.42:0/477681970 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3494060f70 0x7f3494063430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f349c004630 tx=0x7f349c009b30).stop 2024-02-15T18:30:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.882+0000 7f34af4e5700 1 -- 172.21.15.42:0/477681970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a80f5440 msgr2=0x7f34a8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.882+0000 7f34af4e5700 1 --2- 172.21.15.42:0/477681970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a80f5440 0x7f34a8106e60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f34a40093f0 tx=0x7f34a4006ec0).stop 2024-02-15T18:30:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.882+0000 7f34af4e5700 1 -- 172.21.15.42:0/477681970 shutdown_connections 2024-02-15T18:30:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.882+0000 7f34af4e5700 1 --2- 172.21.15.42:0/477681970 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3494060f70 0x7f3494063430 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.882+0000 7f34af4e5700 1 --2- 172.21.15.42:0/477681970 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a80f5440 0x7f34a8106e60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.882+0000 7f34af4e5700 1 --2- 172.21.15.42:0/477681970 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34a80f66e0 0x7f34a8107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.882+0000 7f34af4e5700 1 -- 172.21.15.42:0/477681970 >> 172.21.15.42:0/477681970 conn(0x7f34a80f0ba0 msgr2=0x7f34a805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:24.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.882+0000 7f34af4e5700 1 -- 172.21.15.42:0/477681970 shutdown_connections 2024-02-15T18:30:24.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:24.882+0000 7f34af4e5700 1 -- 172.21.15.42:0/477681970 wait complete. 2024-02-15T18:30:24.920 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:24 smithi191 bash[61634]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.3 2024-02-15T18:30:25.187 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:25 smithi191 bash[61687]: Error: no container with name or ID "ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-15T18:30:25.187 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:23.285086+0000 mgr.smithi042.kfoihb (mgr.24435) 206 : audit [DBG] from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:25.187 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:23.593437+0000 mgr.smithi042.kfoihb (mgr.24435) 207 : audit [DBG] from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:25.187 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:23.887665+0000 mgr.smithi042.kfoihb (mgr.24435) 208 : audit [DBG] from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:25.187 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:24.249631+0000 mon.smithi191 (mon.1) 5 : audit [DBG] from='client.? 172.21.15.42:0/433772688' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:25.187 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.402413+0000 mon.smithi042 (mon.0) 286 : cluster [DBG] osd.3 reported immediately failed by osd.2 2024-02-15T18:30:25.187 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.402440+0000 mon.smithi042 (mon.0) 287 : cluster [INF] osd.3 failed (root=default,host=smithi191) (connection refused reported by osd.2) 2024-02-15T18:30:25.188 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.402566+0000 mon.smithi042 (mon.0) 288 : cluster [DBG] osd.3 reported immediately failed by osd.1 2024-02-15T18:30:25.188 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.402613+0000 mon.smithi042 (mon.0) 289 : cluster [DBG] osd.3 reported immediately failed by osd.1 2024-02-15T18:30:25.188 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.402746+0000 mon.smithi042 (mon.0) 290 : cluster [DBG] osd.3 reported immediately failed by osd.0 2024-02-15T18:30:25.188 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.402817+0000 mon.smithi042 (mon.0) 291 : cluster [DBG] osd.3 reported immediately failed by osd.0 2024-02-15T18:30:25.188 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.403213+0000 mon.smithi042 (mon.0) 292 : cluster [DBG] osd.3 reported immediately failed by osd.2 2024-02-15T18:30:25.188 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.403256+0000 mon.smithi042 (mon.0) 293 : cluster [DBG] osd.3 reported immediately failed by osd.4 2024-02-15T18:30:25.188 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.403354+0000 mon.smithi042 (mon.0) 294 : cluster [DBG] osd.3 reported immediately failed by osd.5 2024-02-15T18:30:25.188 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.403393+0000 mon.smithi042 (mon.0) 295 : cluster [DBG] osd.3 reported immediately failed by osd.4 2024-02-15T18:30:25.188 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.403485+0000 mon.smithi042 (mon.0) 296 : cluster [DBG] osd.3 reported immediately failed by osd.5 2024-02-15T18:30:25.188 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:24.580642+0000 mon.smithi042 (mon.0) 297 : audit [DBG] from='client.? 172.21.15.42:0/2798630360' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:30:25.188 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.803368+0000 mon.smithi042 (mon.0) 298 : cluster [DBG] osd.3 reported immediately failed by osd.2 2024-02-15T18:30:25.188 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.803421+0000 mon.smithi042 (mon.0) 299 : cluster [DBG] osd.3 reported immediately failed by osd.1 2024-02-15T18:30:25.188 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.803526+0000 mon.smithi042 (mon.0) 300 : cluster [DBG] osd.3 reported immediately failed by osd.1 2024-02-15T18:30:25.189 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.803780+0000 mon.smithi042 (mon.0) 301 : cluster [DBG] osd.3 reported immediately failed by osd.0 2024-02-15T18:30:25.189 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.803850+0000 mon.smithi042 (mon.0) 302 : cluster [DBG] osd.3 reported immediately failed by osd.0 2024-02-15T18:30:25.189 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.804213+0000 mon.smithi042 (mon.0) 303 : cluster [DBG] osd.3 reported immediately failed by osd.2 2024-02-15T18:30:25.189 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.804255+0000 mon.smithi042 (mon.0) 304 : cluster [DBG] osd.3 reported immediately failed by osd.5 2024-02-15T18:30:25.189 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.804355+0000 mon.smithi042 (mon.0) 305 : cluster [DBG] osd.3 reported immediately failed by osd.4 2024-02-15T18:30:25.189 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.804424+0000 mon.smithi042 (mon.0) 306 : cluster [DBG] osd.3 reported immediately failed by osd.5 2024-02-15T18:30:25.189 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.804504+0000 mon.smithi042 (mon.0) 307 : cluster [DBG] osd.3 reported immediately failed by osd.4 2024-02-15T18:30:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:23.285086+0000 mgr.smithi042.kfoihb (mgr.24435) 206 : audit [DBG] from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:23.593437+0000 mgr.smithi042.kfoihb (mgr.24435) 207 : audit [DBG] from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:23.887665+0000 mgr.smithi042.kfoihb (mgr.24435) 208 : audit [DBG] from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:24.249631+0000 mon.smithi191 (mon.1) 5 : audit [DBG] from='client.? 172.21.15.42:0/433772688' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.402413+0000 mon.smithi042 (mon.0) 286 : cluster [DBG] osd.3 reported immediately failed by osd.2 2024-02-15T18:30:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.402440+0000 mon.smithi042 (mon.0) 287 : cluster [INF] osd.3 failed (root=default,host=smithi191) (connection refused reported by osd.2) 2024-02-15T18:30:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.402566+0000 mon.smithi042 (mon.0) 288 : cluster [DBG] osd.3 reported immediately failed by osd.1 2024-02-15T18:30:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.402613+0000 mon.smithi042 (mon.0) 289 : cluster [DBG] osd.3 reported immediately failed by osd.1 2024-02-15T18:30:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.402746+0000 mon.smithi042 (mon.0) 290 : cluster [DBG] osd.3 reported immediately failed by osd.0 2024-02-15T18:30:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.402817+0000 mon.smithi042 (mon.0) 291 : cluster [DBG] osd.3 reported immediately failed by osd.0 2024-02-15T18:30:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.403213+0000 mon.smithi042 (mon.0) 292 : cluster [DBG] osd.3 reported immediately failed by osd.2 2024-02-15T18:30:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.403256+0000 mon.smithi042 (mon.0) 293 : cluster [DBG] osd.3 reported immediately failed by osd.4 2024-02-15T18:30:25.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.403354+0000 mon.smithi042 (mon.0) 294 : cluster [DBG] osd.3 reported immediately failed by osd.5 2024-02-15T18:30:25.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.403393+0000 mon.smithi042 (mon.0) 295 : cluster [DBG] osd.3 reported immediately failed by osd.4 2024-02-15T18:30:25.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.403485+0000 mon.smithi042 (mon.0) 296 : cluster [DBG] osd.3 reported immediately failed by osd.5 2024-02-15T18:30:25.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:24.580642+0000 mon.smithi042 (mon.0) 297 : audit [DBG] from='client.? 172.21.15.42:0/2798630360' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:30:25.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.803368+0000 mon.smithi042 (mon.0) 298 : cluster [DBG] osd.3 reported immediately failed by osd.2 2024-02-15T18:30:25.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.803421+0000 mon.smithi042 (mon.0) 299 : cluster [DBG] osd.3 reported immediately failed by osd.1 2024-02-15T18:30:25.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.803526+0000 mon.smithi042 (mon.0) 300 : cluster [DBG] osd.3 reported immediately failed by osd.1 2024-02-15T18:30:25.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.803780+0000 mon.smithi042 (mon.0) 301 : cluster [DBG] osd.3 reported immediately failed by osd.0 2024-02-15T18:30:25.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.803850+0000 mon.smithi042 (mon.0) 302 : cluster [DBG] osd.3 reported immediately failed by osd.0 2024-02-15T18:30:25.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.804213+0000 mon.smithi042 (mon.0) 303 : cluster [DBG] osd.3 reported immediately failed by osd.2 2024-02-15T18:30:25.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.804255+0000 mon.smithi042 (mon.0) 304 : cluster [DBG] osd.3 reported immediately failed by osd.5 2024-02-15T18:30:25.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.804355+0000 mon.smithi042 (mon.0) 305 : cluster [DBG] osd.3 reported immediately failed by osd.4 2024-02-15T18:30:25.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.804424+0000 mon.smithi042 (mon.0) 306 : cluster [DBG] osd.3 reported immediately failed by osd.5 2024-02-15T18:30:25.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.804504+0000 mon.smithi042 (mon.0) 307 : cluster [DBG] osd.3 reported immediately failed by osd.4 2024-02-15T18:30:26.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.731167+0000 mgr.smithi042.kfoihb (mgr.24435) 209 : cluster [DBG] pgmap v164: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:30:26.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:24.881166+0000 mgr.smithi042.kfoihb (mgr.24435) 210 : audit [DBG] from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:26.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.920013+0000 mon.smithi042 (mon.0) 308 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:30:26.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:24.927400+0000 mon.smithi042 (mon.0) 309 : cluster [DBG] osdmap e82: 6 total, 5 up, 6 in 2024-02-15T18:30:26.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:25.836265+0000 mon.smithi042 (mon.0) 310 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:26.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.731167+0000 mgr.smithi042.kfoihb (mgr.24435) 209 : cluster [DBG] pgmap v164: 65 pgs: 65 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:30:26.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:24.881166+0000 mgr.smithi042.kfoihb (mgr.24435) 210 : audit [DBG] from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:26.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.920013+0000 mon.smithi042 (mon.0) 308 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:30:26.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:24.927400+0000 mon.smithi042 (mon.0) 309 : cluster [DBG] osdmap e82: 6 total, 5 up, 6 in 2024-02-15T18:30:26.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:25.836265+0000 mon.smithi042 (mon.0) 310 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:27.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:25.927592+0000 mon.smithi042 (mon.0) 311 : cluster [DBG] osdmap e83: 6 total, 5 up, 6 in 2024-02-15T18:30:27.194 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:26 smithi191 systemd[1]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.3.service: Succeeded. 2024-02-15T18:30:27.194 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:26 smithi191 systemd[1]: Stopped Ceph osd.3 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:30:27.194 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:27 smithi191 systemd[1]: Starting Ceph osd.3 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:30:27.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:25.927592+0000 mon.smithi042 (mon.0) 311 : cluster [DBG] osdmap e83: 6 total, 5 up, 6 in 2024-02-15T18:30:28.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:26.731707+0000 mgr.smithi042.kfoihb (mgr.24435) 211 : cluster [DBG] pgmap v167: 65 pgs: 4 active+undersized+degraded, 11 peering, 5 stale+active+clean, 5 active+undersized, 40 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 14/210 objects degraded (6.667%) 2024-02-15T18:30:28.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:26.969252+0000 mon.smithi042 (mon.0) 312 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 4 pgs peering (PG_AVAILABILITY) 2024-02-15T18:30:28.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:26.969277+0000 mon.smithi042 (mon.0) 313 : cluster [WRN] Health check failed: Degraded data redundancy: 14/210 objects degraded (6.667%), 4 pgs degraded (PG_DEGRADED) 2024-02-15T18:30:28.195 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:28 smithi191 bash[61945]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-02-15T18:30:28.195 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:28 smithi191 bash[61945]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-15T18:30:28.195 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:28 smithi191 bash[61945]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-3 --no-mon-config 2024-02-15T18:30:28.195 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:28 smithi191 bash[61945]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-15T18:30:28.195 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:28 smithi191 bash[61945]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-3/block 2024-02-15T18:30:28.195 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:28 smithi191 bash[61945]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-15T18:30:28.195 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:28 smithi191 bash[61945]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-15T18:30:28.195 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:28 smithi191 bash[61945]: --> ceph-volume lvm activate successful for osd ID: 3 2024-02-15T18:30:28.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:26.731707+0000 mgr.smithi042.kfoihb (mgr.24435) 211 : cluster [DBG] pgmap v167: 65 pgs: 4 active+undersized+degraded, 11 peering, 5 stale+active+clean, 5 active+undersized, 40 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 14/210 objects degraded (6.667%) 2024-02-15T18:30:28.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:26.969252+0000 mon.smithi042 (mon.0) 312 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 4 pgs peering (PG_AVAILABILITY) 2024-02-15T18:30:28.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:26.969277+0000 mon.smithi042 (mon.0) 313 : cluster [WRN] Health check failed: Degraded data redundancy: 14/210 objects degraded (6.667%), 4 pgs degraded (PG_DEGRADED) 2024-02-15T18:30:29.298 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:28.732471+0000 mgr.smithi042.kfoihb (mgr.24435) 212 : cluster [DBG] pgmap v168: 65 pgs: 12 active+undersized+degraded, 11 peering, 15 active+undersized, 27 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 36/210 objects degraded (17.143%) 2024-02-15T18:30:29.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:28.732471+0000 mgr.smithi042.kfoihb (mgr.24435) 212 : cluster [DBG] pgmap v168: 65 pgs: 12 active+undersized+degraded, 11 peering, 15 active+undersized, 27 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 36/210 objects degraded (17.143%) 2024-02-15T18:30:29.694 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:30:29.304+0000 7f492f670700 1 -- 172.21.15.191:0/3310016199 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fa13230680 con 0x55fa131f7400 2024-02-15T18:30:29.694 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:29 smithi191 bash[62218]: c71b210091cb21e6c4f6fc53af5f3baff523983410f8c01ab4d7bd1c1aae5d87 2024-02-15T18:30:29.694 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:29 smithi191 systemd[1]: Started Ceph osd.3 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:30:30.168 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:30:29.824+0000 7f49370fe380 -1 Falling back to public interface 2024-02-15T18:30:30.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:29.474738+0000 mon.smithi042 (mon.0) 314 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:30.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:29.475635+0000 mon.smithi042 (mon.0) 315 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:30.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:29.474738+0000 mon.smithi042 (mon.0) 314 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:30.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:29.475635+0000 mon.smithi042 (mon.0) 315 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:31.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:30.733091+0000 mgr.smithi042.kfoihb (mgr.24435) 213 : cluster [DBG] pgmap v169: 65 pgs: 14 active+undersized+degraded, 7 peering, 17 active+undersized, 27 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 39/210 objects degraded (18.571%) 2024-02-15T18:30:31.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:30.733091+0000 mgr.smithi042.kfoihb (mgr.24435) 213 : cluster [DBG] pgmap v169: 65 pgs: 14 active+undersized+degraded, 7 peering, 17 active+undersized, 27 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 39/210 objects degraded (18.571%) 2024-02-15T18:30:33.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:32.784345+0000 mon.smithi042 (mon.0) 316 : cluster [WRN] Health check update: Degraded data redundancy: 46/210 objects degraded (21.905%), 16 pgs degraded (PG_DEGRADED) 2024-02-15T18:30:33.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:32.784376+0000 mon.smithi042 (mon.0) 317 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-02-15T18:30:33.233 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:32.784345+0000 mon.smithi042 (mon.0) 316 : cluster [WRN] Health check update: Degraded data redundancy: 46/210 objects degraded (21.905%), 16 pgs degraded (PG_DEGRADED) 2024-02-15T18:30:33.233 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:32.784376+0000 mon.smithi042 (mon.0) 317 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-02-15T18:30:34.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:32.733670+0000 mgr.smithi042.kfoihb (mgr.24435) 214 : cluster [DBG] pgmap v170: 65 pgs: 16 active+undersized+degraded, 22 active+undersized, 27 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 46/210 objects degraded (21.905%) 2024-02-15T18:30:34.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:32.920667+0000 mon.smithi042 (mon.0) 318 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:34.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:33.221578+0000 mon.smithi042 (mon.0) 319 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:34.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:33.222522+0000 mon.smithi042 (mon.0) 320 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:30:34.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:33.227587+0000 mon.smithi042 (mon.0) 321 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:34.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:33.233679+0000 mon.smithi042 (mon.0) 322 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:34.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:33.236345+0000 mon.smithi042 (mon.0) 323 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:34.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:33.237794+0000 mon.smithi042 (mon.0) 324 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:34.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:33.239154+0000 mon.smithi042 (mon.0) 325 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:34.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:33.240586+0000 mon.smithi042 (mon.0) 326 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:30:34.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:32.733670+0000 mgr.smithi042.kfoihb (mgr.24435) 214 : cluster [DBG] pgmap v170: 65 pgs: 16 active+undersized+degraded, 22 active+undersized, 27 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 46/210 objects degraded (21.905%) 2024-02-15T18:30:34.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:32.920667+0000 mon.smithi042 (mon.0) 318 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:34.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:33.221578+0000 mon.smithi042 (mon.0) 319 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:30:34.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:33.222522+0000 mon.smithi042 (mon.0) 320 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:30:34.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:33.227587+0000 mon.smithi042 (mon.0) 321 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:34.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:33.233679+0000 mon.smithi042 (mon.0) 322 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:30:34.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:33.236345+0000 mon.smithi042 (mon.0) 323 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:34.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:33.237794+0000 mon.smithi042 (mon.0) 324 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:34.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:33.239154+0000 mon.smithi042 (mon.0) 325 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:34.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:33.240586+0000 mon.smithi042 (mon.0) 326 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:30:35.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:30:33.223276+0000 mgr.smithi042.kfoihb (mgr.24435) 215 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:30:35.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:33.240765+0000 mgr.smithi042.kfoihb (mgr.24435) 216 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:30:35.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:30:33.241642+0000 mgr.smithi042.kfoihb (mgr.24435) 217 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-02-15T18:30:35.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:30:33.223276+0000 mgr.smithi042.kfoihb (mgr.24435) 215 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:30:35.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:33.240765+0000 mgr.smithi042.kfoihb (mgr.24435) 216 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:30:35.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:30:33.241642+0000 mgr.smithi042.kfoihb (mgr.24435) 217 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-02-15T18:30:36.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:34.734175+0000 mgr.smithi042.kfoihb (mgr.24435) 218 : cluster [DBG] pgmap v171: 65 pgs: 16 active+undersized+degraded, 22 active+undersized, 27 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 46/210 objects degraded (21.905%) 2024-02-15T18:30:36.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:34.734175+0000 mgr.smithi042.kfoihb (mgr.24435) 218 : cluster [DBG] pgmap v171: 65 pgs: 16 active+undersized+degraded, 22 active+undersized, 27 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 46/210 objects degraded (21.905%) 2024-02-15T18:30:37.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:36.734715+0000 mgr.smithi042.kfoihb (mgr.24435) 219 : cluster [DBG] pgmap v172: 65 pgs: 16 active+undersized+degraded, 22 active+undersized, 27 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 46/210 objects degraded (21.905%) 2024-02-15T18:30:37.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:36.734715+0000 mgr.smithi042.kfoihb (mgr.24435) 219 : cluster [DBG] pgmap v172: 65 pgs: 16 active+undersized+degraded, 22 active+undersized, 27 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 46/210 objects degraded (21.905%) 2024-02-15T18:30:39.693 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:30:39.356+0000 7f49370fe380 -1 osd.3 81 log_to_monitors {default=true} 2024-02-15T18:30:40.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:38.735365+0000 mgr.smithi042.kfoihb (mgr.24435) 220 : cluster [DBG] pgmap v173: 65 pgs: 16 active+undersized+degraded, 22 active+undersized, 27 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 46/210 objects degraded (21.905%) 2024-02-15T18:30:40.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:39.360799+0000 mon.smithi191 (mon.1) 6 : audit [INF] from='osd.3 [v2:172.21.15.191:6800/4294301607,v1:172.21.15.191:6801/4294301607]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:30:40.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:39.361405+0000 mon.smithi042 (mon.0) 327 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:30:40.193 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:30:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:30:39.814+0000 7f4929de8700 -1 osd.3 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-15T18:30:40.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:38.735365+0000 mgr.smithi042.kfoihb (mgr.24435) 220 : cluster [DBG] pgmap v173: 65 pgs: 16 active+undersized+degraded, 22 active+undersized, 27 active+clean; 193 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 46/210 objects degraded (21.905%) 2024-02-15T18:30:40.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:39.360799+0000 mon.smithi191 (mon.1) 6 : audit [INF] from='osd.3 [v2:172.21.15.191:6800/4294301607,v1:172.21.15.191:6801/4294301607]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:30:40.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:39.361405+0000 mon.smithi042 (mon.0) 327 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-15T18:30:41.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:39.792058+0000 mon.smithi042 (mon.0) 328 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-15T18:30:41.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:39.792122+0000 mon.smithi042 (mon.0) 329 : cluster [DBG] osdmap e84: 6 total, 5 up, 6 in 2024-02-15T18:30:41.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:39.800840+0000 mon.smithi191 (mon.1) 7 : audit [INF] from='osd.3 [v2:172.21.15.191:6800/4294301607,v1:172.21.15.191:6801/4294301607]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:30:41.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:39.804400+0000 mon.smithi042 (mon.0) 330 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:30:41.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:39.792058+0000 mon.smithi042 (mon.0) 328 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-15T18:30:41.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:39.792122+0000 mon.smithi042 (mon.0) 329 : cluster [DBG] osdmap e84: 6 total, 5 up, 6 in 2024-02-15T18:30:41.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:39.800840+0000 mon.smithi191 (mon.1) 7 : audit [INF] from='osd.3 [v2:172.21.15.191:6800/4294301607,v1:172.21.15.191:6801/4294301607]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:30:41.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:39.804400+0000 mon.smithi042 (mon.0) 330 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:30:42.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:40.736033+0000 mgr.smithi042.kfoihb (mgr.24435) 221 : cluster [DBG] pgmap v175: 65 pgs: 16 active+undersized+degraded, 22 active+undersized, 27 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 46/210 objects degraded (21.905%) 2024-02-15T18:30:42.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:40.801763+0000 mon.smithi042 (mon.0) 331 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:30:42.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:40.806580+0000 mon.smithi042 (mon.0) 332 : cluster [INF] osd.3 [v2:172.21.15.191:6800/4294301607,v1:172.21.15.191:6801/4294301607] boot 2024-02-15T18:30:42.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:40.806688+0000 mon.smithi042 (mon.0) 333 : cluster [DBG] osdmap e85: 6 total, 6 up, 6 in 2024-02-15T18:30:42.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:40.809146+0000 mon.smithi042 (mon.0) 334 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:30:42.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:40.836293+0000 mon.smithi042 (mon.0) 335 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:42.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:41.807647+0000 mon.smithi042 (mon.0) 336 : cluster [DBG] osdmap e86: 6 total, 6 up, 6 in 2024-02-15T18:30:42.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:40.736033+0000 mgr.smithi042.kfoihb (mgr.24435) 221 : cluster [DBG] pgmap v175: 65 pgs: 16 active+undersized+degraded, 22 active+undersized, 27 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 46/210 objects degraded (21.905%) 2024-02-15T18:30:42.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:40.801763+0000 mon.smithi042 (mon.0) 331 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:30:42.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:40.806580+0000 mon.smithi042 (mon.0) 332 : cluster [INF] osd.3 [v2:172.21.15.191:6800/4294301607,v1:172.21.15.191:6801/4294301607] boot 2024-02-15T18:30:42.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:40.806688+0000 mon.smithi042 (mon.0) 333 : cluster [DBG] osdmap e85: 6 total, 6 up, 6 in 2024-02-15T18:30:42.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:40.809146+0000 mon.smithi042 (mon.0) 334 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-15T18:30:42.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:40.836293+0000 mon.smithi042 (mon.0) 335 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:42.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:41.807647+0000 mon.smithi042 (mon.0) 336 : cluster [DBG] osdmap e86: 6 total, 6 up, 6 in 2024-02-15T18:30:43.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:42 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:42.820168+0000 mon.smithi042 (mon.0) 337 : cluster [WRN] Health check update: Degraded data redundancy: 24/210 objects degraded (11.429%), 9 pgs degraded (PG_DEGRADED) 2024-02-15T18:30:43.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:42 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:42.820168+0000 mon.smithi042 (mon.0) 337 : cluster [WRN] Health check update: Degraded data redundancy: 24/210 objects degraded (11.429%), 9 pgs degraded (PG_DEGRADED) 2024-02-15T18:30:44.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:42.736521+0000 mgr.smithi042.kfoihb (mgr.24435) 222 : cluster [DBG] pgmap v178: 65 pgs: 1 active, 1 activating, 12 peering, 9 active+undersized+degraded, 10 active+undersized, 32 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 24/210 objects degraded (11.429%) 2024-02-15T18:30:44.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:42.736521+0000 mgr.smithi042.kfoihb (mgr.24435) 222 : cluster [DBG] pgmap v178: 65 pgs: 1 active, 1 activating, 12 peering, 9 active+undersized+degraded, 10 active+undersized, 32 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 24/210 objects degraded (11.429%) 2024-02-15T18:30:46.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:44.736954+0000 mgr.smithi042.kfoihb (mgr.24435) 223 : cluster [DBG] pgmap v179: 65 pgs: 1 active, 1 activating, 12 peering, 9 active+undersized+degraded, 10 active+undersized, 32 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 24/210 objects degraded (11.429%) 2024-02-15T18:30:46.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:44.736954+0000 mgr.smithi042.kfoihb (mgr.24435) 223 : cluster [DBG] pgmap v179: 65 pgs: 1 active, 1 activating, 12 peering, 9 active+undersized+degraded, 10 active+undersized, 32 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 24/210 objects degraded (11.429%) 2024-02-15T18:30:47.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:46.737546+0000 mgr.smithi042.kfoihb (mgr.24435) 224 : cluster [DBG] pgmap v180: 65 pgs: 1 active, 1 activating, 6 peering, 57 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 884 B/s rd, 2 op/s 2024-02-15T18:30:47.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:46 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:46.737546+0000 mgr.smithi042.kfoihb (mgr.24435) 224 : cluster [DBG] pgmap v180: 65 pgs: 1 active, 1 activating, 6 peering, 57 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 884 B/s rd, 2 op/s 2024-02-15T18:30:48.250 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:46.973539+0000 mon.smithi042 (mon.0) 338 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/210 objects degraded (11.429%), 9 pgs degraded) 2024-02-15T18:30:48.250 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:46.973597+0000 mon.smithi042 (mon.0) 339 : cluster [INF] Cluster is now healthy 2024-02-15T18:30:48.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:46.973539+0000 mon.smithi042 (mon.0) 338 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/210 objects degraded (11.429%), 9 pgs degraded) 2024-02-15T18:30:48.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:46.973597+0000 mon.smithi042 (mon.0) 339 : cluster [INF] Cluster is now healthy 2024-02-15T18:30:49.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:48.257298+0000 mon.smithi042 (mon.0) 340 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:30:49.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:48.257530+0000 mgr.smithi042.kfoihb (mgr.24435) 225 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:30:49.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:30:48.258535+0000 mgr.smithi042.kfoihb (mgr.24435) 226 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-02-15T18:30:49.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:48.738166+0000 mgr.smithi042.kfoihb (mgr.24435) 227 : cluster [DBG] pgmap v181: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-02-15T18:30:49.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:48.257298+0000 mon.smithi042 (mon.0) 340 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:30:49.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:48.257530+0000 mgr.smithi042.kfoihb (mgr.24435) 225 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:30:49.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:30:48.258535+0000 mgr.smithi042.kfoihb (mgr.24435) 226 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-02-15T18:30:49.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:48.738166+0000 mgr.smithi042.kfoihb (mgr.24435) 227 : cluster [DBG] pgmap v181: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-02-15T18:30:52.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:50.738643+0000 mgr.smithi042.kfoihb (mgr.24435) 228 : cluster [DBG] pgmap v182: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-02-15T18:30:52.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:50.738643+0000 mgr.smithi042.kfoihb (mgr.24435) 228 : cluster [DBG] pgmap v182: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-02-15T18:30:54.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:52.739184+0000 mgr.smithi042.kfoihb (mgr.24435) 229 : cluster [DBG] pgmap v183: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-15T18:30:54.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:52.739184+0000 mgr.smithi042.kfoihb (mgr.24435) 229 : cluster [DBG] pgmap v183: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-15T18:30:54.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.000+0000 7f54c97d8700 1 -- 172.21.15.42:0/1714309945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c40f6530 msgr2=0x7f54c40f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:54.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.000+0000 7f54c97d8700 1 --2- 172.21.15.42:0/1714309945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c40f6530 0x7f54c40f6950 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f54b4004660 tx=0x7f54b4009b30).stop 2024-02-15T18:30:54.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.000+0000 7f54c97d8700 1 -- 172.21.15.42:0/1714309945 shutdown_connections 2024-02-15T18:30:54.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.000+0000 7f54c97d8700 1 --2- 172.21.15.42:0/1714309945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54c40f77d0 0x7f54c40f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:54.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.000+0000 7f54c97d8700 1 --2- 172.21.15.42:0/1714309945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c40f6530 0x7f54c40f6950 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:54.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.000+0000 7f54c97d8700 1 -- 172.21.15.42:0/1714309945 >> 172.21.15.42:0/1714309945 conn(0x7f54c40f1c90 msgr2=0x7f54c40f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:54.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.000+0000 7f54c97d8700 1 -- 172.21.15.42:0/1714309945 shutdown_connections 2024-02-15T18:30:54.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.000+0000 7f54c97d8700 1 -- 172.21.15.42:0/1714309945 wait complete. 2024-02-15T18:30:54.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.001+0000 7f54c97d8700 1 Processor -- start 2024-02-15T18:30:54.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.001+0000 7f54c97d8700 1 -- start start 2024-02-15T18:30:54.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.001+0000 7f54c97d8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54c40f6530 0x7f54c4106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:54.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.001+0000 7f54c97d8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c40f77d0 0x7f54c41073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:54.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.001+0000 7f54c97d8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c4107960 con 0x7f54c40f77d0 2024-02-15T18:30:54.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.001+0000 7f54c97d8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c4108160 con 0x7f54c40f6530 2024-02-15T18:30:54.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.001+0000 7f54c37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c40f77d0 0x7f54c41073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:54.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.001+0000 7f54c37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c40f77d0 0x7f54c41073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58624/0 (socket says 172.21.15.42:58624) 2024-02-15T18:30:54.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.001+0000 7f54c37fe700 1 -- 172.21.15.42:0/2195192470 learned_addr learned my addr 172.21.15.42:0/2195192470 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:30:54.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.001+0000 7f54c3fff700 1 --2- 172.21.15.42:0/2195192470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54c40f6530 0x7f54c4106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:54.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.002+0000 7f54c37fe700 1 -- 172.21.15.42:0/2195192470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54c40f6530 msgr2=0x7f54c4106ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:54.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.002+0000 7f54c37fe700 1 --2- 172.21.15.42:0/2195192470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54c40f6530 0x7f54c4106ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:54.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.002+0000 7f54c37fe700 1 -- 172.21.15.42:0/2195192470 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b4005040 con 0x7f54c40f77d0 2024-02-15T18:30:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.002+0000 7f54c37fe700 1 --2- 172.21.15.42:0/2195192470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c40f77d0 0x7f54c41073b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f54b800c790 tx=0x7f54b8007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.002+0000 7f54c0ff9700 1 -- 172.21.15.42:0/2195192470 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54b8002590 con 0x7f54c40f77d0 2024-02-15T18:30:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.002+0000 7f54c0ff9700 1 -- 172.21.15.42:0/2195192470 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f54b8002be0 con 0x7f54c40f77d0 2024-02-15T18:30:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.002+0000 7f54c97d8700 1 -- 172.21.15.42:0/2195192470 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c4108440 con 0x7f54c40f77d0 2024-02-15T18:30:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.002+0000 7f54c0ff9700 1 -- 172.21.15.42:0/2195192470 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54b80123b0 con 0x7f54c40f77d0 2024-02-15T18:30:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.002+0000 7f54c97d8700 1 -- 172.21.15.42:0/2195192470 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c4108a00 con 0x7f54c40f77d0 2024-02-15T18:30:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.003+0000 7f54c97d8700 1 -- 172.21.15.42:0/2195192470 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c4059100 con 0x7f54c40f77d0 2024-02-15T18:30:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.004+0000 7f54c0ff9700 1 -- 172.21.15.42:0/2195192470 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f54b801a060 con 0x7f54c40f77d0 2024-02-15T18:30:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.004+0000 7f54c0ff9700 1 --2- 172.21.15.42:0/2195192470 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f54ac060f70 0x7f54ac063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.004+0000 7f54c0ff9700 1 -- 172.21.15.42:0/2195192470 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f54b807c380 con 0x7f54c40f77d0 2024-02-15T18:30:54.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.004+0000 7f54c3fff700 1 --2- 172.21.15.42:0/2195192470 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f54ac060f70 0x7f54ac063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:54.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.005+0000 7f54c3fff700 1 --2- 172.21.15.42:0/2195192470 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f54ac060f70 0x7f54ac063430 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f54b4009f50 tx=0x7f54b4009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:55.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.009+0000 7f54c0ff9700 1 -- 172.21.15.42:0/2195192470 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f54b80501d0 con 0x7f54c40f77d0 2024-02-15T18:30:55.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.200+0000 7f54c97d8700 1 -- 172.21.15.42:0/2195192470 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f54c40fee00 con 0x7f54ac060f70 2024-02-15T18:30:55.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.202+0000 7f54c0ff9700 1 -- 172.21.15.42:0/2195192470 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f54c40fee00 con 0x7f54ac060f70 2024-02-15T18:30:55.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.204+0000 7f54c97d8700 1 -- 172.21.15.42:0/2195192470 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f54ac060f70 msgr2=0x7f54ac063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:55.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.204+0000 7f54c97d8700 1 --2- 172.21.15.42:0/2195192470 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f54ac060f70 0x7f54ac063430 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f54b4009f50 tx=0x7f54b4009b30).stop 2024-02-15T18:30:55.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.204+0000 7f54c97d8700 1 -- 172.21.15.42:0/2195192470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c40f77d0 msgr2=0x7f54c41073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:55.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.204+0000 7f54c97d8700 1 --2- 172.21.15.42:0/2195192470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c40f77d0 0x7f54c41073b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f54b800c790 tx=0x7f54b8007d70).stop 2024-02-15T18:30:55.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.204+0000 7f54c97d8700 1 -- 172.21.15.42:0/2195192470 shutdown_connections 2024-02-15T18:30:55.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.204+0000 7f54c97d8700 1 --2- 172.21.15.42:0/2195192470 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f54ac060f70 0x7f54ac063430 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.204+0000 7f54c97d8700 1 --2- 172.21.15.42:0/2195192470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f54c40f6530 0x7f54c4106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.204+0000 7f54c97d8700 1 --2- 172.21.15.42:0/2195192470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c40f77d0 0x7f54c41073b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.204+0000 7f54c97d8700 1 -- 172.21.15.42:0/2195192470 >> 172.21.15.42:0/2195192470 conn(0x7f54c40f1c90 msgr2=0x7f54c40fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:55.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.204+0000 7f54c97d8700 1 -- 172.21.15.42:0/2195192470 shutdown_connections 2024-02-15T18:30:55.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.204+0000 7f54c97d8700 1 -- 172.21.15.42:0/2195192470 wait complete. 2024-02-15T18:30:55.208 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:30:55.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.307+0000 7fcba7823700 1 -- 172.21.15.42:0/3867165835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba00f7980 msgr2=0x7fcba00f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:55.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.307+0000 7fcba7823700 1 --2- 172.21.15.42:0/3867165835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba00f7980 0x7fcba00f7de0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fcb9c004660 tx=0x7fcb9c009b30).stop 2024-02-15T18:30:55.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.307+0000 7fcba7823700 1 -- 172.21.15.42:0/3867165835 shutdown_connections 2024-02-15T18:30:55.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.307+0000 7fcba7823700 1 --2- 172.21.15.42:0/3867165835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba00f7980 0x7fcba00f7de0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.307+0000 7fcba7823700 1 --2- 172.21.15.42:0/3867165835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcba00f66e0 0x7fcba00f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.307+0000 7fcba7823700 1 -- 172.21.15.42:0/3867165835 >> 172.21.15.42:0/3867165835 conn(0x7fcba00f1e40 msgr2=0x7fcba00f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:55.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.307+0000 7fcba7823700 1 -- 172.21.15.42:0/3867165835 shutdown_connections 2024-02-15T18:30:55.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.307+0000 7fcba7823700 1 -- 172.21.15.42:0/3867165835 wait complete. 2024-02-15T18:30:55.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.307+0000 7fcba7823700 1 Processor -- start 2024-02-15T18:30:55.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.307+0000 7fcba7823700 1 -- start start 2024-02-15T18:30:55.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcba7823700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba00f66e0 0x7fcba0107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:55.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcba7823700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcba00f7980 0x7fcba01074f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:55.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcba7823700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0107aa0 con 0x7fcba00f66e0 2024-02-15T18:30:55.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcba7823700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0107be0 con 0x7fcba00f7980 2024-02-15T18:30:55.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcba55bf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba00f66e0 0x7fcba0107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:55.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcba55bf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba00f66e0 0x7fcba0107000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58634/0 (socket says 172.21.15.42:58634) 2024-02-15T18:30:55.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcba55bf700 1 -- 172.21.15.42:0/851704200 learned_addr learned my addr 172.21.15.42:0/851704200 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:30:55.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcba4dbe700 1 --2- 172.21.15.42:0/851704200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcba00f7980 0x7fcba01074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:55.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcba55bf700 1 -- 172.21.15.42:0/851704200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcba00f7980 msgr2=0x7fcba01074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:55.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcba55bf700 1 --2- 172.21.15.42:0/851704200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcba00f7980 0x7fcba01074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcba55bf700 1 -- 172.21.15.42:0/851704200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c005040 con 0x7fcba00f66e0 2024-02-15T18:30:55.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcba4dbe700 1 --2- 172.21.15.42:0/851704200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcba00f7980 0x7fcba01074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:30:55.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcba55bf700 1 --2- 172.21.15.42:0/851704200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba00f66e0 0x7fcba0107000 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fcb90002700 tx=0x7fcb90009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:55.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcb95ffb700 1 -- 172.21.15.42:0/851704200 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb90009320 con 0x7fcba00f66e0 2024-02-15T18:30:55.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcb95ffb700 1 -- 172.21.15.42:0/851704200 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fcb9000f040 con 0x7fcba00f66e0 2024-02-15T18:30:55.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcba7823700 1 -- 172.21.15.42:0/851704200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba0108580 con 0x7fcba00f66e0 2024-02-15T18:30:55.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.308+0000 7fcb95ffb700 1 -- 172.21.15.42:0/851704200 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb900149b0 con 0x7fcba00f66e0 2024-02-15T18:30:55.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.309+0000 7fcba7823700 1 -- 172.21.15.42:0/851704200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba0108ac0 con 0x7fcba00f66e0 2024-02-15T18:30:55.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.310+0000 7fcba7823700 1 -- 172.21.15.42:0/851704200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba0045ad0 con 0x7fcba00f66e0 2024-02-15T18:30:55.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.310+0000 7fcb95ffb700 1 -- 172.21.15.42:0/851704200 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fcb90009480 con 0x7fcba00f66e0 2024-02-15T18:30:55.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.311+0000 7fcb95ffb700 1 --2- 172.21.15.42:0/851704200 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb8c065330 0x7fcb8c0677f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:55.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.311+0000 7fcb95ffb700 1 -- 172.21.15.42:0/851704200 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fcb9007c360 con 0x7fcba00f66e0 2024-02-15T18:30:55.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.311+0000 7fcba4dbe700 1 --2- 172.21.15.42:0/851704200 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb8c065330 0x7fcb8c0677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:55.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.311+0000 7fcba4dbe700 1 --2- 172.21.15.42:0/851704200 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb8c065330 0x7fcb8c0677f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fcb9c005010 tx=0x7fcb9c009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:55.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.315+0000 7fcb95ffb700 1 -- 172.21.15.42:0/851704200 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcb90050300 con 0x7fcba00f66e0 2024-02-15T18:30:55.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.505+0000 7fcba7823700 1 -- 172.21.15.42:0/851704200 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcba0108d90 con 0x7fcb8c065330 2024-02-15T18:30:55.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.506+0000 7fcb95ffb700 1 -- 172.21.15.42:0/851704200 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fcba0108d90 con 0x7fcb8c065330 2024-02-15T18:30:55.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.508+0000 7fcba7823700 1 -- 172.21.15.42:0/851704200 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb8c065330 msgr2=0x7fcb8c0677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:55.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.508+0000 7fcba7823700 1 --2- 172.21.15.42:0/851704200 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb8c065330 0x7fcb8c0677f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fcb9c005010 tx=0x7fcb9c009b30).stop 2024-02-15T18:30:55.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.508+0000 7fcba7823700 1 -- 172.21.15.42:0/851704200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba00f66e0 msgr2=0x7fcba0107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:55.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.508+0000 7fcba7823700 1 --2- 172.21.15.42:0/851704200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba00f66e0 0x7fcba0107000 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fcb90002700 tx=0x7fcb90009e00).stop 2024-02-15T18:30:55.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.508+0000 7fcba7823700 1 -- 172.21.15.42:0/851704200 shutdown_connections 2024-02-15T18:30:55.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.508+0000 7fcba7823700 1 --2- 172.21.15.42:0/851704200 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb8c065330 0x7fcb8c0677f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.508+0000 7fcba7823700 1 --2- 172.21.15.42:0/851704200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba00f66e0 0x7fcba0107000 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.508+0000 7fcba7823700 1 --2- 172.21.15.42:0/851704200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcba00f7980 0x7fcba01074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.508+0000 7fcba7823700 1 -- 172.21.15.42:0/851704200 >> 172.21.15.42:0/851704200 conn(0x7fcba00f1e40 msgr2=0x7fcba00fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:55.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.508+0000 7fcba7823700 1 -- 172.21.15.42:0/851704200 shutdown_connections 2024-02-15T18:30:55.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.508+0000 7fcba7823700 1 -- 172.21.15.42:0/851704200 wait complete. 2024-02-15T18:30:55.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.608+0000 7f8b491da700 1 -- 172.21.15.42:0/253780571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b440f6860 msgr2=0x7f8b440f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:55.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.608+0000 7f8b491da700 1 --2- 172.21.15.42:0/253780571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b440f6860 0x7f8b440f6cc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f8b38004660 tx=0x7f8b38009b30).stop 2024-02-15T18:30:55.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.608+0000 7f8b491da700 1 -- 172.21.15.42:0/253780571 shutdown_connections 2024-02-15T18:30:55.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.608+0000 7f8b491da700 1 --2- 172.21.15.42:0/253780571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b440f6860 0x7f8b440f6cc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.608+0000 7f8b491da700 1 --2- 172.21.15.42:0/253780571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b440f55c0 0x7f8b440f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.608+0000 7f8b491da700 1 -- 172.21.15.42:0/253780571 >> 172.21.15.42:0/253780571 conn(0x7f8b440f0d40 msgr2=0x7f8b440f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:55.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.608+0000 7f8b491da700 1 -- 172.21.15.42:0/253780571 shutdown_connections 2024-02-15T18:30:55.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.608+0000 7f8b491da700 1 -- 172.21.15.42:0/253780571 wait complete. 2024-02-15T18:30:55.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.609+0000 7f8b491da700 1 Processor -- start 2024-02-15T18:30:55.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.609+0000 7f8b491da700 1 -- start start 2024-02-15T18:30:55.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.609+0000 7f8b491da700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b440f55c0 0x7f8b44106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:55.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.609+0000 7f8b491da700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b440f6860 0x7f8b441074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:55.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.609+0000 7f8b491da700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44107a60 con 0x7f8b440f6860 2024-02-15T18:30:55.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.609+0000 7f8b491da700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b44107ba0 con 0x7f8b440f55c0 2024-02-15T18:30:55.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.609+0000 7f8b4259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b440f6860 0x7f8b441074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:55.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.609+0000 7f8b4259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b440f6860 0x7f8b441074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58644/0 (socket says 172.21.15.42:58644) 2024-02-15T18:30:55.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.609+0000 7f8b4259c700 1 -- 172.21.15.42:0/1301555194 learned_addr learned my addr 172.21.15.42:0/1301555194 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:30:55.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.610+0000 7f8b42d9d700 1 --2- 172.21.15.42:0/1301555194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b440f55c0 0x7f8b44106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:55.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.610+0000 7f8b4259c700 1 -- 172.21.15.42:0/1301555194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b440f55c0 msgr2=0x7f8b44106fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:55.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.610+0000 7f8b4259c700 1 --2- 172.21.15.42:0/1301555194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b440f55c0 0x7f8b44106fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.610+0000 7f8b4259c700 1 -- 172.21.15.42:0/1301555194 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38005040 con 0x7f8b440f6860 2024-02-15T18:30:55.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.610+0000 7f8b42d9d700 1 --2- 172.21.15.42:0/1301555194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b440f55c0 0x7f8b44106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:30:55.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.610+0000 7f8b4259c700 1 --2- 172.21.15.42:0/1301555194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b440f6860 0x7f8b441074b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f8b38009f50 tx=0x7f8b38009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:55.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.610+0000 7f8b2b7fe700 1 -- 172.21.15.42:0/1301555194 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b38018070 con 0x7f8b440f6860 2024-02-15T18:30:55.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.610+0000 7f8b491da700 1 -- 172.21.15.42:0/1301555194 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b441084e0 con 0x7f8b440f6860 2024-02-15T18:30:55.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.610+0000 7f8b2b7fe700 1 -- 172.21.15.42:0/1301555194 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f8b38007d70 con 0x7f8b440f6860 2024-02-15T18:30:55.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.610+0000 7f8b2b7fe700 1 -- 172.21.15.42:0/1301555194 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b3801c620 con 0x7f8b440f6860 2024-02-15T18:30:55.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.610+0000 7f8b491da700 1 -- 172.21.15.42:0/1301555194 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b44108a40 con 0x7f8b440f6860 2024-02-15T18:30:55.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.611+0000 7f8b491da700 1 -- 172.21.15.42:0/1301555194 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b44045ad0 con 0x7f8b440f6860 2024-02-15T18:30:55.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.612+0000 7f8b2b7fe700 1 -- 172.21.15.42:0/1301555194 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f8b3801c8e0 con 0x7f8b440f6860 2024-02-15T18:30:55.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.612+0000 7f8b2b7fe700 1 --2- 172.21.15.42:0/1301555194 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f8b2c060f20 0x7f8b2c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:55.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.612+0000 7f8b2b7fe700 1 -- 172.21.15.42:0/1301555194 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f8b3807e070 con 0x7f8b440f6860 2024-02-15T18:30:55.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.612+0000 7f8b42d9d700 1 --2- 172.21.15.42:0/1301555194 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f8b2c060f20 0x7f8b2c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:55.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.612+0000 7f8b42d9d700 1 --2- 172.21.15.42:0/1301555194 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f8b2c060f20 0x7f8b2c0633e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f8b440f66c0 tx=0x7f8b34009040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:55.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.616+0000 7f8b2b7fe700 1 -- 172.21.15.42:0/1301555194 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8b38051f20 con 0x7f8b440f6860 2024-02-15T18:30:55.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.796+0000 7f8b491da700 1 -- 172.21.15.42:0/1301555194 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b440fdfd0 con 0x7f8b2c060f20 2024-02-15T18:30:55.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.806+0000 7f8b2b7fe700 1 -- 172.21.15.42:0/1301555194 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f8b440fdfd0 con 0x7f8b2c060f20 2024-02-15T18:30:55.800 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:30:55.800 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (10m) 51s ago 12m 21.4M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:30:55.800 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (3m) 51s ago 12m 6643k - 16.2.14-521-g0e714d9a 1fbc92c859dc 2da9d875539a 2024-02-15T18:30:55.800 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (3m) 22s ago 11m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 5c46a68ff62b 2024-02-15T18:30:55.800 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (10m) 51s ago 12m 33.7M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:30:55.800 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (7m) 51s ago 7m 14.9M - 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (7m) 51s ago 7m 17.5M - 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (7m) 22s ago 7m 110M - 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (7m) 22s ago 7m 153M - 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (4m) 51s ago 13m 435M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (4m) 22s ago 11m 389M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (4m) 51s ago 14m 46.9M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (3m) 22s ago 11m 38.8M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (11m) 51s ago 11m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (11m) 22s ago 11m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (2m) 51s ago 10m 250M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc f226bea76cab 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (103s) 51s ago 10m 264M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 5114ed026724 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (57s) 51s ago 9m 11.6M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 2e26b1ec292a 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (26s) 22s ago 9m 15.4M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc c71b210091cb 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (9m) 22s ago 9m 482M 4096M 16.2.4 8d91d370c2b8 24582012e234 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (8m) 22s ago 8m 401M 4096M 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:30:55.801 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (5m) 51s ago 11m 61.0M - 2.18.1 de242295e225 63b6dfb825ed 2024-02-15T18:30:55.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.808+0000 7f8b491da700 1 -- 172.21.15.42:0/1301555194 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f8b2c060f20 msgr2=0x7f8b2c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:55.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.808+0000 7f8b491da700 1 --2- 172.21.15.42:0/1301555194 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f8b2c060f20 0x7f8b2c0633e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f8b440f66c0 tx=0x7f8b34009040).stop 2024-02-15T18:30:55.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.808+0000 7f8b491da700 1 -- 172.21.15.42:0/1301555194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b440f6860 msgr2=0x7f8b441074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:55.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.808+0000 7f8b491da700 1 --2- 172.21.15.42:0/1301555194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b440f6860 0x7f8b441074b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f8b38009f50 tx=0x7f8b38009b30).stop 2024-02-15T18:30:55.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.808+0000 7f8b491da700 1 -- 172.21.15.42:0/1301555194 shutdown_connections 2024-02-15T18:30:55.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.808+0000 7f8b491da700 1 --2- 172.21.15.42:0/1301555194 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f8b2c060f20 0x7f8b2c0633e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.808+0000 7f8b491da700 1 --2- 172.21.15.42:0/1301555194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b440f55c0 0x7f8b44106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.808+0000 7f8b491da700 1 --2- 172.21.15.42:0/1301555194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b440f6860 0x7f8b441074b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.808+0000 7f8b491da700 1 -- 172.21.15.42:0/1301555194 >> 172.21.15.42:0/1301555194 conn(0x7f8b440f0d40 msgr2=0x7f8b4405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:55.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.808+0000 7f8b491da700 1 -- 172.21.15.42:0/1301555194 shutdown_connections 2024-02-15T18:30:55.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.808+0000 7f8b491da700 1 -- 172.21.15.42:0/1301555194 wait complete. 2024-02-15T18:30:55.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.911+0000 7f5911294700 1 -- 172.21.15.42:0/3665099581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f590c0f6860 msgr2=0x7f590c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:55.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.911+0000 7f5911294700 1 --2- 172.21.15.42:0/3665099581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f590c0f6860 0x7f590c0f6cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f5900004660 tx=0x7f5900009b30).stop 2024-02-15T18:30:55.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.911+0000 7f5911294700 1 -- 172.21.15.42:0/3665099581 shutdown_connections 2024-02-15T18:30:55.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.911+0000 7f5911294700 1 --2- 172.21.15.42:0/3665099581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f590c0f6860 0x7f590c0f6cc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.911+0000 7f5911294700 1 --2- 172.21.15.42:0/3665099581 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590c0f55c0 0x7f590c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.911+0000 7f5911294700 1 -- 172.21.15.42:0/3665099581 >> 172.21.15.42:0/3665099581 conn(0x7f590c0f0d40 msgr2=0x7f590c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:55.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.912+0000 7f5911294700 1 -- 172.21.15.42:0/3665099581 shutdown_connections 2024-02-15T18:30:55.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.912+0000 7f5911294700 1 -- 172.21.15.42:0/3665099581 wait complete. 2024-02-15T18:30:55.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.912+0000 7f5911294700 1 Processor -- start 2024-02-15T18:30:55.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.912+0000 7f5911294700 1 -- start start 2024-02-15T18:30:55.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.912+0000 7f5911294700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590c0f55c0 0x7f590c109cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:55.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.912+0000 7f5911294700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f590c0f6860 0x7f590c10c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:55.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.912+0000 7f5911294700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c0604e0 con 0x7f590c0f6860 2024-02-15T18:30:55.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.912+0000 7f5911294700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c060620 con 0x7f590c0f55c0 2024-02-15T18:30:55.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.913+0000 7f590a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f590c0f6860 0x7f590c10c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:55.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.913+0000 7f590a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f590c0f6860 0x7f590c10c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58662/0 (socket says 172.21.15.42:58662) 2024-02-15T18:30:55.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.913+0000 7f590a7fc700 1 -- 172.21.15.42:0/1098588888 learned_addr learned my addr 172.21.15.42:0/1098588888 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:30:55.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.913+0000 7f590affd700 1 --2- 172.21.15.42:0/1098588888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590c0f55c0 0x7f590c109cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:55.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.913+0000 7f590a7fc700 1 -- 172.21.15.42:0/1098588888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590c0f55c0 msgr2=0x7f590c109cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:55.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.913+0000 7f590a7fc700 1 --2- 172.21.15.42:0/1098588888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590c0f55c0 0x7f590c109cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:55.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.913+0000 7f590a7fc700 1 -- 172.21.15.42:0/1098588888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900005040 con 0x7f590c0f6860 2024-02-15T18:30:55.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.913+0000 7f590a7fc700 1 --2- 172.21.15.42:0/1098588888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f590c0f6860 0x7f590c10c1f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f5900009f50 tx=0x7f5900009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:55.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.913+0000 7f58f37fe700 1 -- 172.21.15.42:0/1098588888 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5900018070 con 0x7f590c0f6860 2024-02-15T18:30:55.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.913+0000 7f5911294700 1 -- 172.21.15.42:0/1098588888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590c10c740 con 0x7f590c0f6860 2024-02-15T18:30:55.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.913+0000 7f58f37fe700 1 -- 172.21.15.42:0/1098588888 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f5900007ba0 con 0x7f590c0f6860 2024-02-15T18:30:55.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.913+0000 7f58f37fe700 1 -- 172.21.15.42:0/1098588888 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f590001c420 con 0x7f590c0f6860 2024-02-15T18:30:55.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.913+0000 7f5911294700 1 -- 172.21.15.42:0/1098588888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590c10cc70 con 0x7f590c0f6860 2024-02-15T18:30:55.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.914+0000 7f5911294700 1 -- 172.21.15.42:0/1098588888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590c045ad0 con 0x7f590c0f6860 2024-02-15T18:30:55.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.915+0000 7f58f37fe700 1 -- 172.21.15.42:0/1098588888 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f590001d420 con 0x7f590c0f6860 2024-02-15T18:30:55.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.915+0000 7f58f37fe700 1 --2- 172.21.15.42:0/1098588888 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f58f4060f20 0x7f58f40633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:55.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.915+0000 7f58f37fe700 1 -- 172.21.15.42:0/1098588888 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f590007dda0 con 0x7f590c0f6860 2024-02-15T18:30:55.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.915+0000 7f590affd700 1 --2- 172.21.15.42:0/1098588888 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f58f4060f20 0x7f58f40633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:55.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.916+0000 7f590affd700 1 --2- 172.21.15.42:0/1098588888 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f58f4060f20 0x7f58f40633e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f590c060160 tx=0x7f58fc00b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:55.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:55.919+0000 7f58f37fe700 1 -- 172.21.15.42:0/1098588888 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f590004e1c0 con 0x7f590c0f6860 2024-02-15T18:30:56.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:54.739540+0000 mgr.smithi042.kfoihb (mgr.24435) 230 : cluster [DBG] pgmap v184: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-02-15T18:30:56.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.163+0000 7f5911294700 1 -- 172.21.15.42:0/1098588888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f590c027a00 con 0x7f590c0f6860 2024-02-15T18:30:56.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.163+0000 7f58f37fe700 1 -- 172.21.15.42:0/1098588888 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+820 (secure 0 0 0) 0x7f59000517e0 con 0x7f590c0f6860 2024-02-15T18:30:56.157 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:30:56.157 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:30:56.157 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:30:56.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:30:56.157 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:30:56.157 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:30:56.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:30:56.158 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:30:56.158 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4, 2024-02-15T18:30:56.158 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:30:56.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:30:56.158 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:30:56.158 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:30:56.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:30:56.158 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:30:56.158 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 8, 2024-02-15T18:30:56.158 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2024-02-15T18:30:56.158 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:30:56.158 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:30:56.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.165+0000 7f5911294700 1 -- 172.21.15.42:0/1098588888 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f58f4060f20 msgr2=0x7f58f40633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:56.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.165+0000 7f5911294700 1 --2- 172.21.15.42:0/1098588888 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f58f4060f20 0x7f58f40633e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f590c060160 tx=0x7f58fc00b040).stop 2024-02-15T18:30:56.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.165+0000 7f5911294700 1 -- 172.21.15.42:0/1098588888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f590c0f6860 msgr2=0x7f590c10c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:56.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.165+0000 7f5911294700 1 --2- 172.21.15.42:0/1098588888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f590c0f6860 0x7f590c10c1f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f5900009f50 tx=0x7f5900009b30).stop 2024-02-15T18:30:56.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.165+0000 7f5911294700 1 -- 172.21.15.42:0/1098588888 shutdown_connections 2024-02-15T18:30:56.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.165+0000 7f5911294700 1 --2- 172.21.15.42:0/1098588888 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f58f4060f20 0x7f58f40633e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.165+0000 7f5911294700 1 --2- 172.21.15.42:0/1098588888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590c0f55c0 0x7f590c109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.165+0000 7f5911294700 1 --2- 172.21.15.42:0/1098588888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f590c0f6860 0x7f590c10c1f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.165+0000 7f5911294700 1 -- 172.21.15.42:0/1098588888 >> 172.21.15.42:0/1098588888 conn(0x7f590c0f0d40 msgr2=0x7f590c0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:56.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.165+0000 7f5911294700 1 -- 172.21.15.42:0/1098588888 shutdown_connections 2024-02-15T18:30:56.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.165+0000 7f5911294700 1 -- 172.21.15.42:0/1098588888 wait complete. 2024-02-15T18:30:56.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:54.739540+0000 mgr.smithi042.kfoihb (mgr.24435) 230 : cluster [DBG] pgmap v184: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 938 B/s rd, 1 op/s 2024-02-15T18:30:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.266+0000 7f3035f32700 1 -- 172.21.15.42:0/2510168692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3030060310 msgr2=0x7f30300f9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.266+0000 7f3035f32700 1 --2- 172.21.15.42:0/2510168692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3030060310 0x7f30300f9070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f3024004660 tx=0x7f3024009b30).stop 2024-02-15T18:30:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.267+0000 7f3035f32700 1 -- 172.21.15.42:0/2510168692 shutdown_connections 2024-02-15T18:30:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.267+0000 7f3035f32700 1 --2- 172.21.15.42:0/2510168692 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30300f95f0 0x7f30300fba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.267+0000 7f3035f32700 1 --2- 172.21.15.42:0/2510168692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3030060310 0x7f30300f9070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.267+0000 7f3035f32700 1 -- 172.21.15.42:0/2510168692 >> 172.21.15.42:0/2510168692 conn(0x7f30300f0ad0 msgr2=0x7f30300f2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.267+0000 7f3035f32700 1 -- 172.21.15.42:0/2510168692 shutdown_connections 2024-02-15T18:30:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.267+0000 7f3035f32700 1 -- 172.21.15.42:0/2510168692 wait complete. 2024-02-15T18:30:56.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.267+0000 7f3035f32700 1 Processor -- start 2024-02-15T18:30:56.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.267+0000 7f3035f32700 1 -- start start 2024-02-15T18:30:56.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.267+0000 7f3035f32700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3030060310 0x7f3030106d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:56.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f3035f32700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30300f95f0 0x7f3030107280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:56.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f3035f32700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3030107880 con 0x7f3030060310 2024-02-15T18:30:56.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f3035f32700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3030108090 con 0x7f30300f95f0 2024-02-15T18:30:56.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f3034f30700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3030060310 0x7f3030106d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:56.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f3034f30700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3030060310 0x7f3030106d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58676/0 (socket says 172.21.15.42:58676) 2024-02-15T18:30:56.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f3034f30700 1 -- 172.21.15.42:0/3027573568 learned_addr learned my addr 172.21.15.42:0/3027573568 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:30:56.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f302ffff700 1 --2- 172.21.15.42:0/3027573568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30300f95f0 0x7f3030107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:56.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f3034f30700 1 -- 172.21.15.42:0/3027573568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30300f95f0 msgr2=0x7f3030107280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:56.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f3034f30700 1 --2- 172.21.15.42:0/3027573568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30300f95f0 0x7f3030107280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f3034f30700 1 -- 172.21.15.42:0/3027573568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3024005040 con 0x7f3030060310 2024-02-15T18:30:56.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f3034f30700 1 --2- 172.21.15.42:0/3027573568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3030060310 0x7f3030106d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f3024009f50 tx=0x7f3024009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:56.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f302d7fa700 1 -- 172.21.15.42:0/3027573568 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3024018070 con 0x7f3030060310 2024-02-15T18:30:56.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f3035f32700 1 -- 172.21.15.42:0/3027573568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30301082b0 con 0x7f3030060310 2024-02-15T18:30:56.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f302d7fa700 1 -- 172.21.15.42:0/3027573568 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f3024007ba0 con 0x7f3030060310 2024-02-15T18:30:56.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.268+0000 7f302d7fa700 1 -- 172.21.15.42:0/3027573568 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f302401c420 con 0x7f3030060310 2024-02-15T18:30:56.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.269+0000 7f3035f32700 1 -- 172.21.15.42:0/3027573568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3030108810 con 0x7f3030060310 2024-02-15T18:30:56.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.270+0000 7f302d7fa700 1 -- 172.21.15.42:0/3027573568 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f302401d420 con 0x7f3030060310 2024-02-15T18:30:56.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.270+0000 7f3035f32700 1 -- 172.21.15.42:0/3027573568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3030059100 con 0x7f3030060310 2024-02-15T18:30:56.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.270+0000 7f302d7fa700 1 --2- 172.21.15.42:0/3027573568 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3020060f70 0x7f3020063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:56.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.270+0000 7f302d7fa700 1 -- 172.21.15.42:0/3027573568 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f302407d9e0 con 0x7f3030060310 2024-02-15T18:30:56.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.270+0000 7f302ffff700 1 --2- 172.21.15.42:0/3027573568 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3020060f70 0x7f3020063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:56.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.271+0000 7f302ffff700 1 --2- 172.21.15.42:0/3027573568 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3020060f70 0x7f3020063430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f30300600e0 tx=0x7f301c00b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:56.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.274+0000 7f302d7fa700 1 -- 172.21.15.42:0/3027573568 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f302404de00 con 0x7f3030060310 2024-02-15T18:30:56.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.491+0000 7f3035f32700 1 -- 172.21.15.42:0/3027573568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f30300fed80 con 0x7f3030060310 2024-02-15T18:30:56.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.491+0000 7f302d7fa700 1 -- 172.21.15.42:0/3027573568 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1773 (secure 0 0 0) 0x7f3024051420 con 0x7f3030060310 2024-02-15T18:30:56.485 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:30:56.485 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:30:56.485 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:30:56.485 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:30:56.485 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:30:56.485 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:30:56.485 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:30:56.486 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:30:56.487 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:30:56.488 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:30:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.493+0000 7f3035f32700 1 -- 172.21.15.42:0/3027573568 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3020060f70 msgr2=0x7f3020063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:56.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.493+0000 7f3035f32700 1 --2- 172.21.15.42:0/3027573568 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3020060f70 0x7f3020063430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f30300600e0 tx=0x7f301c00b040).stop 2024-02-15T18:30:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.493+0000 7f3035f32700 1 -- 172.21.15.42:0/3027573568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3030060310 msgr2=0x7f3030106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.493+0000 7f3035f32700 1 --2- 172.21.15.42:0/3027573568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3030060310 0x7f3030106d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f3024009f50 tx=0x7f3024009b30).stop 2024-02-15T18:30:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.493+0000 7f3035f32700 1 -- 172.21.15.42:0/3027573568 shutdown_connections 2024-02-15T18:30:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.493+0000 7f3035f32700 1 --2- 172.21.15.42:0/3027573568 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3020060f70 0x7f3020063430 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.493+0000 7f3035f32700 1 --2- 172.21.15.42:0/3027573568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3030060310 0x7f3030106d90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.493+0000 7f3035f32700 1 --2- 172.21.15.42:0/3027573568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30300f95f0 0x7f3030107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.493+0000 7f3035f32700 1 -- 172.21.15.42:0/3027573568 >> 172.21.15.42:0/3027573568 conn(0x7f30300f0ad0 msgr2=0x7f30300f2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.493+0000 7f3035f32700 1 -- 172.21.15.42:0/3027573568 shutdown_connections 2024-02-15T18:30:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.493+0000 7f3035f32700 1 -- 172.21.15.42:0/3027573568 wait complete. 2024-02-15T18:30:56.489 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:30:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.595+0000 7f0e00c30700 1 -- 172.21.15.42:0/4025020450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dfc0f6860 msgr2=0x7f0dfc0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.595+0000 7f0e00c30700 1 --2- 172.21.15.42:0/4025020450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dfc0f6860 0x7f0dfc0f6cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f0df0004660 tx=0x7f0df0009b30).stop 2024-02-15T18:30:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.595+0000 7f0e00c30700 1 -- 172.21.15.42:0/4025020450 shutdown_connections 2024-02-15T18:30:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.595+0000 7f0e00c30700 1 --2- 172.21.15.42:0/4025020450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dfc0f6860 0x7f0dfc0f6cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.595+0000 7f0e00c30700 1 --2- 172.21.15.42:0/4025020450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dfc0f55c0 0x7f0dfc0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.595+0000 7f0e00c30700 1 -- 172.21.15.42:0/4025020450 >> 172.21.15.42:0/4025020450 conn(0x7f0dfc0f0d40 msgr2=0x7f0dfc0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.595+0000 7f0e00c30700 1 -- 172.21.15.42:0/4025020450 shutdown_connections 2024-02-15T18:30:56.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.595+0000 7f0e00c30700 1 -- 172.21.15.42:0/4025020450 wait complete. 2024-02-15T18:30:56.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.596+0000 7f0e00c30700 1 Processor -- start 2024-02-15T18:30:56.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.596+0000 7f0e00c30700 1 -- start start 2024-02-15T18:30:56.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.596+0000 7f0e00c30700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dfc0f55c0 0x7f0dfc106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:56.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.596+0000 7f0e00c30700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dfc0f6860 0x7f0dfc1074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:56.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.596+0000 7f0e00c30700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc107a60 con 0x7f0dfc0f55c0 2024-02-15T18:30:56.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.596+0000 7f0e00c30700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc107ba0 con 0x7f0dfc0f6860 2024-02-15T18:30:56.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.596+0000 7f0dfa59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dfc0f55c0 0x7f0dfc106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:56.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.596+0000 7f0dfa59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dfc0f55c0 0x7f0dfc106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58682/0 (socket says 172.21.15.42:58682) 2024-02-15T18:30:56.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.596+0000 7f0dfa59c700 1 -- 172.21.15.42:0/2021697682 learned_addr learned my addr 172.21.15.42:0/2021697682 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:30:56.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.596+0000 7f0df9d9b700 1 --2- 172.21.15.42:0/2021697682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dfc0f6860 0x7f0dfc1074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:56.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.596+0000 7f0dfa59c700 1 -- 172.21.15.42:0/2021697682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dfc0f6860 msgr2=0x7f0dfc1074b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.596+0000 7f0dfa59c700 1 --2- 172.21.15.42:0/2021697682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dfc0f6860 0x7f0dfc1074b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.596+0000 7f0dfa59c700 1 -- 172.21.15.42:0/2021697682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df0005040 con 0x7f0dfc0f55c0 2024-02-15T18:30:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.596+0000 7f0dfa59c700 1 --2- 172.21.15.42:0/2021697682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dfc0f55c0 0x7f0dfc106fc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f0dec002700 tx=0x7f0dec009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.597+0000 7f0deaffd700 1 -- 172.21.15.42:0/2021697682 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dec0079c0 con 0x7f0dfc0f55c0 2024-02-15T18:30:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.597+0000 7f0e00c30700 1 -- 172.21.15.42:0/2021697682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dfc108540 con 0x7f0dfc0f55c0 2024-02-15T18:30:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.597+0000 7f0deaffd700 1 -- 172.21.15.42:0/2021697682 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f0dec00f040 con 0x7f0dfc0f55c0 2024-02-15T18:30:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.597+0000 7f0deaffd700 1 -- 172.21.15.42:0/2021697682 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dec014990 con 0x7f0dfc0f55c0 2024-02-15T18:30:56.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.597+0000 7f0e00c30700 1 -- 172.21.15.42:0/2021697682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dfc108a80 con 0x7f0dfc0f55c0 2024-02-15T18:30:56.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.598+0000 7f0e00c30700 1 -- 172.21.15.42:0/2021697682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dfc045ad0 con 0x7f0dfc0f55c0 2024-02-15T18:30:56.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.598+0000 7f0deaffd700 1 -- 172.21.15.42:0/2021697682 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f0dec01b060 con 0x7f0dfc0f55c0 2024-02-15T18:30:56.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.599+0000 7f0deaffd700 1 --2- 172.21.15.42:0/2021697682 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0de4060f20 0x7f0de40633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:30:56.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.599+0000 7f0deaffd700 1 -- 172.21.15.42:0/2021697682 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f0dec07bfd0 con 0x7f0dfc0f55c0 2024-02-15T18:30:56.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.599+0000 7f0df9d9b700 1 --2- 172.21.15.42:0/2021697682 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0de4060f20 0x7f0de40633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:30:56.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.599+0000 7f0df9d9b700 1 --2- 172.21.15.42:0/2021697682 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0de4060f20 0x7f0de40633e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f0df0004630 tx=0x7f0df0009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:30:56.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.603+0000 7f0deaffd700 1 -- 172.21.15.42:0/2021697682 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0dec04fe80 con 0x7f0dfc0f55c0 2024-02-15T18:30:56.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.791+0000 7f0e00c30700 1 -- 172.21.15.42:0/2021697682 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dfc0faca0 con 0x7f0de4060f20 2024-02-15T18:30:56.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.792+0000 7f0deaffd700 1 -- 172.21.15.42:0/2021697682 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f0dfc0faca0 con 0x7f0de4060f20 2024-02-15T18:30:56.786 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:30:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:30:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:30:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:30:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:30:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-02-15T18:30:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-02-15T18:30:56.786 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-02-15T18:30:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:30:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "10/21 daemons upgraded", 2024-02-15T18:30:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading osd daemons", 2024-02-15T18:30:56.787 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:30:56.787 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:30:56.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.794+0000 7f0e00c30700 1 -- 172.21.15.42:0/2021697682 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0de4060f20 msgr2=0x7f0de40633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.794+0000 7f0e00c30700 1 --2- 172.21.15.42:0/2021697682 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0de4060f20 0x7f0de40633e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f0df0004630 tx=0x7f0df0009b30).stop 2024-02-15T18:30:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.794+0000 7f0e00c30700 1 -- 172.21.15.42:0/2021697682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dfc0f55c0 msgr2=0x7f0dfc106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:30:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.794+0000 7f0e00c30700 1 --2- 172.21.15.42:0/2021697682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dfc0f55c0 0x7f0dfc106fc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f0dec002700 tx=0x7f0dec009e00).stop 2024-02-15T18:30:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.794+0000 7f0e00c30700 1 -- 172.21.15.42:0/2021697682 shutdown_connections 2024-02-15T18:30:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.794+0000 7f0e00c30700 1 --2- 172.21.15.42:0/2021697682 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0de4060f20 0x7f0de40633e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.794+0000 7f0e00c30700 1 --2- 172.21.15.42:0/2021697682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0dfc0f55c0 0x7f0dfc106fc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.794+0000 7f0e00c30700 1 --2- 172.21.15.42:0/2021697682 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dfc0f6860 0x7f0dfc1074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:30:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.794+0000 7f0e00c30700 1 -- 172.21.15.42:0/2021697682 >> 172.21.15.42:0/2021697682 conn(0x7f0dfc0f0d40 msgr2=0x7f0dfc05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:30:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.794+0000 7f0e00c30700 1 -- 172.21.15.42:0/2021697682 shutdown_connections 2024-02-15T18:30:56.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:30:56.794+0000 7f0e00c30700 1 -- 172.21.15.42:0/2021697682 wait complete. 2024-02-15T18:30:57.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:55.202471+0000 mgr.smithi042.kfoihb (mgr.24435) 231 : audit [DBG] from='client.34276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:57.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:55.507039+0000 mgr.smithi042.kfoihb (mgr.24435) 232 : audit [DBG] from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:57.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:55.798558+0000 mgr.smithi042.kfoihb (mgr.24435) 233 : audit [DBG] from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:57.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:55.837209+0000 mon.smithi042 (mon.0) 341 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:57.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:55.950624+0000 mon.smithi042 (mon.0) 342 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:57.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:56.164678+0000 mon.smithi042 (mon.0) 343 : audit [DBG] from='client.? 172.21.15.42:0/1098588888' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:57.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:56.492644+0000 mon.smithi042 (mon.0) 344 : audit [DBG] from='client.? 172.21.15.42:0/3027573568' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:30:57.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:55.202471+0000 mgr.smithi042.kfoihb (mgr.24435) 231 : audit [DBG] from='client.34276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:57.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:55.507039+0000 mgr.smithi042.kfoihb (mgr.24435) 232 : audit [DBG] from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:57.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:55.798558+0000 mgr.smithi042.kfoihb (mgr.24435) 233 : audit [DBG] from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:57.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:55.837209+0000 mon.smithi042 (mon.0) 341 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:57.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:55.950624+0000 mon.smithi042 (mon.0) 342 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:30:57.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:56.164678+0000 mon.smithi042 (mon.0) 343 : audit [DBG] from='client.? 172.21.15.42:0/1098588888' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:30:57.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:56.492644+0000 mon.smithi042 (mon.0) 344 : audit [DBG] from='client.? 172.21.15.42:0/3027573568' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:30:58.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:56.739927+0000 mgr.smithi042.kfoihb (mgr.24435) 234 : cluster [DBG] pgmap v185: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-02-15T18:30:58.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:30:56.793165+0000 mgr.smithi042.kfoihb (mgr.24435) 235 : audit [DBG] from='client.34296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:30:58.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:56.739927+0000 mgr.smithi042.kfoihb (mgr.24435) 234 : cluster [DBG] pgmap v185: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-02-15T18:30:58.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:30:56.793165+0000 mgr.smithi042.kfoihb (mgr.24435) 235 : audit [DBG] from='client.34296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:31:00.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:30:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:30:58.740377+0000 mgr.smithi042.kfoihb (mgr.24435) 236 : cluster [DBG] pgmap v186: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:31:00.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:30:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:30:58.740377+0000 mgr.smithi042.kfoihb (mgr.24435) 236 : cluster [DBG] pgmap v186: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:31:02.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:00.741028+0000 mgr.smithi042.kfoihb (mgr.24435) 237 : cluster [DBG] pgmap v187: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:31:02.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:00.741028+0000 mgr.smithi042.kfoihb (mgr.24435) 237 : cluster [DBG] pgmap v187: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:31:04.162 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:02.741495+0000 mgr.smithi042.kfoihb (mgr.24435) 238 : cluster [DBG] pgmap v188: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:31:04.162 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:03.274084+0000 mon.smithi042 (mon.0) 345 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:31:04.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:02.741495+0000 mgr.smithi042.kfoihb (mgr.24435) 238 : cluster [DBG] pgmap v188: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:31:04.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:03.274084+0000 mon.smithi042 (mon.0) 345 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:31:05.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:03.274318+0000 mgr.smithi042.kfoihb (mgr.24435) 239 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:31:05.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:31:03.275388+0000 mgr.smithi042.kfoihb (mgr.24435) 240 : cephadm [INF] Upgrade: osd.4 is safe to restart 2024-02-15T18:31:05.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:03.274318+0000 mgr.smithi042.kfoihb (mgr.24435) 239 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-15T18:31:05.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:31:03.275388+0000 mgr.smithi042.kfoihb (mgr.24435) 240 : cephadm [INF] Upgrade: osd.4 is safe to restart 2024-02-15T18:31:06.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:04.741914+0000 mgr.smithi042.kfoihb (mgr.24435) 241 : cluster [DBG] pgmap v189: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:31:06.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:31:04.917254+0000 mgr.smithi042.kfoihb (mgr.24435) 242 : cephadm [INF] Upgrade: Updating osd.4 2024-02-15T18:31:06.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:04.921456+0000 mon.smithi042 (mon.0) 346 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:06.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:04.922357+0000 mon.smithi042 (mon.0) 347 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-15T18:31:06.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:04.923100+0000 mon.smithi042 (mon.0) 348 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:31:06.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:31:04.924320+0000 mgr.smithi042.kfoihb (mgr.24435) 243 : cephadm [INF] Deploying daemon osd.4 on smithi191 2024-02-15T18:31:06.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:04.741914+0000 mgr.smithi042.kfoihb (mgr.24435) 241 : cluster [DBG] pgmap v189: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:31:06.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:31:04.917254+0000 mgr.smithi042.kfoihb (mgr.24435) 242 : cephadm [INF] Upgrade: Updating osd.4 2024-02-15T18:31:06.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:04.921456+0000 mon.smithi042 (mon.0) 346 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:06.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:04.922357+0000 mon.smithi042 (mon.0) 347 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-15T18:31:06.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:04.923100+0000 mon.smithi042 (mon.0) 348 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:31:06.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:31:04.924320+0000 mgr.smithi042.kfoihb (mgr.24435) 243 : cephadm [INF] Deploying daemon osd.4 on smithi191 2024-02-15T18:31:07.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:06.742617+0000 mgr.smithi042.kfoihb (mgr.24435) 244 : cluster [DBG] pgmap v190: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:31:07.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:06.742617+0000 mgr.smithi042.kfoihb (mgr.24435) 244 : cluster [DBG] pgmap v190: 65 pgs: 65 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:31:08.177 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:07 smithi191 systemd[1]: Stopping Ceph osd.4 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:31:08.443 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.4[48536]: debug 2024-02-15T18:31:08.182+0000 7f5097b8e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:31:08.443 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.4[48536]: debug 2024-02-15T18:31:08.182+0000 7f5097b8e700 -1 osd.4 86 *** Got signal Terminated *** 2024-02-15T18:31:08.443 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.4[48536]: debug 2024-02-15T18:31:08.182+0000 7f5097b8e700 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:31:08.751 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:08 smithi191 bash[63927]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.4 2024-02-15T18:31:08.752 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:08.443471+0000 mon.smithi042 (mon.0) 349 : cluster [DBG] osd.4 reported immediately failed by osd.2 2024-02-15T18:31:08.752 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:08.443505+0000 mon.smithi042 (mon.0) 350 : cluster [INF] osd.4 failed (root=default,host=smithi191) (connection refused reported by osd.2) 2024-02-15T18:31:08.752 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:08.443666+0000 mon.smithi042 (mon.0) 351 : cluster [DBG] osd.4 reported immediately failed by osd.2 2024-02-15T18:31:08.752 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:08.443766+0000 mon.smithi042 (mon.0) 352 : cluster [DBG] osd.4 reported immediately failed by osd.1 2024-02-15T18:31:08.752 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:08.443899+0000 mon.smithi042 (mon.0) 353 : cluster [DBG] osd.4 reported immediately failed by osd.0 2024-02-15T18:31:08.752 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:08.444020+0000 mon.smithi042 (mon.0) 354 : cluster [DBG] osd.4 reported immediately failed by osd.1 2024-02-15T18:31:08.752 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:08.444191+0000 mon.smithi042 (mon.0) 355 : cluster [DBG] osd.4 reported immediately failed by osd.0 2024-02-15T18:31:08.752 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:08.444303+0000 mon.smithi042 (mon.0) 356 : cluster [DBG] osd.4 reported immediately failed by osd.5 2024-02-15T18:31:08.752 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:08.444425+0000 mon.smithi042 (mon.0) 357 : cluster [DBG] osd.4 reported immediately failed by osd.5 2024-02-15T18:31:08.752 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:08.444552+0000 mon.smithi042 (mon.0) 358 : cluster [DBG] osd.4 reported immediately failed by osd.3 2024-02-15T18:31:08.753 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:08.444685+0000 mon.smithi042 (mon.0) 359 : cluster [DBG] osd.4 reported immediately failed by osd.3 2024-02-15T18:31:08.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:08.443471+0000 mon.smithi042 (mon.0) 349 : cluster [DBG] osd.4 reported immediately failed by osd.2 2024-02-15T18:31:08.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:08.443505+0000 mon.smithi042 (mon.0) 350 : cluster [INF] osd.4 failed (root=default,host=smithi191) (connection refused reported by osd.2) 2024-02-15T18:31:08.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:08.443666+0000 mon.smithi042 (mon.0) 351 : cluster [DBG] osd.4 reported immediately failed by osd.2 2024-02-15T18:31:08.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:08.443766+0000 mon.smithi042 (mon.0) 352 : cluster [DBG] osd.4 reported immediately failed by osd.1 2024-02-15T18:31:08.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:08.443899+0000 mon.smithi042 (mon.0) 353 : cluster [DBG] osd.4 reported immediately failed by osd.0 2024-02-15T18:31:08.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:08.444020+0000 mon.smithi042 (mon.0) 354 : cluster [DBG] osd.4 reported immediately failed by osd.1 2024-02-15T18:31:08.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:08.444191+0000 mon.smithi042 (mon.0) 355 : cluster [DBG] osd.4 reported immediately failed by osd.0 2024-02-15T18:31:08.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:08.444303+0000 mon.smithi042 (mon.0) 356 : cluster [DBG] osd.4 reported immediately failed by osd.5 2024-02-15T18:31:08.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:08.444425+0000 mon.smithi042 (mon.0) 357 : cluster [DBG] osd.4 reported immediately failed by osd.5 2024-02-15T18:31:08.807 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:08.444552+0000 mon.smithi042 (mon.0) 358 : cluster [DBG] osd.4 reported immediately failed by osd.3 2024-02-15T18:31:08.807 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:08.444685+0000 mon.smithi042 (mon.0) 359 : cluster [DBG] osd.4 reported immediately failed by osd.3 2024-02-15T18:31:09.062 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:08 smithi191 bash[63982]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.4 2024-02-15T18:31:09.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:08.494160+0000 mon.smithi042 (mon.0) 360 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:31:09.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:08.499080+0000 mon.smithi042 (mon.0) 361 : cluster [DBG] osdmap e87: 6 total, 5 up, 6 in 2024-02-15T18:31:09.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:08.743041+0000 mgr.smithi042.kfoihb (mgr.24435) 245 : cluster [DBG] pgmap v192: 65 pgs: 6 stale+active+clean, 59 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-02-15T18:31:09.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:09.499769+0000 mon.smithi042 (mon.0) 362 : cluster [DBG] osdmap e88: 6 total, 5 up, 6 in 2024-02-15T18:31:09.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:08.494160+0000 mon.smithi042 (mon.0) 360 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:31:09.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:08.499080+0000 mon.smithi042 (mon.0) 361 : cluster [DBG] osdmap e87: 6 total, 5 up, 6 in 2024-02-15T18:31:09.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:08.743041+0000 mgr.smithi042.kfoihb (mgr.24435) 245 : cluster [DBG] pgmap v192: 65 pgs: 6 stale+active+clean, 59 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-02-15T18:31:09.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:09.499769+0000 mon.smithi042 (mon.0) 362 : cluster [DBG] osdmap e88: 6 total, 5 up, 6 in 2024-02-15T18:31:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.700063+0000 mon.smithi042 (mon.0) 363 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1", "id": [0, 4]}]: dispatch 2024-02-15T18:31:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.700369+0000 mon.smithi042 (mon.0) 364 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [0, 4]}]: dispatch 2024-02-15T18:31:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.700558+0000 mon.smithi042 (mon.0) 365 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 4]}]: dispatch 2024-02-15T18:31:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.700745+0000 mon.smithi042 (mon.0) 366 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [3, 2]}]: dispatch 2024-02-15T18:31:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.700943+0000 mon.smithi042 (mon.0) 367 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [3, 2]}]: dispatch 2024-02-15T18:31:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.701150+0000 mon.smithi042 (mon.0) 368 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.19", "id": [0, 5]}]: dispatch 2024-02-15T18:31:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.701331+0000 mon.smithi042 (mon.0) 369 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 0]}]: dispatch 2024-02-15T18:31:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.701518+0000 mon.smithi042 (mon.0) 370 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 5]}]: dispatch 2024-02-15T18:31:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.701704+0000 mon.smithi042 (mon.0) 371 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 2]}]: dispatch 2024-02-15T18:31:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.701910+0000 mon.smithi042 (mon.0) 372 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [3, 0]}]: dispatch 2024-02-15T18:31:11.194 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:10 smithi191 systemd[1]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.4.service: Succeeded. 2024-02-15T18:31:11.194 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:10 smithi191 systemd[1]: Stopped Ceph osd.4 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:31:11.194 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:10 smithi191 systemd[1]: Starting Ceph osd.4 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:31:11.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.700063+0000 mon.smithi042 (mon.0) 363 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1", "id": [0, 4]}]: dispatch 2024-02-15T18:31:11.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.700369+0000 mon.smithi042 (mon.0) 364 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [0, 4]}]: dispatch 2024-02-15T18:31:11.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.700558+0000 mon.smithi042 (mon.0) 365 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 4]}]: dispatch 2024-02-15T18:31:11.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.700745+0000 mon.smithi042 (mon.0) 366 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [3, 2]}]: dispatch 2024-02-15T18:31:11.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.700943+0000 mon.smithi042 (mon.0) 367 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [3, 2]}]: dispatch 2024-02-15T18:31:11.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.701150+0000 mon.smithi042 (mon.0) 368 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.19", "id": [0, 5]}]: dispatch 2024-02-15T18:31:11.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.701331+0000 mon.smithi042 (mon.0) 369 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 0]}]: dispatch 2024-02-15T18:31:11.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.701518+0000 mon.smithi042 (mon.0) 370 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 5]}]: dispatch 2024-02-15T18:31:11.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.701704+0000 mon.smithi042 (mon.0) 371 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 2]}]: dispatch 2024-02-15T18:31:11.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.701910+0000 mon.smithi042 (mon.0) 372 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [3, 0]}]: dispatch 2024-02-15T18:31:12.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:10.743679+0000 mgr.smithi042.kfoihb (mgr.24435) 246 : cluster [DBG] pgmap v194: 65 pgs: 5 active+undersized+degraded, 4 stale+active+clean, 8 active+undersized, 48 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 895 B/s rd, 1 op/s; 18/210 objects degraded (8.571%) 2024-02-15T18:31:12.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.754789+0000 mon.smithi042 (mon.0) 373 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1", "id": [0, 4]}]': finished 2024-02-15T18:31:12.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.754865+0000 mon.smithi042 (mon.0) 374 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [0, 4]}]': finished 2024-02-15T18:31:12.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.754949+0000 mon.smithi042 (mon.0) 375 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 4]}]': finished 2024-02-15T18:31:12.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.755063+0000 mon.smithi042 (mon.0) 376 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [3, 2]}]': finished 2024-02-15T18:31:12.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.755130+0000 mon.smithi042 (mon.0) 377 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [3, 2]}]': finished 2024-02-15T18:31:12.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.755185+0000 mon.smithi042 (mon.0) 378 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.19", "id": [0, 5]}]': finished 2024-02-15T18:31:12.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.755237+0000 mon.smithi042 (mon.0) 379 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 0]}]': finished 2024-02-15T18:31:12.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.755298+0000 mon.smithi042 (mon.0) 380 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 5]}]': finished 2024-02-15T18:31:12.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.755390+0000 mon.smithi042 (mon.0) 381 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 2]}]': finished 2024-02-15T18:31:12.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.755446+0000 mon.smithi042 (mon.0) 382 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [3, 0]}]': finished 2024-02-15T18:31:12.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:10.755485+0000 mon.smithi042 (mon.0) 383 : cluster [DBG] osdmap e89: 6 total, 5 up, 6 in 2024-02-15T18:31:12.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:10.794422+0000 mon.smithi042 (mon.0) 384 : cluster [WRN] Health check failed: Degraded data redundancy: 18/210 objects degraded (8.571%), 5 pgs degraded (PG_DEGRADED) 2024-02-15T18:31:12.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:10.836649+0000 mon.smithi042 (mon.0) 385 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:12.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:10.979958+0000 mon.smithi042 (mon.0) 386 : cluster [DBG] osdmap e90: 6 total, 5 up, 6 in 2024-02-15T18:31:12.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:10.743679+0000 mgr.smithi042.kfoihb (mgr.24435) 246 : cluster [DBG] pgmap v194: 65 pgs: 5 active+undersized+degraded, 4 stale+active+clean, 8 active+undersized, 48 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 895 B/s rd, 1 op/s; 18/210 objects degraded (8.571%) 2024-02-15T18:31:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.754789+0000 mon.smithi042 (mon.0) 373 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1", "id": [0, 4]}]': finished 2024-02-15T18:31:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.754865+0000 mon.smithi042 (mon.0) 374 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [0, 4]}]': finished 2024-02-15T18:31:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.754949+0000 mon.smithi042 (mon.0) 375 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 4]}]': finished 2024-02-15T18:31:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.755063+0000 mon.smithi042 (mon.0) 376 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [3, 2]}]': finished 2024-02-15T18:31:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.755130+0000 mon.smithi042 (mon.0) 377 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.11", "id": [3, 2]}]': finished 2024-02-15T18:31:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.755185+0000 mon.smithi042 (mon.0) 378 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.19", "id": [0, 5]}]': finished 2024-02-15T18:31:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.755237+0000 mon.smithi042 (mon.0) 379 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.a", "id": [1, 0]}]': finished 2024-02-15T18:31:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.755298+0000 mon.smithi042 (mon.0) 380 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [1, 5]}]': finished 2024-02-15T18:31:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.755390+0000 mon.smithi042 (mon.0) 381 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [3, 2]}]': finished 2024-02-15T18:31:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.755446+0000 mon.smithi042 (mon.0) 382 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [3, 0]}]': finished 2024-02-15T18:31:12.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:10.755485+0000 mon.smithi042 (mon.0) 383 : cluster [DBG] osdmap e89: 6 total, 5 up, 6 in 2024-02-15T18:31:12.195 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:11 smithi191 bash[64237]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-02-15T18:31:12.195 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:11 smithi191 bash[64237]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-15T18:31:12.195 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:11 smithi191 bash[64237]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2024-02-15T18:31:12.195 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:11 smithi191 bash[64237]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-15T18:31:12.195 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:11 smithi191 bash[64237]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-4/block 2024-02-15T18:31:12.195 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:11 smithi191 bash[64237]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-15T18:31:12.195 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:11 smithi191 bash[64237]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-15T18:31:12.195 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:11 smithi191 bash[64237]: --> ceph-volume lvm activate successful for osd ID: 4 2024-02-15T18:31:12.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:10.794422+0000 mon.smithi042 (mon.0) 384 : cluster [WRN] Health check failed: Degraded data redundancy: 18/210 objects degraded (8.571%), 5 pgs degraded (PG_DEGRADED) 2024-02-15T18:31:12.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:10.836649+0000 mon.smithi042 (mon.0) 385 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:12.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:10.979958+0000 mon.smithi042 (mon.0) 386 : cluster [DBG] osdmap e90: 6 total, 5 up, 6 in 2024-02-15T18:31:13.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:11.981742+0000 mon.smithi042 (mon.0) 387 : cluster [DBG] osdmap e91: 6 total, 5 up, 6 in 2024-02-15T18:31:13.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:12.744191+0000 mgr.smithi042.kfoihb (mgr.24435) 247 : cluster [DBG] pgmap v198: 65 pgs: 2 remapped+peering, 13 active+undersized+degraded, 2 activating+undersized+remapped, 17 active+undersized, 31 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 35/210 objects degraded (16.667%); 60/210 objects misplaced (28.571%) 2024-02-15T18:31:13.336 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:11.981742+0000 mon.smithi042 (mon.0) 387 : cluster [DBG] osdmap e91: 6 total, 5 up, 6 in 2024-02-15T18:31:13.336 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:12.744191+0000 mgr.smithi042.kfoihb (mgr.24435) 247 : cluster [DBG] pgmap v198: 65 pgs: 2 remapped+peering, 13 active+undersized+degraded, 2 activating+undersized+remapped, 17 active+undersized, 31 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 35/210 objects degraded (16.667%); 60/210 objects misplaced (28.571%) 2024-02-15T18:31:13.693 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:31:13.342+0000 7f18d79a9700 1 -- 172.21.15.191:0/1357105143 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559f95f8a680 con 0x559f95f51400 2024-02-15T18:31:13.693 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:13 smithi191 bash[64506]: 83bd9cdcb3da5fe668736959d26d527cb77d66ab8884da2e376a16594fb56125 2024-02-15T18:31:13.693 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:13 smithi191 systemd[1]: Started Ceph osd.4 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:31:14.193 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:31:13.864+0000 7f18df437380 -1 Falling back to public interface 2024-02-15T18:31:14.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:13.509516+0000 mon.smithi042 (mon.0) 388 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:14.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:13.510364+0000 mon.smithi042 (mon.0) 389 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:31:14.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:13.509516+0000 mon.smithi042 (mon.0) 388 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:14.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:13.510364+0000 mon.smithi042 (mon.0) 389 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:31:15.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:14.510625+0000 mon.smithi042 (mon.0) 390 : cluster [DBG] osdmap e92: 6 total, 5 up, 6 in 2024-02-15T18:31:15.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:14.744547+0000 mgr.smithi042.kfoihb (mgr.24435) 248 : cluster [DBG] pgmap v200: 65 pgs: 2 remapped+peering, 13 active+undersized+degraded, 2 activating+undersized+remapped, 17 active+undersized, 31 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 35/210 objects degraded (16.667%); 60/210 objects misplaced (28.571%) 2024-02-15T18:31:15.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:14.510625+0000 mon.smithi042 (mon.0) 390 : cluster [DBG] osdmap e92: 6 total, 5 up, 6 in 2024-02-15T18:31:15.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:14.744547+0000 mgr.smithi042.kfoihb (mgr.24435) 248 : cluster [DBG] pgmap v200: 65 pgs: 2 remapped+peering, 13 active+undersized+degraded, 2 activating+undersized+remapped, 17 active+undersized, 31 active+clean; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 35/210 objects degraded (16.667%); 60/210 objects misplaced (28.571%) 2024-02-15T18:31:16.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:15.519606+0000 mon.smithi042 (mon.0) 391 : cluster [DBG] osdmap e93: 6 total, 5 up, 6 in 2024-02-15T18:31:16.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:15.977293+0000 mon.smithi042 (mon.0) 392 : cluster [WRN] Health check update: Degraded data redundancy: 35/210 objects degraded (16.667%), 13 pgs degraded (PG_DEGRADED) 2024-02-15T18:31:16.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:15.519606+0000 mon.smithi042 (mon.0) 391 : cluster [DBG] osdmap e93: 6 total, 5 up, 6 in 2024-02-15T18:31:16.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:15.977293+0000 mon.smithi042 (mon.0) 392 : cluster [WRN] Health check update: Degraded data redundancy: 35/210 objects degraded (16.667%), 13 pgs degraded (PG_DEGRADED) 2024-02-15T18:31:17.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:16.520702+0000 mon.smithi042 (mon.0) 393 : cluster [DBG] osdmap e94: 6 total, 5 up, 6 in 2024-02-15T18:31:17.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:16.745065+0000 mgr.smithi042.kfoihb (mgr.24435) 249 : cluster [DBG] pgmap v203: 65 pgs: 29 active+clean, 1 active+recovery_wait+undersized+remapped, 2 active+recovering+undersized+remapped, 14 active+undersized, 1 activating+undersized+remapped, 13 active+undersized+degraded, 3 remapped+peering, 2 active+clean+remapped; 193 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 859 B/s rd, 1 op/s; 35/210 objects degraded (16.667%); 784/210 objects misplaced (373.333%); 0 B/s, 307 objects/s recovering 2024-02-15T18:31:17.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:17.099245+0000 mon.smithi042 (mon.0) 394 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:17.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:17.099924+0000 mon.smithi042 (mon.0) 395 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:31:17.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:17.100545+0000 mon.smithi042 (mon.0) 396 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:31:17.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:17.103801+0000 mon.smithi042 (mon.0) 397 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:17.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:17.108657+0000 mon.smithi042 (mon.0) 398 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:31:17.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:17.110669+0000 mon.smithi042 (mon.0) 399 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:31:17.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:17.111779+0000 mon.smithi042 (mon.0) 400 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:31:17.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:17.112833+0000 mon.smithi042 (mon.0) 401 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:31:17.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:17.114059+0000 mon.smithi042 (mon.0) 402 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:31:17.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:16.520702+0000 mon.smithi042 (mon.0) 393 : cluster [DBG] osdmap e94: 6 total, 5 up, 6 in 2024-02-15T18:31:17.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:16.745065+0000 mgr.smithi042.kfoihb (mgr.24435) 249 : cluster [DBG] pgmap v203: 65 pgs: 29 active+clean, 1 active+recovery_wait+undersized+remapped, 2 active+recovering+undersized+remapped, 14 active+undersized, 1 activating+undersized+remapped, 13 active+undersized+degraded, 3 remapped+peering, 2 active+clean+remapped; 193 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 859 B/s rd, 1 op/s; 35/210 objects degraded (16.667%); 784/210 objects misplaced (373.333%); 0 B/s, 307 objects/s recovering 2024-02-15T18:31:17.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:17.099245+0000 mon.smithi042 (mon.0) 394 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:17.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:17.099924+0000 mon.smithi042 (mon.0) 395 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:31:17.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:17.100545+0000 mon.smithi042 (mon.0) 396 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:31:17.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:17.103801+0000 mon.smithi042 (mon.0) 397 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:17.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:17.108657+0000 mon.smithi042 (mon.0) 398 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:31:17.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:17.110669+0000 mon.smithi042 (mon.0) 399 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:31:17.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:17.111779+0000 mon.smithi042 (mon.0) 400 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:31:17.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:17.112833+0000 mon.smithi042 (mon.0) 401 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:31:17.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:17.114059+0000 mon.smithi042 (mon.0) 402 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:31:18.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:31:17.101060+0000 mgr.smithi042.kfoihb (mgr.24435) 250 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:31:18.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:17.114205+0000 mgr.smithi042.kfoihb (mgr.24435) 251 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:31:18.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:31:17.114823+0000 mgr.smithi042.kfoihb (mgr.24435) 252 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-02-15T18:31:18.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:17.522147+0000 mon.smithi042 (mon.0) 403 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-15T18:31:18.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:31:17.101060+0000 mgr.smithi042.kfoihb (mgr.24435) 250 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:31:18.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:17.114205+0000 mgr.smithi042.kfoihb (mgr.24435) 251 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:31:18.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:31:17.114823+0000 mgr.smithi042.kfoihb (mgr.24435) 252 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-02-15T18:31:18.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:17.522147+0000 mon.smithi042 (mon.0) 403 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-15T18:31:19.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:18.533566+0000 mon.smithi042 (mon.0) 404 : cluster [DBG] osdmap e95: 6 total, 5 up, 6 in 2024-02-15T18:31:19.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:18.745461+0000 mgr.smithi042.kfoihb (mgr.24435) 253 : cluster [DBG] pgmap v205: 65 pgs: 28 active+clean, 1 active+recovery_wait+undersized+remapped, 2 active+recovering+undersized+remapped, 14 active+undersized, 13 active+undersized+degraded, 1 remapped+peering, 6 active+clean+remapped; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 966 B/s rd, 1 op/s; 33/210 objects degraded (15.714%); 754/210 objects misplaced (359.048%); 1.9 MiB/s, 360 objects/s recovering 2024-02-15T18:31:19.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:18.533566+0000 mon.smithi042 (mon.0) 404 : cluster [DBG] osdmap e95: 6 total, 5 up, 6 in 2024-02-15T18:31:19.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:18.745461+0000 mgr.smithi042.kfoihb (mgr.24435) 253 : cluster [DBG] pgmap v205: 65 pgs: 28 active+clean, 1 active+recovery_wait+undersized+remapped, 2 active+recovering+undersized+remapped, 14 active+undersized, 13 active+undersized+degraded, 1 remapped+peering, 6 active+clean+remapped; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 966 B/s rd, 1 op/s; 33/210 objects degraded (15.714%); 754/210 objects misplaced (359.048%); 1.9 MiB/s, 360 objects/s recovering 2024-02-15T18:31:20.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:20 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:19.537724+0000 mon.smithi042 (mon.0) 405 : cluster [DBG] osdmap e96: 6 total, 5 up, 6 in 2024-02-15T18:31:20.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:20 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:19.537724+0000 mon.smithi042 (mon.0) 405 : cluster [DBG] osdmap e96: 6 total, 5 up, 6 in 2024-02-15T18:31:21.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:20.542440+0000 mon.smithi042 (mon.0) 406 : cluster [DBG] osdmap e97: 6 total, 5 up, 6 in 2024-02-15T18:31:21.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:20.746115+0000 mgr.smithi042.kfoihb (mgr.24435) 254 : cluster [DBG] pgmap v208: 65 pgs: 1 remapped+peering, 28 active+clean, 1 active+recovering+undersized+remapped, 14 active+undersized, 13 active+undersized+degraded, 8 active+clean+remapped; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 33/210 objects degraded (15.714%); 7/210 objects misplaced (3.333%); 1.9 MiB/s, 67 objects/s recovering 2024-02-15T18:31:21.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:20.978152+0000 mon.smithi042 (mon.0) 407 : cluster [WRN] Health check update: Degraded data redundancy: 33/210 objects degraded (15.714%), 13 pgs degraded (PG_DEGRADED) 2024-02-15T18:31:21.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:20.542440+0000 mon.smithi042 (mon.0) 406 : cluster [DBG] osdmap e97: 6 total, 5 up, 6 in 2024-02-15T18:31:21.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:20.746115+0000 mgr.smithi042.kfoihb (mgr.24435) 254 : cluster [DBG] pgmap v208: 65 pgs: 1 remapped+peering, 28 active+clean, 1 active+recovering+undersized+remapped, 14 active+undersized, 13 active+undersized+degraded, 8 active+clean+remapped; 193 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 33/210 objects degraded (15.714%); 7/210 objects misplaced (3.333%); 1.9 MiB/s, 67 objects/s recovering 2024-02-15T18:31:21.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:20.978152+0000 mon.smithi042 (mon.0) 407 : cluster [WRN] Health check update: Degraded data redundancy: 33/210 objects degraded (15.714%), 13 pgs degraded (PG_DEGRADED) 2024-02-15T18:31:22.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:22.352687+0000 mon.smithi191 (mon.1) 8 : audit [INF] from='osd.4 [v2:172.21.15.191:6808/2781892647,v1:172.21.15.191:6809/2781892647]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:31:22.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:22.353245+0000 mon.smithi042 (mon.0) 408 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:31:22.694 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:31:22.348+0000 7f18df437380 -1 osd.4 86 log_to_monitors {default=true} 2024-02-15T18:31:22.694 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:31:22 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:31:22.560+0000 7f18ce119700 -1 osd.4 86 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-15T18:31:22.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:22.352687+0000 mon.smithi191 (mon.1) 8 : audit [INF] from='osd.4 [v2:172.21.15.191:6808/2781892647,v1:172.21.15.191:6809/2781892647]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:31:22.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:22 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:22.353245+0000 mon.smithi042 (mon.0) 408 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-15T18:31:23.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:22.552547+0000 mon.smithi042 (mon.0) 409 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-15T18:31:23.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:22.552619+0000 mon.smithi042 (mon.0) 410 : cluster [DBG] osdmap e98: 6 total, 5 up, 6 in 2024-02-15T18:31:23.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:22.554176+0000 mon.smithi191 (mon.1) 9 : audit [INF] from='osd.4 [v2:172.21.15.191:6808/2781892647,v1:172.21.15.191:6809/2781892647]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:31:23.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:22.554638+0000 mon.smithi042 (mon.0) 411 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:31:23.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:22.746566+0000 mgr.smithi042.kfoihb (mgr.24435) 255 : cluster [DBG] pgmap v210: 65 pgs: 1 remapped+peering, 28 active+clean, 14 active+undersized, 13 active+undersized+degraded, 9 active+clean+remapped; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 33/210 objects degraded (15.714%); 7/210 objects misplaced (3.333%); 0 B/s, 174 objects/s recovering 2024-02-15T18:31:23.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:22.552547+0000 mon.smithi042 (mon.0) 409 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-15T18:31:23.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:22.552619+0000 mon.smithi042 (mon.0) 410 : cluster [DBG] osdmap e98: 6 total, 5 up, 6 in 2024-02-15T18:31:23.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:22.554176+0000 mon.smithi191 (mon.1) 9 : audit [INF] from='osd.4 [v2:172.21.15.191:6808/2781892647,v1:172.21.15.191:6809/2781892647]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:31:23.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:22.554638+0000 mon.smithi042 (mon.0) 411 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:31:23.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:22.746566+0000 mgr.smithi042.kfoihb (mgr.24435) 255 : cluster [DBG] pgmap v210: 65 pgs: 1 remapped+peering, 28 active+clean, 14 active+undersized, 13 active+undersized+degraded, 9 active+clean+remapped; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 33/210 objects degraded (15.714%); 7/210 objects misplaced (3.333%); 0 B/s, 174 objects/s recovering 2024-02-15T18:31:24.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:23.550984+0000 mon.smithi042 (mon.0) 412 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:31:24.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:23.555383+0000 mon.smithi042 (mon.0) 413 : cluster [INF] osd.4 [v2:172.21.15.191:6808/2781892647,v1:172.21.15.191:6809/2781892647] boot 2024-02-15T18:31:24.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:23.555430+0000 mon.smithi042 (mon.0) 414 : cluster [DBG] osdmap e99: 6 total, 6 up, 6 in 2024-02-15T18:31:24.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:23.558034+0000 mon.smithi042 (mon.0) 415 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:31:24.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:24 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:24.556956+0000 mon.smithi042 (mon.0) 416 : cluster [DBG] osdmap e100: 6 total, 6 up, 6 in 2024-02-15T18:31:25.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:23.550984+0000 mon.smithi042 (mon.0) 412 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:31:25.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:23.555383+0000 mon.smithi042 (mon.0) 413 : cluster [INF] osd.4 [v2:172.21.15.191:6808/2781892647,v1:172.21.15.191:6809/2781892647] boot 2024-02-15T18:31:25.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:23.555430+0000 mon.smithi042 (mon.0) 414 : cluster [DBG] osdmap e99: 6 total, 6 up, 6 in 2024-02-15T18:31:25.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:23.558034+0000 mon.smithi042 (mon.0) 415 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-15T18:31:25.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:24 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:24.556956+0000 mon.smithi042 (mon.0) 416 : cluster [DBG] osdmap e100: 6 total, 6 up, 6 in 2024-02-15T18:31:25.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:24.747316+0000 mgr.smithi042.kfoihb (mgr.24435) 256 : cluster [DBG] pgmap v213: 65 pgs: 1 remapped+peering, 28 active+clean, 14 active+undersized, 13 active+undersized+degraded, 9 active+clean+remapped; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 33/210 objects degraded (15.714%); 7/210 objects misplaced (3.333%); 0 B/s, 122 objects/s recovering 2024-02-15T18:31:25.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:25.559035+0000 mon.smithi042 (mon.0) 417 : cluster [DBG] osdmap e101: 6 total, 6 up, 6 in 2024-02-15T18:31:26.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:24.747316+0000 mgr.smithi042.kfoihb (mgr.24435) 256 : cluster [DBG] pgmap v213: 65 pgs: 1 remapped+peering, 28 active+clean, 14 active+undersized, 13 active+undersized+degraded, 9 active+clean+remapped; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 33/210 objects degraded (15.714%); 7/210 objects misplaced (3.333%); 0 B/s, 122 objects/s recovering 2024-02-15T18:31:26.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:25.559035+0000 mon.smithi042 (mon.0) 417 : cluster [DBG] osdmap e101: 6 total, 6 up, 6 in 2024-02-15T18:31:26.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.910+0000 7f254c612700 1 -- 172.21.15.42:0/3526897316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25440f55c0 msgr2=0x7f25440f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:26.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.911+0000 7f254c612700 1 --2- 172.21.15.42:0/3526897316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25440f55c0 0x7f25440f59e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f2540004660 tx=0x7f2540009b30).stop 2024-02-15T18:31:26.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.911+0000 7f254c612700 1 -- 172.21.15.42:0/3526897316 shutdown_connections 2024-02-15T18:31:26.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.911+0000 7f254c612700 1 --2- 172.21.15.42:0/3526897316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25440f6860 0x7f25440f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:26.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.911+0000 7f254c612700 1 --2- 172.21.15.42:0/3526897316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25440f55c0 0x7f25440f59e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:26.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.911+0000 7f254c612700 1 -- 172.21.15.42:0/3526897316 >> 172.21.15.42:0/3526897316 conn(0x7f25440f0d40 msgr2=0x7f25440f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:26.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.911+0000 7f254c612700 1 -- 172.21.15.42:0/3526897316 shutdown_connections 2024-02-15T18:31:26.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.911+0000 7f254c612700 1 -- 172.21.15.42:0/3526897316 wait complete. 2024-02-15T18:31:26.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.911+0000 7f254c612700 1 Processor -- start 2024-02-15T18:31:26.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.911+0000 7f254c612700 1 -- start start 2024-02-15T18:31:26.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.911+0000 7f254c612700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25440f55c0 0x7f25440f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:26.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.911+0000 7f254c612700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25440f6860 0x7f25440f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:26.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.911+0000 7f254c612700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25440f8100 con 0x7f25440f55c0 2024-02-15T18:31:26.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.911+0000 7f254c612700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25440f8240 con 0x7f25440f6860 2024-02-15T18:31:26.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.912+0000 7f254a3ae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25440f55c0 0x7f25440f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:26.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.912+0000 7f254a3ae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25440f55c0 0x7f25440f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34290/0 (socket says 172.21.15.42:34290) 2024-02-15T18:31:26.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.912+0000 7f254a3ae700 1 -- 172.21.15.42:0/3158729593 learned_addr learned my addr 172.21.15.42:0/3158729593 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:31:26.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.912+0000 7f2549bad700 1 --2- 172.21.15.42:0/3158729593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25440f6860 0x7f25440f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:26.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.912+0000 7f254a3ae700 1 -- 172.21.15.42:0/3158729593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25440f6860 msgr2=0x7f25440f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:26.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.912+0000 7f254a3ae700 1 --2- 172.21.15.42:0/3158729593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25440f6860 0x7f25440f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:26.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.912+0000 7f254a3ae700 1 -- 172.21.15.42:0/3158729593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2540005040 con 0x7f25440f55c0 2024-02-15T18:31:26.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.912+0000 7f254a3ae700 1 --2- 172.21.15.42:0/3158729593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25440f55c0 0x7f25440f9720 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f2540002040 tx=0x7f254000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:26.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.912+0000 7f253affd700 1 -- 172.21.15.42:0/3158729593 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2540019070 con 0x7f25440f55c0 2024-02-15T18:31:26.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.912+0000 7f253affd700 1 -- 172.21.15.42:0/3158729593 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f2540007e80 con 0x7f25440f55c0 2024-02-15T18:31:26.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.912+0000 7f254c612700 1 -- 172.21.15.42:0/3158729593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25440f84c0 con 0x7f25440f55c0 2024-02-15T18:31:26.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.912+0000 7f253affd700 1 -- 172.21.15.42:0/3158729593 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2540014db0 con 0x7f25440f55c0 2024-02-15T18:31:26.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.912+0000 7f254c612700 1 -- 172.21.15.42:0/3158729593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25440f8a20 con 0x7f25440f55c0 2024-02-15T18:31:26.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.913+0000 7f254c612700 1 -- 172.21.15.42:0/3158729593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2544045ad0 con 0x7f25440f55c0 2024-02-15T18:31:26.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.914+0000 7f253affd700 1 -- 172.21.15.42:0/3158729593 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f2540025070 con 0x7f25440f55c0 2024-02-15T18:31:26.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.914+0000 7f253affd700 1 --2- 172.21.15.42:0/3158729593 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2530060ec0 0x7f2530063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:26.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.914+0000 7f253affd700 1 -- 172.21.15.42:0/3158729593 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6131+0+0 (secure 0 0 0) 0x7f254007d1f0 con 0x7f25440f55c0 2024-02-15T18:31:26.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.914+0000 7f2549bad700 1 --2- 172.21.15.42:0/3158729593 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2530060ec0 0x7f2530063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:26.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.915+0000 7f2549bad700 1 --2- 172.21.15.42:0/3158729593 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2530060ec0 0x7f2530063380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f2534009fd0 tx=0x7f253400b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:26.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:26.919+0000 7f253affd700 1 -- 172.21.15.42:0/3158729593 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f254004d410 con 0x7f25440f55c0 2024-02-15T18:31:27.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.116+0000 7f254c612700 1 -- 172.21.15.42:0/3158729593 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f25440faca0 con 0x7f2530060ec0 2024-02-15T18:31:27.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.117+0000 7f253affd700 1 -- 172.21.15.42:0/3158729593 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f25440faca0 con 0x7f2530060ec0 2024-02-15T18:31:27.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.119+0000 7f254c612700 1 -- 172.21.15.42:0/3158729593 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2530060ec0 msgr2=0x7f2530063380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:27.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.119+0000 7f254c612700 1 --2- 172.21.15.42:0/3158729593 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2530060ec0 0x7f2530063380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f2534009fd0 tx=0x7f253400b040).stop 2024-02-15T18:31:27.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.119+0000 7f254c612700 1 -- 172.21.15.42:0/3158729593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25440f55c0 msgr2=0x7f25440f9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:27.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.119+0000 7f254c612700 1 --2- 172.21.15.42:0/3158729593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25440f55c0 0x7f25440f9720 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f2540002040 tx=0x7f254000b040).stop 2024-02-15T18:31:27.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.119+0000 7f254c612700 1 -- 172.21.15.42:0/3158729593 shutdown_connections 2024-02-15T18:31:27.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.119+0000 7f254c612700 1 --2- 172.21.15.42:0/3158729593 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2530060ec0 0x7f2530063380 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.119+0000 7f254c612700 1 --2- 172.21.15.42:0/3158729593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25440f55c0 0x7f25440f9720 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.119+0000 7f254c612700 1 --2- 172.21.15.42:0/3158729593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25440f6860 0x7f25440f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.119+0000 7f254c612700 1 -- 172.21.15.42:0/3158729593 >> 172.21.15.42:0/3158729593 conn(0x7f25440f0d40 msgr2=0x7f254405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:27.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.119+0000 7f254c612700 1 -- 172.21.15.42:0/3158729593 shutdown_connections 2024-02-15T18:31:27.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.119+0000 7f254c612700 1 -- 172.21.15.42:0/3158729593 wait complete. 2024-02-15T18:31:27.123 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:31:27.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:25.836712+0000 mon.smithi042 (mon.0) 418 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:27.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:25.982845+0000 mon.smithi042 (mon.0) 419 : cluster [DBG] osdmap e102: 6 total, 6 up, 6 in 2024-02-15T18:31:27.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.221+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/3611616278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f780f77d0 msgr2=0x7f3f780f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:27.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.221+0000 7f3f7f0fc700 1 --2- 172.21.15.42:0/3611616278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f780f77d0 0x7f3f780f7c30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f3f68004660 tx=0x7f3f68009b30).stop 2024-02-15T18:31:27.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.221+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/3611616278 shutdown_connections 2024-02-15T18:31:27.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.221+0000 7f3f7f0fc700 1 --2- 172.21.15.42:0/3611616278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f780f77d0 0x7f3f780f7c30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.221+0000 7f3f7f0fc700 1 --2- 172.21.15.42:0/3611616278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f780f6530 0x7f3f780f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.221+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/3611616278 >> 172.21.15.42:0/3611616278 conn(0x7f3f780f1c90 msgr2=0x7f3f780f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.221+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/3611616278 shutdown_connections 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.222+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/3611616278 wait complete. 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.222+0000 7f3f7f0fc700 1 Processor -- start 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.222+0000 7f3f7f0fc700 1 -- start start 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.222+0000 7f3f7f0fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f780f6530 0x7f3f7806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.222+0000 7f3f7f0fc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f780f77d0 0x7f3f78069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.222+0000 7f3f7f0fc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7806a380 con 0x7f3f780f6530 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.222+0000 7f3f7f0fc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7806a4c0 con 0x7f3f780f77d0 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.222+0000 7f3f7e0fa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f780f6530 0x7f3f7806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.222+0000 7f3f7e0fa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f780f6530 0x7f3f7806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34314/0 (socket says 172.21.15.42:34314) 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.222+0000 7f3f7e0fa700 1 -- 172.21.15.42:0/153240469 learned_addr learned my addr 172.21.15.42:0/153240469 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.222+0000 7f3f7d8f9700 1 --2- 172.21.15.42:0/153240469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f780f77d0 0x7f3f78069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.223+0000 7f3f7e0fa700 1 -- 172.21.15.42:0/153240469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f780f77d0 msgr2=0x7f3f78069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:27.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.223+0000 7f3f7e0fa700 1 --2- 172.21.15.42:0/153240469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f780f77d0 0x7f3f78069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.223+0000 7f3f7e0fa700 1 -- 172.21.15.42:0/153240469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f68005040 con 0x7f3f780f6530 2024-02-15T18:31:27.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.223+0000 7f3f7e0fa700 1 --2- 172.21.15.42:0/153240469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f780f6530 0x7f3f7806b9a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f3f7400e7c0 tx=0x7f3f7400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:27.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.223+0000 7f3f6effd700 1 -- 172.21.15.42:0/153240469 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f740092e0 con 0x7f3f780f6530 2024-02-15T18:31:27.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.223+0000 7f3f6effd700 1 -- 172.21.15.42:0/153240469 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f3f7400f040 con 0x7f3f780f6530 2024-02-15T18:31:27.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.223+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/153240469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7806a7a0 con 0x7f3f780f6530 2024-02-15T18:31:27.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.223+0000 7f3f6effd700 1 -- 172.21.15.42:0/153240469 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f74014920 con 0x7f3f780f6530 2024-02-15T18:31:27.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.223+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/153240469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7810ede0 con 0x7f3f780f6530 2024-02-15T18:31:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.224+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/153240469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f78045ad0 con 0x7f3f780f6530 2024-02-15T18:31:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.224+0000 7f3f6effd700 1 -- 172.21.15.42:0/153240469 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f3f7401b030 con 0x7f3f780f6530 2024-02-15T18:31:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.225+0000 7f3f6effd700 1 --2- 172.21.15.42:0/153240469 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3f64060f70 0x7f3f64063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.225+0000 7f3f6effd700 1 -- 172.21.15.42:0/153240469 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6131+0+0 (secure 0 0 0) 0x7f3f7407bf10 con 0x7f3f780f6530 2024-02-15T18:31:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.225+0000 7f3f7d8f9700 1 --2- 172.21.15.42:0/153240469 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3f64060f70 0x7f3f64063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:27.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.225+0000 7f3f7d8f9700 1 --2- 172.21.15.42:0/153240469 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3f64060f70 0x7f3f64063430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f3f68004630 tx=0x7f3f68016040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:27.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.229+0000 7f3f6effd700 1 -- 172.21.15.42:0/153240469 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3f7404c380 con 0x7f3f780f6530 2024-02-15T18:31:27.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:25.836712+0000 mon.smithi042 (mon.0) 418 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:27.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:25.982845+0000 mon.smithi042 (mon.0) 419 : cluster [DBG] osdmap e102: 6 total, 6 up, 6 in 2024-02-15T18:31:27.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.419+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/153240469 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f780fefb0 con 0x7f3f64060f70 2024-02-15T18:31:27.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.421+0000 7f3f6effd700 1 -- 172.21.15.42:0/153240469 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f3f6800f070 con 0x7f3f64060f70 2024-02-15T18:31:27.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.422+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/153240469 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3f64060f70 msgr2=0x7f3f64063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:27.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.422+0000 7f3f7f0fc700 1 --2- 172.21.15.42:0/153240469 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3f64060f70 0x7f3f64063430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f3f68004630 tx=0x7f3f68016040).stop 2024-02-15T18:31:27.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.422+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/153240469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f780f6530 msgr2=0x7f3f7806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:27.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.422+0000 7f3f7f0fc700 1 --2- 172.21.15.42:0/153240469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f780f6530 0x7f3f7806b9a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f3f7400e7c0 tx=0x7f3f7400b040).stop 2024-02-15T18:31:27.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.423+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/153240469 shutdown_connections 2024-02-15T18:31:27.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.423+0000 7f3f7f0fc700 1 --2- 172.21.15.42:0/153240469 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3f64060f70 0x7f3f64063430 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.423+0000 7f3f7f0fc700 1 --2- 172.21.15.42:0/153240469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f780f6530 0x7f3f7806b9a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.423+0000 7f3f7f0fc700 1 --2- 172.21.15.42:0/153240469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f780f77d0 0x7f3f78069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.423+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/153240469 >> 172.21.15.42:0/153240469 conn(0x7f3f780f1c90 msgr2=0x7f3f780fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:27.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.423+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/153240469 shutdown_connections 2024-02-15T18:31:27.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.423+0000 7f3f7f0fc700 1 -- 172.21.15.42:0/153240469 wait complete. 2024-02-15T18:31:27.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.525+0000 7f2f748bc700 1 -- 172.21.15.42:0/1832187655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f700f49a0 msgr2=0x7f2f700f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:27.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.525+0000 7f2f748bc700 1 --2- 172.21.15.42:0/1832187655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f700f49a0 0x7f2f700f4dc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f2f60004660 tx=0x7f2f60009b30).stop 2024-02-15T18:31:27.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.525+0000 7f2f748bc700 1 -- 172.21.15.42:0/1832187655 shutdown_connections 2024-02-15T18:31:27.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.525+0000 7f2f748bc700 1 --2- 172.21.15.42:0/1832187655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f700f5340 0x7f2f700f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.525+0000 7f2f748bc700 1 --2- 172.21.15.42:0/1832187655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f700f49a0 0x7f2f700f4dc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.525+0000 7f2f748bc700 1 -- 172.21.15.42:0/1832187655 >> 172.21.15.42:0/1832187655 conn(0x7f2f700f04c0 msgr2=0x7f2f700f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:27.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.525+0000 7f2f748bc700 1 -- 172.21.15.42:0/1832187655 shutdown_connections 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.525+0000 7f2f748bc700 1 -- 172.21.15.42:0/1832187655 wait complete. 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.526+0000 7f2f748bc700 1 Processor -- start 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.526+0000 7f2f748bc700 1 -- start start 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.526+0000 7f2f748bc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f700f49a0 0x7f2f701070f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.526+0000 7f2f748bc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f700f5340 0x7f2f701075e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.526+0000 7f2f748bc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f70107b90 con 0x7f2f700f49a0 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.526+0000 7f2f748bc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f70107cd0 con 0x7f2f700f5340 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.526+0000 7f2f6e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f700f49a0 0x7f2f701070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.526+0000 7f2f6e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f700f49a0 0x7f2f701070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34324/0 (socket says 172.21.15.42:34324) 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.526+0000 7f2f6e59c700 1 -- 172.21.15.42:0/2386329826 learned_addr learned my addr 172.21.15.42:0/2386329826 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.526+0000 7f2f6dd9b700 1 --2- 172.21.15.42:0/2386329826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f700f5340 0x7f2f701075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.526+0000 7f2f6e59c700 1 -- 172.21.15.42:0/2386329826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f700f5340 msgr2=0x7f2f701075e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.526+0000 7f2f6e59c700 1 --2- 172.21.15.42:0/2386329826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f700f5340 0x7f2f701075e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.526+0000 7f2f6e59c700 1 -- 172.21.15.42:0/2386329826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f58005040 con 0x7f2f700f49a0 2024-02-15T18:31:27.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.527+0000 7f2f6e59c700 1 --2- 172.21.15.42:0/2386329826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f700f49a0 0x7f2f701070f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f2f600043d0 tx=0x7f2f6000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:27.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.527+0000 7f2f66ffd700 1 -- 172.21.15.42:0/2386329826 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f60019070 con 0x7f2f700f49a0 2024-02-15T18:31:27.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.527+0000 7f2f748bc700 1 -- 172.21.15.42:0/2386329826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f60005040 con 0x7f2f700f49a0 2024-02-15T18:31:27.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.527+0000 7f2f66ffd700 1 -- 172.21.15.42:0/2386329826 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f2f60007e80 con 0x7f2f700f49a0 2024-02-15T18:31:27.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.527+0000 7f2f66ffd700 1 -- 172.21.15.42:0/2386329826 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f60014cf0 con 0x7f2f700f49a0 2024-02-15T18:31:27.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.527+0000 7f2f748bc700 1 -- 172.21.15.42:0/2386329826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f701089e0 con 0x7f2f700f49a0 2024-02-15T18:31:27.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.528+0000 7f2f748bc700 1 -- 172.21.15.42:0/2386329826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f70045ad0 con 0x7f2f700f49a0 2024-02-15T18:31:27.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.528+0000 7f2f66ffd700 1 -- 172.21.15.42:0/2386329826 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f2f60025070 con 0x7f2f700f49a0 2024-02-15T18:31:27.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.528+0000 7f2f66ffd700 1 --2- 172.21.15.42:0/2386329826 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2f5c060f20 0x7f2f5c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:27.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.529+0000 7f2f66ffd700 1 -- 172.21.15.42:0/2386329826 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6131+0+0 (secure 0 0 0) 0x7f2f6007d1b0 con 0x7f2f700f49a0 2024-02-15T18:31:27.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.529+0000 7f2f6dd9b700 1 --2- 172.21.15.42:0/2386329826 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2f5c060f20 0x7f2f5c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:27.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.529+0000 7f2f6dd9b700 1 --2- 172.21.15.42:0/2386329826 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2f5c060f20 0x7f2f5c0633e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f2f700f5780 tx=0x7f2f5800b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:27.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.533+0000 7f2f66ffd700 1 -- 172.21.15.42:0/2386329826 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2f60050f20 con 0x7f2f700f49a0 2024-02-15T18:31:27.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.714+0000 7f2f748bc700 1 -- 172.21.15.42:0/2386329826 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f70100f80 con 0x7f2f5c060f20 2024-02-15T18:31:27.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.724+0000 7f2f66ffd700 1 -- 172.21.15.42:0/2386329826 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f2f70100f80 con 0x7f2f5c060f20 2024-02-15T18:31:27.717 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:31:27.717 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (11m) 83s ago 13m 21.4M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:31:27.717 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (3m) 83s ago 13m 6643k - 16.2.14-521-g0e714d9a 1fbc92c859dc 2da9d875539a 2024-02-15T18:31:27.717 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (3m) 10s ago 11m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 5c46a68ff62b 2024-02-15T18:31:27.717 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (11m) 83s ago 12m 33.7M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:31:27.717 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (7m) 83s ago 7m 14.9M - 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (7m) 83s ago 7m 17.5M - 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (7m) 10s ago 7m 110M - 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (7m) 10s ago 7m 153M - 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (5m) 83s ago 14m 435M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (5m) 10s ago 11m 389M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (4m) 83s ago 14m 46.9M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (4m) 10s ago 11m 42.8M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (12m) 83s ago 12m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (11m) 10s ago 11m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (3m) 83s ago 11m 250M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc f226bea76cab 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (2m) 83s ago 10m 264M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 5114ed026724 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (89s) 83s ago 10m 11.6M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 2e26b1ec292a 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (58s) 10s ago 10m 328M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc c71b210091cb 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (15s) 10s ago 9m 13.0M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 83bd9cdcb3da 2024-02-15T18:31:27.718 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (9m) 10s ago 9m 407M 4096M 16.2.4 8d91d370c2b8 e16734588229 2024-02-15T18:31:27.719 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (5m) 83s ago 12m 61.0M - 2.18.1 de242295e225 63b6dfb825ed 2024-02-15T18:31:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.725+0000 7f2f748bc700 1 -- 172.21.15.42:0/2386329826 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2f5c060f20 msgr2=0x7f2f5c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.725+0000 7f2f748bc700 1 --2- 172.21.15.42:0/2386329826 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2f5c060f20 0x7f2f5c0633e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f2f700f5780 tx=0x7f2f5800b040).stop 2024-02-15T18:31:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.725+0000 7f2f748bc700 1 -- 172.21.15.42:0/2386329826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f700f49a0 msgr2=0x7f2f701070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.725+0000 7f2f748bc700 1 --2- 172.21.15.42:0/2386329826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f700f49a0 0x7f2f701070f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f2f600043d0 tx=0x7f2f6000b040).stop 2024-02-15T18:31:27.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.726+0000 7f2f748bc700 1 -- 172.21.15.42:0/2386329826 shutdown_connections 2024-02-15T18:31:27.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.726+0000 7f2f748bc700 1 --2- 172.21.15.42:0/2386329826 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2f5c060f20 0x7f2f5c0633e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.726+0000 7f2f748bc700 1 --2- 172.21.15.42:0/2386329826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f700f49a0 0x7f2f701070f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.726+0000 7f2f748bc700 1 --2- 172.21.15.42:0/2386329826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f700f5340 0x7f2f701075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.726+0000 7f2f748bc700 1 -- 172.21.15.42:0/2386329826 >> 172.21.15.42:0/2386329826 conn(0x7f2f700f04c0 msgr2=0x7f2f700fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:27.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.726+0000 7f2f748bc700 1 -- 172.21.15.42:0/2386329826 shutdown_connections 2024-02-15T18:31:27.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.726+0000 7f2f748bc700 1 -- 172.21.15.42:0/2386329826 wait complete. 2024-02-15T18:31:27.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.825+0000 7fc3b359e700 1 -- 172.21.15.42:0/3370462403 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b4060250 msgr2=0x7fc3b40f91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:27.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.825+0000 7fc3b359e700 1 --2- 172.21.15.42:0/3370462403 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b4060250 0x7fc3b40f91d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fc3a4004660 tx=0x7fc3a4009b30).stop 2024-02-15T18:31:27.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.825+0000 7fc3b359e700 1 -- 172.21.15.42:0/3370462403 shutdown_connections 2024-02-15T18:31:27.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.825+0000 7fc3b359e700 1 --2- 172.21.15.42:0/3370462403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3b40f96c0 0x7fc3b40fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.825+0000 7fc3b359e700 1 --2- 172.21.15.42:0/3370462403 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b4060250 0x7fc3b40f91d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.825+0000 7fc3b359e700 1 -- 172.21.15.42:0/3370462403 >> 172.21.15.42:0/3370462403 conn(0x7fc3b40f0b90 msgr2=0x7fc3b40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.825+0000 7fc3b359e700 1 -- 172.21.15.42:0/3370462403 shutdown_connections 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.826+0000 7fc3b359e700 1 -- 172.21.15.42:0/3370462403 wait complete. 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.826+0000 7fc3b359e700 1 Processor -- start 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.826+0000 7fc3b359e700 1 -- start start 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.826+0000 7fc3b359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3b4060250 0x7fc3b40f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.826+0000 7fc3b359e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b40f96c0 0x7fc3b40f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.826+0000 7fc3b359e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40f5630 con 0x7fc3b40f96c0 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.826+0000 7fc3b359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40f5770 con 0x7fc3b4060250 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.826+0000 7fc3b1d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b40f96c0 0x7fc3b40f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.826+0000 7fc3b1d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b40f96c0 0x7fc3b40f50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34352/0 (socket says 172.21.15.42:34352) 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.826+0000 7fc3b1d9b700 1 -- 172.21.15.42:0/3550654591 learned_addr learned my addr 172.21.15.42:0/3550654591 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.827+0000 7fc3b259c700 1 --2- 172.21.15.42:0/3550654591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3b4060250 0x7fc3b40f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.827+0000 7fc3b1d9b700 1 -- 172.21.15.42:0/3550654591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3b4060250 msgr2=0x7fc3b40f6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.827+0000 7fc3b1d9b700 1 --2- 172.21.15.42:0/3550654591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3b4060250 0x7fc3b40f6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:27.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.827+0000 7fc3b1d9b700 1 -- 172.21.15.42:0/3550654591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a4005040 con 0x7fc3b40f96c0 2024-02-15T18:31:27.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.827+0000 7fc3b1d9b700 1 --2- 172.21.15.42:0/3550654591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b40f96c0 0x7fc3b40f50d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fc3a8002700 tx=0x7fc3a8009da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:27.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.827+0000 7fc3a2ffd700 1 -- 172.21.15.42:0/3550654591 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3a80079c0 con 0x7fc3b40f96c0 2024-02-15T18:31:27.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.827+0000 7fc3a2ffd700 1 -- 172.21.15.42:0/3550654591 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fc3a800f040 con 0x7fc3b40f96c0 2024-02-15T18:31:27.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.827+0000 7fc3a2ffd700 1 -- 172.21.15.42:0/3550654591 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3a80149c0 con 0x7fc3b40f96c0 2024-02-15T18:31:27.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.827+0000 7fc3b359e700 1 -- 172.21.15.42:0/3550654591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b40f5a50 con 0x7fc3b40f96c0 2024-02-15T18:31:27.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.827+0000 7fc3b359e700 1 -- 172.21.15.42:0/3550654591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b410ee40 con 0x7fc3b40f96c0 2024-02-15T18:31:27.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.828+0000 7fc3b359e700 1 -- 172.21.15.42:0/3550654591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b4045ad0 con 0x7fc3b40f96c0 2024-02-15T18:31:27.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.828+0000 7fc3a2ffd700 1 -- 172.21.15.42:0/3550654591 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fc3a801b060 con 0x7fc3b40f96c0 2024-02-15T18:31:27.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.829+0000 7fc3a2ffd700 1 --2- 172.21.15.42:0/3550654591 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc39c060f70 0x7fc39c063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:27.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.829+0000 7fc3a2ffd700 1 -- 172.21.15.42:0/3550654591 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6131+0+0 (secure 0 0 0) 0x7fc3a807c0d0 con 0x7fc3b40f96c0 2024-02-15T18:31:27.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.829+0000 7fc3b259c700 1 --2- 172.21.15.42:0/3550654591 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc39c060f70 0x7fc39c063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:27.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.829+0000 7fc3b259c700 1 --2- 172.21.15.42:0/3550654591 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc39c060f70 0x7fc39c063430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fc3a4004630 tx=0x7fc3a4009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:27.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:27.833+0000 7fc3a2ffd700 1 -- 172.21.15.42:0/3550654591 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc3a804c410 con 0x7fc3b40f96c0 2024-02-15T18:31:28.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.085+0000 7fc3b359e700 1 -- 172.21.15.42:0/3550654591 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc3b4027a00 con 0x7fc3b40f96c0 2024-02-15T18:31:28.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.085+0000 7fc3a2ffd700 1 -- 172.21.15.42:0/3550654591 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+820 (secure 0 0 0) 0x7fc3a804fa30 con 0x7fc3b40f96c0 2024-02-15T18:31:28.079 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:31:28.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:31:28.079 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:31:28.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:31:28.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:31:28.079 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:31:28.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:31:28.079 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:31:28.079 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 5, 2024-02-15T18:31:28.079 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 1 2024-02-15T18:31:28.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:31:28.079 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:31:28.080 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:31:28.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:31:28.080 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:31:28.080 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 9, 2024-02-15T18:31:28.080 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 5 2024-02-15T18:31:28.080 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:31:28.080 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:31:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.087+0000 7fc3b359e700 1 -- 172.21.15.42:0/3550654591 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc39c060f70 msgr2=0x7fc39c063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.087+0000 7fc3b359e700 1 --2- 172.21.15.42:0/3550654591 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc39c060f70 0x7fc39c063430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fc3a4004630 tx=0x7fc3a4009b30).stop 2024-02-15T18:31:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.087+0000 7fc3b359e700 1 -- 172.21.15.42:0/3550654591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b40f96c0 msgr2=0x7fc3b40f50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.087+0000 7fc3b359e700 1 --2- 172.21.15.42:0/3550654591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b40f96c0 0x7fc3b40f50d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fc3a8002700 tx=0x7fc3a8009da0).stop 2024-02-15T18:31:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.087+0000 7fc3b359e700 1 -- 172.21.15.42:0/3550654591 shutdown_connections 2024-02-15T18:31:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.087+0000 7fc3b359e700 1 --2- 172.21.15.42:0/3550654591 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc39c060f70 0x7fc39c063430 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.087+0000 7fc3b359e700 1 --2- 172.21.15.42:0/3550654591 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3b4060250 0x7fc3b40f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.087+0000 7fc3b359e700 1 --2- 172.21.15.42:0/3550654591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b40f96c0 0x7fc3b40f50d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.087+0000 7fc3b359e700 1 -- 172.21.15.42:0/3550654591 >> 172.21.15.42:0/3550654591 conn(0x7fc3b40f0b90 msgr2=0x7fc3b40f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.087+0000 7fc3b359e700 1 -- 172.21.15.42:0/3550654591 shutdown_connections 2024-02-15T18:31:28.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.087+0000 7fc3b359e700 1 -- 172.21.15.42:0/3550654591 wait complete. 2024-02-15T18:31:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.189+0000 7f9f25e0f700 1 -- 172.21.15.42:0/2137514799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f200f6860 msgr2=0x7f9f200f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.189+0000 7f9f25e0f700 1 --2- 172.21.15.42:0/2137514799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f200f6860 0x7f9f200f6cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f9f14004660 tx=0x7f9f14009b30).stop 2024-02-15T18:31:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.189+0000 7f9f25e0f700 1 -- 172.21.15.42:0/2137514799 shutdown_connections 2024-02-15T18:31:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.189+0000 7f9f25e0f700 1 --2- 172.21.15.42:0/2137514799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f200f6860 0x7f9f200f6cc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.189+0000 7f9f25e0f700 1 --2- 172.21.15.42:0/2137514799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f200f55c0 0x7f9f200f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.189+0000 7f9f25e0f700 1 -- 172.21.15.42:0/2137514799 >> 172.21.15.42:0/2137514799 conn(0x7f9f200f0d40 msgr2=0x7f9f200f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.189+0000 7f9f25e0f700 1 -- 172.21.15.42:0/2137514799 shutdown_connections 2024-02-15T18:31:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.189+0000 7f9f25e0f700 1 -- 172.21.15.42:0/2137514799 wait complete. 2024-02-15T18:31:28.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.190+0000 7f9f25e0f700 1 Processor -- start 2024-02-15T18:31:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.190+0000 7f9f25e0f700 1 -- start start 2024-02-15T18:31:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.190+0000 7f9f25e0f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f200f55c0 0x7f9f20109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.190+0000 7f9f25e0f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f200f6860 0x7f9f2010c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.190+0000 7f9f25e0f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200603c0 con 0x7f9f200f55c0 2024-02-15T18:31:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.190+0000 7f9f25e0f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20060500 con 0x7f9f200f6860 2024-02-15T18:31:28.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.190+0000 7f9f1f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f200f55c0 0x7f9f20109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.190+0000 7f9f1f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f200f55c0 0x7f9f20109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34376/0 (socket says 172.21.15.42:34376) 2024-02-15T18:31:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.190+0000 7f9f1f7fe700 1 -- 172.21.15.42:0/1898023452 learned_addr learned my addr 172.21.15.42:0/1898023452 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:31:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.190+0000 7f9f1effd700 1 --2- 172.21.15.42:0/1898023452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f200f6860 0x7f9f2010c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.191+0000 7f9f1f7fe700 1 -- 172.21.15.42:0/1898023452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f200f6860 msgr2=0x7f9f2010c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.191+0000 7f9f1f7fe700 1 --2- 172.21.15.42:0/1898023452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f200f6860 0x7f9f2010c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.191+0000 7f9f1f7fe700 1 -- 172.21.15.42:0/1898023452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f14005040 con 0x7f9f200f55c0 2024-02-15T18:31:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.191+0000 7f9f1f7fe700 1 --2- 172.21.15.42:0/1898023452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f200f55c0 0x7f9f20109cd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f9f1000e7c0 tx=0x7f9f1000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.191+0000 7f9f07fff700 1 -- 172.21.15.42:0/1898023452 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f100092e0 con 0x7f9f200f55c0 2024-02-15T18:31:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.191+0000 7f9f07fff700 1 -- 172.21.15.42:0/1898023452 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f9f1000f040 con 0x7f9f200f55c0 2024-02-15T18:31:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.191+0000 7f9f25e0f700 1 -- 172.21.15.42:0/1898023452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2010c780 con 0x7f9f200f55c0 2024-02-15T18:31:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.191+0000 7f9f07fff700 1 -- 172.21.15.42:0/1898023452 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f10014920 con 0x7f9f200f55c0 2024-02-15T18:31:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.191+0000 7f9f25e0f700 1 -- 172.21.15.42:0/1898023452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2010cd10 con 0x7f9f200f55c0 2024-02-15T18:31:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.192+0000 7f9f25e0f700 1 -- 172.21.15.42:0/1898023452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f20045ad0 con 0x7f9f200f55c0 2024-02-15T18:31:28.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.193+0000 7f9f07fff700 1 -- 172.21.15.42:0/1898023452 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f9f1001b030 con 0x7f9f200f55c0 2024-02-15T18:31:28.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.193+0000 7f9f07fff700 1 --2- 172.21.15.42:0/1898023452 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9f08060f20 0x7f9f080633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:28.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.193+0000 7f9f07fff700 1 -- 172.21.15.42:0/1898023452 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6102+0+0 (secure 0 0 0) 0x7f9f1007c160 con 0x7f9f200f55c0 2024-02-15T18:31:28.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.193+0000 7f9f1effd700 1 --2- 172.21.15.42:0/1898023452 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9f08060f20 0x7f9f080633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:28.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.193+0000 7f9f1effd700 1 --2- 172.21.15.42:0/1898023452 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9f08060f20 0x7f9f080633e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f9f14009720 tx=0x7f9f14016040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:28.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.197+0000 7f9f07fff700 1 -- 172.21.15.42:0/1898023452 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9f1004c4c0 con 0x7f9f200f55c0 2024-02-15T18:31:28.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:26.747876+0000 mgr.smithi042.kfoihb (mgr.24435) 257 : cluster [DBG] pgmap v216: 65 pgs: 2 peering, 1 remapped+peering, 48 active+clean, 1 active+recovering+undersized+remapped, 5 active+undersized, 5 active+undersized+degraded, 3 active+clean+remapped; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 7/210 objects degraded (3.333%); 3/210 objects misplaced (1.429%); 0 B/s, 189 objects/s recovering 2024-02-15T18:31:28.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:26.984936+0000 mon.smithi042 (mon.0) 420 : cluster [DBG] osdmap e103: 6 total, 6 up, 6 in 2024-02-15T18:31:28.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:26.991030+0000 mon.smithi042 (mon.0) 421 : cluster [WRN] Health check update: Degraded data redundancy: 7/210 objects degraded (3.333%), 5 pgs degraded (PG_DEGRADED) 2024-02-15T18:31:28.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:26.991054+0000 mon.smithi042 (mon.0) 422 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-15T18:31:28.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:26.747876+0000 mgr.smithi042.kfoihb (mgr.24435) 257 : cluster [DBG] pgmap v216: 65 pgs: 2 peering, 1 remapped+peering, 48 active+clean, 1 active+recovering+undersized+remapped, 5 active+undersized, 5 active+undersized+degraded, 3 active+clean+remapped; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 7/210 objects degraded (3.333%); 3/210 objects misplaced (1.429%); 0 B/s, 189 objects/s recovering 2024-02-15T18:31:28.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:26.984936+0000 mon.smithi042 (mon.0) 420 : cluster [DBG] osdmap e103: 6 total, 6 up, 6 in 2024-02-15T18:31:28.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:26.991030+0000 mon.smithi042 (mon.0) 421 : cluster [WRN] Health check update: Degraded data redundancy: 7/210 objects degraded (3.333%), 5 pgs degraded (PG_DEGRADED) 2024-02-15T18:31:28.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:26.991054+0000 mon.smithi042 (mon.0) 422 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-15T18:31:28.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.413+0000 7f9f25e0f700 1 -- 172.21.15.42:0/1898023452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9f200fde90 con 0x7f9f200f55c0 2024-02-15T18:31:28.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.413+0000 7f9f07fff700 1 -- 172.21.15.42:0/1898023452 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1773 (secure 0 0 0) 0x7f9f10019020 con 0x7f9f200f55c0 2024-02-15T18:31:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:31:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:31:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:31:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:31:28.407 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:31:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:31:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:31:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:31:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:31:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:31:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:31:28.407 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:31:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:31:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:31:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:31:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:31:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:31:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:31:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:31:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:31:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:31:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:31:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:31:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:31:28.408 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:31:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:31:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:31:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:31:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:31:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:31:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:31:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:31:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:31:28.409 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:31:28.409 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:31:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:31:28.409 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:31:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:31:28.409 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:31:28.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.415+0000 7f9f25e0f700 1 -- 172.21.15.42:0/1898023452 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9f08060f20 msgr2=0x7f9f080633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:28.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.415+0000 7f9f25e0f700 1 --2- 172.21.15.42:0/1898023452 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9f08060f20 0x7f9f080633e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f9f14009720 tx=0x7f9f14016040).stop 2024-02-15T18:31:28.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.415+0000 7f9f25e0f700 1 -- 172.21.15.42:0/1898023452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f200f55c0 msgr2=0x7f9f20109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:28.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.415+0000 7f9f25e0f700 1 --2- 172.21.15.42:0/1898023452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f200f55c0 0x7f9f20109cd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f9f1000e7c0 tx=0x7f9f1000b040).stop 2024-02-15T18:31:28.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.415+0000 7f9f25e0f700 1 -- 172.21.15.42:0/1898023452 shutdown_connections 2024-02-15T18:31:28.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.415+0000 7f9f25e0f700 1 --2- 172.21.15.42:0/1898023452 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9f08060f20 0x7f9f080633e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.415+0000 7f9f25e0f700 1 --2- 172.21.15.42:0/1898023452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f200f55c0 0x7f9f20109cd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.415+0000 7f9f25e0f700 1 --2- 172.21.15.42:0/1898023452 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f200f6860 0x7f9f2010c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.415+0000 7f9f25e0f700 1 -- 172.21.15.42:0/1898023452 >> 172.21.15.42:0/1898023452 conn(0x7f9f200f0d40 msgr2=0x7f9f200fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:28.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.415+0000 7f9f25e0f700 1 -- 172.21.15.42:0/1898023452 shutdown_connections 2024-02-15T18:31:28.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.415+0000 7f9f25e0f700 1 -- 172.21.15.42:0/1898023452 wait complete. 2024-02-15T18:31:28.411 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:31:28.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.518+0000 7f0151bca700 1 -- 172.21.15.42:0/3082847686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f014c0fbbf0 msgr2=0x7f014c0fdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:28.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.518+0000 7f0151bca700 1 --2- 172.21.15.42:0/3082847686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f014c0fbbf0 0x7f014c0fdfe0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f0140004660 tx=0x7f0140009b30).stop 2024-02-15T18:31:28.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.518+0000 7f0151bca700 1 -- 172.21.15.42:0/3082847686 shutdown_connections 2024-02-15T18:31:28.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.518+0000 7f0151bca700 1 --2- 172.21.15.42:0/3082847686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f014c0fbbf0 0x7f014c0fdfe0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.518+0000 7f0151bca700 1 --2- 172.21.15.42:0/3082847686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f014c0602b0 0x7f014c0fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.518+0000 7f0151bca700 1 -- 172.21.15.42:0/3082847686 >> 172.21.15.42:0/3082847686 conn(0x7f014c0f0d40 msgr2=0x7f014c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:28.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.518+0000 7f0151bca700 1 -- 172.21.15.42:0/3082847686 shutdown_connections 2024-02-15T18:31:28.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.518+0000 7f0151bca700 1 -- 172.21.15.42:0/3082847686 wait complete. 2024-02-15T18:31:28.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.518+0000 7f0151bca700 1 Processor -- start 2024-02-15T18:31:28.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.518+0000 7f0151bca700 1 -- start start 2024-02-15T18:31:28.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f0151bca700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f014c0602b0 0x7f014c1091c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:28.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f0151bca700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f014c0fbbf0 0x7f014c1096b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f0151bca700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c109c60 con 0x7f014c0602b0 2024-02-15T18:31:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f0151bca700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c109da0 con 0x7f014c0fbbf0 2024-02-15T18:31:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f014b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f014c0602b0 0x7f014c1091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f014b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f014c0602b0 0x7f014c1091c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34396/0 (socket says 172.21.15.42:34396) 2024-02-15T18:31:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f014b7fe700 1 -- 172.21.15.42:0/3225927189 learned_addr learned my addr 172.21.15.42:0/3225927189 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:31:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f014affd700 1 --2- 172.21.15.42:0/3225927189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f014c0fbbf0 0x7f014c1096b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f014b7fe700 1 -- 172.21.15.42:0/3225927189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f014c0fbbf0 msgr2=0x7f014c1096b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f014b7fe700 1 --2- 172.21.15.42:0/3225927189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f014c0fbbf0 0x7f014c1096b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f014b7fe700 1 -- 172.21.15.42:0/3225927189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0140005040 con 0x7f014c0602b0 2024-02-15T18:31:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f014affd700 1 --2- 172.21.15.42:0/3225927189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f014c0fbbf0 0x7f014c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:31:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f014b7fe700 1 --2- 172.21.15.42:0/3225927189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f014c0602b0 0x7f014c1091c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f013c00e7f0 tx=0x7f013c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f0133fff700 1 -- 172.21.15.42:0/3225927189 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f013c0092e0 con 0x7f014c0602b0 2024-02-15T18:31:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f0151bca700 1 -- 172.21.15.42:0/3225927189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014c10a740 con 0x7f014c0602b0 2024-02-15T18:31:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f0133fff700 1 -- 172.21.15.42:0/3225927189 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f013c00f040 con 0x7f014c0602b0 2024-02-15T18:31:28.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f0151bca700 1 -- 172.21.15.42:0/3225927189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014c10ad00 con 0x7f014c0602b0 2024-02-15T18:31:28.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.519+0000 7f0133fff700 1 -- 172.21.15.42:0/3225927189 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f013c014880 con 0x7f014c0602b0 2024-02-15T18:31:28.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.520+0000 7f0151bca700 1 -- 172.21.15.42:0/3225927189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014c045ad0 con 0x7f014c0602b0 2024-02-15T18:31:28.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.521+0000 7f0133fff700 1 -- 172.21.15.42:0/3225927189 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f013c01b030 con 0x7f014c0602b0 2024-02-15T18:31:28.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.521+0000 7f0133fff700 1 --2- 172.21.15.42:0/3225927189 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0134060f20 0x7f01340633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:28.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.521+0000 7f0133fff700 1 -- 172.21.15.42:0/3225927189 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6102+0+0 (secure 0 0 0) 0x7f013c07bff0 con 0x7f014c0602b0 2024-02-15T18:31:28.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.521+0000 7f014affd700 1 --2- 172.21.15.42:0/3225927189 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0134060f20 0x7f01340633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:28.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.522+0000 7f014affd700 1 --2- 172.21.15.42:0/3225927189 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0134060f20 0x7f01340633e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f0140009720 tx=0x7f014000b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:28.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.526+0000 7f0133fff700 1 -- 172.21.15.42:0/3225927189 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f013c04fe30 con 0x7f014c0602b0 2024-02-15T18:31:28.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.715+0000 7f0151bca700 1 -- 172.21.15.42:0/3225927189 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f014c101260 con 0x7f0134060f20 2024-02-15T18:31:28.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.717+0000 7f0133fff700 1 -- 172.21.15.42:0/3225927189 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f014c101260 con 0x7f0134060f20 2024-02-15T18:31:28.710 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:31:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:31:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:31:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:31:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:31:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-02-15T18:31:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-02-15T18:31:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-02-15T18:31:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:31:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "11/21 daemons upgraded", 2024-02-15T18:31:28.710 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading osd daemons", 2024-02-15T18:31:28.711 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:31:28.711 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:31:28.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.718+0000 7f0151bca700 1 -- 172.21.15.42:0/3225927189 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0134060f20 msgr2=0x7f01340633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:28.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.718+0000 7f0151bca700 1 --2- 172.21.15.42:0/3225927189 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0134060f20 0x7f01340633e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f0140009720 tx=0x7f014000b040).stop 2024-02-15T18:31:28.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.718+0000 7f0151bca700 1 -- 172.21.15.42:0/3225927189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f014c0602b0 msgr2=0x7f014c1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:28.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.718+0000 7f0151bca700 1 --2- 172.21.15.42:0/3225927189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f014c0602b0 0x7f014c1091c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f013c00e7f0 tx=0x7f013c00b040).stop 2024-02-15T18:31:28.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.718+0000 7f0151bca700 1 -- 172.21.15.42:0/3225927189 shutdown_connections 2024-02-15T18:31:28.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.718+0000 7f0151bca700 1 --2- 172.21.15.42:0/3225927189 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0134060f20 0x7f01340633e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.718+0000 7f0151bca700 1 --2- 172.21.15.42:0/3225927189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f014c0602b0 0x7f014c1091c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.718+0000 7f0151bca700 1 --2- 172.21.15.42:0/3225927189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f014c0fbbf0 0x7f014c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:28.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.718+0000 7f0151bca700 1 -- 172.21.15.42:0/3225927189 >> 172.21.15.42:0/3225927189 conn(0x7f014c0f0d40 msgr2=0x7f014c0f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:28.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.718+0000 7f0151bca700 1 -- 172.21.15.42:0/3225927189 shutdown_connections 2024-02-15T18:31:28.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:28.719+0000 7f0151bca700 1 -- 172.21.15.42:0/3225927189 wait complete. 2024-02-15T18:31:29.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:27.118144+0000 mgr.smithi042.kfoihb (mgr.24435) 258 : audit [DBG] from='client.34302 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:31:29.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:27.421609+0000 mgr.smithi042.kfoihb (mgr.24435) 259 : audit [DBG] from='client.34306 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:31:29.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:27.716300+0000 mgr.smithi042.kfoihb (mgr.24435) 260 : audit [DBG] from='client.34310 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:31:29.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:27.995106+0000 mon.smithi042 (mon.0) 423 : cluster [DBG] osdmap e104: 6 total, 6 up, 6 in 2024-02-15T18:31:29.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:28.086517+0000 mon.smithi042 (mon.0) 424 : audit [DBG] from='client.? 172.21.15.42:0/3550654591' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:31:29.196 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:28.414644+0000 mon.smithi042 (mon.0) 425 : audit [DBG] from='client.? 172.21.15.42:0/1898023452' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:31:29.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:27.118144+0000 mgr.smithi042.kfoihb (mgr.24435) 258 : audit [DBG] from='client.34302 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:31:29.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:27.421609+0000 mgr.smithi042.kfoihb (mgr.24435) 259 : audit [DBG] from='client.34306 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:31:29.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:27.716300+0000 mgr.smithi042.kfoihb (mgr.24435) 260 : audit [DBG] from='client.34310 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:31:29.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:27.995106+0000 mon.smithi042 (mon.0) 423 : cluster [DBG] osdmap e104: 6 total, 6 up, 6 in 2024-02-15T18:31:29.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:28.086517+0000 mon.smithi042 (mon.0) 424 : audit [DBG] from='client.? 172.21.15.42:0/3550654591' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:31:29.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:28.414644+0000 mon.smithi042 (mon.0) 425 : audit [DBG] from='client.? 172.21.15.42:0/1898023452' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:31:30.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:28.717534+0000 mgr.smithi042.kfoihb (mgr.24435) 261 : audit [DBG] from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:31:30.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:28.748398+0000 mgr.smithi042.kfoihb (mgr.24435) 262 : cluster [DBG] pgmap v219: 65 pgs: 2 peering, 1 remapped+peering, 61 active+clean, 1 active+recovering+undersized+remapped; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 1/210 objects misplaced (0.476%); 0 B/s, 189 objects/s recovering 2024-02-15T18:31:30.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:28.992914+0000 mon.smithi042 (mon.0) 426 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/210 objects degraded (3.333%), 5 pgs degraded) 2024-02-15T18:31:30.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:28.992937+0000 mon.smithi042 (mon.0) 427 : cluster [INF] Cluster is now healthy 2024-02-15T18:31:30.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:28.998128+0000 mon.smithi042 (mon.0) 428 : cluster [DBG] osdmap e105: 6 total, 6 up, 6 in 2024-02-15T18:31:30.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:28.717534+0000 mgr.smithi042.kfoihb (mgr.24435) 261 : audit [DBG] from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:31:30.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:28.748398+0000 mgr.smithi042.kfoihb (mgr.24435) 262 : cluster [DBG] pgmap v219: 65 pgs: 2 peering, 1 remapped+peering, 61 active+clean, 1 active+recovering+undersized+remapped; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 1/210 objects misplaced (0.476%); 0 B/s, 189 objects/s recovering 2024-02-15T18:31:30.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:28.992914+0000 mon.smithi042 (mon.0) 426 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/210 objects degraded (3.333%), 5 pgs degraded) 2024-02-15T18:31:30.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:28.992937+0000 mon.smithi042 (mon.0) 427 : cluster [INF] Cluster is now healthy 2024-02-15T18:31:30.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:28.998128+0000 mon.smithi042 (mon.0) 428 : cluster [DBG] osdmap e105: 6 total, 6 up, 6 in 2024-02-15T18:31:32.111 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:30.748930+0000 mgr.smithi042.kfoihb (mgr.24435) 263 : cluster [DBG] pgmap v221: 65 pgs: 64 active+clean, 1 active+recovering+undersized+remapped; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 644 B/s rd, 1 op/s; 1/210 objects misplaced (0.476%); 2.8 MiB/s, 10 objects/s recovering 2024-02-15T18:31:32.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:30.748930+0000 mgr.smithi042.kfoihb (mgr.24435) 263 : cluster [DBG] pgmap v221: 65 pgs: 64 active+clean, 1 active+recovering+undersized+remapped; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 644 B/s rd, 1 op/s; 1/210 objects misplaced (0.476%); 2.8 MiB/s, 10 objects/s recovering 2024-02-15T18:31:33.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:32.118975+0000 mon.smithi042 (mon.0) 429 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:31:33.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:32.118975+0000 mon.smithi042 (mon.0) 429 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:31:34.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:32.119217+0000 mgr.smithi042.kfoihb (mgr.24435) 264 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:31:34.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:31:32.120274+0000 mgr.smithi042.kfoihb (mgr.24435) 265 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-02-15T18:31:34.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:32.749362+0000 mgr.smithi042.kfoihb (mgr.24435) 266 : cluster [DBG] pgmap v222: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 2.9 MiB/s, 8 objects/s recovering 2024-02-15T18:31:34.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:32.119217+0000 mgr.smithi042.kfoihb (mgr.24435) 264 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:31:34.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:31:32.120274+0000 mgr.smithi042.kfoihb (mgr.24435) 265 : cephadm [INF] Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-02-15T18:31:34.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:32.749362+0000 mgr.smithi042.kfoihb (mgr.24435) 266 : cluster [DBG] pgmap v222: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 2.9 MiB/s, 8 objects/s recovering 2024-02-15T18:31:36.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:34.749770+0000 mgr.smithi042.kfoihb (mgr.24435) 267 : cluster [DBG] pgmap v223: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 395 B/s rd, 1 op/s; 2.2 MiB/s, 6 objects/s recovering 2024-02-15T18:31:36.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:34.749770+0000 mgr.smithi042.kfoihb (mgr.24435) 267 : cluster [DBG] pgmap v223: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 395 B/s rd, 1 op/s; 2.2 MiB/s, 6 objects/s recovering 2024-02-15T18:31:37.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:36.750248+0000 mgr.smithi042.kfoihb (mgr.24435) 268 : cluster [DBG] pgmap v224: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 935 B/s rd, 2 op/s; 2.0 MiB/s, 5 objects/s recovering 2024-02-15T18:31:37.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:36.750248+0000 mgr.smithi042.kfoihb (mgr.24435) 268 : cluster [DBG] pgmap v224: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 935 B/s rd, 2 op/s; 2.0 MiB/s, 5 objects/s recovering 2024-02-15T18:31:40.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:38.750766+0000 mgr.smithi042.kfoihb (mgr.24435) 269 : cluster [DBG] pgmap v225: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 819 B/s rd, 1 op/s; 1.7 MiB/s, 4 objects/s recovering 2024-02-15T18:31:40.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:38.750766+0000 mgr.smithi042.kfoihb (mgr.24435) 269 : cluster [DBG] pgmap v225: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 819 B/s rd, 1 op/s; 1.7 MiB/s, 4 objects/s recovering 2024-02-15T18:31:42.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:40.751306+0000 mgr.smithi042.kfoihb (mgr.24435) 270 : cluster [DBG] pgmap v226: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 1.5 MiB/s, 4 objects/s recovering 2024-02-15T18:31:42.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:40.837382+0000 mon.smithi042 (mon.0) 430 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:42.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:40.969143+0000 mon.smithi042 (mon.0) 431 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:42.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:40.751306+0000 mgr.smithi042.kfoihb (mgr.24435) 270 : cluster [DBG] pgmap v226: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 1.5 MiB/s, 4 objects/s recovering 2024-02-15T18:31:42.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:40.837382+0000 mon.smithi042 (mon.0) 430 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:42.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:40.969143+0000 mon.smithi042 (mon.0) 431 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:44.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:42.751884+0000 mgr.smithi042.kfoihb (mgr.24435) 271 : cluster [DBG] pgmap v227: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 341 KiB/s, 0 objects/s recovering 2024-02-15T18:31:44.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:42.751884+0000 mgr.smithi042.kfoihb (mgr.24435) 271 : cluster [DBG] pgmap v227: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 341 KiB/s, 0 objects/s recovering 2024-02-15T18:31:46.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:44.752290+0000 mgr.smithi042.kfoihb (mgr.24435) 272 : cluster [DBG] pgmap v228: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:31:46.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:44.752290+0000 mgr.smithi042.kfoihb (mgr.24435) 272 : cluster [DBG] pgmap v228: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:31:47.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:46 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:46.752752+0000 mgr.smithi042.kfoihb (mgr.24435) 273 : cluster [DBG] pgmap v229: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:31:47.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:46 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:46.752752+0000 mgr.smithi042.kfoihb (mgr.24435) 273 : cluster [DBG] pgmap v229: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:31:48.302 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:47.135877+0000 mon.smithi042 (mon.0) 432 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:31:48.302 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:47.136127+0000 mgr.smithi042.kfoihb (mgr.24435) 274 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:31:48.302 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:31:47.137192+0000 mgr.smithi042.kfoihb (mgr.24435) 275 : cephadm [INF] Upgrade: osd.5 is safe to restart 2024-02-15T18:31:48.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:47.135877+0000 mon.smithi042 (mon.0) 432 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:31:48.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:47.136127+0000 mgr.smithi042.kfoihb (mgr.24435) 274 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-15T18:31:48.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:31:47.137192+0000 mgr.smithi042.kfoihb (mgr.24435) 275 : cephadm [INF] Upgrade: osd.5 is safe to restart 2024-02-15T18:31:49.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:48.753296+0000 mgr.smithi042.kfoihb (mgr.24435) 276 : cluster [DBG] pgmap v230: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:31:49.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:31:48.760964+0000 mgr.smithi042.kfoihb (mgr.24435) 277 : cephadm [INF] Upgrade: Updating osd.5 2024-02-15T18:31:49.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:48.766317+0000 mon.smithi042 (mon.0) 433 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:49.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:48.767160+0000 mon.smithi042 (mon.0) 434 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-15T18:31:49.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:48.767897+0000 mon.smithi042 (mon.0) 435 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:31:49.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:31:48.769129+0000 mgr.smithi042.kfoihb (mgr.24435) 278 : cephadm [INF] Deploying daemon osd.5 on smithi191 2024-02-15T18:31:50.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:48.753296+0000 mgr.smithi042.kfoihb (mgr.24435) 276 : cluster [DBG] pgmap v230: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:31:50.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:31:48.760964+0000 mgr.smithi042.kfoihb (mgr.24435) 277 : cephadm [INF] Upgrade: Updating osd.5 2024-02-15T18:31:50.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:48.766317+0000 mon.smithi042 (mon.0) 433 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:50.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:48.767160+0000 mon.smithi042 (mon.0) 434 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-15T18:31:50.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:48.767897+0000 mon.smithi042 (mon.0) 435 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:31:50.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:31:48.769129+0000 mgr.smithi042.kfoihb (mgr.24435) 278 : cephadm [INF] Deploying daemon osd.5 on smithi191 2024-02-15T18:31:52.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:50.753899+0000 mgr.smithi042.kfoihb (mgr.24435) 279 : cluster [DBG] pgmap v231: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:31:52.122 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:50.753899+0000 mgr.smithi042.kfoihb (mgr.24435) 279 : cluster [DBG] pgmap v231: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:31:52.443 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:52 smithi191 systemd[1]: Stopping Ceph osd.5 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:31:52.693 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.5[52193]: debug 2024-02-15T18:31:52.449+0000 7fe9f9710700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:31:52.693 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.5[52193]: debug 2024-02-15T18:31:52.449+0000 7fe9f9710700 -1 osd.5 105 *** Got signal Terminated *** 2024-02-15T18:31:52.693 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.5[52193]: debug 2024-02-15T18:31:52.449+0000 7fe9f9710700 -1 osd.5 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:31:53.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:52.702507+0000 mon.smithi042 (mon.0) 436 : cluster [DBG] osd.5 reported immediately failed by osd.1 2024-02-15T18:31:53.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:52.702543+0000 mon.smithi042 (mon.0) 437 : cluster [INF] osd.5 failed (root=default,host=smithi191) (connection refused reported by osd.1) 2024-02-15T18:31:53.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:52.702704+0000 mon.smithi042 (mon.0) 438 : cluster [DBG] osd.5 reported immediately failed by osd.1 2024-02-15T18:31:53.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:52.702780+0000 mon.smithi042 (mon.0) 439 : cluster [DBG] osd.5 reported immediately failed by osd.2 2024-02-15T18:31:53.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:52.702960+0000 mon.smithi042 (mon.0) 440 : cluster [DBG] osd.5 reported immediately failed by osd.3 2024-02-15T18:31:53.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:52.703090+0000 mon.smithi042 (mon.0) 441 : cluster [DBG] osd.5 reported immediately failed by osd.2 2024-02-15T18:31:53.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:52.703191+0000 mon.smithi042 (mon.0) 442 : cluster [DBG] osd.5 reported immediately failed by osd.3 2024-02-15T18:31:53.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:52.703361+0000 mon.smithi042 (mon.0) 443 : cluster [DBG] osd.5 reported immediately failed by osd.4 2024-02-15T18:31:53.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:52.703462+0000 mon.smithi042 (mon.0) 444 : cluster [DBG] osd.5 reported immediately failed by osd.0 2024-02-15T18:31:53.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:52.703626+0000 mon.smithi042 (mon.0) 445 : cluster [DBG] osd.5 reported immediately failed by osd.4 2024-02-15T18:31:53.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:52.703726+0000 mon.smithi042 (mon.0) 446 : cluster [DBG] osd.5 reported immediately failed by osd.0 2024-02-15T18:31:53.134 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:52 smithi191 bash[66156]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.5 2024-02-15T18:31:53.134 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:52.702507+0000 mon.smithi042 (mon.0) 436 : cluster [DBG] osd.5 reported immediately failed by osd.1 2024-02-15T18:31:53.134 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:52.702543+0000 mon.smithi042 (mon.0) 437 : cluster [INF] osd.5 failed (root=default,host=smithi191) (connection refused reported by osd.1) 2024-02-15T18:31:53.135 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:52.702704+0000 mon.smithi042 (mon.0) 438 : cluster [DBG] osd.5 reported immediately failed by osd.1 2024-02-15T18:31:53.135 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:52.702780+0000 mon.smithi042 (mon.0) 439 : cluster [DBG] osd.5 reported immediately failed by osd.2 2024-02-15T18:31:53.135 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:52.702960+0000 mon.smithi042 (mon.0) 440 : cluster [DBG] osd.5 reported immediately failed by osd.3 2024-02-15T18:31:53.135 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:52.703090+0000 mon.smithi042 (mon.0) 441 : cluster [DBG] osd.5 reported immediately failed by osd.2 2024-02-15T18:31:53.135 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:52.703191+0000 mon.smithi042 (mon.0) 442 : cluster [DBG] osd.5 reported immediately failed by osd.3 2024-02-15T18:31:53.135 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:52.703361+0000 mon.smithi042 (mon.0) 443 : cluster [DBG] osd.5 reported immediately failed by osd.4 2024-02-15T18:31:53.135 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:52.703462+0000 mon.smithi042 (mon.0) 444 : cluster [DBG] osd.5 reported immediately failed by osd.0 2024-02-15T18:31:53.135 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:52.703626+0000 mon.smithi042 (mon.0) 445 : cluster [DBG] osd.5 reported immediately failed by osd.4 2024-02-15T18:31:53.135 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:52.703726+0000 mon.smithi042 (mon.0) 446 : cluster [DBG] osd.5 reported immediately failed by osd.0 2024-02-15T18:31:53.442 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:53 smithi191 bash[66210]: Error: no container with name or ID "ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-15T18:31:54.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:52.754522+0000 mgr.smithi042.kfoihb (mgr.24435) 280 : cluster [DBG] pgmap v232: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:31:54.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:52.768822+0000 mon.smithi042 (mon.0) 447 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:31:54.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:52.776120+0000 mon.smithi042 (mon.0) 448 : cluster [DBG] osdmap e106: 6 total, 5 up, 6 in 2024-02-15T18:31:54.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:52.754522+0000 mgr.smithi042.kfoihb (mgr.24435) 280 : cluster [DBG] pgmap v232: 65 pgs: 65 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:31:54.244 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:52.768822+0000 mon.smithi042 (mon.0) 447 : cluster [WRN] Health check failed: 1 osds down (OSD_DOWN) 2024-02-15T18:31:54.244 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:52.776120+0000 mon.smithi042 (mon.0) 448 : cluster [DBG] osdmap e106: 6 total, 5 up, 6 in 2024-02-15T18:31:55.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:53.779710+0000 mon.smithi042 (mon.0) 449 : cluster [DBG] osdmap e107: 6 total, 5 up, 6 in 2024-02-15T18:31:55.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:53.779710+0000 mon.smithi042 (mon.0) 449 : cluster [DBG] osdmap e107: 6 total, 5 up, 6 in 2024-02-15T18:31:55.193 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:54 smithi191 systemd[1]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.5.service: Succeeded. 2024-02-15T18:31:55.194 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:54 smithi191 systemd[1]: Stopped Ceph osd.5 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:31:55.194 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:55 smithi191 systemd[1]: Starting Ceph osd.5 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:31:56.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:54.755128+0000 mgr.smithi042.kfoihb (mgr.24435) 281 : cluster [DBG] pgmap v235: 65 pgs: 15 stale+active+clean, 50 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-02-15T18:31:56.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:54.755128+0000 mgr.smithi042.kfoihb (mgr.24435) 281 : cluster [DBG] pgmap v235: 65 pgs: 15 stale+active+clean, 50 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-02-15T18:31:56.193 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:55 smithi191 bash[66468]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-02-15T18:31:56.193 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:55 smithi191 bash[66468]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-15T18:31:56.193 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:55 smithi191 bash[66468]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-5 --no-mon-config 2024-02-15T18:31:56.193 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:55 smithi191 bash[66468]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-15T18:31:56.194 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:55 smithi191 bash[66468]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-5/block 2024-02-15T18:31:56.194 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:55 smithi191 bash[66468]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-15T18:31:56.194 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:55 smithi191 bash[66468]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-15T18:31:56.194 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:55 smithi191 bash[66468]: --> ceph-volume lvm activate successful for osd ID: 5 2024-02-15T18:31:57.123 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:55.837567+0000 mon.smithi042 (mon.0) 450 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:57.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:55.837567+0000 mon.smithi042 (mon.0) 450 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:57.443 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:31:57.129+0000 7fb324cd1700 1 -- 172.21.15.191:0/475559723 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555c14fe2680 con 0x555c14fa9000 2024-02-15T18:31:57.443 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:57 smithi191 bash[66716]: d45e566ed2881603684084474cd4d4e14facf9de94ae8e57ad640758e9781376 2024-02-15T18:31:57.443 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:57 smithi191 systemd[1]: Started Ceph osd.5 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:31:57.943 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:31:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:31:57.647+0000 7fb32c75f380 -1 Falling back to public interface 2024-02-15T18:31:57.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:56.755779+0000 mgr.smithi042.kfoihb (mgr.24435) 282 : cluster [DBG] pgmap v236: 65 pgs: 11 active+undersized, 7 active+undersized+degraded, 6 stale+active+clean, 41 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 22/210 objects degraded (10.476%) 2024-02-15T18:31:57.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:56.987221+0000 mon.smithi042 (mon.0) 451 : cluster [WRN] Health check failed: Degraded data redundancy: 22/210 objects degraded (10.476%), 7 pgs degraded (PG_DEGRADED) 2024-02-15T18:31:57.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:57.249936+0000 mon.smithi042 (mon.0) 452 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:57.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:57.250847+0000 mon.smithi042 (mon.0) 453 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:31:58.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:56.755779+0000 mgr.smithi042.kfoihb (mgr.24435) 282 : cluster [DBG] pgmap v236: 65 pgs: 11 active+undersized, 7 active+undersized+degraded, 6 stale+active+clean, 41 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 22/210 objects degraded (10.476%) 2024-02-15T18:31:58.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:56.987221+0000 mon.smithi042 (mon.0) 451 : cluster [WRN] Health check failed: Degraded data redundancy: 22/210 objects degraded (10.476%), 7 pgs degraded (PG_DEGRADED) 2024-02-15T18:31:58.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:57.249936+0000 mon.smithi042 (mon.0) 452 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:31:58.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:57.250847+0000 mon.smithi042 (mon.0) 453 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:31:58.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.835+0000 7fea04834700 1 -- 172.21.15.42:0/3562666602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea000f6ff0 msgr2=0x7fea000f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:58.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.835+0000 7fea04834700 1 --2- 172.21.15.42:0/3562666602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea000f6ff0 0x7fea000f7a20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fe9f0004660 tx=0x7fe9f0009b30).stop 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.835+0000 7fea04834700 1 -- 172.21.15.42:0/3562666602 shutdown_connections 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.835+0000 7fea04834700 1 --2- 172.21.15.42:0/3562666602 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea000f6ff0 0x7fea000f7a20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.835+0000 7fea04834700 1 --2- 172.21.15.42:0/3562666602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea000f66e0 0x7fea000f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.835+0000 7fea04834700 1 -- 172.21.15.42:0/3562666602 >> 172.21.15.42:0/3562666602 conn(0x7fea000f1f00 msgr2=0x7fea000f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.835+0000 7fea04834700 1 -- 172.21.15.42:0/3562666602 shutdown_connections 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.835+0000 7fea04834700 1 -- 172.21.15.42:0/3562666602 wait complete. 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.835+0000 7fea04834700 1 Processor -- start 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.835+0000 7fea04834700 1 -- start start 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fea04834700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea000f66e0 0x7fea00106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fea04834700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea000f6ff0 0x7fea001074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fea04834700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea00107a60 con 0x7fea000f6ff0 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fea04834700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea00107ba0 con 0x7fea000f66e0 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fe9fdd9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea000f6ff0 0x7fea001074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fe9fdd9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea000f6ff0 0x7fea001074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42526/0 (socket says 172.21.15.42:42526) 2024-02-15T18:31:58.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fe9fdd9b700 1 -- 172.21.15.42:0/2933539259 learned_addr learned my addr 172.21.15.42:0/2933539259 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:31:58.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fe9fe59c700 1 --2- 172.21.15.42:0/2933539259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea000f66e0 0x7fea00106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:58.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fe9fdd9b700 1 -- 172.21.15.42:0/2933539259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea000f66e0 msgr2=0x7fea00106fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:58.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fe9fdd9b700 1 --2- 172.21.15.42:0/2933539259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea000f66e0 0x7fea00106fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:58.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fe9fdd9b700 1 -- 172.21.15.42:0/2933539259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0005040 con 0x7fea000f6ff0 2024-02-15T18:31:58.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fe9fdd9b700 1 --2- 172.21.15.42:0/2933539259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea000f6ff0 0x7fea001074b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fe9f0000c00 tx=0x7fe9f000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:58.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fe9f6ffd700 1 -- 172.21.15.42:0/2933539259 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9f0019070 con 0x7fea000f6ff0 2024-02-15T18:31:58.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fea04834700 1 -- 172.21.15.42:0/2933539259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea001084e0 con 0x7fea000f6ff0 2024-02-15T18:31:58.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fe9f6ffd700 1 -- 172.21.15.42:0/2933539259 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fe9f0007e80 con 0x7fea000f6ff0 2024-02-15T18:31:58.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.836+0000 7fe9f6ffd700 1 -- 172.21.15.42:0/2933539259 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9f0014db0 con 0x7fea000f6ff0 2024-02-15T18:31:58.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.837+0000 7fea04834700 1 -- 172.21.15.42:0/2933539259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea00108a40 con 0x7fea000f6ff0 2024-02-15T18:31:58.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.838+0000 7fea04834700 1 -- 172.21.15.42:0/2933539259 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea00045ad0 con 0x7fea000f6ff0 2024-02-15T18:31:58.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.838+0000 7fe9f6ffd700 1 -- 172.21.15.42:0/2933539259 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fe9f0025070 con 0x7fea000f6ff0 2024-02-15T18:31:58.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.838+0000 7fe9f6ffd700 1 --2- 172.21.15.42:0/2933539259 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe9ec060ec0 0x7fe9ec063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:58.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.838+0000 7fe9f6ffd700 1 -- 172.21.15.42:0/2933539259 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6102+0+0 (secure 0 0 0) 0x7fe9f007d140 con 0x7fea000f6ff0 2024-02-15T18:31:58.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.839+0000 7fe9fe59c700 1 --2- 172.21.15.42:0/2933539259 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe9ec060ec0 0x7fe9ec063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:58.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.839+0000 7fe9fe59c700 1 --2- 172.21.15.42:0/2933539259 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe9ec060ec0 0x7fe9ec063380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fe9e8009fd0 tx=0x7fe9e800b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:58.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:58.843+0000 7fe9f6ffd700 1 -- 172.21.15.42:0/2933539259 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe9f0050ed0 con 0x7fea000f6ff0 2024-02-15T18:31:59.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.031+0000 7fea04834700 1 -- 172.21.15.42:0/2933539259 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea000ff150 con 0x7fe9ec060ec0 2024-02-15T18:31:59.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.032+0000 7fe9f6ffd700 1 -- 172.21.15.42:0/2933539259 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fea000ff150 con 0x7fe9ec060ec0 2024-02-15T18:31:59.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.034+0000 7fea04834700 1 -- 172.21.15.42:0/2933539259 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe9ec060ec0 msgr2=0x7fe9ec063380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.034+0000 7fea04834700 1 --2- 172.21.15.42:0/2933539259 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe9ec060ec0 0x7fe9ec063380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fe9e8009fd0 tx=0x7fe9e800b040).stop 2024-02-15T18:31:59.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.034+0000 7fea04834700 1 -- 172.21.15.42:0/2933539259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea000f6ff0 msgr2=0x7fea001074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.034+0000 7fea04834700 1 --2- 172.21.15.42:0/2933539259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea000f6ff0 0x7fea001074b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fe9f0000c00 tx=0x7fe9f000b040).stop 2024-02-15T18:31:59.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.034+0000 7fea04834700 1 -- 172.21.15.42:0/2933539259 shutdown_connections 2024-02-15T18:31:59.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.034+0000 7fea04834700 1 --2- 172.21.15.42:0/2933539259 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe9ec060ec0 0x7fe9ec063380 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.034+0000 7fea04834700 1 --2- 172.21.15.42:0/2933539259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea000f66e0 0x7fea00106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.034+0000 7fea04834700 1 --2- 172.21.15.42:0/2933539259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea000f6ff0 0x7fea001074b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.034+0000 7fea04834700 1 -- 172.21.15.42:0/2933539259 >> 172.21.15.42:0/2933539259 conn(0x7fea000f1f00 msgr2=0x7fea000fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:59.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.034+0000 7fea04834700 1 -- 172.21.15.42:0/2933539259 shutdown_connections 2024-02-15T18:31:59.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.034+0000 7fea04834700 1 -- 172.21.15.42:0/2933539259 wait complete. 2024-02-15T18:31:59.037 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:31:59.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.137+0000 7f281bc70700 1 -- 172.21.15.42:0/302370067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28140f66e0 msgr2=0x7f28140f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.137+0000 7f281bc70700 1 --2- 172.21.15.42:0/302370067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28140f66e0 0x7f28140f6b00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f2804004660 tx=0x7f2804009b30).stop 2024-02-15T18:31:59.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.137+0000 7f281bc70700 1 -- 172.21.15.42:0/302370067 shutdown_connections 2024-02-15T18:31:59.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.137+0000 7f281bc70700 1 --2- 172.21.15.42:0/302370067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28140f7980 0x7f28140f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.137+0000 7f281bc70700 1 --2- 172.21.15.42:0/302370067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28140f66e0 0x7f28140f6b00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.137+0000 7f281bc70700 1 -- 172.21.15.42:0/302370067 >> 172.21.15.42:0/302370067 conn(0x7f28140f1e40 msgr2=0x7f28140f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:59.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.137+0000 7f281bc70700 1 -- 172.21.15.42:0/302370067 shutdown_connections 2024-02-15T18:31:59.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.137+0000 7f281bc70700 1 -- 172.21.15.42:0/302370067 wait complete. 2024-02-15T18:31:59.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.137+0000 7f281bc70700 1 Processor -- start 2024-02-15T18:31:59.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.138+0000 7f281bc70700 1 -- start start 2024-02-15T18:31:59.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.138+0000 7f281bc70700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28140f66e0 0x7f2814107030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:59.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.138+0000 7f281bc70700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28140f7980 0x7f2814107520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:59.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.138+0000 7f281bc70700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2814107ad0 con 0x7f28140f7980 2024-02-15T18:31:59.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.138+0000 7f281bc70700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2814107c10 con 0x7f28140f66e0 2024-02-15T18:31:59.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.138+0000 7f281920b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28140f7980 0x7f2814107520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:59.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.138+0000 7f281920b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28140f7980 0x7f2814107520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42552/0 (socket says 172.21.15.42:42552) 2024-02-15T18:31:59.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.138+0000 7f281920b700 1 -- 172.21.15.42:0/256929611 learned_addr learned my addr 172.21.15.42:0/256929611 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:31:59.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.138+0000 7f2819a0c700 1 --2- 172.21.15.42:0/256929611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28140f66e0 0x7f2814107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:59.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.138+0000 7f281920b700 1 -- 172.21.15.42:0/256929611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28140f66e0 msgr2=0x7f2814107030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.138+0000 7f281920b700 1 --2- 172.21.15.42:0/256929611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28140f66e0 0x7f2814107030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.138+0000 7f281920b700 1 -- 172.21.15.42:0/256929611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2804005040 con 0x7f28140f7980 2024-02-15T18:31:59.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.138+0000 7f281920b700 1 --2- 172.21.15.42:0/256929611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28140f7980 0x7f2814107520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f28100026d0 tx=0x7f2810009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:59.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.139+0000 7f280a7fc700 1 -- 172.21.15.42:0/256929611 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28100079c0 con 0x7f28140f7980 2024-02-15T18:31:59.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.139+0000 7f281bc70700 1 -- 172.21.15.42:0/256929611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28141085b0 con 0x7f28140f7980 2024-02-15T18:31:59.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.139+0000 7f280a7fc700 1 -- 172.21.15.42:0/256929611 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f281000f040 con 0x7f28140f7980 2024-02-15T18:31:59.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.139+0000 7f280a7fc700 1 -- 172.21.15.42:0/256929611 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28100149f0 con 0x7f28140f7980 2024-02-15T18:31:59.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.139+0000 7f281bc70700 1 -- 172.21.15.42:0/256929611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2814108b70 con 0x7f28140f7980 2024-02-15T18:31:59.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.139+0000 7f281bc70700 1 -- 172.21.15.42:0/256929611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2814045ad0 con 0x7f28140f7980 2024-02-15T18:31:59.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.140+0000 7f280a7fc700 1 -- 172.21.15.42:0/256929611 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f2810007b20 con 0x7f28140f7980 2024-02-15T18:31:59.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.140+0000 7f280a7fc700 1 --2- 172.21.15.42:0/256929611 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2800060f20 0x7f28000633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:59.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.140+0000 7f280a7fc700 1 -- 172.21.15.42:0/256929611 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6102+0+0 (secure 0 0 0) 0x7f281007c200 con 0x7f28140f7980 2024-02-15T18:31:59.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.140+0000 7f2819a0c700 1 --2- 172.21.15.42:0/256929611 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2800060f20 0x7f28000633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:59.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.141+0000 7f2819a0c700 1 --2- 172.21.15.42:0/256929611 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2800060f20 0x7f28000633e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f2804004630 tx=0x7f2804009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:59.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.145+0000 7f280a7fc700 1 -- 172.21.15.42:0/256929611 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f281004ff90 con 0x7f28140f7980 2024-02-15T18:31:59.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.335+0000 7f281bc70700 1 -- 172.21.15.42:0/256929611 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2814108df0 con 0x7f2800060f20 2024-02-15T18:31:59.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.336+0000 7f280a7fc700 1 -- 172.21.15.42:0/256929611 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f2814108df0 con 0x7f2800060f20 2024-02-15T18:31:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.338+0000 7f281bc70700 1 -- 172.21.15.42:0/256929611 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2800060f20 msgr2=0x7f28000633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.338+0000 7f281bc70700 1 --2- 172.21.15.42:0/256929611 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2800060f20 0x7f28000633e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f2804004630 tx=0x7f2804009b30).stop 2024-02-15T18:31:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.338+0000 7f281bc70700 1 -- 172.21.15.42:0/256929611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28140f7980 msgr2=0x7f2814107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.338+0000 7f281bc70700 1 --2- 172.21.15.42:0/256929611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28140f7980 0x7f2814107520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f28100026d0 tx=0x7f2810009de0).stop 2024-02-15T18:31:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.338+0000 7f281bc70700 1 -- 172.21.15.42:0/256929611 shutdown_connections 2024-02-15T18:31:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.338+0000 7f281bc70700 1 --2- 172.21.15.42:0/256929611 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2800060f20 0x7f28000633e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.338+0000 7f281bc70700 1 --2- 172.21.15.42:0/256929611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f28140f66e0 0x7f2814107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.338+0000 7f281bc70700 1 --2- 172.21.15.42:0/256929611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28140f7980 0x7f2814107520 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.338+0000 7f281bc70700 1 -- 172.21.15.42:0/256929611 >> 172.21.15.42:0/256929611 conn(0x7f28140f1e40 msgr2=0x7f28140fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.338+0000 7f281bc70700 1 -- 172.21.15.42:0/256929611 shutdown_connections 2024-02-15T18:31:59.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.338+0000 7f281bc70700 1 -- 172.21.15.42:0/256929611 wait complete. 2024-02-15T18:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.439+0000 7fdf748f6700 1 -- 172.21.15.42:0/4051774445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf700f6860 msgr2=0x7fdf700f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.439+0000 7fdf748f6700 1 --2- 172.21.15.42:0/4051774445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf700f6860 0x7fdf700f6cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fdf58004660 tx=0x7fdf58009b30).stop 2024-02-15T18:31:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.439+0000 7fdf748f6700 1 -- 172.21.15.42:0/4051774445 shutdown_connections 2024-02-15T18:31:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.439+0000 7fdf748f6700 1 --2- 172.21.15.42:0/4051774445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf700f6860 0x7fdf700f6cc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.439+0000 7fdf748f6700 1 --2- 172.21.15.42:0/4051774445 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf700f55c0 0x7fdf700f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.439+0000 7fdf748f6700 1 -- 172.21.15.42:0/4051774445 >> 172.21.15.42:0/4051774445 conn(0x7fdf700f0d40 msgr2=0x7fdf700f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.439+0000 7fdf748f6700 1 -- 172.21.15.42:0/4051774445 shutdown_connections 2024-02-15T18:31:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.439+0000 7fdf748f6700 1 -- 172.21.15.42:0/4051774445 wait complete. 2024-02-15T18:31:59.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.440+0000 7fdf748f6700 1 Processor -- start 2024-02-15T18:31:59.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.440+0000 7fdf748f6700 1 -- start start 2024-02-15T18:31:59.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.440+0000 7fdf748f6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf700f55c0 0x7fdf70109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:59.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.440+0000 7fdf748f6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf700f6860 0x7fdf7010c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.440+0000 7fdf748f6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700603e0 con 0x7fdf700f6860 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.440+0000 7fdf748f6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf70060520 con 0x7fdf700f55c0 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.440+0000 7fdf6dd9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf700f6860 0x7fdf7010c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.440+0000 7fdf6dd9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf700f6860 0x7fdf7010c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42570/0 (socket says 172.21.15.42:42570) 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.440+0000 7fdf6dd9b700 1 -- 172.21.15.42:0/2047423043 learned_addr learned my addr 172.21.15.42:0/2047423043 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.440+0000 7fdf6e59c700 1 --2- 172.21.15.42:0/2047423043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf700f55c0 0x7fdf70109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.440+0000 7fdf6dd9b700 1 -- 172.21.15.42:0/2047423043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf700f55c0 msgr2=0x7fdf70109cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.440+0000 7fdf6dd9b700 1 --2- 172.21.15.42:0/2047423043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf700f55c0 0x7fdf70109cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.440+0000 7fdf6dd9b700 1 -- 172.21.15.42:0/2047423043 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf58005040 con 0x7fdf700f6860 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.441+0000 7fdf6dd9b700 1 --2- 172.21.15.42:0/2047423043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf700f6860 0x7fdf7010c1b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fdf58009f50 tx=0x7fdf58009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.441+0000 7fdf66ffd700 1 -- 172.21.15.42:0/2047423043 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf58018070 con 0x7fdf700f6860 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.441+0000 7fdf66ffd700 1 -- 172.21.15.42:0/2047423043 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fdf58007ba0 con 0x7fdf700f6860 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.441+0000 7fdf748f6700 1 -- 172.21.15.42:0/2047423043 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7010c700 con 0x7fdf700f6860 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.441+0000 7fdf66ffd700 1 -- 172.21.15.42:0/2047423043 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf5801c420 con 0x7fdf700f6860 2024-02-15T18:31:59.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.441+0000 7fdf748f6700 1 -- 172.21.15.42:0/2047423043 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7010cbe0 con 0x7fdf700f6860 2024-02-15T18:31:59.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.442+0000 7fdf748f6700 1 -- 172.21.15.42:0/2047423043 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf70045ad0 con 0x7fdf700f6860 2024-02-15T18:31:59.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.442+0000 7fdf66ffd700 1 -- 172.21.15.42:0/2047423043 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fdf5801d420 con 0x7fdf700f6860 2024-02-15T18:31:59.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.442+0000 7fdf66ffd700 1 --2- 172.21.15.42:0/2047423043 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdf5c060f20 0x7fdf5c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:59.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.443+0000 7fdf66ffd700 1 -- 172.21.15.42:0/2047423043 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6102+0+0 (secure 0 0 0) 0x7fdf5807dee0 con 0x7fdf700f6860 2024-02-15T18:31:59.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.443+0000 7fdf6e59c700 1 --2- 172.21.15.42:0/2047423043 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdf5c060f20 0x7fdf5c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:59.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.443+0000 7fdf6e59c700 1 --2- 172.21.15.42:0/2047423043 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdf5c060f20 0x7fdf5c0633e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fdf70060060 tx=0x7fdf6000b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:59.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.447+0000 7fdf66ffd700 1 -- 172.21.15.42:0/2047423043 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdf5804e1e0 con 0x7fdf700f6860 2024-02-15T18:31:59.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.629+0000 7fdf748f6700 1 -- 172.21.15.42:0/2047423043 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf70057f10 con 0x7fdf5c060f20 2024-02-15T18:31:59.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.638+0000 7fdf66ffd700 1 -- 172.21.15.42:0/2047423043 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fdf70057f10 con 0x7fdf5c060f20 2024-02-15T18:31:59.632 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:31:59.632 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (11m) 115s ago 13m 21.4M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:31:59.632 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (4m) 115s ago 13m 6643k - 16.2.14-521-g0e714d9a 1fbc92c859dc 2da9d875539a 2024-02-15T18:31:59.632 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (4m) 42s ago 12m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 5c46a68ff62b 2024-02-15T18:31:59.632 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (11m) 115s ago 13m 33.7M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:31:59.632 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (8m) 115s ago 8m 14.9M - 16.2.4 8d91d370c2b8 bd400ddeecd2 2024-02-15T18:31:59.632 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (8m) 115s ago 8m 17.5M - 16.2.4 8d91d370c2b8 0377c9d13c47 2024-02-15T18:31:59.632 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (8m) 42s ago 8m 110M - 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:31:59.632 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (8m) 42s ago 8m 153M - 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:31:59.632 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (5m) 115s ago 14m 435M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:31:59.632 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (5m) 42s ago 12m 389M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:31:59.633 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (5m) 115s ago 15m 46.9M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:31:59.633 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (4m) 42s ago 12m 42.8M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:31:59.633 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (13m) 115s ago 13m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:31:59.633 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (12m) 42s ago 12m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:31:59.633 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (3m) 115s ago 11m 250M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc f226bea76cab 2024-02-15T18:31:59.633 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (2m) 115s ago 11m 264M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 5114ed026724 2024-02-15T18:31:59.633 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (2m) 115s ago 10m 11.6M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 2e26b1ec292a 2024-02-15T18:31:59.633 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (90s) 42s ago 10m 328M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc c71b210091cb 2024-02-15T18:31:59.633 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (46s) 42s ago 10m 13.0M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 83bd9cdcb3da 2024-02-15T18:31:59.633 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 starting - - - 4096M 2024-02-15T18:31:59.633 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (6m) 115s ago 12m 61.0M - 2.18.1 de242295e225 63b6dfb825ed 2024-02-15T18:31:59.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.640+0000 7fdf748f6700 1 -- 172.21.15.42:0/2047423043 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdf5c060f20 msgr2=0x7fdf5c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.640+0000 7fdf748f6700 1 --2- 172.21.15.42:0/2047423043 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdf5c060f20 0x7fdf5c0633e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fdf70060060 tx=0x7fdf6000b040).stop 2024-02-15T18:31:59.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.640+0000 7fdf748f6700 1 -- 172.21.15.42:0/2047423043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf700f6860 msgr2=0x7fdf7010c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.640+0000 7fdf748f6700 1 --2- 172.21.15.42:0/2047423043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf700f6860 0x7fdf7010c1b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fdf58009f50 tx=0x7fdf58009b30).stop 2024-02-15T18:31:59.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.640+0000 7fdf748f6700 1 -- 172.21.15.42:0/2047423043 shutdown_connections 2024-02-15T18:31:59.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.640+0000 7fdf748f6700 1 --2- 172.21.15.42:0/2047423043 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdf5c060f20 0x7fdf5c0633e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.640+0000 7fdf748f6700 1 --2- 172.21.15.42:0/2047423043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf700f55c0 0x7fdf70109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.640+0000 7fdf748f6700 1 --2- 172.21.15.42:0/2047423043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf700f6860 0x7fdf7010c1b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.640+0000 7fdf748f6700 1 -- 172.21.15.42:0/2047423043 >> 172.21.15.42:0/2047423043 conn(0x7fdf700f0d40 msgr2=0x7fdf700fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:59.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.640+0000 7fdf748f6700 1 -- 172.21.15.42:0/2047423043 shutdown_connections 2024-02-15T18:31:59.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.640+0000 7fdf748f6700 1 -- 172.21.15.42:0/2047423043 wait complete. 2024-02-15T18:31:59.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.740+0000 7f4a20f06700 1 -- 172.21.15.42:0/756874492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a1c0f6530 msgr2=0x7f4a1c0f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.740+0000 7f4a20f06700 1 --2- 172.21.15.42:0/756874492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a1c0f6530 0x7f4a1c0f6950 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f4a0c004660 tx=0x7f4a0c009b30).stop 2024-02-15T18:31:59.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.740+0000 7f4a20f06700 1 -- 172.21.15.42:0/756874492 shutdown_connections 2024-02-15T18:31:59.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.740+0000 7f4a20f06700 1 --2- 172.21.15.42:0/756874492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1c0f77d0 0x7f4a1c0f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.740+0000 7f4a20f06700 1 --2- 172.21.15.42:0/756874492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a1c0f6530 0x7f4a1c0f6950 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.740+0000 7f4a20f06700 1 -- 172.21.15.42:0/756874492 >> 172.21.15.42:0/756874492 conn(0x7f4a1c0f1cd0 msgr2=0x7f4a1c0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:59.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.740+0000 7f4a20f06700 1 -- 172.21.15.42:0/756874492 shutdown_connections 2024-02-15T18:31:59.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.741+0000 7f4a20f06700 1 -- 172.21.15.42:0/756874492 wait complete. 2024-02-15T18:31:59.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.741+0000 7f4a20f06700 1 Processor -- start 2024-02-15T18:31:59.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.741+0000 7f4a20f06700 1 -- start start 2024-02-15T18:31:59.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.741+0000 7f4a20f06700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a1c0f6530 0x7f4a1c106e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:59.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.741+0000 7f4a20f06700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1c0f77d0 0x7f4a1c107340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:59.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.741+0000 7f4a20f06700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c107940 con 0x7f4a1c0f6530 2024-02-15T18:31:59.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.741+0000 7f4a20f06700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c108150 con 0x7f4a1c0f77d0 2024-02-15T18:31:59.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.742+0000 7f4a1b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a1c0f6530 0x7f4a1c106e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:59.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.742+0000 7f4a1b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a1c0f6530 0x7f4a1c106e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42578/0 (socket says 172.21.15.42:42578) 2024-02-15T18:31:59.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.742+0000 7f4a1b7fe700 1 -- 172.21.15.42:0/1359908582 learned_addr learned my addr 172.21.15.42:0/1359908582 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:31:59.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.742+0000 7f4a1affd700 1 --2- 172.21.15.42:0/1359908582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1c0f77d0 0x7f4a1c107340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:59.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.742+0000 7f4a1b7fe700 1 -- 172.21.15.42:0/1359908582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1c0f77d0 msgr2=0x7f4a1c107340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.742+0000 7f4a1b7fe700 1 --2- 172.21.15.42:0/1359908582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1c0f77d0 0x7f4a1c107340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.742+0000 7f4a1b7fe700 1 -- 172.21.15.42:0/1359908582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c005040 con 0x7f4a1c0f6530 2024-02-15T18:31:59.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.742+0000 7f4a1b7fe700 1 --2- 172.21.15.42:0/1359908582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a1c0f6530 0x7f4a1c106e50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f4a0c004e30 tx=0x7f4a0c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.742+0000 7f4a13fff700 1 -- 172.21.15.42:0/1359908582 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a0c018070 con 0x7f4a1c0f6530 2024-02-15T18:31:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.742+0000 7f4a20f06700 1 -- 172.21.15.42:0/1359908582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c108370 con 0x7f4a1c0f6530 2024-02-15T18:31:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.742+0000 7f4a13fff700 1 -- 172.21.15.42:0/1359908582 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f4a0c007ba0 con 0x7f4a1c0f6530 2024-02-15T18:31:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.742+0000 7f4a20f06700 1 -- 172.21.15.42:0/1359908582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c1088d0 con 0x7f4a1c0f6530 2024-02-15T18:31:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.742+0000 7f4a13fff700 1 -- 172.21.15.42:0/1359908582 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a0c027440 con 0x7f4a1c0f6530 2024-02-15T18:31:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.743+0000 7f4a20f06700 1 -- 172.21.15.42:0/1359908582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c059100 con 0x7f4a1c0f6530 2024-02-15T18:31:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.744+0000 7f4a13fff700 1 -- 172.21.15.42:0/1359908582 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f4a0c01d9a0 con 0x7f4a1c0f6530 2024-02-15T18:31:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.744+0000 7f4a13fff700 1 --2- 172.21.15.42:0/1359908582 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4a08060f70 0x7f4a08063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:31:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.744+0000 7f4a13fff700 1 -- 172.21.15.42:0/1359908582 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6102+0+0 (secure 0 0 0) 0x7f4a0c07ddd0 con 0x7f4a1c0f6530 2024-02-15T18:31:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.744+0000 7f4a1affd700 1 --2- 172.21.15.42:0/1359908582 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4a08060f70 0x7f4a08063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:31:59.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.744+0000 7f4a1affd700 1 --2- 172.21.15.42:0/1359908582 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4a08060f70 0x7f4a08063430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f4a040096c0 tx=0x7f4a04009be0).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:31:59.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.748+0000 7f4a13fff700 1 -- 172.21.15.42:0/1359908582 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4a0c04e130 con 0x7f4a1c0f6530 2024-02-15T18:31:59.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.993+0000 7f4a20f06700 1 -- 172.21.15.42:0/1359908582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4a1c0fee00 con 0x7f4a1c0f6530 2024-02-15T18:31:59.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.993+0000 7f4a13fff700 1 -- 172.21.15.42:0/1359908582 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+726 (secure 0 0 0) 0x7f4a0c051750 con 0x7f4a1c0f6530 2024-02-15T18:31:59.987 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:31:59.987 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:31:59.987 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:31:59.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:31:59.987 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:31:59.987 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:31:59.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:31:59.987 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:31:59.987 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 5 2024-02-15T18:31:59.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:31:59.987 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:31:59.987 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:31:59.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:31:59.988 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:31:59.988 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 9, 2024-02-15T18:31:59.988 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2024-02-15T18:31:59.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:31:59.988 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:31:59.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.995+0000 7f4a20f06700 1 -- 172.21.15.42:0/1359908582 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4a08060f70 msgr2=0x7f4a08063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.995+0000 7f4a20f06700 1 --2- 172.21.15.42:0/1359908582 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4a08060f70 0x7f4a08063430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f4a040096c0 tx=0x7f4a04009be0).stop 2024-02-15T18:31:59.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.995+0000 7f4a20f06700 1 -- 172.21.15.42:0/1359908582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a1c0f6530 msgr2=0x7f4a1c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:31:59.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.995+0000 7f4a20f06700 1 --2- 172.21.15.42:0/1359908582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a1c0f6530 0x7f4a1c106e50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f4a0c004e30 tx=0x7f4a0c009b30).stop 2024-02-15T18:31:59.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.995+0000 7f4a20f06700 1 -- 172.21.15.42:0/1359908582 shutdown_connections 2024-02-15T18:31:59.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.995+0000 7f4a20f06700 1 --2- 172.21.15.42:0/1359908582 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4a08060f70 0x7f4a08063430 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.995+0000 7f4a20f06700 1 --2- 172.21.15.42:0/1359908582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a1c0f6530 0x7f4a1c106e50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.995+0000 7f4a20f06700 1 --2- 172.21.15.42:0/1359908582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1c0f77d0 0x7f4a1c107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:31:59.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.995+0000 7f4a20f06700 1 -- 172.21.15.42:0/1359908582 >> 172.21.15.42:0/1359908582 conn(0x7f4a1c0f1cd0 msgr2=0x7f4a1c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:31:59.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.996+0000 7f4a20f06700 1 -- 172.21.15.42:0/1359908582 shutdown_connections 2024-02-15T18:31:59.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:31:59.996+0000 7f4a20f06700 1 -- 172.21.15.42:0/1359908582 wait complete. 2024-02-15T18:32:00.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.096+0000 7fbc5359e700 1 -- 172.21.15.42:0/3161828580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6530 msgr2=0x7fbc540f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.096+0000 7fbc5359e700 1 --2- 172.21.15.42:0/3161828580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6530 0x7fbc540f6950 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fbc44004660 tx=0x7fbc44009b30).stop 2024-02-15T18:32:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.096+0000 7fbc5359e700 1 -- 172.21.15.42:0/3161828580 shutdown_connections 2024-02-15T18:32:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.096+0000 7fbc5359e700 1 --2- 172.21.15.42:0/3161828580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc540f77d0 0x7fbc540f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.096+0000 7fbc5359e700 1 --2- 172.21.15.42:0/3161828580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6530 0x7fbc540f6950 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.096+0000 7fbc5359e700 1 -- 172.21.15.42:0/3161828580 >> 172.21.15.42:0/3161828580 conn(0x7fbc540f1c90 msgr2=0x7fbc540f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.096+0000 7fbc5359e700 1 -- 172.21.15.42:0/3161828580 shutdown_connections 2024-02-15T18:32:00.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.096+0000 7fbc5359e700 1 -- 172.21.15.42:0/3161828580 wait complete. 2024-02-15T18:32:00.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.097+0000 7fbc5359e700 1 Processor -- start 2024-02-15T18:32:00.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.097+0000 7fbc5359e700 1 -- start start 2024-02-15T18:32:00.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.097+0000 7fbc5359e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6530 0x7fbc54106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:00.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc5359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc540f77d0 0x7fbc541073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:00.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc5359e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54107960 con 0x7fbc540f6530 2024-02-15T18:32:00.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc5359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54108160 con 0x7fbc540f77d0 2024-02-15T18:32:00.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc5259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6530 0x7fbc54106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:00.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc5259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6530 0x7fbc54106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42594/0 (socket says 172.21.15.42:42594) 2024-02-15T18:32:00.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc5259c700 1 -- 172.21.15.42:0/1250446569 learned_addr learned my addr 172.21.15.42:0/1250446569 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:32:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc51d9b700 1 --2- 172.21.15.42:0/1250446569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc540f77d0 0x7fbc541073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc5259c700 1 -- 172.21.15.42:0/1250446569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc540f77d0 msgr2=0x7fbc541073b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc5259c700 1 --2- 172.21.15.42:0/1250446569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc540f77d0 0x7fbc541073b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc5259c700 1 -- 172.21.15.42:0/1250446569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc44005040 con 0x7fbc540f6530 2024-02-15T18:32:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc51d9b700 1 --2- 172.21.15.42:0/1250446569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc540f77d0 0x7fbc541073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:32:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc5259c700 1 --2- 172.21.15.42:0/1250446569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6530 0x7fbc54106ec0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fbc44004e30 tx=0x7fbc44009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc42ffd700 1 -- 172.21.15.42:0/1250446569 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc44018070 con 0x7fbc540f6530 2024-02-15T18:32:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc42ffd700 1 -- 172.21.15.42:0/1250446569 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fbc44007ba0 con 0x7fbc540f6530 2024-02-15T18:32:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc42ffd700 1 -- 172.21.15.42:0/1250446569 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc4401c620 con 0x7fbc540f6530 2024-02-15T18:32:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc5359e700 1 -- 172.21.15.42:0/1250446569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc541083e0 con 0x7fbc540f6530 2024-02-15T18:32:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.098+0000 7fbc5359e700 1 -- 172.21.15.42:0/1250446569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc541088f0 con 0x7fbc540f6530 2024-02-15T18:32:00.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.099+0000 7fbc5359e700 1 -- 172.21.15.42:0/1250446569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc54059100 con 0x7fbc540f6530 2024-02-15T18:32:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.100+0000 7fbc42ffd700 1 -- 172.21.15.42:0/1250446569 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fbc4401d9a0 con 0x7fbc540f6530 2024-02-15T18:32:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.100+0000 7fbc42ffd700 1 --2- 172.21.15.42:0/1250446569 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc3c060f70 0x7fbc3c063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.100+0000 7fbc42ffd700 1 -- 172.21.15.42:0/1250446569 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6102+0+0 (secure 0 0 0) 0x7fbc4407dfc0 con 0x7fbc540f6530 2024-02-15T18:32:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.100+0000 7fbc51d9b700 1 --2- 172.21.15.42:0/1250446569 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc3c060f70 0x7fbc3c063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:00.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.101+0000 7fbc51d9b700 1 --2- 172.21.15.42:0/1250446569 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc3c060f70 0x7fbc3c063430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fbc48009740 tx=0x7fbc48009be0).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:00.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.104+0000 7fbc42ffd700 1 -- 172.21.15.42:0/1250446569 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbc44051d50 con 0x7fbc540f6530 2024-02-15T18:32:00.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:31:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:31:58.756350+0000 mgr.smithi042.kfoihb (mgr.24435) 283 : cluster [DBG] pgmap v237: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 127 B/s rd, 0 op/s; 30/210 objects degraded (14.286%) 2024-02-15T18:32:00.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:31:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:31:58.756350+0000 mgr.smithi042.kfoihb (mgr.24435) 283 : cluster [DBG] pgmap v237: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 127 B/s rd, 0 op/s; 30/210 objects degraded (14.286%) 2024-02-15T18:32:00.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.325+0000 7fbc5359e700 1 -- 172.21.15.42:0/1250446569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbc540fee00 con 0x7fbc540f6530 2024-02-15T18:32:00.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.326+0000 7fbc42ffd700 1 -- 172.21.15.42:0/1250446569 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1773 (secure 0 0 0) 0x7fbc44051940 con 0x7fbc540f6530 2024-02-15T18:32:00.319 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-02-15T18:32:00.319 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:32:00.319 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:32:00.319 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:32:00.319 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:32:00.319 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:32:00.319 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:32:00.319 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 11 2024-02-15T18:32:00.319 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:32:00.319 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:32:00.319 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:23:55.523717+0000 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24263} 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:32:00.320 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:32:00.321 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:32:00.321 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:32:00.321 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:32:00.321 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:32:00.321 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:32:00.321 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{0:24263} state up:active seq 2 join_fscid=1 addr [v2:172.21.15.191:6824/1971151000,v1:172.21.15.191:6825/1971151000] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:32:00.321 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{0:24277} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.191:6826/126692509,v1:172.21.15.191:6827/126692509] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:32:00.321 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:32:00.321 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:32:00.321 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:32:00.321 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:32:00.321 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{-1:14442} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6826/1892158366,v1:172.21.15.42:6827/1892158366] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:32:00.321 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:14450} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.42:6828/2580627796,v1:172.21.15.42:6829/2580627796] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:32:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.327+0000 7fbc5359e700 1 -- 172.21.15.42:0/1250446569 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc3c060f70 msgr2=0x7fbc3c063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.327+0000 7fbc5359e700 1 --2- 172.21.15.42:0/1250446569 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc3c060f70 0x7fbc3c063430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fbc48009740 tx=0x7fbc48009be0).stop 2024-02-15T18:32:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.327+0000 7fbc5359e700 1 -- 172.21.15.42:0/1250446569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6530 msgr2=0x7fbc54106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.327+0000 7fbc5359e700 1 --2- 172.21.15.42:0/1250446569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6530 0x7fbc54106ec0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fbc44004e30 tx=0x7fbc44009b30).stop 2024-02-15T18:32:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.328+0000 7fbc5359e700 1 -- 172.21.15.42:0/1250446569 shutdown_connections 2024-02-15T18:32:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.328+0000 7fbc5359e700 1 --2- 172.21.15.42:0/1250446569 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbc3c060f70 0x7fbc3c063430 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.328+0000 7fbc5359e700 1 --2- 172.21.15.42:0/1250446569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc540f6530 0x7fbc54106ec0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.328+0000 7fbc5359e700 1 --2- 172.21.15.42:0/1250446569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc540f77d0 0x7fbc541073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:00.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.328+0000 7fbc5359e700 1 -- 172.21.15.42:0/1250446569 >> 172.21.15.42:0/1250446569 conn(0x7fbc540f1c90 msgr2=0x7fbc540fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:00.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.328+0000 7fbc5359e700 1 -- 172.21.15.42:0/1250446569 shutdown_connections 2024-02-15T18:32:00.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.328+0000 7fbc5359e700 1 -- 172.21.15.42:0/1250446569 wait complete. 2024-02-15T18:32:00.323 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:32:00.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.430+0000 7ff4c975a700 1 -- 172.21.15.42:0/3809809243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4c40f66e0 msgr2=0x7ff4c40f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:00.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.430+0000 7ff4c975a700 1 --2- 172.21.15.42:0/3809809243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4c40f66e0 0x7ff4c40f6b00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7ff4b8004660 tx=0x7ff4b8009b30).stop 2024-02-15T18:32:00.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.430+0000 7ff4c975a700 1 -- 172.21.15.42:0/3809809243 shutdown_connections 2024-02-15T18:32:00.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.430+0000 7ff4c975a700 1 --2- 172.21.15.42:0/3809809243 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4c40f7980 0x7ff4c40f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:00.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.430+0000 7ff4c975a700 1 --2- 172.21.15.42:0/3809809243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4c40f66e0 0x7ff4c40f6b00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:00.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.430+0000 7ff4c975a700 1 -- 172.21.15.42:0/3809809243 >> 172.21.15.42:0/3809809243 conn(0x7ff4c40f1e40 msgr2=0x7ff4c40f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:00.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.430+0000 7ff4c975a700 1 -- 172.21.15.42:0/3809809243 shutdown_connections 2024-02-15T18:32:00.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.430+0000 7ff4c975a700 1 -- 172.21.15.42:0/3809809243 wait complete. 2024-02-15T18:32:00.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.431+0000 7ff4c975a700 1 Processor -- start 2024-02-15T18:32:00.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.431+0000 7ff4c975a700 1 -- start start 2024-02-15T18:32:00.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.431+0000 7ff4c975a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4c40f66e0 0x7ff4c4107030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:00.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.431+0000 7ff4c975a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4c40f7980 0x7ff4c4107520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:00.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.431+0000 7ff4c975a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c4107ad0 con 0x7ff4c40f66e0 2024-02-15T18:32:00.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.431+0000 7ff4c975a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c4107c10 con 0x7ff4c40f7980 2024-02-15T18:32:00.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.431+0000 7ff4c2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4c40f66e0 0x7ff4c4107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:00.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.431+0000 7ff4c2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4c40f66e0 0x7ff4c4107030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42612/0 (socket says 172.21.15.42:42612) 2024-02-15T18:32:00.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.431+0000 7ff4c2ffd700 1 -- 172.21.15.42:0/1007052761 learned_addr learned my addr 172.21.15.42:0/1007052761 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:32:00.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.431+0000 7ff4c27fc700 1 --2- 172.21.15.42:0/1007052761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4c40f7980 0x7ff4c4107520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:00.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.431+0000 7ff4c2ffd700 1 -- 172.21.15.42:0/1007052761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4c40f7980 msgr2=0x7ff4c4107520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:00.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.431+0000 7ff4c2ffd700 1 --2- 172.21.15.42:0/1007052761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4c40f7980 0x7ff4c4107520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:00.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.431+0000 7ff4c2ffd700 1 -- 172.21.15.42:0/1007052761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b8005040 con 0x7ff4c40f66e0 2024-02-15T18:32:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.431+0000 7ff4c2ffd700 1 --2- 172.21.15.42:0/1007052761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4c40f66e0 0x7ff4c4107030 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7ff4b8004e30 tx=0x7ff4b8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.432+0000 7ff4ab7fe700 1 -- 172.21.15.42:0/1007052761 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4b8018070 con 0x7ff4c40f66e0 2024-02-15T18:32:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.432+0000 7ff4ab7fe700 1 -- 172.21.15.42:0/1007052761 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7ff4b8007ba0 con 0x7ff4c40f66e0 2024-02-15T18:32:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.432+0000 7ff4ab7fe700 1 -- 172.21.15.42:0/1007052761 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4b801c3c0 con 0x7ff4c40f66e0 2024-02-15T18:32:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.432+0000 7ff4c975a700 1 -- 172.21.15.42:0/1007052761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c4108550 con 0x7ff4c40f66e0 2024-02-15T18:32:00.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.432+0000 7ff4c975a700 1 -- 172.21.15.42:0/1007052761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c4108ab0 con 0x7ff4c40f66e0 2024-02-15T18:32:00.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.433+0000 7ff4c975a700 1 -- 172.21.15.42:0/1007052761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c4045ad0 con 0x7ff4c40f66e0 2024-02-15T18:32:00.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.433+0000 7ff4ab7fe700 1 -- 172.21.15.42:0/1007052761 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7ff4b801d420 con 0x7ff4c40f66e0 2024-02-15T18:32:00.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.433+0000 7ff4ab7fe700 1 --2- 172.21.15.42:0/1007052761 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff4ac060f20 0x7ff4ac0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:00.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.434+0000 7ff4ab7fe700 1 -- 172.21.15.42:0/1007052761 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6102+0+0 (secure 0 0 0) 0x7ff4b807dd50 con 0x7ff4c40f66e0 2024-02-15T18:32:00.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.434+0000 7ff4c27fc700 1 --2- 172.21.15.42:0/1007052761 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff4ac060f20 0x7ff4ac0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:00.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.434+0000 7ff4c27fc700 1 --2- 172.21.15.42:0/1007052761 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff4ac060f20 0x7ff4ac0633e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7ff4b4004c40 tx=0x7ff4b400b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:00.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.438+0000 7ff4ab7fe700 1 -- 172.21.15.42:0/1007052761 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff4b8051b60 con 0x7ff4c40f66e0 2024-02-15T18:32:00.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.631+0000 7ff4c975a700 1 -- 172.21.15.42:0/1007052761 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4c40ff130 con 0x7ff4ac060f20 2024-02-15T18:32:00.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.632+0000 7ff4ab7fe700 1 -- 172.21.15.42:0/1007052761 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff4c40ff130 con 0x7ff4ac060f20 2024-02-15T18:32:00.625 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:32:00.625 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:32:00.625 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:32:00.625 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:32:00.625 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:32:00.625 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-02-15T18:32:00.626 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-02-15T18:32:00.626 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-02-15T18:32:00.626 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-02-15T18:32:00.626 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:32:00.626 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "12/21 daemons upgraded", 2024-02-15T18:32:00.626 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading osd daemons", 2024-02-15T18:32:00.626 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:32:00.626 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:32:00.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.634+0000 7ff4c975a700 1 -- 172.21.15.42:0/1007052761 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff4ac060f20 msgr2=0x7ff4ac0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:00.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.634+0000 7ff4c975a700 1 --2- 172.21.15.42:0/1007052761 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff4ac060f20 0x7ff4ac0633e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7ff4b4004c40 tx=0x7ff4b400b040).stop 2024-02-15T18:32:00.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.634+0000 7ff4c975a700 1 -- 172.21.15.42:0/1007052761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4c40f66e0 msgr2=0x7ff4c4107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:00.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.634+0000 7ff4c975a700 1 --2- 172.21.15.42:0/1007052761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4c40f66e0 0x7ff4c4107030 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7ff4b8004e30 tx=0x7ff4b8009b30).stop 2024-02-15T18:32:00.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.634+0000 7ff4c975a700 1 -- 172.21.15.42:0/1007052761 shutdown_connections 2024-02-15T18:32:00.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.634+0000 7ff4c975a700 1 --2- 172.21.15.42:0/1007052761 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff4ac060f20 0x7ff4ac0633e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:00.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.634+0000 7ff4c975a700 1 --2- 172.21.15.42:0/1007052761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4c40f66e0 0x7ff4c4107030 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:00.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.634+0000 7ff4c975a700 1 --2- 172.21.15.42:0/1007052761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4c40f7980 0x7ff4c4107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:00.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.634+0000 7ff4c975a700 1 -- 172.21.15.42:0/1007052761 >> 172.21.15.42:0/1007052761 conn(0x7ff4c40f1e40 msgr2=0x7ff4c40fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:00.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.634+0000 7ff4c975a700 1 -- 172.21.15.42:0/1007052761 shutdown_connections 2024-02-15T18:32:00.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:00.634+0000 7ff4c975a700 1 -- 172.21.15.42:0/1007052761 wait complete. 2024-02-15T18:32:01.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:59.033149+0000 mgr.smithi042.kfoihb (mgr.24435) 284 : audit [DBG] from='client.34326 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:01.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:59.336880+0000 mgr.smithi042.kfoihb (mgr.24435) 285 : audit [DBG] from='client.34330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:01.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:59.631257+0000 mgr.smithi042.kfoihb (mgr.24435) 286 : audit [DBG] from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:01.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:31:59.994614+0000 mon.smithi042 (mon.0) 454 : audit [DBG] from='client.? 172.21.15.42:0/1359908582' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:01.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.327076+0000 mon.smithi042 (mon.0) 455 : audit [DBG] from='client.? 172.21.15.42:0/1250446569' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:32:01.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.498281+0000 mon.smithi042 (mon.0) 456 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:01.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.806682+0000 mon.smithi042 (mon.0) 457 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:01.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.807383+0000 mon.smithi042 (mon.0) 458 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.811543+0000 mon.smithi042 (mon.0) 459 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.816402+0000 mon.smithi042 (mon.0) 460 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.818609+0000 mon.smithi042 (mon.0) 461 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.819736+0000 mon.smithi042 (mon.0) 462 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.820824+0000 mon.smithi042 (mon.0) 463 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.822213+0000 mon.smithi042 (mon.0) 464 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.826166+0000 mon.smithi042 (mon.0) 465 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.826802+0000 mon.smithi042 (mon.0) 466 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.829796+0000 mon.smithi042 (mon.0) 467 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.830389+0000 mon.smithi042 (mon.0) 468 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.833335+0000 mon.smithi042 (mon.0) 469 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.833878+0000 mon.smithi042 (mon.0) 470 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.836484+0000 mon.smithi042 (mon.0) 471 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.837059+0000 mon.smithi042 (mon.0) 472 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-15T18:32:01.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.839660+0000 mon.smithi042 (mon.0) 473 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-15T18:32:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.840226+0000 mon.smithi042 (mon.0) 474 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-15T18:32:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.842774+0000 mon.smithi042 (mon.0) 475 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-15T18:32:01.195 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.843339+0000 mon.smithi042 (mon.0) 476 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-15T18:32:01.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:59.033149+0000 mgr.smithi042.kfoihb (mgr.24435) 284 : audit [DBG] from='client.34326 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:01.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:59.336880+0000 mgr.smithi042.kfoihb (mgr.24435) 285 : audit [DBG] from='client.34330 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:01.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:59.631257+0000 mgr.smithi042.kfoihb (mgr.24435) 286 : audit [DBG] from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:01.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:31:59.994614+0000 mon.smithi042 (mon.0) 454 : audit [DBG] from='client.? 172.21.15.42:0/1359908582' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:01.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.327076+0000 mon.smithi042 (mon.0) 455 : audit [DBG] from='client.? 172.21.15.42:0/1250446569' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:32:01.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.498281+0000 mon.smithi042 (mon.0) 456 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:01.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.806682+0000 mon.smithi042 (mon.0) 457 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.807383+0000 mon.smithi042 (mon.0) 458 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.811543+0000 mon.smithi042 (mon.0) 459 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.816402+0000 mon.smithi042 (mon.0) 460 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.818609+0000 mon.smithi042 (mon.0) 461 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.819736+0000 mon.smithi042 (mon.0) 462 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.820824+0000 mon.smithi042 (mon.0) 463 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.822213+0000 mon.smithi042 (mon.0) 464 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.826166+0000 mon.smithi042 (mon.0) 465 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.826802+0000 mon.smithi042 (mon.0) 466 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.829796+0000 mon.smithi042 (mon.0) 467 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.830389+0000 mon.smithi042 (mon.0) 468 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.833335+0000 mon.smithi042 (mon.0) 469 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.833878+0000 mon.smithi042 (mon.0) 470 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.836484+0000 mon.smithi042 (mon.0) 471 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-15T18:32:01.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.837059+0000 mon.smithi042 (mon.0) 472 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-15T18:32:01.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.839660+0000 mon.smithi042 (mon.0) 473 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-15T18:32:01.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.840226+0000 mon.smithi042 (mon.0) 474 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-15T18:32:01.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.842774+0000 mon.smithi042 (mon.0) 475 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-15T18:32:01.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.843339+0000 mon.smithi042 (mon.0) 476 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-15T18:32:02.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.633134+0000 mgr.smithi042.kfoihb (mgr.24435) 287 : audit [DBG] from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:02.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:00.757012+0000 mgr.smithi042.kfoihb (mgr.24435) 288 : cluster [DBG] pgmap v238: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 30/210 objects degraded (14.286%) 2024-02-15T18:32:02.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:00.807953+0000 mgr.smithi042.kfoihb (mgr.24435) 289 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:32:02.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:00.822739+0000 mgr.smithi042.kfoihb (mgr.24435) 290 : cephadm [INF] Upgrade: Setting container_image for all osd 2024-02-15T18:32:02.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.846164+0000 mon.smithi042 (mon.0) 477 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-15T18:32:02.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:00.847958+0000 mgr.smithi042.kfoihb (mgr.24435) 291 : cephadm [INF] Upgrade: Disabling standby-replay for filesystem cephfs 2024-02-15T18:32:02.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.852756+0000 mon.smithi042 (mon.0) 478 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:02.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:00.853135+0000 mon.smithi042 (mon.0) 479 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-02-15T18:32:02.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:00.856927+0000 mon.smithi042 (mon.0) 480 : cluster [DBG] osdmap e108: 6 total, 5 up, 6 in 2024-02-15T18:32:02.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.633134+0000 mgr.smithi042.kfoihb (mgr.24435) 287 : audit [DBG] from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:02.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:00.757012+0000 mgr.smithi042.kfoihb (mgr.24435) 288 : cluster [DBG] pgmap v238: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 30/210 objects degraded (14.286%) 2024-02-15T18:32:02.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:00.807953+0000 mgr.smithi042.kfoihb (mgr.24435) 289 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:32:02.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:00.822739+0000 mgr.smithi042.kfoihb (mgr.24435) 290 : cephadm [INF] Upgrade: Setting container_image for all osd 2024-02-15T18:32:02.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.846164+0000 mon.smithi042 (mon.0) 477 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-15T18:32:02.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:00.847958+0000 mgr.smithi042.kfoihb (mgr.24435) 291 : cephadm [INF] Upgrade: Disabling standby-replay for filesystem cephfs 2024-02-15T18:32:02.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.852756+0000 mon.smithi042 (mon.0) 478 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:02.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:00.853135+0000 mon.smithi042 (mon.0) 479 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-02-15T18:32:02.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:00.856927+0000 mon.smithi042 (mon.0) 480 : cluster [DBG] osdmap e108: 6 total, 5 up, 6 in 2024-02-15T18:32:02.849 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:01.855344+0000 mon.smithi042 (mon.0) 481 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-02-15T18:32:02.850 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:01.855464+0000 mon.smithi042 (mon.0) 482 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 2 up:standby 2024-02-15T18:32:02.850 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:01.856419+0000 mon.smithi042 (mon.0) 483 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:02.850 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:01.858297+0000 mon.smithi042 (mon.0) 484 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:02.850 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:01.859158+0000 mon.smithi042 (mon.0) 485 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:32:02.850 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:01.859926+0000 mgr.smithi042.kfoihb (mgr.24435) 292 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:32:02.850 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:01.863545+0000 mon.smithi042 (mon.0) 486 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:03.035 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:01.869564+0000 mon.smithi042 (mon.0) 487 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:03.035 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:01.872189+0000 mon.smithi042 (mon.0) 488 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:03.035 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:01.873606+0000 mon.smithi042 (mon.0) 489 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:03.035 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:01.874935+0000 mon.smithi042 (mon.0) 490 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:03.035 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:01.876665+0000 mon.smithi042 (mon.0) 491 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:03.035 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:01.878552+0000 mon.smithi042 (mon.0) 492 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi042.vtwvlo"]}]: dispatch 2024-02-15T18:32:03.035 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:01.879194+0000 mgr.smithi042.kfoihb (mgr.24435) 293 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi042.vtwvlo 2024-02-15T18:32:03.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:01.855344+0000 mon.smithi042 (mon.0) 481 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-02-15T18:32:03.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:01.855464+0000 mon.smithi042 (mon.0) 482 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 2 up:standby 2024-02-15T18:32:03.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:01.856419+0000 mon.smithi042 (mon.0) 483 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:03.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:01.858297+0000 mon.smithi042 (mon.0) 484 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:03.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:01.859158+0000 mon.smithi042 (mon.0) 485 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:32:03.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:01.859926+0000 mgr.smithi042.kfoihb (mgr.24435) 292 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:32:03.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:01.863545+0000 mon.smithi042 (mon.0) 486 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:03.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:01.869564+0000 mon.smithi042 (mon.0) 487 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:03.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:01.872189+0000 mon.smithi042 (mon.0) 488 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:03.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:01.873606+0000 mon.smithi042 (mon.0) 489 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:03.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:01.874935+0000 mon.smithi042 (mon.0) 490 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:03.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:01.876665+0000 mon.smithi042 (mon.0) 491 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:03.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:01.878552+0000 mon.smithi042 (mon.0) 492 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi042.vtwvlo"]}]: dispatch 2024-02-15T18:32:03.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:01.879194+0000 mgr.smithi042.kfoihb (mgr.24435) 293 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi042.vtwvlo 2024-02-15T18:32:04.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:02.757651+0000 mgr.smithi042.kfoihb (mgr.24435) 294 : cluster [DBG] pgmap v240: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 684 B/s rd, 1 op/s; 30/210 objects degraded (14.286%) 2024-02-15T18:32:04.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:02.866774+0000 mon.smithi042 (mon.0) 493 : cluster [DBG] mds.? [v2:172.21.15.191:6826/1725194059,v1:172.21.15.191:6827/1725194059] up:boot 2024-02-15T18:32:04.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:02.866824+0000 mon.smithi042 (mon.0) 494 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 3 up:standby 2024-02-15T18:32:04.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:02.866926+0000 mon.smithi042 (mon.0) 495 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.erlpxp"}]: dispatch 2024-02-15T18:32:04.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:03.496051+0000 mon.smithi042 (mon.0) 496 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:04.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:03.496960+0000 mon.smithi042 (mon.0) 497 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.vtwvlo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:04.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:03.497725+0000 mon.smithi042 (mon.0) 498 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:04.247 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:02.757651+0000 mgr.smithi042.kfoihb (mgr.24435) 294 : cluster [DBG] pgmap v240: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 684 B/s rd, 1 op/s; 30/210 objects degraded (14.286%) 2024-02-15T18:32:04.247 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:02.866774+0000 mon.smithi042 (mon.0) 493 : cluster [DBG] mds.? [v2:172.21.15.191:6826/1725194059,v1:172.21.15.191:6827/1725194059] up:boot 2024-02-15T18:32:04.248 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:02.866824+0000 mon.smithi042 (mon.0) 494 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 3 up:standby 2024-02-15T18:32:04.248 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:02.866926+0000 mon.smithi042 (mon.0) 495 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.erlpxp"}]: dispatch 2024-02-15T18:32:04.248 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:03.496051+0000 mon.smithi042 (mon.0) 496 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:04.248 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:03.496960+0000 mon.smithi042 (mon.0) 497 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.vtwvlo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:04.248 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:03.497725+0000 mon.smithi042 (mon.0) 498 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:04.692 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:32:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:32:04.408+0000 7fb32c75f380 -1 osd.5 105 log_to_monitors {default=true} 2024-02-15T18:32:04.957 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:03.491446+0000 mgr.smithi042.kfoihb (mgr.24435) 295 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi042.vtwvlo 2024-02-15T18:32:04.958 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:03.498394+0000 mgr.smithi042.kfoihb (mgr.24435) 296 : cephadm [INF] Deploying daemon mds.cephfs.smithi042.vtwvlo on smithi042 2024-02-15T18:32:04.958 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:04.413478+0000 mon.smithi191 (mon.1) 10 : audit [INF] from='osd.5 [v2:172.21.15.191:6816/4219355336,v1:172.21.15.191:6817/4219355336]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:32:04.958 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:04.413919+0000 mon.smithi042 (mon.0) 499 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:32:05.192 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:32:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:32:04.889+0000 7fb31bc42700 -1 osd.5 105 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-15T18:32:05.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:03.491446+0000 mgr.smithi042.kfoihb (mgr.24435) 295 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi042.vtwvlo 2024-02-15T18:32:05.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:03.498394+0000 mgr.smithi042.kfoihb (mgr.24435) 296 : cephadm [INF] Deploying daemon mds.cephfs.smithi042.vtwvlo on smithi042 2024-02-15T18:32:05.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:04.413478+0000 mon.smithi191 (mon.1) 10 : audit [INF] from='osd.5 [v2:172.21.15.191:6816/4219355336,v1:172.21.15.191:6817/4219355336]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:32:05.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:04.413919+0000 mon.smithi042 (mon.0) 499 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-15T18:32:06.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:04.758273+0000 mgr.smithi042.kfoihb (mgr.24435) 297 : cluster [DBG] pgmap v241: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 614 B/s rd, 1 op/s; 30/210 objects degraded (14.286%) 2024-02-15T18:32:06.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:04.875730+0000 mon.smithi042 (mon.0) 500 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-15T18:32:06.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:04.875800+0000 mon.smithi042 (mon.0) 501 : cluster [DBG] osdmap e109: 6 total, 5 up, 6 in 2024-02-15T18:32:06.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:04.876303+0000 mon.smithi191 (mon.1) 11 : audit [INF] from='osd.5 [v2:172.21.15.191:6816/4219355336,v1:172.21.15.191:6817/4219355336]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:32:06.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:04.876768+0000 mon.smithi042 (mon.0) 502 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:32:06.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:05.244435+0000 mon.smithi042 (mon.0) 503 : cluster [WRN] daemon mds.cephfs.smithi042.vtwvlo compat changed unexpectedly 2024-02-15T18:32:06.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:05.244868+0000 mon.smithi042 (mon.0) 504 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:32:06.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:05.248775+0000 mon.smithi042 (mon.0) 505 : cluster [INF] osd.5 [v2:172.21.15.191:6816/4219355336,v1:172.21.15.191:6817/4219355336] boot 2024-02-15T18:32:06.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:05.248824+0000 mon.smithi042 (mon.0) 506 : cluster [DBG] osdmap e110: 6 total, 6 up, 6 in 2024-02-15T18:32:06.194 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:05.249006+0000 mon.smithi042 (mon.0) 507 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:32:06.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:04.758273+0000 mgr.smithi042.kfoihb (mgr.24435) 297 : cluster [DBG] pgmap v241: 65 pgs: 18 active+undersized, 12 active+undersized+degraded, 35 active+clean; 193 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 614 B/s rd, 1 op/s; 30/210 objects degraded (14.286%) 2024-02-15T18:32:06.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:04.875730+0000 mon.smithi042 (mon.0) 500 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-15T18:32:06.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:04.875800+0000 mon.smithi042 (mon.0) 501 : cluster [DBG] osdmap e109: 6 total, 5 up, 6 in 2024-02-15T18:32:06.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:04.876303+0000 mon.smithi191 (mon.1) 11 : audit [INF] from='osd.5 [v2:172.21.15.191:6816/4219355336,v1:172.21.15.191:6817/4219355336]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:32:06.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:04.876768+0000 mon.smithi042 (mon.0) 502 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-15T18:32:06.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:05.244435+0000 mon.smithi042 (mon.0) 503 : cluster [WRN] daemon mds.cephfs.smithi042.vtwvlo compat changed unexpectedly 2024-02-15T18:32:06.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:05.244868+0000 mon.smithi042 (mon.0) 504 : cluster [INF] Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-15T18:32:06.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:05.248775+0000 mon.smithi042 (mon.0) 505 : cluster [INF] osd.5 [v2:172.21.15.191:6816/4219355336,v1:172.21.15.191:6817/4219355336] boot 2024-02-15T18:32:06.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:05.248824+0000 mon.smithi042 (mon.0) 506 : cluster [DBG] osdmap e110: 6 total, 6 up, 6 in 2024-02-15T18:32:06.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:05.249006+0000 mon.smithi042 (mon.0) 507 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-15T18:32:07.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:05.879614+0000 mon.smithi042 (mon.0) 508 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 2 up:standby 2024-02-15T18:32:07.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:05.987204+0000 mon.smithi042 (mon.0) 509 : cluster [WRN] Health check update: Degraded data redundancy: 30/210 objects degraded (14.286%), 12 pgs degraded (PG_DEGRADED) 2024-02-15T18:32:07.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:06.249957+0000 mon.smithi042 (mon.0) 510 : cluster [DBG] osdmap e111: 6 total, 6 up, 6 in 2024-02-15T18:32:07.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:06 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:06.691443+0000 mon.smithi042 (mon.0) 511 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.vtwvlo"}]: dispatch 2024-02-15T18:32:07.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:05.879614+0000 mon.smithi042 (mon.0) 508 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 2 up:standby 2024-02-15T18:32:07.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:05.987204+0000 mon.smithi042 (mon.0) 509 : cluster [WRN] Health check update: Degraded data redundancy: 30/210 objects degraded (14.286%), 12 pgs degraded (PG_DEGRADED) 2024-02-15T18:32:07.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:06.249957+0000 mon.smithi042 (mon.0) 510 : cluster [DBG] osdmap e111: 6 total, 6 up, 6 in 2024-02-15T18:32:07.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:06 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:06.691443+0000 mon.smithi042 (mon.0) 511 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.vtwvlo"}]: dispatch 2024-02-15T18:32:07.913 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:06.758821+0000 mgr.smithi042.kfoihb (mgr.24435) 298 : cluster [DBG] pgmap v245: 65 pgs: 4 peering, 15 active+undersized, 11 active+undersized+degraded, 35 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 346 B/s rd, 0 op/s; 27/210 objects degraded (12.857%) 2024-02-15T18:32:08.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:06.758821+0000 mgr.smithi042.kfoihb (mgr.24435) 298 : cluster [DBG] pgmap v245: 65 pgs: 4 peering, 15 active+undersized, 11 active+undersized+degraded, 35 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 346 B/s rd, 0 op/s; 27/210 objects degraded (12.857%) 2024-02-15T18:32:10.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:08.759370+0000 mgr.smithi042.kfoihb (mgr.24435) 299 : cluster [DBG] pgmap v246: 65 pgs: 4 peering, 12 active+undersized, 8 active+undersized+degraded, 41 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 341 B/s rd, 0 op/s; 23/210 objects degraded (10.952%) 2024-02-15T18:32:10.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:09.246056+0000 mon.smithi042 (mon.0) 512 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:10.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:09.246819+0000 mon.smithi042 (mon.0) 513 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:10.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:08.759370+0000 mgr.smithi042.kfoihb (mgr.24435) 299 : cluster [DBG] pgmap v246: 65 pgs: 4 peering, 12 active+undersized, 8 active+undersized+degraded, 41 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 341 B/s rd, 0 op/s; 23/210 objects degraded (10.952%) 2024-02-15T18:32:10.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:09.246056+0000 mon.smithi042 (mon.0) 512 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:10.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:09.246819+0000 mon.smithi042 (mon.0) 513 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:11.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:09.885736+0000 mon.smithi042 (mon.0) 514 : cluster [DBG] mds.? [v2:172.21.15.42:6826/2962563293,v1:172.21.15.42:6827/2962563293] up:boot 2024-02-15T18:32:11.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:09.885785+0000 mon.smithi042 (mon.0) 515 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 3 up:standby 2024-02-15T18:32:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:09.886069+0000 mon.smithi042 (mon.0) 516 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.vtwvlo"}]: dispatch 2024-02-15T18:32:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:10.837994+0000 mon.smithi042 (mon.0) 517 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:11.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:09.885736+0000 mon.smithi042 (mon.0) 514 : cluster [DBG] mds.? [v2:172.21.15.42:6826/2962563293,v1:172.21.15.42:6827/2962563293] up:boot 2024-02-15T18:32:11.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:09.885785+0000 mon.smithi042 (mon.0) 515 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 3 up:standby 2024-02-15T18:32:11.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:09.886069+0000 mon.smithi042 (mon.0) 516 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.vtwvlo"}]: dispatch 2024-02-15T18:32:11.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:10.837994+0000 mon.smithi042 (mon.0) 517 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:12.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:10.759878+0000 mgr.smithi042.kfoihb (mgr.24435) 300 : cluster [DBG] pgmap v247: 65 pgs: 2 peering, 63 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 170 B/s rd, 0 op/s 2024-02-15T18:32:12.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:10.887058+0000 mon.smithi042 (mon.0) 518 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/210 objects degraded (10.952%), 8 pgs degraded) 2024-02-15T18:32:12.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:10.887114+0000 mon.smithi042 (mon.0) 519 : cluster [INF] Cluster is now healthy 2024-02-15T18:32:12.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:10.759878+0000 mgr.smithi042.kfoihb (mgr.24435) 300 : cluster [DBG] pgmap v247: 65 pgs: 2 peering, 63 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 170 B/s rd, 0 op/s 2024-02-15T18:32:12.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:10.887058+0000 mon.smithi042 (mon.0) 518 : cluster [INF] Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/210 objects degraded (10.952%), 8 pgs degraded) 2024-02-15T18:32:12.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:10.887114+0000 mon.smithi042 (mon.0) 519 : cluster [INF] Cluster is now healthy 2024-02-15T18:32:14.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:12.760402+0000 mgr.smithi042.kfoihb (mgr.24435) 301 : cluster [DBG] pgmap v248: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-02-15T18:32:14.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:12.760402+0000 mgr.smithi042.kfoihb (mgr.24435) 301 : cluster [DBG] pgmap v248: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-02-15T18:32:16.012 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:14.760763+0000 mgr.smithi042.kfoihb (mgr.24435) 302 : cluster [DBG] pgmap v249: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-02-15T18:32:16.012 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:14.923546+0000 mon.smithi042 (mon.0) 520 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:16.012 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:14.924361+0000 mon.smithi042 (mon.0) 521 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:16.012 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:14.925047+0000 mon.smithi042 (mon.0) 522 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:32:16.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:14.925630+0000 mgr.smithi042.kfoihb (mgr.24435) 303 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:32:16.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:14.928559+0000 mon.smithi042 (mon.0) 523 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:16.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:14.933391+0000 mon.smithi042 (mon.0) 524 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:16.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:14.935602+0000 mon.smithi042 (mon.0) 525 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:16.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:14.936718+0000 mon.smithi042 (mon.0) 526 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:16.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:14.937806+0000 mon.smithi042 (mon.0) 527 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:16.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:14.939241+0000 mon.smithi042 (mon.0) 528 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:16.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:14.940839+0000 mon.smithi042 (mon.0) 529 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi042.idnxkc"]}]: dispatch 2024-02-15T18:32:16.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:14.941345+0000 mgr.smithi042.kfoihb (mgr.24435) 304 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi042.idnxkc 2024-02-15T18:32:16.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:14.760763+0000 mgr.smithi042.kfoihb (mgr.24435) 302 : cluster [DBG] pgmap v249: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-02-15T18:32:16.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:14.923546+0000 mon.smithi042 (mon.0) 520 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:16.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:14.924361+0000 mon.smithi042 (mon.0) 521 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:16.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:14.925047+0000 mon.smithi042 (mon.0) 522 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:32:16.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:14.925630+0000 mgr.smithi042.kfoihb (mgr.24435) 303 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:32:16.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:14.928559+0000 mon.smithi042 (mon.0) 523 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:16.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:14.933391+0000 mon.smithi042 (mon.0) 524 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:16.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:14.935602+0000 mon.smithi042 (mon.0) 525 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:16.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:14.936718+0000 mon.smithi042 (mon.0) 526 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:16.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:14.937806+0000 mon.smithi042 (mon.0) 527 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:16.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:14.939241+0000 mon.smithi042 (mon.0) 528 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:16.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:14.940839+0000 mon.smithi042 (mon.0) 529 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi042.idnxkc"]}]: dispatch 2024-02-15T18:32:16.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:14.941345+0000 mgr.smithi042.kfoihb (mgr.24435) 304 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi042.idnxkc 2024-02-15T18:32:17.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:16.383584+0000 mgr.smithi042.kfoihb (mgr.24435) 305 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi042.idnxkc 2024-02-15T18:32:17.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:16.388203+0000 mon.smithi042 (mon.0) 530 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:17.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:16.389098+0000 mon.smithi042 (mon.0) 531 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.idnxkc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:17.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:16.389952+0000 mon.smithi042 (mon.0) 532 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:17.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:16.390631+0000 mgr.smithi042.kfoihb (mgr.24435) 306 : cephadm [INF] Deploying daemon mds.cephfs.smithi042.idnxkc on smithi042 2024-02-15T18:32:17.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:16.761262+0000 mgr.smithi042.kfoihb (mgr.24435) 307 : cluster [DBG] pgmap v250: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:17.716 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:16.383584+0000 mgr.smithi042.kfoihb (mgr.24435) 305 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi042.idnxkc 2024-02-15T18:32:17.716 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:16.388203+0000 mon.smithi042 (mon.0) 530 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:17.716 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:16.389098+0000 mon.smithi042 (mon.0) 531 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.idnxkc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:17.716 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:16.389952+0000 mon.smithi042 (mon.0) 532 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:17.716 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:16.390631+0000 mgr.smithi042.kfoihb (mgr.24435) 306 : cephadm [INF] Deploying daemon mds.cephfs.smithi042.idnxkc on smithi042 2024-02-15T18:32:17.716 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:16.761262+0000 mgr.smithi042.kfoihb (mgr.24435) 307 : cluster [DBG] pgmap v250: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:18.692 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:17.995020+0000 mon.smithi042 (mon.0) 533 : cluster [WRN] daemon mds.cephfs.smithi042.idnxkc compat changed unexpectedly 2024-02-15T18:32:18.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:17.998709+0000 mon.smithi042 (mon.0) 534 : cluster [DBG] osdmap e112: 6 total, 6 up, 6 in 2024-02-15T18:32:18.805 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:17.995020+0000 mon.smithi042 (mon.0) 533 : cluster [WRN] daemon mds.cephfs.smithi042.idnxkc compat changed unexpectedly 2024-02-15T18:32:18.805 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:17.998709+0000 mon.smithi042 (mon.0) 534 : cluster [DBG] osdmap e112: 6 total, 6 up, 6 in 2024-02-15T18:32:19.692 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:18.397147+0000 mon.smithi042 (mon.0) 535 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 2 up:standby 2024-02-15T18:32:19.692 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:18.761606+0000 mgr.smithi042.kfoihb (mgr.24435) 308 : cluster [DBG] pgmap v252: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:19.805 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:18.397147+0000 mon.smithi042 (mon.0) 535 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 2 up:standby 2024-02-15T18:32:19.805 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:18.761606+0000 mgr.smithi042.kfoihb (mgr.24435) 308 : cluster [DBG] pgmap v252: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:22.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:20.762172+0000 mgr.smithi042.kfoihb (mgr.24435) 309 : cluster [DBG] pgmap v253: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:22.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:20.988613+0000 mon.smithi042 (mon.0) 536 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:22.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:20.762172+0000 mgr.smithi042.kfoihb (mgr.24435) 309 : cluster [DBG] pgmap v253: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:22.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:20.988613+0000 mon.smithi042 (mon.0) 536 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:23.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:22.762524+0000 mgr.smithi042.kfoihb (mgr.24435) 310 : cluster [DBG] pgmap v254: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:23.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:22.763856+0000 mon.smithi042 (mon.0) 537 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:23.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:22.764632+0000 mon.smithi042 (mon.0) 538 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:23.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:22.819584+0000 mon.smithi042 (mon.0) 539 : cluster [DBG] mds.? [v2:172.21.15.42:6828/1562993914,v1:172.21.15.42:6829/1562993914] up:boot 2024-02-15T18:32:23.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:22.819648+0000 mon.smithi042 (mon.0) 540 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 3 up:standby 2024-02-15T18:32:23.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:22.819763+0000 mon.smithi042 (mon.0) 541 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.idnxkc"}]: dispatch 2024-02-15T18:32:24.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:22.762524+0000 mgr.smithi042.kfoihb (mgr.24435) 310 : cluster [DBG] pgmap v254: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:24.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:22.763856+0000 mon.smithi042 (mon.0) 537 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:24.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:22.764632+0000 mon.smithi042 (mon.0) 538 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:24.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:22.819584+0000 mon.smithi042 (mon.0) 539 : cluster [DBG] mds.? [v2:172.21.15.42:6828/1562993914,v1:172.21.15.42:6829/1562993914] up:boot 2024-02-15T18:32:24.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:22.819648+0000 mon.smithi042 (mon.0) 540 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi191.icwcnb=up:active} 3 up:standby 2024-02-15T18:32:24.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:22.819763+0000 mon.smithi042 (mon.0) 541 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.idnxkc"}]: dispatch 2024-02-15T18:32:26.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:24.762865+0000 mgr.smithi042.kfoihb (mgr.24435) 311 : cluster [DBG] pgmap v255: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:26.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:24.762865+0000 mgr.smithi042.kfoihb (mgr.24435) 311 : cluster [DBG] pgmap v255: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:27.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:26 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:25.838237+0000 mon.smithi042 (mon.0) 542 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:27.237 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:26 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:25.838237+0000 mon.smithi042 (mon.0) 542 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:28.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:26.763300+0000 mgr.smithi042.kfoihb (mgr.24435) 312 : cluster [DBG] pgmap v256: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:28.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:26.763300+0000 mgr.smithi042.kfoihb (mgr.24435) 312 : cluster [DBG] pgmap v256: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:29.013 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:27.915858+0000 mon.smithi042 (mon.0) 543 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:29.013 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:27.916627+0000 mon.smithi042 (mon.0) 544 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:29.013 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:27.917230+0000 mon.smithi042 (mon.0) 545 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:32:29.013 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:27.917742+0000 mgr.smithi042.kfoihb (mgr.24435) 313 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:32:29.013 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:27.920725+0000 mon.smithi042 (mon.0) 546 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:29.014 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:27.925339+0000 mon.smithi042 (mon.0) 547 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:29.014 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:27.927314+0000 mon.smithi042 (mon.0) 548 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:29.014 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:27.928340+0000 mon.smithi042 (mon.0) 549 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:29.014 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:27.929344+0000 mon.smithi042 (mon.0) 550 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:29.014 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:27.930638+0000 mon.smithi042 (mon.0) 551 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:29.014 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:27.932124+0000 mon.smithi042 (mon.0) 552 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi191.icwcnb"]}]: dispatch 2024-02-15T18:32:29.014 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:28 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:27.932586+0000 mgr.smithi042.kfoihb (mgr.24435) 314 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi191.icwcnb 2024-02-15T18:32:29.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:27.915858+0000 mon.smithi042 (mon.0) 543 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:29.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:27.916627+0000 mon.smithi042 (mon.0) 544 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:29.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:27.917230+0000 mon.smithi042 (mon.0) 545 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:32:29.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:27.917742+0000 mgr.smithi042.kfoihb (mgr.24435) 313 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:32:29.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:27.920725+0000 mon.smithi042 (mon.0) 546 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:29.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:27.925339+0000 mon.smithi042 (mon.0) 547 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:29.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:27.927314+0000 mon.smithi042 (mon.0) 548 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:29.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:27.928340+0000 mon.smithi042 (mon.0) 549 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:29.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:27.929344+0000 mon.smithi042 (mon.0) 550 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:29.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:27.930638+0000 mon.smithi042 (mon.0) 551 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:29.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:27.932124+0000 mon.smithi042 (mon.0) 552 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi191.icwcnb"]}]: dispatch 2024-02-15T18:32:29.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:28 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:27.932586+0000 mgr.smithi042.kfoihb (mgr.24435) 314 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi191.icwcnb 2024-02-15T18:32:30.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:28.763791+0000 mgr.smithi042.kfoihb (mgr.24435) 315 : cluster [DBG] pgmap v257: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:30.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:29.462403+0000 mon.smithi042 (mon.0) 553 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:30.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:29.463509+0000 mon.smithi042 (mon.0) 554 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi191.icwcnb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:30.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:29.464506+0000 mon.smithi042 (mon.0) 555 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:30.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:28.763791+0000 mgr.smithi042.kfoihb (mgr.24435) 315 : cluster [DBG] pgmap v257: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:30.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:29.462403+0000 mon.smithi042 (mon.0) 553 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:30.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:29.463509+0000 mon.smithi042 (mon.0) 554 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi191.icwcnb", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:30.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:29.464506+0000 mon.smithi042 (mon.0) 555 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.750+0000 7f135900e700 1 -- 172.21.15.42:0/1832940063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13540f7980 msgr2=0x7f13540f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.750+0000 7f135900e700 1 --2- 172.21.15.42:0/1832940063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13540f7980 0x7f13540f7de0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f1348004660 tx=0x7f1348009b30).stop 2024-02-15T18:32:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.750+0000 7f135900e700 1 -- 172.21.15.42:0/1832940063 shutdown_connections 2024-02-15T18:32:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.750+0000 7f135900e700 1 --2- 172.21.15.42:0/1832940063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13540f7980 0x7f13540f7de0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.750+0000 7f135900e700 1 --2- 172.21.15.42:0/1832940063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13540f66e0 0x7f13540f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.750+0000 7f135900e700 1 -- 172.21.15.42:0/1832940063 >> 172.21.15.42:0/1832940063 conn(0x7f13540f1e40 msgr2=0x7f13540f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.750+0000 7f135900e700 1 -- 172.21.15.42:0/1832940063 shutdown_connections 2024-02-15T18:32:30.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.750+0000 7f135900e700 1 -- 172.21.15.42:0/1832940063 wait complete. 2024-02-15T18:32:30.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.751+0000 7f135900e700 1 Processor -- start 2024-02-15T18:32:30.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.751+0000 7f135900e700 1 -- start start 2024-02-15T18:32:30.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.751+0000 7f135900e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13540f66e0 0x7f1354107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:30.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.751+0000 7f135900e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13540f7980 0x7f13541074f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:30.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.751+0000 7f135900e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1354107aa0 con 0x7f13540f7980 2024-02-15T18:32:30.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.751+0000 7f135900e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1354107be0 con 0x7f13540f66e0 2024-02-15T18:32:30.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.751+0000 7f135259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13540f7980 0x7f13541074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:30.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.751+0000 7f1352d9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13540f66e0 0x7f1354107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:30.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.751+0000 7f135259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13540f7980 0x7f13541074f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50498/0 (socket says 172.21.15.42:50498) 2024-02-15T18:32:30.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.751+0000 7f135259c700 1 -- 172.21.15.42:0/1289528582 learned_addr learned my addr 172.21.15.42:0/1289528582 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:32:30.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.752+0000 7f135259c700 1 -- 172.21.15.42:0/1289528582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13540f66e0 msgr2=0x7f1354107000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:30.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.752+0000 7f135259c700 1 --2- 172.21.15.42:0/1289528582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13540f66e0 0x7f1354107000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:30.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.752+0000 7f135259c700 1 -- 172.21.15.42:0/1289528582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1348005040 con 0x7f13540f7980 2024-02-15T18:32:30.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.752+0000 7f135259c700 1 --2- 172.21.15.42:0/1289528582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13540f7980 0x7f13541074f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f1348009f50 tx=0x7f1348009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:30.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.752+0000 7f133b7fe700 1 -- 172.21.15.42:0/1289528582 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1348018070 con 0x7f13540f7980 2024-02-15T18:32:30.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.752+0000 7f133b7fe700 1 -- 172.21.15.42:0/1289528582 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f1348007ba0 con 0x7f13540f7980 2024-02-15T18:32:30.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.752+0000 7f135900e700 1 -- 172.21.15.42:0/1289528582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1354108520 con 0x7f13540f7980 2024-02-15T18:32:30.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.752+0000 7f133b7fe700 1 -- 172.21.15.42:0/1289528582 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f134801c420 con 0x7f13540f7980 2024-02-15T18:32:30.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.752+0000 7f135900e700 1 -- 172.21.15.42:0/1289528582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1354108a00 con 0x7f13540f7980 2024-02-15T18:32:30.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.753+0000 7f135900e700 1 -- 172.21.15.42:0/1289528582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1354045ad0 con 0x7f13540f7980 2024-02-15T18:32:30.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.754+0000 7f133b7fe700 1 -- 172.21.15.42:0/1289528582 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f134801d420 con 0x7f13540f7980 2024-02-15T18:32:30.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.754+0000 7f133b7fe700 1 --2- 172.21.15.42:0/1289528582 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f133c060f20 0x7f133c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:30.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.754+0000 7f133b7fe700 1 -- 172.21.15.42:0/1289528582 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6102+0+0 (secure 0 0 0) 0x7f134807ddf0 con 0x7f13540f7980 2024-02-15T18:32:30.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.754+0000 7f1352d9d700 1 --2- 172.21.15.42:0/1289528582 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f133c060f20 0x7f133c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:30.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.755+0000 7f1352d9d700 1 --2- 172.21.15.42:0/1289528582 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f133c060f20 0x7f133c0633e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f13540f77e0 tx=0x7f134400b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:30.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.758+0000 7f133b7fe700 1 -- 172.21.15.42:0/1289528582 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1348051ba0 con 0x7f13540f7980 2024-02-15T18:32:30.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.949+0000 7f135900e700 1 -- 172.21.15.42:0/1289528582 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13540ff120 con 0x7f133c060f20 2024-02-15T18:32:30.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.950+0000 7f133b7fe700 1 -- 172.21.15.42:0/1289528582 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f13540ff120 con 0x7f133c060f20 2024-02-15T18:32:30.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.951+0000 7f135900e700 1 -- 172.21.15.42:0/1289528582 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f133c060f20 msgr2=0x7f133c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:30.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.951+0000 7f135900e700 1 --2- 172.21.15.42:0/1289528582 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f133c060f20 0x7f133c0633e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f13540f77e0 tx=0x7f134400b040).stop 2024-02-15T18:32:30.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.951+0000 7f135900e700 1 -- 172.21.15.42:0/1289528582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13540f7980 msgr2=0x7f13541074f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:30.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.951+0000 7f135900e700 1 --2- 172.21.15.42:0/1289528582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13540f7980 0x7f13541074f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f1348009f50 tx=0x7f1348009b30).stop 2024-02-15T18:32:30.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.952+0000 7f135900e700 1 -- 172.21.15.42:0/1289528582 shutdown_connections 2024-02-15T18:32:30.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.952+0000 7f135900e700 1 --2- 172.21.15.42:0/1289528582 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f133c060f20 0x7f133c0633e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:30.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.952+0000 7f135900e700 1 --2- 172.21.15.42:0/1289528582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f13540f66e0 0x7f1354107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:30.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.952+0000 7f135900e700 1 --2- 172.21.15.42:0/1289528582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13540f7980 0x7f13541074f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:30.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.952+0000 7f135900e700 1 -- 172.21.15.42:0/1289528582 >> 172.21.15.42:0/1289528582 conn(0x7f13540f1e40 msgr2=0x7f13540fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:30.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.952+0000 7f135900e700 1 -- 172.21.15.42:0/1289528582 shutdown_connections 2024-02-15T18:32:30.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:30.952+0000 7f135900e700 1 -- 172.21.15.42:0/1289528582 wait complete. 2024-02-15T18:32:30.955 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:32:31.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.054+0000 7f44f02bc700 1 -- 172.21.15.42:0/1664262793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44e80f9880 msgr2=0x7f44e80fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:31.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.054+0000 7f44f02bc700 1 --2- 172.21.15.42:0/1664262793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44e80f9880 0x7f44e80fbc70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f44d8004660 tx=0x7f44d8009b30).stop 2024-02-15T18:32:31.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.054+0000 7f44f02bc700 1 -- 172.21.15.42:0/1664262793 shutdown_connections 2024-02-15T18:32:31.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.054+0000 7f44f02bc700 1 --2- 172.21.15.42:0/1664262793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44e80f9880 0x7f44e80fbc70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.054+0000 7f44f02bc700 1 --2- 172.21.15.42:0/1664262793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44e80602b0 0x7f44e80f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.054+0000 7f44f02bc700 1 -- 172.21.15.42:0/1664262793 >> 172.21.15.42:0/1664262793 conn(0x7f44e80f0d40 msgr2=0x7f44e80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:31.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.055+0000 7f44f02bc700 1 -- 172.21.15.42:0/1664262793 shutdown_connections 2024-02-15T18:32:31.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.055+0000 7f44f02bc700 1 -- 172.21.15.42:0/1664262793 wait complete. 2024-02-15T18:32:31.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.055+0000 7f44f02bc700 1 Processor -- start 2024-02-15T18:32:31.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.055+0000 7f44f02bc700 1 -- start start 2024-02-15T18:32:31.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.055+0000 7f44f02bc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44e80602b0 0x7f44e80f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:31.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.055+0000 7f44f02bc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44e80f9880 0x7f44e80f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:31.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.055+0000 7f44f02bc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80f5760 con 0x7f44e80f9880 2024-02-15T18:32:31.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.055+0000 7f44f02bc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80f58a0 con 0x7f44e80602b0 2024-02-15T18:32:31.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.055+0000 7f44ed857700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44e80f9880 0x7f44e80f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:31.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.055+0000 7f44ed857700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44e80f9880 0x7f44e80f5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50502/0 (socket says 172.21.15.42:50502) 2024-02-15T18:32:31.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.055+0000 7f44ed857700 1 -- 172.21.15.42:0/3554418145 learned_addr learned my addr 172.21.15.42:0/3554418145 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:32:31.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.055+0000 7f44ee058700 1 --2- 172.21.15.42:0/3554418145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44e80602b0 0x7f44e80f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:31.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.056+0000 7f44ed857700 1 -- 172.21.15.42:0/3554418145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44e80602b0 msgr2=0x7f44e80f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:31.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.056+0000 7f44ed857700 1 --2- 172.21.15.42:0/3554418145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44e80602b0 0x7f44e80f6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.056+0000 7f44ed857700 1 -- 172.21.15.42:0/3554418145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d8005040 con 0x7f44e80f9880 2024-02-15T18:32:31.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.056+0000 7f44ed857700 1 --2- 172.21.15.42:0/3554418145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44e80f9880 0x7f44e80f5200 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f44d8004e30 tx=0x7f44d8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:31.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.056+0000 7f44deffd700 1 -- 172.21.15.42:0/3554418145 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44d8018070 con 0x7f44e80f9880 2024-02-15T18:32:31.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.056+0000 7f44deffd700 1 -- 172.21.15.42:0/3554418145 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f44d8007ba0 con 0x7f44e80f9880 2024-02-15T18:32:31.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.056+0000 7f44f02bc700 1 -- 172.21.15.42:0/3554418145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e80f5b20 con 0x7f44e80f9880 2024-02-15T18:32:31.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.056+0000 7f44deffd700 1 -- 172.21.15.42:0/3554418145 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44d801c3c0 con 0x7f44e80f9880 2024-02-15T18:32:31.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.056+0000 7f44f02bc700 1 -- 172.21.15.42:0/3554418145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e80f6000 con 0x7f44e80f9880 2024-02-15T18:32:31.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.057+0000 7f44f02bc700 1 -- 172.21.15.42:0/3554418145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e8045ad0 con 0x7f44e80f9880 2024-02-15T18:32:31.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.057+0000 7f44deffd700 1 -- 172.21.15.42:0/3554418145 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f44d801d420 con 0x7f44e80f9880 2024-02-15T18:32:31.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.058+0000 7f44deffd700 1 --2- 172.21.15.42:0/3554418145 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f44d4060f20 0x7f44d40633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:31.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.058+0000 7f44deffd700 1 -- 172.21.15.42:0/3554418145 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6102+0+0 (secure 0 0 0) 0x7f44d807de20 con 0x7f44e80f9880 2024-02-15T18:32:31.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.058+0000 7f44ee058700 1 --2- 172.21.15.42:0/3554418145 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f44d4060f20 0x7f44d40633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:31.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.058+0000 7f44ee058700 1 --2- 172.21.15.42:0/3554418145 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f44d4060f20 0x7f44d40633e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f44e80f6b10 tx=0x7f44e400b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:31.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.062+0000 7f44deffd700 1 -- 172.21.15.42:0/3554418145 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f44d804e230 con 0x7f44e80f9880 2024-02-15T18:32:31.182 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:29.457432+0000 mgr.smithi042.kfoihb (mgr.24435) 316 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi191.icwcnb 2024-02-15T18:32:31.182 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:30 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:29.465348+0000 mgr.smithi042.kfoihb (mgr.24435) 317 : cephadm [INF] Deploying daemon mds.cephfs.smithi191.icwcnb on smithi191 2024-02-15T18:32:31.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.254+0000 7f44f02bc700 1 -- 172.21.15.42:0/3554418145 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f44e80ff0a0 con 0x7f44d4060f20 2024-02-15T18:32:31.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.255+0000 7f44deffd700 1 -- 172.21.15.42:0/3554418145 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f44e80ff0a0 con 0x7f44d4060f20 2024-02-15T18:32:31.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.257+0000 7f44f02bc700 1 -- 172.21.15.42:0/3554418145 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f44d4060f20 msgr2=0x7f44d40633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:31.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.257+0000 7f44f02bc700 1 --2- 172.21.15.42:0/3554418145 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f44d4060f20 0x7f44d40633e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f44e80f6b10 tx=0x7f44e400b040).stop 2024-02-15T18:32:31.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.257+0000 7f44f02bc700 1 -- 172.21.15.42:0/3554418145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44e80f9880 msgr2=0x7f44e80f5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:31.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.257+0000 7f44f02bc700 1 --2- 172.21.15.42:0/3554418145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44e80f9880 0x7f44e80f5200 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f44d8004e30 tx=0x7f44d8009b30).stop 2024-02-15T18:32:31.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.257+0000 7f44f02bc700 1 -- 172.21.15.42:0/3554418145 shutdown_connections 2024-02-15T18:32:31.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.257+0000 7f44f02bc700 1 --2- 172.21.15.42:0/3554418145 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f44d4060f20 0x7f44d40633e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.257+0000 7f44f02bc700 1 --2- 172.21.15.42:0/3554418145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44e80602b0 0x7f44e80f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.257+0000 7f44f02bc700 1 --2- 172.21.15.42:0/3554418145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f44e80f9880 0x7f44e80f5200 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.257+0000 7f44f02bc700 1 -- 172.21.15.42:0/3554418145 >> 172.21.15.42:0/3554418145 conn(0x7f44e80f0d40 msgr2=0x7f44e80f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:31.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.258+0000 7f44f02bc700 1 -- 172.21.15.42:0/3554418145 shutdown_connections 2024-02-15T18:32:31.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.258+0000 7f44f02bc700 1 -- 172.21.15.42:0/3554418145 wait complete. 2024-02-15T18:32:31.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:29.457432+0000 mgr.smithi042.kfoihb (mgr.24435) 316 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi191.icwcnb 2024-02-15T18:32:31.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:30 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:29.465348+0000 mgr.smithi042.kfoihb (mgr.24435) 317 : cephadm [INF] Deploying daemon mds.cephfs.smithi191.icwcnb on smithi191 2024-02-15T18:32:31.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.359+0000 7f0f8ed69700 1 -- 172.21.15.42:0/282373618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f88060310 msgr2=0x7f0f880f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:31.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.359+0000 7f0f8ed69700 1 --2- 172.21.15.42:0/282373618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f88060310 0x7f0f880f9110 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f0f800099e0 tx=0x7f0f80004d10).stop 2024-02-15T18:32:31.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.359+0000 7f0f8ed69700 1 -- 172.21.15.42:0/282373618 shutdown_connections 2024-02-15T18:32:31.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.359+0000 7f0f8ed69700 1 --2- 172.21.15.42:0/282373618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f880f9690 0x7f0f880fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:32:31.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.359+0000 7f0f8ed69700 1 --2- 172.21.15.42:0/282373618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f88060310 0x7f0f880f9110 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.359+0000 7f0f8ed69700 1 -- 172.21.15.42:0/282373618 >> 172.21.15.42:0/282373618 conn(0x7f0f880f0b90 msgr2=0x7f0f880f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:31.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.359+0000 7f0f8ed69700 1 -- 172.21.15.42:0/282373618 shutdown_connections 2024-02-15T18:32:31.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.360+0000 7f0f8ed69700 1 -- 172.21.15.42:0/282373618 wait complete. 2024-02-15T18:32:31.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.360+0000 7f0f8ed69700 1 Processor -- start 2024-02-15T18:32:31.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.360+0000 7f0f8ed69700 1 -- start start 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.360+0000 7f0f8ed69700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f88060310 0x7f0f880f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f8ed69700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f880f9690 0x7f0f880f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f8ed69700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f880f5600 con 0x7f0f88060310 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f8ed69700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f880f5740 con 0x7f0f880f9690 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f8dd67700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f88060310 0x7f0f880f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f8dd67700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f88060310 0x7f0f880f6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50526/0 (socket says 172.21.15.42:50526) 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f8dd67700 1 -- 172.21.15.42:0/3978561174 learned_addr learned my addr 172.21.15.42:0/3978561174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f8d566700 1 --2- 172.21.15.42:0/3978561174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f880f9690 0x7f0f880f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f8dd67700 1 -- 172.21.15.42:0/3978561174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f880f9690 msgr2=0x7f0f880f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f8dd67700 1 --2- 172.21.15.42:0/3978561174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f880f9690 0x7f0f880f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f8dd67700 1 -- 172.21.15.42:0/3978561174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f80009640 con 0x7f0f88060310 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f8dd67700 1 --2- 172.21.15.42:0/3978561174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f88060310 0x7f0f880f6b90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f0f80007e40 tx=0x7f0f80004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f867fc700 1 -- 172.21.15.42:0/3978561174 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f8000e030 con 0x7f0f88060310 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f8ed69700 1 -- 172.21.15.42:0/3978561174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f880f59c0 con 0x7f0f88060310 2024-02-15T18:32:31.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f867fc700 1 -- 172.21.15.42:0/3978561174 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f0f8000bea0 con 0x7f0f88060310 2024-02-15T18:32:31.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f867fc700 1 -- 172.21.15.42:0/3978561174 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f80018070 con 0x7f0f88060310 2024-02-15T18:32:31.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.361+0000 7f0f8ed69700 1 -- 172.21.15.42:0/3978561174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8810f060 con 0x7f0f88060310 2024-02-15T18:32:31.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.362+0000 7f0f8ed69700 1 -- 172.21.15.42:0/3978561174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f88059100 con 0x7f0f88060310 2024-02-15T18:32:31.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.363+0000 7f0f867fc700 1 -- 172.21.15.42:0/3978561174 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f0f8001db80 con 0x7f0f88060310 2024-02-15T18:32:31.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.363+0000 7f0f867fc700 1 --2- 172.21.15.42:0/3978561174 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0f70060f70 0x7f0f70063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:31.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.363+0000 7f0f867fc700 1 -- 172.21.15.42:0/3978561174 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f0f8007e290 con 0x7f0f88060310 2024-02-15T18:32:31.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.363+0000 7f0f8d566700 1 --2- 172.21.15.42:0/3978561174 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0f70060f70 0x7f0f70063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:31.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.364+0000 7f0f8d566700 1 --2- 172.21.15.42:0/3978561174 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0f70060f70 0x7f0f70063430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f0f880600e0 tx=0x7f0f7c008040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.368+0000 7f0f867fc700 1 -- 172.21.15.42:0/3978561174 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0f8004e610 con 0x7f0f88060310 2024-02-15T18:32:31.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.552+0000 7f0f8ed69700 1 -- 172.21.15.42:0/3978561174 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f880fef90 con 0x7f0f70060f70 2024-02-15T18:32:31.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.562+0000 7f0f867fc700 1 -- 172.21.15.42:0/3978561174 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f0f880fef90 con 0x7f0f70060f70 2024-02-15T18:32:31.555 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:32:31.555 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (12m) 3s ago 14m 21.4M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:32:31.555 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (4m) 3s ago 14m 6643k - 16.2.14-521-g0e714d9a 1fbc92c859dc 2da9d875539a 2024-02-15T18:32:31.555 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (4m) 31s ago 13m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 5c46a68ff62b 2024-02-15T18:32:31.555 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (12m) 3s ago 13m 33.9M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:32:31.555 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (9s) 3s ago 8m 12.2M - 16.2.14-521-g0e714d9a 1fbc92c859dc 39e47ffbe32b 2024-02-15T18:32:31.555 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (22s) 3s ago 8m 11.5M - 16.2.14-521-g0e714d9a 1fbc92c859dc 372b8d41fced 2024-02-15T18:32:31.555 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (8m) 31s ago 8m 110M - 16.2.4 8d91d370c2b8 2bcdc26ba314 2024-02-15T18:32:31.555 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (8m) 31s ago 8m 153M - 16.2.4 8d91d370c2b8 6af6af573376 2024-02-15T18:32:31.555 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (6m) 3s ago 15m 438M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:32:31.555 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (6m) 31s ago 12m 389M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:32:31.555 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (5m) 3s ago 15m 59.7M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:32:31.556 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (5m) 31s ago 12m 48.0M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:32:31.556 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (13m) 3s ago 13m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:32:31.556 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (12m) 31s ago 12m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:32:31.556 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (4m) 3s ago 12m 310M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc f226bea76cab 2024-02-15T18:32:31.556 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (3m) 3s ago 11m 271M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 5114ed026724 2024-02-15T18:32:31.556 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (2m) 3s ago 11m 190M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 2e26b1ec292a 2024-02-15T18:32:31.556 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (2m) 31s ago 11m 331M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc c71b210091cb 2024-02-15T18:32:31.556 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (78s) 31s ago 10m 299M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 83bd9cdcb3da 2024-02-15T18:32:31.556 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (34s) 31s ago 10m 13.2M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc d45e566ed288 2024-02-15T18:32:31.556 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (6m) 3s ago 13m 63.6M - 2.18.1 de242295e225 63b6dfb825ed 2024-02-15T18:32:31.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.564+0000 7f0f8ed69700 1 -- 172.21.15.42:0/3978561174 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0f70060f70 msgr2=0x7f0f70063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:31.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.564+0000 7f0f8ed69700 1 --2- 172.21.15.42:0/3978561174 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0f70060f70 0x7f0f70063430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f0f880600e0 tx=0x7f0f7c008040).stop 2024-02-15T18:32:31.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.564+0000 7f0f8ed69700 1 -- 172.21.15.42:0/3978561174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f88060310 msgr2=0x7f0f880f6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:31.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.564+0000 7f0f8ed69700 1 --2- 172.21.15.42:0/3978561174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f88060310 0x7f0f880f6b90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f0f80007e40 tx=0x7f0f80004d10).stop 2024-02-15T18:32:31.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.564+0000 7f0f8ed69700 1 -- 172.21.15.42:0/3978561174 shutdown_connections 2024-02-15T18:32:31.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.564+0000 7f0f8ed69700 1 --2- 172.21.15.42:0/3978561174 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0f70060f70 0x7f0f70063430 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.564+0000 7f0f8ed69700 1 --2- 172.21.15.42:0/3978561174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f88060310 0x7f0f880f6b90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.564+0000 7f0f8ed69700 1 --2- 172.21.15.42:0/3978561174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f880f9690 0x7f0f880f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.564+0000 7f0f8ed69700 1 -- 172.21.15.42:0/3978561174 >> 172.21.15.42:0/3978561174 conn(0x7f0f880f0b90 msgr2=0x7f0f880f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:31.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.564+0000 7f0f8ed69700 1 -- 172.21.15.42:0/3978561174 shutdown_connections 2024-02-15T18:32:31.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.564+0000 7f0f8ed69700 1 -- 172.21.15.42:0/3978561174 wait complete. 2024-02-15T18:32:31.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.664+0000 7fcb449a9700 1 -- 172.21.15.42:0/3619632233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb400f6530 msgr2=0x7fcb400f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:31.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.664+0000 7fcb449a9700 1 --2- 172.21.15.42:0/3619632233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb400f6530 0x7fcb400f6950 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fcb30004660 tx=0x7fcb30009b30).stop 2024-02-15T18:32:31.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.664+0000 7fcb449a9700 1 -- 172.21.15.42:0/3619632233 shutdown_connections 2024-02-15T18:32:31.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.664+0000 7fcb449a9700 1 --2- 172.21.15.42:0/3619632233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb400f77d0 0x7fcb400f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.664+0000 7fcb449a9700 1 --2- 172.21.15.42:0/3619632233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb400f6530 0x7fcb400f6950 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.664+0000 7fcb449a9700 1 -- 172.21.15.42:0/3619632233 >> 172.21.15.42:0/3619632233 conn(0x7fcb400f1c90 msgr2=0x7fcb400f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:31.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.664+0000 7fcb449a9700 1 -- 172.21.15.42:0/3619632233 shutdown_connections 2024-02-15T18:32:31.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.665+0000 7fcb449a9700 1 -- 172.21.15.42:0/3619632233 wait complete. 2024-02-15T18:32:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.665+0000 7fcb449a9700 1 Processor -- start 2024-02-15T18:32:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.665+0000 7fcb449a9700 1 -- start start 2024-02-15T18:32:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.665+0000 7fcb449a9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb400f6530 0x7fcb40106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.665+0000 7fcb449a9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb400f77d0 0x7fcb401073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.665+0000 7fcb449a9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb40107960 con 0x7fcb400f77d0 2024-02-15T18:32:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.665+0000 7fcb449a9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb40108160 con 0x7fcb400f6530 2024-02-15T18:32:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.666+0000 7fcb3effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb400f77d0 0x7fcb401073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.666+0000 7fcb3effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb400f77d0 0x7fcb401073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52708/0 (socket says 172.21.15.42:52708) 2024-02-15T18:32:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.666+0000 7fcb3effd700 1 -- 172.21.15.42:0/2335258245 learned_addr learned my addr 172.21.15.42:0/2335258245 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:32:31.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.666+0000 7fcb3f7fe700 1 --2- 172.21.15.42:0/2335258245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb400f6530 0x7fcb40106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:31.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.666+0000 7fcb3effd700 1 -- 172.21.15.42:0/2335258245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb400f6530 msgr2=0x7fcb40106ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:31.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.666+0000 7fcb3effd700 1 --2- 172.21.15.42:0/2335258245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb400f6530 0x7fcb40106ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.666+0000 7fcb3effd700 1 -- 172.21.15.42:0/2335258245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb30005040 con 0x7fcb400f77d0 2024-02-15T18:32:31.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.666+0000 7fcb3effd700 1 --2- 172.21.15.42:0/2335258245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb400f77d0 0x7fcb401073b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fcb34002700 tx=0x7fcb34009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:31.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.666+0000 7fcb27fff700 1 -- 172.21.15.42:0/2335258245 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb34009320 con 0x7fcb400f77d0 2024-02-15T18:32:31.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.666+0000 7fcb449a9700 1 -- 172.21.15.42:0/2335258245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb40108440 con 0x7fcb400f77d0 2024-02-15T18:32:31.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.666+0000 7fcb27fff700 1 -- 172.21.15.42:0/2335258245 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fcb3400f040 con 0x7fcb400f77d0 2024-02-15T18:32:31.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.666+0000 7fcb27fff700 1 -- 172.21.15.42:0/2335258245 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb340149a0 con 0x7fcb400f77d0 2024-02-15T18:32:31.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.666+0000 7fcb449a9700 1 -- 172.21.15.42:0/2335258245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb40108980 con 0x7fcb400f77d0 2024-02-15T18:32:31.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.667+0000 7fcb449a9700 1 -- 172.21.15.42:0/2335258245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb40059100 con 0x7fcb400f77d0 2024-02-15T18:32:31.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.667+0000 7fcb27fff700 1 -- 172.21.15.42:0/2335258245 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fcb340077b0 con 0x7fcb400f77d0 2024-02-15T18:32:31.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.668+0000 7fcb27fff700 1 --2- 172.21.15.42:0/2335258245 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb28060f70 0x7fcb28063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:31.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.668+0000 7fcb27fff700 1 -- 172.21.15.42:0/2335258245 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fcb3407c510 con 0x7fcb400f77d0 2024-02-15T18:32:31.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.668+0000 7fcb3f7fe700 1 --2- 172.21.15.42:0/2335258245 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb28060f70 0x7fcb28063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:31.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.668+0000 7fcb3f7fe700 1 --2- 172.21.15.42:0/2335258245 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb28060f70 0x7fcb28063430 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fcb30004630 tx=0x7fcb30009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:31.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.672+0000 7fcb27fff700 1 -- 172.21.15.42:0/2335258245 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcb34050250 con 0x7fcb400f77d0 2024-02-15T18:32:31.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.914+0000 7fcb449a9700 1 -- 172.21.15.42:0/2335258245 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcb400feea0 con 0x7fcb400f77d0 2024-02-15T18:32:31.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.915+0000 7fcb27fff700 1 -- 172.21.15.42:0/2335258245 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7fcb3404fe40 con 0x7fcb400f77d0 2024-02-15T18:32:31.907 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:32:31.907 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:32:31.907 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:32:31.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:32:31.907 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:32:31.907 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:32:31.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:32:31.908 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:32:31.908 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 6 2024-02-15T18:32:31.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:32:31.908 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:32:31.908 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2, 2024-02-15T18:32:31.908 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:32:31.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:32:31.908 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:32:31.908 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 12, 2024-02-15T18:32:31.908 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2024-02-15T18:32:31.908 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:32:31.908 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:32:31.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.916+0000 7fcb449a9700 1 -- 172.21.15.42:0/2335258245 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb28060f70 msgr2=0x7fcb28063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:31.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.916+0000 7fcb449a9700 1 --2- 172.21.15.42:0/2335258245 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb28060f70 0x7fcb28063430 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fcb30004630 tx=0x7fcb30009b30).stop 2024-02-15T18:32:31.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.916+0000 7fcb449a9700 1 -- 172.21.15.42:0/2335258245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb400f77d0 msgr2=0x7fcb401073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:31.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.916+0000 7fcb449a9700 1 --2- 172.21.15.42:0/2335258245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb400f77d0 0x7fcb401073b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fcb34002700 tx=0x7fcb34009e00).stop 2024-02-15T18:32:31.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.916+0000 7fcb449a9700 1 -- 172.21.15.42:0/2335258245 shutdown_connections 2024-02-15T18:32:31.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.916+0000 7fcb449a9700 1 --2- 172.21.15.42:0/2335258245 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcb28060f70 0x7fcb28063430 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.916+0000 7fcb449a9700 1 --2- 172.21.15.42:0/2335258245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb400f6530 0x7fcb40106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.916+0000 7fcb449a9700 1 --2- 172.21.15.42:0/2335258245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb400f77d0 0x7fcb401073b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:31.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.916+0000 7fcb449a9700 1 -- 172.21.15.42:0/2335258245 >> 172.21.15.42:0/2335258245 conn(0x7fcb400f1c90 msgr2=0x7fcb400fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:31.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.917+0000 7fcb449a9700 1 -- 172.21.15.42:0/2335258245 shutdown_connections 2024-02-15T18:32:31.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:31.917+0000 7fcb449a9700 1 -- 172.21.15.42:0/2335258245 wait complete. 2024-02-15T18:32:32.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.027+0000 7f1958914700 1 -- 172.21.15.42:0/1577724809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19540f77d0 msgr2=0x7f19540f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:32.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.027+0000 7f1958914700 1 --2- 172.21.15.42:0/1577724809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19540f77d0 0x7f19540f7c30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f1948004660 tx=0x7f1948009b30).stop 2024-02-15T18:32:32.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.027+0000 7f1958914700 1 -- 172.21.15.42:0/1577724809 shutdown_connections 2024-02-15T18:32:32.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.027+0000 7f1958914700 1 --2- 172.21.15.42:0/1577724809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19540f77d0 0x7f19540f7c30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.027+0000 7f1958914700 1 --2- 172.21.15.42:0/1577724809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19540f6530 0x7f19540f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:32:32.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.027+0000 7f1958914700 1 -- 172.21.15.42:0/1577724809 >> 172.21.15.42:0/1577724809 conn(0x7f19540f1c90 msgr2=0x7f19540f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:32.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.027+0000 7f1958914700 1 -- 172.21.15.42:0/1577724809 shutdown_connections 2024-02-15T18:32:32.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.027+0000 7f1958914700 1 -- 172.21.15.42:0/1577724809 wait complete. 2024-02-15T18:32:32.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.028+0000 7f1958914700 1 Processor -- start 2024-02-15T18:32:32.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.028+0000 7f1958914700 1 -- start start 2024-02-15T18:32:32.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.028+0000 7f1958914700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19540f6530 0x7f1954106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:32.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.029+0000 7f1958914700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19540f77d0 0x7f19541073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:32.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.029+0000 7f1958914700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1954107a40 con 0x7f19540f77d0 2024-02-15T18:32:32.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.029+0000 7f1958914700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19541081c0 con 0x7f19540f6530 2024-02-15T18:32:32.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.029+0000 7f1952d9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19540f6530 0x7f1954106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:32.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.029+0000 7f195259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19540f77d0 0x7f19541073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:32.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.029+0000 7f1952d9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19540f6530 0x7f1954106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.42:53382/0 (socket says 172.21.15.42:53382) 2024-02-15T18:32:32.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.029+0000 7f1952d9d700 1 -- 172.21.15.42:0/227440699 learned_addr learned my addr 172.21.15.42:0/227440699 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:32:32.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.029+0000 7f1952d9d700 1 -- 172.21.15.42:0/227440699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19540f77d0 msgr2=0x7f19541073b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:32.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.029+0000 7f1952d9d700 1 --2- 172.21.15.42:0/227440699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19540f77d0 0x7f19541073b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.029+0000 7f1952d9d700 1 -- 172.21.15.42:0/227440699 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1948005040 con 0x7f19540f6530 2024-02-15T18:32:32.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.029+0000 7f195259c700 1 --2- 172.21.15.42:0/227440699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19540f77d0 0x7f19541073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:32:32.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.029+0000 7f1952d9d700 1 --2- 172.21.15.42:0/227440699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19540f6530 0x7f1954106ec0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f19440093f0 tx=0x7f1944006ec0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:32.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.030+0000 7f193b7fe700 1 -- 172.21.15.42:0/227440699 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1944013640 con 0x7f19540f6530 2024-02-15T18:32:32.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.030+0000 7f1958914700 1 -- 172.21.15.42:0/227440699 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1954108440 con 0x7f19540f6530 2024-02-15T18:32:32.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.030+0000 7f1958914700 1 -- 172.21.15.42:0/227440699 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1954108980 con 0x7f19540f6530 2024-02-15T18:32:32.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.030+0000 7f193b7fe700 1 -- 172.21.15.42:0/227440699 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f1944013c90 con 0x7f19540f6530 2024-02-15T18:32:32.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.030+0000 7f193b7fe700 1 -- 172.21.15.42:0/227440699 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1944012de0 con 0x7f19540f6530 2024-02-15T18:32:32.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.031+0000 7f1958914700 1 -- 172.21.15.42:0/227440699 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1934004fa0 con 0x7f19540f6530 2024-02-15T18:32:32.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.031+0000 7f193b7fe700 1 -- 172.21.15.42:0/227440699 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f194401a060 con 0x7f19540f6530 2024-02-15T18:32:32.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.032+0000 7f193b7fe700 1 --2- 172.21.15.42:0/227440699 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f193c060f70 0x7f193c063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:32.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.032+0000 7f193b7fe700 1 -- 172.21.15.42:0/227440699 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f194407c110 con 0x7f19540f6530 2024-02-15T18:32:32.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.032+0000 7f195259c700 1 --2- 172.21.15.42:0/227440699 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f193c060f70 0x7f193c063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:32.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.032+0000 7f195259c700 1 --2- 172.21.15.42:0/227440699 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f193c060f70 0x7f193c063430 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f1948013fd0 tx=0x7f1948009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:32.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.036+0000 7f193b7fe700 1 -- 172.21.15.42:0/227440699 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f194404c420 con 0x7f19540f6530 2024-02-15T18:32:32.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:30.764197+0000 mgr.smithi042.kfoihb (mgr.24435) 318 : cluster [DBG] pgmap v258: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:32.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:30.950680+0000 mgr.smithi042.kfoihb (mgr.24435) 319 : audit [DBG] from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:32.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:31.193685+0000 mon.smithi042 (mon.0) 556 : cluster [DBG] osdmap e113: 6 total, 6 up, 6 in 2024-02-15T18:32:32.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:31.915782+0000 mon.smithi042 (mon.0) 557 : audit [DBG] from='client.? 172.21.15.42:0/2335258245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:32.193 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:32:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:32:32.013+0000 7f490e4fa700 -1 bdev(0x55fa13f6a800 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 1 2024-02-15T18:32:32.193 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:32:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:32:32.014+0000 7f490ccf7700 -1 bdev(0x55fa13f6a800 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 1 2024-02-15T18:32:32.193 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:32:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:32:32.014+0000 7f490f4fc700 -1 bdev(0x55fa13f6a800 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 5 2024-02-15T18:32:32.193 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:32:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:32:32.016+0000 7f4911d01700 -1 bdev(0x55fa13f6a800 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 13 2024-02-15T18:32:32.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.260+0000 7f1958914700 1 -- 172.21.15.42:0/227440699 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1934005e80 con 0x7f19540f6530 2024-02-15T18:32:32.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.261+0000 7f193b7fe700 1 -- 172.21.15.42:0/227440699 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1593 (secure 0 0 0) 0x7f194404fa40 con 0x7f19540f6530 2024-02-15T18:32:32.253 INFO:teuthology.orchestra.run.smithi042.stdout:e21 2024-02-15T18:32:32.253 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:32:32.253 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:32:32.253 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 21 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:32:31.997905+0000 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 113 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:32:32.254 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:32:32.255 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:32:32.255 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=34356} 2024-02-15T18:32:32.255 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:32:32.255 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:32:32.255 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:32:32.255 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:32:32.255 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:32:32.255 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:32:32.255 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:32:32.255 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:32:32.255 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{0:34356} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2962563293,v1:172.21.15.42:6827/2962563293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:32:32.255 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:32:32.255 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:32:32.255 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:32:32.256 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:32:32.256 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/1562993914,v1:172.21.15.42:6829/1562993914] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:32:32.256 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{-1:44313} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.191:6826/1725194059,v1:172.21.15.191:6827/1725194059] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:32:32.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.262+0000 7f1958914700 1 -- 172.21.15.42:0/227440699 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f193c060f70 msgr2=0x7f193c063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:32.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.262+0000 7f1958914700 1 --2- 172.21.15.42:0/227440699 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f193c060f70 0x7f193c063430 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f1948013fd0 tx=0x7f1948009b30).stop 2024-02-15T18:32:32.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.262+0000 7f1958914700 1 -- 172.21.15.42:0/227440699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19540f6530 msgr2=0x7f1954106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:32.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.262+0000 7f1958914700 1 --2- 172.21.15.42:0/227440699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19540f6530 0x7f1954106ec0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f19440093f0 tx=0x7f1944006ec0).stop 2024-02-15T18:32:32.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.262+0000 7f1958914700 1 -- 172.21.15.42:0/227440699 shutdown_connections 2024-02-15T18:32:32.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.262+0000 7f1958914700 1 --2- 172.21.15.42:0/227440699 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f193c060f70 0x7f193c063430 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.262+0000 7f1958914700 1 --2- 172.21.15.42:0/227440699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19540f6530 0x7f1954106ec0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.262+0000 7f1958914700 1 --2- 172.21.15.42:0/227440699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19540f77d0 0x7f19541073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.262+0000 7f1958914700 1 -- 172.21.15.42:0/227440699 >> 172.21.15.42:0/227440699 conn(0x7f19540f1c90 msgr2=0x7f19540fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:32.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.263+0000 7f1958914700 1 -- 172.21.15.42:0/227440699 shutdown_connections 2024-02-15T18:32:32.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.263+0000 7f1958914700 1 -- 172.21.15.42:0/227440699 wait complete. 2024-02-15T18:32:32.257 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 21 2024-02-15T18:32:32.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:30.764197+0000 mgr.smithi042.kfoihb (mgr.24435) 318 : cluster [DBG] pgmap v258: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-02-15T18:32:32.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:30.950680+0000 mgr.smithi042.kfoihb (mgr.24435) 319 : audit [DBG] from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:32.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:31.193685+0000 mon.smithi042 (mon.0) 556 : cluster [DBG] osdmap e113: 6 total, 6 up, 6 in 2024-02-15T18:32:32.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:31.915782+0000 mon.smithi042 (mon.0) 557 : audit [DBG] from='client.? 172.21.15.42:0/2335258245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:32.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:32:31.993+0000 7fd934584700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-15T18:32:32.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.367+0000 7f551588f700 1 -- 172.21.15.42:0/3889191931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5510060310 msgr2=0x7f55100f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:32.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.367+0000 7f551588f700 1 --2- 172.21.15.42:0/3889191931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5510060310 0x7f55100f9110 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f5504004660 tx=0x7f5504009b30).stop 2024-02-15T18:32:32.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.367+0000 7f551588f700 1 -- 172.21.15.42:0/3889191931 shutdown_connections 2024-02-15T18:32:32.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.367+0000 7f551588f700 1 --2- 172.21.15.42:0/3889191931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55100f9690 0x7f55100fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.367+0000 7f551588f700 1 --2- 172.21.15.42:0/3889191931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5510060310 0x7f55100f9110 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.367+0000 7f551588f700 1 -- 172.21.15.42:0/3889191931 >> 172.21.15.42:0/3889191931 conn(0x7f55100f0b90 msgr2=0x7f55100f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:32.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.367+0000 7f551588f700 1 -- 172.21.15.42:0/3889191931 shutdown_connections 2024-02-15T18:32:32.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.367+0000 7f551588f700 1 -- 172.21.15.42:0/3889191931 wait complete. 2024-02-15T18:32:32.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.368+0000 7f551588f700 1 Processor -- start 2024-02-15T18:32:32.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.368+0000 7f551588f700 1 -- start start 2024-02-15T18:32:32.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.368+0000 7f551588f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5510060310 0x7f5510106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:32.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.368+0000 7f551588f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55100f9690 0x7f5510107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:32.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.368+0000 7f551588f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5510107900 con 0x7f5510060310 2024-02-15T18:32:32.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.368+0000 7f551588f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5510108110 con 0x7f55100f9690 2024-02-15T18:32:32.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.368+0000 7f551488d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5510060310 0x7f5510106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:32.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.368+0000 7f551488d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5510060310 0x7f5510106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52746/0 (socket says 172.21.15.42:52746) 2024-02-15T18:32:32.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.368+0000 7f551488d700 1 -- 172.21.15.42:0/2835309900 learned_addr learned my addr 172.21.15.42:0/2835309900 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:32:32.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.368+0000 7f550ffff700 1 --2- 172.21.15.42:0/2835309900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55100f9690 0x7f5510107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:32.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.369+0000 7f551488d700 1 -- 172.21.15.42:0/2835309900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55100f9690 msgr2=0x7f5510107300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:32.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.369+0000 7f551488d700 1 --2- 172.21.15.42:0/2835309900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55100f9690 0x7f5510107300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.369+0000 7f551488d700 1 -- 172.21.15.42:0/2835309900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5504005040 con 0x7f5510060310 2024-02-15T18:32:32.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.369+0000 7f551488d700 1 --2- 172.21.15.42:0/2835309900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5510060310 0x7f5510106e10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f5504000c00 tx=0x7f550400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:32.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.369+0000 7f550d7fa700 1 -- 172.21.15.42:0/2835309900 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5504019070 con 0x7f5510060310 2024-02-15T18:32:32.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.369+0000 7f550d7fa700 1 -- 172.21.15.42:0/2835309900 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f5504007e80 con 0x7f5510060310 2024-02-15T18:32:32.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.369+0000 7f551588f700 1 -- 172.21.15.42:0/2835309900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5510108330 con 0x7f5510060310 2024-02-15T18:32:32.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.369+0000 7f550d7fa700 1 -- 172.21.15.42:0/2835309900 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5504014db0 con 0x7f5510060310 2024-02-15T18:32:32.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.369+0000 7f551588f700 1 -- 172.21.15.42:0/2835309900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5510108890 con 0x7f5510060310 2024-02-15T18:32:32.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.370+0000 7f551588f700 1 -- 172.21.15.42:0/2835309900 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5510059100 con 0x7f5510060310 2024-02-15T18:32:32.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.371+0000 7f550d7fa700 1 -- 172.21.15.42:0/2835309900 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f5504025070 con 0x7f5510060310 2024-02-15T18:32:32.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.371+0000 7f550d7fa700 1 --2- 172.21.15.42:0/2835309900 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f5500060f10 0x7f55000633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:32:32.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.372+0000 7f550d7fa700 1 -- 172.21.15.42:0/2835309900 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f550407d1b0 con 0x7f5510060310 2024-02-15T18:32:32.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.372+0000 7f550ffff700 1 --2- 172.21.15.42:0/2835309900 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f5500060f10 0x7f55000633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:32:32.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.372+0000 7f550ffff700 1 --2- 172.21.15.42:0/2835309900 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f5500060f10 0x7f55000633d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f54fc009fd0 tx=0x7f54fc00b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:32:32.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.376+0000 7f550d7fa700 1 -- 172.21.15.42:0/2835309900 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f550404d4d0 con 0x7f5510060310 2024-02-15T18:32:32.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.589+0000 7f551588f700 1 -- 172.21.15.42:0/2835309900 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55100fee90 con 0x7f5500060f10 2024-02-15T18:32:32.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.590+0000 7f550d7fa700 1 -- 172.21.15.42:0/2835309900 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f55100fee90 con 0x7f5500060f10 2024-02-15T18:32:32.583 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:32:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:32:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:32:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:32:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:32:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-02-15T18:32:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-02-15T18:32:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-02-15T18:32:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-02-15T18:32:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:32:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "14/21 daemons upgraded", 2024-02-15T18:32:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-02-15T18:32:32.583 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:32:32.583 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:32:32.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.592+0000 7f551588f700 1 -- 172.21.15.42:0/2835309900 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f5500060f10 msgr2=0x7f55000633d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:32.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.592+0000 7f551588f700 1 --2- 172.21.15.42:0/2835309900 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f5500060f10 0x7f55000633d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f54fc009fd0 tx=0x7f54fc00b040).stop 2024-02-15T18:32:32.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.592+0000 7f551588f700 1 -- 172.21.15.42:0/2835309900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5510060310 msgr2=0x7f5510106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:32:32.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.592+0000 7f551588f700 1 --2- 172.21.15.42:0/2835309900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5510060310 0x7f5510106e10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f5504000c00 tx=0x7f550400b040).stop 2024-02-15T18:32:32.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.592+0000 7f551588f700 1 -- 172.21.15.42:0/2835309900 shutdown_connections 2024-02-15T18:32:32.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.592+0000 7f551588f700 1 --2- 172.21.15.42:0/2835309900 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f5500060f10 0x7f55000633d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.592+0000 7f551588f700 1 --2- 172.21.15.42:0/2835309900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5510060310 0x7f5510106e10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.592+0000 7f551588f700 1 --2- 172.21.15.42:0/2835309900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55100f9690 0x7f5510107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:32:32.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.592+0000 7f551588f700 1 -- 172.21.15.42:0/2835309900 >> 172.21.15.42:0/2835309900 conn(0x7f55100f0b90 msgr2=0x7f55100f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:32:32.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.592+0000 7f551588f700 1 -- 172.21.15.42:0/2835309900 shutdown_connections 2024-02-15T18:32:32.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:32:32.592+0000 7f551588f700 1 -- 172.21.15.42:0/2835309900 wait complete. 2024-02-15T18:32:33.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:31.256437+0000 mgr.smithi042.kfoihb (mgr.24435) 320 : audit [DBG] from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:33.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:31.554464+0000 mgr.smithi042.kfoihb (mgr.24435) 321 : audit [DBG] from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:33.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:31.994366+0000 mon.smithi042 (mon.0) 558 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-15T18:32:33.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:31.994393+0000 mon.smithi042 (mon.0) 559 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-15T18:32:33.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:31.997869+0000 mon.smithi042 (mon.0) 560 : cluster [INF] Standby daemon mds.cephfs.smithi042.vtwvlo assigned to filesystem cephfs as rank 0 2024-02-15T18:32:33.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:31.997956+0000 mon.smithi042 (mon.0) 561 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-15T18:32:33.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:31.998128+0000 mon.smithi042 (mon.0) 562 : cluster [DBG] fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-15T18:32:33.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:32.000981+0000 mon.smithi042 (mon.0) 563 : cluster [DBG] fsmap cephfs:1/1 {0=cephfs.smithi042.vtwvlo=up:replay} 2 up:standby 2024-02-15T18:32:33.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:32 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:32.261670+0000 mon.smithi191 (mon.1) 12 : audit [DBG] from='client.? 172.21.15.42:0/227440699' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:32:33.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:31.256437+0000 mgr.smithi042.kfoihb (mgr.24435) 320 : audit [DBG] from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:33.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:31.554464+0000 mgr.smithi042.kfoihb (mgr.24435) 321 : audit [DBG] from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:33.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:31.994366+0000 mon.smithi042 (mon.0) 558 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-15T18:32:33.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:31.994393+0000 mon.smithi042 (mon.0) 559 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-15T18:32:33.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:31.997869+0000 mon.smithi042 (mon.0) 560 : cluster [INF] Standby daemon mds.cephfs.smithi042.vtwvlo assigned to filesystem cephfs as rank 0 2024-02-15T18:32:33.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:31.997956+0000 mon.smithi042 (mon.0) 561 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-15T18:32:33.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:31.998128+0000 mon.smithi042 (mon.0) 562 : cluster [DBG] fsmap cephfs:0/1 3 up:standby, 1 failed 2024-02-15T18:32:33.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:32.000981+0000 mon.smithi042 (mon.0) 563 : cluster [DBG] fsmap cephfs:1/1 {0=cephfs.smithi042.vtwvlo=up:replay} 2 up:standby 2024-02-15T18:32:33.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:32 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:32.261670+0000 mon.smithi191 (mon.1) 12 : audit [DBG] from='client.? 172.21.15.42:0/227440699' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:32:34.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:32.590768+0000 mgr.smithi042.kfoihb (mgr.24435) 322 : audit [DBG] from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:34.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:32.764558+0000 mgr.smithi042.kfoihb (mgr.24435) 323 : cluster [DBG] pgmap v260: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-02-15T18:32:34.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:32.590768+0000 mgr.smithi042.kfoihb (mgr.24435) 322 : audit [DBG] from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:32:34.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:32.764558+0000 mgr.smithi042.kfoihb (mgr.24435) 323 : cluster [DBG] pgmap v260: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-02-15T18:32:36.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:34.764861+0000 mgr.smithi042.kfoihb (mgr.24435) 324 : cluster [DBG] pgmap v261: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-02-15T18:32:36.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:34.764861+0000 mgr.smithi042.kfoihb (mgr.24435) 324 : cluster [DBG] pgmap v261: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 102 B/s rd, 0 op/s 2024-02-15T18:32:37.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:36.765285+0000 mgr.smithi042.kfoihb (mgr.24435) 325 : cluster [DBG] pgmap v262: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 8.4 MiB/s rd, 2 op/s 2024-02-15T18:32:37.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:36.765285+0000 mgr.smithi042.kfoihb (mgr.24435) 325 : cluster [DBG] pgmap v262: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 8.4 MiB/s rd, 2 op/s 2024-02-15T18:32:40.096 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:38.765645+0000 mgr.smithi042.kfoihb (mgr.24435) 326 : cluster [DBG] pgmap v263: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-02-15T18:32:40.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:38.765645+0000 mgr.smithi042.kfoihb (mgr.24435) 326 : cluster [DBG] pgmap v263: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-02-15T18:32:41.831 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:40.766051+0000 mgr.smithi042.kfoihb (mgr.24435) 327 : cluster [DBG] pgmap v264: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-02-15T18:32:41.831 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:40.838350+0000 mon.smithi042 (mon.0) 564 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:41.831 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:41.611691+0000 mon.smithi042 (mon.0) 565 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:41.831 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:41.612427+0000 mon.smithi042 (mon.0) 566 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:42.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:40.766051+0000 mgr.smithi042.kfoihb (mgr.24435) 327 : cluster [DBG] pgmap v264: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-02-15T18:32:42.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:40.838350+0000 mon.smithi042 (mon.0) 564 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:42.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:41.611691+0000 mon.smithi042 (mon.0) 565 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:42.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:41.612427+0000 mon.smithi042 (mon.0) 566 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:43.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:41.999800+0000 mon.smithi042 (mon.0) 567 : cluster [DBG] mds.? [v2:172.21.15.191:6824/540280968,v1:172.21.15.191:6825/540280968] up:boot 2024-02-15T18:32:43.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:41.999844+0000 mon.smithi042 (mon.0) 568 : cluster [DBG] mds.? [v2:172.21.15.42:6826/2962563293,v1:172.21.15.42:6827/2962563293] up:reconnect 2024-02-15T18:32:43.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:41.999890+0000 mon.smithi042 (mon.0) 569 : cluster [DBG] fsmap cephfs:1/1 {0=cephfs.smithi042.vtwvlo=up:reconnect} 3 up:standby 2024-02-15T18:32:43.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:42.000005+0000 mon.smithi042 (mon.0) 570 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.icwcnb"}]: dispatch 2024-02-15T18:32:43.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:42.001910+0000 mds.cephfs.smithi042.vtwvlo (mds.0) 1 : cluster [DBG] reconnect by client.24325 v1:172.21.15.42:0/1124938238 after 0.00399997 2024-02-15T18:32:43.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:42.006111+0000 mds.cephfs.smithi042.vtwvlo (mds.0) 2 : cluster [DBG] reconnect by client.24331 v1:192.168.0.1:0/1083392431 after 0.00899993 2024-02-15T18:32:43.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:42.766661+0000 mgr.smithi042.kfoihb (mgr.24435) 328 : cluster [DBG] pgmap v265: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-02-15T18:32:43.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:41.999800+0000 mon.smithi042 (mon.0) 567 : cluster [DBG] mds.? [v2:172.21.15.191:6824/540280968,v1:172.21.15.191:6825/540280968] up:boot 2024-02-15T18:32:43.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:41.999844+0000 mon.smithi042 (mon.0) 568 : cluster [DBG] mds.? [v2:172.21.15.42:6826/2962563293,v1:172.21.15.42:6827/2962563293] up:reconnect 2024-02-15T18:32:43.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:41.999890+0000 mon.smithi042 (mon.0) 569 : cluster [DBG] fsmap cephfs:1/1 {0=cephfs.smithi042.vtwvlo=up:reconnect} 3 up:standby 2024-02-15T18:32:43.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:42.000005+0000 mon.smithi042 (mon.0) 570 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.icwcnb"}]: dispatch 2024-02-15T18:32:43.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:42.001910+0000 mds.cephfs.smithi042.vtwvlo (mds.0) 1 : cluster [DBG] reconnect by client.24325 v1:172.21.15.42:0/1124938238 after 0.00399997 2024-02-15T18:32:43.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:42.006111+0000 mds.cephfs.smithi042.vtwvlo (mds.0) 2 : cluster [DBG] reconnect by client.24331 v1:192.168.0.1:0/1083392431 after 0.00899993 2024-02-15T18:32:43.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:42.766661+0000 mgr.smithi042.kfoihb (mgr.24435) 328 : cluster [DBG] pgmap v265: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-02-15T18:32:44.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:43.005277+0000 mon.smithi042 (mon.0) 571 : cluster [DBG] mds.? [v2:172.21.15.42:6826/2962563293,v1:172.21.15.42:6827/2962563293] up:rejoin 2024-02-15T18:32:44.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:43.005328+0000 mon.smithi042 (mon.0) 572 : cluster [DBG] fsmap cephfs:1/1 {0=cephfs.smithi042.vtwvlo=up:rejoin} 3 up:standby 2024-02-15T18:32:44.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:44 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:43.082434+0000 mon.smithi042 (mon.0) 573 : cluster [INF] daemon mds.cephfs.smithi042.vtwvlo is now active in filesystem cephfs as rank 0 2024-02-15T18:32:44.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:43.005277+0000 mon.smithi042 (mon.0) 571 : cluster [DBG] mds.? [v2:172.21.15.42:6826/2962563293,v1:172.21.15.42:6827/2962563293] up:rejoin 2024-02-15T18:32:44.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:43.005328+0000 mon.smithi042 (mon.0) 572 : cluster [DBG] fsmap cephfs:1/1 {0=cephfs.smithi042.vtwvlo=up:rejoin} 3 up:standby 2024-02-15T18:32:44.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:43.082434+0000 mon.smithi042 (mon.0) 573 : cluster [INF] daemon mds.cephfs.smithi042.vtwvlo is now active in filesystem cephfs as rank 0 2024-02-15T18:32:45.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:44.003131+0000 mon.smithi042 (mon.0) 574 : cluster [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-15T18:32:45.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:44.003158+0000 mon.smithi042 (mon.0) 575 : cluster [INF] Cluster is now healthy 2024-02-15T18:32:45.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:44.010663+0000 mon.smithi042 (mon.0) 576 : cluster [DBG] mds.? [v2:172.21.15.42:6826/2962563293,v1:172.21.15.42:6827/2962563293] up:active 2024-02-15T18:32:45.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:44.010734+0000 mon.smithi042 (mon.0) 577 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi042.vtwvlo=up:active} 3 up:standby 2024-02-15T18:32:45.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:44.646768+0000 mon.smithi042 (mon.0) 578 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:45.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:44.647740+0000 mon.smithi042 (mon.0) 579 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:45.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:44.648704+0000 mon.smithi042 (mon.0) 580 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:32:45.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:44.649431+0000 mgr.smithi042.kfoihb (mgr.24435) 329 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:32:45.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:44.653598+0000 mon.smithi042 (mon.0) 581 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:45.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:44.659673+0000 mon.smithi042 (mon.0) 582 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:45.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:44.662658+0000 mon.smithi042 (mon.0) 583 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:45.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:44.664111+0000 mon.smithi042 (mon.0) 584 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:45.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:44.665460+0000 mon.smithi042 (mon.0) 585 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:45.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:44.667172+0000 mon.smithi042 (mon.0) 586 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:45.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:44.669182+0000 mon.smithi042 (mon.0) 587 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi191.erlpxp"]}]: dispatch 2024-02-15T18:32:45.306 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:44.669953+0000 mgr.smithi042.kfoihb (mgr.24435) 330 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi191.erlpxp 2024-02-15T18:32:45.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:44.767084+0000 mgr.smithi042.kfoihb (mgr.24435) 331 : cluster [DBG] pgmap v266: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 16 MiB/s rd, 4 op/s 2024-02-15T18:32:45.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:44.003131+0000 mon.smithi042 (mon.0) 574 : cluster [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-15T18:32:45.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:44.003158+0000 mon.smithi042 (mon.0) 575 : cluster [INF] Cluster is now healthy 2024-02-15T18:32:45.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:44.010663+0000 mon.smithi042 (mon.0) 576 : cluster [DBG] mds.? [v2:172.21.15.42:6826/2962563293,v1:172.21.15.42:6827/2962563293] up:active 2024-02-15T18:32:45.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:44.010734+0000 mon.smithi042 (mon.0) 577 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi042.vtwvlo=up:active} 3 up:standby 2024-02-15T18:32:45.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:44.646768+0000 mon.smithi042 (mon.0) 578 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:45.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:44.647740+0000 mon.smithi042 (mon.0) 579 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:45.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:44.648704+0000 mon.smithi042 (mon.0) 580 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:32:45.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:44.649431+0000 mgr.smithi042.kfoihb (mgr.24435) 329 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:32:45.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:44.653598+0000 mon.smithi042 (mon.0) 581 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:45.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:44.659673+0000 mon.smithi042 (mon.0) 582 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:45.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:44.662658+0000 mon.smithi042 (mon.0) 583 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:45.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:44.664111+0000 mon.smithi042 (mon.0) 584 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:45.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:44.665460+0000 mon.smithi042 (mon.0) 585 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:45.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:44.667172+0000 mon.smithi042 (mon.0) 586 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:45.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:44.669182+0000 mon.smithi042 (mon.0) 587 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi191.erlpxp"]}]: dispatch 2024-02-15T18:32:45.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:44.669953+0000 mgr.smithi042.kfoihb (mgr.24435) 330 : cephadm [INF] Upgrade: It appears safe to stop mds.cephfs.smithi191.erlpxp 2024-02-15T18:32:45.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:44.767084+0000 mgr.smithi042.kfoihb (mgr.24435) 331 : cluster [DBG] pgmap v266: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 16 MiB/s rd, 4 op/s 2024-02-15T18:32:47.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:46.201553+0000 mgr.smithi042.kfoihb (mgr.24435) 332 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi191.erlpxp 2024-02-15T18:32:47.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:46.206329+0000 mon.smithi042 (mon.0) 588 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:47.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:46.207413+0000 mon.smithi042 (mon.0) 589 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi191.erlpxp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:47.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:46.208378+0000 mon.smithi042 (mon.0) 590 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:47.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:46.209123+0000 mgr.smithi042.kfoihb (mgr.24435) 333 : cephadm [INF] Deploying daemon mds.cephfs.smithi191.erlpxp on smithi191 2024-02-15T18:32:47.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:46.767580+0000 mgr.smithi042.kfoihb (mgr.24435) 334 : cluster [DBG] pgmap v267: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 16 MiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-02-15T18:32:47.680 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:46.201553+0000 mgr.smithi042.kfoihb (mgr.24435) 332 : cephadm [INF] Upgrade: Updating mds.cephfs.smithi191.erlpxp 2024-02-15T18:32:47.680 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:46.206329+0000 mon.smithi042 (mon.0) 588 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:47.681 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:46.207413+0000 mon.smithi042 (mon.0) 589 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi191.erlpxp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-15T18:32:47.681 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:46.208378+0000 mon.smithi042 (mon.0) 590 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:47.681 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:46.209123+0000 mgr.smithi042.kfoihb (mgr.24435) 333 : cephadm [INF] Deploying daemon mds.cephfs.smithi191.erlpxp on smithi191 2024-02-15T18:32:47.681 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:46.767580+0000 mgr.smithi042.kfoihb (mgr.24435) 334 : cluster [DBG] pgmap v267: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 16 MiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-02-15T18:32:48.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:47.968362+0000 mon.smithi042 (mon.0) 591 : cluster [WRN] daemon mds.cephfs.smithi191.erlpxp compat changed unexpectedly 2024-02-15T18:32:48.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:47.973061+0000 mon.smithi042 (mon.0) 592 : cluster [DBG] osdmap e114: 6 total, 6 up, 6 in 2024-02-15T18:32:48.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:47.968362+0000 mon.smithi042 (mon.0) 591 : cluster [WRN] daemon mds.cephfs.smithi191.erlpxp compat changed unexpectedly 2024-02-15T18:32:48.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:47.973061+0000 mon.smithi042 (mon.0) 592 : cluster [DBG] osdmap e114: 6 total, 6 up, 6 in 2024-02-15T18:32:49.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:48.214965+0000 mon.smithi042 (mon.0) 593 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi042.vtwvlo=up:active} 2 up:standby 2024-02-15T18:32:49.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:48.768080+0000 mgr.smithi042.kfoihb (mgr.24435) 335 : cluster [DBG] pgmap v269: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 7.1 MiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-02-15T18:32:49.692 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:48.214965+0000 mon.smithi042 (mon.0) 593 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi042.vtwvlo=up:active} 2 up:standby 2024-02-15T18:32:49.692 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:48.768080+0000 mgr.smithi042.kfoihb (mgr.24435) 335 : cluster [DBG] pgmap v269: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 7.1 MiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-02-15T18:32:52.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:50.768772+0000 mgr.smithi042.kfoihb (mgr.24435) 336 : cluster [DBG] pgmap v270: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-02-15T18:32:52.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:50.768772+0000 mgr.smithi042.kfoihb (mgr.24435) 336 : cluster [DBG] pgmap v270: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-02-15T18:32:53.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:51.847165+0000 mon.smithi042 (mon.0) 594 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:53.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:51.848036+0000 mon.smithi042 (mon.0) 595 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:53.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:52.001481+0000 mon.smithi042 (mon.0) 596 : cluster [DBG] mds.? [v2:172.21.15.191:6826/289198038,v1:172.21.15.191:6827/289198038] up:boot 2024-02-15T18:32:53.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:52.001537+0000 mon.smithi042 (mon.0) 597 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi042.vtwvlo=up:active} 3 up:standby 2024-02-15T18:32:53.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:52.001656+0000 mon.smithi042 (mon.0) 598 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.erlpxp"}]: dispatch 2024-02-15T18:32:53.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:51.847165+0000 mon.smithi042 (mon.0) 594 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:53.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:51.848036+0000 mon.smithi042 (mon.0) 595 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:53.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:52.001481+0000 mon.smithi042 (mon.0) 596 : cluster [DBG] mds.? [v2:172.21.15.191:6826/289198038,v1:172.21.15.191:6827/289198038] up:boot 2024-02-15T18:32:53.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:52.001537+0000 mon.smithi042 (mon.0) 597 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi042.vtwvlo=up:active} 3 up:standby 2024-02-15T18:32:53.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:52 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:52.001656+0000 mon.smithi042 (mon.0) 598 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi191.erlpxp"}]: dispatch 2024-02-15T18:32:54.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:52.769320+0000 mgr.smithi042.kfoihb (mgr.24435) 337 : cluster [DBG] pgmap v271: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-02-15T18:32:54.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:52.769320+0000 mgr.smithi042.kfoihb (mgr.24435) 337 : cluster [DBG] pgmap v271: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-02-15T18:32:55.442 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:32:55.259+0000 7f490ccf7700 -1 bdev(0x55fa13f6a800 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 2 2024-02-15T18:32:55.442 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:32:55.259+0000 7f490f4fc700 -1 bdev(0x55fa13f6a800 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 3 2024-02-15T18:32:55.442 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:32:55.261+0000 7f49104fe700 -1 bdev(0x55fa13f6a800 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 10 2024-02-15T18:32:55.442 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:32:55.262+0000 7f490dcf9700 -1 bdev(0x55fa13f6a800 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 14 2024-02-15T18:32:55.442 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:32:55.262+0000 7f4912502700 -1 bdev(0x55fa13f6a800 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 17 2024-02-15T18:32:55.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.196788+0000 mon.smithi042 (mon.0) 599 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:55.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.197701+0000 mon.smithi042 (mon.0) 600 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:55.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.198617+0000 mon.smithi042 (mon.0) 601 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:32:55.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:54.199370+0000 mgr.smithi042.kfoihb (mgr.24435) 338 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:32:55.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.203114+0000 mon.smithi042 (mon.0) 602 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:55.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.209259+0000 mon.smithi042 (mon.0) 603 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:55.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.211932+0000 mon.smithi042 (mon.0) 604 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:55.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.213301+0000 mon.smithi042 (mon.0) 605 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:55.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.214622+0000 mon.smithi042 (mon.0) 606 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:55.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.216297+0000 mon.smithi042 (mon.0) 607 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:55.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.218220+0000 mon.smithi042 (mon.0) 608 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:55.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:54.218869+0000 mgr.smithi042.kfoihb (mgr.24435) 339 : cephadm [INF] Upgrade: Setting container_image for all mds 2024-02-15T18:32:55.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.223261+0000 mon.smithi042 (mon.0) 609 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:55.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.224063+0000 mon.smithi042 (mon.0) 610 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.idnxkc"}]: dispatch 2024-02-15T18:32:55.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.227650+0000 mon.smithi042 (mon.0) 611 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.idnxkc"}]': finished 2024-02-15T18:32:55.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.228385+0000 mon.smithi042 (mon.0) 612 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.vtwvlo"}]: dispatch 2024-02-15T18:32:55.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.231827+0000 mon.smithi042 (mon.0) 613 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.vtwvlo"}]': finished 2024-02-15T18:32:55.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.232554+0000 mon.smithi042 (mon.0) 614 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi191.erlpxp"}]: dispatch 2024-02-15T18:32:55.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.235773+0000 mon.smithi042 (mon.0) 615 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi191.erlpxp"}]': finished 2024-02-15T18:32:55.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.236483+0000 mon.smithi042 (mon.0) 616 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi191.icwcnb"}]: dispatch 2024-02-15T18:32:55.445 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.239698+0000 mon.smithi042 (mon.0) 617 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi191.icwcnb"}]': finished 2024-02-15T18:32:55.445 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:54.240441+0000 mgr.smithi042.kfoihb (mgr.24435) 340 : cephadm [INF] Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-02-15T18:32:55.445 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:54.240668+0000 mon.smithi042 (mon.0) 618 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-02-15T18:32:55.445 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:54.769868+0000 mgr.smithi042.kfoihb (mgr.24435) 341 : cluster [DBG] pgmap v272: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-02-15T18:32:55.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.196788+0000 mon.smithi042 (mon.0) 599 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:55.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.197701+0000 mon.smithi042 (mon.0) 600 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:32:55.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.198617+0000 mon.smithi042 (mon.0) 601 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:32:55.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:54.199370+0000 mgr.smithi042.kfoihb (mgr.24435) 338 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:32:55.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.203114+0000 mon.smithi042 (mon.0) 602 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:55.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.209259+0000 mon.smithi042 (mon.0) 603 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:32:55.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.211932+0000 mon.smithi042 (mon.0) 604 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:55.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.213301+0000 mon.smithi042 (mon.0) 605 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:55.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.214622+0000 mon.smithi042 (mon.0) 606 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:55.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.216297+0000 mon.smithi042 (mon.0) 607 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:55.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.218220+0000 mon.smithi042 (mon.0) 608 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:55.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:54.218869+0000 mgr.smithi042.kfoihb (mgr.24435) 339 : cephadm [INF] Upgrade: Setting container_image for all mds 2024-02-15T18:32:55.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.223261+0000 mon.smithi042 (mon.0) 609 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:55.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.224063+0000 mon.smithi042 (mon.0) 610 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.idnxkc"}]: dispatch 2024-02-15T18:32:55.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.227650+0000 mon.smithi042 (mon.0) 611 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.idnxkc"}]': finished 2024-02-15T18:32:55.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.228385+0000 mon.smithi042 (mon.0) 612 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.vtwvlo"}]: dispatch 2024-02-15T18:32:55.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.231827+0000 mon.smithi042 (mon.0) 613 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.vtwvlo"}]': finished 2024-02-15T18:32:55.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.232554+0000 mon.smithi042 (mon.0) 614 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi191.erlpxp"}]: dispatch 2024-02-15T18:32:55.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.235773+0000 mon.smithi042 (mon.0) 615 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi191.erlpxp"}]': finished 2024-02-15T18:32:55.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.236483+0000 mon.smithi042 (mon.0) 616 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi191.icwcnb"}]: dispatch 2024-02-15T18:32:55.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.239698+0000 mon.smithi042 (mon.0) 617 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi191.icwcnb"}]': finished 2024-02-15T18:32:55.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:54.240441+0000 mgr.smithi042.kfoihb (mgr.24435) 340 : cephadm [INF] Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-02-15T18:32:55.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:54.240668+0000 mon.smithi042 (mon.0) 618 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-02-15T18:32:55.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:54.769868+0000 mgr.smithi042.kfoihb (mgr.24435) 341 : cluster [DBG] pgmap v272: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-02-15T18:32:56.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:55.244179+0000 mon.smithi042 (mon.0) 619 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-02-15T18:32:56.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:55.244475+0000 mon.smithi042 (mon.0) 620 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi042.vtwvlo=up:active} 3 up:standby 2024-02-15T18:32:56.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:55.249718+0000 mon.smithi042 (mon.0) 621 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi042.vtwvlo=up:active} 1 up:standby-replay 2 up:standby 2024-02-15T18:32:56.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:55.255526+0000 mon.smithi042 (mon.0) 622 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:56.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:55.257286+0000 mon.smithi042 (mon.0) 623 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:56.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:55.258600+0000 mgr.smithi042.kfoihb (mgr.24435) 342 : cephadm [INF] Upgrade: Setting container_image for all rgw 2024-02-15T18:32:56.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:55.272842+0000 mon.smithi042 (mon.0) 624 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:56.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:55.274585+0000 mon.smithi042 (mon.0) 625 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:56.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:55.275568+0000 mgr.smithi042.kfoihb (mgr.24435) 343 : cephadm [INF] Upgrade: Setting container_image for all rbd-mirror 2024-02-15T18:32:56.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:55.281236+0000 mon.smithi042 (mon.0) 626 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:56.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:55.283074+0000 mon.smithi042 (mon.0) 627 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:56.556 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:55.284670+0000 mon.smithi042 (mon.0) 628 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:56.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:55.285538+0000 mgr.smithi042.kfoihb (mgr.24435) 344 : cephadm [INF] Upgrade: Setting container_image for all iscsi 2024-02-15T18:32:56.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:55.295811+0000 mon.smithi042 (mon.0) 629 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:56.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:55.297130+0000 mon.smithi042 (mon.0) 630 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:56.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:55.297918+0000 mgr.smithi042.kfoihb (mgr.24435) 345 : cephadm [INF] Upgrade: Setting container_image for all nfs 2024-02-15T18:32:56.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:55.303713+0000 mon.smithi042 (mon.0) 631 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:56.557 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:56 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:32:55.838760+0000 mon.smithi042 (mon.0) 632 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:56.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:55.244179+0000 mon.smithi042 (mon.0) 619 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-02-15T18:32:56.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:55.244475+0000 mon.smithi042 (mon.0) 620 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi042.vtwvlo=up:active} 3 up:standby 2024-02-15T18:32:56.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:55.249718+0000 mon.smithi042 (mon.0) 621 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi042.vtwvlo=up:active} 1 up:standby-replay 2 up:standby 2024-02-15T18:32:56.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:55.255526+0000 mon.smithi042 (mon.0) 622 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:56.693 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:55.257286+0000 mon.smithi042 (mon.0) 623 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:56.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:55.258600+0000 mgr.smithi042.kfoihb (mgr.24435) 342 : cephadm [INF] Upgrade: Setting container_image for all rgw 2024-02-15T18:32:56.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:55.272842+0000 mon.smithi042 (mon.0) 624 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:56.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:55.274585+0000 mon.smithi042 (mon.0) 625 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:56.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:55.275568+0000 mgr.smithi042.kfoihb (mgr.24435) 343 : cephadm [INF] Upgrade: Setting container_image for all rbd-mirror 2024-02-15T18:32:56.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:55.281236+0000 mon.smithi042 (mon.0) 626 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:56.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:55.283074+0000 mon.smithi042 (mon.0) 627 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:56.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:55.284670+0000 mon.smithi042 (mon.0) 628 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:56.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:55.285538+0000 mgr.smithi042.kfoihb (mgr.24435) 344 : cephadm [INF] Upgrade: Setting container_image for all iscsi 2024-02-15T18:32:56.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:55.295811+0000 mon.smithi042 (mon.0) 629 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:56.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:55.297130+0000 mon.smithi042 (mon.0) 630 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:32:56.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:55.297918+0000 mgr.smithi042.kfoihb (mgr.24435) 345 : cephadm [INF] Upgrade: Setting container_image for all nfs 2024-02-15T18:32:56.694 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:55.303713+0000 mon.smithi042 (mon.0) 631 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:56.695 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:32:55.838760+0000 mon.smithi042 (mon.0) 632 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:32:57.555 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:56.770309+0000 mgr.smithi042.kfoihb (mgr.24435) 346 : cluster [DBG] pgmap v273: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 B/s wr, 1 op/s 2024-02-15T18:32:57.692 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:56.770309+0000 mgr.smithi042.kfoihb (mgr.24435) 346 : cluster [DBG] pgmap v273: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 B/s wr, 1 op/s 2024-02-15T18:33:00.075 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:32:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:32:58.770732+0000 mgr.smithi042.kfoihb (mgr.24435) 347 : cluster [DBG] pgmap v274: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 1 op/s 2024-02-15T18:33:00.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:32:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:32:58.770732+0000 mgr.smithi042.kfoihb (mgr.24435) 347 : cluster [DBG] pgmap v274: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 1 op/s 2024-02-15T18:33:01.089 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:59.255785+0000 mgr.smithi042.kfoihb (mgr.24435) 348 : cephadm [INF] Upgrade: Updating node-exporter.smithi042 (1/2) 2024-02-15T18:33:01.089 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:00 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:32:59.256008+0000 mgr.smithi042.kfoihb (mgr.24435) 349 : cephadm [INF] Deploying daemon node-exporter.smithi042 on smithi042 2024-02-15T18:33:01.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:59.255785+0000 mgr.smithi042.kfoihb (mgr.24435) 348 : cephadm [INF] Upgrade: Updating node-exporter.smithi042 (1/2) 2024-02-15T18:33:01.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:32:59.256008+0000 mgr.smithi042.kfoihb (mgr.24435) 349 : cephadm [INF] Deploying daemon node-exporter.smithi042 on smithi042 2024-02-15T18:33:02.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:00.771283+0000 mgr.smithi042.kfoihb (mgr.24435) 350 : cluster [DBG] pgmap v275: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-02-15T18:33:02.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:00.771283+0000 mgr.smithi042.kfoihb (mgr.24435) 350 : cluster [DBG] pgmap v275: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-02-15T18:33:02.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.741+0000 7fd68b232700 1 -- 172.21.15.42:0/3016106590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6840f6860 msgr2=0x7fd6840f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.741+0000 7fd68b232700 1 --2- 172.21.15.42:0/3016106590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6840f6860 0x7fd6840f6cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fd670004660 tx=0x7fd670009b30).stop 2024-02-15T18:33:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.741+0000 7fd68b232700 1 -- 172.21.15.42:0/3016106590 shutdown_connections 2024-02-15T18:33:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.741+0000 7fd68b232700 1 --2- 172.21.15.42:0/3016106590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6840f6860 0x7fd6840f6cc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.741+0000 7fd68b232700 1 --2- 172.21.15.42:0/3016106590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6840f55c0 0x7fd6840f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.741+0000 7fd68b232700 1 -- 172.21.15.42:0/3016106590 >> 172.21.15.42:0/3016106590 conn(0x7fd6840f0d40 msgr2=0x7fd6840f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.741+0000 7fd68b232700 1 -- 172.21.15.42:0/3016106590 shutdown_connections 2024-02-15T18:33:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.741+0000 7fd68b232700 1 -- 172.21.15.42:0/3016106590 wait complete. 2024-02-15T18:33:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.742+0000 7fd68b232700 1 Processor -- start 2024-02-15T18:33:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.742+0000 7fd68b232700 1 -- start start 2024-02-15T18:33:02.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.742+0000 7fd68b232700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6840f55c0 0x7fd68410c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.742+0000 7fd68b232700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6840f6860 0x7fd68410e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.742+0000 7fd68b232700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840602d0 con 0x7fd6840f55c0 2024-02-15T18:33:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.742+0000 7fd68b232700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd684060410 con 0x7fd6840f6860 2024-02-15T18:33:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.743+0000 7fd688fce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6840f55c0 0x7fd68410c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.743+0000 7fd683fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6840f6860 0x7fd68410e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.743+0000 7fd688fce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6840f55c0 0x7fd68410c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43198/0 (socket says 172.21.15.42:43198) 2024-02-15T18:33:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.743+0000 7fd688fce700 1 -- 172.21.15.42:0/2529480304 learned_addr learned my addr 172.21.15.42:0/2529480304 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:33:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.743+0000 7fd688fce700 1 -- 172.21.15.42:0/2529480304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6840f6860 msgr2=0x7fd68410e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.743+0000 7fd688fce700 1 --2- 172.21.15.42:0/2529480304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6840f6860 0x7fd68410e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.743+0000 7fd688fce700 1 -- 172.21.15.42:0/2529480304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd670005040 con 0x7fd6840f55c0 2024-02-15T18:33:02.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.743+0000 7fd688fce700 1 --2- 172.21.15.42:0/2529480304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6840f55c0 0x7fd68410c0b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fd67800e7c0 tx=0x7fd67800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.743+0000 7fd6817fa700 1 -- 172.21.15.42:0/2529480304 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6780092e0 con 0x7fd6840f55c0 2024-02-15T18:33:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.743+0000 7fd6817fa700 1 -- 172.21.15.42:0/2529480304 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fd67800f040 con 0x7fd6840f55c0 2024-02-15T18:33:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.743+0000 7fd6817fa700 1 -- 172.21.15.42:0/2529480304 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd678014920 con 0x7fd6840f55c0 2024-02-15T18:33:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.743+0000 7fd68b232700 1 -- 172.21.15.42:0/2529480304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68410eaa0 con 0x7fd6840f55c0 2024-02-15T18:33:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.744+0000 7fd68b232700 1 -- 172.21.15.42:0/2529480304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68410eec0 con 0x7fd6840f55c0 2024-02-15T18:33:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.745+0000 7fd68b232700 1 -- 172.21.15.42:0/2529480304 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd684045ad0 con 0x7fd6840f55c0 2024-02-15T18:33:02.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.746+0000 7fd6817fa700 1 -- 172.21.15.42:0/2529480304 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fd67801b030 con 0x7fd6840f55c0 2024-02-15T18:33:02.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.746+0000 7fd6817fa700 1 --2- 172.21.15.42:0/2529480304 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd674060f20 0x7fd6740633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:02.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.746+0000 7fd6817fa700 1 -- 172.21.15.42:0/2529480304 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fd67807c680 con 0x7fd6840f55c0 2024-02-15T18:33:02.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.750+0000 7fd6817fa700 1 -- 172.21.15.42:0/2529480304 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd67804c990 con 0x7fd6840f55c0 2024-02-15T18:33:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.752+0000 7fd683fff700 1 --2- 172.21.15.42:0/2529480304 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd674060f20 0x7fd6740633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:02.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.752+0000 7fd683fff700 1 --2- 172.21.15.42:0/2529480304 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd674060f20 0x7fd6740633e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd670009720 tx=0x7fd670016040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:02.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.959+0000 7fd68b232700 1 -- 172.21.15.42:0/2529480304 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd684057f10 con 0x7fd674060f20 2024-02-15T18:33:02.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.960+0000 7fd6817fa700 1 -- 172.21.15.42:0/2529480304 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fd684057f10 con 0x7fd674060f20 2024-02-15T18:33:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.962+0000 7fd68b232700 1 -- 172.21.15.42:0/2529480304 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd674060f20 msgr2=0x7fd6740633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.962+0000 7fd68b232700 1 --2- 172.21.15.42:0/2529480304 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd674060f20 0x7fd6740633e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd670009720 tx=0x7fd670016040).stop 2024-02-15T18:33:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.962+0000 7fd68b232700 1 -- 172.21.15.42:0/2529480304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6840f55c0 msgr2=0x7fd68410c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.962+0000 7fd68b232700 1 --2- 172.21.15.42:0/2529480304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6840f55c0 0x7fd68410c0b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fd67800e7c0 tx=0x7fd67800b040).stop 2024-02-15T18:33:02.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.962+0000 7fd68b232700 1 -- 172.21.15.42:0/2529480304 shutdown_connections 2024-02-15T18:33:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.962+0000 7fd68b232700 1 --2- 172.21.15.42:0/2529480304 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd674060f20 0x7fd6740633e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.962+0000 7fd68b232700 1 --2- 172.21.15.42:0/2529480304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6840f55c0 0x7fd68410c0b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.962+0000 7fd68b232700 1 --2- 172.21.15.42:0/2529480304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6840f6860 0x7fd68410e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.962+0000 7fd68b232700 1 -- 172.21.15.42:0/2529480304 >> 172.21.15.42:0/2529480304 conn(0x7fd6840f0d40 msgr2=0x7fd6840fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.962+0000 7fd68b232700 1 -- 172.21.15.42:0/2529480304 shutdown_connections 2024-02-15T18:33:02.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:02.962+0000 7fd68b232700 1 -- 172.21.15.42:0/2529480304 wait complete. 2024-02-15T18:33:02.965 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:33:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.077+0000 7f4029e53700 1 -- 172.21.15.42:0/2776574986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40240f5a50 msgr2=0x7f40240f5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.077+0000 7f4029e53700 1 --2- 172.21.15.42:0/2776574986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40240f5a50 0x7f40240f5e70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f401c0099e0 tx=0x7f401c004d10).stop 2024-02-15T18:33:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.077+0000 7f4029e53700 1 -- 172.21.15.42:0/2776574986 shutdown_connections 2024-02-15T18:33:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.077+0000 7f4029e53700 1 --2- 172.21.15.42:0/2776574986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40240f47a0 0x7f40240f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.077+0000 7f4029e53700 1 --2- 172.21.15.42:0/2776574986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40240f5a50 0x7f40240f5e70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.077+0000 7f4029e53700 1 -- 172.21.15.42:0/2776574986 >> 172.21.15.42:0/2776574986 conn(0x7f40240f04c0 msgr2=0x7f40240f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.078+0000 7f4029e53700 1 -- 172.21.15.42:0/2776574986 shutdown_connections 2024-02-15T18:33:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.078+0000 7f4029e53700 1 -- 172.21.15.42:0/2776574986 wait complete. 2024-02-15T18:33:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.078+0000 7f4029e53700 1 Processor -- start 2024-02-15T18:33:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.079+0000 7f4029e53700 1 -- start start 2024-02-15T18:33:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.079+0000 7f4029e53700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40240f47a0 0x7f402410d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.079+0000 7f4029e53700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40240f5a50 0x7f402410dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.079+0000 7f4029e53700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402410e190 con 0x7f40240f5a50 2024-02-15T18:33:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.079+0000 7f4029e53700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402410e2d0 con 0x7f40240f47a0 2024-02-15T18:33:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.079+0000 7f4022ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40240f5a50 0x7f402410dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.079+0000 7f4022ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40240f5a50 0x7f402410dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43210/0 (socket says 172.21.15.42:43210) 2024-02-15T18:33:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.079+0000 7f4022ffd700 1 -- 172.21.15.42:0/4021761397 learned_addr learned my addr 172.21.15.42:0/4021761397 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:33:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.079+0000 7f40237fe700 1 --2- 172.21.15.42:0/4021761397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40240f47a0 0x7f402410d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.080+0000 7f4022ffd700 1 -- 172.21.15.42:0/4021761397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40240f47a0 msgr2=0x7f402410d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:03.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.080+0000 7f4022ffd700 1 --2- 172.21.15.42:0/4021761397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40240f47a0 0x7f402410d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.080+0000 7f4022ffd700 1 -- 172.21.15.42:0/4021761397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c009640 con 0x7f40240f5a50 2024-02-15T18:33:03.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.080+0000 7f4022ffd700 1 --2- 172.21.15.42:0/4021761397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40240f5a50 0x7f402410dbe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f401800e7f0 tx=0x7f401800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:03.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.080+0000 7f400bfff700 1 -- 172.21.15.42:0/4021761397 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40180092e0 con 0x7f40240f5a50 2024-02-15T18:33:03.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.080+0000 7f400bfff700 1 -- 172.21.15.42:0/4021761397 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f401800f040 con 0x7f40240f5a50 2024-02-15T18:33:03.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.080+0000 7f400bfff700 1 -- 172.21.15.42:0/4021761397 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4018014850 con 0x7f40240f5a50 2024-02-15T18:33:03.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.080+0000 7f4029e53700 1 -- 172.21.15.42:0/4021761397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402410ec70 con 0x7f40240f5a50 2024-02-15T18:33:03.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.080+0000 7f4029e53700 1 -- 172.21.15.42:0/4021761397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402410f230 con 0x7f40240f5a50 2024-02-15T18:33:03.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.081+0000 7f4029e53700 1 -- 172.21.15.42:0/4021761397 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4024045ad0 con 0x7f40240f5a50 2024-02-15T18:33:03.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.082+0000 7f400bfff700 1 -- 172.21.15.42:0/4021761397 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f401801b030 con 0x7f40240f5a50 2024-02-15T18:33:03.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.082+0000 7f400bfff700 1 --2- 172.21.15.42:0/4021761397 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f400c060f70 0x7f400c063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:03.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.082+0000 7f400bfff700 1 -- 172.21.15.42:0/4021761397 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f401807bf40 con 0x7f40240f5a50 2024-02-15T18:33:03.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.083+0000 7f40237fe700 1 --2- 172.21.15.42:0/4021761397 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f400c060f70 0x7f400c063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:03.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.083+0000 7f40237fe700 1 --2- 172.21.15.42:0/4021761397 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f400c060f70 0x7f400c063430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f401c003380 tx=0x7f401c004d10).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:03.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.091+0000 7f400bfff700 1 -- 172.21.15.42:0/4021761397 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f40180aaf60 con 0x7f40240f5a50 2024-02-15T18:33:03.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.309+0000 7f4029e53700 1 -- 172.21.15.42:0/4021761397 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f402410f4a0 con 0x7f400c060f70 2024-02-15T18:33:03.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.311+0000 7f400bfff700 1 -- 172.21.15.42:0/4021761397 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f402410f4a0 con 0x7f400c060f70 2024-02-15T18:33:03.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.313+0000 7f4029e53700 1 -- 172.21.15.42:0/4021761397 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f400c060f70 msgr2=0x7f400c063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.313+0000 7f4029e53700 1 --2- 172.21.15.42:0/4021761397 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f400c060f70 0x7f400c063430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f401c003380 tx=0x7f401c004d10).stop 2024-02-15T18:33:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.313+0000 7f4029e53700 1 -- 172.21.15.42:0/4021761397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40240f5a50 msgr2=0x7f402410dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.313+0000 7f4029e53700 1 --2- 172.21.15.42:0/4021761397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40240f5a50 0x7f402410dbe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f401800e7f0 tx=0x7f401800b040).stop 2024-02-15T18:33:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.313+0000 7f4029e53700 1 -- 172.21.15.42:0/4021761397 shutdown_connections 2024-02-15T18:33:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.313+0000 7f4029e53700 1 --2- 172.21.15.42:0/4021761397 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f400c060f70 0x7f400c063430 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.313+0000 7f4029e53700 1 --2- 172.21.15.42:0/4021761397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40240f47a0 0x7f402410d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.313+0000 7f4029e53700 1 --2- 172.21.15.42:0/4021761397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40240f5a50 0x7f402410dbe0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.313+0000 7f4029e53700 1 -- 172.21.15.42:0/4021761397 >> 172.21.15.42:0/4021761397 conn(0x7f40240f04c0 msgr2=0x7f40240f9200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.313+0000 7f4029e53700 1 -- 172.21.15.42:0/4021761397 shutdown_connections 2024-02-15T18:33:03.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.313+0000 7f4029e53700 1 -- 172.21.15.42:0/4021761397 wait complete. 2024-02-15T18:33:03.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.433+0000 7f2a3eb88700 1 -- 172.21.15.42:0/3663557298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a380f6c70 msgr2=0x7f2a380f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:03.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.433+0000 7f2a3eb88700 1 --2- 172.21.15.42:0/3663557298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a380f6c70 0x7f2a380f7090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f2a2c004660 tx=0x7f2a2c009b30).stop 2024-02-15T18:33:03.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.433+0000 7f2a3eb88700 1 -- 172.21.15.42:0/3663557298 shutdown_connections 2024-02-15T18:33:03.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.433+0000 7f2a3eb88700 1 --2- 172.21.15.42:0/3663557298 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a380f7610 0x7f2a380fdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.433+0000 7f2a3eb88700 1 --2- 172.21.15.42:0/3663557298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a380f6c70 0x7f2a380f7090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.433+0000 7f2a3eb88700 1 -- 172.21.15.42:0/3663557298 >> 172.21.15.42:0/3663557298 conn(0x7f2a380f04c0 msgr2=0x7f2a380f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:03.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.433+0000 7f2a3eb88700 1 -- 172.21.15.42:0/3663557298 shutdown_connections 2024-02-15T18:33:03.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.433+0000 7f2a3eb88700 1 -- 172.21.15.42:0/3663557298 wait complete. 2024-02-15T18:33:03.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.434+0000 7f2a3eb88700 1 Processor -- start 2024-02-15T18:33:03.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.434+0000 7f2a3eb88700 1 -- start start 2024-02-15T18:33:03.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.434+0000 7f2a3eb88700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a380f6c70 0x7f2a38109220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:03.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.434+0000 7f2a3eb88700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a380f7610 0x7f2a38109710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:03.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.434+0000 7f2a3eb88700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a38109cc0 con 0x7f2a380f6c70 2024-02-15T18:33:03.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.434+0000 7f2a3eb88700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a38109e00 con 0x7f2a380f7610 2024-02-15T18:33:03.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.435+0000 7f2a3c924700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a380f6c70 0x7f2a38109220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:03.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.435+0000 7f2a3c924700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a380f6c70 0x7f2a38109220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43224/0 (socket says 172.21.15.42:43224) 2024-02-15T18:33:03.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.435+0000 7f2a3c924700 1 -- 172.21.15.42:0/4250304230 learned_addr learned my addr 172.21.15.42:0/4250304230 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:33:03.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.435+0000 7f2a37fff700 1 --2- 172.21.15.42:0/4250304230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a380f7610 0x7f2a38109710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:03.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.435+0000 7f2a3c924700 1 -- 172.21.15.42:0/4250304230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a380f7610 msgr2=0x7f2a38109710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:03.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.435+0000 7f2a3c924700 1 --2- 172.21.15.42:0/4250304230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a380f7610 0x7f2a38109710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.435+0000 7f2a3c924700 1 -- 172.21.15.42:0/4250304230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a24005040 con 0x7f2a380f6c70 2024-02-15T18:33:03.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.435+0000 7f2a37fff700 1 --2- 172.21.15.42:0/4250304230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a380f7610 0x7f2a38109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:33:03.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.435+0000 7f2a3c924700 1 --2- 172.21.15.42:0/4250304230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a380f6c70 0x7f2a38109220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f2a2c000c00 tx=0x7f2a2c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:03.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.435+0000 7f2a357fa700 1 -- 172.21.15.42:0/4250304230 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a2c019070 con 0x7f2a380f6c70 2024-02-15T18:33:03.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.435+0000 7f2a357fa700 1 -- 172.21.15.42:0/4250304230 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f2a2c007e80 con 0x7f2a380f6c70 2024-02-15T18:33:03.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.435+0000 7f2a3eb88700 1 -- 172.21.15.42:0/4250304230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c005040 con 0x7f2a380f6c70 2024-02-15T18:33:03.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.436+0000 7f2a357fa700 1 -- 172.21.15.42:0/4250304230 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a2c014e80 con 0x7f2a380f6c70 2024-02-15T18:33:03.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.436+0000 7f2a3eb88700 1 -- 172.21.15.42:0/4250304230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3810ab10 con 0x7f2a380f6c70 2024-02-15T18:33:03.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.437+0000 7f2a3eb88700 1 -- 172.21.15.42:0/4250304230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a380f4150 con 0x7f2a380f6c70 2024-02-15T18:33:03.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.437+0000 7f2a357fa700 1 -- 172.21.15.42:0/4250304230 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f2a2c014620 con 0x7f2a380f6c70 2024-02-15T18:33:03.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.438+0000 7f2a357fa700 1 --2- 172.21.15.42:0/4250304230 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2a28060ec0 0x7f2a28063380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:03.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.438+0000 7f2a357fa700 1 -- 172.21.15.42:0/4250304230 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f2a2c07cae0 con 0x7f2a380f6c70 2024-02-15T18:33:03.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.438+0000 7f2a37fff700 1 --2- 172.21.15.42:0/4250304230 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2a28060ec0 0x7f2a28063380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:03.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.438+0000 7f2a37fff700 1 --2- 172.21.15.42:0/4250304230 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2a28060ec0 0x7f2a28063380 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f2a380f7a90 tx=0x7f2a24009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:03.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.446+0000 7f2a357fa700 1 -- 172.21.15.42:0/4250304230 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2a2c04af60 con 0x7f2a380f6c70 2024-02-15T18:33:03.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.679+0000 7f2a3eb88700 1 -- 172.21.15.42:0/4250304230 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a381031e0 con 0x7f2a28060ec0 2024-02-15T18:33:03.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.697+0000 7f2a357fa700 1 -- 172.21.15.42:0/4250304230 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f2a381031e0 con 0x7f2a28060ec0 2024-02-15T18:33:03.690 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:33:03.690 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (12m) 35s ago 14m 21.4M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:33:03.690 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (5m) 35s ago 14m 6643k - 16.2.14-521-g0e714d9a 1fbc92c859dc 2da9d875539a 2024-02-15T18:33:03.690 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (5m) 9s ago 13m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 5c46a68ff62b 2024-02-15T18:33:03.690 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (12m) 35s ago 14m 33.9M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:33:03.690 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (41s) 35s ago 9m 12.2M - 16.2.14-521-g0e714d9a 1fbc92c859dc 39e47ffbe32b 2024-02-15T18:33:03.690 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (54s) 35s ago 9m 11.5M - 16.2.14-521-g0e714d9a 1fbc92c859dc 372b8d41fced 2024-02-15T18:33:03.690 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (12s) 9s ago 9m 11.3M - 16.2.14-521-g0e714d9a 1fbc92c859dc 4eb6a75798d1 2024-02-15T18:33:03.690 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (22s) 9s ago 9m 12.2M - 16.2.14-521-g0e714d9a 1fbc92c859dc 551cd0ff7860 2024-02-15T18:33:03.690 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (7m) 35s ago 16m 438M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:33:03.690 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (6m) 9s ago 13m 390M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:33:03.691 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (6m) 35s ago 16m 59.7M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:33:03.691 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (5m) 9s ago 13m 53.9M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:33:03.691 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (14m) 35s ago 14m 17.0M - 0.18.1 e5a616e4b9cf 388191816a46 2024-02-15T18:33:03.691 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (13m) 9s ago 13m 17.2M - 0.18.1 e5a616e4b9cf 6596d7ad4f86 2024-02-15T18:33:03.691 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (4m) 35s ago 12m 310M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc f226bea76cab 2024-02-15T18:33:03.691 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (3m) 35s ago 12m 271M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 5114ed026724 2024-02-15T18:33:03.691 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (3m) 35s ago 11m 190M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 2e26b1ec292a 2024-02-15T18:33:03.691 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (2m) 9s ago 11m 338M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc c71b210091cb 2024-02-15T18:33:03.691 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (111s) 9s ago 11m 285M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 83bd9cdcb3da 2024-02-15T18:33:03.691 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (67s) 9s ago 10m 243M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc d45e566ed288 2024-02-15T18:33:03.691 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (7m) 35s ago 13m 63.6M - 2.18.1 de242295e225 63b6dfb825ed 2024-02-15T18:33:03.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.700+0000 7f2a3eb88700 1 -- 172.21.15.42:0/4250304230 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2a28060ec0 msgr2=0x7f2a28063380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:03.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.700+0000 7f2a3eb88700 1 --2- 172.21.15.42:0/4250304230 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2a28060ec0 0x7f2a28063380 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f2a380f7a90 tx=0x7f2a24009b30).stop 2024-02-15T18:33:03.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.700+0000 7f2a3eb88700 1 -- 172.21.15.42:0/4250304230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a380f6c70 msgr2=0x7f2a38109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:03.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.700+0000 7f2a3eb88700 1 --2- 172.21.15.42:0/4250304230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a380f6c70 0x7f2a38109220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f2a2c000c00 tx=0x7f2a2c00b040).stop 2024-02-15T18:33:03.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.700+0000 7f2a3eb88700 1 -- 172.21.15.42:0/4250304230 shutdown_connections 2024-02-15T18:33:03.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.700+0000 7f2a3eb88700 1 --2- 172.21.15.42:0/4250304230 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2a28060ec0 0x7f2a28063380 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.700+0000 7f2a3eb88700 1 --2- 172.21.15.42:0/4250304230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a380f6c70 0x7f2a38109220 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.700+0000 7f2a3eb88700 1 --2- 172.21.15.42:0/4250304230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a380f7610 0x7f2a38109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.700+0000 7f2a3eb88700 1 -- 172.21.15.42:0/4250304230 >> 172.21.15.42:0/4250304230 conn(0x7f2a380f04c0 msgr2=0x7f2a380f28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:03.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.701+0000 7f2a3eb88700 1 -- 172.21.15.42:0/4250304230 shutdown_connections 2024-02-15T18:33:03.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.701+0000 7f2a3eb88700 1 -- 172.21.15.42:0/4250304230 wait complete. 2024-02-15T18:33:03.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.806+0000 7f0690793700 1 -- 172.21.15.42:0/3813874606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06880f55c0 msgr2=0x7f06880f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:03.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.806+0000 7f0690793700 1 --2- 172.21.15.42:0/3813874606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06880f55c0 0x7f06880f59e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f0684004660 tx=0x7f0684009b30).stop 2024-02-15T18:33:03.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.806+0000 7f0690793700 1 -- 172.21.15.42:0/3813874606 shutdown_connections 2024-02-15T18:33:03.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.806+0000 7f0690793700 1 --2- 172.21.15.42:0/3813874606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06880f6860 0x7f06880f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.806+0000 7f0690793700 1 --2- 172.21.15.42:0/3813874606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06880f55c0 0x7f06880f59e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.806+0000 7f0690793700 1 -- 172.21.15.42:0/3813874606 >> 172.21.15.42:0/3813874606 conn(0x7f06880f0d40 msgr2=0x7f06880f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:03.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.806+0000 7f0690793700 1 -- 172.21.15.42:0/3813874606 shutdown_connections 2024-02-15T18:33:03.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.806+0000 7f0690793700 1 -- 172.21.15.42:0/3813874606 wait complete. 2024-02-15T18:33:03.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.807+0000 7f0690793700 1 Processor -- start 2024-02-15T18:33:03.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.807+0000 7f0690793700 1 -- start start 2024-02-15T18:33:03.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.807+0000 7f0690793700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06880f55c0 0x7f0688105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:03.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.807+0000 7f0690793700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06880f6860 0x7f06881063a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:03.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.807+0000 7f0690793700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0688106950 con 0x7f06880f55c0 2024-02-15T18:33:03.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.807+0000 7f0690793700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0688106a90 con 0x7f06880f6860 2024-02-15T18:33:03.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.807+0000 7f068e52f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06880f55c0 0x7f0688105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:03.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.807+0000 7f068e52f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06880f55c0 0x7f0688105eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43238/0 (socket says 172.21.15.42:43238) 2024-02-15T18:33:03.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.807+0000 7f068e52f700 1 -- 172.21.15.42:0/4079383345 learned_addr learned my addr 172.21.15.42:0/4079383345 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:33:03.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.807+0000 7f068dd2e700 1 --2- 172.21.15.42:0/4079383345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06880f6860 0x7f06881063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:03.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.807+0000 7f068e52f700 1 -- 172.21.15.42:0/4079383345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06880f6860 msgr2=0x7f06881063a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:03.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.807+0000 7f068e52f700 1 --2- 172.21.15.42:0/4079383345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06880f6860 0x7f06881063a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:03.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.807+0000 7f068e52f700 1 -- 172.21.15.42:0/4079383345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0684005040 con 0x7f06880f55c0 2024-02-15T18:33:03.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.808+0000 7f068e52f700 1 --2- 172.21.15.42:0/4079383345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06880f55c0 0x7f0688105eb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f0684013fd0 tx=0x7f0684009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:03.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.808+0000 7f067effd700 1 -- 172.21.15.42:0/4079383345 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0684018070 con 0x7f06880f55c0 2024-02-15T18:33:03.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.808+0000 7f0690793700 1 -- 172.21.15.42:0/4079383345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06881073d0 con 0x7f06880f55c0 2024-02-15T18:33:03.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.808+0000 7f067effd700 1 -- 172.21.15.42:0/4079383345 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f0684007c70 con 0x7f06880f55c0 2024-02-15T18:33:03.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.808+0000 7f067effd700 1 -- 172.21.15.42:0/4079383345 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f068401c620 con 0x7f06880f55c0 2024-02-15T18:33:03.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.808+0000 7f0690793700 1 -- 172.21.15.42:0/4079383345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0688107930 con 0x7f06880f55c0 2024-02-15T18:33:03.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.809+0000 7f0690793700 1 -- 172.21.15.42:0/4079383345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0688045ad0 con 0x7f06880f55c0 2024-02-15T18:33:03.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.809+0000 7f067effd700 1 -- 172.21.15.42:0/4079383345 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f068401d420 con 0x7f06880f55c0 2024-02-15T18:33:03.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.810+0000 7f067effd700 1 --2- 172.21.15.42:0/4079383345 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0674060f20 0x7f06740633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:03.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.810+0000 7f067effd700 1 -- 172.21.15.42:0/4079383345 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f068407dff0 con 0x7f06880f55c0 2024-02-15T18:33:03.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.810+0000 7f068dd2e700 1 --2- 172.21.15.42:0/4079383345 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0674060f20 0x7f06740633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:03.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.810+0000 7f068dd2e700 1 --2- 172.21.15.42:0/4079383345 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0674060f20 0x7f06740633e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f06780041d0 tx=0x7f0678009040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:03.814+0000 7f067effd700 1 -- 172.21.15.42:0/4079383345 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0684051db0 con 0x7f06880f55c0 2024-02-15T18:33:04.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.061+0000 7f0690793700 1 -- 172.21.15.42:0/4079383345 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f06880fdf60 con 0x7f06880f55c0 2024-02-15T18:33:04.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.062+0000 7f067effd700 1 -- 172.21.15.42:0/4079383345 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+648 (secure 0 0 0) 0x7f0684007de0 con 0x7f06880f55c0 2024-02-15T18:33:04.054 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:33:04.054 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:33:04.055 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:33:04.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:33:04.055 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:33:04.055 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:33:04.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:33:04.055 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:33:04.055 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 6 2024-02-15T18:33:04.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:33:04.055 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:33:04.055 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4 2024-02-15T18:33:04.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:33:04.055 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:33:04.055 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 14 2024-02-15T18:33:04.055 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:33:04.056 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:33:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.063+0000 7f0690793700 1 -- 172.21.15.42:0/4079383345 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0674060f20 msgr2=0x7f06740633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:04.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.063+0000 7f0690793700 1 --2- 172.21.15.42:0/4079383345 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0674060f20 0x7f06740633e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f06780041d0 tx=0x7f0678009040).stop 2024-02-15T18:33:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.064+0000 7f0690793700 1 -- 172.21.15.42:0/4079383345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06880f55c0 msgr2=0x7f0688105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.064+0000 7f0690793700 1 --2- 172.21.15.42:0/4079383345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06880f55c0 0x7f0688105eb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f0684013fd0 tx=0x7f0684009b30).stop 2024-02-15T18:33:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.064+0000 7f0690793700 1 -- 172.21.15.42:0/4079383345 shutdown_connections 2024-02-15T18:33:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.064+0000 7f0690793700 1 --2- 172.21.15.42:0/4079383345 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f0674060f20 0x7f06740633e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.064+0000 7f0690793700 1 --2- 172.21.15.42:0/4079383345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06880f55c0 0x7f0688105eb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.064+0000 7f0690793700 1 --2- 172.21.15.42:0/4079383345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06880f6860 0x7f06881063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.064+0000 7f0690793700 1 -- 172.21.15.42:0/4079383345 >> 172.21.15.42:0/4079383345 conn(0x7f06880f0d40 msgr2=0x7f06880fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.064+0000 7f0690793700 1 -- 172.21.15.42:0/4079383345 shutdown_connections 2024-02-15T18:33:04.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.064+0000 7f0690793700 1 -- 172.21.15.42:0/4079383345 wait complete. 2024-02-15T18:33:04.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.172+0000 7fc65c548700 1 -- 172.21.15.42:0/2100274927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6540f66e0 msgr2=0x7fc6540f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:04.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.172+0000 7fc65c548700 1 --2- 172.21.15.42:0/2100274927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6540f66e0 0x7fc6540f6b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fc650004660 tx=0x7fc650009b30).stop 2024-02-15T18:33:04.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.172+0000 7fc65c548700 1 -- 172.21.15.42:0/2100274927 shutdown_connections 2024-02-15T18:33:04.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.172+0000 7fc65c548700 1 --2- 172.21.15.42:0/2100274927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6540f7980 0x7fc6540f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:04.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.172+0000 7fc65c548700 1 --2- 172.21.15.42:0/2100274927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6540f66e0 0x7fc6540f6b00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:04.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.172+0000 7fc65c548700 1 -- 172.21.15.42:0/2100274927 >> 172.21.15.42:0/2100274927 conn(0x7fc6540f1e40 msgr2=0x7fc6540f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:04.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.172+0000 7fc65c548700 1 -- 172.21.15.42:0/2100274927 shutdown_connections 2024-02-15T18:33:04.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.172+0000 7fc65c548700 1 -- 172.21.15.42:0/2100274927 wait complete. 2024-02-15T18:33:04.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.172+0000 7fc65c548700 1 Processor -- start 2024-02-15T18:33:04.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.172+0000 7fc65c548700 1 -- start start 2024-02-15T18:33:04.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.173+0000 7fc65c548700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6540f66e0 0x7fc654107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:04.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.173+0000 7fc65c548700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6540f7980 0x7fc6541074f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:04.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.173+0000 7fc65c548700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654107aa0 con 0x7fc6540f7980 2024-02-15T18:33:04.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.173+0000 7fc65c548700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654107be0 con 0x7fc6540f66e0 2024-02-15T18:33:04.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.173+0000 7fc659ae3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6540f7980 0x7fc6541074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:04.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.173+0000 7fc659ae3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6540f7980 0x7fc6541074f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43252/0 (socket says 172.21.15.42:43252) 2024-02-15T18:33:04.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.173+0000 7fc659ae3700 1 -- 172.21.15.42:0/2916136068 learned_addr learned my addr 172.21.15.42:0/2916136068 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:33:04.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.173+0000 7fc659ae3700 1 -- 172.21.15.42:0/2916136068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6540f66e0 msgr2=0x7fc654107000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:04.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.173+0000 7fc659ae3700 1 --2- 172.21.15.42:0/2916136068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6540f66e0 0x7fc654107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:33:04.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.173+0000 7fc659ae3700 1 -- 172.21.15.42:0/2916136068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc650005040 con 0x7fc6540f7980 2024-02-15T18:33:04.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.173+0000 7fc659ae3700 1 --2- 172.21.15.42:0/2916136068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6540f7980 0x7fc6541074f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fc644002700 tx=0x7fc644009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:04.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.173+0000 7fc64affd700 1 -- 172.21.15.42:0/2916136068 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc644009320 con 0x7fc6540f7980 2024-02-15T18:33:04.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.173+0000 7fc65c548700 1 -- 172.21.15.42:0/2916136068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc654108580 con 0x7fc6540f7980 2024-02-15T18:33:04.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.174+0000 7fc65c548700 1 -- 172.21.15.42:0/2916136068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc654108ac0 con 0x7fc6540f7980 2024-02-15T18:33:04.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.174+0000 7fc64affd700 1 -- 172.21.15.42:0/2916136068 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fc64400f040 con 0x7fc6540f7980 2024-02-15T18:33:04.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.174+0000 7fc64affd700 1 -- 172.21.15.42:0/2916136068 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc644014970 con 0x7fc6540f7980 2024-02-15T18:33:04.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.175+0000 7fc65c548700 1 -- 172.21.15.42:0/2916136068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc63c004fa0 con 0x7fc6540f7980 2024-02-15T18:33:04.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.176+0000 7fc64affd700 1 -- 172.21.15.42:0/2916136068 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fc644009480 con 0x7fc6540f7980 2024-02-15T18:33:04.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.176+0000 7fc64affd700 1 --2- 172.21.15.42:0/2916136068 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc640060f70 0x7fc640063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:04.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.176+0000 7fc64affd700 1 -- 172.21.15.42:0/2916136068 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fc64407c5f0 con 0x7fc6540f7980 2024-02-15T18:33:04.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.176+0000 7fc65a2e4700 1 --2- 172.21.15.42:0/2916136068 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc640060f70 0x7fc640063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:04.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.177+0000 7fc65a2e4700 1 --2- 172.21.15.42:0/2916136068 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc640060f70 0x7fc640063430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fc650009f50 tx=0x7fc650009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:04.173 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:02.771653+0000 mgr.smithi042.kfoihb (mgr.24435) 351 : cluster [DBG] pgmap v276: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-02-15T18:33:04.173 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:02.960718+0000 mgr.smithi042.kfoihb (mgr.24435) 352 : audit [DBG] from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:04.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.184+0000 7fc64affd700 1 -- 172.21.15.42:0/2916136068 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc644050270 con 0x7fc6540f7980 2024-02-15T18:33:04.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:02.771653+0000 mgr.smithi042.kfoihb (mgr.24435) 351 : cluster [DBG] pgmap v276: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-02-15T18:33:04.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:02.960718+0000 mgr.smithi042.kfoihb (mgr.24435) 352 : audit [DBG] from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:04.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.436+0000 7fc65c548700 1 -- 172.21.15.42:0/2916136068 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc63c005e80 con 0x7fc6540f7980 2024-02-15T18:33:04.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.436+0000 7fc64affd700 1 -- 172.21.15.42:0/2916136068 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1774 (secure 0 0 0) 0x7fc64404fe60 con 0x7fc6540f7980 2024-02-15T18:33:04.429 INFO:teuthology.orchestra.run.smithi042.stdout:e28 2024-02-15T18:33:04.429 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:33:04.429 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:33:04.429 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:33:04.429 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:33:04.429 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:33:04.430 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:33:04.430 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 28 2024-02-15T18:33:04.430 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:33:04.430 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:33:04.430 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:32:55.244270+0000 2024-02-15T18:33:04.430 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:33:04.430 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:33:04.430 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:33:04.430 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:33:04.430 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:33:04.430 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:33:04.430 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 113 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=34356} 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{0:34356} state up:active seq 12 join_fscid=1 addr [v2:172.21.15.42:6826/2962563293,v1:172.21.15.42:6827/2962563293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:33:04.431 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{0:34360} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/1562993914,v1:172.21.15.42:6829/1562993914] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:33:04.432 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:33:04.432 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:33:04.432 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:33:04.432 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:33:04.432 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.191:6824/540280968,v1:172.21.15.191:6825/540280968] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:33:04.432 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.191:6826/289198038,v1:172.21.15.191:6827/289198038] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:33:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.438+0000 7fc65c548700 1 -- 172.21.15.42:0/2916136068 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc640060f70 msgr2=0x7fc640063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.438+0000 7fc65c548700 1 --2- 172.21.15.42:0/2916136068 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc640060f70 0x7fc640063430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fc650009f50 tx=0x7fc650009b30).stop 2024-02-15T18:33:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.438+0000 7fc65c548700 1 -- 172.21.15.42:0/2916136068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6540f7980 msgr2=0x7fc6541074f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.438+0000 7fc65c548700 1 --2- 172.21.15.42:0/2916136068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6540f7980 0x7fc6541074f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fc644002700 tx=0x7fc644009e00).stop 2024-02-15T18:33:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.438+0000 7fc65c548700 1 -- 172.21.15.42:0/2916136068 shutdown_connections 2024-02-15T18:33:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.438+0000 7fc65c548700 1 --2- 172.21.15.42:0/2916136068 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc640060f70 0x7fc640063430 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.438+0000 7fc65c548700 1 --2- 172.21.15.42:0/2916136068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc6540f66e0 0x7fc654107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:33:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.438+0000 7fc65c548700 1 --2- 172.21.15.42:0/2916136068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6540f7980 0x7fc6541074f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.438+0000 7fc65c548700 1 -- 172.21.15.42:0/2916136068 >> 172.21.15.42:0/2916136068 conn(0x7fc6540f1e40 msgr2=0x7fc6540fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.438+0000 7fc65c548700 1 -- 172.21.15.42:0/2916136068 shutdown_connections 2024-02-15T18:33:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.438+0000 7fc65c548700 1 -- 172.21.15.42:0/2916136068 wait complete. 2024-02-15T18:33:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 28 2024-02-15T18:33:04.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.551+0000 7f9fcd40b700 1 -- 172.21.15.42:0/2862578617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fc80fba90 msgr2=0x7f9fc80fde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:04.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.551+0000 7f9fcd40b700 1 --2- 172.21.15.42:0/2862578617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fc80fba90 0x7f9fc80fde80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f9fbc0045a0 tx=0x7f9fbc009b30).stop 2024-02-15T18:33:04.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.551+0000 7f9fcd40b700 1 -- 172.21.15.42:0/2862578617 shutdown_connections 2024-02-15T18:33:04.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.551+0000 7f9fcd40b700 1 --2- 172.21.15.42:0/2862578617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fc80fba90 0x7f9fc80fde80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:04.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.551+0000 7f9fcd40b700 1 --2- 172.21.15.42:0/2862578617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc80602b0 0x7f9fc80fb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:04.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.551+0000 7f9fcd40b700 1 -- 172.21.15.42:0/2862578617 >> 172.21.15.42:0/2862578617 conn(0x7f9fc80f0bc0 msgr2=0x7f9fc80f2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:04.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.551+0000 7f9fcd40b700 1 -- 172.21.15.42:0/2862578617 shutdown_connections 2024-02-15T18:33:04.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.551+0000 7f9fcd40b700 1 -- 172.21.15.42:0/2862578617 wait complete. 2024-02-15T18:33:04.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.552+0000 7f9fcd40b700 1 Processor -- start 2024-02-15T18:33:04.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.552+0000 7f9fcd40b700 1 -- start start 2024-02-15T18:33:04.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.552+0000 7f9fcd40b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fc80602b0 0x7f9fc8108fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:04.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.552+0000 7f9fcd40b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc80fba90 0x7f9fc81094d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:04.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.552+0000 7f9fcd40b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc8109a80 con 0x7f9fc80fba90 2024-02-15T18:33:04.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.552+0000 7f9fcd40b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc8109bc0 con 0x7f9fc80602b0 2024-02-15T18:33:04.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.552+0000 7f9fc67fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc80fba90 0x7f9fc81094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:04.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.552+0000 7f9fc67fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc80fba90 0x7f9fc81094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43260/0 (socket says 172.21.15.42:43260) 2024-02-15T18:33:04.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.553+0000 7f9fc67fc700 1 -- 172.21.15.42:0/4069994926 learned_addr learned my addr 172.21.15.42:0/4069994926 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:33:04.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.553+0000 7f9fc6ffd700 1 --2- 172.21.15.42:0/4069994926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fc80602b0 0x7f9fc8108fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:04.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.553+0000 7f9fc67fc700 1 -- 172.21.15.42:0/4069994926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fc80602b0 msgr2=0x7f9fc8108fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:04.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.553+0000 7f9fc67fc700 1 --2- 172.21.15.42:0/4069994926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fc80602b0 0x7f9fc8108fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:04.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.553+0000 7f9fc67fc700 1 -- 172.21.15.42:0/4069994926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc005040 con 0x7f9fc80fba90 2024-02-15T18:33:04.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.553+0000 7f9fc67fc700 1 --2- 172.21.15.42:0/4069994926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc80fba90 0x7f9fc81094d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f9fbc014040 tx=0x7f9fbc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:04.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.553+0000 7f9faf7fe700 1 -- 172.21.15.42:0/4069994926 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fbc00e030 con 0x7f9fc80fba90 2024-02-15T18:33:04.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.553+0000 7f9faf7fe700 1 -- 172.21.15.42:0/4069994926 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f9fbc007ba0 con 0x7f9fc80fba90 2024-02-15T18:33:04.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.553+0000 7f9fcd40b700 1 -- 172.21.15.42:0/4069994926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc810a500 con 0x7f9fc80fba90 2024-02-15T18:33:04.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.553+0000 7f9faf7fe700 1 -- 172.21.15.42:0/4069994926 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fbc019070 con 0x7f9fc80fba90 2024-02-15T18:33:04.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.553+0000 7f9fcd40b700 1 -- 172.21.15.42:0/4069994926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc810a9e0 con 0x7f9fc80fba90 2024-02-15T18:33:04.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.555+0000 7f9fcd40b700 1 -- 172.21.15.42:0/4069994926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc8045ad0 con 0x7f9fc80fba90 2024-02-15T18:33:04.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.555+0000 7f9faf7fe700 1 -- 172.21.15.42:0/4069994926 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f9fbc025070 con 0x7f9fc80fba90 2024-02-15T18:33:04.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.556+0000 7f9faf7fe700 1 --2- 172.21.15.42:0/4069994926 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9fb0060f10 0x7f9fb00633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:04.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.556+0000 7f9faf7fe700 1 -- 172.21.15.42:0/4069994926 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f9fbc00f070 con 0x7f9fc80fba90 2024-02-15T18:33:04.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.556+0000 7f9fc6ffd700 1 --2- 172.21.15.42:0/4069994926 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9fb0060f10 0x7f9fb00633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:04.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.556+0000 7f9fc6ffd700 1 --2- 172.21.15.42:0/4069994926 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9fb0060f10 0x7f9fb00633d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f9fb8009880 tx=0x7f9fb800b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:04.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.564+0000 7f9faf7fe700 1 -- 172.21.15.42:0/4069994926 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9fbc080030 con 0x7f9fc80fba90 2024-02-15T18:33:04.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.778+0000 7f9fcd40b700 1 -- 172.21.15.42:0/4069994926 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fc8101140 con 0x7f9fb0060f10 2024-02-15T18:33:04.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.779+0000 7f9faf7fe700 1 -- 172.21.15.42:0/4069994926 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f9fc8101140 con 0x7f9fb0060f10 2024-02-15T18:33:04.772 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:33:04.772 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:33:04.772 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:33:04.772 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:33:04.772 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:33:04.772 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-02-15T18:33:04.772 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-02-15T18:33:04.772 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-02-15T18:33:04.772 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr", 2024-02-15T18:33:04.772 INFO:teuthology.orchestra.run.smithi042.stdout: "crash" 2024-02-15T18:33:04.772 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:33:04.773 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/21 daemons upgraded", 2024-02-15T18:33:04.773 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading node-exporter daemons", 2024-02-15T18:33:04.773 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:33:04.773 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:33:04.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.781+0000 7f9fcd40b700 1 -- 172.21.15.42:0/4069994926 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9fb0060f10 msgr2=0x7f9fb00633d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:04.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.781+0000 7f9fcd40b700 1 --2- 172.21.15.42:0/4069994926 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9fb0060f10 0x7f9fb00633d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f9fb8009880 tx=0x7f9fb800b040).stop 2024-02-15T18:33:04.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.781+0000 7f9fcd40b700 1 -- 172.21.15.42:0/4069994926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc80fba90 msgr2=0x7f9fc81094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:04.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.781+0000 7f9fcd40b700 1 --2- 172.21.15.42:0/4069994926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc80fba90 0x7f9fc81094d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f9fbc014040 tx=0x7f9fbc00b040).stop 2024-02-15T18:33:04.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.781+0000 7f9fcd40b700 1 -- 172.21.15.42:0/4069994926 shutdown_connections 2024-02-15T18:33:04.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.781+0000 7f9fcd40b700 1 --2- 172.21.15.42:0/4069994926 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f9fb0060f10 0x7f9fb00633d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:04.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.781+0000 7f9fcd40b700 1 --2- 172.21.15.42:0/4069994926 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fc80602b0 0x7f9fc8108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:04.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.781+0000 7f9fcd40b700 1 --2- 172.21.15.42:0/4069994926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9fc80fba90 0x7f9fc81094d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:04.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.781+0000 7f9fcd40b700 1 -- 172.21.15.42:0/4069994926 >> 172.21.15.42:0/4069994926 conn(0x7f9fc80f0bc0 msgr2=0x7f9fc80f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:04.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.781+0000 7f9fcd40b700 1 -- 172.21.15.42:0/4069994926 shutdown_connections 2024-02-15T18:33:04.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:04.781+0000 7f9fcd40b700 1 -- 172.21.15.42:0/4069994926 wait complete. 2024-02-15T18:33:05.087 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:03.311489+0000 mgr.smithi042.kfoihb (mgr.24435) 353 : audit [DBG] from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:05.087 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:03.680675+0000 mgr.smithi042.kfoihb (mgr.24435) 354 : audit [DBG] from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:05.087 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:04.062947+0000 mon.smithi042 (mon.0) 633 : audit [DBG] from='client.? 172.21.15.42:0/4079383345' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:05.087 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:04.437424+0000 mon.smithi042 (mon.0) 634 : audit [DBG] from='client.? 172.21.15.42:0/2916136068' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:33:05.087 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:04 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:04.577349+0000 mon.smithi042 (mon.0) 635 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:05.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:03.311489+0000 mgr.smithi042.kfoihb (mgr.24435) 353 : audit [DBG] from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:05.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:03.680675+0000 mgr.smithi042.kfoihb (mgr.24435) 354 : audit [DBG] from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:05.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:04.062947+0000 mon.smithi042 (mon.0) 633 : audit [DBG] from='client.? 172.21.15.42:0/4079383345' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:05.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:04.437424+0000 mon.smithi042 (mon.0) 634 : audit [DBG] from='client.? 172.21.15.42:0/2916136068' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:33:05.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:04 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:04.577349+0000 mon.smithi042 (mon.0) 635 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:06.106 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:04.771959+0000 mgr.smithi042.kfoihb (mgr.24435) 355 : cluster [DBG] pgmap v277: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-02-15T18:33:06.106 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:04.780111+0000 mgr.smithi042.kfoihb (mgr.24435) 356 : audit [DBG] from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:06.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:04.771959+0000 mgr.smithi042.kfoihb (mgr.24435) 355 : cluster [DBG] pgmap v277: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-02-15T18:33:06.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:04.780111+0000 mgr.smithi042.kfoihb (mgr.24435) 356 : audit [DBG] from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:07.164 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:33:06.134480+0000 mgr.smithi042.kfoihb (mgr.24435) 357 : cephadm [INF] Upgrade: Updating node-exporter.smithi191 (2/2) 2024-02-15T18:33:07.164 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:33:06.134676+0000 mgr.smithi042.kfoihb (mgr.24435) 358 : cephadm [INF] Deploying daemon node-exporter.smithi191 on smithi191 2024-02-15T18:33:07.164 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:06.772389+0000 mgr.smithi042.kfoihb (mgr.24435) 359 : cluster [DBG] pgmap v278: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 16 MiB/s rd, 4 op/s 2024-02-15T18:33:07.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:33:06.134480+0000 mgr.smithi042.kfoihb (mgr.24435) 357 : cephadm [INF] Upgrade: Updating node-exporter.smithi191 (2/2) 2024-02-15T18:33:07.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:33:06.134676+0000 mgr.smithi042.kfoihb (mgr.24435) 358 : cephadm [INF] Deploying daemon node-exporter.smithi191 on smithi191 2024-02-15T18:33:07.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:06.772389+0000 mgr.smithi042.kfoihb (mgr.24435) 359 : cluster [DBG] pgmap v278: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 16 MiB/s rd, 4 op/s 2024-02-15T18:33:10.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:08.772896+0000 mgr.smithi042.kfoihb (mgr.24435) 360 : cluster [DBG] pgmap v279: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-02-15T18:33:10.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:08.772896+0000 mgr.smithi042.kfoihb (mgr.24435) 360 : cluster [DBG] pgmap v279: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-02-15T18:33:12.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:10.773371+0000 mgr.smithi042.kfoihb (mgr.24435) 361 : cluster [DBG] pgmap v280: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-02-15T18:33:12.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:10.966666+0000 mon.smithi042 (mon.0) 636 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:12.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:10.967599+0000 mon.smithi042 (mon.0) 637 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:12.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:10.773371+0000 mgr.smithi042.kfoihb (mgr.24435) 361 : cluster [DBG] pgmap v280: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-02-15T18:33:12.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:10.966666+0000 mon.smithi042 (mon.0) 636 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:12.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:10.967599+0000 mon.smithi042 (mon.0) 637 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:14.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:12.773885+0000 mgr.smithi042.kfoihb (mgr.24435) 362 : cluster [DBG] pgmap v281: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 5.2 MiB/s rd, 2 op/s 2024-02-15T18:33:14.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:13.143167+0000 mon.smithi042 (mon.0) 638 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:14.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:12.773885+0000 mgr.smithi042.kfoihb (mgr.24435) 362 : cluster [DBG] pgmap v281: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 5.2 MiB/s rd, 2 op/s 2024-02-15T18:33:14.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:13.143167+0000 mon.smithi042 (mon.0) 638 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:16.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:14.774198+0000 mgr.smithi042.kfoihb (mgr.24435) 363 : cluster [DBG] pgmap v282: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 1 op/s 2024-02-15T18:33:16.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:14.774198+0000 mgr.smithi042.kfoihb (mgr.24435) 363 : cluster [DBG] pgmap v282: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 1 op/s 2024-02-15T18:33:17.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:15.873294+0000 mon.smithi042 (mon.0) 639 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:17.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.175726+0000 mon.smithi042 (mon.0) 640 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:33:17.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.176403+0000 mon.smithi042 (mon.0) 641 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:33:17.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.180437+0000 mon.smithi042 (mon.0) 642 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:17.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.185078+0000 mon.smithi042 (mon.0) 643 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:17.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.187018+0000 mon.smithi042 (mon.0) 644 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.188023+0000 mon.smithi042 (mon.0) 645 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.188965+0000 mon.smithi042 (mon.0) 646 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.190264+0000 mon.smithi042 (mon.0) 647 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.191957+0000 mon.smithi042 (mon.0) 648 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.192944+0000 mon.smithi042 (mon.0) 649 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.193930+0000 mon.smithi042 (mon.0) 650 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.194898+0000 mon.smithi042 (mon.0) 651 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.195879+0000 mon.smithi042 (mon.0) 652 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.196847+0000 mon.smithi042 (mon.0) 653 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:16 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:16.198023+0000 mon.smithi042 (mon.0) 654 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:15.873294+0000 mon.smithi042 (mon.0) 639 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:17.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.175726+0000 mon.smithi042 (mon.0) 640 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:33:17.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.176403+0000 mon.smithi042 (mon.0) 641 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:33:17.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.180437+0000 mon.smithi042 (mon.0) 642 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:17.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.185078+0000 mon.smithi042 (mon.0) 643 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:17.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.187018+0000 mon.smithi042 (mon.0) 644 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.188023+0000 mon.smithi042 (mon.0) 645 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.188965+0000 mon.smithi042 (mon.0) 646 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.190264+0000 mon.smithi042 (mon.0) 647 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.191957+0000 mon.smithi042 (mon.0) 648 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.192944+0000 mon.smithi042 (mon.0) 649 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.193930+0000 mon.smithi042 (mon.0) 650 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.194898+0000 mon.smithi042 (mon.0) 651 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.195879+0000 mon.smithi042 (mon.0) 652 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.196847+0000 mon.smithi042 (mon.0) 653 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:17.193 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:16 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:16.198023+0000 mon.smithi042 (mon.0) 654 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:18.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:33:16.176922+0000 mgr.smithi042.kfoihb (mgr.24435) 364 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:33:18.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:16.774831+0000 mgr.smithi042.kfoihb (mgr.24435) 365 : cluster [DBG] pgmap v283: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2 op/s 2024-02-15T18:33:18.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:33:16.176922+0000 mgr.smithi042.kfoihb (mgr.24435) 364 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:33:18.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:16.774831+0000 mgr.smithi042.kfoihb (mgr.24435) 365 : cluster [DBG] pgmap v283: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2 op/s 2024-02-15T18:33:19.054 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:33:17.896572+0000 mgr.smithi042.kfoihb (mgr.24435) 366 : cephadm [INF] Upgrade: Updating prometheus.smithi042 2024-02-15T18:33:19.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:18 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:33:17.908367+0000 mgr.smithi042.kfoihb (mgr.24435) 367 : cephadm [INF] Deploying daemon prometheus.smithi042 on smithi042 2024-02-15T18:33:19.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:33:17.896572+0000 mgr.smithi042.kfoihb (mgr.24435) 366 : cephadm [INF] Upgrade: Updating prometheus.smithi042 2024-02-15T18:33:19.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:18 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:33:17.908367+0000 mgr.smithi042.kfoihb (mgr.24435) 367 : cephadm [INF] Deploying daemon prometheus.smithi042 on smithi042 2024-02-15T18:33:20.039 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:18.775263+0000 mgr.smithi042.kfoihb (mgr.24435) 368 : cluster [DBG] pgmap v284: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 342 KiB/s rd, 1 op/s 2024-02-15T18:33:20.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:18.775263+0000 mgr.smithi042.kfoihb (mgr.24435) 368 : cluster [DBG] pgmap v284: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 342 KiB/s rd, 1 op/s 2024-02-15T18:33:22.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:20.775854+0000 mgr.smithi042.kfoihb (mgr.24435) 369 : cluster [DBG] pgmap v285: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:22.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:20.775854+0000 mgr.smithi042.kfoihb (mgr.24435) 369 : cluster [DBG] pgmap v285: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:24.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:22.776206+0000 mgr.smithi042.kfoihb (mgr.24435) 370 : cluster [DBG] pgmap v286: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:24.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:23.203204+0000 mon.smithi042 (mon.0) 655 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:24.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:23.204001+0000 mon.smithi042 (mon.0) 656 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:24.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:22.776206+0000 mgr.smithi042.kfoihb (mgr.24435) 370 : cluster [DBG] pgmap v286: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:24.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:23.203204+0000 mon.smithi042 (mon.0) 655 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:24.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:23.204001+0000 mon.smithi042 (mon.0) 656 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:26.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:24.776501+0000 mgr.smithi042.kfoihb (mgr.24435) 371 : cluster [DBG] pgmap v287: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:26.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:24.776501+0000 mgr.smithi042.kfoihb (mgr.24435) 371 : cluster [DBG] pgmap v287: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:27.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:26.776956+0000 mgr.smithi042.kfoihb (mgr.24435) 372 : cluster [DBG] pgmap v288: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:27.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:26.776956+0000 mgr.smithi042.kfoihb (mgr.24435) 372 : cluster [DBG] pgmap v288: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:30.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:28.777337+0000 mgr.smithi042.kfoihb (mgr.24435) 373 : cluster [DBG] pgmap v289: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:30.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:28.777337+0000 mgr.smithi042.kfoihb (mgr.24435) 373 : cluster [DBG] pgmap v289: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:32.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:30.777900+0000 mgr.smithi042.kfoihb (mgr.24435) 374 : cluster [DBG] pgmap v290: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:32.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:30.777900+0000 mgr.smithi042.kfoihb (mgr.24435) 374 : cluster [DBG] pgmap v290: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:33.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.006662+0000 mon.smithi042 (mon.0) 657 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:33.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.007415+0000 mon.smithi042 (mon.0) 658 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:33:33.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.008022+0000 mon.smithi042 (mon.0) 659 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:33:33.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:33:32.008542+0000 mgr.smithi042.kfoihb (mgr.24435) 375 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:33:33.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.011353+0000 mon.smithi042 (mon.0) 660 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:33.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.017202+0000 mon.smithi042 (mon.0) 661 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:33:33.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.017557+0000 mgr.smithi042.kfoihb (mgr.24435) 376 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:33:33.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.018883+0000 mon.smithi042 (mon.0) 662 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:33.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.020877+0000 mon.smithi042 (mon.0) 663 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.021945+0000 mon.smithi042 (mon.0) 664 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.022969+0000 mon.smithi042 (mon.0) 665 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.024328+0000 mon.smithi042 (mon.0) 666 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.025886+0000 mon.smithi042 (mon.0) 667 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.026745+0000 mon.smithi042 (mon.0) 668 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.027581+0000 mon.smithi042 (mon.0) 669 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.028422+0000 mon.smithi042 (mon.0) 670 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.029248+0000 mon.smithi042 (mon.0) 671 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.030070+0000 mon.smithi042 (mon.0) 672 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.031198+0000 mon.smithi042 (mon.0) 673 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:32.032212+0000 mon.smithi042 (mon.0) 674 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:32.778335+0000 mgr.smithi042.kfoihb (mgr.24435) 377 : cluster [DBG] pgmap v291: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:33.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.006662+0000 mon.smithi042 (mon.0) 657 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:33.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.007415+0000 mon.smithi042 (mon.0) 658 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:33:33.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.008022+0000 mon.smithi042 (mon.0) 659 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:33:33.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:33:32.008542+0000 mgr.smithi042.kfoihb (mgr.24435) 375 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:33:33.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.011353+0000 mon.smithi042 (mon.0) 660 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:33.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.017202+0000 mon.smithi042 (mon.0) 661 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:33:33.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.017557+0000 mgr.smithi042.kfoihb (mgr.24435) 376 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-15T18:33:33.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.018883+0000 mon.smithi042 (mon.0) 662 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:33.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.020877+0000 mon.smithi042 (mon.0) 663 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.021945+0000 mon.smithi042 (mon.0) 664 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.022969+0000 mon.smithi042 (mon.0) 665 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.024328+0000 mon.smithi042 (mon.0) 666 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.025886+0000 mon.smithi042 (mon.0) 667 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.026745+0000 mon.smithi042 (mon.0) 668 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.027581+0000 mon.smithi042 (mon.0) 669 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.028422+0000 mon.smithi042 (mon.0) 670 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.029248+0000 mon.smithi042 (mon.0) 671 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.030070+0000 mon.smithi042 (mon.0) 672 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.443 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.031198+0000 mon.smithi042 (mon.0) 673 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:32.032212+0000 mon.smithi042 (mon.0) 674 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:33.444 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:32.778335+0000 mgr.smithi042.kfoihb (mgr.24435) 377 : cluster [DBG] pgmap v291: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:34.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:33:33.504500+0000 mgr.smithi042.kfoihb (mgr.24435) 378 : cephadm [INF] Upgrade: Updating alertmanager.smithi042 2024-02-15T18:33:34.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:34 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:33:33.512084+0000 mgr.smithi042.kfoihb (mgr.24435) 379 : cephadm [INF] Deploying daemon alertmanager.smithi042 on smithi042 2024-02-15T18:33:34.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:33:33.504500+0000 mgr.smithi042.kfoihb (mgr.24435) 378 : cephadm [INF] Upgrade: Updating alertmanager.smithi042 2024-02-15T18:33:34.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:34 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:33:33.512084+0000 mgr.smithi042.kfoihb (mgr.24435) 379 : cephadm [INF] Deploying daemon alertmanager.smithi042 on smithi042 2024-02-15T18:33:34.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.898+0000 7f52e149c700 1 -- 172.21.15.42:0/1721976026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc0f55e0 msgr2=0x7f52dc0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:34.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.898+0000 7f52e149c700 1 --2- 172.21.15.42:0/1721976026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc0f55e0 0x7f52dc0f5a00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f52cc004660 tx=0x7f52cc009b30).stop 2024-02-15T18:33:34.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.898+0000 7f52e149c700 1 -- 172.21.15.42:0/1721976026 shutdown_connections 2024-02-15T18:33:34.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.898+0000 7f52e149c700 1 --2- 172.21.15.42:0/1721976026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52dc0f5ef0 0x7f52dc0f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:34.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.898+0000 7f52e149c700 1 --2- 172.21.15.42:0/1721976026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc0f55e0 0x7f52dc0f5a00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:34.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.898+0000 7f52e149c700 1 -- 172.21.15.42:0/1721976026 >> 172.21.15.42:0/1721976026 conn(0x7f52dc0f0de0 msgr2=0x7f52dc0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:34.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.898+0000 7f52e149c700 1 -- 172.21.15.42:0/1721976026 shutdown_connections 2024-02-15T18:33:34.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.898+0000 7f52e149c700 1 -- 172.21.15.42:0/1721976026 wait complete. 2024-02-15T18:33:34.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.898+0000 7f52e149c700 1 Processor -- start 2024-02-15T18:33:34.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.899+0000 7f52e149c700 1 -- start start 2024-02-15T18:33:34.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.899+0000 7f52e149c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc0f55e0 0x7f52dc0f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:34.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.899+0000 7f52e149c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52dc0f5ef0 0x7f52dc0f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:34.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.899+0000 7f52e149c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc0f81c0 con 0x7f52dc0f55e0 2024-02-15T18:33:34.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.899+0000 7f52e149c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc0f8300 con 0x7f52dc0f5ef0 2024-02-15T18:33:34.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.899+0000 7f52daffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc0f55e0 0x7f52dc0f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:34.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.899+0000 7f52daffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc0f55e0 0x7f52dc0f97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44230/0 (socket says 172.21.15.42:44230) 2024-02-15T18:33:34.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.899+0000 7f52daffd700 1 -- 172.21.15.42:0/3588524525 learned_addr learned my addr 172.21.15.42:0/3588524525 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:33:34.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.899+0000 7f52da7fc700 1 --2- 172.21.15.42:0/3588524525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52dc0f5ef0 0x7f52dc0f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:34.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.899+0000 7f52daffd700 1 -- 172.21.15.42:0/3588524525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52dc0f5ef0 msgr2=0x7f52dc0f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:34.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.899+0000 7f52daffd700 1 --2- 172.21.15.42:0/3588524525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52dc0f5ef0 0x7f52dc0f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:34.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.899+0000 7f52daffd700 1 -- 172.21.15.42:0/3588524525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc005040 con 0x7f52dc0f55e0 2024-02-15T18:33:34.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.899+0000 7f52da7fc700 1 --2- 172.21.15.42:0/3588524525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52dc0f5ef0 0x7f52dc0f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:33:34.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.899+0000 7f52daffd700 1 --2- 172.21.15.42:0/3588524525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc0f55e0 0x7f52dc0f97e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f52cc009f50 tx=0x7f52cc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:34.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.900+0000 7f52c37fe700 1 -- 172.21.15.42:0/3588524525 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52cc018070 con 0x7f52dc0f55e0 2024-02-15T18:33:34.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.900+0000 7f52c37fe700 1 -- 172.21.15.42:0/3588524525 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f52cc007ba0 con 0x7f52dc0f55e0 2024-02-15T18:33:34.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.900+0000 7f52c37fe700 1 -- 172.21.15.42:0/3588524525 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52cc01c620 con 0x7f52dc0f55e0 2024-02-15T18:33:34.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.900+0000 7f52e149c700 1 -- 172.21.15.42:0/3588524525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc0f8580 con 0x7f52dc0f55e0 2024-02-15T18:33:34.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.900+0000 7f52e149c700 1 -- 172.21.15.42:0/3588524525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc0f8ae0 con 0x7f52dc0f55e0 2024-02-15T18:33:34.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.901+0000 7f52e149c700 1 -- 172.21.15.42:0/3588524525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52dc045ad0 con 0x7f52dc0f55e0 2024-02-15T18:33:34.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.901+0000 7f52c37fe700 1 -- 172.21.15.42:0/3588524525 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f52cc01d420 con 0x7f52dc0f55e0 2024-02-15T18:33:34.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.901+0000 7f52c37fe700 1 --2- 172.21.15.42:0/3588524525 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f52c4060f20 0x7f52c40633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:34.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.901+0000 7f52c37fe700 1 -- 172.21.15.42:0/3588524525 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f52cc07e010 con 0x7f52dc0f55e0 2024-02-15T18:33:34.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.901+0000 7f52da7fc700 1 --2- 172.21.15.42:0/3588524525 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f52c4060f20 0x7f52c40633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:34.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.902+0000 7f52da7fc700 1 --2- 172.21.15.42:0/3588524525 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f52c4060f20 0x7f52c40633e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f52d0009780 tx=0x7f52d0009c80).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:34.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:34.906+0000 7f52c37fe700 1 -- 172.21.15.42:0/3588524525 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f52cc04e340 con 0x7f52dc0f55e0 2024-02-15T18:33:35.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.098+0000 7f52e149c700 1 -- 172.21.15.42:0/3588524525 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f52dc0fad60 con 0x7f52c4060f20 2024-02-15T18:33:35.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.100+0000 7f52c37fe700 1 -- 172.21.15.42:0/3588524525 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7f52dc0fad60 con 0x7f52c4060f20 2024-02-15T18:33:35.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.101+0000 7f52e149c700 1 -- 172.21.15.42:0/3588524525 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f52c4060f20 msgr2=0x7f52c40633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.101+0000 7f52e149c700 1 --2- 172.21.15.42:0/3588524525 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f52c4060f20 0x7f52c40633e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f52d0009780 tx=0x7f52d0009c80).stop 2024-02-15T18:33:35.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.101+0000 7f52e149c700 1 -- 172.21.15.42:0/3588524525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc0f55e0 msgr2=0x7f52dc0f97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.101+0000 7f52e149c700 1 --2- 172.21.15.42:0/3588524525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc0f55e0 0x7f52dc0f97e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f52cc009f50 tx=0x7f52cc009b30).stop 2024-02-15T18:33:35.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.101+0000 7f52e149c700 1 -- 172.21.15.42:0/3588524525 shutdown_connections 2024-02-15T18:33:35.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.101+0000 7f52e149c700 1 --2- 172.21.15.42:0/3588524525 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f52c4060f20 0x7f52c40633e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.101+0000 7f52e149c700 1 --2- 172.21.15.42:0/3588524525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc0f55e0 0x7f52dc0f97e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.101+0000 7f52e149c700 1 --2- 172.21.15.42:0/3588524525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52dc0f5ef0 0x7f52dc0f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.101+0000 7f52e149c700 1 -- 172.21.15.42:0/3588524525 >> 172.21.15.42:0/3588524525 conn(0x7f52dc0f0de0 msgr2=0x7f52dc05ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:35.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.101+0000 7f52e149c700 1 -- 172.21.15.42:0/3588524525 shutdown_connections 2024-02-15T18:33:35.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.101+0000 7f52e149c700 1 -- 172.21.15.42:0/3588524525 wait complete. 2024-02-15T18:33:35.103 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:33:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.209+0000 7f6849411700 1 -- 172.21.15.42:0/2935830854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6844060290 msgr2=0x7f68440f9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.209+0000 7f6849411700 1 --2- 172.21.15.42:0/2935830854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6844060290 0x7f68440f9190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f6834004660 tx=0x7f6834009b30).stop 2024-02-15T18:33:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.209+0000 7f6849411700 1 -- 172.21.15.42:0/2935830854 shutdown_connections 2024-02-15T18:33:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.209+0000 7f6849411700 1 --2- 172.21.15.42:0/2935830854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68440f9710 0x7f68440fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.209+0000 7f6849411700 1 --2- 172.21.15.42:0/2935830854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6844060290 0x7f68440f9190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.209+0000 7f6849411700 1 -- 172.21.15.42:0/2935830854 >> 172.21.15.42:0/2935830854 conn(0x7f68440f0b90 msgr2=0x7f68440f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:35.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.209+0000 7f6849411700 1 -- 172.21.15.42:0/2935830854 shutdown_connections 2024-02-15T18:33:35.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.210+0000 7f6849411700 1 -- 172.21.15.42:0/2935830854 wait complete. 2024-02-15T18:33:35.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.210+0000 7f6849411700 1 Processor -- start 2024-02-15T18:33:35.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.210+0000 7f6849411700 1 -- start start 2024-02-15T18:33:35.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.210+0000 7f6849411700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6844060290 0x7f6844106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.210+0000 7f6849411700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68440f9710 0x7f68441073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.210+0000 7f6849411700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6844107960 con 0x7f68440f9710 2024-02-15T18:33:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.210+0000 7f6849411700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6844108160 con 0x7f6844060290 2024-02-15T18:33:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.211+0000 7f68437fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68440f9710 0x7f68441073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.211+0000 7f68437fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68440f9710 0x7f68441073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44244/0 (socket says 172.21.15.42:44244) 2024-02-15T18:33:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.211+0000 7f68437fe700 1 -- 172.21.15.42:0/2222926931 learned_addr learned my addr 172.21.15.42:0/2222926931 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:33:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.211+0000 7f6843fff700 1 --2- 172.21.15.42:0/2222926931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6844060290 0x7f6844106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.211+0000 7f68437fe700 1 -- 172.21.15.42:0/2222926931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6844060290 msgr2=0x7f6844106ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.211+0000 7f68437fe700 1 --2- 172.21.15.42:0/2222926931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6844060290 0x7f6844106ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.211+0000 7f68437fe700 1 -- 172.21.15.42:0/2222926931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6834005040 con 0x7f68440f9710 2024-02-15T18:33:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.211+0000 7f68437fe700 1 --2- 172.21.15.42:0/2222926931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68440f9710 0x7f68441073b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f6838002700 tx=0x7f6838009da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.211+0000 7f6840ff9700 1 -- 172.21.15.42:0/2222926931 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68380079c0 con 0x7f68440f9710 2024-02-15T18:33:35.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.211+0000 7f6840ff9700 1 -- 172.21.15.42:0/2222926931 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f683800f040 con 0x7f68440f9710 2024-02-15T18:33:35.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.211+0000 7f6840ff9700 1 -- 172.21.15.42:0/2222926931 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68380149b0 con 0x7f68440f9710 2024-02-15T18:33:35.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.211+0000 7f6849411700 1 -- 172.21.15.42:0/2222926931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6844108440 con 0x7f68440f9710 2024-02-15T18:33:35.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.211+0000 7f6849411700 1 -- 172.21.15.42:0/2222926931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6844108a00 con 0x7f68440f9710 2024-02-15T18:33:35.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.212+0000 7f6849411700 1 -- 172.21.15.42:0/2222926931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6844059100 con 0x7f68440f9710 2024-02-15T18:33:35.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.213+0000 7f6840ff9700 1 -- 172.21.15.42:0/2222926931 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f683801b060 con 0x7f68440f9710 2024-02-15T18:33:35.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.213+0000 7f6840ff9700 1 --2- 172.21.15.42:0/2222926931 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f682c060f70 0x7f682c063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.213+0000 7f6840ff9700 1 -- 172.21.15.42:0/2222926931 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f683807c020 con 0x7f68440f9710 2024-02-15T18:33:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.213+0000 7f6843fff700 1 --2- 172.21.15.42:0/2222926931 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f682c060f70 0x7f682c063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:35.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.214+0000 7f6843fff700 1 --2- 172.21.15.42:0/2222926931 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f682c060f70 0x7f682c063430 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f6834004630 tx=0x7f6834009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:35.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.218+0000 7f6840ff9700 1 -- 172.21.15.42:0/2222926931 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f683804fde0 con 0x7f68440f9710 2024-02-15T18:33:35.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:34.778704+0000 mgr.smithi042.kfoihb (mgr.24435) 380 : cluster [DBG] pgmap v292: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:35.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.415+0000 7f6849411700 1 -- 172.21.15.42:0/2222926931 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68440008d0 con 0x7f682c060f70 2024-02-15T18:33:35.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.416+0000 7f6840ff9700 1 -- 172.21.15.42:0/2222926931 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7f68440008d0 con 0x7f682c060f70 2024-02-15T18:33:35.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.419+0000 7f6849411700 1 -- 172.21.15.42:0/2222926931 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f682c060f70 msgr2=0x7f682c063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.419+0000 7f6849411700 1 --2- 172.21.15.42:0/2222926931 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f682c060f70 0x7f682c063430 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f6834004630 tx=0x7f6834009b30).stop 2024-02-15T18:33:35.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.419+0000 7f6849411700 1 -- 172.21.15.42:0/2222926931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68440f9710 msgr2=0x7f68441073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.419+0000 7f6849411700 1 --2- 172.21.15.42:0/2222926931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68440f9710 0x7f68441073b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f6838002700 tx=0x7f6838009da0).stop 2024-02-15T18:33:35.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.419+0000 7f6849411700 1 -- 172.21.15.42:0/2222926931 shutdown_connections 2024-02-15T18:33:35.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.419+0000 7f6849411700 1 --2- 172.21.15.42:0/2222926931 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f682c060f70 0x7f682c063430 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.419+0000 7f6849411700 1 --2- 172.21.15.42:0/2222926931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6844060290 0x7f6844106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.419+0000 7f6849411700 1 --2- 172.21.15.42:0/2222926931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68440f9710 0x7f68441073b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.419+0000 7f6849411700 1 -- 172.21.15.42:0/2222926931 >> 172.21.15.42:0/2222926931 conn(0x7f68440f0b90 msgr2=0x7f68440f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:35.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.419+0000 7f6849411700 1 -- 172.21.15.42:0/2222926931 shutdown_connections 2024-02-15T18:33:35.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.419+0000 7f6849411700 1 -- 172.21.15.42:0/2222926931 wait complete. 2024-02-15T18:33:35.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:34.778704+0000 mgr.smithi042.kfoihb (mgr.24435) 380 : cluster [DBG] pgmap v292: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:35.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.525+0000 7fc25359e700 1 -- 172.21.15.42:0/1158115599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2540f5430 msgr2=0x7fc2540f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.525+0000 7fc25359e700 1 --2- 172.21.15.42:0/1158115599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2540f5430 0x7fc2540f5850 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fc244004660 tx=0x7fc244009b30).stop 2024-02-15T18:33:35.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.525+0000 7fc25359e700 1 -- 172.21.15.42:0/1158115599 shutdown_connections 2024-02-15T18:33:35.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.525+0000 7fc25359e700 1 --2- 172.21.15.42:0/1158115599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2540f66d0 0x7fc2540f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:33:35.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.525+0000 7fc25359e700 1 --2- 172.21.15.42:0/1158115599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2540f5430 0x7fc2540f5850 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.525+0000 7fc25359e700 1 -- 172.21.15.42:0/1158115599 >> 172.21.15.42:0/1158115599 conn(0x7fc2540f0b90 msgr2=0x7fc2540f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:35.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.526+0000 7fc25359e700 1 -- 172.21.15.42:0/1158115599 shutdown_connections 2024-02-15T18:33:35.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.526+0000 7fc25359e700 1 -- 172.21.15.42:0/1158115599 wait complete. 2024-02-15T18:33:35.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.526+0000 7fc25359e700 1 Processor -- start 2024-02-15T18:33:35.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.526+0000 7fc25359e700 1 -- start start 2024-02-15T18:33:35.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.526+0000 7fc25359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2540f5430 0x7fc2540f9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:35.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.526+0000 7fc25359e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2540f66d0 0x7fc2540f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:35.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.526+0000 7fc25359e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540f8000 con 0x7fc2540f66d0 2024-02-15T18:33:35.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.526+0000 7fc25359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540f8140 con 0x7fc2540f5430 2024-02-15T18:33:35.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.526+0000 7fc251d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2540f66d0 0x7fc2540f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:35.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.526+0000 7fc251d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2540f66d0 0x7fc2540f7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44262/0 (socket says 172.21.15.42:44262) 2024-02-15T18:33:35.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.527+0000 7fc251d9b700 1 -- 172.21.15.42:0/1740059316 learned_addr learned my addr 172.21.15.42:0/1740059316 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:33:35.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.527+0000 7fc25259c700 1 --2- 172.21.15.42:0/1740059316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2540f5430 0x7fc2540f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:35.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.527+0000 7fc251d9b700 1 -- 172.21.15.42:0/1740059316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2540f5430 msgr2=0x7fc2540f9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.527+0000 7fc251d9b700 1 --2- 172.21.15.42:0/1740059316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2540f5430 0x7fc2540f9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.527+0000 7fc251d9b700 1 -- 172.21.15.42:0/1740059316 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc244005040 con 0x7fc2540f66d0 2024-02-15T18:33:35.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.527+0000 7fc25259c700 1 --2- 172.21.15.42:0/1740059316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2540f5430 0x7fc2540f9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:33:35.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.527+0000 7fc251d9b700 1 --2- 172.21.15.42:0/1740059316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2540f66d0 0x7fc2540f7a10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fc24800b410 tx=0x7fc248006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:35.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.527+0000 7fc242ffd700 1 -- 172.21.15.42:0/1740059316 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2480125f0 con 0x7fc2540f66d0 2024-02-15T18:33:35.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.527+0000 7fc25359e700 1 -- 172.21.15.42:0/1740059316 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2540f8420 con 0x7fc2540f66d0 2024-02-15T18:33:35.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.527+0000 7fc242ffd700 1 -- 172.21.15.42:0/1740059316 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fc24800be30 con 0x7fc2540f66d0 2024-02-15T18:33:35.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.527+0000 7fc242ffd700 1 -- 172.21.15.42:0/1740059316 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc248002b70 con 0x7fc2540f66d0 2024-02-15T18:33:35.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.527+0000 7fc25359e700 1 -- 172.21.15.42:0/1740059316 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25410ee90 con 0x7fc2540f66d0 2024-02-15T18:33:35.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.528+0000 7fc25359e700 1 -- 172.21.15.42:0/1740059316 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc254045ad0 con 0x7fc2540f66d0 2024-02-15T18:33:35.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.529+0000 7fc242ffd700 1 -- 172.21.15.42:0/1740059316 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fc24800f030 con 0x7fc2540f66d0 2024-02-15T18:33:35.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.529+0000 7fc242ffd700 1 --2- 172.21.15.42:0/1740059316 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc23c060f70 0x7fc23c063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:35.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.529+0000 7fc242ffd700 1 -- 172.21.15.42:0/1740059316 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fc24807c6a0 con 0x7fc2540f66d0 2024-02-15T18:33:35.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.530+0000 7fc25259c700 1 --2- 172.21.15.42:0/1740059316 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc23c060f70 0x7fc23c063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:35.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.530+0000 7fc25259c700 1 --2- 172.21.15.42:0/1740059316 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc23c060f70 0x7fc23c063430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fc2540f9320 tx=0x7fc244009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:35.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.534+0000 7fc242ffd700 1 -- 172.21.15.42:0/1740059316 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc24804c950 con 0x7fc2540f66d0 2024-02-15T18:33:35.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.756+0000 7fc25359e700 1 -- 172.21.15.42:0/1740059316 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2540fab10 con 0x7fc23c060f70 2024-02-15T18:33:35.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.768+0000 7fc242ffd700 1 -- 172.21.15.42:0/1740059316 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fc2540fab10 con 0x7fc23c060f70 2024-02-15T18:33:35.760 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:33:35.760 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (13m) 3s ago 15m 21.4M - 0.20.0 0881eb8f169f 9328d0e8d42a 2024-02-15T18:33:35.760 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (5m) 3s ago 15m 6643k - 16.2.14-521-g0e714d9a 1fbc92c859dc 2da9d875539a 2024-02-15T18:33:35.760 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (5m) 22s ago 14m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 5c46a68ff62b 2024-02-15T18:33:35.760 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (13m) 3s ago 14m 35.5M - 6.7.4 557c83e11646 5d8f5cea7db4 2024-02-15T18:33:35.760 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (73s) 3s ago 9m 82.4M - 16.2.14-521-g0e714d9a 1fbc92c859dc 39e47ffbe32b 2024-02-15T18:33:35.760 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (87s) 3s ago 9m 80.4M - 16.2.14-521-g0e714d9a 1fbc92c859dc 372b8d41fced 2024-02-15T18:33:35.760 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (44s) 22s ago 9m 11.5M - 16.2.14-521-g0e714d9a 1fbc92c859dc 4eb6a75798d1 2024-02-15T18:33:35.760 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (54s) 22s ago 9m 12.3M - 16.2.14-521-g0e714d9a 1fbc92c859dc 551cd0ff7860 2024-02-15T18:33:35.760 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (7m) 3s ago 16m 441M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:33:35.761 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (7m) 22s ago 14m 390M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:33:35.761 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (6m) 3s ago 16m 64.5M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:33:35.761 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (6m) 22s ago 13m 56.0M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:33:35.761 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (31s) 3s ago 14m 12.6M - 1.3.1 1dbe0e931976 e80f2a2363ff 2024-02-15T18:33:35.761 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (25s) 22s ago 13m 11.3M - 1.3.1 1dbe0e931976 ee2cf38a118f 2024-02-15T18:33:35.761 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (5m) 3s ago 13m 309M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc f226bea76cab 2024-02-15T18:33:35.761 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (4m) 3s ago 12m 279M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 5114ed026724 2024-02-15T18:33:35.761 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (3m) 3s ago 12m 190M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 2e26b1ec292a 2024-02-15T18:33:35.761 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (3m) 22s ago 12m 339M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc c71b210091cb 2024-02-15T18:33:35.761 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (2m) 22s ago 11m 285M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 83bd9cdcb3da 2024-02-15T18:33:35.761 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (99s) 22s ago 11m 243M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc d45e566ed288 2024-02-15T18:33:35.761 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (13s) 3s ago 14m 35.4M - 2.33.4 514e6a882f6e b0c2ac6c79f3 2024-02-15T18:33:35.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.769+0000 7fc25359e700 1 -- 172.21.15.42:0/1740059316 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc23c060f70 msgr2=0x7fc23c063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.769+0000 7fc25359e700 1 --2- 172.21.15.42:0/1740059316 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc23c060f70 0x7fc23c063430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fc2540f9320 tx=0x7fc244009b30).stop 2024-02-15T18:33:35.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.770+0000 7fc25359e700 1 -- 172.21.15.42:0/1740059316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2540f66d0 msgr2=0x7fc2540f7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.770+0000 7fc25359e700 1 --2- 172.21.15.42:0/1740059316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2540f66d0 0x7fc2540f7a10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7fc24800b410 tx=0x7fc248006ef0).stop 2024-02-15T18:33:35.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.770+0000 7fc25359e700 1 -- 172.21.15.42:0/1740059316 shutdown_connections 2024-02-15T18:33:35.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.770+0000 7fc25359e700 1 --2- 172.21.15.42:0/1740059316 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fc23c060f70 0x7fc23c063430 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.770+0000 7fc25359e700 1 --2- 172.21.15.42:0/1740059316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2540f5430 0x7fc2540f9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.770+0000 7fc25359e700 1 --2- 172.21.15.42:0/1740059316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2540f66d0 0x7fc2540f7a10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.770+0000 7fc25359e700 1 -- 172.21.15.42:0/1740059316 >> 172.21.15.42:0/1740059316 conn(0x7fc2540f0b90 msgr2=0x7fc25405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:35.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.770+0000 7fc25359e700 1 -- 172.21.15.42:0/1740059316 shutdown_connections 2024-02-15T18:33:35.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.770+0000 7fc25359e700 1 -- 172.21.15.42:0/1740059316 wait complete. 2024-02-15T18:33:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.881+0000 7fbae2700700 1 -- 172.21.15.42:0/3713835433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbadc0f77d0 msgr2=0x7fbadc0f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.881+0000 7fbae2700700 1 --2- 172.21.15.42:0/3713835433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbadc0f77d0 0x7fbadc0f7c30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fbacc004660 tx=0x7fbacc009b30).stop 2024-02-15T18:33:35.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.881+0000 7fbae2700700 1 -- 172.21.15.42:0/3713835433 shutdown_connections 2024-02-15T18:33:35.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.881+0000 7fbae2700700 1 --2- 172.21.15.42:0/3713835433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbadc0f77d0 0x7fbadc0f7c30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.881+0000 7fbae2700700 1 --2- 172.21.15.42:0/3713835433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbadc0f6530 0x7fbadc0f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:33:35.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.881+0000 7fbae2700700 1 -- 172.21.15.42:0/3713835433 >> 172.21.15.42:0/3713835433 conn(0x7fbadc0f1c90 msgr2=0x7fbadc0f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:35.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.881+0000 7fbae2700700 1 -- 172.21.15.42:0/3713835433 shutdown_connections 2024-02-15T18:33:35.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.881+0000 7fbae2700700 1 -- 172.21.15.42:0/3713835433 wait complete. 2024-02-15T18:33:35.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.882+0000 7fbae2700700 1 Processor -- start 2024-02-15T18:33:35.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.882+0000 7fbae2700700 1 -- start start 2024-02-15T18:33:35.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.882+0000 7fbae2700700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbadc0f6530 0x7fbadc1090d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:35.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.882+0000 7fbae2700700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbadc0f77d0 0x7fbadc1095c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:35.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.882+0000 7fbae2700700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc109c00 con 0x7fbadc0f6530 2024-02-15T18:33:35.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.882+0000 7fbae2700700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc10a4b0 con 0x7fbadc0f77d0 2024-02-15T18:33:35.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.882+0000 7fbae16fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbadc0f6530 0x7fbadc1090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:35.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.882+0000 7fbae16fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbadc0f6530 0x7fbadc1090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44278/0 (socket says 172.21.15.42:44278) 2024-02-15T18:33:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.882+0000 7fbae16fe700 1 -- 172.21.15.42:0/3001460171 learned_addr learned my addr 172.21.15.42:0/3001460171 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:33:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.883+0000 7fbae0efd700 1 --2- 172.21.15.42:0/3001460171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbadc0f77d0 0x7fbadc1095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.883+0000 7fbae16fe700 1 -- 172.21.15.42:0/3001460171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbadc0f77d0 msgr2=0x7fbadc1095c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.883+0000 7fbae16fe700 1 --2- 172.21.15.42:0/3001460171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbadc0f77d0 0x7fbadc1095c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.883+0000 7fbae16fe700 1 -- 172.21.15.42:0/3001460171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc005040 con 0x7fbadc0f6530 2024-02-15T18:33:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.883+0000 7fbae0efd700 1 --2- 172.21.15.42:0/3001460171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbadc0f77d0 0x7fbadc1095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:33:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.883+0000 7fbae16fe700 1 --2- 172.21.15.42:0/3001460171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbadc0f6530 0x7fbadc1090d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fbad8009360 tx=0x7fbad8006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.883+0000 7fbad1ffb700 1 -- 172.21.15.42:0/3001460171 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbad8013460 con 0x7fbadc0f6530 2024-02-15T18:33:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.883+0000 7fbad1ffb700 1 -- 172.21.15.42:0/3001460171 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fbad8013ab0 con 0x7fbadc0f6530 2024-02-15T18:33:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.883+0000 7fbae2700700 1 -- 172.21.15.42:0/3001460171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbadc10a790 con 0x7fbadc0f6530 2024-02-15T18:33:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.883+0000 7fbad1ffb700 1 -- 172.21.15.42:0/3001460171 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbad800db40 con 0x7fbadc0f6530 2024-02-15T18:33:35.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.883+0000 7fbae2700700 1 -- 172.21.15.42:0/3001460171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbadc10ad50 con 0x7fbadc0f6530 2024-02-15T18:33:35.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.884+0000 7fbae2700700 1 -- 172.21.15.42:0/3001460171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbadc059100 con 0x7fbadc0f6530 2024-02-15T18:33:35.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.885+0000 7fbad1ffb700 1 -- 172.21.15.42:0/3001460171 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fbad801a060 con 0x7fbadc0f6530 2024-02-15T18:33:35.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.885+0000 7fbad1ffb700 1 --2- 172.21.15.42:0/3001460171 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbac8060f70 0x7fbac8063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:35.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.885+0000 7fbad1ffb700 1 -- 172.21.15.42:0/3001460171 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fbad807c100 con 0x7fbadc0f6530 2024-02-15T18:33:35.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.885+0000 7fbae0efd700 1 --2- 172.21.15.42:0/3001460171 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbac8060f70 0x7fbac8063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:35.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.886+0000 7fbae0efd700 1 --2- 172.21.15.42:0/3001460171 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbac8060f70 0x7fbac8063430 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fbacc005010 tx=0x7fbacc009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:35.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:35.891+0000 7fbad1ffb700 1 -- 172.21.15.42:0/3001460171 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbad804feb0 con 0x7fbadc0f6530 2024-02-15T18:33:36.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.154+0000 7fbae2700700 1 -- 172.21.15.42:0/3001460171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbadc027a00 con 0x7fbadc0f6530 2024-02-15T18:33:36.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.155+0000 7fbad1ffb700 1 -- 172.21.15.42:0/3001460171 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+648 (secure 0 0 0) 0x7fbad804faa0 con 0x7fbadc0f6530 2024-02-15T18:33:36.147 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:33:36.147 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:33:36.147 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:33:36.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:33:36.147 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:33:36.147 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:33:36.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:33:36.147 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:33:36.147 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 6 2024-02-15T18:33:36.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:33:36.147 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:33:36.147 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4 2024-02-15T18:33:36.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:33:36.148 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:33:36.148 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 14 2024-02-15T18:33:36.148 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:33:36.148 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:33:36.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.156+0000 7fbae2700700 1 -- 172.21.15.42:0/3001460171 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbac8060f70 msgr2=0x7fbac8063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:36.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.156+0000 7fbae2700700 1 --2- 172.21.15.42:0/3001460171 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbac8060f70 0x7fbac8063430 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fbacc005010 tx=0x7fbacc009b30).stop 2024-02-15T18:33:36.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.156+0000 7fbae2700700 1 -- 172.21.15.42:0/3001460171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbadc0f6530 msgr2=0x7fbadc1090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:36.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.156+0000 7fbae2700700 1 --2- 172.21.15.42:0/3001460171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbadc0f6530 0x7fbadc1090d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fbad8009360 tx=0x7fbad8006e10).stop 2024-02-15T18:33:36.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.157+0000 7fbae2700700 1 -- 172.21.15.42:0/3001460171 shutdown_connections 2024-02-15T18:33:36.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.157+0000 7fbae2700700 1 --2- 172.21.15.42:0/3001460171 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbac8060f70 0x7fbac8063430 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:36.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.157+0000 7fbae2700700 1 --2- 172.21.15.42:0/3001460171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbadc0f6530 0x7fbadc1090d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:36.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.157+0000 7fbae2700700 1 --2- 172.21.15.42:0/3001460171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbadc0f77d0 0x7fbadc1095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:36.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.157+0000 7fbae2700700 1 -- 172.21.15.42:0/3001460171 >> 172.21.15.42:0/3001460171 conn(0x7fbadc0f1c90 msgr2=0x7fbadc06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:36.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.157+0000 7fbae2700700 1 -- 172.21.15.42:0/3001460171 shutdown_connections 2024-02-15T18:33:36.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.157+0000 7fbae2700700 1 -- 172.21.15.42:0/3001460171 wait complete. 2024-02-15T18:33:36.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.260+0000 7f4642fb0700 1 -- 172.21.15.42:0/2337942111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c0f6860 msgr2=0x7f463c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:36.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.260+0000 7f4642fb0700 1 --2- 172.21.15.42:0/2337942111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c0f6860 0x7f463c0f6cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f462c004660 tx=0x7f462c009b30).stop 2024-02-15T18:33:36.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.260+0000 7f4642fb0700 1 -- 172.21.15.42:0/2337942111 shutdown_connections 2024-02-15T18:33:36.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.260+0000 7f4642fb0700 1 --2- 172.21.15.42:0/2337942111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c0f6860 0x7f463c0f6cc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:36.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.260+0000 7f4642fb0700 1 --2- 172.21.15.42:0/2337942111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f463c0f55c0 0x7f463c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:36.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.260+0000 7f4642fb0700 1 -- 172.21.15.42:0/2337942111 >> 172.21.15.42:0/2337942111 conn(0x7f463c0f0d40 msgr2=0x7f463c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:36.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.260+0000 7f4642fb0700 1 -- 172.21.15.42:0/2337942111 shutdown_connections 2024-02-15T18:33:36.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.260+0000 7f4642fb0700 1 -- 172.21.15.42:0/2337942111 wait complete. 2024-02-15T18:33:36.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.261+0000 7f4642fb0700 1 Processor -- start 2024-02-15T18:33:36.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.261+0000 7f4642fb0700 1 -- start start 2024-02-15T18:33:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.261+0000 7f4642fb0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c0f55c0 0x7f463c0f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.261+0000 7f4642fb0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f463c0f6860 0x7f463c0f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.261+0000 7f4642fb0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0f8920 con 0x7f463c0f55c0 2024-02-15T18:33:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.261+0000 7f4642fb0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c0f8a60 con 0x7f463c0f6860 2024-02-15T18:33:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.261+0000 7f4640d4c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c0f55c0 0x7f463c0f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.261+0000 7f4640d4c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c0f55c0 0x7f463c0f9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44294/0 (socket says 172.21.15.42:44294) 2024-02-15T18:33:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.261+0000 7f4640d4c700 1 -- 172.21.15.42:0/1168833468 learned_addr learned my addr 172.21.15.42:0/1168833468 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:33:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.261+0000 7f4640d4c700 1 -- 172.21.15.42:0/1168833468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f463c0f6860 msgr2=0x7f463c0f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.261+0000 7f4640d4c700 1 --2- 172.21.15.42:0/1168833468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f463c0f6860 0x7f463c0f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:33:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.261+0000 7f4640d4c700 1 -- 172.21.15.42:0/1168833468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c005040 con 0x7f463c0f55c0 2024-02-15T18:33:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.262+0000 7f4640d4c700 1 --2- 172.21.15.42:0/1168833468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c0f55c0 0x7f463c0f9f40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f4638002700 tx=0x7f4638009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.262+0000 7f46317fa700 1 -- 172.21.15.42:0/1168833468 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4638009320 con 0x7f463c0f55c0 2024-02-15T18:33:36.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.262+0000 7f4642fb0700 1 -- 172.21.15.42:0/1168833468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463c0f8d40 con 0x7f463c0f55c0 2024-02-15T18:33:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.262+0000 7f46317fa700 1 -- 172.21.15.42:0/1168833468 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f463800f040 con 0x7f463c0f55c0 2024-02-15T18:33:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.262+0000 7f46317fa700 1 -- 172.21.15.42:0/1168833468 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46380149d0 con 0x7f463c0f55c0 2024-02-15T18:33:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.262+0000 7f4642fb0700 1 -- 172.21.15.42:0/1168833468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463c10f000 con 0x7f463c0f55c0 2024-02-15T18:33:36.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.263+0000 7f4642fb0700 1 -- 172.21.15.42:0/1168833468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463c045ad0 con 0x7f463c0f55c0 2024-02-15T18:33:36.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.263+0000 7f46317fa700 1 -- 172.21.15.42:0/1168833468 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f46380077b0 con 0x7f463c0f55c0 2024-02-15T18:33:36.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.264+0000 7f46317fa700 1 --2- 172.21.15.42:0/1168833468 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4624060f20 0x7f46240633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:36.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.264+0000 7f46317fa700 1 -- 172.21.15.42:0/1168833468 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f463807c4e0 con 0x7f463c0f55c0 2024-02-15T18:33:36.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.264+0000 7f4633fff700 1 --2- 172.21.15.42:0/1168833468 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4624060f20 0x7f46240633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:36.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.264+0000 7f4633fff700 1 --2- 172.21.15.42:0/1168833468 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4624060f20 0x7f46240633e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f462c009f50 tx=0x7f462c009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:36.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.268+0000 7f46317fa700 1 -- 172.21.15.42:0/1168833468 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f463804c7f0 con 0x7f463c0f55c0 2024-02-15T18:33:36.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:35.100429+0000 mgr.smithi042.kfoihb (mgr.24435) 381 : audit [DBG] from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:36.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:35.417523+0000 mgr.smithi042.kfoihb (mgr.24435) 382 : audit [DBG] from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:36.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:36 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:35.757991+0000 mgr.smithi042.kfoihb (mgr.24435) 383 : audit [DBG] from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:36.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:35.100429+0000 mgr.smithi042.kfoihb (mgr.24435) 381 : audit [DBG] from='client.34420 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:36.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:35.417523+0000 mgr.smithi042.kfoihb (mgr.24435) 382 : audit [DBG] from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:36.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:36 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:35.757991+0000 mgr.smithi042.kfoihb (mgr.24435) 383 : audit [DBG] from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:36.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.532+0000 7f4642fb0700 1 -- 172.21.15.42:0/1168833468 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f463c10f770 con 0x7f463c0f55c0 2024-02-15T18:33:36.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.532+0000 7f46317fa700 1 -- 172.21.15.42:0/1168833468 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1774 (secure 0 0 0) 0x7f463804fe10 con 0x7f463c0f55c0 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:e28 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 28 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:32:55.244270+0000 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:33:36.525 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:33:36.526 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:33:36.526 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:33:36.526 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 113 2024-02-15T18:33:36.526 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:33:36.526 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:33:36.526 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:33:36.526 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=34356} 2024-02-15T18:33:36.526 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:33:36.526 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:33:36.526 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:33:36.526 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:33:36.526 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:33:36.526 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:33:36.527 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:33:36.527 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:33:36.527 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{0:34356} state up:active seq 12 join_fscid=1 addr [v2:172.21.15.42:6826/2962563293,v1:172.21.15.42:6827/2962563293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:33:36.527 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{0:34360} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/1562993914,v1:172.21.15.42:6829/1562993914] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:33:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:33:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:33:36.527 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:33:36.527 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:33:36.527 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.191:6824/540280968,v1:172.21.15.191:6825/540280968] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:33:36.527 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.191:6826/289198038,v1:172.21.15.191:6827/289198038] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:33:36.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.534+0000 7f4642fb0700 1 -- 172.21.15.42:0/1168833468 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4624060f20 msgr2=0x7f46240633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:36.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.534+0000 7f4642fb0700 1 --2- 172.21.15.42:0/1168833468 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4624060f20 0x7f46240633e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f462c009f50 tx=0x7f462c009b30).stop 2024-02-15T18:33:36.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.534+0000 7f4642fb0700 1 -- 172.21.15.42:0/1168833468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c0f55c0 msgr2=0x7f463c0f9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:36.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.534+0000 7f4642fb0700 1 --2- 172.21.15.42:0/1168833468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c0f55c0 0x7f463c0f9f40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f4638002700 tx=0x7f4638009d80).stop 2024-02-15T18:33:36.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.534+0000 7f4642fb0700 1 -- 172.21.15.42:0/1168833468 shutdown_connections 2024-02-15T18:33:36.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.534+0000 7f4642fb0700 1 --2- 172.21.15.42:0/1168833468 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4624060f20 0x7f46240633e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:36.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.534+0000 7f4642fb0700 1 --2- 172.21.15.42:0/1168833468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f463c0f55c0 0x7f463c0f9f40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:36.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.534+0000 7f4642fb0700 1 --2- 172.21.15.42:0/1168833468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f463c0f6860 0x7f463c0f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:33:36.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.534+0000 7f4642fb0700 1 -- 172.21.15.42:0/1168833468 >> 172.21.15.42:0/1168833468 conn(0x7f463c0f0d40 msgr2=0x7f463c05f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:36.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.534+0000 7f4642fb0700 1 -- 172.21.15.42:0/1168833468 shutdown_connections 2024-02-15T18:33:36.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.534+0000 7f4642fb0700 1 -- 172.21.15.42:0/1168833468 wait complete. 2024-02-15T18:33:36.529 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 28 2024-02-15T18:33:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.639+0000 7f68256fc700 1 -- 172.21.15.42:0/2882294306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68200f55e0 msgr2=0x7f68200f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.639+0000 7f68256fc700 1 --2- 172.21.15.42:0/2882294306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68200f55e0 0x7f68200f5a00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f6814004660 tx=0x7f6814009b30).stop 2024-02-15T18:33:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.639+0000 7f68256fc700 1 -- 172.21.15.42:0/2882294306 shutdown_connections 2024-02-15T18:33:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.639+0000 7f68256fc700 1 --2- 172.21.15.42:0/2882294306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68200f5ef0 0x7f68200f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.639+0000 7f68256fc700 1 --2- 172.21.15.42:0/2882294306 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68200f55e0 0x7f68200f5a00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:36.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.639+0000 7f68256fc700 1 -- 172.21.15.42:0/2882294306 >> 172.21.15.42:0/2882294306 conn(0x7f68200f0de0 msgr2=0x7f68200f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:36.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.639+0000 7f68256fc700 1 -- 172.21.15.42:0/2882294306 shutdown_connections 2024-02-15T18:33:36.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.639+0000 7f68256fc700 1 -- 172.21.15.42:0/2882294306 wait complete. 2024-02-15T18:33:36.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.640+0000 7f68256fc700 1 Processor -- start 2024-02-15T18:33:36.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.640+0000 7f68256fc700 1 -- start start 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.640+0000 7f68256fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68200f55e0 0x7f6820109200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.640+0000 7f68256fc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68200f5ef0 0x7f68201096f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.640+0000 7f68256fc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6820109ca0 con 0x7f68200f55e0 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.640+0000 7f68256fc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6820109de0 con 0x7f68200f5ef0 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.640+0000 7f681effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68200f55e0 0x7f6820109200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.640+0000 7f681effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68200f55e0 0x7f6820109200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44304/0 (socket says 172.21.15.42:44304) 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.640+0000 7f681effd700 1 -- 172.21.15.42:0/2603625850 learned_addr learned my addr 172.21.15.42:0/2603625850 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.640+0000 7f681e7fc700 1 --2- 172.21.15.42:0/2603625850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68200f5ef0 0x7f68201096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.640+0000 7f681effd700 1 -- 172.21.15.42:0/2603625850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68200f5ef0 msgr2=0x7f68201096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.640+0000 7f681effd700 1 --2- 172.21.15.42:0/2603625850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68200f5ef0 0x7f68201096f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.640+0000 7f681effd700 1 -- 172.21.15.42:0/2603625850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6814005040 con 0x7f68200f55e0 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.640+0000 7f681effd700 1 --2- 172.21.15.42:0/2603625850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68200f55e0 0x7f6820109200 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f6814009f50 tx=0x7f6814009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.641+0000 7f68077fe700 1 -- 172.21.15.42:0/2603625850 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6814018070 con 0x7f68200f55e0 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.641+0000 7f68077fe700 1 -- 172.21.15.42:0/2603625850 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f6814007ba0 con 0x7f68200f55e0 2024-02-15T18:33:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.641+0000 7f68256fc700 1 -- 172.21.15.42:0/2603625850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f682010a720 con 0x7f68200f55e0 2024-02-15T18:33:36.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.641+0000 7f68077fe700 1 -- 172.21.15.42:0/2603625850 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f681401c420 con 0x7f68200f55e0 2024-02-15T18:33:36.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.641+0000 7f68256fc700 1 -- 172.21.15.42:0/2603625850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f682010ac80 con 0x7f68200f55e0 2024-02-15T18:33:36.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.642+0000 7f68256fc700 1 -- 172.21.15.42:0/2603625850 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6820045ad0 con 0x7f68200f55e0 2024-02-15T18:33:36.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.642+0000 7f68077fe700 1 -- 172.21.15.42:0/2603625850 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f681401d420 con 0x7f68200f55e0 2024-02-15T18:33:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.642+0000 7f68077fe700 1 --2- 172.21.15.42:0/2603625850 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6808060f20 0x7f68080633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:33:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.642+0000 7f68077fe700 1 -- 172.21.15.42:0/2603625850 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f681407de40 con 0x7f68200f55e0 2024-02-15T18:33:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.643+0000 7f681e7fc700 1 --2- 172.21.15.42:0/2603625850 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6808060f20 0x7f68080633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:33:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.643+0000 7f681e7fc700 1 --2- 172.21.15.42:0/2603625850 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6808060f20 0x7f68080633e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f6810004c40 tx=0x7f681000b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:33:36.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.647+0000 7f68077fe700 1 -- 172.21.15.42:0/2603625850 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6814051ba0 con 0x7f68200f55e0 2024-02-15T18:33:36.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.864+0000 7f68256fc700 1 -- 172.21.15.42:0/2603625850 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68200fb380 con 0x7f6808060f20 2024-02-15T18:33:36.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.865+0000 7f68077fe700 1 -- 172.21.15.42:0/2603625850 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+440 (secure 0 0 0) 0x7f68200fb380 con 0x7f6808060f20 2024-02-15T18:33:36.857 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:33:36.857 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:33:36.858 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:33:36.858 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:33:36.858 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:33:36.858 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-02-15T18:33:36.858 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-02-15T18:33:36.858 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-02-15T18:33:36.858 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr", 2024-02-15T18:33:36.858 INFO:teuthology.orchestra.run.smithi042.stdout: "crash" 2024-02-15T18:33:36.858 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:33:36.858 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/21 daemons upgraded", 2024-02-15T18:33:36.858 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading alertmanager daemons", 2024-02-15T18:33:36.858 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:33:36.858 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:33:36.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.867+0000 7f68256fc700 1 -- 172.21.15.42:0/2603625850 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6808060f20 msgr2=0x7f68080633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:36.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.867+0000 7f68256fc700 1 --2- 172.21.15.42:0/2603625850 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6808060f20 0x7f68080633e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f6810004c40 tx=0x7f681000b040).stop 2024-02-15T18:33:36.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.867+0000 7f68256fc700 1 -- 172.21.15.42:0/2603625850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68200f55e0 msgr2=0x7f6820109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:33:36.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.867+0000 7f68256fc700 1 --2- 172.21.15.42:0/2603625850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68200f55e0 0x7f6820109200 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f6814009f50 tx=0x7f6814009b30).stop 2024-02-15T18:33:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.867+0000 7f68256fc700 1 -- 172.21.15.42:0/2603625850 shutdown_connections 2024-02-15T18:33:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.867+0000 7f68256fc700 1 --2- 172.21.15.42:0/2603625850 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6808060f20 0x7f68080633e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.867+0000 7f68256fc700 1 --2- 172.21.15.42:0/2603625850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68200f55e0 0x7f6820109200 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.867+0000 7f68256fc700 1 --2- 172.21.15.42:0/2603625850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68200f5ef0 0x7f68201096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:33:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.867+0000 7f68256fc700 1 -- 172.21.15.42:0/2603625850 >> 172.21.15.42:0/2603625850 conn(0x7f68200f0de0 msgr2=0x7f682005f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:33:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.867+0000 7f68256fc700 1 -- 172.21.15.42:0/2603625850 shutdown_connections 2024-02-15T18:33:36.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:33:36.867+0000 7f68256fc700 1 -- 172.21.15.42:0/2603625850 wait complete. 2024-02-15T18:33:37.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:36.155845+0000 mon.smithi042 (mon.0) 675 : audit [DBG] from='client.? 172.21.15.42:0/3001460171' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:37.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:36.533635+0000 mon.smithi042 (mon.0) 676 : audit [DBG] from='client.? 172.21.15.42:0/1168833468' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:33:37.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:36.779188+0000 mgr.smithi042.kfoihb (mgr.24435) 384 : cluster [DBG] pgmap v293: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:37.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:36.866348+0000 mgr.smithi042.kfoihb (mgr.24435) 385 : audit [DBG] from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:37.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:36.155845+0000 mon.smithi042 (mon.0) 675 : audit [DBG] from='client.? 172.21.15.42:0/3001460171' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:37.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:36.533635+0000 mon.smithi042 (mon.0) 676 : audit [DBG] from='client.? 172.21.15.42:0/1168833468' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:33:37.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:36.779188+0000 mgr.smithi042.kfoihb (mgr.24435) 384 : cluster [DBG] pgmap v293: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:37.442 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:36.866348+0000 mgr.smithi042.kfoihb (mgr.24435) 385 : audit [DBG] from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:33:39.965 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:38.779658+0000 mgr.smithi042.kfoihb (mgr.24435) 386 : cluster [DBG] pgmap v294: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-02-15T18:33:40.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:38.779658+0000 mgr.smithi042.kfoihb (mgr.24435) 386 : cluster [DBG] pgmap v294: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-02-15T18:33:42.158 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:40.780160+0000 mgr.smithi042.kfoihb (mgr.24435) 387 : cluster [DBG] pgmap v295: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:42.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:40.780160+0000 mgr.smithi042.kfoihb (mgr.24435) 387 : cluster [DBG] pgmap v295: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:44.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:42.780602+0000 mgr.smithi042.kfoihb (mgr.24435) 388 : cluster [DBG] pgmap v296: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-02-15T18:33:44.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:43.185712+0000 mon.smithi042 (mon.0) 677 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:44.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:43.186444+0000 mon.smithi042 (mon.0) 678 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:44.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:42.780602+0000 mgr.smithi042.kfoihb (mgr.24435) 388 : cluster [DBG] pgmap v296: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-02-15T18:33:44.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:43.185712+0000 mon.smithi042 (mon.0) 677 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:44.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:43.186444+0000 mon.smithi042 (mon.0) 678 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:46.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:44.780906+0000 mgr.smithi042.kfoihb (mgr.24435) 389 : cluster [DBG] pgmap v297: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-02-15T18:33:46.204 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:44.780906+0000 mgr.smithi042.kfoihb (mgr.24435) 389 : cluster [DBG] pgmap v297: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-02-15T18:33:47.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:46.781619+0000 mgr.smithi042.kfoihb (mgr.24435) 390 : cluster [DBG] pgmap v298: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:47.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:46.781619+0000 mgr.smithi042.kfoihb (mgr.24435) 390 : cluster [DBG] pgmap v298: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:48.805 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.660063+0000 mon.smithi042 (mon.0) 679 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:48.805 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.660869+0000 mon.smithi042 (mon.0) 680 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:33:48.805 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.661504+0000 mon.smithi042 (mon.0) 681 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:33:48.805 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:33:47.662084+0000 mgr.smithi042.kfoihb (mgr.24435) 391 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:33:48.805 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.665209+0000 mon.smithi042 (mon.0) 682 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:48.805 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.672177+0000 mon.smithi042 (mon.0) 683 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:33:48.805 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.672497+0000 mgr.smithi042.kfoihb (mgr.24435) 392 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.673292+0000 mon.smithi042 (mon.0) 684 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi042.front.sepia.ceph.com:9093"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.673535+0000 mgr.smithi042.kfoihb (mgr.24435) 393 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi042.front.sepia.ceph.com:9093"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.677008+0000 mon.smithi042 (mon.0) 685 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.679041+0000 mon.smithi042 (mon.0) 686 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.682776+0000 mon.smithi042 (mon.0) 687 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.684794+0000 mon.smithi042 (mon.0) 688 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.686218+0000 mon.smithi042 (mon.0) 689 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.687975+0000 mon.smithi042 (mon.0) 690 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.689613+0000 mon.smithi042 (mon.0) 691 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.690471+0000 mon.smithi042 (mon.0) 692 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.691376+0000 mon.smithi042 (mon.0) 693 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.692308+0000 mon.smithi042 (mon.0) 694 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.693549+0000 mon.smithi042 (mon.0) 695 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.694677+0000 mon.smithi042 (mon.0) 696 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.696186+0000 mon.smithi042 (mon.0) 697 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.806 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.697302+0000 mon.smithi042 (mon.0) 698 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.807 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:48 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:47.698395+0000 mon.smithi042 (mon.0) 699 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.660063+0000 mon.smithi042 (mon.0) 679 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:48.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.660869+0000 mon.smithi042 (mon.0) 680 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:33:48.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.661504+0000 mon.smithi042 (mon.0) 681 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:33:48.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:33:47.662084+0000 mgr.smithi042.kfoihb (mgr.24435) 391 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:33:48.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.665209+0000 mon.smithi042 (mon.0) 682 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:48.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.672177+0000 mon.smithi042 (mon.0) 683 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:33:48.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.672497+0000 mgr.smithi042.kfoihb (mgr.24435) 392 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-15T18:33:48.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.673292+0000 mon.smithi042 (mon.0) 684 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi042.front.sepia.ceph.com:9093"}]: dispatch 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.673535+0000 mgr.smithi042.kfoihb (mgr.24435) 393 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi042.front.sepia.ceph.com:9093"}]: dispatch 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.677008+0000 mon.smithi042 (mon.0) 685 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.679041+0000 mon.smithi042 (mon.0) 686 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.682776+0000 mon.smithi042 (mon.0) 687 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.684794+0000 mon.smithi042 (mon.0) 688 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.686218+0000 mon.smithi042 (mon.0) 689 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.687975+0000 mon.smithi042 (mon.0) 690 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.689613+0000 mon.smithi042 (mon.0) 691 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.690471+0000 mon.smithi042 (mon.0) 692 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.691376+0000 mon.smithi042 (mon.0) 693 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.692308+0000 mon.smithi042 (mon.0) 694 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.693549+0000 mon.smithi042 (mon.0) 695 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.694677+0000 mon.smithi042 (mon.0) 696 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.696186+0000 mon.smithi042 (mon.0) 697 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.697302+0000 mon.smithi042 (mon.0) 698 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:48.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:47.698395+0000 mon.smithi042 (mon.0) 699 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:33:49.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:48.782027+0000 mgr.smithi042.kfoihb (mgr.24435) 394 : cluster [DBG] pgmap v299: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:49.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:49.326805+0000 mon.smithi042 (mon.0) 700 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:49.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:49.329227+0000 mon.smithi042 (mon.0) 701 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:49.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:49.549283+0000 mon.smithi042 (mon.0) 702 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-15T18:33:50.054 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:48.782027+0000 mgr.smithi042.kfoihb (mgr.24435) 394 : cluster [DBG] pgmap v299: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:50.054 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:49.326805+0000 mon.smithi042 (mon.0) 700 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:50.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:49.329227+0000 mon.smithi042 (mon.0) 701 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:33:50.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:49.549283+0000 mon.smithi042 (mon.0) 702 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-15T18:33:50.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:33:49.177075+0000 mgr.smithi042.kfoihb (mgr.24435) 395 : cephadm [INF] Upgrade: Updating grafana.smithi042 2024-02-15T18:33:50.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:33:49.549687+0000 mgr.smithi042.kfoihb (mgr.24435) 396 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-15T18:33:50.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:33:49.553836+0000 mgr.smithi042.kfoihb (mgr.24435) 397 : cephadm [INF] Deploying daemon grafana.smithi042 on smithi042 2024-02-15T18:33:51.054 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:33:49.177075+0000 mgr.smithi042.kfoihb (mgr.24435) 395 : cephadm [INF] Upgrade: Updating grafana.smithi042 2024-02-15T18:33:51.054 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:33:49.549687+0000 mgr.smithi042.kfoihb (mgr.24435) 396 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-15T18:33:51.054 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:50 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:33:49.553836+0000 mgr.smithi042.kfoihb (mgr.24435) 397 : cephadm [INF] Deploying daemon grafana.smithi042 on smithi042 2024-02-15T18:33:51.941 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:50.782435+0000 mgr.smithi042.kfoihb (mgr.24435) 398 : cluster [DBG] pgmap v300: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:52.054 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:50.782435+0000 mgr.smithi042.kfoihb (mgr.24435) 398 : cluster [DBG] pgmap v300: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:54.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:52.782814+0000 mgr.smithi042.kfoihb (mgr.24435) 399 : cluster [DBG] pgmap v301: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:54.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:52.782814+0000 mgr.smithi042.kfoihb (mgr.24435) 399 : cluster [DBG] pgmap v301: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:56.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:54.783104+0000 mgr.smithi042.kfoihb (mgr.24435) 400 : cluster [DBG] pgmap v302: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:56.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:54.783104+0000 mgr.smithi042.kfoihb (mgr.24435) 400 : cluster [DBG] pgmap v302: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:33:57.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:56.783488+0000 mgr.smithi042.kfoihb (mgr.24435) 401 : cluster [DBG] pgmap v303: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:33:57.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:56.783488+0000 mgr.smithi042.kfoihb (mgr.24435) 401 : cluster [DBG] pgmap v303: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:00.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:33:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:33:58.783839+0000 mgr.smithi042.kfoihb (mgr.24435) 402 : cluster [DBG] pgmap v304: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:00.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:33:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:33:58.783839+0000 mgr.smithi042.kfoihb (mgr.24435) 402 : cluster [DBG] pgmap v304: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:01.952 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:00.784327+0000 mgr.smithi042.kfoihb (mgr.24435) 403 : cluster [DBG] pgmap v305: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:02.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:00.784327+0000 mgr.smithi042.kfoihb (mgr.24435) 403 : cluster [DBG] pgmap v305: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:04.037 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:02.784860+0000 mgr.smithi042.kfoihb (mgr.24435) 404 : cluster [DBG] pgmap v306: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:04.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:02.784860+0000 mgr.smithi042.kfoihb (mgr.24435) 404 : cluster [DBG] pgmap v306: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:06.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:04.785171+0000 mgr.smithi042.kfoihb (mgr.24435) 405 : cluster [DBG] pgmap v307: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:06.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:05.043804+0000 mon.smithi042 (mon.0) 703 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:06.192 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:05.044575+0000 mon.smithi042 (mon.0) 704 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:34:06.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:04.785171+0000 mgr.smithi042.kfoihb (mgr.24435) 405 : cluster [DBG] pgmap v307: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:06.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:05.043804+0000 mon.smithi042 (mon.0) 703 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:06.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:05.044575+0000 mon.smithi042 (mon.0) 704 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:34:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.972+0000 7f0301105700 1 -- 172.21.15.42:0/3767200665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02fc0f55c0 msgr2=0x7f02fc0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.972+0000 7f0301105700 1 --2- 172.21.15.42:0/3767200665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02fc0f55c0 0x7f02fc0f59e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f02f4004660 tx=0x7f02f4009b30).stop 2024-02-15T18:34:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.973+0000 7f0301105700 1 -- 172.21.15.42:0/3767200665 shutdown_connections 2024-02-15T18:34:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.973+0000 7f0301105700 1 --2- 172.21.15.42:0/3767200665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc0f6860 0x7f02fc0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:34:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.973+0000 7f0301105700 1 --2- 172.21.15.42:0/3767200665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02fc0f55c0 0x7f02fc0f59e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.973+0000 7f0301105700 1 -- 172.21.15.42:0/3767200665 >> 172.21.15.42:0/3767200665 conn(0x7f02fc0f0d40 msgr2=0x7f02fc0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.973+0000 7f0301105700 1 -- 172.21.15.42:0/3767200665 shutdown_connections 2024-02-15T18:34:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.973+0000 7f0301105700 1 -- 172.21.15.42:0/3767200665 wait complete. 2024-02-15T18:34:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.973+0000 7f0301105700 1 Processor -- start 2024-02-15T18:34:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.973+0000 7f0301105700 1 -- start start 2024-02-15T18:34:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f0301105700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02fc0f55c0 0x7f02fc109270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f0301105700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc0f6860 0x7f02fc109760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f0301105700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc109da0 con 0x7f02fc0f55c0 2024-02-15T18:34:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f0301105700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc10a650 con 0x7f02fc0f6860 2024-02-15T18:34:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f02fad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02fc0f55c0 0x7f02fc109270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f02fad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02fc0f55c0 0x7f02fc109270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55500/0 (socket says 172.21.15.42:55500) 2024-02-15T18:34:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f02fad9d700 1 -- 172.21.15.42:0/4163992043 learned_addr learned my addr 172.21.15.42:0/4163992043 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:34:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f02fa59c700 1 --2- 172.21.15.42:0/4163992043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc0f6860 0x7f02fc109760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f02fad9d700 1 -- 172.21.15.42:0/4163992043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc0f6860 msgr2=0x7f02fc109760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f02fad9d700 1 --2- 172.21.15.42:0/4163992043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc0f6860 0x7f02fc109760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f02fad9d700 1 -- 172.21.15.42:0/4163992043 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f4005040 con 0x7f02fc0f55c0 2024-02-15T18:34:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f02fad9d700 1 --2- 172.21.15.42:0/4163992043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02fc0f55c0 0x7f02fc109270 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f02f4004620 tx=0x7f02f4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f02f37fe700 1 -- 172.21.15.42:0/4163992043 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02f4018070 con 0x7f02fc0f55c0 2024-02-15T18:34:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f02f37fe700 1 -- 172.21.15.42:0/4163992043 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f02f4007df0 con 0x7f02fc0f55c0 2024-02-15T18:34:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f0301105700 1 -- 172.21.15.42:0/4163992043 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02fc10a8d0 con 0x7f02fc0f55c0 2024-02-15T18:34:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.974+0000 7f02f37fe700 1 -- 172.21.15.42:0/4163992043 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02f401c7b0 con 0x7f02fc0f55c0 2024-02-15T18:34:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.975+0000 7f0301105700 1 -- 172.21.15.42:0/4163992043 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02fc10adb0 con 0x7f02fc0f55c0 2024-02-15T18:34:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.976+0000 7f0301105700 1 -- 172.21.15.42:0/4163992043 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02fc0f49b0 con 0x7f02fc0f55c0 2024-02-15T18:34:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.976+0000 7f02f37fe700 1 -- 172.21.15.42:0/4163992043 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f02f401c910 con 0x7f02fc0f55c0 2024-02-15T18:34:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.976+0000 7f02f37fe700 1 --2- 172.21.15.42:0/4163992043 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f02e8060f20 0x7f02e80633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.977+0000 7f02f37fe700 1 -- 172.21.15.42:0/4163992043 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f02f407e180 con 0x7f02fc0f55c0 2024-02-15T18:34:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.977+0000 7f02fa59c700 1 --2- 172.21.15.42:0/4163992043 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f02e8060f20 0x7f02e80633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.977+0000 7f02fa59c700 1 --2- 172.21.15.42:0/4163992043 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f02e8060f20 0x7f02e80633e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f02e400b790 tx=0x7f02e4008040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:06.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:06.982+0000 7f02f37fe700 1 -- 172.21.15.42:0/4163992043 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f02f4051ee0 con 0x7f02fc0f55c0 2024-02-15T18:34:07.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.200+0000 7f0301105700 1 -- 172.21.15.42:0/4163992043 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f02fc0faaa0 con 0x7f02e8060f20 2024-02-15T18:34:07.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.201+0000 7f02f37fe700 1 -- 172.21.15.42:0/4163992043 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f02fc0faaa0 con 0x7f02e8060f20 2024-02-15T18:34:07.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.203+0000 7f0301105700 1 -- 172.21.15.42:0/4163992043 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f02e8060f20 msgr2=0x7f02e80633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:07.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.203+0000 7f0301105700 1 --2- 172.21.15.42:0/4163992043 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f02e8060f20 0x7f02e80633e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f02e400b790 tx=0x7f02e4008040).stop 2024-02-15T18:34:07.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.203+0000 7f0301105700 1 -- 172.21.15.42:0/4163992043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02fc0f55c0 msgr2=0x7f02fc109270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:07.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.203+0000 7f0301105700 1 --2- 172.21.15.42:0/4163992043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02fc0f55c0 0x7f02fc109270 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f02f4004620 tx=0x7f02f4009b30).stop 2024-02-15T18:34:07.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.203+0000 7f0301105700 1 -- 172.21.15.42:0/4163992043 shutdown_connections 2024-02-15T18:34:07.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.203+0000 7f0301105700 1 --2- 172.21.15.42:0/4163992043 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f02e8060f20 0x7f02e80633e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.203+0000 7f0301105700 1 --2- 172.21.15.42:0/4163992043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02fc0f55c0 0x7f02fc109270 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.203+0000 7f0301105700 1 --2- 172.21.15.42:0/4163992043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc0f6860 0x7f02fc109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.203+0000 7f0301105700 1 -- 172.21.15.42:0/4163992043 >> 172.21.15.42:0/4163992043 conn(0x7f02fc0f0d40 msgr2=0x7f02fc05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:07.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.203+0000 7f0301105700 1 -- 172.21.15.42:0/4163992043 shutdown_connections 2024-02-15T18:34:07.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.203+0000 7f0301105700 1 -- 172.21.15.42:0/4163992043 wait complete. 2024-02-15T18:34:07.205 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:34:07.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.306+0000 7fcf60be9700 1 -- 172.21.15.42:0/3073316721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf5c0f6860 msgr2=0x7fcf5c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.306+0000 7fcf60be9700 1 --2- 172.21.15.42:0/3073316721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf5c0f6860 0x7fcf5c0f6cc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fcf44004660 tx=0x7fcf44009b30).stop 2024-02-15T18:34:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.306+0000 7fcf60be9700 1 -- 172.21.15.42:0/3073316721 shutdown_connections 2024-02-15T18:34:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.306+0000 7fcf60be9700 1 --2- 172.21.15.42:0/3073316721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf5c0f6860 0x7fcf5c0f6cc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.306+0000 7fcf60be9700 1 --2- 172.21.15.42:0/3073316721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf5c0f55c0 0x7fcf5c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.306+0000 7fcf60be9700 1 -- 172.21.15.42:0/3073316721 >> 172.21.15.42:0/3073316721 conn(0x7fcf5c0f0d40 msgr2=0x7fcf5c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.306+0000 7fcf60be9700 1 -- 172.21.15.42:0/3073316721 shutdown_connections 2024-02-15T18:34:07.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.306+0000 7fcf60be9700 1 -- 172.21.15.42:0/3073316721 wait complete. 2024-02-15T18:34:07.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf60be9700 1 Processor -- start 2024-02-15T18:34:07.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf60be9700 1 -- start start 2024-02-15T18:34:07.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf60be9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf5c0f55c0 0x7fcf5c106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf60be9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf5c0f6860 0x7fcf5c1074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf60be9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c107a60 con 0x7fcf5c0f55c0 2024-02-15T18:34:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf60be9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c107ba0 con 0x7fcf5c0f6860 2024-02-15T18:34:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf5a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf5c0f55c0 0x7fcf5c106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf5a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf5c0f55c0 0x7fcf5c106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55508/0 (socket says 172.21.15.42:55508) 2024-02-15T18:34:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf5a59c700 1 -- 172.21.15.42:0/2025491576 learned_addr learned my addr 172.21.15.42:0/2025491576 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:34:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf59d9b700 1 --2- 172.21.15.42:0/2025491576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf5c0f6860 0x7fcf5c1074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf5a59c700 1 -- 172.21.15.42:0/2025491576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf5c0f6860 msgr2=0x7fcf5c1074b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf5a59c700 1 --2- 172.21.15.42:0/2025491576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf5c0f6860 0x7fcf5c1074b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf5a59c700 1 -- 172.21.15.42:0/2025491576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf44005040 con 0x7fcf5c0f55c0 2024-02-15T18:34:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf59d9b700 1 --2- 172.21.15.42:0/2025491576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf5c0f6860 0x7fcf5c1074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:34:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.307+0000 7fcf5a59c700 1 --2- 172.21.15.42:0/2025491576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf5c0f55c0 0x7fcf5c106fc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fcf4c00d8d0 tx=0x7fcf4c004f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:07.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.308+0000 7fcf52ffd700 1 -- 172.21.15.42:0/2025491576 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf4c00de70 con 0x7fcf5c0f55c0 2024-02-15T18:34:07.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.308+0000 7fcf52ffd700 1 -- 172.21.15.42:0/2025491576 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fcf4c00f040 con 0x7fcf5c0f55c0 2024-02-15T18:34:07.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.308+0000 7fcf60be9700 1 -- 172.21.15.42:0/2025491576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5c108540 con 0x7fcf5c0f55c0 2024-02-15T18:34:07.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.308+0000 7fcf52ffd700 1 -- 172.21.15.42:0/2025491576 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf4c013be0 con 0x7fcf5c0f55c0 2024-02-15T18:34:07.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.308+0000 7fcf60be9700 1 -- 172.21.15.42:0/2025491576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c108a80 con 0x7fcf5c0f55c0 2024-02-15T18:34:07.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.308+0000 7fcf60be9700 1 -- 172.21.15.42:0/2025491576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5c045ad0 con 0x7fcf5c0f55c0 2024-02-15T18:34:07.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.309+0000 7fcf52ffd700 1 -- 172.21.15.42:0/2025491576 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fcf4c01a030 con 0x7fcf5c0f55c0 2024-02-15T18:34:07.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.310+0000 7fcf52ffd700 1 --2- 172.21.15.42:0/2025491576 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf48060f20 0x7fcf480633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:07.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.310+0000 7fcf52ffd700 1 -- 172.21.15.42:0/2025491576 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fcf4c07c8a0 con 0x7fcf5c0f55c0 2024-02-15T18:34:07.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.310+0000 7fcf59d9b700 1 --2- 172.21.15.42:0/2025491576 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf48060f20 0x7fcf480633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:07.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.310+0000 7fcf59d9b700 1 --2- 172.21.15.42:0/2025491576 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf48060f20 0x7fcf480633e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fcf44005010 tx=0x7fcf44009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:07.369 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:06.785549+0000 mgr.smithi042.kfoihb (mgr.24435) 406 : cluster [DBG] pgmap v308: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:07.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.313+0000 7fcf52ffd700 1 -- 172.21.15.42:0/2025491576 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcf4c050600 con 0x7fcf5c0f55c0 2024-02-15T18:34:07.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:06.785549+0000 mgr.smithi042.kfoihb (mgr.24435) 406 : cluster [DBG] pgmap v308: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:07.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.515+0000 7fcf60be9700 1 -- 172.21.15.42:0/2025491576 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf5c0faca0 con 0x7fcf48060f20 2024-02-15T18:34:07.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.516+0000 7fcf52ffd700 1 -- 172.21.15.42:0/2025491576 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7fcf5c0faca0 con 0x7fcf48060f20 2024-02-15T18:34:07.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.518+0000 7fcf60be9700 1 -- 172.21.15.42:0/2025491576 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf48060f20 msgr2=0x7fcf480633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:07.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.518+0000 7fcf60be9700 1 --2- 172.21.15.42:0/2025491576 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf48060f20 0x7fcf480633e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fcf44005010 tx=0x7fcf44009b30).stop 2024-02-15T18:34:07.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.518+0000 7fcf60be9700 1 -- 172.21.15.42:0/2025491576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf5c0f55c0 msgr2=0x7fcf5c106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:07.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.518+0000 7fcf60be9700 1 --2- 172.21.15.42:0/2025491576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf5c0f55c0 0x7fcf5c106fc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fcf4c00d8d0 tx=0x7fcf4c004f00).stop 2024-02-15T18:34:07.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.518+0000 7fcf60be9700 1 -- 172.21.15.42:0/2025491576 shutdown_connections 2024-02-15T18:34:07.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.518+0000 7fcf60be9700 1 --2- 172.21.15.42:0/2025491576 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf48060f20 0x7fcf480633e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.518+0000 7fcf60be9700 1 --2- 172.21.15.42:0/2025491576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf5c0f55c0 0x7fcf5c106fc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.518+0000 7fcf60be9700 1 --2- 172.21.15.42:0/2025491576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf5c0f6860 0x7fcf5c1074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.518+0000 7fcf60be9700 1 -- 172.21.15.42:0/2025491576 >> 172.21.15.42:0/2025491576 conn(0x7fcf5c0f0d40 msgr2=0x7fcf5c05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:07.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.519+0000 7fcf60be9700 1 -- 172.21.15.42:0/2025491576 shutdown_connections 2024-02-15T18:34:07.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.519+0000 7fcf60be9700 1 -- 172.21.15.42:0/2025491576 wait complete. 2024-02-15T18:34:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.619+0000 7fcc24d40700 1 -- 172.21.15.42:0/3000009963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc200f77d0 msgr2=0x7fcc200f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.619+0000 7fcc24d40700 1 --2- 172.21.15.42:0/3000009963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc200f77d0 0x7fcc200f7c30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fcc14004660 tx=0x7fcc14009b30).stop 2024-02-15T18:34:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.619+0000 7fcc24d40700 1 -- 172.21.15.42:0/3000009963 shutdown_connections 2024-02-15T18:34:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.619+0000 7fcc24d40700 1 --2- 172.21.15.42:0/3000009963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc200f77d0 0x7fcc200f7c30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.619+0000 7fcc24d40700 1 --2- 172.21.15.42:0/3000009963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc200f6530 0x7fcc200f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.619+0000 7fcc24d40700 1 -- 172.21.15.42:0/3000009963 >> 172.21.15.42:0/3000009963 conn(0x7fcc200f1c90 msgr2=0x7fcc200f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:07.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.619+0000 7fcc24d40700 1 -- 172.21.15.42:0/3000009963 shutdown_connections 2024-02-15T18:34:07.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.619+0000 7fcc24d40700 1 -- 172.21.15.42:0/3000009963 wait complete. 2024-02-15T18:34:07.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.620+0000 7fcc24d40700 1 Processor -- start 2024-02-15T18:34:07.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.620+0000 7fcc24d40700 1 -- start start 2024-02-15T18:34:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.620+0000 7fcc24d40700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc200f6530 0x7fcc20106e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.620+0000 7fcc24d40700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc200f77d0 0x7fcc20107310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.620+0000 7fcc24d40700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc20107910 con 0x7fcc200f77d0 2024-02-15T18:34:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.620+0000 7fcc24d40700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc20108120 con 0x7fcc200f6530 2024-02-15T18:34:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.620+0000 7fcc1effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc200f77d0 0x7fcc20107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.620+0000 7fcc1effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc200f77d0 0x7fcc20107310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55532/0 (socket says 172.21.15.42:55532) 2024-02-15T18:34:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.620+0000 7fcc1effd700 1 -- 172.21.15.42:0/364458195 learned_addr learned my addr 172.21.15.42:0/364458195 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:34:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.621+0000 7fcc1f7fe700 1 --2- 172.21.15.42:0/364458195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc200f6530 0x7fcc20106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.621+0000 7fcc1effd700 1 -- 172.21.15.42:0/364458195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc200f6530 msgr2=0x7fcc20106e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.621+0000 7fcc1effd700 1 --2- 172.21.15.42:0/364458195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc200f6530 0x7fcc20106e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.621+0000 7fcc1effd700 1 -- 172.21.15.42:0/364458195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc14005040 con 0x7fcc200f77d0 2024-02-15T18:34:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.621+0000 7fcc1f7fe700 1 --2- 172.21.15.42:0/364458195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc200f6530 0x7fcc20106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:34:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.621+0000 7fcc1effd700 1 --2- 172.21.15.42:0/364458195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc200f77d0 0x7fcc20107310 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fcc14013fb0 tx=0x7fcc14009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:07.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.621+0000 7fcc07fff700 1 -- 172.21.15.42:0/364458195 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc14018070 con 0x7fcc200f77d0 2024-02-15T18:34:07.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.621+0000 7fcc07fff700 1 -- 172.21.15.42:0/364458195 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fcc140097d0 con 0x7fcc200f77d0 2024-02-15T18:34:07.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.621+0000 7fcc24d40700 1 -- 172.21.15.42:0/364458195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc20108340 con 0x7fcc200f77d0 2024-02-15T18:34:07.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.621+0000 7fcc07fff700 1 -- 172.21.15.42:0/364458195 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc1401c7c0 con 0x7fcc200f77d0 2024-02-15T18:34:07.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.621+0000 7fcc24d40700 1 -- 172.21.15.42:0/364458195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc201088a0 con 0x7fcc200f77d0 2024-02-15T18:34:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.622+0000 7fcc24d40700 1 -- 172.21.15.42:0/364458195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc20059100 con 0x7fcc200f77d0 2024-02-15T18:34:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.622+0000 7fcc07fff700 1 -- 172.21.15.42:0/364458195 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fcc1401c920 con 0x7fcc200f77d0 2024-02-15T18:34:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.623+0000 7fcc07fff700 1 --2- 172.21.15.42:0/364458195 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcc08060f70 0x7fcc08063430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.623+0000 7fcc07fff700 1 -- 172.21.15.42:0/364458195 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fcc1407e0e0 con 0x7fcc200f77d0 2024-02-15T18:34:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.623+0000 7fcc1f7fe700 1 --2- 172.21.15.42:0/364458195 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcc08060f70 0x7fcc08063430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:07.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.623+0000 7fcc1f7fe700 1 --2- 172.21.15.42:0/364458195 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcc08060f70 0x7fcc08063430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fcc20106b90 tx=0x7fcc10009040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:07.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.627+0000 7fcc07fff700 1 -- 172.21.15.42:0/364458195 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcc1404e470 con 0x7fcc200f77d0 2024-02-15T18:34:07.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.812+0000 7fcc24d40700 1 -- 172.21.15.42:0/364458195 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc200fef90 con 0x7fcc08060f70 2024-02-15T18:34:07.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.822+0000 7fcc07fff700 1 -- 172.21.15.42:0/364458195 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7fcc200fef90 con 0x7fcc08060f70 2024-02-15T18:34:07.814 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:34:07.814 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (25s) 20s ago 15m 13.5M - 0.23.0 ba2b418f427c d6474309eb10 2024-02-15T18:34:07.814 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (6m) 20s ago 15m 6643k - 16.2.14-521-g0e714d9a 1fbc92c859dc 2da9d875539a 2024-02-15T18:34:07.814 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (6m) 54s ago 14m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 5c46a68ff62b 2024-02-15T18:34:07.814 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 starting - - - - 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (105s) 20s ago 10m 82.4M - 16.2.14-521-g0e714d9a 1fbc92c859dc 39e47ffbe32b 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (119s) 20s ago 10m 80.5M - 16.2.14-521-g0e714d9a 1fbc92c859dc 372b8d41fced 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (76s) 54s ago 10m 11.5M - 16.2.14-521-g0e714d9a 1fbc92c859dc 4eb6a75798d1 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (86s) 54s ago 10m 12.3M - 16.2.14-521-g0e714d9a 1fbc92c859dc 551cd0ff7860 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (8m) 20s ago 17m 441M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (7m) 54s ago 14m 390M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (7m) 20s ago 17m 57.6M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (6m) 54s ago 14m 56.0M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (63s) 20s ago 15m 13.2M - 1.3.1 1dbe0e931976 e80f2a2363ff 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (57s) 54s ago 14m 11.3M - 1.3.1 1dbe0e931976 ee2cf38a118f 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (6m) 20s ago 13m 309M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc f226bea76cab 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (4m) 20s ago 13m 279M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 5114ed026724 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (4m) 20s ago 13m 190M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 2e26b1ec292a 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (3m) 54s ago 12m 339M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc c71b210091cb 2024-02-15T18:34:07.815 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (2m) 54s ago 12m 285M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 83bd9cdcb3da 2024-02-15T18:34:07.816 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (2m) 54s ago 11m 243M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc d45e566ed288 2024-02-15T18:34:07.816 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (45s) 20s ago 14m 59.9M - 2.33.4 514e6a882f6e b0c2ac6c79f3 2024-02-15T18:34:07.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.824+0000 7fcc24d40700 1 -- 172.21.15.42:0/364458195 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcc08060f70 msgr2=0x7fcc08063430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:07.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.824+0000 7fcc24d40700 1 --2- 172.21.15.42:0/364458195 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcc08060f70 0x7fcc08063430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fcc20106b90 tx=0x7fcc10009040).stop 2024-02-15T18:34:07.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.824+0000 7fcc24d40700 1 -- 172.21.15.42:0/364458195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc200f77d0 msgr2=0x7fcc20107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:07.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.824+0000 7fcc24d40700 1 --2- 172.21.15.42:0/364458195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc200f77d0 0x7fcc20107310 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fcc14013fb0 tx=0x7fcc14009b30).stop 2024-02-15T18:34:07.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.824+0000 7fcc24d40700 1 -- 172.21.15.42:0/364458195 shutdown_connections 2024-02-15T18:34:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.824+0000 7fcc24d40700 1 --2- 172.21.15.42:0/364458195 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcc08060f70 0x7fcc08063430 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.824+0000 7fcc24d40700 1 --2- 172.21.15.42:0/364458195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc200f6530 0x7fcc20106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.824+0000 7fcc24d40700 1 --2- 172.21.15.42:0/364458195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc200f77d0 0x7fcc20107310 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.824+0000 7fcc24d40700 1 -- 172.21.15.42:0/364458195 >> 172.21.15.42:0/364458195 conn(0x7fcc200f1c90 msgr2=0x7fcc200fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.824+0000 7fcc24d40700 1 -- 172.21.15.42:0/364458195 shutdown_connections 2024-02-15T18:34:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.824+0000 7fcc24d40700 1 -- 172.21.15.42:0/364458195 wait complete. 2024-02-15T18:34:07.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.931+0000 7fab0b617700 1 -- 172.21.15.42:0/3693779768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab040f66e0 msgr2=0x7fab040f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:07.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.931+0000 7fab0b617700 1 --2- 172.21.15.42:0/3693779768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab040f66e0 0x7fab040f6b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7faaf40099e0 tx=0x7faaf4004d10).stop 2024-02-15T18:34:07.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.931+0000 7fab0b617700 1 -- 172.21.15.42:0/3693779768 shutdown_connections 2024-02-15T18:34:07.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.931+0000 7fab0b617700 1 --2- 172.21.15.42:0/3693779768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab040f7980 0x7fab040f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.931+0000 7fab0b617700 1 --2- 172.21.15.42:0/3693779768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab040f66e0 0x7fab040f6b00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.931+0000 7fab0b617700 1 -- 172.21.15.42:0/3693779768 >> 172.21.15.42:0/3693779768 conn(0x7fab040f1e40 msgr2=0x7fab040f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:07.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.931+0000 7fab0b617700 1 -- 172.21.15.42:0/3693779768 shutdown_connections 2024-02-15T18:34:07.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.931+0000 7fab0b617700 1 -- 172.21.15.42:0/3693779768 wait complete. 2024-02-15T18:34:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab0b617700 1 Processor -- start 2024-02-15T18:34:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab0b617700 1 -- start start 2024-02-15T18:34:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab0b617700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab040f66e0 0x7fab04107170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab0b617700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab040f7980 0x7fab04107660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab0b617700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab04107c10 con 0x7fab040f66e0 2024-02-15T18:34:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab0b617700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab04107d50 con 0x7fab040f7980 2024-02-15T18:34:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab093b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab040f66e0 0x7fab04107170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab093b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab040f66e0 0x7fab04107170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55550/0 (socket says 172.21.15.42:55550) 2024-02-15T18:34:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab093b3700 1 -- 172.21.15.42:0/3066387242 learned_addr learned my addr 172.21.15.42:0/3066387242 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:34:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab08bb2700 1 --2- 172.21.15.42:0/3066387242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab040f7980 0x7fab04107660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab093b3700 1 -- 172.21.15.42:0/3066387242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab040f7980 msgr2=0x7fab04107660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab093b3700 1 --2- 172.21.15.42:0/3066387242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab040f7980 0x7fab04107660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab093b3700 1 -- 172.21.15.42:0/3066387242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf4009640 con 0x7fab040f66e0 2024-02-15T18:34:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab08bb2700 1 --2- 172.21.15.42:0/3066387242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab040f7980 0x7fab04107660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:34:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.932+0000 7fab093b3700 1 --2- 172.21.15.42:0/3066387242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab040f66e0 0x7fab04107170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7faaf4007f70 tx=0x7faaf4004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.933+0000 7fab01ffb700 1 -- 172.21.15.42:0/3066387242 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaf400e070 con 0x7fab040f66e0 2024-02-15T18:34:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.933+0000 7fab01ffb700 1 -- 172.21.15.42:0/3066387242 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7faaf400be70 con 0x7fab040f66e0 2024-02-15T18:34:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.933+0000 7fab0b617700 1 -- 172.21.15.42:0/3066387242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab04108690 con 0x7fab040f66e0 2024-02-15T18:34:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.933+0000 7fab01ffb700 1 -- 172.21.15.42:0/3066387242 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaf4018070 con 0x7fab040f66e0 2024-02-15T18:34:07.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.933+0000 7fab0b617700 1 -- 172.21.15.42:0/3066387242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab04108b70 con 0x7fab040f66e0 2024-02-15T18:34:07.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.934+0000 7fab0b617700 1 -- 172.21.15.42:0/3066387242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab04045ad0 con 0x7fab040f66e0 2024-02-15T18:34:07.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.934+0000 7fab01ffb700 1 -- 172.21.15.42:0/3066387242 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7faaf4024060 con 0x7fab040f66e0 2024-02-15T18:34:07.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.935+0000 7fab01ffb700 1 --2- 172.21.15.42:0/3066387242 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7faaec060f20 0x7faaec0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:07.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.935+0000 7fab01ffb700 1 -- 172.21.15.42:0/3066387242 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7faaf407d920 con 0x7fab040f66e0 2024-02-15T18:34:07.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.935+0000 7fab08bb2700 1 --2- 172.21.15.42:0/3066387242 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7faaec060f20 0x7faaec0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:07.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.935+0000 7fab08bb2700 1 --2- 172.21.15.42:0/3066387242 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7faaec060f20 0x7faaec0633e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7faaf8009740 tx=0x7faaf8009be0).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:07.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:07.939+0000 7fab01ffb700 1 -- 172.21.15.42:0/3066387242 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7faaf40516e0 con 0x7fab040f66e0 2024-02-15T18:34:08.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.189+0000 7fab0b617700 1 -- 172.21.15.42:0/3066387242 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fab04027a00 con 0x7fab040f66e0 2024-02-15T18:34:08.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.189+0000 7fab01ffb700 1 -- 172.21.15.42:0/3066387242 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+648 (secure 0 0 0) 0x7faaf40512d0 con 0x7fab040f66e0 2024-02-15T18:34:08.181 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:34:08.181 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:34:08.181 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:34:08.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:34:08.181 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:34:08.181 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:34:08.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:34:08.181 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:34:08.181 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 6 2024-02-15T18:34:08.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:34:08.182 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:34:08.182 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4 2024-02-15T18:34:08.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:34:08.182 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:34:08.182 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 14 2024-02-15T18:34:08.182 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:34:08.182 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:34:08.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.191+0000 7fab0b617700 1 -- 172.21.15.42:0/3066387242 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7faaec060f20 msgr2=0x7faaec0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:08.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.191+0000 7fab0b617700 1 --2- 172.21.15.42:0/3066387242 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7faaec060f20 0x7faaec0633e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7faaf8009740 tx=0x7faaf8009be0).stop 2024-02-15T18:34:08.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.191+0000 7fab0b617700 1 -- 172.21.15.42:0/3066387242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab040f66e0 msgr2=0x7fab04107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:08.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.191+0000 7fab0b617700 1 --2- 172.21.15.42:0/3066387242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab040f66e0 0x7fab04107170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7faaf4007f70 tx=0x7faaf4004d10).stop 2024-02-15T18:34:08.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.191+0000 7fab0b617700 1 -- 172.21.15.42:0/3066387242 shutdown_connections 2024-02-15T18:34:08.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.191+0000 7fab0b617700 1 --2- 172.21.15.42:0/3066387242 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7faaec060f20 0x7faaec0633e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:08.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.191+0000 7fab0b617700 1 --2- 172.21.15.42:0/3066387242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab040f66e0 0x7fab04107170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:08.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.191+0000 7fab0b617700 1 --2- 172.21.15.42:0/3066387242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab040f7980 0x7fab04107660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:08.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.191+0000 7fab0b617700 1 -- 172.21.15.42:0/3066387242 >> 172.21.15.42:0/3066387242 conn(0x7fab040f1e40 msgr2=0x7fab040fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:08.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.191+0000 7fab0b617700 1 -- 172.21.15.42:0/3066387242 shutdown_connections 2024-02-15T18:34:08.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.191+0000 7fab0b617700 1 -- 172.21.15.42:0/3066387242 wait complete. 2024-02-15T18:34:08.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.293+0000 7fca59b6f700 1 -- 172.21.15.42:0/3601635309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca540f66e0 msgr2=0x7fca540f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:08.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.293+0000 7fca59b6f700 1 --2- 172.21.15.42:0/3601635309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca540f66e0 0x7fca540f6b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7fca44004660 tx=0x7fca44009b30).stop 2024-02-15T18:34:08.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.293+0000 7fca59b6f700 1 -- 172.21.15.42:0/3601635309 shutdown_connections 2024-02-15T18:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.293+0000 7fca59b6f700 1 --2- 172.21.15.42:0/3601635309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca540f7980 0x7fca540f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.293+0000 7fca59b6f700 1 --2- 172.21.15.42:0/3601635309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca540f66e0 0x7fca540f6b00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.293+0000 7fca59b6f700 1 -- 172.21.15.42:0/3601635309 >> 172.21.15.42:0/3601635309 conn(0x7fca540f1e40 msgr2=0x7fca540f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.293+0000 7fca59b6f700 1 -- 172.21.15.42:0/3601635309 shutdown_connections 2024-02-15T18:34:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.293+0000 7fca59b6f700 1 -- 172.21.15.42:0/3601635309 wait complete. 2024-02-15T18:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.293+0000 7fca59b6f700 1 Processor -- start 2024-02-15T18:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.294+0000 7fca59b6f700 1 -- start start 2024-02-15T18:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.294+0000 7fca59b6f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca540f66e0 0x7fca5406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.294+0000 7fca59b6f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca540f7980 0x7fca54069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.294+0000 7fca59b6f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5406a380 con 0x7fca540f66e0 2024-02-15T18:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.294+0000 7fca59b6f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5406a4c0 con 0x7fca540f7980 2024-02-15T18:34:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.294+0000 7fca537fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca540f66e0 0x7fca5406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.294+0000 7fca537fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca540f66e0 0x7fca5406b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55560/0 (socket says 172.21.15.42:55560) 2024-02-15T18:34:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.294+0000 7fca537fe700 1 -- 172.21.15.42:0/1030612950 learned_addr learned my addr 172.21.15.42:0/1030612950 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:34:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.294+0000 7fca52ffd700 1 --2- 172.21.15.42:0/1030612950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca540f7980 0x7fca54069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.294+0000 7fca537fe700 1 -- 172.21.15.42:0/1030612950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca540f7980 msgr2=0x7fca54069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.294+0000 7fca537fe700 1 --2- 172.21.15.42:0/1030612950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca540f7980 0x7fca54069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.294+0000 7fca537fe700 1 -- 172.21.15.42:0/1030612950 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca44005040 con 0x7fca540f66e0 2024-02-15T18:34:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.294+0000 7fca537fe700 1 --2- 172.21.15.42:0/1030612950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca540f66e0 0x7fca5406b9a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fca44013fd0 tx=0x7fca44009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.295+0000 7fca3bfff700 1 -- 172.21.15.42:0/1030612950 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca44018070 con 0x7fca540f66e0 2024-02-15T18:34:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.295+0000 7fca3bfff700 1 -- 172.21.15.42:0/1030612950 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7fca44007c70 con 0x7fca540f66e0 2024-02-15T18:34:08.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.295+0000 7fca59b6f700 1 -- 172.21.15.42:0/1030612950 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5406a740 con 0x7fca540f66e0 2024-02-15T18:34:08.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.295+0000 7fca3bfff700 1 -- 172.21.15.42:0/1030612950 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca4401c620 con 0x7fca540f66e0 2024-02-15T18:34:08.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.295+0000 7fca59b6f700 1 -- 172.21.15.42:0/1030612950 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5406aca0 con 0x7fca540f66e0 2024-02-15T18:34:08.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.296+0000 7fca59b6f700 1 -- 172.21.15.42:0/1030612950 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca54045ad0 con 0x7fca540f66e0 2024-02-15T18:34:08.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.297+0000 7fca3bfff700 1 -- 172.21.15.42:0/1030612950 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fca4401d420 con 0x7fca540f66e0 2024-02-15T18:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.297+0000 7fca3bfff700 1 --2- 172.21.15.42:0/1030612950 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fca3c060f20 0x7fca3c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.298+0000 7fca3bfff700 1 -- 172.21.15.42:0/1030612950 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fca4407e6c0 con 0x7fca540f66e0 2024-02-15T18:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.298+0000 7fca52ffd700 1 --2- 172.21.15.42:0/1030612950 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fca3c060f20 0x7fca3c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:08.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.298+0000 7fca52ffd700 1 --2- 172.21.15.42:0/1030612950 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fca3c060f20 0x7fca3c0633e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fca480041d0 tx=0x7fca48009040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:08.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.301+0000 7fca3bfff700 1 -- 172.21.15.42:0/1030612950 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fca4404ea50 con 0x7fca540f66e0 2024-02-15T18:34:08.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:07.201878+0000 mgr.smithi042.kfoihb (mgr.24435) 407 : audit [DBG] from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:08.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:07.517437+0000 mgr.smithi042.kfoihb (mgr.24435) 408 : audit [DBG] from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:08.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:07.813984+0000 mgr.smithi042.kfoihb (mgr.24435) 409 : audit [DBG] from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:08.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:07.201878+0000 mgr.smithi042.kfoihb (mgr.24435) 407 : audit [DBG] from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:08.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:07.517437+0000 mgr.smithi042.kfoihb (mgr.24435) 408 : audit [DBG] from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:08.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:07.813984+0000 mgr.smithi042.kfoihb (mgr.24435) 409 : audit [DBG] from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:08.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.519+0000 7fca59b6f700 1 -- 172.21.15.42:0/1030612950 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fca540586e0 con 0x7fca540f66e0 2024-02-15T18:34:08.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.519+0000 7fca3bfff700 1 -- 172.21.15.42:0/1030612950 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1774 (secure 0 0 0) 0x7fca44052070 con 0x7fca540f66e0 2024-02-15T18:34:08.511 INFO:teuthology.orchestra.run.smithi042.stdout:e28 2024-02-15T18:34:08.511 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:34:08.511 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:34:08.511 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:34:08.511 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:34:08.511 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 28 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:32:55.244270+0000 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 113 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:34:08.512 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=34356} 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{0:34356} state up:active seq 12 join_fscid=1 addr [v2:172.21.15.42:6826/2962563293,v1:172.21.15.42:6827/2962563293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{0:34360} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/1562993914,v1:172.21.15.42:6829/1562993914] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:34:08.513 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.191:6824/540280968,v1:172.21.15.191:6825/540280968] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:34:08.514 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.191:6826/289198038,v1:172.21.15.191:6827/289198038] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:34:08.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.521+0000 7fca59b6f700 1 -- 172.21.15.42:0/1030612950 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fca3c060f20 msgr2=0x7fca3c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:08.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.521+0000 7fca59b6f700 1 --2- 172.21.15.42:0/1030612950 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fca3c060f20 0x7fca3c0633e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fca480041d0 tx=0x7fca48009040).stop 2024-02-15T18:34:08.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.521+0000 7fca59b6f700 1 -- 172.21.15.42:0/1030612950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca540f66e0 msgr2=0x7fca5406b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:08.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.521+0000 7fca59b6f700 1 --2- 172.21.15.42:0/1030612950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca540f66e0 0x7fca5406b9a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fca44013fd0 tx=0x7fca44009b30).stop 2024-02-15T18:34:08.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.521+0000 7fca59b6f700 1 -- 172.21.15.42:0/1030612950 shutdown_connections 2024-02-15T18:34:08.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.521+0000 7fca59b6f700 1 --2- 172.21.15.42:0/1030612950 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fca3c060f20 0x7fca3c0633e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:08.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.521+0000 7fca59b6f700 1 --2- 172.21.15.42:0/1030612950 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca540f66e0 0x7fca5406b9a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:08.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.521+0000 7fca59b6f700 1 --2- 172.21.15.42:0/1030612950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca540f7980 0x7fca54069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:08.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.521+0000 7fca59b6f700 1 -- 172.21.15.42:0/1030612950 >> 172.21.15.42:0/1030612950 conn(0x7fca540f1e40 msgr2=0x7fca540fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:08.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.522+0000 7fca59b6f700 1 -- 172.21.15.42:0/1030612950 shutdown_connections 2024-02-15T18:34:08.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.522+0000 7fca59b6f700 1 -- 172.21.15.42:0/1030612950 wait complete. 2024-02-15T18:34:08.515 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 28 2024-02-15T18:34:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.624+0000 7f359e677700 1 -- 172.21.15.42:0/1607361264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35980f6860 msgr2=0x7f35980f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:08.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.624+0000 7f359e677700 1 --2- 172.21.15.42:0/1607361264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35980f6860 0x7f35980f6cc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f3580004660 tx=0x7f3580009b30).stop 2024-02-15T18:34:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.624+0000 7f359e677700 1 -- 172.21.15.42:0/1607361264 shutdown_connections 2024-02-15T18:34:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.624+0000 7f359e677700 1 --2- 172.21.15.42:0/1607361264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35980f6860 0x7f35980f6cc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.624+0000 7f359e677700 1 --2- 172.21.15.42:0/1607361264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35980f55c0 0x7f35980f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:34:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.624+0000 7f359e677700 1 -- 172.21.15.42:0/1607361264 >> 172.21.15.42:0/1607361264 conn(0x7f35980f0d40 msgr2=0x7f35980f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.624+0000 7f359e677700 1 -- 172.21.15.42:0/1607361264 shutdown_connections 2024-02-15T18:34:08.616 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.624+0000 7f359e677700 1 -- 172.21.15.42:0/1607361264 wait complete. 2024-02-15T18:34:08.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.624+0000 7f359e677700 1 Processor -- start 2024-02-15T18:34:08.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.624+0000 7f359e677700 1 -- start start 2024-02-15T18:34:08.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f359e677700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35980f55c0 0x7f3598107050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:08.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f359e677700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35980f6860 0x7f3598107540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:08.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f359e677700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3598107b80 con 0x7f35980f6860 2024-02-15T18:34:08.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f359e677700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3598108430 con 0x7f35980f55c0 2024-02-15T18:34:08.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f35977fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35980f6860 0x7f3598107540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:08.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f35977fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35980f6860 0x7f3598107540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55582/0 (socket says 172.21.15.42:55582) 2024-02-15T18:34:08.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f35977fe700 1 -- 172.21.15.42:0/932987224 learned_addr learned my addr 172.21.15.42:0/932987224 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:34:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f3597fff700 1 --2- 172.21.15.42:0/932987224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35980f55c0 0x7f3598107050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f35977fe700 1 -- 172.21.15.42:0/932987224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35980f55c0 msgr2=0x7f3598107050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f35977fe700 1 --2- 172.21.15.42:0/932987224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35980f55c0 0x7f3598107050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f35977fe700 1 -- 172.21.15.42:0/932987224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3580005040 con 0x7f35980f6860 2024-02-15T18:34:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f35977fe700 1 --2- 172.21.15.42:0/932987224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35980f6860 0x7f3598107540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f3580009f50 tx=0x7f3580009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f3594ff9700 1 -- 172.21.15.42:0/932987224 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3580018070 con 0x7f35980f6860 2024-02-15T18:34:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f359e677700 1 -- 172.21.15.42:0/932987224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35981086b0 con 0x7f35980f6860 2024-02-15T18:34:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f3594ff9700 1 -- 172.21.15.42:0/932987224 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1137+0+0 (secure 0 0 0) 0x7f3580007d70 con 0x7f35980f6860 2024-02-15T18:34:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f3594ff9700 1 -- 172.21.15.42:0/932987224 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358001c620 con 0x7f35980f6860 2024-02-15T18:34:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.625+0000 7f359e677700 1 -- 172.21.15.42:0/932987224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3598108b90 con 0x7f35980f6860 2024-02-15T18:34:08.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.626+0000 7f359e677700 1 -- 172.21.15.42:0/932987224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35980f49b0 con 0x7f35980f6860 2024-02-15T18:34:08.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.627+0000 7f3594ff9700 1 -- 172.21.15.42:0/932987224 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f358001c8e0 con 0x7f35980f6860 2024-02-15T18:34:08.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.627+0000 7f3594ff9700 1 --2- 172.21.15.42:0/932987224 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3584060f20 0x7f35840633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:08.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.627+0000 7f3594ff9700 1 -- 172.21.15.42:0/932987224 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f358007e1e0 con 0x7f35980f6860 2024-02-15T18:34:08.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.627+0000 7f3597fff700 1 --2- 172.21.15.42:0/932987224 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3584060f20 0x7f35840633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:08.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.628+0000 7f3597fff700 1 --2- 172.21.15.42:0/932987224 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3584060f20 0x7f35840633e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f35980f66c0 tx=0x7f3588008040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:08.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.631+0000 7f3594ff9700 1 -- 172.21.15.42:0/932987224 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3580051f20 con 0x7f35980f6860 2024-02-15T18:34:08.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.841+0000 7f359e677700 1 -- 172.21.15.42:0/932987224 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35980ff110 con 0x7f3584060f20 2024-02-15T18:34:08.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.842+0000 7f3594ff9700 1 -- 172.21.15.42:0/932987224 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f35980ff110 con 0x7f3584060f20 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc", 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr", 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "crash" 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/21 daemons upgraded", 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading grafana daemons", 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-02-15T18:34:08.834 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:34:08.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.844+0000 7f359e677700 1 -- 172.21.15.42:0/932987224 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3584060f20 msgr2=0x7f35840633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:08.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.844+0000 7f359e677700 1 --2- 172.21.15.42:0/932987224 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3584060f20 0x7f35840633e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f35980f66c0 tx=0x7f3588008040).stop 2024-02-15T18:34:08.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.844+0000 7f359e677700 1 -- 172.21.15.42:0/932987224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35980f6860 msgr2=0x7f3598107540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:08.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.844+0000 7f359e677700 1 --2- 172.21.15.42:0/932987224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35980f6860 0x7f3598107540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f3580009f50 tx=0x7f3580009b30).stop 2024-02-15T18:34:08.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.844+0000 7f359e677700 1 -- 172.21.15.42:0/932987224 shutdown_connections 2024-02-15T18:34:08.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.844+0000 7f359e677700 1 --2- 172.21.15.42:0/932987224 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3584060f20 0x7f35840633e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:08.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.844+0000 7f359e677700 1 --2- 172.21.15.42:0/932987224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35980f55c0 0x7f3598107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:08.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.844+0000 7f359e677700 1 --2- 172.21.15.42:0/932987224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35980f6860 0x7f3598107540 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:08.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.844+0000 7f359e677700 1 -- 172.21.15.42:0/932987224 >> 172.21.15.42:0/932987224 conn(0x7f35980f0d40 msgr2=0x7f35980fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:08.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.844+0000 7f359e677700 1 -- 172.21.15.42:0/932987224 shutdown_connections 2024-02-15T18:34:08.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:08.844+0000 7f359e677700 1 -- 172.21.15.42:0/932987224 wait complete. 2024-02-15T18:34:09.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:08.190423+0000 mon.smithi042 (mon.0) 705 : audit [DBG] from='client.? 172.21.15.42:0/3066387242' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:09.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:08.520673+0000 mon.smithi042 (mon.0) 706 : audit [DBG] from='client.? 172.21.15.42:0/1030612950' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:34:09.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:08.785970+0000 mgr.smithi042.kfoihb (mgr.24435) 410 : cluster [DBG] pgmap v309: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:09.305 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:08.842752+0000 mgr.smithi042.kfoihb (mgr.24435) 411 : audit [DBG] from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:09.364 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:08.190423+0000 mon.smithi042 (mon.0) 705 : audit [DBG] from='client.? 172.21.15.42:0/3066387242' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:09.364 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:08.520673+0000 mon.smithi042 (mon.0) 706 : audit [DBG] from='client.? 172.21.15.42:0/1030612950' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:34:09.364 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:08.785970+0000 mgr.smithi042.kfoihb (mgr.24435) 410 : cluster [DBG] pgmap v309: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:09.364 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:08.842752+0000 mgr.smithi042.kfoihb (mgr.24435) 411 : audit [DBG] from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:10.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.634926+0000 mon.smithi042 (mon.0) 707 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:10.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.635697+0000 mon.smithi042 (mon.0) 708 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:34:10.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.636297+0000 mon.smithi042 (mon.0) 709 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:34:10.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:34:09.636821+0000 mgr.smithi042.kfoihb (mgr.24435) 412 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:34:10.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.639759+0000 mon.smithi042 (mon.0) 710 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:10.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.645714+0000 mon.smithi042 (mon.0) 711 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:34:10.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.646070+0000 mgr.smithi042.kfoihb (mgr.24435) 413 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:34:10.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.646795+0000 mon.smithi042 (mon.0) 712 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi042.front.sepia.ceph.com:3000"}]: dispatch 2024-02-15T18:34:10.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.647088+0000 mgr.smithi042.kfoihb (mgr.24435) 414 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi042.front.sepia.ceph.com:3000"}]: dispatch 2024-02-15T18:34:10.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.650535+0000 mon.smithi042 (mon.0) 713 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:10.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.652560+0000 mon.smithi042 (mon.0) 714 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:34:10.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.656366+0000 mon.smithi042 (mon.0) 715 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:10.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.657863+0000 mon.smithi042 (mon.0) 716 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:10.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.659542+0000 mon.smithi042 (mon.0) 717 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:10.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.660983+0000 mon.smithi042 (mon.0) 718 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:10.943 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.663045+0000 mon.smithi042 (mon.0) 719 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:10.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.664096+0000 mon.smithi042 (mon.0) 720 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:10.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.664958+0000 mon.smithi042 (mon.0) 721 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:10.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.666182+0000 mon.smithi042 (mon.0) 722 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:10.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.667184+0000 mon.smithi042 (mon.0) 723 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:10.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.668126+0000 mon.smithi042 (mon.0) 724 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:10.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.669676+0000 mon.smithi042 (mon.0) 725 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:10.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.670896+0000 mon.smithi042 (mon.0) 726 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:10.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.671924+0000 mon.smithi042 (mon.0) 727 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:10.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.672937+0000 mon.smithi042 (mon.0) 728 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:10.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:34:09.673482+0000 mgr.smithi042.kfoihb (mgr.24435) 415 : cephadm [INF] Upgrade: Finalizing container_image settings 2024-02-15T18:34:10.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.677116+0000 mon.smithi042 (mon.0) 729 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:10.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.677706+0000 mon.smithi042 (mon.0) 730 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-15T18:34:10.944 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.682101+0000 mon.smithi042 (mon.0) 731 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.682630+0000 mon.smithi042 (mon.0) 732 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.687825+0000 mon.smithi042 (mon.0) 733 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.688383+0000 mon.smithi042 (mon.0) 734 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.692950+0000 mon.smithi042 (mon.0) 735 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.693507+0000 mon.smithi042 (mon.0) 736 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.698474+0000 mon.smithi042 (mon.0) 737 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.699013+0000 mon.smithi042 (mon.0) 738 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.703615+0000 mon.smithi042 (mon.0) 739 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.704165+0000 mon.smithi042 (mon.0) 740 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.709207+0000 mon.smithi042 (mon.0) 741 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.709708+0000 mon.smithi042 (mon.0) 742 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.714541+0000 mon.smithi042 (mon.0) 743 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.715075+0000 mon.smithi042 (mon.0) 744 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.715553+0000 mon.smithi042 (mon.0) 745 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-15T18:34:10.945 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.720164+0000 mon.smithi042 (mon.0) 746 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-15T18:34:10.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.720694+0000 mon.smithi042 (mon.0) 747 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-15T18:34:10.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.725179+0000 mon.smithi042 (mon.0) 748 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-15T18:34:10.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.725737+0000 mon.smithi042 (mon.0) 749 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:34:10.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.726247+0000 mon.smithi042 (mon.0) 750 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:34:10.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.726761+0000 mon.smithi042 (mon.0) 751 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:34:10.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.727294+0000 mon.smithi042 (mon.0) 752 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:34:10.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.727813+0000 mon.smithi042 (mon.0) 753 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-02-15T18:34:10.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.730487+0000 mon.smithi042 (mon.0) 754 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-02-15T18:34:10.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:34:09.730955+0000 mgr.smithi042.kfoihb (mgr.24435) 416 : cephadm [INF] Upgrade: Complete! 2024-02-15T18:34:10.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.731308+0000 mon.smithi042 (mon.0) 755 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-15T18:34:10.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.735108+0000 mon.smithi042 (mon.0) 756 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-15T18:34:10.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.735441+0000 mon.smithi042 (mon.0) 757 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:34:10.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.736650+0000 mon.smithi042 (mon.0) 758 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:34:10.946 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.737191+0000 mon.smithi042 (mon.0) 759 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:34:10.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:34:09.737689+0000 mgr.smithi042.kfoihb (mgr.24435) 417 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:34:10.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.740713+0000 mon.smithi042 (mon.0) 760 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:10.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.745486+0000 mon.smithi042 (mon.0) 761 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:34:10.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.746691+0000 mon.smithi042 (mon.0) 762 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:34:10.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.747255+0000 mon.smithi042 (mon.0) 763 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:34:10.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:34:09.747757+0000 mgr.smithi042.kfoihb (mgr.24435) 418 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:34:10.947 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:09.750834+0000 mon.smithi042 (mon.0) 764 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:11.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.634926+0000 mon.smithi042 (mon.0) 707 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:11.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.635697+0000 mon.smithi042 (mon.0) 708 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:34:11.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.636297+0000 mon.smithi042 (mon.0) 709 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:34:11.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:34:09.636821+0000 mgr.smithi042.kfoihb (mgr.24435) 412 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:34:11.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.639759+0000 mon.smithi042 (mon.0) 710 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:11.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.645714+0000 mon.smithi042 (mon.0) 711 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.646070+0000 mgr.smithi042.kfoihb (mgr.24435) 413 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.646795+0000 mon.smithi042 (mon.0) 712 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi042.front.sepia.ceph.com:3000"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.647088+0000 mgr.smithi042.kfoihb (mgr.24435) 414 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi042.front.sepia.ceph.com:3000"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.650535+0000 mon.smithi042 (mon.0) 713 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.652560+0000 mon.smithi042 (mon.0) 714 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.656366+0000 mon.smithi042 (mon.0) 715 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.657863+0000 mon.smithi042 (mon.0) 716 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.659542+0000 mon.smithi042 (mon.0) 717 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.660983+0000 mon.smithi042 (mon.0) 718 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.663045+0000 mon.smithi042 (mon.0) 719 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.664096+0000 mon.smithi042 (mon.0) 720 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.664958+0000 mon.smithi042 (mon.0) 721 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.666182+0000 mon.smithi042 (mon.0) 722 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.667184+0000 mon.smithi042 (mon.0) 723 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.668126+0000 mon.smithi042 (mon.0) 724 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:11.056 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.669676+0000 mon.smithi042 (mon.0) 725 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.670896+0000 mon.smithi042 (mon.0) 726 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.671924+0000 mon.smithi042 (mon.0) 727 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.672937+0000 mon.smithi042 (mon.0) 728 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:34:09.673482+0000 mgr.smithi042.kfoihb (mgr.24435) 415 : cephadm [INF] Upgrade: Finalizing container_image settings 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.677116+0000 mon.smithi042 (mon.0) 729 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.677706+0000 mon.smithi042 (mon.0) 730 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.682101+0000 mon.smithi042 (mon.0) 731 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.682630+0000 mon.smithi042 (mon.0) 732 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.687825+0000 mon.smithi042 (mon.0) 733 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.688383+0000 mon.smithi042 (mon.0) 734 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.692950+0000 mon.smithi042 (mon.0) 735 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.693507+0000 mon.smithi042 (mon.0) 736 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.698474+0000 mon.smithi042 (mon.0) 737 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.699013+0000 mon.smithi042 (mon.0) 738 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-15T18:34:11.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.703615+0000 mon.smithi042 (mon.0) 739 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.704165+0000 mon.smithi042 (mon.0) 740 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.709207+0000 mon.smithi042 (mon.0) 741 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.709708+0000 mon.smithi042 (mon.0) 742 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.714541+0000 mon.smithi042 (mon.0) 743 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.715075+0000 mon.smithi042 (mon.0) 744 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.715553+0000 mon.smithi042 (mon.0) 745 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.720164+0000 mon.smithi042 (mon.0) 746 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.720694+0000 mon.smithi042 (mon.0) 747 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.725179+0000 mon.smithi042 (mon.0) 748 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.725737+0000 mon.smithi042 (mon.0) 749 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.726247+0000 mon.smithi042 (mon.0) 750 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.726761+0000 mon.smithi042 (mon.0) 751 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.727294+0000 mon.smithi042 (mon.0) 752 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.727813+0000 mon.smithi042 (mon.0) 753 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.730487+0000 mon.smithi042 (mon.0) 754 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-02-15T18:34:11.058 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:34:09.730955+0000 mgr.smithi042.kfoihb (mgr.24435) 416 : cephadm [INF] Upgrade: Complete! 2024-02-15T18:34:11.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.731308+0000 mon.smithi042 (mon.0) 755 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-15T18:34:11.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.735108+0000 mon.smithi042 (mon.0) 756 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-15T18:34:11.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.735441+0000 mon.smithi042 (mon.0) 757 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:34:11.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.736650+0000 mon.smithi042 (mon.0) 758 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:34:11.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.737191+0000 mon.smithi042 (mon.0) 759 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:34:11.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:34:09.737689+0000 mgr.smithi042.kfoihb (mgr.24435) 417 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:34:11.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.740713+0000 mon.smithi042 (mon.0) 760 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:11.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.745486+0000 mon.smithi042 (mon.0) 761 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:34:11.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.746691+0000 mon.smithi042 (mon.0) 762 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:34:11.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.747255+0000 mon.smithi042 (mon.0) 763 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:34:11.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:34:09.747757+0000 mgr.smithi042.kfoihb (mgr.24435) 418 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:34:11.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:09.750834+0000 mon.smithi042 (mon.0) 764 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:11.941 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:10.786636+0000 mgr.smithi042.kfoihb (mgr.24435) 419 : cluster [DBG] pgmap v310: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:11.942 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:11.022782+0000 mon.smithi042 (mon.0) 765 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:12.054 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:10.786636+0000 mgr.smithi042.kfoihb (mgr.24435) 419 : cluster [DBG] pgmap v310: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:12.055 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:11.022782+0000 mon.smithi042 (mon.0) 765 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:34:14.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:12.787094+0000 mgr.smithi042.kfoihb (mgr.24435) 420 : cluster [DBG] pgmap v311: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:14.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:12.787094+0000 mgr.smithi042.kfoihb (mgr.24435) 420 : cluster [DBG] pgmap v311: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:16.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:14.787477+0000 mgr.smithi042.kfoihb (mgr.24435) 421 : cluster [DBG] pgmap v312: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:16.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:14.787477+0000 mgr.smithi042.kfoihb (mgr.24435) 421 : cluster [DBG] pgmap v312: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:17.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:16.788047+0000 mgr.smithi042.kfoihb (mgr.24435) 422 : cluster [DBG] pgmap v313: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:17.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:16.788047+0000 mgr.smithi042.kfoihb (mgr.24435) 422 : cluster [DBG] pgmap v313: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:20.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:18.788498+0000 mgr.smithi042.kfoihb (mgr.24435) 423 : cluster [DBG] pgmap v314: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:20.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:18.788498+0000 mgr.smithi042.kfoihb (mgr.24435) 423 : cluster [DBG] pgmap v314: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:22.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:20.789081+0000 mgr.smithi042.kfoihb (mgr.24435) 424 : cluster [DBG] pgmap v315: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:22.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:20.789081+0000 mgr.smithi042.kfoihb (mgr.24435) 424 : cluster [DBG] pgmap v315: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:24.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:22.789463+0000 mgr.smithi042.kfoihb (mgr.24435) 425 : cluster [DBG] pgmap v316: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:24.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:22.789463+0000 mgr.smithi042.kfoihb (mgr.24435) 425 : cluster [DBG] pgmap v316: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:26.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:24.789803+0000 mgr.smithi042.kfoihb (mgr.24435) 426 : cluster [DBG] pgmap v317: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:26.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:24.789803+0000 mgr.smithi042.kfoihb (mgr.24435) 426 : cluster [DBG] pgmap v317: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:27.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:26.790377+0000 mgr.smithi042.kfoihb (mgr.24435) 427 : cluster [DBG] pgmap v318: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:27.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:26.790377+0000 mgr.smithi042.kfoihb (mgr.24435) 427 : cluster [DBG] pgmap v318: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:30.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:28.790817+0000 mgr.smithi042.kfoihb (mgr.24435) 428 : cluster [DBG] pgmap v319: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:30.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:28.790817+0000 mgr.smithi042.kfoihb (mgr.24435) 428 : cluster [DBG] pgmap v319: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:32.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:30.791361+0000 mgr.smithi042.kfoihb (mgr.24435) 429 : cluster [DBG] pgmap v320: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:32.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:30.791361+0000 mgr.smithi042.kfoihb (mgr.24435) 429 : cluster [DBG] pgmap v320: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:34.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:32.791883+0000 mgr.smithi042.kfoihb (mgr.24435) 430 : cluster [DBG] pgmap v321: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:34.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:32.791883+0000 mgr.smithi042.kfoihb (mgr.24435) 430 : cluster [DBG] pgmap v321: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:36.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:34.792332+0000 mgr.smithi042.kfoihb (mgr.24435) 431 : cluster [DBG] pgmap v322: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:36.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:34.792332+0000 mgr.smithi042.kfoihb (mgr.24435) 431 : cluster [DBG] pgmap v322: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:37.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:36.792831+0000 mgr.smithi042.kfoihb (mgr.24435) 432 : cluster [DBG] pgmap v323: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:37.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:36.792831+0000 mgr.smithi042.kfoihb (mgr.24435) 432 : cluster [DBG] pgmap v323: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.959+0000 7ff27fb7a700 1 -- 172.21.15.42:0/2777848099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2780f6860 msgr2=0x7ff2780f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.959+0000 7ff27fb7a700 1 --2- 172.21.15.42:0/2777848099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2780f6860 0x7ff2780f6cc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7ff26c004660 tx=0x7ff26c009b30).stop 2024-02-15T18:34:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.959+0000 7ff27fb7a700 1 -- 172.21.15.42:0/2777848099 shutdown_connections 2024-02-15T18:34:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.959+0000 7ff27fb7a700 1 --2- 172.21.15.42:0/2777848099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2780f6860 0x7ff2780f6cc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.959+0000 7ff27fb7a700 1 --2- 172.21.15.42:0/2777848099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2780f55c0 0x7ff2780f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.959+0000 7ff27fb7a700 1 -- 172.21.15.42:0/2777848099 >> 172.21.15.42:0/2777848099 conn(0x7ff2780f0d40 msgr2=0x7ff2780f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.960+0000 7ff27fb7a700 1 -- 172.21.15.42:0/2777848099 shutdown_connections 2024-02-15T18:34:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.960+0000 7ff27fb7a700 1 -- 172.21.15.42:0/2777848099 wait complete. 2024-02-15T18:34:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.960+0000 7ff27fb7a700 1 Processor -- start 2024-02-15T18:34:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.960+0000 7ff27fb7a700 1 -- start start 2024-02-15T18:34:38.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.960+0000 7ff27fb7a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2780f55c0 0x7ff278109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.960+0000 7ff27fb7a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2780f6860 0x7ff27810c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.960+0000 7ff27fb7a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780603c0 con 0x7ff2780f6860 2024-02-15T18:34:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.960+0000 7ff27fb7a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278060500 con 0x7ff2780f55c0 2024-02-15T18:34:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.960+0000 7ff27d115700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2780f6860 0x7ff27810c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.960+0000 7ff27d115700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2780f6860 0x7ff27810c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42712/0 (socket says 172.21.15.42:42712) 2024-02-15T18:34:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.960+0000 7ff27d115700 1 -- 172.21.15.42:0/3359777471 learned_addr learned my addr 172.21.15.42:0/3359777471 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:34:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.961+0000 7ff27d916700 1 --2- 172.21.15.42:0/3359777471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2780f55c0 0x7ff278109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.961+0000 7ff27d916700 1 -- 172.21.15.42:0/3359777471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2780f6860 msgr2=0x7ff27810c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.961+0000 7ff27d916700 1 --2- 172.21.15.42:0/3359777471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2780f6860 0x7ff27810c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.961+0000 7ff27d916700 1 -- 172.21.15.42:0/3359777471 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c005040 con 0x7ff2780f55c0 2024-02-15T18:34:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.961+0000 7ff27d115700 1 --2- 172.21.15.42:0/3359777471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2780f6860 0x7ff27810c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-15T18:34:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.961+0000 7ff27d916700 1 --2- 172.21.15.42:0/3359777471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2780f55c0 0x7ff278109cd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7ff2780f66c0 tx=0x7ff27400b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.961+0000 7ff26a7fc700 1 -- 172.21.15.42:0/3359777471 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2740092e0 con 0x7ff2780f55c0 2024-02-15T18:34:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.961+0000 7ff27fb7a700 1 -- 172.21.15.42:0/3359777471 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27810c780 con 0x7ff2780f55c0 2024-02-15T18:34:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.961+0000 7ff26a7fc700 1 -- 172.21.15.42:0/3359777471 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7ff27400f040 con 0x7ff2780f55c0 2024-02-15T18:34:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.961+0000 7ff27fb7a700 1 -- 172.21.15.42:0/3359777471 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27810cd10 con 0x7ff2780f55c0 2024-02-15T18:34:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.961+0000 7ff26a7fc700 1 -- 172.21.15.42:0/3359777471 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff274014900 con 0x7ff2780f55c0 2024-02-15T18:34:38.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.962+0000 7ff27fb7a700 1 -- 172.21.15.42:0/3359777471 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2780f49b0 con 0x7ff2780f55c0 2024-02-15T18:34:38.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.963+0000 7ff26a7fc700 1 -- 172.21.15.42:0/3359777471 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7ff27401b030 con 0x7ff2780f55c0 2024-02-15T18:34:38.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.963+0000 7ff26a7fc700 1 --2- 172.21.15.42:0/3359777471 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff264060f20 0x7ff2640633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:38.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.963+0000 7ff26a7fc700 1 -- 172.21.15.42:0/3359777471 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7ff27407bf00 con 0x7ff2780f55c0 2024-02-15T18:34:38.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.963+0000 7ff27d115700 1 --2- 172.21.15.42:0/3359777471 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff264060f20 0x7ff2640633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:38.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.963+0000 7ff27d115700 1 --2- 172.21.15.42:0/3359777471 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff264060f20 0x7ff2640633e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7ff26c009d60 tx=0x7ff26c016040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:38.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:38.968+0000 7ff26a7fc700 1 -- 172.21.15.42:0/3359777471 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff27404c1a0 con 0x7ff2780f55c0 2024-02-15T18:34:39.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:39.158+0000 7ff27fb7a700 1 -- 172.21.15.42:0/3359777471 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2780008d0 con 0x7ff264060f20 2024-02-15T18:34:39.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:39.159+0000 7ff26a7fc700 1 -- 172.21.15.42:0/3359777471 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7ff2780008d0 con 0x7ff264060f20 2024-02-15T18:34:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:39.161+0000 7ff27fb7a700 1 -- 172.21.15.42:0/3359777471 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff264060f20 msgr2=0x7ff2640633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:39.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:39.161+0000 7ff27fb7a700 1 --2- 172.21.15.42:0/3359777471 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff264060f20 0x7ff2640633e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7ff26c009d60 tx=0x7ff26c016040).stop 2024-02-15T18:34:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:39.161+0000 7ff27fb7a700 1 -- 172.21.15.42:0/3359777471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2780f55c0 msgr2=0x7ff278109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:39.161+0000 7ff27fb7a700 1 --2- 172.21.15.42:0/3359777471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2780f55c0 0x7ff278109cd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7ff2780f66c0 tx=0x7ff27400b040).stop 2024-02-15T18:34:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:39.161+0000 7ff27fb7a700 1 -- 172.21.15.42:0/3359777471 shutdown_connections 2024-02-15T18:34:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:39.161+0000 7ff27fb7a700 1 --2- 172.21.15.42:0/3359777471 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7ff264060f20 0x7ff2640633e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:39.161+0000 7ff27fb7a700 1 --2- 172.21.15.42:0/3359777471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2780f55c0 0x7ff278109cd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:39.161+0000 7ff27fb7a700 1 --2- 172.21.15.42:0/3359777471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2780f6860 0x7ff27810c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:39.161+0000 7ff27fb7a700 1 -- 172.21.15.42:0/3359777471 >> 172.21.15.42:0/3359777471 conn(0x7ff2780f0d40 msgr2=0x7ff2780fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:39.161+0000 7ff27fb7a700 1 -- 172.21.15.42:0/3359777471 shutdown_connections 2024-02-15T18:34:39.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:39.161+0000 7ff27fb7a700 1 -- 172.21.15.42:0/3359777471 wait complete. 2024-02-15T18:34:39.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph orch ps' 2024-02-15T18:34:40.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:38.793448+0000 mgr.smithi042.kfoihb (mgr.24435) 433 : cluster [DBG] pgmap v324: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:40.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:38.793448+0000 mgr.smithi042.kfoihb (mgr.24435) 433 : cluster [DBG] pgmap v324: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:41.054 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:40 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:39.159772+0000 mgr.smithi042.kfoihb (mgr.24435) 434 : audit [DBG] from='client.44405 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:41.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:39.159772+0000 mgr.smithi042.kfoihb (mgr.24435) 434 : audit [DBG] from='client.44405 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:42.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.025+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/1858002755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ac00f66e0 msgr2=0x7f8ac00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:42.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.025+0000 7f8ac5bc2700 1 --2- 172.21.15.42:0/1858002755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ac00f66e0 0x7f8ac00f6b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f8ab00099e0 tx=0x7f8ab0004d10).stop 2024-02-15T18:34:42.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.025+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/1858002755 shutdown_connections 2024-02-15T18:34:42.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.025+0000 7f8ac5bc2700 1 --2- 172.21.15.42:0/1858002755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac00f7980 0x7f8ac00f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:34:42.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.025+0000 7f8ac5bc2700 1 --2- 172.21.15.42:0/1858002755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ac00f66e0 0x7f8ac00f6b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:42.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.025+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/1858002755 >> 172.21.15.42:0/1858002755 conn(0x7f8ac00f1e40 msgr2=0x7f8ac00f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:42.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.025+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/1858002755 shutdown_connections 2024-02-15T18:34:42.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.025+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/1858002755 wait complete. 2024-02-15T18:34:42.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.026+0000 7f8ac5bc2700 1 Processor -- start 2024-02-15T18:34:42.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.026+0000 7f8ac5bc2700 1 -- start start 2024-02-15T18:34:42.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.026+0000 7f8ac5bc2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac00f66e0 0x7f8ac0107260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:42.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.026+0000 7f8ac5bc2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ac00f7980 0x7f8ac0107750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:42.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.026+0000 7f8ac5bc2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac0107d90 con 0x7f8ac00f7980 2024-02-15T18:34:42.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.026+0000 7f8ac5bc2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac0108640 con 0x7f8ac00f66e0 2024-02-15T18:34:42.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.026+0000 7f8abeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ac00f7980 0x7f8ac0107750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:42.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.026+0000 7f8abeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ac00f7980 0x7f8ac0107750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40782/0 (socket says 172.21.15.42:40782) 2024-02-15T18:34:42.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.026+0000 7f8abeffd700 1 -- 172.21.15.42:0/3908041736 learned_addr learned my addr 172.21.15.42:0/3908041736 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:34:42.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.026+0000 7f8abeffd700 1 -- 172.21.15.42:0/3908041736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac00f66e0 msgr2=0x7f8ac0107260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:42.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.026+0000 7f8abf7fe700 1 --2- 172.21.15.42:0/3908041736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac00f66e0 0x7f8ac0107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:42.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.026+0000 7f8abeffd700 1 --2- 172.21.15.42:0/3908041736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac00f66e0 0x7f8ac0107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:42.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.026+0000 7f8abeffd700 1 -- 172.21.15.42:0/3908041736 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab0009640 con 0x7f8ac00f7980 2024-02-15T18:34:42.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.027+0000 7f8abeffd700 1 --2- 172.21.15.42:0/3908041736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ac00f7980 0x7f8ac0107750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f8aa800b410 tx=0x7f8aa8006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:42.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.027+0000 7f8ab7fff700 1 -- 172.21.15.42:0/3908041736 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8aa80125f0 con 0x7f8ac00f7980 2024-02-15T18:34:42.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.027+0000 7f8ab7fff700 1 -- 172.21.15.42:0/3908041736 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f8aa8012c40 con 0x7f8ac00f7980 2024-02-15T18:34:42.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.027+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/3908041736 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac0108920 con 0x7f8ac00f7980 2024-02-15T18:34:42.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.027+0000 7f8ab7fff700 1 -- 172.21.15.42:0/3908041736 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8aa8002b80 con 0x7f8ac00f7980 2024-02-15T18:34:42.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.027+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/3908041736 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac0108e60 con 0x7f8ac00f7980 2024-02-15T18:34:42.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.028+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/3908041736 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac00f5ad0 con 0x7f8ac00f7980 2024-02-15T18:34:42.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.028+0000 7f8ab7fff700 1 -- 172.21.15.42:0/3908041736 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f8aa800f030 con 0x7f8ac00f7980 2024-02-15T18:34:42.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.029+0000 7f8ab7fff700 1 --2- 172.21.15.42:0/3908041736 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f8aac060f20 0x7f8aac0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:42.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.029+0000 7f8ab7fff700 1 -- 172.21.15.42:0/3908041736 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f8aa807c7f0 con 0x7f8ac00f7980 2024-02-15T18:34:42.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.029+0000 7f8abf7fe700 1 --2- 172.21.15.42:0/3908041736 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f8aac060f20 0x7f8aac0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:42.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.029+0000 7f8abf7fe700 1 --2- 172.21.15.42:0/3908041736 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f8aac060f20 0x7f8aac0633e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f8ab000b7a0 tx=0x7f8ab0004d10).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:42.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.033+0000 7f8ab7fff700 1 -- 172.21.15.42:0/3908041736 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8aa8050470 con 0x7f8ac00f7980 2024-02-15T18:34:42.054 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:40.793871+0000 mgr.smithi042.kfoihb (mgr.24435) 435 : cluster [DBG] pgmap v325: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:42.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:40.793871+0000 mgr.smithi042.kfoihb (mgr.24435) 435 : cluster [DBG] pgmap v325: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:42.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.215+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/3908041736 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ac00ff110 con 0x7f8aac060f20 2024-02-15T18:34:42.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.225+0000 7f8ab7fff700 1 -- 172.21.15.42:0/3908041736 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f8ac00ff110 con 0x7f8aac060f20 2024-02-15T18:34:42.217 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:34:42.217 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (59s) 32s ago 16m 13.9M - 0.23.0 ba2b418f427c d6474309eb10 2024-02-15T18:34:42.217 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (6m) 32s ago 16m 6643k - 16.2.14-521-g0e714d9a 1fbc92c859dc 2da9d875539a 2024-02-15T18:34:42.217 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (6m) 89s ago 15m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 5c46a68ff62b 2024-02-15T18:34:42.217 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (37s) 32s ago 15m 42.2M - 8.3.5 dad864ee21e9 4fbf722d87dc 2024-02-15T18:34:42.217 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (2m) 32s ago 10m 82.4M - 16.2.14-521-g0e714d9a 1fbc92c859dc 39e47ffbe32b 2024-02-15T18:34:42.217 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (2m) 32s ago 10m 80.5M - 16.2.14-521-g0e714d9a 1fbc92c859dc 372b8d41fced 2024-02-15T18:34:42.217 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (110s) 89s ago 10m 11.5M - 16.2.14-521-g0e714d9a 1fbc92c859dc 4eb6a75798d1 2024-02-15T18:34:42.217 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (2m) 89s ago 10m 12.3M - 16.2.14-521-g0e714d9a 1fbc92c859dc 551cd0ff7860 2024-02-15T18:34:42.217 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (8m) 32s ago 17m 444M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:34:42.217 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (8m) 89s ago 15m 390M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:34:42.217 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (7m) 32s ago 17m 57.9M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:34:42.218 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (7m) 89s ago 15m 56.0M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:34:42.218 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (98s) 32s ago 15m 14.2M - 1.3.1 1dbe0e931976 e80f2a2363ff 2024-02-15T18:34:42.218 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (91s) 89s ago 14m 11.3M - 1.3.1 1dbe0e931976 ee2cf38a118f 2024-02-15T18:34:42.218 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (6m) 32s ago 14m 309M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc f226bea76cab 2024-02-15T18:34:42.218 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (5m) 32s ago 13m 279M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 5114ed026724 2024-02-15T18:34:42.218 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (4m) 32s ago 13m 190M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 2e26b1ec292a 2024-02-15T18:34:42.218 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (4m) 89s ago 13m 339M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc c71b210091cb 2024-02-15T18:34:42.218 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (3m) 89s ago 12m 285M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 83bd9cdcb3da 2024-02-15T18:34:42.218 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (2m) 89s ago 12m 243M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc d45e566ed288 2024-02-15T18:34:42.218 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (79s) 32s ago 15m 60.1M - 2.33.4 514e6a882f6e b0c2ac6c79f3 2024-02-15T18:34:42.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.227+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/3908041736 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f8aac060f20 msgr2=0x7f8aac0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:42.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.227+0000 7f8ac5bc2700 1 --2- 172.21.15.42:0/3908041736 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f8aac060f20 0x7f8aac0633e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f8ab000b7a0 tx=0x7f8ab0004d10).stop 2024-02-15T18:34:42.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.227+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/3908041736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ac00f7980 msgr2=0x7f8ac0107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:42.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.227+0000 7f8ac5bc2700 1 --2- 172.21.15.42:0/3908041736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ac00f7980 0x7f8ac0107750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f8aa800b410 tx=0x7f8aa8006ef0).stop 2024-02-15T18:34:42.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.227+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/3908041736 shutdown_connections 2024-02-15T18:34:42.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.227+0000 7f8ac5bc2700 1 --2- 172.21.15.42:0/3908041736 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f8aac060f20 0x7f8aac0633e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:42.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.227+0000 7f8ac5bc2700 1 --2- 172.21.15.42:0/3908041736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac00f66e0 0x7f8ac0107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:42.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.227+0000 7f8ac5bc2700 1 --2- 172.21.15.42:0/3908041736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ac00f7980 0x7f8ac0107750 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:42.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.227+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/3908041736 >> 172.21.15.42:0/3908041736 conn(0x7f8ac00f1e40 msgr2=0x7f8ac00fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:42.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.227+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/3908041736 shutdown_connections 2024-02-15T18:34:42.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:42.227+0000 7f8ac5bc2700 1 -- 172.21.15.42:0/3908041736 wait complete. 2024-02-15T18:34:42.868 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph versions' 2024-02-15T18:34:44.102 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:42.217480+0000 mgr.smithi042.kfoihb (mgr.24435) 436 : audit [DBG] from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:44.102 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:42.794394+0000 mgr.smithi042.kfoihb (mgr.24435) 437 : cluster [DBG] pgmap v326: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:44.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:42.217480+0000 mgr.smithi042.kfoihb (mgr.24435) 436 : audit [DBG] from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:34:44.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:42.794394+0000 mgr.smithi042.kfoihb (mgr.24435) 437 : cluster [DBG] pgmap v326: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:44.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.971+0000 7f62c20d8700 1 -- 172.21.15.42:0/805914146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62bc0f66e0 msgr2=0x7f62bc0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:44.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.971+0000 7f62c20d8700 1 --2- 172.21.15.42:0/805914146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62bc0f66e0 0x7f62bc0f6b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f62a4004660 tx=0x7f62a4009b30).stop 2024-02-15T18:34:44.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.971+0000 7f62c20d8700 1 -- 172.21.15.42:0/805914146 shutdown_connections 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.971+0000 7f62c20d8700 1 --2- 172.21.15.42:0/805914146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62bc0f7980 0x7f62bc0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.971+0000 7f62c20d8700 1 --2- 172.21.15.42:0/805914146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62bc0f66e0 0x7f62bc0f6b00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.971+0000 7f62c20d8700 1 -- 172.21.15.42:0/805914146 >> 172.21.15.42:0/805914146 conn(0x7f62bc0f1e40 msgr2=0x7f62bc0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.971+0000 7f62c20d8700 1 -- 172.21.15.42:0/805914146 shutdown_connections 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.971+0000 7f62c20d8700 1 -- 172.21.15.42:0/805914146 wait complete. 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.972+0000 7f62c20d8700 1 Processor -- start 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.972+0000 7f62c20d8700 1 -- start start 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.972+0000 7f62c20d8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62bc0f66e0 0x7f62bc109250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.972+0000 7f62c20d8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62bc0f7980 0x7f62bc109740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.972+0000 7f62c20d8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc109cf0 con 0x7f62bc0f66e0 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.972+0000 7f62c20d8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc109e30 con 0x7f62bc0f7980 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.972+0000 7f62bb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62bc0f66e0 0x7f62bc109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.972+0000 7f62bb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62bc0f66e0 0x7f62bc109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40794/0 (socket says 172.21.15.42:40794) 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.972+0000 7f62bb7fe700 1 -- 172.21.15.42:0/1778406882 learned_addr learned my addr 172.21.15.42:0/1778406882 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.972+0000 7f62baffd700 1 --2- 172.21.15.42:0/1778406882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62bc0f7980 0x7f62bc109740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:44.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.972+0000 7f62bb7fe700 1 -- 172.21.15.42:0/1778406882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62bc0f7980 msgr2=0x7f62bc109740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:44.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.972+0000 7f62bb7fe700 1 --2- 172.21.15.42:0/1778406882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62bc0f7980 0x7f62bc109740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:44.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.972+0000 7f62bb7fe700 1 -- 172.21.15.42:0/1778406882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a4005040 con 0x7f62bc0f66e0 2024-02-15T18:34:44.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.973+0000 7f62bb7fe700 1 --2- 172.21.15.42:0/1778406882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62bc0f66e0 0x7f62bc109250 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f62a4004620 tx=0x7f62a4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:44.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.973+0000 7f62b3fff700 1 -- 172.21.15.42:0/1778406882 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62a4018070 con 0x7f62bc0f66e0 2024-02-15T18:34:44.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.973+0000 7f62b3fff700 1 -- 172.21.15.42:0/1778406882 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f62a4007d70 con 0x7f62bc0f66e0 2024-02-15T18:34:44.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.973+0000 7f62b3fff700 1 -- 172.21.15.42:0/1778406882 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62a401c740 con 0x7f62bc0f66e0 2024-02-15T18:34:44.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.973+0000 7f62c20d8700 1 -- 172.21.15.42:0/1778406882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62bc10a770 con 0x7f62bc0f66e0 2024-02-15T18:34:44.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.973+0000 7f62c20d8700 1 -- 172.21.15.42:0/1778406882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62bc10ac80 con 0x7f62bc0f66e0 2024-02-15T18:34:44.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.974+0000 7f62c20d8700 1 -- 172.21.15.42:0/1778406882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62bc045ad0 con 0x7f62bc0f66e0 2024-02-15T18:34:44.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.974+0000 7f62b3fff700 1 -- 172.21.15.42:0/1778406882 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f62a401db10 con 0x7f62bc0f66e0 2024-02-15T18:34:44.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.975+0000 7f62b3fff700 1 --2- 172.21.15.42:0/1778406882 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f62a8060f20 0x7f62a80633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:34:44.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.975+0000 7f62b3fff700 1 -- 172.21.15.42:0/1778406882 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f62a407e1d0 con 0x7f62bc0f66e0 2024-02-15T18:34:44.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.975+0000 7f62baffd700 1 --2- 172.21.15.42:0/1778406882 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f62a8060f20 0x7f62a80633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:34:44.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.975+0000 7f62baffd700 1 --2- 172.21.15.42:0/1778406882 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f62a8060f20 0x7f62a80633e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f62ac0041d0 tx=0x7f62ac009040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:34:44.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:44.979+0000 7f62b3fff700 1 -- 172.21.15.42:0/1778406882 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f62a4051eb0 con 0x7f62bc0f66e0 2024-02-15T18:34:45.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:45.222+0000 7f62c20d8700 1 -- 172.21.15.42:0/1778406882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f62bc0ff0c0 con 0x7f62bc0f66e0 2024-02-15T18:34:45.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:45.223+0000 7f62b3fff700 1 -- 172.21.15.42:0/1778406882 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+648 (secure 0 0 0) 0x7f62a4051aa0 con 0x7f62bc0f66e0 2024-02-15T18:34:45.214 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 6 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 14 2024-02-15T18:34:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:34:45.216 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:34:45.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:45.225+0000 7f62c20d8700 1 -- 172.21.15.42:0/1778406882 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f62a8060f20 msgr2=0x7f62a80633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:45.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:45.225+0000 7f62c20d8700 1 --2- 172.21.15.42:0/1778406882 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f62a8060f20 0x7f62a80633e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f62ac0041d0 tx=0x7f62ac009040).stop 2024-02-15T18:34:45.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:45.225+0000 7f62c20d8700 1 -- 172.21.15.42:0/1778406882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62bc0f66e0 msgr2=0x7f62bc109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:34:45.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:45.225+0000 7f62c20d8700 1 --2- 172.21.15.42:0/1778406882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62bc0f66e0 0x7f62bc109250 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f62a4004620 tx=0x7f62a4009b30).stop 2024-02-15T18:34:45.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:45.225+0000 7f62c20d8700 1 -- 172.21.15.42:0/1778406882 shutdown_connections 2024-02-15T18:34:45.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:45.225+0000 7f62c20d8700 1 --2- 172.21.15.42:0/1778406882 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f62a8060f20 0x7f62a80633e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:45.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:45.225+0000 7f62c20d8700 1 --2- 172.21.15.42:0/1778406882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62bc0f66e0 0x7f62bc109250 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:45.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:45.225+0000 7f62c20d8700 1 --2- 172.21.15.42:0/1778406882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62bc0f7980 0x7f62bc109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:34:45.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:45.225+0000 7f62c20d8700 1 -- 172.21.15.42:0/1778406882 >> 172.21.15.42:0/1778406882 conn(0x7f62bc0f1e40 msgr2=0x7f62bc06a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:34:45.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:45.225+0000 7f62c20d8700 1 -- 172.21.15.42:0/1778406882 shutdown_connections 2024-02-15T18:34:45.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:34:45.225+0000 7f62c20d8700 1 -- 172.21.15.42:0/1778406882 wait complete. 2024-02-15T18:34:45.911 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-02-15T18:34:46.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:44.794769+0000 mgr.smithi042.kfoihb (mgr.24435) 438 : cluster [DBG] pgmap v327: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:46.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:34:45.224042+0000 mon.smithi042 (mon.0) 766 : audit [DBG] from='client.? 172.21.15.42:0/1778406882' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:46.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:44.794769+0000 mgr.smithi042.kfoihb (mgr.24435) 438 : cluster [DBG] pgmap v327: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:46.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:34:45.224042+0000 mon.smithi042 (mon.0) 766 : audit [DBG] from='client.? 172.21.15.42:0/1778406882' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:34:47.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:46.795431+0000 mgr.smithi042.kfoihb (mgr.24435) 439 : cluster [DBG] pgmap v328: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:47.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:46.795431+0000 mgr.smithi042.kfoihb (mgr.24435) 439 : cluster [DBG] pgmap v328: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:47.954 INFO:teuthology.orchestra.run.smithi042.stdout:wait for servicemap items w/ changing names to refresh 2024-02-15T18:34:48.680 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'sleep 60' 2024-02-15T18:34:50.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:48.795787+0000 mgr.smithi042.kfoihb (mgr.24435) 440 : cluster [DBG] pgmap v329: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:50.230 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:48.795787+0000 mgr.smithi042.kfoihb (mgr.24435) 440 : cluster [DBG] pgmap v329: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:52.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:50.796360+0000 mgr.smithi042.kfoihb (mgr.24435) 441 : cluster [DBG] pgmap v330: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:52.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:50.796360+0000 mgr.smithi042.kfoihb (mgr.24435) 441 : cluster [DBG] pgmap v330: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:54.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:52.796871+0000 mgr.smithi042.kfoihb (mgr.24435) 442 : cluster [DBG] pgmap v331: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:54.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:52.796871+0000 mgr.smithi042.kfoihb (mgr.24435) 442 : cluster [DBG] pgmap v331: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:56.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:54.797408+0000 mgr.smithi042.kfoihb (mgr.24435) 443 : cluster [DBG] pgmap v332: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:56.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:54.797408+0000 mgr.smithi042.kfoihb (mgr.24435) 443 : cluster [DBG] pgmap v332: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:34:57.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:56.798010+0000 mgr.smithi042.kfoihb (mgr.24435) 444 : cluster [DBG] pgmap v333: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:34:57.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:56.798010+0000 mgr.smithi042.kfoihb (mgr.24435) 444 : cluster [DBG] pgmap v333: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:00.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:34:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:34:58.798683+0000 mgr.smithi042.kfoihb (mgr.24435) 445 : cluster [DBG] pgmap v334: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:00.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:34:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:34:58.798683+0000 mgr.smithi042.kfoihb (mgr.24435) 445 : cluster [DBG] pgmap v334: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:02.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:00.799273+0000 mgr.smithi042.kfoihb (mgr.24435) 446 : cluster [DBG] pgmap v335: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:02.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:00.799273+0000 mgr.smithi042.kfoihb (mgr.24435) 446 : cluster [DBG] pgmap v335: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:04.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:02.799754+0000 mgr.smithi042.kfoihb (mgr.24435) 447 : cluster [DBG] pgmap v336: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:04.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:02.799754+0000 mgr.smithi042.kfoihb (mgr.24435) 447 : cluster [DBG] pgmap v336: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:06.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:04.800407+0000 mgr.smithi042.kfoihb (mgr.24435) 448 : cluster [DBG] pgmap v337: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:06.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:04.800407+0000 mgr.smithi042.kfoihb (mgr.24435) 448 : cluster [DBG] pgmap v337: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:07.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:06.800944+0000 mgr.smithi042.kfoihb (mgr.24435) 449 : cluster [DBG] pgmap v338: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:07.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:06.800944+0000 mgr.smithi042.kfoihb (mgr.24435) 449 : cluster [DBG] pgmap v338: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:10.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:08.801525+0000 mgr.smithi042.kfoihb (mgr.24435) 450 : cluster [DBG] pgmap v339: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:10.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:35:09.755714+0000 mon.smithi042 (mon.0) 767 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:35:10.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:08.801525+0000 mgr.smithi042.kfoihb (mgr.24435) 450 : cluster [DBG] pgmap v339: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:10.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:35:09.755714+0000 mon.smithi042 (mon.0) 767 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:35:11.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:35:10.073615+0000 mon.smithi042 (mon.0) 768 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:35:11.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:35:10.074305+0000 mon.smithi042 (mon.0) 769 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:35:11.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:35:10.078275+0000 mon.smithi042 (mon.0) 770 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:35:11.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:35:10.073615+0000 mon.smithi042 (mon.0) 768 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:35:11.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:35:10.074305+0000 mon.smithi042 (mon.0) 769 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:35:11.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:35:10.078275+0000 mon.smithi042 (mon.0) 770 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:35:12.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:35:10.074841+0000 mgr.smithi042.kfoihb (mgr.24435) 451 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:35:12.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:10.802094+0000 mgr.smithi042.kfoihb (mgr.24435) 452 : cluster [DBG] pgmap v340: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:12.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:35:10.074841+0000 mgr.smithi042.kfoihb (mgr.24435) 451 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:35:12.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:10.802094+0000 mgr.smithi042.kfoihb (mgr.24435) 452 : cluster [DBG] pgmap v340: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:14.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:12.802619+0000 mgr.smithi042.kfoihb (mgr.24435) 453 : cluster [DBG] pgmap v341: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:14.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:12.802619+0000 mgr.smithi042.kfoihb (mgr.24435) 453 : cluster [DBG] pgmap v341: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:16.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:14.803192+0000 mgr.smithi042.kfoihb (mgr.24435) 454 : cluster [DBG] pgmap v342: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:16.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:15 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:14.803192+0000 mgr.smithi042.kfoihb (mgr.24435) 454 : cluster [DBG] pgmap v342: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:17.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:16.803916+0000 mgr.smithi042.kfoihb (mgr.24435) 455 : cluster [DBG] pgmap v343: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:17.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:17 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:16.803916+0000 mgr.smithi042.kfoihb (mgr.24435) 455 : cluster [DBG] pgmap v343: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:20.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:19 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:18.804450+0000 mgr.smithi042.kfoihb (mgr.24435) 456 : cluster [DBG] pgmap v344: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:20.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:19 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:18.804450+0000 mgr.smithi042.kfoihb (mgr.24435) 456 : cluster [DBG] pgmap v344: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:22.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:21 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:20.805123+0000 mgr.smithi042.kfoihb (mgr.24435) 457 : cluster [DBG] pgmap v345: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:22.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:21 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:20.805123+0000 mgr.smithi042.kfoihb (mgr.24435) 457 : cluster [DBG] pgmap v345: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:24.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:23 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:22.805551+0000 mgr.smithi042.kfoihb (mgr.24435) 458 : cluster [DBG] pgmap v346: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:24.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:23 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:22.805551+0000 mgr.smithi042.kfoihb (mgr.24435) 458 : cluster [DBG] pgmap v346: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:26.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:25 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:24.806094+0000 mgr.smithi042.kfoihb (mgr.24435) 459 : cluster [DBG] pgmap v347: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:26.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:24.806094+0000 mgr.smithi042.kfoihb (mgr.24435) 459 : cluster [DBG] pgmap v347: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:27.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:27 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:26.806634+0000 mgr.smithi042.kfoihb (mgr.24435) 460 : cluster [DBG] pgmap v348: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:27.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:27 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:26.806634+0000 mgr.smithi042.kfoihb (mgr.24435) 460 : cluster [DBG] pgmap v348: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:30.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:29 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:28.807165+0000 mgr.smithi042.kfoihb (mgr.24435) 461 : cluster [DBG] pgmap v349: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:30.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:29 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:28.807165+0000 mgr.smithi042.kfoihb (mgr.24435) 461 : cluster [DBG] pgmap v349: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:32.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:31 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:30.807764+0000 mgr.smithi042.kfoihb (mgr.24435) 462 : cluster [DBG] pgmap v350: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:32.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:31 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:30.807764+0000 mgr.smithi042.kfoihb (mgr.24435) 462 : cluster [DBG] pgmap v350: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:34.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:33 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:32.808411+0000 mgr.smithi042.kfoihb (mgr.24435) 463 : cluster [DBG] pgmap v351: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:34.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:32.808411+0000 mgr.smithi042.kfoihb (mgr.24435) 463 : cluster [DBG] pgmap v351: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:36.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:35 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:34.808778+0000 mgr.smithi042.kfoihb (mgr.24435) 464 : cluster [DBG] pgmap v352: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:36.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:35 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:34.808778+0000 mgr.smithi042.kfoihb (mgr.24435) 464 : cluster [DBG] pgmap v352: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:37.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:37 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:36.809344+0000 mgr.smithi042.kfoihb (mgr.24435) 465 : cluster [DBG] pgmap v353: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:37.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:37 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:36.809344+0000 mgr.smithi042.kfoihb (mgr.24435) 465 : cluster [DBG] pgmap v353: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:40.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:39 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:38.809907+0000 mgr.smithi042.kfoihb (mgr.24435) 466 : cluster [DBG] pgmap v354: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:40.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:39 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:38.809907+0000 mgr.smithi042.kfoihb (mgr.24435) 466 : cluster [DBG] pgmap v354: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:42.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:41 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:40.810585+0000 mgr.smithi042.kfoihb (mgr.24435) 467 : cluster [DBG] pgmap v355: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:42.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:41 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:40.810585+0000 mgr.smithi042.kfoihb (mgr.24435) 467 : cluster [DBG] pgmap v355: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:44.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:42.811192+0000 mgr.smithi042.kfoihb (mgr.24435) 468 : cluster [DBG] pgmap v356: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:44.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:43 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:42.811192+0000 mgr.smithi042.kfoihb (mgr.24435) 468 : cluster [DBG] pgmap v356: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:46.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:44.811598+0000 mgr.smithi042.kfoihb (mgr.24435) 469 : cluster [DBG] pgmap v357: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:46.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:45 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:44.811598+0000 mgr.smithi042.kfoihb (mgr.24435) 469 : cluster [DBG] pgmap v357: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:47.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:47 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:46.812130+0000 mgr.smithi042.kfoihb (mgr.24435) 470 : cluster [DBG] pgmap v358: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:47.440 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:46.812130+0000 mgr.smithi042.kfoihb (mgr.24435) 470 : cluster [DBG] pgmap v358: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:50.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:48.812598+0000 mgr.smithi042.kfoihb (mgr.24435) 471 : cluster [DBG] pgmap v359: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:50.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:49 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:48.812598+0000 mgr.smithi042.kfoihb (mgr.24435) 471 : cluster [DBG] pgmap v359: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:51.501 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph orch ps' 2024-02-15T18:35:52.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:50.813083+0000 mgr.smithi042.kfoihb (mgr.24435) 472 : cluster [DBG] pgmap v360: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:52.244 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:51 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:50.813083+0000 mgr.smithi042.kfoihb (mgr.24435) 472 : cluster [DBG] pgmap v360: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.514+0000 7fef8f0eb700 1 -- 172.21.15.42:0/2730129674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef880f6860 msgr2=0x7fef880f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.514+0000 7fef8f0eb700 1 --2- 172.21.15.42:0/2730129674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef880f6860 0x7fef880f6cc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fef74004660 tx=0x7fef74009b30).stop 2024-02-15T18:35:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.514+0000 7fef8f0eb700 1 -- 172.21.15.42:0/2730129674 shutdown_connections 2024-02-15T18:35:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.514+0000 7fef8f0eb700 1 --2- 172.21.15.42:0/2730129674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef880f6860 0x7fef880f6cc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.514+0000 7fef8f0eb700 1 --2- 172.21.15.42:0/2730129674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef880f55c0 0x7fef880f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:53.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.514+0000 7fef8f0eb700 1 -- 172.21.15.42:0/2730129674 >> 172.21.15.42:0/2730129674 conn(0x7fef880f0d40 msgr2=0x7fef880f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.514+0000 7fef8f0eb700 1 -- 172.21.15.42:0/2730129674 shutdown_connections 2024-02-15T18:35:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.514+0000 7fef8f0eb700 1 -- 172.21.15.42:0/2730129674 wait complete. 2024-02-15T18:35:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.515+0000 7fef8f0eb700 1 Processor -- start 2024-02-15T18:35:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.515+0000 7fef8f0eb700 1 -- start start 2024-02-15T18:35:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.515+0000 7fef8f0eb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef880f55c0 0x7fef88105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.515+0000 7fef8f0eb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef880f6860 0x7fef881063a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.515+0000 7fef8f0eb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef88106950 con 0x7fef880f55c0 2024-02-15T18:35:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.515+0000 7fef8f0eb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef88106a90 con 0x7fef880f6860 2024-02-15T18:35:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.515+0000 7fef8ce87700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef880f55c0 0x7fef88105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.515+0000 7fef8ce87700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef880f55c0 0x7fef88105eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50568/0 (socket says 172.21.15.42:50568) 2024-02-15T18:35:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.515+0000 7fef8ce87700 1 -- 172.21.15.42:0/4154456889 learned_addr learned my addr 172.21.15.42:0/4154456889 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:35:53.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.516+0000 7fef87fff700 1 --2- 172.21.15.42:0/4154456889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef880f6860 0x7fef881063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:53.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.516+0000 7fef8ce87700 1 -- 172.21.15.42:0/4154456889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef880f6860 msgr2=0x7fef881063a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:53.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.516+0000 7fef8ce87700 1 --2- 172.21.15.42:0/4154456889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef880f6860 0x7fef881063a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:53.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.516+0000 7fef8ce87700 1 -- 172.21.15.42:0/4154456889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef74005040 con 0x7fef880f55c0 2024-02-15T18:35:53.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.516+0000 7fef87fff700 1 --2- 172.21.15.42:0/4154456889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef880f6860 0x7fef881063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:35:53.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.516+0000 7fef8ce87700 1 --2- 172.21.15.42:0/4154456889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef880f55c0 0x7fef88105eb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fef7c002700 tx=0x7fef7c009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:53.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.516+0000 7fef857fa700 1 -- 172.21.15.42:0/4154456889 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef7c009320 con 0x7fef880f55c0 2024-02-15T18:35:53.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.516+0000 7fef857fa700 1 -- 172.21.15.42:0/4154456889 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fef7c00f040 con 0x7fef880f55c0 2024-02-15T18:35:53.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.516+0000 7fef8f0eb700 1 -- 172.21.15.42:0/4154456889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef88107430 con 0x7fef880f55c0 2024-02-15T18:35:53.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.516+0000 7fef857fa700 1 -- 172.21.15.42:0/4154456889 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef7c0149d0 con 0x7fef880f55c0 2024-02-15T18:35:53.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.516+0000 7fef8f0eb700 1 -- 172.21.15.42:0/4154456889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef881079f0 con 0x7fef880f55c0 2024-02-15T18:35:53.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.517+0000 7fef8f0eb700 1 -- 172.21.15.42:0/4154456889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef88045ad0 con 0x7fef880f55c0 2024-02-15T18:35:53.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.518+0000 7fef857fa700 1 -- 172.21.15.42:0/4154456889 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fef7c0077b0 con 0x7fef880f55c0 2024-02-15T18:35:53.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.518+0000 7fef857fa700 1 --2- 172.21.15.42:0/4154456889 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fef78060f20 0x7fef780633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:53.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.518+0000 7fef857fa700 1 -- 172.21.15.42:0/4154456889 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fef7c07c4f0 con 0x7fef880f55c0 2024-02-15T18:35:53.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.518+0000 7fef87fff700 1 --2- 172.21.15.42:0/4154456889 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fef78060f20 0x7fef780633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:53.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.518+0000 7fef87fff700 1 --2- 172.21.15.42:0/4154456889 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fef78060f20 0x7fef780633e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fef74005010 tx=0x7fef74009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:53.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.522+0000 7fef857fa700 1 -- 172.21.15.42:0/4154456889 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fef7c050230 con 0x7fef880f55c0 2024-02-15T18:35:53.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.704+0000 7fef8f0eb700 1 -- 172.21.15.42:0/4154456889 --> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fef88107c70 con 0x7fef78060f20 2024-02-15T18:35:53.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.714+0000 7fef857fa700 1 -- 172.21.15.42:0/4154456889 <== mgr.24435 v2:172.21.15.42:6800/3393955655 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7fef88107c70 con 0x7fef78060f20 2024-02-15T18:35:53.705 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-15T18:35:53.705 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (2m) 104s ago 17m 13.9M - 0.23.0 ba2b418f427c d6474309eb10 2024-02-15T18:35:53.705 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (8m) 104s ago 17m 6643k - 16.2.14-521-g0e714d9a 1fbc92c859dc 2da9d875539a 2024-02-15T18:35:53.705 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi191 smithi191 running (8m) 2m ago 16m 6639k - 16.2.14-521-g0e714d9a 1fbc92c859dc 5c46a68ff62b 2024-02-15T18:35:53.705 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (109s) 104s ago 17m 42.2M - 8.3.5 dad864ee21e9 4fbf722d87dc 2024-02-15T18:35:53.705 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.idnxkc smithi042 running (3m) 104s ago 12m 82.4M - 16.2.14-521-g0e714d9a 1fbc92c859dc 39e47ffbe32b 2024-02-15T18:35:53.705 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.vtwvlo smithi042 running (3m) 104s ago 12m 80.5M - 16.2.14-521-g0e714d9a 1fbc92c859dc 372b8d41fced 2024-02-15T18:35:53.705 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.erlpxp smithi191 running (3m) 2m ago 12m 11.5M - 16.2.14-521-g0e714d9a 1fbc92c859dc 4eb6a75798d1 2024-02-15T18:35:53.705 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi191.icwcnb smithi191 running (3m) 2m ago 12m 12.3M - 16.2.14-521-g0e714d9a 1fbc92c859dc 551cd0ff7860 2024-02-15T18:35:53.705 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.kfoihb smithi042 *:8443,9283 running (9m) 104s ago 18m 444M - 16.2.14-521-g0e714d9a 1fbc92c859dc b9376e735fce 2024-02-15T18:35:53.706 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi191.gftdzo smithi191 *:8443,9283 running (9m) 2m ago 16m 390M - 16.2.14-521-g0e714d9a 1fbc92c859dc 9e42e9d0ba58 2024-02-15T18:35:53.706 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (9m) 104s ago 19m 57.9M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc ff538d2bc6d9 2024-02-15T18:35:53.706 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi191 smithi191 running (8m) 2m ago 16m 56.0M 2048M 16.2.14-521-g0e714d9a 1fbc92c859dc 7abbf59144ba 2024-02-15T18:35:53.706 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (2m) 104s ago 16m 14.2M - 1.3.1 1dbe0e931976 e80f2a2363ff 2024-02-15T18:35:53.706 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi191 smithi191 *:9100 running (2m) 2m ago 16m 11.3M - 1.3.1 1dbe0e931976 ee2cf38a118f 2024-02-15T18:35:53.706 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (7m) 104s ago 15m 309M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc f226bea76cab 2024-02-15T18:35:53.706 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (6m) 104s ago 15m 279M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 5114ed026724 2024-02-15T18:35:53.706 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (5m) 104s ago 14m 190M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 2e26b1ec292a 2024-02-15T18:35:53.706 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi191 running (5m) 2m ago 14m 339M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc c71b210091cb 2024-02-15T18:35:53.706 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi191 running (4m) 2m ago 14m 285M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc 83bd9cdcb3da 2024-02-15T18:35:53.706 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi191 running (3m) 2m ago 13m 243M 4096M 16.2.14-521-g0e714d9a 1fbc92c859dc d45e566ed288 2024-02-15T18:35:53.706 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (2m) 104s ago 16m 60.1M - 2.33.4 514e6a882f6e b0c2ac6c79f3 2024-02-15T18:35:53.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.715+0000 7fef8f0eb700 1 -- 172.21.15.42:0/4154456889 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fef78060f20 msgr2=0x7fef780633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:53.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.715+0000 7fef8f0eb700 1 --2- 172.21.15.42:0/4154456889 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fef78060f20 0x7fef780633e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fef74005010 tx=0x7fef74009b30).stop 2024-02-15T18:35:53.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.715+0000 7fef8f0eb700 1 -- 172.21.15.42:0/4154456889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef880f55c0 msgr2=0x7fef88105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:53.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.715+0000 7fef8f0eb700 1 --2- 172.21.15.42:0/4154456889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef880f55c0 0x7fef88105eb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fef7c002700 tx=0x7fef7c009d80).stop 2024-02-15T18:35:53.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.715+0000 7fef8f0eb700 1 -- 172.21.15.42:0/4154456889 shutdown_connections 2024-02-15T18:35:53.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.715+0000 7fef8f0eb700 1 --2- 172.21.15.42:0/4154456889 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fef78060f20 0x7fef780633e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:53.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.715+0000 7fef8f0eb700 1 --2- 172.21.15.42:0/4154456889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef880f55c0 0x7fef88105eb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:53.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.715+0000 7fef8f0eb700 1 --2- 172.21.15.42:0/4154456889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef880f6860 0x7fef881063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:53.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.715+0000 7fef8f0eb700 1 -- 172.21.15.42:0/4154456889 >> 172.21.15.42:0/4154456889 conn(0x7fef880f0d40 msgr2=0x7fef880fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:53.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.716+0000 7fef8f0eb700 1 -- 172.21.15.42:0/4154456889 shutdown_connections 2024-02-15T18:35:53.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:53.716+0000 7fef8f0eb700 1 -- 172.21.15.42:0/4154456889 wait complete. 2024-02-15T18:35:54.051 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:53 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:52.813840+0000 mgr.smithi042.kfoihb (mgr.24435) 473 : cluster [DBG] pgmap v361: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:54.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:52.813840+0000 mgr.smithi042.kfoihb (mgr.24435) 473 : cluster [DBG] pgmap v361: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:54.400 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph versions' 2024-02-15T18:35:55.111 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:54 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:35:53.706113+0000 mgr.smithi042.kfoihb (mgr.24435) 474 : audit [DBG] from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:55.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:35:53.706113+0000 mgr.smithi042.kfoihb (mgr.24435) 474 : audit [DBG] from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-15T18:35:56.104 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:55 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:54.814327+0000 mgr.smithi042.kfoihb (mgr.24435) 475 : cluster [DBG] pgmap v362: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:56.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:54.814327+0000 mgr.smithi042.kfoihb (mgr.24435) 475 : cluster [DBG] pgmap v362: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:35:56.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.363+0000 7f00a2b06700 1 -- 172.21.15.42:0/540212403 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f009c0f7980 msgr2=0x7f009c0f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:56.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.363+0000 7f00a2b06700 1 --2- 172.21.15.42:0/540212403 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f009c0f7980 0x7f009c0f7de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f0088004660 tx=0x7f0088009b30).stop 2024-02-15T18:35:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.363+0000 7f00a2b06700 1 -- 172.21.15.42:0/540212403 shutdown_connections 2024-02-15T18:35:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.363+0000 7f00a2b06700 1 --2- 172.21.15.42:0/540212403 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f009c0f7980 0x7f009c0f7de0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.363+0000 7f00a2b06700 1 --2- 172.21.15.42:0/540212403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f009c0f66e0 0x7f009c0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:35:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.363+0000 7f00a2b06700 1 -- 172.21.15.42:0/540212403 >> 172.21.15.42:0/540212403 conn(0x7f009c0f1e40 msgr2=0x7f009c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.363+0000 7f00a2b06700 1 -- 172.21.15.42:0/540212403 shutdown_connections 2024-02-15T18:35:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.363+0000 7f00a2b06700 1 -- 172.21.15.42:0/540212403 wait complete. 2024-02-15T18:35:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.364+0000 7f00a2b06700 1 Processor -- start 2024-02-15T18:35:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.364+0000 7f00a2b06700 1 -- start start 2024-02-15T18:35:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.364+0000 7f00a2b06700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f009c0f66e0 0x7f009c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.364+0000 7f00a2b06700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f009c0f7980 0x7f009c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.364+0000 7f00a2b06700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c06a410 con 0x7f009c0f7980 2024-02-15T18:35:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.364+0000 7f00a2b06700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c06a550 con 0x7f009c0f66e0 2024-02-15T18:35:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.364+0000 7f009bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f009c0f7980 0x7f009c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:56.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.364+0000 7f009bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f009c0f7980 0x7f009c069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50580/0 (socket says 172.21.15.42:50580) 2024-02-15T18:35:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.364+0000 7f009bfff700 1 -- 172.21.15.42:0/620485768 learned_addr learned my addr 172.21.15.42:0/620485768 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:35:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.365+0000 7f009bfff700 1 -- 172.21.15.42:0/620485768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f009c0f66e0 msgr2=0x7f009c06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.365+0000 7f009bfff700 1 --2- 172.21.15.42:0/620485768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f009c0f66e0 0x7f009c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:35:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.365+0000 7f009bfff700 1 -- 172.21.15.42:0/620485768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0088005040 con 0x7f009c0f7980 2024-02-15T18:35:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.365+0000 7f009bfff700 1 --2- 172.21.15.42:0/620485768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f009c0f7980 0x7f009c069e20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f0088009f50 tx=0x7f0088009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.365+0000 7f00997fa700 1 -- 172.21.15.42:0/620485768 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0088018070 con 0x7f009c0f7980 2024-02-15T18:35:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.365+0000 7f00997fa700 1 -- 172.21.15.42:0/620485768 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f0088007d70 con 0x7f009c0f7980 2024-02-15T18:35:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.365+0000 7f00997fa700 1 -- 172.21.15.42:0/620485768 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f008801c620 con 0x7f009c0f7980 2024-02-15T18:35:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.365+0000 7f00a2b06700 1 -- 172.21.15.42:0/620485768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009c06a7d0 con 0x7f009c0f7980 2024-02-15T18:35:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.365+0000 7f00a2b06700 1 -- 172.21.15.42:0/620485768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009c10f080 con 0x7f009c0f7980 2024-02-15T18:35:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.366+0000 7f00a2b06700 1 -- 172.21.15.42:0/620485768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009c0f5ad0 con 0x7f009c0f7980 2024-02-15T18:35:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.366+0000 7f00997fa700 1 -- 172.21.15.42:0/620485768 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f008801c8e0 con 0x7f009c0f7980 2024-02-15T18:35:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.366+0000 7f00997fa700 1 --2- 172.21.15.42:0/620485768 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f008c060f20 0x7f008c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:56.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.367+0000 7f00997fa700 1 -- 172.21.15.42:0/620485768 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f008807e2a0 con 0x7f009c0f7980 2024-02-15T18:35:56.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.367+0000 7f00a08a2700 1 --2- 172.21.15.42:0/620485768 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f008c060f20 0x7f008c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:56.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.367+0000 7f00a08a2700 1 --2- 172.21.15.42:0/620485768 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f008c060f20 0x7f008c0633e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f009c06b730 tx=0x7f0090006ab0).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:56.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.371+0000 7f00997fa700 1 -- 172.21.15.42:0/620485768 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f008804e4f0 con 0x7f009c0f7980 2024-02-15T18:35:56.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.614+0000 7f00a2b06700 1 -- 172.21.15.42:0/620485768 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f009c0ff050 con 0x7f009c0f7980 2024-02-15T18:35:56.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.615+0000 7f00997fa700 1 -- 172.21.15.42:0/620485768 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+648 (secure 0 0 0) 0x7f0088051b10 con 0x7f009c0f7980 2024-02-15T18:35:56.605 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 2 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 6 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 4 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)": 14 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-02-15T18:35:56.606 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-02-15T18:35:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.616+0000 7f00a2b06700 1 -- 172.21.15.42:0/620485768 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f008c060f20 msgr2=0x7f008c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.616+0000 7f00a2b06700 1 --2- 172.21.15.42:0/620485768 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f008c060f20 0x7f008c0633e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f009c06b730 tx=0x7f0090006ab0).stop 2024-02-15T18:35:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.616+0000 7f00a2b06700 1 -- 172.21.15.42:0/620485768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f009c0f7980 msgr2=0x7f009c069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.616+0000 7f00a2b06700 1 --2- 172.21.15.42:0/620485768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f009c0f7980 0x7f009c069e20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f0088009f50 tx=0x7f0088009b30).stop 2024-02-15T18:35:56.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.616+0000 7f00a2b06700 1 -- 172.21.15.42:0/620485768 shutdown_connections 2024-02-15T18:35:56.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.616+0000 7f00a2b06700 1 --2- 172.21.15.42:0/620485768 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f008c060f20 0x7f008c0633e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:56.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.616+0000 7f00a2b06700 1 --2- 172.21.15.42:0/620485768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f009c0f66e0 0x7f009c06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:35:56.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.616+0000 7f00a2b06700 1 --2- 172.21.15.42:0/620485768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f009c0f7980 0x7f009c069e20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:56.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.616+0000 7f00a2b06700 1 -- 172.21.15.42:0/620485768 >> 172.21.15.42:0/620485768 conn(0x7f009c0f1e40 msgr2=0x7f009c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:56.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.616+0000 7f00a2b06700 1 -- 172.21.15.42:0/620485768 shutdown_connections 2024-02-15T18:35:56.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:56.616+0000 7f00a2b06700 1 -- 172.21.15.42:0/620485768 wait complete. 2024-02-15T18:35:57.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:35:56.615733+0000 mon.smithi042 (mon.0) 771 : audit [DBG] from='client.? 172.21.15.42:0/620485768' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:35:57.230 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:57 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:56.814753+0000 mgr.smithi042.kfoihb (mgr.24435) 476 : cluster [DBG] pgmap v363: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:57.251 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-02-15T18:35:57.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:35:56.615733+0000 mon.smithi042 (mon.0) 771 : audit [DBG] from='client.? 172.21.15.42:0/620485768' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:35:57.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:56.814753+0000 mgr.smithi042.kfoihb (mgr.24435) 476 : cluster [DBG] pgmap v363: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:35:59.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.256+0000 7fe852dc5700 1 -- 172.21.15.42:0/1592341505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84c06a350 msgr2=0x7fe84c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:59.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.256+0000 7fe852dc5700 1 --2- 172.21.15.42:0/1592341505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84c06a350 0x7fe84c06a7d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fe838004660 tx=0x7fe838009b30).stop 2024-02-15T18:35:59.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.256+0000 7fe852dc5700 1 -- 172.21.15.42:0/1592341505 shutdown_connections 2024-02-15T18:35:59.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.256+0000 7fe852dc5700 1 --2- 172.21.15.42:0/1592341505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84c06a350 0x7fe84c06a7d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:59.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.256+0000 7fe852dc5700 1 --2- 172.21.15.42:0/1592341505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c06b960 0x7fe84c069e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:59.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.256+0000 7fe852dc5700 1 -- 172.21.15.42:0/1592341505 >> 172.21.15.42:0/1592341505 conn(0x7fe84c0f1e40 msgr2=0x7fe84c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:59.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.256+0000 7fe852dc5700 1 -- 172.21.15.42:0/1592341505 shutdown_connections 2024-02-15T18:35:59.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.257+0000 7fe852dc5700 1 -- 172.21.15.42:0/1592341505 wait complete. 2024-02-15T18:35:59.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.257+0000 7fe852dc5700 1 Processor -- start 2024-02-15T18:35:59.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.257+0000 7fe852dc5700 1 -- start start 2024-02-15T18:35:59.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.257+0000 7fe852dc5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c06a350 0x7fe84c1091c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:59.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.257+0000 7fe852dc5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84c06b960 0x7fe84c1096b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:59.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.257+0000 7fe852dc5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c109c60 con 0x7fe84c06b960 2024-02-15T18:35:59.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.257+0000 7fe852dc5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c109da0 con 0x7fe84c06a350 2024-02-15T18:35:59.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.257+0000 7fe84bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84c06b960 0x7fe84c1096b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:59.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.258+0000 7fe84bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84c06b960 0x7fe84c1096b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50584/0 (socket says 172.21.15.42:50584) 2024-02-15T18:35:59.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.258+0000 7fe84bfff700 1 -- 172.21.15.42:0/2902484369 learned_addr learned my addr 172.21.15.42:0/2902484369 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:35:59.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.258+0000 7fe850b61700 1 --2- 172.21.15.42:0/2902484369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c06a350 0x7fe84c1091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:59.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.258+0000 7fe84bfff700 1 -- 172.21.15.42:0/2902484369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c06a350 msgr2=0x7fe84c1091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:59.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.258+0000 7fe84bfff700 1 --2- 172.21.15.42:0/2902484369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c06a350 0x7fe84c1091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:59.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.258+0000 7fe84bfff700 1 -- 172.21.15.42:0/2902484369 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe838005040 con 0x7fe84c06b960 2024-02-15T18:35:59.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.258+0000 7fe84bfff700 1 --2- 172.21.15.42:0/2902484369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84c06b960 0x7fe84c1096b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fe838009f50 tx=0x7fe838009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:59.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.258+0000 7fe8497fa700 1 -- 172.21.15.42:0/2902484369 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe838018070 con 0x7fe84c06b960 2024-02-15T18:35:59.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.258+0000 7fe8497fa700 1 -- 172.21.15.42:0/2902484369 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fe838007ba0 con 0x7fe84c06b960 2024-02-15T18:35:59.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.258+0000 7fe852dc5700 1 -- 172.21.15.42:0/2902484369 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c10a6e0 con 0x7fe84c06b960 2024-02-15T18:35:59.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.258+0000 7fe8497fa700 1 -- 172.21.15.42:0/2902484369 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe83801c420 con 0x7fe84c06b960 2024-02-15T18:35:59.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.258+0000 7fe852dc5700 1 -- 172.21.15.42:0/2902484369 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84c10ac40 con 0x7fe84c06b960 2024-02-15T18:35:59.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.259+0000 7fe852dc5700 1 -- 172.21.15.42:0/2902484369 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84c045ad0 con 0x7fe84c06b960 2024-02-15T18:35:59.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.260+0000 7fe8497fa700 1 -- 172.21.15.42:0/2902484369 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fe83801d420 con 0x7fe84c06b960 2024-02-15T18:35:59.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.260+0000 7fe8497fa700 1 --2- 172.21.15.42:0/2902484369 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe83c060f20 0x7fe83c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:35:59.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.260+0000 7fe8497fa700 1 -- 172.21.15.42:0/2902484369 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fe83807df30 con 0x7fe84c06b960 2024-02-15T18:35:59.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.260+0000 7fe850b61700 1 --2- 172.21.15.42:0/2902484369 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe83c060f20 0x7fe83c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:35:59.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.261+0000 7fe850b61700 1 --2- 172.21.15.42:0/2902484369 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe83c060f20 0x7fe83c0633e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fe840004c40 tx=0x7fe84000b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:35:59.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.264+0000 7fe8497fa700 1 -- 172.21.15.42:0/2902484369 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe8380acf60 con 0x7fe84c06b960 2024-02-15T18:35:59.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.507+0000 7fe852dc5700 1 -- 172.21.15.42:0/2902484369 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe84c027a00 con 0x7fe84c06b960 2024-02-15T18:35:59.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.508+0000 7fe8497fa700 1 -- 172.21.15.42:0/2902484369 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+648 (secure 0 0 0) 0x7fe838051c10 con 0x7fe84c06b960 2024-02-15T18:35:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.510+0000 7fe852dc5700 1 -- 172.21.15.42:0/2902484369 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe83c060f20 msgr2=0x7fe83c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.510+0000 7fe852dc5700 1 --2- 172.21.15.42:0/2902484369 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe83c060f20 0x7fe83c0633e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fe840004c40 tx=0x7fe84000b040).stop 2024-02-15T18:35:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.510+0000 7fe852dc5700 1 -- 172.21.15.42:0/2902484369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84c06b960 msgr2=0x7fe84c1096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:35:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.510+0000 7fe852dc5700 1 --2- 172.21.15.42:0/2902484369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84c06b960 0x7fe84c1096b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fe838009f50 tx=0x7fe838009b30).stop 2024-02-15T18:35:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.510+0000 7fe852dc5700 1 -- 172.21.15.42:0/2902484369 shutdown_connections 2024-02-15T18:35:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.510+0000 7fe852dc5700 1 --2- 172.21.15.42:0/2902484369 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe83c060f20 0x7fe83c0633e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.510+0000 7fe852dc5700 1 --2- 172.21.15.42:0/2902484369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe84c06a350 0x7fe84c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.510+0000 7fe852dc5700 1 --2- 172.21.15.42:0/2902484369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe84c06b960 0x7fe84c1096b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:35:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.510+0000 7fe852dc5700 1 -- 172.21.15.42:0/2902484369 >> 172.21.15.42:0/2902484369 conn(0x7fe84c0f1e40 msgr2=0x7fe84c0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:35:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.510+0000 7fe852dc5700 1 -- 172.21.15.42:0/2902484369 shutdown_connections 2024-02-15T18:35:59.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:35:59.510+0000 7fe852dc5700 1 -- 172.21.15.42:0/2902484369 wait complete. 2024-02-15T18:35:59.511 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-02-15T18:36:00.144 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:35:58.815300+0000 mgr.smithi042.kfoihb (mgr.24435) 477 : cluster [DBG] pgmap v364: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:36:00.144 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:35:59 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:35:59.509103+0000 mon.smithi042 (mon.0) 772 : audit [DBG] from='client.? 172.21.15.42:0/2902484369' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:00.167 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -e sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-02-15T18:36:00.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:35:58.815300+0000 mgr.smithi042.kfoihb (mgr.24435) 477 : cluster [DBG] pgmap v364: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:36:00.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:35:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:35:59.509103+0000 mon.smithi042 (mon.0) 772 : audit [DBG] from='client.? 172.21.15.42:0/2902484369' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:02.054 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:01 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:36:00.815692+0000 mgr.smithi042.kfoihb (mgr.24435) 478 : cluster [DBG] pgmap v365: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:36:02.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.073+0000 7f51cc663700 1 -- 172.21.15.42:0/1917209779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c40f6860 msgr2=0x7f51c40f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:02.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.073+0000 7f51cc663700 1 --2- 172.21.15.42:0/1917209779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c40f6860 0x7f51c40f6cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f51b4004660 tx=0x7f51b4009b30).stop 2024-02-15T18:36:02.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.074+0000 7f51cc663700 1 -- 172.21.15.42:0/1917209779 shutdown_connections 2024-02-15T18:36:02.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.074+0000 7f51cc663700 1 --2- 172.21.15.42:0/1917209779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c40f6860 0x7f51c40f6cc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:02.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.074+0000 7f51cc663700 1 --2- 172.21.15.42:0/1917209779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51c40f55c0 0x7f51c40f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:02.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.074+0000 7f51cc663700 1 -- 172.21.15.42:0/1917209779 >> 172.21.15.42:0/1917209779 conn(0x7f51c40f0d40 msgr2=0x7f51c40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:02.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.074+0000 7f51cc663700 1 -- 172.21.15.42:0/1917209779 shutdown_connections 2024-02-15T18:36:02.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.074+0000 7f51cc663700 1 -- 172.21.15.42:0/1917209779 wait complete. 2024-02-15T18:36:02.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.074+0000 7f51cc663700 1 Processor -- start 2024-02-15T18:36:02.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.074+0000 7f51cc663700 1 -- start start 2024-02-15T18:36:02.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.074+0000 7f51cc663700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c40f55c0 0x7f51c4106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:02.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.074+0000 7f51cc663700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51c40f6860 0x7f51c41074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:02.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.074+0000 7f51cc663700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c4107a60 con 0x7f51c40f55c0 2024-02-15T18:36:02.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.074+0000 7f51cc663700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c4107ba0 con 0x7f51c40f6860 2024-02-15T18:36:02.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.075+0000 7f51ca3ff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c40f55c0 0x7f51c4106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:02.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.075+0000 7f51ca3ff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c40f55c0 0x7f51c4106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39828/0 (socket says 172.21.15.42:39828) 2024-02-15T18:36:02.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.075+0000 7f51ca3ff700 1 -- 172.21.15.42:0/83073614 learned_addr learned my addr 172.21.15.42:0/83073614 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:02.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.075+0000 7f51c9bfe700 1 --2- 172.21.15.42:0/83073614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51c40f6860 0x7f51c41074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:02.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.075+0000 7f51ca3ff700 1 -- 172.21.15.42:0/83073614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51c40f6860 msgr2=0x7f51c41074b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:02.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.075+0000 7f51ca3ff700 1 --2- 172.21.15.42:0/83073614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51c40f6860 0x7f51c41074b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.075+0000 7f51ca3ff700 1 -- 172.21.15.42:0/83073614 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b4005040 con 0x7f51c40f55c0 2024-02-15T18:36:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.075+0000 7f51ca3ff700 1 --2- 172.21.15.42:0/83073614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c40f55c0 0x7f51c4106fc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f51c000c790 tx=0x7f51c0007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.075+0000 7f51baffd700 1 -- 172.21.15.42:0/83073614 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51c000e680 con 0x7f51c40f55c0 2024-02-15T18:36:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.075+0000 7f51baffd700 1 -- 172.21.15.42:0/83073614 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f51c000ecd0 con 0x7f51c40f55c0 2024-02-15T18:36:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.075+0000 7f51cc663700 1 -- 172.21.15.42:0/83073614 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c4108540 con 0x7f51c40f55c0 2024-02-15T18:36:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.075+0000 7f51baffd700 1 -- 172.21.15.42:0/83073614 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51c0012e40 con 0x7f51c40f55c0 2024-02-15T18:36:02.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.075+0000 7f51cc663700 1 -- 172.21.15.42:0/83073614 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c4108b00 con 0x7f51c40f55c0 2024-02-15T18:36:02.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.076+0000 7f51cc663700 1 -- 172.21.15.42:0/83073614 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c4045ad0 con 0x7f51c40f55c0 2024-02-15T18:36:02.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.077+0000 7f51baffd700 1 -- 172.21.15.42:0/83073614 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f51c0010070 con 0x7f51c40f55c0 2024-02-15T18:36:02.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.077+0000 7f51baffd700 1 --2- 172.21.15.42:0/83073614 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f51b0060f20 0x7f51b00633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:02.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.077+0000 7f51baffd700 1 -- 172.21.15.42:0/83073614 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f51c007c2d0 con 0x7f51c40f55c0 2024-02-15T18:36:02.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.077+0000 7f51c9bfe700 1 --2- 172.21.15.42:0/83073614 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f51b0060f20 0x7f51b00633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:02.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.077+0000 7f51c9bfe700 1 --2- 172.21.15.42:0/83073614 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f51b0060f20 0x7f51b00633e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f51b4004e30 tx=0x7f51b4009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:02.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.081+0000 7f51baffd700 1 -- 172.21.15.42:0/83073614 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f51c0050030 con 0x7f51c40f55c0 2024-02-15T18:36:02.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:01 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:36:00.815692+0000 mgr.smithi042.kfoihb (mgr.24435) 478 : cluster [DBG] pgmap v365: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:36:02.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.322+0000 7f51cc663700 1 -- 172.21.15.42:0/83073614 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f51c410d5e0 con 0x7f51c40f55c0 2024-02-15T18:36:02.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.322+0000 7f51baffd700 1 -- 172.21.15.42:0/83073614 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+648 (secure 0 0 0) 0x7f51c004fc20 con 0x7f51c40f55c0 2024-02-15T18:36:02.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.324+0000 7f51cc663700 1 -- 172.21.15.42:0/83073614 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f51b0060f20 msgr2=0x7f51b00633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:02.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.324+0000 7f51cc663700 1 --2- 172.21.15.42:0/83073614 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f51b0060f20 0x7f51b00633e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f51b4004e30 tx=0x7f51b4009b30).stop 2024-02-15T18:36:02.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.324+0000 7f51cc663700 1 -- 172.21.15.42:0/83073614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c40f55c0 msgr2=0x7f51c4106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:02.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.324+0000 7f51cc663700 1 --2- 172.21.15.42:0/83073614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c40f55c0 0x7f51c4106fc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f51c000c790 tx=0x7f51c0007d90).stop 2024-02-15T18:36:02.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.324+0000 7f51cc663700 1 -- 172.21.15.42:0/83073614 shutdown_connections 2024-02-15T18:36:02.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.324+0000 7f51cc663700 1 --2- 172.21.15.42:0/83073614 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f51b0060f20 0x7f51b00633e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:02.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.324+0000 7f51cc663700 1 --2- 172.21.15.42:0/83073614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c40f55c0 0x7f51c4106fc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:02.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.324+0000 7f51cc663700 1 --2- 172.21.15.42:0/83073614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51c40f6860 0x7f51c41074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:02.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.324+0000 7f51cc663700 1 -- 172.21.15.42:0/83073614 >> 172.21.15.42:0/83073614 conn(0x7f51c40f0d40 msgr2=0x7f51c405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:02.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.324+0000 7f51cc663700 1 -- 172.21.15.42:0/83073614 shutdown_connections 2024-02-15T18:36:02.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:02.324+0000 7f51cc663700 1 -- 172.21.15.42:0/83073614 wait complete. 2024-02-15T18:36:02.325 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable)" 2024-02-15T18:36:02.942 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:02 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:02.323502+0000 mon.smithi042 (mon.0) 773 : audit [DBG] from='client.? 172.21.15.42:0/83073614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:02.962 DEBUG:teuthology.parallel:result is None 2024-02-15T18:36:02.963 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-15T18:36:02.975 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-02-15T18:36:02.975 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-15T18:36:03.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:02 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:02.323502+0000 mon.smithi042 (mon.0) 773 : audit [DBG] from='client.? 172.21.15.42:0/83073614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-15T18:36:04.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:03 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:36:02.816241+0000 mgr.smithi042.kfoihb (mgr.24435) 479 : cluster [DBG] pgmap v366: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:36:04.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:03 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:36:02.816241+0000 mgr.smithi042.kfoihb (mgr.24435) 479 : cluster [DBG] pgmap v366: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:36:05.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.124+0000 7fe77586c700 1 -- 172.21.15.42:0/1815701609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7700f6ff0 msgr2=0x7fe7700f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:05.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.124+0000 7fe77586c700 1 --2- 172.21.15.42:0/1815701609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7700f6ff0 0x7fe7700f7a20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fe758004660 tx=0x7fe758009b30).stop 2024-02-15T18:36:05.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.124+0000 7fe77586c700 1 -- 172.21.15.42:0/1815701609 shutdown_connections 2024-02-15T18:36:05.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.124+0000 7fe77586c700 1 --2- 172.21.15.42:0/1815701609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7700f6ff0 0x7fe7700f7a20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:05.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.124+0000 7fe77586c700 1 --2- 172.21.15.42:0/1815701609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7700f66e0 0x7fe7700f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:05.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.124+0000 7fe77586c700 1 -- 172.21.15.42:0/1815701609 >> 172.21.15.42:0/1815701609 conn(0x7fe7700f1f00 msgr2=0x7fe7700f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:05.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.124+0000 7fe77586c700 1 -- 172.21.15.42:0/1815701609 shutdown_connections 2024-02-15T18:36:05.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.124+0000 7fe77586c700 1 -- 172.21.15.42:0/1815701609 wait complete. 2024-02-15T18:36:05.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.125+0000 7fe77586c700 1 Processor -- start 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.125+0000 7fe77586c700 1 -- start start 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.125+0000 7fe77586c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7700f66e0 0x7fe7701091e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.125+0000 7fe77586c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7700f6ff0 0x7fe7701096d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.125+0000 7fe77586c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe770109c80 con 0x7fe7700f66e0 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.125+0000 7fe77586c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe770109dc0 con 0x7fe7700f6ff0 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.126+0000 7fe76effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7700f66e0 0x7fe7701091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.126+0000 7fe76effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7700f66e0 0x7fe7701091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39856/0 (socket says 172.21.15.42:39856) 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.126+0000 7fe76effd700 1 -- 172.21.15.42:0/3795193310 learned_addr learned my addr 172.21.15.42:0/3795193310 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.126+0000 7fe76e7fc700 1 --2- 172.21.15.42:0/3795193310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7700f6ff0 0x7fe7701096d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.126+0000 7fe76effd700 1 -- 172.21.15.42:0/3795193310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7700f6ff0 msgr2=0x7fe7701096d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.126+0000 7fe76effd700 1 --2- 172.21.15.42:0/3795193310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7700f6ff0 0x7fe7701096d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.126+0000 7fe76effd700 1 -- 172.21.15.42:0/3795193310 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe758005040 con 0x7fe7700f66e0 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.126+0000 7fe76effd700 1 --2- 172.21.15.42:0/3795193310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7700f66e0 0x7fe7701091e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fe76000e7f0 tx=0x7fe76000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.126+0000 7fe7677fe700 1 -- 172.21.15.42:0/3795193310 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7600092e0 con 0x7fe7700f66e0 2024-02-15T18:36:05.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.126+0000 7fe77586c700 1 -- 172.21.15.42:0/3795193310 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77010a760 con 0x7fe7700f66e0 2024-02-15T18:36:05.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.126+0000 7fe7677fe700 1 -- 172.21.15.42:0/3795193310 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fe76000f040 con 0x7fe7700f66e0 2024-02-15T18:36:05.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.126+0000 7fe7677fe700 1 -- 172.21.15.42:0/3795193310 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe760014850 con 0x7fe7700f66e0 2024-02-15T18:36:05.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.126+0000 7fe77586c700 1 -- 172.21.15.42:0/3795193310 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77010ad20 con 0x7fe7700f66e0 2024-02-15T18:36:05.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.127+0000 7fe77586c700 1 -- 172.21.15.42:0/3795193310 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe770045ad0 con 0x7fe7700f66e0 2024-02-15T18:36:05.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.128+0000 7fe7677fe700 1 -- 172.21.15.42:0/3795193310 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fe76001b030 con 0x7fe7700f66e0 2024-02-15T18:36:05.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.128+0000 7fe7677fe700 1 --2- 172.21.15.42:0/3795193310 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe75c060f20 0x7fe75c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:05.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.128+0000 7fe7677fe700 1 -- 172.21.15.42:0/3795193310 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fe76007bff0 con 0x7fe7700f66e0 2024-02-15T18:36:05.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.128+0000 7fe76e7fc700 1 --2- 172.21.15.42:0/3795193310 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe75c060f20 0x7fe75c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:05.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.129+0000 7fe76e7fc700 1 --2- 172.21.15.42:0/3795193310 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe75c060f20 0x7fe75c0633e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fe758009720 tx=0x7fe758016040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:05.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.132+0000 7fe7677fe700 1 -- 172.21.15.42:0/3795193310 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe76004fde0 con 0x7fe7700f66e0 2024-02-15T18:36:05.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.353+0000 7fe77586c700 1 -- 172.21.15.42:0/3795193310 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe7700ff0f0 con 0x7fe7700f66e0 2024-02-15T18:36:05.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.353+0000 7fe7677fe700 1 -- 172.21.15.42:0/3795193310 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1774 (secure 0 0 0) 0x7fe760019020 con 0x7fe7700f66e0 2024-02-15T18:36:05.344 INFO:teuthology.orchestra.run.smithi042.stdout:e28 2024-02-15T18:36:05.345 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-15T18:36:05.345 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:36:05.345 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-02-15T18:36:05.345 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:05.345 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-02-15T18:36:05.345 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-02-15T18:36:05.345 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 28 2024-02-15T18:36:05.345 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 2024-02-15T18:36:05.345 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-02-15T18:23:40.469506+0000 2024-02-15T18:36:05.345 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-02-15T18:32:55.244270+0000 2024-02-15T18:36:05.345 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-02-15T18:36:05.345 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-02-15T18:36:05.346 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-02-15T18:36:05.346 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-02-15T18:36:05.346 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-02-15T18:36:05.346 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-02-15T18:36:05.346 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-02-15T18:36:05.346 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 113 2024-02-15T18:36:05.346 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-15T18:36:05.346 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-02-15T18:36:05.346 INFO:teuthology.orchestra.run.smithi042.stdout:in 0 2024-02-15T18:36:05.346 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=34356} 2024-02-15T18:36:05.346 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-02-15T18:36:05.346 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-02-15T18:36:05.346 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-02-15T18:36:05.346 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-02-15T18:36:05.347 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-02-15T18:36:05.347 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-02-15T18:36:05.347 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-02-15T18:36:05.347 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-02-15T18:36:05.347 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.vtwvlo{0:34356} state up:active seq 12 join_fscid=1 addr [v2:172.21.15.42:6826/2962563293,v1:172.21.15.42:6827/2962563293] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:36:05.347 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.idnxkc{0:34360} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/1562993914,v1:172.21.15.42:6829/1562993914] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:36:05.347 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:05.347 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:05.347 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-02-15T18:36:05.347 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:05.347 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.icwcnb{-1:44343} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.191:6824/540280968,v1:172.21.15.191:6825/540280968] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:36:05.347 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi191.erlpxp{-1:44347} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.191:6826/289198038,v1:172.21.15.191:6827/289198038] compat {c=[1],r=[1],i=[7ff]}] 2024-02-15T18:36:05.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.355+0000 7fe77586c700 1 -- 172.21.15.42:0/3795193310 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe75c060f20 msgr2=0x7fe75c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:05.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.355+0000 7fe77586c700 1 --2- 172.21.15.42:0/3795193310 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe75c060f20 0x7fe75c0633e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fe758009720 tx=0x7fe758016040).stop 2024-02-15T18:36:05.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.355+0000 7fe77586c700 1 -- 172.21.15.42:0/3795193310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7700f66e0 msgr2=0x7fe7701091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:05.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.355+0000 7fe77586c700 1 --2- 172.21.15.42:0/3795193310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7700f66e0 0x7fe7701091e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fe76000e7f0 tx=0x7fe76000b040).stop 2024-02-15T18:36:05.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.355+0000 7fe77586c700 1 -- 172.21.15.42:0/3795193310 shutdown_connections 2024-02-15T18:36:05.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.355+0000 7fe77586c700 1 --2- 172.21.15.42:0/3795193310 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe75c060f20 0x7fe75c0633e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:05.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.355+0000 7fe77586c700 1 --2- 172.21.15.42:0/3795193310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7700f66e0 0x7fe7701091e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:05.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.355+0000 7fe77586c700 1 --2- 172.21.15.42:0/3795193310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7700f6ff0 0x7fe7701096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:05.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.355+0000 7fe77586c700 1 -- 172.21.15.42:0/3795193310 >> 172.21.15.42:0/3795193310 conn(0x7fe7700f1f00 msgr2=0x7fe77006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:05.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.355+0000 7fe77586c700 1 -- 172.21.15.42:0/3795193310 shutdown_connections 2024-02-15T18:36:05.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:05.355+0000 7fe77586c700 1 -- 172.21.15.42:0/3795193310 wait complete. 2024-02-15T18:36:05.349 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 28 2024-02-15T18:36:05.909 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-02-15T18:36:05.920 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-15T18:36:05.946 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:36:04.816580+0000 mgr.smithi042.kfoihb (mgr.24435) 480 : cluster [DBG] pgmap v367: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:36:05.946 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:05 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:05.354618+0000 mon.smithi042 (mon.0) 774 : audit [DBG] from='client.? 172.21.15.42:0/3795193310' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:36:06.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.052+0000 7f4b05bb4700 1 -- 172.21.15.42:0/888714974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b000f5cc0 msgr2=0x7f4b000f66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:06.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.052+0000 7f4b05bb4700 1 --2- 172.21.15.42:0/888714974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b000f5cc0 0x7f4b000f66f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f4ae8004660 tx=0x7f4ae8009b30).stop 2024-02-15T18:36:06.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.052+0000 7f4b05bb4700 1 -- 172.21.15.42:0/888714974 shutdown_connections 2024-02-15T18:36:06.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.052+0000 7f4b05bb4700 1 --2- 172.21.15.42:0/888714974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b000f5cc0 0x7f4b000f66f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.053+0000 7f4b05bb4700 1 --2- 172.21.15.42:0/888714974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b000f53b0 0x7f4b000f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.053+0000 7f4b05bb4700 1 -- 172.21.15.42:0/888714974 >> 172.21.15.42:0/888714974 conn(0x7f4b000f0bd0 msgr2=0x7f4b000f2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:06.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.053+0000 7f4b05bb4700 1 -- 172.21.15.42:0/888714974 shutdown_connections 2024-02-15T18:36:06.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.053+0000 7f4b05bb4700 1 -- 172.21.15.42:0/888714974 wait complete. 2024-02-15T18:36:06.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.053+0000 7f4b05bb4700 1 Processor -- start 2024-02-15T18:36:06.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.053+0000 7f4b05bb4700 1 -- start start 2024-02-15T18:36:06.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.053+0000 7f4b05bb4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b000f53b0 0x7f4b00105b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:06.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.053+0000 7f4b05bb4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b000f5cc0 0x7f4b00106010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:06.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.053+0000 7f4b05bb4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b001065c0 con 0x7f4b000f5cc0 2024-02-15T18:36:06.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.053+0000 7f4b05bb4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00106700 con 0x7f4b000f53b0 2024-02-15T18:36:06.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.054+0000 7f4af6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b000f5cc0 0x7f4b00106010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:06.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.054+0000 7f4af6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b000f5cc0 0x7f4b00106010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39868/0 (socket says 172.21.15.42:39868) 2024-02-15T18:36:06.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.054+0000 7f4af6ffd700 1 -- 172.21.15.42:0/1102121379 learned_addr learned my addr 172.21.15.42:0/1102121379 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:06.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.054+0000 7f4aff7fe700 1 --2- 172.21.15.42:0/1102121379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b000f53b0 0x7f4b00105b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:06.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.054+0000 7f4af6ffd700 1 -- 172.21.15.42:0/1102121379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b000f53b0 msgr2=0x7f4b00105b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:06.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.054+0000 7f4af6ffd700 1 --2- 172.21.15.42:0/1102121379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b000f53b0 0x7f4b00105b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.054+0000 7f4af6ffd700 1 -- 172.21.15.42:0/1102121379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae8005040 con 0x7f4b000f5cc0 2024-02-15T18:36:06.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.054+0000 7f4af6ffd700 1 --2- 172.21.15.42:0/1102121379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b000f5cc0 0x7f4b00106010 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f4ae8009f50 tx=0x7f4ae8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:06.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.054+0000 7f4afcff9700 1 -- 172.21.15.42:0/1102121379 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ae8018070 con 0x7f4b000f5cc0 2024-02-15T18:36:06.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.054+0000 7f4afcff9700 1 -- 172.21.15.42:0/1102121379 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f4ae8007c70 con 0x7f4b000f5cc0 2024-02-15T18:36:06.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.054+0000 7f4b05bb4700 1 -- 172.21.15.42:0/1102121379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0010b140 con 0x7f4b000f5cc0 2024-02-15T18:36:06.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.054+0000 7f4afcff9700 1 -- 172.21.15.42:0/1102121379 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ae801c620 con 0x7f4b000f5cc0 2024-02-15T18:36:06.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.054+0000 7f4b05bb4700 1 -- 172.21.15.42:0/1102121379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0010b700 con 0x7f4b000f5cc0 2024-02-15T18:36:06.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.055+0000 7f4b05bb4700 1 -- 172.21.15.42:0/1102121379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b00027a00 con 0x7f4b000f5cc0 2024-02-15T18:36:06.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.056+0000 7f4afcff9700 1 -- 172.21.15.42:0/1102121379 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f4ae801d420 con 0x7f4b000f5cc0 2024-02-15T18:36:06.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.056+0000 7f4afcff9700 1 --2- 172.21.15.42:0/1102121379 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4aec060fe0 0x7f4aec0634a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:06.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.056+0000 7f4afcff9700 1 -- 172.21.15.42:0/1102121379 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f4ae807e1e0 con 0x7f4b000f5cc0 2024-02-15T18:36:06.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.056+0000 7f4aff7fe700 1 --2- 172.21.15.42:0/1102121379 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4aec060fe0 0x7f4aec0634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:06.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.056+0000 7f4aff7fe700 1 --2- 172.21.15.42:0/1102121379 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4aec060fe0 0x7f4aec0634a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f4af0007fd0 tx=0x7f4af0009040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:06.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.060+0000 7f4afcff9700 1 -- 172.21.15.42:0/1102121379 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4ae8051f20 con 0x7f4b000f5cc0 2024-02-15T18:36:06.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:36:04.816580+0000 mgr.smithi042.kfoihb (mgr.24435) 480 : cluster [DBG] pgmap v367: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:36:06.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:05 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:05.354618+0000 mon.smithi042 (mon.0) 774 : audit [DBG] from='client.? 172.21.15.42:0/3795193310' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-15T18:36:06.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.234+0000 7f4b05bb4700 1 -- 172.21.15.42:0/1102121379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b000fdcb0 con 0x7f4b000f5cc0 2024-02-15T18:36:06.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.235+0000 7f4afcff9700 1 -- 172.21.15.42:0/1102121379 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f4ae8051b10 con 0x7f4b000f5cc0 2024-02-15T18:36:06.226 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:06.226 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":114,"fsid":"51458a44-cc2e-11ee-95ba-87774f69a715","created":"2024-02-15T18:16:53.406642+0000","modified":"2024-02-15T18:32:47.968424+0000","last_up_change":"2024-02-15T18:32:05.244472+0000","last_in_change":"2024-02-15T18:21:58.584131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-15T18:21:26.655770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-15T18:23:39.196684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-15T18:23:39.900559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42281c08-af07-43a5-bcfd-d476bc745782","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":110,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4030668481},{"type":"v1","addr":"172.21.15.42:6803","nonce":4030668481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4030668481},{"type":"v1","addr":"172.21.15.42:6805","nonce":4030668481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4030668481},{"type":"v1","addr":"172.21.15.42:6809","nonce":4030668481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4030668481},{"type":"v1","addr":"172.21.15.42:6807","nonce":4030668481}]},"public_addr":"172.21.15.42:6803/4030668481","cluster_addr":"172.21.15.42:6805/4030668481","heartbeat_back_addr":"172.21.15.42:6809/4030668481","heartbeat_front_addr":"172.21.15.42:6807/4030668481","state":["exists","up"]},{"osd":1,"uuid":"8f67d329-7d44-4361-be02-f8b56b3e5713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":65,"up_from":71,"up_thru":110,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":4294036630},{"type":"v1","addr":"172.21.15.42:6811","nonce":4294036630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":4294036630},{"type":"v1","addr":"172.21.15.42:6813","nonce":4294036630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":4294036630},{"type":"v1","addr":"172.21.15.42:6817","nonce":4294036630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":4294036630},{"type":"v1","addr":"172.21.15.42:6815","nonce":4294036630}]},"public_addr":"172.21.15.42:6811/4294036630","cluster_addr":"172.21.15.42:6813/4294036630","heartbeat_back_addr":"172.21.15.42:6817/4294036630","heartbeat_front_addr":"172.21.15.42:6815/4294036630","state":["exists","up"]},{"osd":2,"uuid":"680f6437-b450-4353-a761-b50d1388bcd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":76,"up_from":80,"up_thru":110,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":1789911666},{"type":"v1","addr":"172.21.15.42:6819","nonce":1789911666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":1789911666},{"type":"v1","addr":"172.21.15.42:6821","nonce":1789911666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":1789911666},{"type":"v1","addr":"172.21.15.42:6825","nonce":1789911666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":1789911666},{"type":"v1","addr":"172.21.15.42:6823","nonce":1789911666}]},"public_addr":"172.21.15.42:6819/1789911666","cluster_addr":"172.21.15.42:6821/1789911666","heartbeat_back_addr":"172.21.15.42:6825/1789911666","heartbeat_front_addr":"172.21.15.42:6823/1789911666","state":["exists","up"]},{"osd":3,"uuid":"6aeea641-3735-4f11-b95f-40c45d4dcd07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":81,"up_from":85,"up_thru":110,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":4294301607},{"type":"v1","addr":"172.21.15.191:6801","nonce":4294301607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":4294301607},{"type":"v1","addr":"172.21.15.191:6803","nonce":4294301607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":4294301607},{"type":"v1","addr":"172.21.15.191:6807","nonce":4294301607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":4294301607},{"type":"v1","addr":"172.21.15.191:6805","nonce":4294301607}]},"public_addr":"172.21.15.191:6801/4294301607","cluster_addr":"172.21.15.191:6803/4294301607","heartbeat_back_addr":"172.21.15.191:6807/4294301607","heartbeat_front_addr":"172.21.15.191:6805/4294301607","state":["exists","up"]},{"osd":4,"uuid":"7d9839d2-bd60-4bae-a16f-ebf56f204acc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":33,"last_clean_end":86,"up_from":99,"up_thru":110,"down_at":87,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":2781892647},{"type":"v1","addr":"172.21.15.191:6809","nonce":2781892647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":2781892647},{"type":"v1","addr":"172.21.15.191:6811","nonce":2781892647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":2781892647},{"type":"v1","addr":"172.21.15.191:6815","nonce":2781892647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":2781892647},{"type":"v1","addr":"172.21.15.191:6813","nonce":2781892647}]},"public_addr":"172.21.15.191:6809/2781892647","cluster_addr":"172.21.15.191:6811/2781892647","heartbeat_back_addr":"172.21.15.191:6815/2781892647","heartbeat_front_addr":"172.21.15.191:6813/2781892647","state":["exists","up"]},{"osd":5,"uuid":"a75bb280-bd87-4961-918b-18c426f31c97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":38,"last_clean_end":105,"up_from":110,"up_thru":110,"down_at":106,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":4219355336},{"type":"v1","addr":"172.21.15.191:6817","nonce":4219355336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":4219355336},{"type":"v1","addr":"172.21.15.191:6819","nonce":4219355336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":4219355336},{"type":"v1","addr":"172.21.15.191:6823","nonce":4219355336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":4219355336},{"type":"v1","addr":"172.21.15.191:6821","nonce":4219355336}]},"public_addr":"172.21.15.191:6817/4219355336","cluster_addr":"172.21.15.191:6819/4219355336","heartbeat_back_addr":"172.21.15.191:6823/4219355336","heartbeat_front_addr":"172.21.15.191:6821/4219355336","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:20:17.479921+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:20:47.951852+0000","dead_epoch":66},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:11.894830+0000","dead_epoch":77},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:31.013756+0000","dead_epoch":82},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:21:52.493759+0000","dead_epoch":87},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-15T18:22:14.397700+0000","dead_epoch":106}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1","mappings":[{"from":0,"to":4}]},{"pgid":"2.2","mappings":[{"from":0,"to":4}]},{"pgid":"2.8","mappings":[{"from":3,"to":4}]},{"pgid":"2.a","mappings":[{"from":3,"to":2}]},{"pgid":"2.11","mappings":[{"from":3,"to":2}]},{"pgid":"2.19","mappings":[{"from":0,"to":5}]},{"pgid":"3.a","mappings":[{"from":1,"to":0}]},{"pgid":"3.b","mappings":[{"from":1,"to":5}]},{"pgid":"3.11","mappings":[{"from":3,"to":2}]},{"pgid":"3.14","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.191:6825/1971151000":"2024-02-16T18:32:31.190181+0000","172.21.15.42:0/4019907068":"2024-02-16T18:25:30.877399+0000","172.21.15.42:0/1703009783":"2024-02-16T18:17:56.705000+0000","172.21.15.191:0/4208150245":"2024-02-16T18:26:09.671710+0000","172.21.15.42:6801/1691818213":"2024-02-16T18:25:30.877399+0000","172.21.15.191:6824/1971151000":"2024-02-16T18:32:31.190181+0000","172.21.15.42:6800/1691818213":"2024-02-16T18:25:30.877399+0000","172.21.15.42:0/4136985555":"2024-02-16T18:17:40.441172+0000","172.21.15.42:6800/481027993":"2024-02-16T18:17:56.705000+0000","172.21.15.191:6828/2058110468":"2024-02-16T18:26:09.671710+0000","172.21.15.191:0/3734172468":"2024-02-16T18:26:09.671710+0000","172.21.15.42:6800/3051492708":"2024-02-16T18:17:40.441172+0000","172.21.15.191:6829/2058110468":"2024-02-16T18:26:09.671710+0000","172.21.15.42:6800/1746226634":"2024-02-16T18:17:18.618949+0000","172.21.15.42:0/1489061810":"2024-02-16T18:25:30.877399+0000","172.21.15.42:0/1046171692":"2024-02-16T18:17:18.618949+0000","172.21.15.42:6801/3051492708":"2024-02-16T18:17:40.441172+0000","172.21.15.42:0/1629344114":"2024-02-16T18:17:56.705000+0000","172.21.15.191:0/3377990551":"2024-02-16T18:26:09.671710+0000","172.21.15.191:0/3285510517":"2024-02-16T18:26:09.671710+0000","172.21.15.42:6801/1746226634":"2024-02-16T18:17:18.618949+0000","172.21.15.42:6801/481027993":"2024-02-16T18:17:56.705000+0000","172.21.15.42:0/4044100":"2024-02-16T18:17:18.618949+0000","172.21.15.42:0/3157909680":"2024-02-16T18:17:40.441172+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-15T18:36:06.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.236+0000 7f4b05bb4700 1 -- 172.21.15.42:0/1102121379 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4aec060fe0 msgr2=0x7f4aec0634a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:06.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.236+0000 7f4b05bb4700 1 --2- 172.21.15.42:0/1102121379 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4aec060fe0 0x7f4aec0634a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f4af0007fd0 tx=0x7f4af0009040).stop 2024-02-15T18:36:06.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.236+0000 7f4b05bb4700 1 -- 172.21.15.42:0/1102121379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b000f5cc0 msgr2=0x7f4b00106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:06.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.236+0000 7f4b05bb4700 1 --2- 172.21.15.42:0/1102121379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b000f5cc0 0x7f4b00106010 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f4ae8009f50 tx=0x7f4ae8009b30).stop 2024-02-15T18:36:06.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.236+0000 7f4b05bb4700 1 -- 172.21.15.42:0/1102121379 shutdown_connections 2024-02-15T18:36:06.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.236+0000 7f4b05bb4700 1 --2- 172.21.15.42:0/1102121379 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f4aec060fe0 0x7f4aec0634a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.236+0000 7f4b05bb4700 1 --2- 172.21.15.42:0/1102121379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b000f53b0 0x7f4b00105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.236+0000 7f4b05bb4700 1 --2- 172.21.15.42:0/1102121379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b000f5cc0 0x7f4b00106010 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.236+0000 7f4b05bb4700 1 -- 172.21.15.42:0/1102121379 >> 172.21.15.42:0/1102121379 conn(0x7f4b000f0bd0 msgr2=0x7f4b000fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:06.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.237+0000 7f4b05bb4700 1 -- 172.21.15.42:0/1102121379 shutdown_connections 2024-02-15T18:36:06.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.237+0000 7f4b05bb4700 1 -- 172.21.15.42:0/1102121379 wait complete. 2024-02-15T18:36:06.240 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-15T18:21:26.655770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-15T18:23:39.196684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-15T18:23:39.900559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-15T18:36:06.240 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2024-02-15T18:36:06.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.413+0000 7fe8ed02c700 1 -- 172.21.15.42:0/2229747860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e80f5ca0 msgr2=0x7fe8e80f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:06.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.413+0000 7fe8ed02c700 1 --2- 172.21.15.42:0/2229747860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e80f5ca0 0x7fe8e80f66d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fe8d8004660 tx=0x7fe8d8009b30).stop 2024-02-15T18:36:06.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.413+0000 7fe8ed02c700 1 -- 172.21.15.42:0/2229747860 shutdown_connections 2024-02-15T18:36:06.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.413+0000 7fe8ed02c700 1 --2- 172.21.15.42:0/2229747860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e80f5ca0 0x7fe8e80f66d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.413+0000 7fe8ed02c700 1 --2- 172.21.15.42:0/2229747860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8e80f5390 0x7fe8e80f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.413+0000 7fe8ed02c700 1 -- 172.21.15.42:0/2229747860 >> 172.21.15.42:0/2229747860 conn(0x7fe8e80f0b70 msgr2=0x7fe8e80f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.413+0000 7fe8ed02c700 1 -- 172.21.15.42:0/2229747860 shutdown_connections 2024-02-15T18:36:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.413+0000 7fe8ed02c700 1 -- 172.21.15.42:0/2229747860 wait complete. 2024-02-15T18:36:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.414+0000 7fe8ed02c700 1 Processor -- start 2024-02-15T18:36:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.414+0000 7fe8ed02c700 1 -- start start 2024-02-15T18:36:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.414+0000 7fe8ed02c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8e80f5390 0x7fe8e8106ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.414+0000 7fe8ed02c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e80f5ca0 0x7fe8e81071d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.414+0000 7fe8ed02c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81077d0 con 0x7fe8e80f5ca0 2024-02-15T18:36:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.414+0000 7fe8ed02c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e8107910 con 0x7fe8e80f5390 2024-02-15T18:36:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.414+0000 7fe8e659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e80f5ca0 0x7fe8e81071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.414+0000 7fe8e659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e80f5ca0 0x7fe8e81071d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39884/0 (socket says 172.21.15.42:39884) 2024-02-15T18:36:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.414+0000 7fe8e659c700 1 -- 172.21.15.42:0/1268113428 learned_addr learned my addr 172.21.15.42:0/1268113428 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.414+0000 7fe8e6d9d700 1 --2- 172.21.15.42:0/1268113428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8e80f5390 0x7fe8e8106ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.415+0000 7fe8e659c700 1 -- 172.21.15.42:0/1268113428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8e80f5390 msgr2=0x7fe8e8106ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.415+0000 7fe8e659c700 1 --2- 172.21.15.42:0/1268113428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8e80f5390 0x7fe8e8106ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.415+0000 7fe8e659c700 1 -- 172.21.15.42:0/1268113428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d8005040 con 0x7fe8e80f5ca0 2024-02-15T18:36:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.415+0000 7fe8e659c700 1 --2- 172.21.15.42:0/1268113428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e80f5ca0 0x7fe8e81071d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fe8d8009f50 tx=0x7fe8d8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.415+0000 7fe8cf7fe700 1 -- 172.21.15.42:0/1268113428 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8d8018070 con 0x7fe8e80f5ca0 2024-02-15T18:36:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.415+0000 7fe8cf7fe700 1 -- 172.21.15.42:0/1268113428 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fe8d8007ba0 con 0x7fe8e80f5ca0 2024-02-15T18:36:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.415+0000 7fe8ed02c700 1 -- 172.21.15.42:0/1268113428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e810c2c0 con 0x7fe8e80f5ca0 2024-02-15T18:36:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.415+0000 7fe8cf7fe700 1 -- 172.21.15.42:0/1268113428 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8d801c420 con 0x7fe8e80f5ca0 2024-02-15T18:36:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.415+0000 7fe8ed02c700 1 -- 172.21.15.42:0/1268113428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e810c820 con 0x7fe8e80f5ca0 2024-02-15T18:36:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.416+0000 7fe8ed02c700 1 -- 172.21.15.42:0/1268113428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e80fdc90 con 0x7fe8e80f5ca0 2024-02-15T18:36:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.417+0000 7fe8cf7fe700 1 -- 172.21.15.42:0/1268113428 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fe8d801d420 con 0x7fe8e80f5ca0 2024-02-15T18:36:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.417+0000 7fe8cf7fe700 1 --2- 172.21.15.42:0/1268113428 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe8d0060f20 0x7fe8d00633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.417+0000 7fe8cf7fe700 1 -- 172.21.15.42:0/1268113428 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fe8d807de40 con 0x7fe8e80f5ca0 2024-02-15T18:36:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.417+0000 7fe8e6d9d700 1 --2- 172.21.15.42:0/1268113428 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe8d0060f20 0x7fe8d00633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.417+0000 7fe8e6d9d700 1 --2- 172.21.15.42:0/1268113428 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe8d0060f20 0x7fe8d00633e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fe8e81081d0 tx=0x7fe8dc00b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:06.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.421+0000 7fe8cf7fe700 1 -- 172.21.15.42:0/1268113428 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe8d804e170 con 0x7fe8e80f5ca0 2024-02-15T18:36:06.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.592+0000 7fe8ed02c700 1 -- 172.21.15.42:0/1268113428 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe8e810ceb0 con 0x7fe8e80f5ca0 2024-02-15T18:36:06.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.593+0000 7fe8cf7fe700 1 -- 172.21.15.42:0/1268113428 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v114) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe8d8051790 con 0x7fe8e80f5ca0 2024-02-15T18:36:06.584 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-02-15T18:36:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.594+0000 7fe8ed02c700 1 -- 172.21.15.42:0/1268113428 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe8d0060f20 msgr2=0x7fe8d00633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.594+0000 7fe8ed02c700 1 --2- 172.21.15.42:0/1268113428 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe8d0060f20 0x7fe8d00633e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fe8e81081d0 tx=0x7fe8dc00b040).stop 2024-02-15T18:36:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.595+0000 7fe8ed02c700 1 -- 172.21.15.42:0/1268113428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e80f5ca0 msgr2=0x7fe8e81071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.595+0000 7fe8ed02c700 1 --2- 172.21.15.42:0/1268113428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e80f5ca0 0x7fe8e81071d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fe8d8009f50 tx=0x7fe8d8009b30).stop 2024-02-15T18:36:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.595+0000 7fe8ed02c700 1 -- 172.21.15.42:0/1268113428 shutdown_connections 2024-02-15T18:36:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.595+0000 7fe8ed02c700 1 --2- 172.21.15.42:0/1268113428 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fe8d0060f20 0x7fe8d00633e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.595+0000 7fe8ed02c700 1 --2- 172.21.15.42:0/1268113428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8e80f5390 0x7fe8e8106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.595+0000 7fe8ed02c700 1 --2- 172.21.15.42:0/1268113428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8e80f5ca0 0x7fe8e81071d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.595+0000 7fe8ed02c700 1 -- 172.21.15.42:0/1268113428 >> 172.21.15.42:0/1268113428 conn(0x7fe8e80f0b70 msgr2=0x7fe8e805fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.595+0000 7fe8ed02c700 1 -- 172.21.15.42:0/1268113428 shutdown_connections 2024-02-15T18:36:06.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.595+0000 7fe8ed02c700 1 -- 172.21.15.42:0/1268113428 wait complete. 2024-02-15T18:36:06.597 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-15T18:36:06.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.730+0000 7fd74d85c700 1 -- 172.21.15.42:0/3836116364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7480f6490 msgr2=0x7fd7480f68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:06.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.730+0000 7fd74d85c700 1 --2- 172.21.15.42:0/3836116364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7480f6490 0x7fd7480f68b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fd730004660 tx=0x7fd730009b30).stop 2024-02-15T18:36:06.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.730+0000 7fd74d85c700 1 -- 172.21.15.42:0/3836116364 shutdown_connections 2024-02-15T18:36:06.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.730+0000 7fd74d85c700 1 --2- 172.21.15.42:0/3836116364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7480f7730 0x7fd7480f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.730+0000 7fd74d85c700 1 --2- 172.21.15.42:0/3836116364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7480f6490 0x7fd7480f68b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.730+0000 7fd74d85c700 1 -- 172.21.15.42:0/3836116364 >> 172.21.15.42:0/3836116364 conn(0x7fd7480f1bd0 msgr2=0x7fd7480f4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:06.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.730+0000 7fd74d85c700 1 -- 172.21.15.42:0/3836116364 shutdown_connections 2024-02-15T18:36:06.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.730+0000 7fd74d85c700 1 -- 172.21.15.42:0/3836116364 wait complete. 2024-02-15T18:36:06.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.730+0000 7fd74d85c700 1 Processor -- start 2024-02-15T18:36:06.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.730+0000 7fd74d85c700 1 -- start start 2024-02-15T18:36:06.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd74d85c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7480f6490 0x7fd748106d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:06.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd74d85c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7480f7730 0x7fd748107200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:06.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd74d85c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd748107800 con 0x7fd7480f6490 2024-02-15T18:36:06.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd74d85c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd748107940 con 0x7fd7480f7730 2024-02-15T18:36:06.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd746ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7480f6490 0x7fd748106d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:06.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd746ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7480f6490 0x7fd748106d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39892/0 (socket says 172.21.15.42:39892) 2024-02-15T18:36:06.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd746ffd700 1 -- 172.21.15.42:0/4098734775 learned_addr learned my addr 172.21.15.42:0/4098734775 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:06.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd746ffd700 1 -- 172.21.15.42:0/4098734775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7480f7730 msgr2=0x7fd748107200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:06.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd7467fc700 1 --2- 172.21.15.42:0/4098734775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7480f7730 0x7fd748107200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:06.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd746ffd700 1 --2- 172.21.15.42:0/4098734775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7480f7730 0x7fd748107200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd746ffd700 1 -- 172.21.15.42:0/4098734775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd730005040 con 0x7fd7480f6490 2024-02-15T18:36:06.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd746ffd700 1 --2- 172.21.15.42:0/4098734775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7480f6490 0x7fd748106d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fd730013fd0 tx=0x7fd730009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:06.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd73f7fe700 1 -- 172.21.15.42:0/4098734775 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd730018070 con 0x7fd7480f6490 2024-02-15T18:36:06.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd73f7fe700 1 -- 172.21.15.42:0/4098734775 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fd730007d70 con 0x7fd7480f6490 2024-02-15T18:36:06.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd74d85c700 1 -- 172.21.15.42:0/4098734775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74810c2f0 con 0x7fd7480f6490 2024-02-15T18:36:06.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd73f7fe700 1 -- 172.21.15.42:0/4098734775 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd73001c620 con 0x7fd7480f6490 2024-02-15T18:36:06.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.731+0000 7fd74d85c700 1 -- 172.21.15.42:0/4098734775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74810c850 con 0x7fd7480f6490 2024-02-15T18:36:06.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.732+0000 7fd74d85c700 1 -- 172.21.15.42:0/4098734775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd748027a00 con 0x7fd7480f6490 2024-02-15T18:36:06.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.733+0000 7fd73f7fe700 1 -- 172.21.15.42:0/4098734775 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fd73001d420 con 0x7fd7480f6490 2024-02-15T18:36:06.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.733+0000 7fd73f7fe700 1 --2- 172.21.15.42:0/4098734775 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd734060f20 0x7fd7340633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:06.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.733+0000 7fd73f7fe700 1 -- 172.21.15.42:0/4098734775 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fd73007e010 con 0x7fd7480f6490 2024-02-15T18:36:06.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.733+0000 7fd7467fc700 1 --2- 172.21.15.42:0/4098734775 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd734060f20 0x7fd7340633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:06.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.733+0000 7fd7467fc700 1 --2- 172.21.15.42:0/4098734775 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd734060f20 0x7fd7340633e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fd7380049e0 tx=0x7fd738009040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:06.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.737+0000 7fd73f7fe700 1 -- 172.21.15.42:0/4098734775 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd73004e410 con 0x7fd7480f6490 2024-02-15T18:36:06.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.909+0000 7fd74d85c700 1 -- 172.21.15.42:0/4098734775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fd7480fed60 con 0x7fd7480f6490 2024-02-15T18:36:06.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.909+0000 7fd73f7fe700 1 -- 172.21.15.42:0/4098734775 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v114) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd730051a30 con 0x7fd7480f6490 2024-02-15T18:36:06.901 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-02-15T18:36:06.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.911+0000 7fd74d85c700 1 -- 172.21.15.42:0/4098734775 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd734060f20 msgr2=0x7fd7340633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.911+0000 7fd74d85c700 1 --2- 172.21.15.42:0/4098734775 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd734060f20 0x7fd7340633e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fd7380049e0 tx=0x7fd738009040).stop 2024-02-15T18:36:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.911+0000 7fd74d85c700 1 -- 172.21.15.42:0/4098734775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7480f6490 msgr2=0x7fd748106d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.911+0000 7fd74d85c700 1 --2- 172.21.15.42:0/4098734775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7480f6490 0x7fd748106d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7fd730013fd0 tx=0x7fd730009b30).stop 2024-02-15T18:36:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.911+0000 7fd74d85c700 1 -- 172.21.15.42:0/4098734775 shutdown_connections 2024-02-15T18:36:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.911+0000 7fd74d85c700 1 --2- 172.21.15.42:0/4098734775 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd734060f20 0x7fd7340633e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.911+0000 7fd74d85c700 1 --2- 172.21.15.42:0/4098734775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7480f6490 0x7fd748106d10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.911+0000 7fd74d85c700 1 --2- 172.21.15.42:0/4098734775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7480f7730 0x7fd748107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.911+0000 7fd74d85c700 1 -- 172.21.15.42:0/4098734775 >> 172.21.15.42:0/4098734775 conn(0x7fd7480f1bd0 msgr2=0x7fd7480fb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.912+0000 7fd74d85c700 1 -- 172.21.15.42:0/4098734775 shutdown_connections 2024-02-15T18:36:06.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:06.912+0000 7fd74d85c700 1 -- 172.21.15.42:0/4098734775 wait complete. 2024-02-15T18:36:06.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-15T18:36:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.047+0000 7f56a24a3700 1 -- 172.21.15.42:0/826607146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c0f9660 msgr2=0x7f569c0fba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.047+0000 7f56a24a3700 1 --2- 172.21.15.42:0/826607146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c0f9660 0x7f569c0fba50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f5684004660 tx=0x7f5684009b30).stop 2024-02-15T18:36:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.047+0000 7f56a24a3700 1 -- 172.21.15.42:0/826607146 shutdown_connections 2024-02-15T18:36:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.047+0000 7f56a24a3700 1 --2- 172.21.15.42:0/826607146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c0f9660 0x7f569c0fba50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.047+0000 7f56a24a3700 1 --2- 172.21.15.42:0/826607146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f569c05fdd0 0x7f569c0f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.047+0000 7f56a24a3700 1 -- 172.21.15.42:0/826607146 >> 172.21.15.42:0/826607146 conn(0x7f569c0f0b70 msgr2=0x7f569c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.047+0000 7f56a24a3700 1 -- 172.21.15.42:0/826607146 shutdown_connections 2024-02-15T18:36:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.047+0000 7f56a24a3700 1 -- 172.21.15.42:0/826607146 wait complete. 2024-02-15T18:36:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.047+0000 7f56a24a3700 1 Processor -- start 2024-02-15T18:36:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.048+0000 7f56a24a3700 1 -- start start 2024-02-15T18:36:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.048+0000 7f56a24a3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f569c05fdd0 0x7f569c0f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:07.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.048+0000 7f56a24a3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c0f9660 0x7f569c0f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.048+0000 7f56a24a3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0f6fc0 con 0x7f569c0f9660 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.048+0000 7f56a24a3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0f5560 con 0x7f569c05fdd0 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.048+0000 7f569b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c0f9660 0x7f569c0f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.048+0000 7f569b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c0f9660 0x7f569c0f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39914/0 (socket says 172.21.15.42:39914) 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.048+0000 7f569b7fe700 1 -- 172.21.15.42:0/2322716107 learned_addr learned my addr 172.21.15.42:0/2322716107 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.048+0000 7f569bfff700 1 --2- 172.21.15.42:0/2322716107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f569c05fdd0 0x7f569c0f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.048+0000 7f569b7fe700 1 -- 172.21.15.42:0/2322716107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f569c05fdd0 msgr2=0x7f569c0f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.048+0000 7f569b7fe700 1 --2- 172.21.15.42:0/2322716107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f569c05fdd0 0x7f569c0f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.048+0000 7f569b7fe700 1 -- 172.21.15.42:0/2322716107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5684005040 con 0x7f569c0f9660 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.048+0000 7f569b7fe700 1 --2- 172.21.15.42:0/2322716107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c0f9660 0x7f569c0f5070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f5684013fd0 tx=0x7f5684009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.049+0000 7f5698ff9700 1 -- 172.21.15.42:0/2322716107 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5684018070 con 0x7f569c0f9660 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.049+0000 7f5698ff9700 1 -- 172.21.15.42:0/2322716107 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f56840097d0 con 0x7f569c0f9660 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.049+0000 7f56a24a3700 1 -- 172.21.15.42:0/2322716107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c0f5780 con 0x7f569c0f9660 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.049+0000 7f5698ff9700 1 -- 172.21.15.42:0/2322716107 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f568401c3c0 con 0x7f569c0f9660 2024-02-15T18:36:07.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.049+0000 7f56a24a3700 1 -- 172.21.15.42:0/2322716107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c0f5c60 con 0x7f569c0f9660 2024-02-15T18:36:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.050+0000 7f56a24a3700 1 -- 172.21.15.42:0/2322716107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569c046330 con 0x7f569c0f9660 2024-02-15T18:36:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.050+0000 7f5698ff9700 1 -- 172.21.15.42:0/2322716107 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f568401d420 con 0x7f569c0f9660 2024-02-15T18:36:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.051+0000 7f5698ff9700 1 --2- 172.21.15.42:0/2322716107 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f5688060f20 0x7f56880633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.051+0000 7f5698ff9700 1 -- 172.21.15.42:0/2322716107 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f568407de00 con 0x7f569c0f9660 2024-02-15T18:36:07.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.051+0000 7f569bfff700 1 --2- 172.21.15.42:0/2322716107 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f5688060f20 0x7f56880633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:07.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.051+0000 7f569bfff700 1 --2- 172.21.15.42:0/2322716107 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f5688060f20 0x7f56880633e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f569c05fba0 tx=0x7f568c00b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.057+0000 7f5698ff9700 1 -- 172.21.15.42:0/2322716107 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5684051bc0 con 0x7f569c0f9660 2024-02-15T18:36:07.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.227+0000 7f56a24a3700 1 -- 172.21.15.42:0/2322716107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f569c027a00 con 0x7f569c0f9660 2024-02-15T18:36:07.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.227+0000 7f5698ff9700 1 -- 172.21.15.42:0/2322716107 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v114) v1 ==== 107+0+11 (secure 0 0 0) 0x7f56840517b0 con 0x7f569c0f9660 2024-02-15T18:36:07.218 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-02-15T18:36:07.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.229+0000 7f56a24a3700 1 -- 172.21.15.42:0/2322716107 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f5688060f20 msgr2=0x7f56880633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:07.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.229+0000 7f56a24a3700 1 --2- 172.21.15.42:0/2322716107 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f5688060f20 0x7f56880633e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f569c05fba0 tx=0x7f568c00b040).stop 2024-02-15T18:36:07.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.229+0000 7f56a24a3700 1 -- 172.21.15.42:0/2322716107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c0f9660 msgr2=0x7f569c0f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:07.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.229+0000 7f56a24a3700 1 --2- 172.21.15.42:0/2322716107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c0f9660 0x7f569c0f5070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f5684013fd0 tx=0x7f5684009b30).stop 2024-02-15T18:36:07.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.229+0000 7f56a24a3700 1 -- 172.21.15.42:0/2322716107 shutdown_connections 2024-02-15T18:36:07.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.229+0000 7f56a24a3700 1 --2- 172.21.15.42:0/2322716107 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f5688060f20 0x7f56880633e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.229+0000 7f56a24a3700 1 --2- 172.21.15.42:0/2322716107 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f569c05fdd0 0x7f569c0f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.229+0000 7f56a24a3700 1 --2- 172.21.15.42:0/2322716107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f569c0f9660 0x7f569c0f5070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.229+0000 7f56a24a3700 1 -- 172.21.15.42:0/2322716107 >> 172.21.15.42:0/2322716107 conn(0x7f569c0f0b70 msgr2=0x7f569c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:07.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.229+0000 7f56a24a3700 1 -- 172.21.15.42:0/2322716107 shutdown_connections 2024-02-15T18:36:07.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.229+0000 7f56a24a3700 1 -- 172.21.15.42:0/2322716107 wait complete. 2024-02-15T18:36:07.231 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:06.235554+0000 mon.smithi042 (mon.0) 775 : audit [DBG] from='client.? 172.21.15.42:0/1102121379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:36:07.231 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:06.593885+0000 mon.smithi042 (mon.0) 776 : audit [DBG] from='client.? 172.21.15.42:0/1268113428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-15T18:36:07.231 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:36:06.817111+0000 mgr.smithi042.kfoihb (mgr.24435) 481 : cluster [DBG] pgmap v368: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:36:07.231 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:07 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:06.910708+0000 mon.smithi042 (mon.0) 777 : audit [DBG] from='client.? 172.21.15.42:0/4098734775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-15T18:36:07.232 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-15T18:36:07.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.361+0000 7f3ebad51700 1 -- 172.21.15.42:0/4220449684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eb405fe80 msgr2=0x7f3eb40f9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:07.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.361+0000 7f3ebad51700 1 --2- 172.21.15.42:0/4220449684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eb405fe80 0x7f3eb40f9130 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f3ea8004660 tx=0x7f3ea8009b30).stop 2024-02-15T18:36:07.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.361+0000 7f3ebad51700 1 -- 172.21.15.42:0/4220449684 shutdown_connections 2024-02-15T18:36:07.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.361+0000 7f3ebad51700 1 --2- 172.21.15.42:0/4220449684 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3eb40f9620 0x7f3eb40fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.361+0000 7f3ebad51700 1 --2- 172.21.15.42:0/4220449684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eb405fe80 0x7f3eb40f9130 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.361+0000 7f3ebad51700 1 -- 172.21.15.42:0/4220449684 >> 172.21.15.42:0/4220449684 conn(0x7f3eb40f0bd0 msgr2=0x7f3eb40f2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.361+0000 7f3ebad51700 1 -- 172.21.15.42:0/4220449684 shutdown_connections 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.361+0000 7f3ebad51700 1 -- 172.21.15.42:0/4220449684 wait complete. 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.362+0000 7f3ebad51700 1 Processor -- start 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.362+0000 7f3ebad51700 1 -- start start 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.362+0000 7f3ebad51700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3eb405fe80 0x7f3eb4106c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.362+0000 7f3ebad51700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eb40f9620 0x7f3eb4107120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.362+0000 7f3ebad51700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb4107720 con 0x7f3eb40f9620 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.362+0000 7f3ebad51700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb4107860 con 0x7f3eb405fe80 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.362+0000 7f3eb3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eb40f9620 0x7f3eb4107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.362+0000 7f3eb3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eb40f9620 0x7f3eb4107120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39932/0 (socket says 172.21.15.42:39932) 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.362+0000 7f3eb3fff700 1 -- 172.21.15.42:0/2981857750 learned_addr learned my addr 172.21.15.42:0/2981857750 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.362+0000 7f3eb3fff700 1 -- 172.21.15.42:0/2981857750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3eb405fe80 msgr2=0x7f3eb4106c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.362+0000 7f3eb3fff700 1 --2- 172.21.15.42:0/2981857750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3eb405fe80 0x7f3eb4106c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.362+0000 7f3eb3fff700 1 -- 172.21.15.42:0/2981857750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea8005040 con 0x7f3eb40f9620 2024-02-15T18:36:07.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.363+0000 7f3eb3fff700 1 --2- 172.21.15.42:0/2981857750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eb40f9620 0x7f3eb4107120 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f3ea000c790 tx=0x7f3ea0007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:07.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.363+0000 7f3eb17fa700 1 -- 172.21.15.42:0/2981857750 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ea000e680 con 0x7f3eb40f9620 2024-02-15T18:36:07.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.363+0000 7f3eb17fa700 1 -- 172.21.15.42:0/2981857750 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f3ea000ecd0 con 0x7f3eb40f9620 2024-02-15T18:36:07.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.363+0000 7f3ebad51700 1 -- 172.21.15.42:0/2981857750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb410c270 con 0x7f3eb40f9620 2024-02-15T18:36:07.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.363+0000 7f3eb17fa700 1 -- 172.21.15.42:0/2981857750 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ea0013ea0 con 0x7f3eb40f9620 2024-02-15T18:36:07.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.363+0000 7f3ebad51700 1 -- 172.21.15.42:0/2981857750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb410c830 con 0x7f3eb40f9620 2024-02-15T18:36:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.364+0000 7f3ebad51700 1 -- 172.21.15.42:0/2981857750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb4027a00 con 0x7f3eb40f9620 2024-02-15T18:36:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.364+0000 7f3eb17fa700 1 -- 172.21.15.42:0/2981857750 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f3ea001a060 con 0x7f3eb40f9620 2024-02-15T18:36:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.364+0000 7f3eb17fa700 1 --2- 172.21.15.42:0/2981857750 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3ea4060fe0 0x7f3ea40634a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.364+0000 7f3eb17fa700 1 -- 172.21.15.42:0/2981857750 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f3ea007c510 con 0x7f3eb40f9620 2024-02-15T18:36:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.364+0000 7f3eb8aed700 1 --2- 172.21.15.42:0/2981857750 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3ea4060fe0 0x7f3ea40634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:07.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.365+0000 7f3eb8aed700 1 --2- 172.21.15.42:0/2981857750 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3ea4060fe0 0x7f3ea40634a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f3ea8000c00 tx=0x7f3ea8009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:07.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.368+0000 7f3eb17fa700 1 -- 172.21.15.42:0/2981857750 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3ea004c990 con 0x7f3eb40f9620 2024-02-15T18:36:07.440 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:06.235554+0000 mon.smithi042 (mon.0) 775 : audit [DBG] from='client.? 172.21.15.42:0/1102121379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-15T18:36:07.440 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:06.593885+0000 mon.smithi042 (mon.0) 776 : audit [DBG] from='client.? 172.21.15.42:0/1268113428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-15T18:36:07.440 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:36:06.817111+0000 mgr.smithi042.kfoihb (mgr.24435) 481 : cluster [DBG] pgmap v368: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:36:07.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:07 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:06.910708+0000 mon.smithi042 (mon.0) 777 : audit [DBG] from='client.? 172.21.15.42:0/4098734775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-15T18:36:07.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.583+0000 7f3ebad51700 1 -- 172.21.15.42:0/2981857750 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3eb40fed90 con 0x7f3eb40f9620 2024-02-15T18:36:07.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.584+0000 7f3eb17fa700 1 -- 172.21.15.42:0/2981857750 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4591 (secure 0 0 0) 0x7f3ea004ffb0 con 0x7f3eb40f9620 2024-02-15T18:36:07.574 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:07.575 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44343,"name":"cephfs.smithi191.icwcnb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6825/540280968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":540280968},{"type":"v1","addr":"172.21.15.191:6825","nonce":540280968}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44347,"name":"cephfs.smithi191.erlpxp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6827/289198038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":289198038},{"type":"v1","addr":"172.21.15.191:6827","nonce":289198038}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:55.244270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34356":{"gid":34356,"name":"cephfs.smithi042.vtwvlo","rank":0,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.42:6827/2962563293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2962563293},{"type":"v1","addr":"172.21.15.42:6827","nonce":2962563293}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34360":{"gid":34360,"name":"cephfs.smithi042.idnxkc","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6829/1562993914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1562993914},{"type":"v1","addr":"172.21.15.42:6829","nonce":1562993914}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.585+0000 7f3ebad51700 1 -- 172.21.15.42:0/2981857750 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3ea4060fe0 msgr2=0x7f3ea40634a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.585+0000 7f3ebad51700 1 --2- 172.21.15.42:0/2981857750 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3ea4060fe0 0x7f3ea40634a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f3ea8000c00 tx=0x7f3ea8009b30).stop 2024-02-15T18:36:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.585+0000 7f3ebad51700 1 -- 172.21.15.42:0/2981857750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eb40f9620 msgr2=0x7f3eb4107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.585+0000 7f3ebad51700 1 --2- 172.21.15.42:0/2981857750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eb40f9620 0x7f3eb4107120 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f3ea000c790 tx=0x7f3ea0007d70).stop 2024-02-15T18:36:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.585+0000 7f3ebad51700 1 -- 172.21.15.42:0/2981857750 shutdown_connections 2024-02-15T18:36:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.585+0000 7f3ebad51700 1 --2- 172.21.15.42:0/2981857750 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3ea4060fe0 0x7f3ea40634a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.585+0000 7f3ebad51700 1 --2- 172.21.15.42:0/2981857750 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3eb405fe80 0x7f3eb4106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:36:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.585+0000 7f3ebad51700 1 --2- 172.21.15.42:0/2981857750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3eb40f9620 0x7f3eb4107120 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.585+0000 7f3ebad51700 1 -- 172.21.15.42:0/2981857750 >> 172.21.15.42:0/2981857750 conn(0x7f3eb40f0bd0 msgr2=0x7f3eb40f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.586+0000 7f3ebad51700 1 -- 172.21.15.42:0/2981857750 shutdown_connections 2024-02-15T18:36:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.586+0000 7f3ebad51700 1 -- 172.21.15.42:0/2981857750 wait complete. 2024-02-15T18:36:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 28 2024-02-15T18:36:07.588 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 11, 'max_mds': 1, 'flags': 50} 2024-02-15T18:36:07.589 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 12 2024-02-15T18:36:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.719+0000 7fdd55594700 1 -- 172.21.15.42:0/3133611218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd500f7b10 msgr2=0x7fdd500f7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.719+0000 7fdd55594700 1 --2- 172.21.15.42:0/3133611218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd500f7b10 0x7fdd500f7f30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fdd38004660 tx=0x7fdd38009b30).stop 2024-02-15T18:36:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.719+0000 7fdd55594700 1 -- 172.21.15.42:0/3133611218 shutdown_connections 2024-02-15T18:36:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.719+0000 7fdd55594700 1 --2- 172.21.15.42:0/3133611218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd500f8420 0x7fdd500f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.719+0000 7fdd55594700 1 --2- 172.21.15.42:0/3133611218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd500f7b10 0x7fdd500f7f30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.719+0000 7fdd55594700 1 -- 172.21.15.42:0/3133611218 >> 172.21.15.42:0/3133611218 conn(0x7fdd500f0370 msgr2=0x7fdd500f2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.719+0000 7fdd55594700 1 -- 172.21.15.42:0/3133611218 shutdown_connections 2024-02-15T18:36:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.719+0000 7fdd55594700 1 -- 172.21.15.42:0/3133611218 wait complete. 2024-02-15T18:36:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd55594700 1 Processor -- start 2024-02-15T18:36:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd55594700 1 -- start start 2024-02-15T18:36:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd55594700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd500f7b10 0x7fdd500f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:07.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd55594700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd500f8420 0x7fdd500f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd55594700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500f77b0 con 0x7fdd500f8420 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd55594700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd500f5d50 con 0x7fdd500f7b10 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd4e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd500f8420 0x7fdd500f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd4e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd500f8420 0x7fdd500f5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39946/0 (socket says 172.21.15.42:39946) 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd4e7fc700 1 -- 172.21.15.42:0/3649008431 learned_addr learned my addr 172.21.15.42:0/3649008431 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd4effd700 1 --2- 172.21.15.42:0/3649008431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd500f7b10 0x7fdd500f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd4e7fc700 1 -- 172.21.15.42:0/3649008431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd500f7b10 msgr2=0x7fdd500f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd4e7fc700 1 --2- 172.21.15.42:0/3649008431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd500f7b10 0x7fdd500f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd4e7fc700 1 -- 172.21.15.42:0/3649008431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd38005040 con 0x7fdd500f8420 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd4effd700 1 --2- 172.21.15.42:0/3649008431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd500f7b10 0x7fdd500f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd4e7fc700 1 --2- 172.21.15.42:0/3649008431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd500f8420 0x7fdd500f5860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fdd4000c530 tx=0x7fdd40007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.720+0000 7fdd477fe700 1 -- 172.21.15.42:0/3649008431 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd4000f650 con 0x7fdd500f8420 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.721+0000 7fdd477fe700 1 -- 172.21.15.42:0/3649008431 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fdd4000fca0 con 0x7fdd500f8420 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.721+0000 7fdd55594700 1 -- 172.21.15.42:0/3649008431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd500f5fd0 con 0x7fdd500f8420 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.721+0000 7fdd477fe700 1 -- 172.21.15.42:0/3649008431 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd40012e10 con 0x7fdd500f8420 2024-02-15T18:36:07.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.721+0000 7fdd55594700 1 -- 172.21.15.42:0/3649008431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd500f6590 con 0x7fdd500f8420 2024-02-15T18:36:07.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.721+0000 7fdd55594700 1 -- 172.21.15.42:0/3649008431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd50046330 con 0x7fdd500f8420 2024-02-15T18:36:07.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.722+0000 7fdd477fe700 1 -- 172.21.15.42:0/3649008431 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fdd40010070 con 0x7fdd500f8420 2024-02-15T18:36:07.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.722+0000 7fdd477fe700 1 --2- 172.21.15.42:0/3649008431 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdd3c060f20 0x7fdd3c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:07.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.722+0000 7fdd477fe700 1 -- 172.21.15.42:0/3649008431 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fdd4007bfb0 con 0x7fdd500f8420 2024-02-15T18:36:07.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.722+0000 7fdd4effd700 1 --2- 172.21.15.42:0/3649008431 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdd3c060f20 0x7fdd3c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:07.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.723+0000 7fdd4effd700 1 --2- 172.21.15.42:0/3649008431 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdd3c060f20 0x7fdd3c0633e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fdd38004630 tx=0x7fdd38009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:07.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.727+0000 7fdd477fe700 1 -- 172.21.15.42:0/3649008431 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdd40047f60 con 0x7fdd500f8420 2024-02-15T18:36:07.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.941+0000 7fdd55594700 1 -- 172.21.15.42:0/3649008431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 12, "format": "json"} v 0) v1 -- 0x7fdd50027a00 con 0x7fdd500f8420 2024-02-15T18:36:07.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.941+0000 7fdd477fe700 1 -- 172.21.15.42:0/3649008431 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 12, "format": "json"}]=0 dumped fsmap epoch 12 v28) v1 ==== 107+0+4591 (secure 0 0 0) 0x7fdd4004fcf0 con 0x7fdd500f8420 2024-02-15T18:36:07.932 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:07.932 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"cephfs.smithi042.vtwvlo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.42:6827/1892158366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1892158366},{"type":"v1","addr":"172.21.15.42:6827","nonce":1892158366}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14450,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.42:6829/2580627796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2580627796},{"type":"v1","addr":"172.21.15.42:6829","nonce":2580627796}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":11,"flags":50,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:23:55.523717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24263":{"gid":24263,"name":"cephfs.smithi191.icwcnb","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.191:6825/1971151000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1971151000},{"type":"v1","addr":"172.21.15.191:6825","nonce":1971151000}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24277":{"gid":24277,"name":"cephfs.smithi191.erlpxp","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.191:6827/126692509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":126692509},{"type":"v1","addr":"172.21.15.191:6827","nonce":126692509}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:07.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.943+0000 7fdd55594700 1 -- 172.21.15.42:0/3649008431 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdd3c060f20 msgr2=0x7fdd3c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.943+0000 7fdd55594700 1 --2- 172.21.15.42:0/3649008431 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdd3c060f20 0x7fdd3c0633e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fdd38004630 tx=0x7fdd38009b30).stop 2024-02-15T18:36:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.943+0000 7fdd55594700 1 -- 172.21.15.42:0/3649008431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd500f8420 msgr2=0x7fdd500f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.943+0000 7fdd55594700 1 --2- 172.21.15.42:0/3649008431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd500f8420 0x7fdd500f5860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fdd4000c530 tx=0x7fdd40007e20).stop 2024-02-15T18:36:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.943+0000 7fdd55594700 1 -- 172.21.15.42:0/3649008431 shutdown_connections 2024-02-15T18:36:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.943+0000 7fdd55594700 1 --2- 172.21.15.42:0/3649008431 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdd3c060f20 0x7fdd3c0633e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.943+0000 7fdd55594700 1 --2- 172.21.15.42:0/3649008431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd500f7b10 0x7fdd500f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.943+0000 7fdd55594700 1 --2- 172.21.15.42:0/3649008431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd500f8420 0x7fdd500f5860 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.943+0000 7fdd55594700 1 -- 172.21.15.42:0/3649008431 >> 172.21.15.42:0/3649008431 conn(0x7fdd500f0370 msgr2=0x7fdd500fb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.943+0000 7fdd55594700 1 -- 172.21.15.42:0/3649008431 shutdown_connections 2024-02-15T18:36:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:07.943+0000 7fdd55594700 1 -- 172.21.15.42:0/3649008431 wait complete. 2024-02-15T18:36:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 12 2024-02-15T18:36:07.946 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 13 2024-02-15T18:36:08.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.077+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/634859450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9e80f5390 msgr2=0x7fd9e80f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.077+0000 7fd9ecc9c700 1 --2- 172.21.15.42:0/634859450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9e80f5390 0x7fd9e80f57b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fd9d0004660 tx=0x7fd9d0009b30).stop 2024-02-15T18:36:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.077+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/634859450 shutdown_connections 2024-02-15T18:36:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.077+0000 7fd9ecc9c700 1 --2- 172.21.15.42:0/634859450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9e80f5ca0 0x7fd9e80f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.077+0000 7fd9ecc9c700 1 --2- 172.21.15.42:0/634859450 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9e80f5390 0x7fd9e80f57b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.077+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/634859450 >> 172.21.15.42:0/634859450 conn(0x7fd9e80f0b70 msgr2=0x7fd9e80f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.077+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/634859450 shutdown_connections 2024-02-15T18:36:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.077+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/634859450 wait complete. 2024-02-15T18:36:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.077+0000 7fd9ecc9c700 1 Processor -- start 2024-02-15T18:36:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.077+0000 7fd9ecc9c700 1 -- start start 2024-02-15T18:36:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9ecc9c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9e80f5390 0x7fd9e8106cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9ecc9c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9e80f5ca0 0x7fd9e81071c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:08.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9ecc9c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e81077c0 con 0x7fd9e80f5ca0 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9ecc9c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e8107900 con 0x7fd9e80f5390 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9e5d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9e80f5ca0 0x7fd9e81071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9e5d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9e80f5ca0 0x7fd9e81071c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39966/0 (socket says 172.21.15.42:39966) 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9e5d9b700 1 -- 172.21.15.42:0/1572343848 learned_addr learned my addr 172.21.15.42:0/1572343848 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9e659c700 1 --2- 172.21.15.42:0/1572343848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9e80f5390 0x7fd9e8106cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9e5d9b700 1 -- 172.21.15.42:0/1572343848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9e80f5390 msgr2=0x7fd9e8106cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9e5d9b700 1 --2- 172.21.15.42:0/1572343848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9e80f5390 0x7fd9e8106cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9e5d9b700 1 -- 172.21.15.42:0/1572343848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d0005040 con 0x7fd9e80f5ca0 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9e659c700 1 --2- 172.21.15.42:0/1572343848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9e80f5390 0x7fd9e8106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9e5d9b700 1 --2- 172.21.15.42:0/1572343848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9e80f5ca0 0x7fd9e81071c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fd9d80026d0 tx=0x7fd9d8009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9deffd700 1 -- 172.21.15.42:0/1572343848 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9d8009320 con 0x7fd9e80f5ca0 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/1572343848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e810c310 con 0x7fd9e80f5ca0 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9deffd700 1 -- 172.21.15.42:0/1572343848 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fd9d800f040 con 0x7fd9e80f5ca0 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9deffd700 1 -- 172.21.15.42:0/1572343848 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9d80149b0 con 0x7fd9e80f5ca0 2024-02-15T18:36:08.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.078+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/1572343848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e810c8d0 con 0x7fd9e80f5ca0 2024-02-15T18:36:08.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.079+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/1572343848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e80fdc90 con 0x7fd9e80f5ca0 2024-02-15T18:36:08.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.080+0000 7fd9deffd700 1 -- 172.21.15.42:0/1572343848 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fd9d8009480 con 0x7fd9e80f5ca0 2024-02-15T18:36:08.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.080+0000 7fd9deffd700 1 --2- 172.21.15.42:0/1572343848 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd9d4060f20 0x7fd9d40633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:08.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.080+0000 7fd9deffd700 1 -- 172.21.15.42:0/1572343848 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fd9d807c840 con 0x7fd9e80f5ca0 2024-02-15T18:36:08.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.080+0000 7fd9e659c700 1 --2- 172.21.15.42:0/1572343848 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd9d4060f20 0x7fd9d40633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:08.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.081+0000 7fd9e659c700 1 --2- 172.21.15.42:0/1572343848 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd9d4060f20 0x7fd9d40633e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fd9d0004e30 tx=0x7fd9d0009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:08.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.084+0000 7fd9deffd700 1 -- 172.21.15.42:0/1572343848 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd9d804cb50 con 0x7fd9e80f5ca0 2024-02-15T18:36:08.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.304+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/1572343848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 13, "format": "json"} v 0) v1 -- 0x7fd9e810cf70 con 0x7fd9e80f5ca0 2024-02-15T18:36:08.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.304+0000 7fd9deffd700 1 -- 172.21.15.42:0/1572343848 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 13, "format": "json"}]=0 dumped fsmap epoch 13 v28) v1 ==== 107+0+4590 (secure 0 0 0) 0x7fd9d8050170 con 0x7fd9e80f5ca0 2024-02-15T18:36:08.295 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:08.295 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"cephfs.smithi042.vtwvlo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.42:6827/1892158366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1892158366},{"type":"v1","addr":"172.21.15.42:6827","nonce":1892158366}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14450,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.42:6829/2580627796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2580627796},{"type":"v1","addr":"172.21.15.42:6829","nonce":2580627796}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":11,"flags":50,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:23:55.523717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24263":{"gid":24263,"name":"cephfs.smithi191.icwcnb","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.191:6825/1971151000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1971151000},{"type":"v1","addr":"172.21.15.191:6825","nonce":1971151000}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24277":{"gid":24277,"name":"cephfs.smithi191.erlpxp","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.191:6827/126692509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":126692509},{"type":"v1","addr":"172.21.15.191:6827","nonce":126692509}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:08.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.306+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/1572343848 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd9d4060f20 msgr2=0x7fd9d40633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:08.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.306+0000 7fd9ecc9c700 1 --2- 172.21.15.42:0/1572343848 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd9d4060f20 0x7fd9d40633e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fd9d0004e30 tx=0x7fd9d0009b30).stop 2024-02-15T18:36:08.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.306+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/1572343848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9e80f5ca0 msgr2=0x7fd9e81071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:08.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.306+0000 7fd9ecc9c700 1 --2- 172.21.15.42:0/1572343848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9e80f5ca0 0x7fd9e81071c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fd9d80026d0 tx=0x7fd9d8009de0).stop 2024-02-15T18:36:08.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.306+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/1572343848 shutdown_connections 2024-02-15T18:36:08.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.306+0000 7fd9ecc9c700 1 --2- 172.21.15.42:0/1572343848 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fd9d4060f20 0x7fd9d40633e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.306+0000 7fd9ecc9c700 1 --2- 172.21.15.42:0/1572343848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9e80f5390 0x7fd9e8106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.306+0000 7fd9ecc9c700 1 --2- 172.21.15.42:0/1572343848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd9e80f5ca0 0x7fd9e81071c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.306+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/1572343848 >> 172.21.15.42:0/1572343848 conn(0x7fd9e80f0b70 msgr2=0x7fd9e805fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:08.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.306+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/1572343848 shutdown_connections 2024-02-15T18:36:08.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.306+0000 7fd9ecc9c700 1 -- 172.21.15.42:0/1572343848 wait complete. 2024-02-15T18:36:08.298 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-02-15T18:36:08.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:07.228371+0000 mon.smithi042 (mon.0) 778 : audit [DBG] from='client.? 172.21.15.42:0/2322716107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-15T18:36:08.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:07.584706+0000 mon.smithi042 (mon.0) 779 : audit [DBG] from='client.? 172.21.15.42:0/2981857750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-15T18:36:08.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:08 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:07.942528+0000 mon.smithi042 (mon.0) 780 : audit [DBG] from='client.? 172.21.15.42:0/3649008431' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 12, "format": "json"}]: dispatch 2024-02-15T18:36:08.309 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 14 2024-02-15T18:36:08.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.439+0000 7fb8056ce700 1 -- 172.21.15.42:0/2023617659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8000f5390 msgr2=0x7fb8000f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.439+0000 7fb8056ce700 1 --2- 172.21.15.42:0/2023617659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8000f5390 0x7fb8000f57b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fb7f4004660 tx=0x7fb7f4009b30).stop 2024-02-15T18:36:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.439+0000 7fb8056ce700 1 -- 172.21.15.42:0/2023617659 shutdown_connections 2024-02-15T18:36:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.439+0000 7fb8056ce700 1 --2- 172.21.15.42:0/2023617659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8000f6630 0x7fb8000f6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.439+0000 7fb8056ce700 1 --2- 172.21.15.42:0/2023617659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8000f5390 0x7fb8000f57b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.439+0000 7fb8056ce700 1 -- 172.21.15.42:0/2023617659 >> 172.21.15.42:0/2023617659 conn(0x7fb8000f0ad0 msgr2=0x7fb8000f2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.440+0000 7fb8056ce700 1 -- 172.21.15.42:0/2023617659 shutdown_connections 2024-02-15T18:36:08.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.440+0000 7fb8056ce700 1 -- 172.21.15.42:0/2023617659 wait complete. 2024-02-15T18:36:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.440+0000 7fb8056ce700 1 Processor -- start 2024-02-15T18:36:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.440+0000 7fb8056ce700 1 -- start start 2024-02-15T18:36:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.440+0000 7fb8056ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8000f5390 0x7fb800105bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.440+0000 7fb8056ce700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8000f6630 0x7fb8001060b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.440+0000 7fb8056ce700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb800106660 con 0x7fb8000f5390 2024-02-15T18:36:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.440+0000 7fb8056ce700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8001067a0 con 0x7fb8000f6630 2024-02-15T18:36:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.441+0000 7fb7feffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8000f5390 0x7fb800105bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.441+0000 7fb7feffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8000f5390 0x7fb800105bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39988/0 (socket says 172.21.15.42:39988) 2024-02-15T18:36:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.441+0000 7fb7feffd700 1 -- 172.21.15.42:0/1443539927 learned_addr learned my addr 172.21.15.42:0/1443539927 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.441+0000 7fb7fe7fc700 1 --2- 172.21.15.42:0/1443539927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8000f6630 0x7fb8001060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.441+0000 7fb7feffd700 1 -- 172.21.15.42:0/1443539927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8000f6630 msgr2=0x7fb8001060b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:08.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.441+0000 7fb7feffd700 1 --2- 172.21.15.42:0/1443539927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8000f6630 0x7fb8001060b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.441+0000 7fb7feffd700 1 -- 172.21.15.42:0/1443539927 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f4005040 con 0x7fb8000f5390 2024-02-15T18:36:08.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.441+0000 7fb7feffd700 1 --2- 172.21.15.42:0/1443539927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8000f5390 0x7fb800105bc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fb7f4004e30 tx=0x7fb7f4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:08.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.441+0000 7fb7e77fe700 1 -- 172.21.15.42:0/1443539927 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7f401a070 con 0x7fb8000f5390 2024-02-15T18:36:08.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.441+0000 7fb8056ce700 1 -- 172.21.15.42:0/1443539927 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80005f650 con 0x7fb8000f5390 2024-02-15T18:36:08.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.441+0000 7fb7e77fe700 1 -- 172.21.15.42:0/1443539927 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fb7f401faf0 con 0x7fb8000f5390 2024-02-15T18:36:08.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.441+0000 7fb7e77fe700 1 -- 172.21.15.42:0/1443539927 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7f401e8a0 con 0x7fb8000f5390 2024-02-15T18:36:08.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.441+0000 7fb8056ce700 1 -- 172.21.15.42:0/1443539927 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80005fc10 con 0x7fb8000f5390 2024-02-15T18:36:08.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.442+0000 7fb8056ce700 1 -- 172.21.15.42:0/1443539927 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8000fdc60 con 0x7fb8000f5390 2024-02-15T18:36:08.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.443+0000 7fb7e77fe700 1 -- 172.21.15.42:0/1443539927 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fb7f401fc60 con 0x7fb8000f5390 2024-02-15T18:36:08.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.443+0000 7fb7e77fe700 1 --2- 172.21.15.42:0/1443539927 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb7e8060f20 0x7fb7e80633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:08.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.443+0000 7fb7e77fe700 1 -- 172.21.15.42:0/1443539927 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fb7f4080310 con 0x7fb8000f5390 2024-02-15T18:36:08.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.443+0000 7fb7fe7fc700 1 --2- 172.21.15.42:0/1443539927 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb7e8060f20 0x7fb7e80633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:08.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.444+0000 7fb7fe7fc700 1 --2- 172.21.15.42:0/1443539927 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb7e8060f20 0x7fb7e80633e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fb7f0004c40 tx=0x7fb7f000b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:08.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.447+0000 7fb7e77fe700 1 -- 172.21.15.42:0/1443539927 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb7f4054130 con 0x7fb8000f5390 2024-02-15T18:36:08.440 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:07.228371+0000 mon.smithi042 (mon.0) 778 : audit [DBG] from='client.? 172.21.15.42:0/2322716107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-15T18:36:08.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:07.584706+0000 mon.smithi042 (mon.0) 779 : audit [DBG] from='client.? 172.21.15.42:0/2981857750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-15T18:36:08.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:08 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:07.942528+0000 mon.smithi042 (mon.0) 780 : audit [DBG] from='client.? 172.21.15.42:0/3649008431' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 12, "format": "json"}]: dispatch 2024-02-15T18:36:08.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.663+0000 7fb8056ce700 1 -- 172.21.15.42:0/1443539927 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 14, "format": "json"} v 0) v1 -- 0x7fb80010b180 con 0x7fb8000f5390 2024-02-15T18:36:08.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.663+0000 7fb7e77fe700 1 -- 172.21.15.42:0/1443539927 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 14, "format": "json"}]=0 dumped fsmap epoch 14 v28) v1 ==== 107+0+3804 (secure 0 0 0) 0x7fb7f4053d20 con 0x7fb8000f5390 2024-02-15T18:36:08.655 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:08.655 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"cephfs.smithi042.vtwvlo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.42:6827/1892158366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1892158366},{"type":"v1","addr":"172.21.15.42:6827","nonce":1892158366}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14450,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.42:6829/2580627796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2580627796},{"type":"v1","addr":"172.21.15.42:6829","nonce":2580627796}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:01.852374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24263":{"gid":24263,"name":"cephfs.smithi191.icwcnb","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.191:6825/1971151000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1971151000},{"type":"v1","addr":"172.21.15.191:6825","nonce":1971151000}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:08.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.665+0000 7fb8056ce700 1 -- 172.21.15.42:0/1443539927 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb7e8060f20 msgr2=0x7fb7e80633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:08.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.665+0000 7fb8056ce700 1 --2- 172.21.15.42:0/1443539927 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb7e8060f20 0x7fb7e80633e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fb7f0004c40 tx=0x7fb7f000b040).stop 2024-02-15T18:36:08.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.665+0000 7fb8056ce700 1 -- 172.21.15.42:0/1443539927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8000f5390 msgr2=0x7fb800105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:08.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.665+0000 7fb8056ce700 1 --2- 172.21.15.42:0/1443539927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8000f5390 0x7fb800105bc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fb7f4004e30 tx=0x7fb7f4009b30).stop 2024-02-15T18:36:08.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.665+0000 7fb8056ce700 1 -- 172.21.15.42:0/1443539927 shutdown_connections 2024-02-15T18:36:08.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.665+0000 7fb8056ce700 1 --2- 172.21.15.42:0/1443539927 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fb7e8060f20 0x7fb7e80633e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.665+0000 7fb8056ce700 1 --2- 172.21.15.42:0/1443539927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8000f5390 0x7fb800105bc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.665+0000 7fb8056ce700 1 --2- 172.21.15.42:0/1443539927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8000f6630 0x7fb8001060b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.665+0000 7fb8056ce700 1 -- 172.21.15.42:0/1443539927 >> 172.21.15.42:0/1443539927 conn(0x7fb8000f0ad0 msgr2=0x7fb8000fa080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:08.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.665+0000 7fb8056ce700 1 -- 172.21.15.42:0/1443539927 shutdown_connections 2024-02-15T18:36:08.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.665+0000 7fb8056ce700 1 -- 172.21.15.42:0/1443539927 wait complete. 2024-02-15T18:36:08.657 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 14 2024-02-15T18:36:08.668 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 14 2024-02-15T18:36:08.668 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 15 2024-02-15T18:36:08.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.798+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/1508159115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe980f5ca0 msgr2=0x7fbe980f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:08.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.798+0000 7fbe9e8a2700 1 --2- 172.21.15.42:0/1508159115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe980f5ca0 0x7fbe980f66d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fbe8c004660 tx=0x7fbe8c009b30).stop 2024-02-15T18:36:08.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.798+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/1508159115 shutdown_connections 2024-02-15T18:36:08.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.798+0000 7fbe9e8a2700 1 --2- 172.21.15.42:0/1508159115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe980f5ca0 0x7fbe980f66d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.798+0000 7fbe9e8a2700 1 --2- 172.21.15.42:0/1508159115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe980f5390 0x7fbe980f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.798+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/1508159115 >> 172.21.15.42:0/1508159115 conn(0x7fbe980f0b70 msgr2=0x7fbe980f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:08.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.798+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/1508159115 shutdown_connections 2024-02-15T18:36:08.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.798+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/1508159115 wait complete. 2024-02-15T18:36:08.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.799+0000 7fbe9e8a2700 1 Processor -- start 2024-02-15T18:36:08.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.799+0000 7fbe9e8a2700 1 -- start start 2024-02-15T18:36:08.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.799+0000 7fbe9e8a2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe980f5390 0x7fbe98106cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:08.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.799+0000 7fbe9e8a2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe980f5ca0 0x7fbe981071c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:08.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.799+0000 7fbe9e8a2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe981077c0 con 0x7fbe980f5ca0 2024-02-15T18:36:08.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.799+0000 7fbe9e8a2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe98107900 con 0x7fbe980f5390 2024-02-15T18:36:08.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.799+0000 7fbe977fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe980f5ca0 0x7fbe981071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:08.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.799+0000 7fbe977fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe980f5ca0 0x7fbe981071c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40008/0 (socket says 172.21.15.42:40008) 2024-02-15T18:36:08.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.799+0000 7fbe977fe700 1 -- 172.21.15.42:0/976885909 learned_addr learned my addr 172.21.15.42:0/976885909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:08.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.800+0000 7fbe97fff700 1 --2- 172.21.15.42:0/976885909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe980f5390 0x7fbe98106cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:08.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.800+0000 7fbe977fe700 1 -- 172.21.15.42:0/976885909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe980f5390 msgr2=0x7fbe98106cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:08.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.800+0000 7fbe977fe700 1 --2- 172.21.15.42:0/976885909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe980f5390 0x7fbe98106cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:08.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.800+0000 7fbe977fe700 1 -- 172.21.15.42:0/976885909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c005040 con 0x7fbe980f5ca0 2024-02-15T18:36:08.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.800+0000 7fbe977fe700 1 --2- 172.21.15.42:0/976885909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe980f5ca0 0x7fbe981071c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fbe8c000c00 tx=0x7fbe8c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:08.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.800+0000 7fbe94ff9700 1 -- 172.21.15.42:0/976885909 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe8c019070 con 0x7fbe980f5ca0 2024-02-15T18:36:08.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.800+0000 7fbe94ff9700 1 -- 172.21.15.42:0/976885909 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fbe8c014420 con 0x7fbe980f5ca0 2024-02-15T18:36:08.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.800+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/976885909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9810c2b0 con 0x7fbe980f5ca0 2024-02-15T18:36:08.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.800+0000 7fbe94ff9700 1 -- 172.21.15.42:0/976885909 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe8c01d600 con 0x7fbe980f5ca0 2024-02-15T18:36:08.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.800+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/976885909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9810c810 con 0x7fbe980f5ca0 2024-02-15T18:36:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.801+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/976885909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe980fdc90 con 0x7fbe980f5ca0 2024-02-15T18:36:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.802+0000 7fbe94ff9700 1 -- 172.21.15.42:0/976885909 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fbe8c025070 con 0x7fbe980f5ca0 2024-02-15T18:36:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.802+0000 7fbe94ff9700 1 --2- 172.21.15.42:0/976885909 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbe80060f20 0x7fbe800633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.802+0000 7fbe94ff9700 1 -- 172.21.15.42:0/976885909 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fbe8c07d6a0 con 0x7fbe980f5ca0 2024-02-15T18:36:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.802+0000 7fbe97fff700 1 --2- 172.21.15.42:0/976885909 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbe80060f20 0x7fbe800633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.803+0000 7fbe97fff700 1 --2- 172.21.15.42:0/976885909 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbe80060f20 0x7fbe800633e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fbe88009da0 tx=0x7fbe8800b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:08.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:08.806+0000 7fbe94ff9700 1 -- 172.21.15.42:0/976885909 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbe8c04d970 con 0x7fbe980f5ca0 2024-02-15T18:36:09.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.021+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/976885909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 15, "format": "json"} v 0) v1 -- 0x7fbe9810ce50 con 0x7fbe980f5ca0 2024-02-15T18:36:09.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.021+0000 7fbe94ff9700 1 -- 172.21.15.42:0/976885909 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 15, "format": "json"}]=0 dumped fsmap epoch 15 v28) v1 ==== 107+0+4586 (secure 0 0 0) 0x7fbe8c023020 con 0x7fbe980f5ca0 2024-02-15T18:36:09.012 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:09.012 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"cephfs.smithi042.vtwvlo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.42:6827/1892158366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1892158366},{"type":"v1","addr":"172.21.15.42:6827","nonce":1892158366}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14450,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.42:6829/2580627796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2580627796},{"type":"v1","addr":"172.21.15.42:6829","nonce":2580627796}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":44313,"name":"cephfs.smithi191.erlpxp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6827/1725194059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1725194059},{"type":"v1","addr":"172.21.15.191:6827","nonce":1725194059}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:01.852374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24263":{"gid":24263,"name":"cephfs.smithi191.icwcnb","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.191:6825/1971151000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1971151000},{"type":"v1","addr":"172.21.15.191:6825","nonce":1971151000}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:09.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.023+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/976885909 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbe80060f20 msgr2=0x7fbe800633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:09.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.023+0000 7fbe9e8a2700 1 --2- 172.21.15.42:0/976885909 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbe80060f20 0x7fbe800633e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fbe88009da0 tx=0x7fbe8800b040).stop 2024-02-15T18:36:09.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.023+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/976885909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe980f5ca0 msgr2=0x7fbe981071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:09.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.023+0000 7fbe9e8a2700 1 --2- 172.21.15.42:0/976885909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe980f5ca0 0x7fbe981071c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fbe8c000c00 tx=0x7fbe8c00b040).stop 2024-02-15T18:36:09.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.023+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/976885909 shutdown_connections 2024-02-15T18:36:09.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.023+0000 7fbe9e8a2700 1 --2- 172.21.15.42:0/976885909 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fbe80060f20 0x7fbe800633e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.023+0000 7fbe9e8a2700 1 --2- 172.21.15.42:0/976885909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe980f5390 0x7fbe98106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.023+0000 7fbe9e8a2700 1 --2- 172.21.15.42:0/976885909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe980f5ca0 0x7fbe981071c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.023+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/976885909 >> 172.21.15.42:0/976885909 conn(0x7fbe980f0b70 msgr2=0x7fbe9805f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.023+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/976885909 shutdown_connections 2024-02-15T18:36:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.023+0000 7fbe9e8a2700 1 -- 172.21.15.42:0/976885909 wait complete. 2024-02-15T18:36:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 15 2024-02-15T18:36:09.026 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 15 2024-02-15T18:36:09.026 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-02-15T18:36:09.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.157+0000 7efbfdb7f700 1 -- 172.21.15.42:0/3699814505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbf80f5390 msgr2=0x7efbf80f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:09.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.157+0000 7efbfdb7f700 1 --2- 172.21.15.42:0/3699814505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbf80f5390 0x7efbf80f57b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7efbe8004660 tx=0x7efbe8009b30).stop 2024-02-15T18:36:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.157+0000 7efbfdb7f700 1 -- 172.21.15.42:0/3699814505 shutdown_connections 2024-02-15T18:36:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.157+0000 7efbfdb7f700 1 --2- 172.21.15.42:0/3699814505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efbf80f5ca0 0x7efbf80f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.157+0000 7efbfdb7f700 1 --2- 172.21.15.42:0/3699814505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbf80f5390 0x7efbf80f57b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.157+0000 7efbfdb7f700 1 -- 172.21.15.42:0/3699814505 >> 172.21.15.42:0/3699814505 conn(0x7efbf80f0b70 msgr2=0x7efbf80f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.157+0000 7efbfdb7f700 1 -- 172.21.15.42:0/3699814505 shutdown_connections 2024-02-15T18:36:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.157+0000 7efbfdb7f700 1 -- 172.21.15.42:0/3699814505 wait complete. 2024-02-15T18:36:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.157+0000 7efbfdb7f700 1 Processor -- start 2024-02-15T18:36:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbfdb7f700 1 -- start start 2024-02-15T18:36:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbfdb7f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbf80f5390 0x7efbf80f93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbfdb7f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efbf80f5ca0 0x7efbf80f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbfdb7f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf80f9960 con 0x7efbf80f5390 2024-02-15T18:36:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbfdb7f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf80f7f00 con 0x7efbf80f5ca0 2024-02-15T18:36:09.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbf77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbf80f5390 0x7efbf80f93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:09.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbf77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbf80f5390 0x7efbf80f93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40016/0 (socket says 172.21.15.42:40016) 2024-02-15T18:36:09.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbf77fe700 1 -- 172.21.15.42:0/2948743091 learned_addr learned my addr 172.21.15.42:0/2948743091 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:09.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbf77fe700 1 -- 172.21.15.42:0/2948743091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efbf80f5ca0 msgr2=0x7efbf80f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:09.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbf77fe700 1 --2- 172.21.15.42:0/2948743091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efbf80f5ca0 0x7efbf80f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:36:09.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbf77fe700 1 -- 172.21.15.42:0/2948743091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe8005040 con 0x7efbf80f5390 2024-02-15T18:36:09.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbf77fe700 1 --2- 172.21.15.42:0/2948743091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbf80f5390 0x7efbf80f93b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7efbe8009f50 tx=0x7efbe8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:09.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbeffff700 1 -- 172.21.15.42:0/2948743091 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbe8018070 con 0x7efbf80f5390 2024-02-15T18:36:09.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbeffff700 1 -- 172.21.15.42:0/2948743091 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7efbe8007d70 con 0x7efbf80f5390 2024-02-15T18:36:09.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbfdb7f700 1 -- 172.21.15.42:0/2948743091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf80f8120 con 0x7efbf80f5390 2024-02-15T18:36:09.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.158+0000 7efbeffff700 1 -- 172.21.15.42:0/2948743091 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efbe801c620 con 0x7efbf80f5390 2024-02-15T18:36:09.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.159+0000 7efbfdb7f700 1 -- 172.21.15.42:0/2948743091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf80f8680 con 0x7efbf80f5390 2024-02-15T18:36:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.159+0000 7efbfdb7f700 1 -- 172.21.15.42:0/2948743091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf8046330 con 0x7efbf80f5390 2024-02-15T18:36:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.160+0000 7efbeffff700 1 -- 172.21.15.42:0/2948743091 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7efbe801c890 con 0x7efbf80f5390 2024-02-15T18:36:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.160+0000 7efbeffff700 1 --2- 172.21.15.42:0/2948743091 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7efbe4060f20 0x7efbe40633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.160+0000 7efbeffff700 1 -- 172.21.15.42:0/2948743091 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7efbe807e0e0 con 0x7efbf80f5390 2024-02-15T18:36:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.160+0000 7efbf6ffd700 1 --2- 172.21.15.42:0/2948743091 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7efbe4060f20 0x7efbe40633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:09.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.161+0000 7efbf6ffd700 1 --2- 172.21.15.42:0/2948743091 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7efbe4060f20 0x7efbe40633e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7efbe0004870 tx=0x7efbe00079c0).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:09.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.164+0000 7efbeffff700 1 -- 172.21.15.42:0/2948743091 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7efbe8051ea0 con 0x7efbf80f5390 2024-02-15T18:36:09.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:08.305343+0000 mon.smithi042 (mon.0) 781 : audit [DBG] from='client.? 172.21.15.42:0/1572343848' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 13, "format": "json"}]: dispatch 2024-02-15T18:36:09.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:08.664624+0000 mon.smithi042 (mon.0) 782 : audit [DBG] from='client.? 172.21.15.42:0/1443539927' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-02-15T18:36:09.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:36:08.817445+0000 mgr.smithi042.kfoihb (mgr.24435) 482 : cluster [DBG] pgmap v369: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:36:09.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:09 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:09.022233+0000 mon.smithi042 (mon.0) 783 : audit [DBG] from='client.? 172.21.15.42:0/976885909' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-02-15T18:36:09.362 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:08.305343+0000 mon.smithi042 (mon.0) 781 : audit [DBG] from='client.? 172.21.15.42:0/1572343848' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 13, "format": "json"}]: dispatch 2024-02-15T18:36:09.363 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:08.664624+0000 mon.smithi042 (mon.0) 782 : audit [DBG] from='client.? 172.21.15.42:0/1443539927' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-02-15T18:36:09.363 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:36:08.817445+0000 mgr.smithi042.kfoihb (mgr.24435) 482 : cluster [DBG] pgmap v369: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:36:09.363 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:09 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:09.022233+0000 mon.smithi042 (mon.0) 783 : audit [DBG] from='client.? 172.21.15.42:0/976885909' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-02-15T18:36:09.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.381+0000 7efbfdb7f700 1 -- 172.21.15.42:0/2948743091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7efbf810f440 con 0x7efbf80f5390 2024-02-15T18:36:09.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.382+0000 7efbeffff700 1 -- 172.21.15.42:0/2948743091 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v28) v1 ==== 107+0+3807 (secure 0 0 0) 0x7efbe8051a90 con 0x7efbf80f5390 2024-02-15T18:36:09.373 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:09.373 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14450,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.42:6829/2580627796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2580627796},{"type":"v1","addr":"172.21.15.42:6829","nonce":2580627796}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":44313,"name":"cephfs.smithi191.erlpxp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6827/1725194059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1725194059},{"type":"v1","addr":"172.21.15.191:6827","nonce":1725194059}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:01.852374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24263":{"gid":24263,"name":"cephfs.smithi191.icwcnb","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.191:6825/1971151000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1971151000},{"type":"v1","addr":"172.21.15.191:6825","nonce":1971151000}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.384+0000 7efbfdb7f700 1 -- 172.21.15.42:0/2948743091 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7efbe4060f20 msgr2=0x7efbe40633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.384+0000 7efbfdb7f700 1 --2- 172.21.15.42:0/2948743091 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7efbe4060f20 0x7efbe40633e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7efbe0004870 tx=0x7efbe00079c0).stop 2024-02-15T18:36:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.384+0000 7efbfdb7f700 1 -- 172.21.15.42:0/2948743091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbf80f5390 msgr2=0x7efbf80f93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.384+0000 7efbfdb7f700 1 --2- 172.21.15.42:0/2948743091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbf80f5390 0x7efbf80f93b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7efbe8009f50 tx=0x7efbe8009b30).stop 2024-02-15T18:36:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.384+0000 7efbfdb7f700 1 -- 172.21.15.42:0/2948743091 shutdown_connections 2024-02-15T18:36:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.384+0000 7efbfdb7f700 1 --2- 172.21.15.42:0/2948743091 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7efbe4060f20 0x7efbe40633e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.384+0000 7efbfdb7f700 1 --2- 172.21.15.42:0/2948743091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efbf80f5390 0x7efbf80f93b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.384+0000 7efbfdb7f700 1 --2- 172.21.15.42:0/2948743091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efbf80f5ca0 0x7efbf80f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-15T18:36:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.384+0000 7efbfdb7f700 1 -- 172.21.15.42:0/2948743091 >> 172.21.15.42:0/2948743091 conn(0x7efbf80f0b70 msgr2=0x7efbf805fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.384+0000 7efbfdb7f700 1 -- 172.21.15.42:0/2948743091 shutdown_connections 2024-02-15T18:36:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.384+0000 7efbfdb7f700 1 -- 172.21.15.42:0/2948743091 wait complete. 2024-02-15T18:36:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 16 2024-02-15T18:36:09.387 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 16 2024-02-15T18:36:09.387 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-02-15T18:36:09.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.518+0000 7f9a144cd700 1 -- 172.21.15.42:0/1130259025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a0c0f5390 msgr2=0x7f9a0c0f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:09.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.518+0000 7f9a144cd700 1 --2- 172.21.15.42:0/1130259025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a0c0f5390 0x7f9a0c0f57b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f9a00004660 tx=0x7f9a00009b30).stop 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.518+0000 7f9a144cd700 1 -- 172.21.15.42:0/1130259025 shutdown_connections 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.518+0000 7f9a144cd700 1 --2- 172.21.15.42:0/1130259025 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c0f5ca0 0x7f9a0c0f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.518+0000 7f9a144cd700 1 --2- 172.21.15.42:0/1130259025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a0c0f5390 0x7f9a0c0f57b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.518+0000 7f9a144cd700 1 -- 172.21.15.42:0/1130259025 >> 172.21.15.42:0/1130259025 conn(0x7f9a0c0f0b70 msgr2=0x7f9a0c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.518+0000 7f9a144cd700 1 -- 172.21.15.42:0/1130259025 shutdown_connections 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.518+0000 7f9a144cd700 1 -- 172.21.15.42:0/1130259025 wait complete. 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.518+0000 7f9a144cd700 1 Processor -- start 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.518+0000 7f9a144cd700 1 -- start start 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.518+0000 7f9a144cd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c0f5390 0x7f9a0c05fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.518+0000 7f9a144cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a0c0f5ca0 0x7f9a0c10bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f9a144cd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c10bfb0 con 0x7f9a0c0f5ca0 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f9a144cd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c10c0f0 con 0x7f9a0c0f5390 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f9a11a68700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a0c0f5ca0 0x7f9a0c10bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f9a11a68700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a0c0f5ca0 0x7f9a0c10bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40030/0 (socket says 172.21.15.42:40030) 2024-02-15T18:36:09.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f9a11a68700 1 -- 172.21.15.42:0/4136268099 learned_addr learned my addr 172.21.15.42:0/4136268099 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f9a11a68700 1 -- 172.21.15.42:0/4136268099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c0f5390 msgr2=0x7f9a0c05fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f9a12269700 1 --2- 172.21.15.42:0/4136268099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c0f5390 0x7f9a0c05fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f9a11a68700 1 --2- 172.21.15.42:0/4136268099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c0f5390 0x7f9a0c05fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f9a11a68700 1 -- 172.21.15.42:0/4136268099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00005040 con 0x7f9a0c0f5ca0 2024-02-15T18:36:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f9a12269700 1 --2- 172.21.15.42:0/4136268099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c0f5390 0x7f9a0c05fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:36:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f9a11a68700 1 --2- 172.21.15.42:0/4136268099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a0c0f5ca0 0x7f9a0c10bac0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f9a0800c790 tx=0x7f9a08007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f99feffd700 1 -- 172.21.15.42:0/4136268099 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a0800e680 con 0x7f9a0c0f5ca0 2024-02-15T18:36:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f99feffd700 1 -- 172.21.15.42:0/4136268099 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f9a0800ecd0 con 0x7f9a0c0f5ca0 2024-02-15T18:36:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f9a144cd700 1 -- 172.21.15.42:0/4136268099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c10c350 con 0x7f9a0c0f5ca0 2024-02-15T18:36:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f99feffd700 1 -- 172.21.15.42:0/4136268099 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a08013ea0 con 0x7f9a0c0f5ca0 2024-02-15T18:36:09.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.519+0000 7f9a144cd700 1 -- 172.21.15.42:0/4136268099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c10c8e0 con 0x7f9a0c0f5ca0 2024-02-15T18:36:09.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.520+0000 7f9a144cd700 1 -- 172.21.15.42:0/4136268099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c046330 con 0x7f9a0c0f5ca0 2024-02-15T18:36:09.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.521+0000 7f99feffd700 1 -- 172.21.15.42:0/4136268099 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f9a0801a060 con 0x7f9a0c0f5ca0 2024-02-15T18:36:09.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.521+0000 7f99feffd700 1 --2- 172.21.15.42:0/4136268099 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f99f8060f20 0x7f99f80633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:09.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.521+0000 7f99feffd700 1 -- 172.21.15.42:0/4136268099 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f9a0807c4a0 con 0x7f9a0c0f5ca0 2024-02-15T18:36:09.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.521+0000 7f9a12269700 1 --2- 172.21.15.42:0/4136268099 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f99f8060f20 0x7f99f80633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:09.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.522+0000 7f9a12269700 1 --2- 172.21.15.42:0/4136268099 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f99f8060f20 0x7f99f80633e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f9a00009d60 tx=0x7f9a00009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:09.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.525+0000 7f99feffd700 1 -- 172.21.15.42:0/4136268099 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9a08050180 con 0x7f9a0c0f5ca0 2024-02-15T18:36:09.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.738+0000 7f9a144cd700 1 -- 172.21.15.42:0/4136268099 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f9a0c0586e0 con 0x7f9a0c0f5ca0 2024-02-15T18:36:09.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.739+0000 7f99feffd700 1 -- 172.21.15.42:0/4136268099 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v28) v1 ==== 107+0+4586 (secure 0 0 0) 0x7f9a0804fd70 con 0x7f9a0c0f5ca0 2024-02-15T18:36:09.730 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:09.730 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14450,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.42:6829/2580627796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":2580627796},{"type":"v1","addr":"172.21.15.42:6829","nonce":2580627796}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34356,"name":"cephfs.smithi042.vtwvlo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2962563293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2962563293},{"type":"v1","addr":"172.21.15.42:6827","nonce":2962563293}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":44313,"name":"cephfs.smithi191.erlpxp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6827/1725194059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1725194059},{"type":"v1","addr":"172.21.15.191:6827","nonce":1725194059}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:01.852374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24263":{"gid":24263,"name":"cephfs.smithi191.icwcnb","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.191:6825/1971151000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1971151000},{"type":"v1","addr":"172.21.15.191:6825","nonce":1971151000}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:09.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.741+0000 7f9a144cd700 1 -- 172.21.15.42:0/4136268099 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f99f8060f20 msgr2=0x7f99f80633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.741+0000 7f9a144cd700 1 --2- 172.21.15.42:0/4136268099 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f99f8060f20 0x7f99f80633e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f9a00009d60 tx=0x7f9a00009b30).stop 2024-02-15T18:36:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.741+0000 7f9a144cd700 1 -- 172.21.15.42:0/4136268099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a0c0f5ca0 msgr2=0x7f9a0c10bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.741+0000 7f9a144cd700 1 --2- 172.21.15.42:0/4136268099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a0c0f5ca0 0x7f9a0c10bac0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f9a0800c790 tx=0x7f9a08007d70).stop 2024-02-15T18:36:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.741+0000 7f9a144cd700 1 -- 172.21.15.42:0/4136268099 shutdown_connections 2024-02-15T18:36:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.741+0000 7f9a144cd700 1 --2- 172.21.15.42:0/4136268099 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f99f8060f20 0x7f99f80633e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.741+0000 7f9a144cd700 1 --2- 172.21.15.42:0/4136268099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0c0f5390 0x7f9a0c05fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.741+0000 7f9a144cd700 1 --2- 172.21.15.42:0/4136268099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a0c0f5ca0 0x7f9a0c10bac0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.741+0000 7f9a144cd700 1 -- 172.21.15.42:0/4136268099 >> 172.21.15.42:0/4136268099 conn(0x7f9a0c0f0b70 msgr2=0x7f9a0c0fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.741+0000 7f9a144cd700 1 -- 172.21.15.42:0/4136268099 shutdown_connections 2024-02-15T18:36:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.741+0000 7f9a144cd700 1 -- 172.21.15.42:0/4136268099 wait complete. 2024-02-15T18:36:09.733 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 17 2024-02-15T18:36:09.744 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 17 2024-02-15T18:36:09.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-02-15T18:36:09.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.875+0000 7fdb913cb700 1 -- 172.21.15.42:0/3085186231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8c05fe30 msgr2=0x7fdb8c0f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:09.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.875+0000 7fdb913cb700 1 --2- 172.21.15.42:0/3085186231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8c05fe30 0x7fdb8c0f9110 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fdb80004660 tx=0x7fdb80009b30).stop 2024-02-15T18:36:09.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.875+0000 7fdb913cb700 1 -- 172.21.15.42:0/3085186231 shutdown_connections 2024-02-15T18:36:09.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.875+0000 7fdb913cb700 1 --2- 172.21.15.42:0/3085186231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb8c0f9600 0x7fdb8c0fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.875+0000 7fdb913cb700 1 --2- 172.21.15.42:0/3085186231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8c05fe30 0x7fdb8c0f9110 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.875+0000 7fdb913cb700 1 -- 172.21.15.42:0/3085186231 >> 172.21.15.42:0/3085186231 conn(0x7fdb8c0f0b70 msgr2=0x7fdb8c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:09.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.875+0000 7fdb913cb700 1 -- 172.21.15.42:0/3085186231 shutdown_connections 2024-02-15T18:36:09.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.875+0000 7fdb913cb700 1 -- 172.21.15.42:0/3085186231 wait complete. 2024-02-15T18:36:09.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.875+0000 7fdb913cb700 1 Processor -- start 2024-02-15T18:36:09.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.875+0000 7fdb913cb700 1 -- start start 2024-02-15T18:36:09.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb913cb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb8c05fe30 0x7fdb8c0f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:09.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb913cb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8c0f9600 0x7fdb8c0f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:09.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb913cb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0f6f60 con 0x7fdb8c0f9600 2024-02-15T18:36:09.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb913cb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0f5500 con 0x7fdb8c05fe30 2024-02-15T18:36:09.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb8a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8c0f9600 0x7fdb8c0f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:09.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb8a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8c0f9600 0x7fdb8c0f5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40050/0 (socket says 172.21.15.42:40050) 2024-02-15T18:36:09.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb8a7fc700 1 -- 172.21.15.42:0/99810944 learned_addr learned my addr 172.21.15.42:0/99810944 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:09.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb8affd700 1 --2- 172.21.15.42:0/99810944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb8c05fe30 0x7fdb8c0f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:09.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb8a7fc700 1 -- 172.21.15.42:0/99810944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb8c05fe30 msgr2=0x7fdb8c0f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:09.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb8a7fc700 1 --2- 172.21.15.42:0/99810944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb8c05fe30 0x7fdb8c0f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:09.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb8a7fc700 1 -- 172.21.15.42:0/99810944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb80005040 con 0x7fdb8c0f9600 2024-02-15T18:36:09.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb8a7fc700 1 --2- 172.21.15.42:0/99810944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8c0f9600 0x7fdb8c0f5010 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fdb7c0026d0 tx=0x7fdb7c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:09.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb737fe700 1 -- 172.21.15.42:0/99810944 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb7c0079c0 con 0x7fdb8c0f9600 2024-02-15T18:36:09.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb737fe700 1 -- 172.21.15.42:0/99810944 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fdb7c00f040 con 0x7fdb8c0f9600 2024-02-15T18:36:09.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb913cb700 1 -- 172.21.15.42:0/99810944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8c0f5780 con 0x7fdb8c0f9600 2024-02-15T18:36:09.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb737fe700 1 -- 172.21.15.42:0/99810944 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb7c0149f0 con 0x7fdb8c0f9600 2024-02-15T18:36:09.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.876+0000 7fdb913cb700 1 -- 172.21.15.42:0/99810944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c0f5d40 con 0x7fdb8c0f9600 2024-02-15T18:36:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.877+0000 7fdb913cb700 1 -- 172.21.15.42:0/99810944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8c027a00 con 0x7fdb8c0f9600 2024-02-15T18:36:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.878+0000 7fdb737fe700 1 -- 172.21.15.42:0/99810944 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fdb7c014b50 con 0x7fdb8c0f9600 2024-02-15T18:36:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.878+0000 7fdb737fe700 1 --2- 172.21.15.42:0/99810944 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdb74060f20 0x7fdb740633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.878+0000 7fdb737fe700 1 -- 172.21.15.42:0/99810944 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fdb7c07c120 con 0x7fdb8c0f9600 2024-02-15T18:36:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.878+0000 7fdb8affd700 1 --2- 172.21.15.42:0/99810944 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdb74060f20 0x7fdb740633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:09.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.879+0000 7fdb8affd700 1 --2- 172.21.15.42:0/99810944 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdb74060f20 0x7fdb740633e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fdb80004630 tx=0x7fdb80009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:09.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:09.882+0000 7fdb737fe700 1 -- 172.21.15.42:0/99810944 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdb7c04fee0 con 0x7fdb8c0f9600 2024-02-15T18:36:10.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.102+0000 7fdb913cb700 1 -- 172.21.15.42:0/99810944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7fdb8c0fed70 con 0x7fdb8c0f9600 2024-02-15T18:36:10.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.103+0000 7fdb737fe700 1 -- 172.21.15.42:0/99810944 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v28) v1 ==== 107+0+3807 (secure 0 0 0) 0x7fdb7c04fad0 con 0x7fdb8c0f9600 2024-02-15T18:36:10.094 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:10.094 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34356,"name":"cephfs.smithi042.vtwvlo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2962563293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2962563293},{"type":"v1","addr":"172.21.15.42:6827","nonce":2962563293}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":44313,"name":"cephfs.smithi191.erlpxp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6827/1725194059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1725194059},{"type":"v1","addr":"172.21.15.191:6827","nonce":1725194059}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:01.852374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24263":{"gid":24263,"name":"cephfs.smithi191.icwcnb","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.191:6825/1971151000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1971151000},{"type":"v1","addr":"172.21.15.191:6825","nonce":1971151000}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:10.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.105+0000 7fdb913cb700 1 -- 172.21.15.42:0/99810944 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdb74060f20 msgr2=0x7fdb740633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:10.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.105+0000 7fdb913cb700 1 --2- 172.21.15.42:0/99810944 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdb74060f20 0x7fdb740633e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fdb80004630 tx=0x7fdb80009b30).stop 2024-02-15T18:36:10.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.105+0000 7fdb913cb700 1 -- 172.21.15.42:0/99810944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8c0f9600 msgr2=0x7fdb8c0f5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:10.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.105+0000 7fdb913cb700 1 --2- 172.21.15.42:0/99810944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8c0f9600 0x7fdb8c0f5010 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fdb7c0026d0 tx=0x7fdb7c009de0).stop 2024-02-15T18:36:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.105+0000 7fdb913cb700 1 -- 172.21.15.42:0/99810944 shutdown_connections 2024-02-15T18:36:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.105+0000 7fdb913cb700 1 --2- 172.21.15.42:0/99810944 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fdb74060f20 0x7fdb740633e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.105+0000 7fdb913cb700 1 --2- 172.21.15.42:0/99810944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdb8c05fe30 0x7fdb8c0f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.105+0000 7fdb913cb700 1 --2- 172.21.15.42:0/99810944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8c0f9600 0x7fdb8c0f5010 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.105+0000 7fdb913cb700 1 -- 172.21.15.42:0/99810944 >> 172.21.15.42:0/99810944 conn(0x7fdb8c0f0b70 msgr2=0x7fdb8c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.105+0000 7fdb913cb700 1 -- 172.21.15.42:0/99810944 shutdown_connections 2024-02-15T18:36:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.105+0000 7fdb913cb700 1 -- 172.21.15.42:0/99810944 wait complete. 2024-02-15T18:36:10.097 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 18 2024-02-15T18:36:10.108 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 18 2024-02-15T18:36:10.109 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-02-15T18:36:10.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.240+0000 7fcf418b9700 1 -- 172.21.15.42:0/4016912001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3c0f7760 msgr2=0x7fcf3c0f7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:10.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.240+0000 7fcf418b9700 1 --2- 172.21.15.42:0/4016912001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3c0f7760 0x7fcf3c0f7bc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fcf30004660 tx=0x7fcf30009b30).stop 2024-02-15T18:36:10.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.240+0000 7fcf418b9700 1 -- 172.21.15.42:0/4016912001 shutdown_connections 2024-02-15T18:36:10.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.240+0000 7fcf418b9700 1 --2- 172.21.15.42:0/4016912001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3c0f7760 0x7fcf3c0f7bc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.240+0000 7fcf418b9700 1 --2- 172.21.15.42:0/4016912001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf3c0f64c0 0x7fcf3c0f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.240+0000 7fcf418b9700 1 -- 172.21.15.42:0/4016912001 >> 172.21.15.42:0/4016912001 conn(0x7fcf3c0f1c00 msgr2=0x7fcf3c0f4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:10.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.241+0000 7fcf418b9700 1 -- 172.21.15.42:0/4016912001 shutdown_connections 2024-02-15T18:36:10.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.241+0000 7fcf418b9700 1 -- 172.21.15.42:0/4016912001 wait complete. 2024-02-15T18:36:10.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.241+0000 7fcf418b9700 1 Processor -- start 2024-02-15T18:36:10.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.241+0000 7fcf418b9700 1 -- start start 2024-02-15T18:36:10.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.242+0000 7fcf418b9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3c0f64c0 0x7fcf3c06b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:10.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.242+0000 7fcf418b9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf3c0f7760 0x7fcf3c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:10.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.242+0000 7fcf418b9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c06bd70 con 0x7fcf3c0f64c0 2024-02-15T18:36:10.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.242+0000 7fcf418b9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c06a310 con 0x7fcf3c0f7760 2024-02-15T18:36:10.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.242+0000 7fcf3affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3c0f64c0 0x7fcf3c06b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:10.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.242+0000 7fcf3affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3c0f64c0 0x7fcf3c06b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40072/0 (socket says 172.21.15.42:40072) 2024-02-15T18:36:10.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.242+0000 7fcf3affd700 1 -- 172.21.15.42:0/1026249520 learned_addr learned my addr 172.21.15.42:0/1026249520 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:10.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.242+0000 7fcf3a7fc700 1 --2- 172.21.15.42:0/1026249520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf3c0f7760 0x7fcf3c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:10.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.242+0000 7fcf3affd700 1 -- 172.21.15.42:0/1026249520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf3c0f7760 msgr2=0x7fcf3c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:10.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.242+0000 7fcf3affd700 1 --2- 172.21.15.42:0/1026249520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf3c0f7760 0x7fcf3c069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.242+0000 7fcf3affd700 1 -- 172.21.15.42:0/1026249520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf30005040 con 0x7fcf3c0f64c0 2024-02-15T18:36:10.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.242+0000 7fcf3affd700 1 --2- 172.21.15.42:0/1026249520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3c0f64c0 0x7fcf3c06b7c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fcf2c002700 tx=0x7fcf2c009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:10.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.242+0000 7fcf237fe700 1 -- 172.21.15.42:0/1026249520 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf2c0079c0 con 0x7fcf3c0f64c0 2024-02-15T18:36:10.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.243+0000 7fcf418b9700 1 -- 172.21.15.42:0/1026249520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3c06a590 con 0x7fcf3c0f64c0 2024-02-15T18:36:10.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.243+0000 7fcf237fe700 1 -- 172.21.15.42:0/1026249520 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fcf2c00f040 con 0x7fcf3c0f64c0 2024-02-15T18:36:10.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.243+0000 7fcf237fe700 1 -- 172.21.15.42:0/1026249520 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf2c014940 con 0x7fcf3c0f64c0 2024-02-15T18:36:10.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.243+0000 7fcf418b9700 1 -- 172.21.15.42:0/1026249520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3c06ab50 con 0x7fcf3c0f64c0 2024-02-15T18:36:10.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.244+0000 7fcf418b9700 1 -- 172.21.15.42:0/1026249520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3c027a00 con 0x7fcf3c0f64c0 2024-02-15T18:36:10.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.244+0000 7fcf237fe700 1 -- 172.21.15.42:0/1026249520 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fcf2c01b060 con 0x7fcf3c0f64c0 2024-02-15T18:36:10.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.244+0000 7fcf237fe700 1 --2- 172.21.15.42:0/1026249520 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf24060f20 0x7fcf240633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:10.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.244+0000 7fcf237fe700 1 -- 172.21.15.42:0/1026249520 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fcf2c07c140 con 0x7fcf3c0f64c0 2024-02-15T18:36:10.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.244+0000 7fcf3a7fc700 1 --2- 172.21.15.42:0/1026249520 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf24060f20 0x7fcf240633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:10.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.245+0000 7fcf3a7fc700 1 --2- 172.21.15.42:0/1026249520 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf24060f20 0x7fcf240633e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fcf30004630 tx=0x7fcf30009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:10.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.248+0000 7fcf237fe700 1 -- 172.21.15.42:0/1026249520 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcf2c04fe80 con 0x7fcf3c0f64c0 2024-02-15T18:36:10.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:09.383191+0000 mon.smithi042 (mon.0) 784 : audit [DBG] from='client.? 172.21.15.42:0/2948743091' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-02-15T18:36:10.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:10 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:09.740205+0000 mon.smithi042 (mon.0) 785 : audit [DBG] from='client.? 172.21.15.42:0/4136268099' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-02-15T18:36:10.440 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:09.383191+0000 mon.smithi042 (mon.0) 784 : audit [DBG] from='client.? 172.21.15.42:0/2948743091' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-02-15T18:36:10.440 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:10 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:09.740205+0000 mon.smithi042 (mon.0) 785 : audit [DBG] from='client.? 172.21.15.42:0/4136268099' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-02-15T18:36:10.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.468+0000 7fcf418b9700 1 -- 172.21.15.42:0/1026249520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fcf3c0fed90 con 0x7fcf3c0f64c0 2024-02-15T18:36:10.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.469+0000 7fcf237fe700 1 -- 172.21.15.42:0/1026249520 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v28) v1 ==== 107+0+4586 (secure 0 0 0) 0x7fcf2c04fa70 con 0x7fcf3c0f64c0 2024-02-15T18:36:10.460 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:10.460 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34356,"name":"cephfs.smithi042.vtwvlo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2962563293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2962563293},{"type":"v1","addr":"172.21.15.42:6827","nonce":2962563293}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34360,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1562993914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1562993914},{"type":"v1","addr":"172.21.15.42:6829","nonce":1562993914}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44313,"name":"cephfs.smithi191.erlpxp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6827/1725194059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1725194059},{"type":"v1","addr":"172.21.15.191:6827","nonce":1725194059}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:01.852374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24263":{"gid":24263,"name":"cephfs.smithi191.icwcnb","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.15.191:6825/1971151000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":1971151000},{"type":"v1","addr":"172.21.15.191:6825","nonce":1971151000}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:10.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.471+0000 7fcf418b9700 1 -- 172.21.15.42:0/1026249520 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf24060f20 msgr2=0x7fcf240633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:10.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.471+0000 7fcf418b9700 1 --2- 172.21.15.42:0/1026249520 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf24060f20 0x7fcf240633e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fcf30004630 tx=0x7fcf30009b30).stop 2024-02-15T18:36:10.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.471+0000 7fcf418b9700 1 -- 172.21.15.42:0/1026249520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3c0f64c0 msgr2=0x7fcf3c06b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:10.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.471+0000 7fcf418b9700 1 --2- 172.21.15.42:0/1026249520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3c0f64c0 0x7fcf3c06b7c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fcf2c002700 tx=0x7fcf2c009d80).stop 2024-02-15T18:36:10.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.471+0000 7fcf418b9700 1 -- 172.21.15.42:0/1026249520 shutdown_connections 2024-02-15T18:36:10.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.471+0000 7fcf418b9700 1 --2- 172.21.15.42:0/1026249520 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf24060f20 0x7fcf240633e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.471+0000 7fcf418b9700 1 --2- 172.21.15.42:0/1026249520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf3c0f64c0 0x7fcf3c06b7c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.471+0000 7fcf418b9700 1 --2- 172.21.15.42:0/1026249520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf3c0f7760 0x7fcf3c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.471+0000 7fcf418b9700 1 -- 172.21.15.42:0/1026249520 >> 172.21.15.42:0/1026249520 conn(0x7fcf3c0f1c00 msgr2=0x7fcf3c0fb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:10.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.471+0000 7fcf418b9700 1 -- 172.21.15.42:0/1026249520 shutdown_connections 2024-02-15T18:36:10.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.471+0000 7fcf418b9700 1 -- 172.21.15.42:0/1026249520 wait complete. 2024-02-15T18:36:10.464 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-02-15T18:36:10.474 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-02-15T18:36:10.474 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-02-15T18:36:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.605+0000 7fce899ed700 1 -- 172.21.15.42:0/327421928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce840f5ca0 msgr2=0x7fce840f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.605+0000 7fce899ed700 1 --2- 172.21.15.42:0/327421928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce840f5ca0 0x7fce840f66d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fce74004660 tx=0x7fce74009b30).stop 2024-02-15T18:36:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.605+0000 7fce899ed700 1 -- 172.21.15.42:0/327421928 shutdown_connections 2024-02-15T18:36:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.605+0000 7fce899ed700 1 --2- 172.21.15.42:0/327421928 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce840f5ca0 0x7fce840f66d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.605+0000 7fce899ed700 1 --2- 172.21.15.42:0/327421928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce840f5390 0x7fce840f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.605+0000 7fce899ed700 1 -- 172.21.15.42:0/327421928 >> 172.21.15.42:0/327421928 conn(0x7fce840f0b70 msgr2=0x7fce840f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.605+0000 7fce899ed700 1 -- 172.21.15.42:0/327421928 shutdown_connections 2024-02-15T18:36:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.605+0000 7fce899ed700 1 -- 172.21.15.42:0/327421928 wait complete. 2024-02-15T18:36:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce899ed700 1 Processor -- start 2024-02-15T18:36:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce899ed700 1 -- start start 2024-02-15T18:36:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce899ed700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce840f5390 0x7fce8405fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:10.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce899ed700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce840f5ca0 0x7fce8410baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:10.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce899ed700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8410bfe0 con 0x7fce840f5ca0 2024-02-15T18:36:10.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce899ed700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8410c120 con 0x7fce840f5390 2024-02-15T18:36:10.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce827fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce840f5ca0 0x7fce8410baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:10.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce827fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce840f5ca0 0x7fce8410baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40090/0 (socket says 172.21.15.42:40090) 2024-02-15T18:36:10.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce827fc700 1 -- 172.21.15.42:0/4056485654 learned_addr learned my addr 172.21.15.42:0/4056485654 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:10.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce82ffd700 1 --2- 172.21.15.42:0/4056485654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce840f5390 0x7fce8405fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:10.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce827fc700 1 -- 172.21.15.42:0/4056485654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce840f5390 msgr2=0x7fce8405fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:10.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce827fc700 1 --2- 172.21.15.42:0/4056485654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce840f5390 0x7fce8405fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce827fc700 1 -- 172.21.15.42:0/4056485654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74005040 con 0x7fce840f5ca0 2024-02-15T18:36:10.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce827fc700 1 --2- 172.21.15.42:0/4056485654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce840f5ca0 0x7fce8410baf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fce74009f50 tx=0x7fce74009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:10.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce6b7fe700 1 -- 172.21.15.42:0/4056485654 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce74018070 con 0x7fce840f5ca0 2024-02-15T18:36:10.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce6b7fe700 1 -- 172.21.15.42:0/4056485654 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fce74007ba0 con 0x7fce840f5ca0 2024-02-15T18:36:10.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.606+0000 7fce899ed700 1 -- 172.21.15.42:0/4056485654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8410c320 con 0x7fce840f5ca0 2024-02-15T18:36:10.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.607+0000 7fce6b7fe700 1 -- 172.21.15.42:0/4056485654 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce7401c420 con 0x7fce840f5ca0 2024-02-15T18:36:10.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.607+0000 7fce899ed700 1 -- 172.21.15.42:0/4056485654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8410c850 con 0x7fce840f5ca0 2024-02-15T18:36:10.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.607+0000 7fce899ed700 1 -- 172.21.15.42:0/4056485654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce84027a00 con 0x7fce840f5ca0 2024-02-15T18:36:10.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.608+0000 7fce6b7fe700 1 -- 172.21.15.42:0/4056485654 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fce7401d420 con 0x7fce840f5ca0 2024-02-15T18:36:10.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.608+0000 7fce6b7fe700 1 --2- 172.21.15.42:0/4056485654 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fce6c060f20 0x7fce6c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:10.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.608+0000 7fce6b7fe700 1 -- 172.21.15.42:0/4056485654 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fce7407df10 con 0x7fce840f5ca0 2024-02-15T18:36:10.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.608+0000 7fce82ffd700 1 --2- 172.21.15.42:0/4056485654 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fce6c060f20 0x7fce6c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:10.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.609+0000 7fce82ffd700 1 --2- 172.21.15.42:0/4056485654 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fce6c060f20 0x7fce6c0633e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fce78004c40 tx=0x7fce7800b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:10.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.613+0000 7fce6b7fe700 1 -- 172.21.15.42:0/4056485654 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fce74051bf0 con 0x7fce840f5ca0 2024-02-15T18:36:10.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.829+0000 7fce899ed700 1 -- 172.21.15.42:0/4056485654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7fce840586e0 con 0x7fce840f5ca0 2024-02-15T18:36:10.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.830+0000 7fce6b7fe700 1 -- 172.21.15.42:0/4056485654 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v28) v1 ==== 107+0+3796 (secure 0 0 0) 0x7fce740517e0 con 0x7fce840f5ca0 2024-02-15T18:36:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:10.821 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34356,"name":"cephfs.smithi042.vtwvlo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2962563293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2962563293},{"type":"v1","addr":"172.21.15.42:6827","nonce":2962563293}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":34360,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1562993914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1562993914},{"type":"v1","addr":"172.21.15.42:6829","nonce":1562993914}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44313,"name":"cephfs.smithi191.erlpxp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6827/1725194059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1725194059},{"type":"v1","addr":"172.21.15.191:6827","nonce":1725194059}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:31.994247+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:10.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.832+0000 7fce899ed700 1 -- 172.21.15.42:0/4056485654 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fce6c060f20 msgr2=0x7fce6c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:10.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.832+0000 7fce899ed700 1 --2- 172.21.15.42:0/4056485654 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fce6c060f20 0x7fce6c0633e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fce78004c40 tx=0x7fce7800b040).stop 2024-02-15T18:36:10.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.832+0000 7fce899ed700 1 -- 172.21.15.42:0/4056485654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce840f5ca0 msgr2=0x7fce8410baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:10.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.832+0000 7fce899ed700 1 --2- 172.21.15.42:0/4056485654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce840f5ca0 0x7fce8410baf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fce74009f50 tx=0x7fce74009b30).stop 2024-02-15T18:36:10.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.832+0000 7fce899ed700 1 -- 172.21.15.42:0/4056485654 shutdown_connections 2024-02-15T18:36:10.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.832+0000 7fce899ed700 1 --2- 172.21.15.42:0/4056485654 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fce6c060f20 0x7fce6c0633e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.832+0000 7fce899ed700 1 --2- 172.21.15.42:0/4056485654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce840f5390 0x7fce8405fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.832+0000 7fce899ed700 1 --2- 172.21.15.42:0/4056485654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce840f5ca0 0x7fce8410baf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.832+0000 7fce899ed700 1 -- 172.21.15.42:0/4056485654 >> 172.21.15.42:0/4056485654 conn(0x7fce840f0b70 msgr2=0x7fce840fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:10.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.832+0000 7fce899ed700 1 -- 172.21.15.42:0/4056485654 shutdown_connections 2024-02-15T18:36:10.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.832+0000 7fce899ed700 1 -- 172.21.15.42:0/4056485654 wait complete. 2024-02-15T18:36:10.825 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 20 2024-02-15T18:36:10.835 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-02-15T18:36:10.835 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-02-15T18:36:10.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.966+0000 7f38266b8700 1 -- 172.21.15.42:0/1193605842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38200f5ca0 msgr2=0x7f38200f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:10.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.966+0000 7f38266b8700 1 --2- 172.21.15.42:0/1193605842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38200f5ca0 0x7f38200f66d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f3814004660 tx=0x7f3814009b30).stop 2024-02-15T18:36:10.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.966+0000 7f38266b8700 1 -- 172.21.15.42:0/1193605842 shutdown_connections 2024-02-15T18:36:10.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.966+0000 7f38266b8700 1 --2- 172.21.15.42:0/1193605842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38200f5ca0 0x7f38200f66d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.966+0000 7f38266b8700 1 --2- 172.21.15.42:0/1193605842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38200f5390 0x7f38200f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.966+0000 7f38266b8700 1 -- 172.21.15.42:0/1193605842 >> 172.21.15.42:0/1193605842 conn(0x7f38200f0b70 msgr2=0x7f38200f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:10.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.966+0000 7f38266b8700 1 -- 172.21.15.42:0/1193605842 shutdown_connections 2024-02-15T18:36:10.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.966+0000 7f38266b8700 1 -- 172.21.15.42:0/1193605842 wait complete. 2024-02-15T18:36:10.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.967+0000 7f38266b8700 1 Processor -- start 2024-02-15T18:36:10.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.967+0000 7f38266b8700 1 -- start start 2024-02-15T18:36:10.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.967+0000 7f38266b8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38200f5390 0x7f38200f9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:10.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.967+0000 7f38266b8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38200f5ca0 0x7f38200f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:10.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.967+0000 7f38266b8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38200fa180 con 0x7f38200f5ca0 2024-02-15T18:36:10.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.967+0000 7f38266b8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38200f8720 con 0x7f38200f5390 2024-02-15T18:36:10.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.967+0000 7f381f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38200f5ca0 0x7f38200f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:10.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.967+0000 7f381f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38200f5ca0 0x7f38200f8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40114/0 (socket says 172.21.15.42:40114) 2024-02-15T18:36:10.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.967+0000 7f381f7fe700 1 -- 172.21.15.42:0/1087969726 learned_addr learned my addr 172.21.15.42:0/1087969726 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:10.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.967+0000 7f381ffff700 1 --2- 172.21.15.42:0/1087969726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38200f5390 0x7f38200f9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:10.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.967+0000 7f381f7fe700 1 -- 172.21.15.42:0/1087969726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38200f5390 msgr2=0x7f38200f9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:10.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.967+0000 7f381f7fe700 1 --2- 172.21.15.42:0/1087969726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38200f5390 0x7f38200f9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:10.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.967+0000 7f381f7fe700 1 -- 172.21.15.42:0/1087969726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3814005040 con 0x7f38200f5ca0 2024-02-15T18:36:10.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.967+0000 7f381f7fe700 1 --2- 172.21.15.42:0/1087969726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38200f5ca0 0x7f38200f8230 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f3814009f50 tx=0x7f3814009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:10.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.968+0000 7f381cff9700 1 -- 172.21.15.42:0/1087969726 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3814018070 con 0x7f38200f5ca0 2024-02-15T18:36:10.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.968+0000 7f381cff9700 1 -- 172.21.15.42:0/1087969726 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f3814007d70 con 0x7f38200f5ca0 2024-02-15T18:36:10.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.968+0000 7f38266b8700 1 -- 172.21.15.42:0/1087969726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38200f8940 con 0x7f38200f5ca0 2024-02-15T18:36:10.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.968+0000 7f381cff9700 1 -- 172.21.15.42:0/1087969726 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f381401c620 con 0x7f38200f5ca0 2024-02-15T18:36:10.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.968+0000 7f38266b8700 1 -- 172.21.15.42:0/1087969726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38200f8ea0 con 0x7f38200f5ca0 2024-02-15T18:36:10.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.969+0000 7f38266b8700 1 -- 172.21.15.42:0/1087969726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3820046330 con 0x7f38200f5ca0 2024-02-15T18:36:10.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.969+0000 7f381cff9700 1 -- 172.21.15.42:0/1087969726 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f381401c8e0 con 0x7f38200f5ca0 2024-02-15T18:36:10.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.969+0000 7f381cff9700 1 --2- 172.21.15.42:0/1087969726 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3808060f20 0x7f38080633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:10.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.969+0000 7f381cff9700 1 -- 172.21.15.42:0/1087969726 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f381407e1e0 con 0x7f38200f5ca0 2024-02-15T18:36:10.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.969+0000 7f381ffff700 1 --2- 172.21.15.42:0/1087969726 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3808060f20 0x7f38080633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:10.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.970+0000 7f381ffff700 1 --2- 172.21.15.42:0/1087969726 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3808060f20 0x7f38080633e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f38100041d0 tx=0x7f3810009040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:10.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:10.974+0000 7f381cff9700 1 -- 172.21.15.42:0/1087969726 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3814051f20 con 0x7f38200f5ca0 2024-02-15T18:36:11.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.195+0000 7f38266b8700 1 -- 172.21.15.42:0/1087969726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f382010f390 con 0x7f38200f5ca0 2024-02-15T18:36:11.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.195+0000 7f381cff9700 1 -- 172.21.15.42:0/1087969726 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v28) v1 ==== 107+0+3807 (secure 0 0 0) 0x7f3814051b10 con 0x7f38200f5ca0 2024-02-15T18:36:11.186 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:11.186 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34360,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1562993914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1562993914},{"type":"v1","addr":"172.21.15.42:6829","nonce":1562993914}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44313,"name":"cephfs.smithi191.erlpxp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6827/1725194059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1725194059},{"type":"v1","addr":"172.21.15.191:6827","nonce":1725194059}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:31.997905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34356":{"gid":34356,"name":"cephfs.smithi042.vtwvlo","rank":0,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.42:6827/2962563293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2962563293},{"type":"v1","addr":"172.21.15.42:6827","nonce":2962563293}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:11.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.197+0000 7f38266b8700 1 -- 172.21.15.42:0/1087969726 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3808060f20 msgr2=0x7f38080633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:11.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.197+0000 7f38266b8700 1 --2- 172.21.15.42:0/1087969726 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3808060f20 0x7f38080633e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f38100041d0 tx=0x7f3810009040).stop 2024-02-15T18:36:11.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.197+0000 7f38266b8700 1 -- 172.21.15.42:0/1087969726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38200f5ca0 msgr2=0x7f38200f8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:11.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.197+0000 7f38266b8700 1 --2- 172.21.15.42:0/1087969726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38200f5ca0 0x7f38200f8230 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f3814009f50 tx=0x7f3814009b30).stop 2024-02-15T18:36:11.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.197+0000 7f38266b8700 1 -- 172.21.15.42:0/1087969726 shutdown_connections 2024-02-15T18:36:11.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.197+0000 7f38266b8700 1 --2- 172.21.15.42:0/1087969726 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3808060f20 0x7f38080633e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.197+0000 7f38266b8700 1 --2- 172.21.15.42:0/1087969726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38200f5390 0x7f38200f9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.197+0000 7f38266b8700 1 --2- 172.21.15.42:0/1087969726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38200f5ca0 0x7f38200f8230 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.197+0000 7f38266b8700 1 -- 172.21.15.42:0/1087969726 >> 172.21.15.42:0/1087969726 conn(0x7f38200f0b70 msgr2=0x7f382005f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:11.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.198+0000 7f38266b8700 1 -- 172.21.15.42:0/1087969726 shutdown_connections 2024-02-15T18:36:11.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.198+0000 7f38266b8700 1 -- 172.21.15.42:0/1087969726 wait complete. 2024-02-15T18:36:11.190 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 21 2024-02-15T18:36:11.199 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:10.082906+0000 mon.smithi042 (mon.0) 786 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:11.200 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:10.104075+0000 mon.smithi042 (mon.0) 787 : audit [DBG] from='client.? 172.21.15.42:0/99810944' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-02-15T18:36:11.200 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:10.398765+0000 mon.smithi042 (mon.0) 788 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:11.200 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:10.399547+0000 mon.smithi042 (mon.0) 789 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:36:11.200 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cephadm 2024-02-15T18:36:10.400125+0000 mgr.smithi042.kfoihb (mgr.24435) 483 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:36:11.200 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:10.403587+0000 mon.smithi042 (mon.0) 790 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:36:11.200 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:10.469910+0000 mon.smithi042 (mon.0) 791 : audit [DBG] from='client.? 172.21.15.42:0/1026249520' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-15T18:36:11.200 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:36:10.817798+0000 mgr.smithi042.kfoihb (mgr.24435) 484 : cluster [DBG] pgmap v370: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:36:11.200 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:11 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:10.830961+0000 mon.smithi042 (mon.0) 792 : audit [DBG] from='client.? 172.21.15.42:0/4056485654' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-15T18:36:11.201 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-02-15T18:36:11.201 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-02-15T18:36:11.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.332+0000 7f2dca401700 1 -- 172.21.15.42:0/412009980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dc40f5ca0 msgr2=0x7f2dc40f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:11.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.332+0000 7f2dca401700 1 --2- 172.21.15.42:0/412009980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dc40f5ca0 0x7f2dc40f66d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f2db8004660 tx=0x7f2db8009b30).stop 2024-02-15T18:36:11.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.332+0000 7f2dca401700 1 -- 172.21.15.42:0/412009980 shutdown_connections 2024-02-15T18:36:11.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.332+0000 7f2dca401700 1 --2- 172.21.15.42:0/412009980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dc40f5ca0 0x7f2dc40f66d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.332+0000 7f2dca401700 1 --2- 172.21.15.42:0/412009980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc40f5390 0x7f2dc40f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.332+0000 7f2dca401700 1 -- 172.21.15.42:0/412009980 >> 172.21.15.42:0/412009980 conn(0x7f2dc40f0b70 msgr2=0x7f2dc40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:11.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.332+0000 7f2dca401700 1 -- 172.21.15.42:0/412009980 shutdown_connections 2024-02-15T18:36:11.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.332+0000 7f2dca401700 1 -- 172.21.15.42:0/412009980 wait complete. 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.333+0000 7f2dca401700 1 Processor -- start 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.333+0000 7f2dca401700 1 -- start start 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.333+0000 7f2dca401700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc40f5390 0x7f2dc4106cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.333+0000 7f2dca401700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dc40f5ca0 0x7f2dc41071c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.333+0000 7f2dca401700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc41077c0 con 0x7f2dc40f5ca0 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.333+0000 7f2dca401700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc4107900 con 0x7f2dc40f5390 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.333+0000 7f2dc37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dc40f5ca0 0x7f2dc41071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.333+0000 7f2dc37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dc40f5ca0 0x7f2dc41071c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40142/0 (socket says 172.21.15.42:40142) 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.333+0000 7f2dc37fe700 1 -- 172.21.15.42:0/4144351121 learned_addr learned my addr 172.21.15.42:0/4144351121 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.333+0000 7f2dc3fff700 1 --2- 172.21.15.42:0/4144351121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc40f5390 0x7f2dc4106cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.333+0000 7f2dc37fe700 1 -- 172.21.15.42:0/4144351121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc40f5390 msgr2=0x7f2dc4106cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.333+0000 7f2dc37fe700 1 --2- 172.21.15.42:0/4144351121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc40f5390 0x7f2dc4106cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.333+0000 7f2dc37fe700 1 -- 172.21.15.42:0/4144351121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db8005040 con 0x7f2dc40f5ca0 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.333+0000 7f2dc3fff700 1 --2- 172.21.15.42:0/4144351121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc40f5390 0x7f2dc4106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.334+0000 7f2dc37fe700 1 --2- 172.21.15.42:0/4144351121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dc40f5ca0 0x7f2dc41071c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f2db8009f50 tx=0x7f2db8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:11.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.334+0000 7f2dc0ff9700 1 -- 172.21.15.42:0/4144351121 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2db8018070 con 0x7f2dc40f5ca0 2024-02-15T18:36:11.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.334+0000 7f2dc0ff9700 1 -- 172.21.15.42:0/4144351121 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f2db8007ba0 con 0x7f2dc40f5ca0 2024-02-15T18:36:11.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.334+0000 7f2dc0ff9700 1 -- 172.21.15.42:0/4144351121 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2db801c620 con 0x7f2dc40f5ca0 2024-02-15T18:36:11.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.334+0000 7f2dca401700 1 -- 172.21.15.42:0/4144351121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc410c2b0 con 0x7f2dc40f5ca0 2024-02-15T18:36:11.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.334+0000 7f2dca401700 1 -- 172.21.15.42:0/4144351121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc410c790 con 0x7f2dc40f5ca0 2024-02-15T18:36:11.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.335+0000 7f2dca401700 1 -- 172.21.15.42:0/4144351121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc40fdc90 con 0x7f2dc40f5ca0 2024-02-15T18:36:11.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.335+0000 7f2dc0ff9700 1 -- 172.21.15.42:0/4144351121 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f2db801d420 con 0x7f2dc40f5ca0 2024-02-15T18:36:11.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.335+0000 7f2dc0ff9700 1 --2- 172.21.15.42:0/4144351121 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2dac060f20 0x7f2dac0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:11.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.336+0000 7f2dc0ff9700 1 -- 172.21.15.42:0/4144351121 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f2db807e010 con 0x7f2dc40f5ca0 2024-02-15T18:36:11.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.336+0000 7f2dc3fff700 1 --2- 172.21.15.42:0/4144351121 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2dac060f20 0x7f2dac0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:11.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.336+0000 7f2dc3fff700 1 --2- 172.21.15.42:0/4144351121 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2dac060f20 0x7f2dac0633e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f2dc41081c0 tx=0x7f2db4009c80).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:11.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.340+0000 7f2dc0ff9700 1 -- 172.21.15.42:0/4144351121 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2db804e340 con 0x7f2dc40f5ca0 2024-02-15T18:36:11.440 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:10.082906+0000 mon.smithi042 (mon.0) 786 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-15T18:36:11.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:10.104075+0000 mon.smithi042 (mon.0) 787 : audit [DBG] from='client.? 172.21.15.42:0/99810944' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-02-15T18:36:11.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:10.398765+0000 mon.smithi042 (mon.0) 788 : audit [DBG] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-15T18:36:11.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:10.399547+0000 mon.smithi042 (mon.0) 789 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-15T18:36:11.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cephadm 2024-02-15T18:36:10.400125+0000 mgr.smithi042.kfoihb (mgr.24435) 483 : cephadm [WRN] unable to calc client keyring client.admin placement PlacementSpec(label='_admin'): Cannot place : No matching hosts for label _admin 2024-02-15T18:36:11.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:10.403587+0000 mon.smithi042 (mon.0) 790 : audit [INF] from='mgr.24435 172.21.15.42:0/2352022149' entity='mgr.smithi042.kfoihb' 2024-02-15T18:36:11.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:10.469910+0000 mon.smithi042 (mon.0) 791 : audit [DBG] from='client.? 172.21.15.42:0/1026249520' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-15T18:36:11.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:36:10.817798+0000 mgr.smithi042.kfoihb (mgr.24435) 484 : cluster [DBG] pgmap v370: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-02-15T18:36:11.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:11 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:10.830961+0000 mon.smithi042 (mon.0) 792 : audit [DBG] from='client.? 172.21.15.42:0/4056485654' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-15T18:36:11.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.553+0000 7f2dca401700 1 -- 172.21.15.42:0/4144351121 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f2dc410d2e0 con 0x7f2dc40f5ca0 2024-02-15T18:36:11.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.553+0000 7f2dc0ff9700 1 -- 172.21.15.42:0/4144351121 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v28) v1 ==== 107+0+4590 (secure 0 0 0) 0x7f2db8051960 con 0x7f2dc40f5ca0 2024-02-15T18:36:11.544 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:11.544 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34360,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1562993914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1562993914},{"type":"v1","addr":"172.21.15.42:6829","nonce":1562993914}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44313,"name":"cephfs.smithi191.erlpxp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6827/1725194059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1725194059},{"type":"v1","addr":"172.21.15.191:6827","nonce":1725194059}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":44343,"name":"cephfs.smithi191.icwcnb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6825/540280968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":540280968},{"type":"v1","addr":"172.21.15.191:6825","nonce":540280968}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:41.996349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34356":{"gid":34356,"name":"cephfs.smithi042.vtwvlo","rank":0,"incarnation":21,"state":"up:reconnect","state_seq":10,"addr":"172.21.15.42:6827/2962563293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2962563293},{"type":"v1","addr":"172.21.15.42:6827","nonce":2962563293}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:11.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.555+0000 7f2dca401700 1 -- 172.21.15.42:0/4144351121 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2dac060f20 msgr2=0x7f2dac0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:11.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.555+0000 7f2dca401700 1 --2- 172.21.15.42:0/4144351121 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2dac060f20 0x7f2dac0633e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f2dc41081c0 tx=0x7f2db4009c80).stop 2024-02-15T18:36:11.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.555+0000 7f2dca401700 1 -- 172.21.15.42:0/4144351121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dc40f5ca0 msgr2=0x7f2dc41071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:11.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.555+0000 7f2dca401700 1 --2- 172.21.15.42:0/4144351121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dc40f5ca0 0x7f2dc41071c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f2db8009f50 tx=0x7f2db8009b30).stop 2024-02-15T18:36:11.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.555+0000 7f2dca401700 1 -- 172.21.15.42:0/4144351121 shutdown_connections 2024-02-15T18:36:11.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.555+0000 7f2dca401700 1 --2- 172.21.15.42:0/4144351121 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2dac060f20 0x7f2dac0633e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.555+0000 7f2dca401700 1 --2- 172.21.15.42:0/4144351121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dc40f5390 0x7f2dc4106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.555+0000 7f2dca401700 1 --2- 172.21.15.42:0/4144351121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dc40f5ca0 0x7f2dc41071c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.555+0000 7f2dca401700 1 -- 172.21.15.42:0/4144351121 >> 172.21.15.42:0/4144351121 conn(0x7f2dc40f0b70 msgr2=0x7f2dc405f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:11.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.555+0000 7f2dca401700 1 -- 172.21.15.42:0/4144351121 shutdown_connections 2024-02-15T18:36:11.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.555+0000 7f2dca401700 1 -- 172.21.15.42:0/4144351121 wait complete. 2024-02-15T18:36:11.547 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 22 2024-02-15T18:36:11.558 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-02-15T18:36:11.558 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-02-15T18:36:11.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.689+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2106771176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba405fe30 msgr2=0x7f3ba40f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:11.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.689+0000 7f3ba9e6b700 1 --2- 172.21.15.42:0/2106771176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba405fe30 0x7f3ba40f9110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f3b94004660 tx=0x7f3b94009b30).stop 2024-02-15T18:36:11.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.689+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2106771176 shutdown_connections 2024-02-15T18:36:11.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.689+0000 7f3ba9e6b700 1 --2- 172.21.15.42:0/2106771176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ba40f9600 0x7f3ba40fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.689+0000 7f3ba9e6b700 1 --2- 172.21.15.42:0/2106771176 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba405fe30 0x7f3ba40f9110 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.689+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2106771176 >> 172.21.15.42:0/2106771176 conn(0x7f3ba40f0b70 msgr2=0x7f3ba40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.689+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2106771176 shutdown_connections 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.689+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2106771176 wait complete. 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.689+0000 7f3ba9e6b700 1 Processor -- start 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.690+0000 7f3ba9e6b700 1 -- start start 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.690+0000 7f3ba9e6b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ba405fe30 0x7f3ba4106c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.690+0000 7f3ba9e6b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba40f9600 0x7f3ba4107180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.690+0000 7f3ba9e6b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba4107780 con 0x7f3ba40f9600 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.690+0000 7f3ba9e6b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba41078f0 con 0x7f3ba405fe30 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.690+0000 7f3ba2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba40f9600 0x7f3ba4107180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.690+0000 7f3ba2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba40f9600 0x7f3ba4107180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43276/0 (socket says 172.21.15.42:43276) 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.690+0000 7f3ba2ffd700 1 -- 172.21.15.42:0/2074196669 learned_addr learned my addr 172.21.15.42:0/2074196669 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.690+0000 7f3ba37fe700 1 --2- 172.21.15.42:0/2074196669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ba405fe30 0x7f3ba4106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.690+0000 7f3ba2ffd700 1 -- 172.21.15.42:0/2074196669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ba405fe30 msgr2=0x7f3ba4106c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:11.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.690+0000 7f3ba2ffd700 1 --2- 172.21.15.42:0/2074196669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ba405fe30 0x7f3ba4106c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.690+0000 7f3ba2ffd700 1 -- 172.21.15.42:0/2074196669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b94005040 con 0x7f3ba40f9600 2024-02-15T18:36:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.690+0000 7f3ba2ffd700 1 --2- 172.21.15.42:0/2074196669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba40f9600 0x7f3ba4107180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f3b980026d0 tx=0x7f3b98009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.691+0000 7f3b8bfff700 1 -- 172.21.15.42:0/2074196669 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b98009320 con 0x7f3ba40f9600 2024-02-15T18:36:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.691+0000 7f3b8bfff700 1 -- 172.21.15.42:0/2074196669 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f3b9800f040 con 0x7f3ba40f9600 2024-02-15T18:36:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.691+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2074196669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba410c360 con 0x7f3ba40f9600 2024-02-15T18:36:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.691+0000 7f3b8bfff700 1 -- 172.21.15.42:0/2074196669 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b980149b0 con 0x7f3ba40f9600 2024-02-15T18:36:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.691+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2074196669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba410c8f0 con 0x7f3ba40f9600 2024-02-15T18:36:11.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.691+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2074196669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba4046330 con 0x7f3ba40f9600 2024-02-15T18:36:11.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.692+0000 7f3b8bfff700 1 -- 172.21.15.42:0/2074196669 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f3b98009480 con 0x7f3ba40f9600 2024-02-15T18:36:11.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.692+0000 7f3b8bfff700 1 --2- 172.21.15.42:0/2074196669 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3b8c060f20 0x7f3b8c0633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:11.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.692+0000 7f3b8bfff700 1 -- 172.21.15.42:0/2074196669 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f3b9807c720 con 0x7f3ba40f9600 2024-02-15T18:36:11.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.692+0000 7f3ba37fe700 1 --2- 172.21.15.42:0/2074196669 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3b8c060f20 0x7f3b8c0633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:11.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.693+0000 7f3ba37fe700 1 --2- 172.21.15.42:0/2074196669 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3b8c060f20 0x7f3b8c0633e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f3ba4107b80 tx=0x7f3b94009b30).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:11.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.697+0000 7f3b8bfff700 1 -- 172.21.15.42:0/2074196669 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3b9804cab0 con 0x7f3ba40f9600 2024-02-15T18:36:11.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.914+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2074196669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f3ba40586e0 con 0x7f3ba40f9600 2024-02-15T18:36:11.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.915+0000 7f3b8bfff700 1 -- 172.21.15.42:0/2074196669 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v28) v1 ==== 107+0+4587 (secure 0 0 0) 0x7f3b980500d0 con 0x7f3ba40f9600 2024-02-15T18:36:11.905 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:11.906 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34360,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1562993914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1562993914},{"type":"v1","addr":"172.21.15.42:6829","nonce":1562993914}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44313,"name":"cephfs.smithi191.erlpxp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6827/1725194059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1725194059},{"type":"v1","addr":"172.21.15.191:6827","nonce":1725194059}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":44343,"name":"cephfs.smithi191.icwcnb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6825/540280968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":540280968},{"type":"v1","addr":"172.21.15.191:6825","nonce":540280968}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:42.998038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34356":{"gid":34356,"name":"cephfs.smithi042.vtwvlo","rank":0,"incarnation":21,"state":"up:rejoin","state_seq":11,"addr":"172.21.15.42:6827/2962563293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2962563293},{"type":"v1","addr":"172.21.15.42:6827","nonce":2962563293}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:11.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.916+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2074196669 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3b8c060f20 msgr2=0x7f3b8c0633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:11.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.916+0000 7f3ba9e6b700 1 --2- 172.21.15.42:0/2074196669 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3b8c060f20 0x7f3b8c0633e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f3ba4107b80 tx=0x7f3b94009b30).stop 2024-02-15T18:36:11.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.917+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2074196669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba40f9600 msgr2=0x7f3ba4107180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:11.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.917+0000 7f3ba9e6b700 1 --2- 172.21.15.42:0/2074196669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba40f9600 0x7f3ba4107180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f3b980026d0 tx=0x7f3b98009de0).stop 2024-02-15T18:36:11.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.917+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2074196669 shutdown_connections 2024-02-15T18:36:11.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.917+0000 7f3ba9e6b700 1 --2- 172.21.15.42:0/2074196669 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f3b8c060f20 0x7f3b8c0633e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.917+0000 7f3ba9e6b700 1 --2- 172.21.15.42:0/2074196669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ba405fe30 0x7f3ba4106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.917+0000 7f3ba9e6b700 1 --2- 172.21.15.42:0/2074196669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ba40f9600 0x7f3ba4107180 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:11.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.917+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2074196669 >> 172.21.15.42:0/2074196669 conn(0x7f3ba40f0b70 msgr2=0x7f3ba40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:11.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.917+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2074196669 shutdown_connections 2024-02-15T18:36:11.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:11.917+0000 7f3ba9e6b700 1 -- 172.21.15.42:0/2074196669 wait complete. 2024-02-15T18:36:11.909 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 23 2024-02-15T18:36:11.920 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-02-15T18:36:11.920 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-02-15T18:36:12.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.050+0000 7fcfa3a99700 1 -- 172.21.15.42:0/1434415171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf9c0f9600 msgr2=0x7fcf9c0fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:12.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.050+0000 7fcfa3a99700 1 --2- 172.21.15.42:0/1434415171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf9c0f9600 0x7fcf9c0fb9f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fcf90004660 tx=0x7fcf90009b30).stop 2024-02-15T18:36:12.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.050+0000 7fcfa3a99700 1 -- 172.21.15.42:0/1434415171 shutdown_connections 2024-02-15T18:36:12.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.050+0000 7fcfa3a99700 1 --2- 172.21.15.42:0/1434415171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf9c0f9600 0x7fcf9c0fb9f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.050+0000 7fcfa3a99700 1 --2- 172.21.15.42:0/1434415171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf9c05fe30 0x7fcf9c0f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.050+0000 7fcfa3a99700 1 -- 172.21.15.42:0/1434415171 >> 172.21.15.42:0/1434415171 conn(0x7fcf9c0f0b70 msgr2=0x7fcf9c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:12.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.050+0000 7fcfa3a99700 1 -- 172.21.15.42:0/1434415171 shutdown_connections 2024-02-15T18:36:12.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.050+0000 7fcfa3a99700 1 -- 172.21.15.42:0/1434415171 wait complete. 2024-02-15T18:36:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.050+0000 7fcfa3a99700 1 Processor -- start 2024-02-15T18:36:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.050+0000 7fcfa3a99700 1 -- start start 2024-02-15T18:36:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcfa3a99700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf9c05fe30 0x7fcf9c106cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:12.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcfa3a99700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf9c0f9600 0x7fcf9c1071c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcfa3a99700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c1077c0 con 0x7fcf9c0f9600 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcfa3a99700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c107900 con 0x7fcf9c05fe30 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcfa1034700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf9c0f9600 0x7fcf9c1071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcfa1034700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf9c0f9600 0x7fcf9c1071c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43292/0 (socket says 172.21.15.42:43292) 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcfa1034700 1 -- 172.21.15.42:0/4008115394 learned_addr learned my addr 172.21.15.42:0/4008115394 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcfa1835700 1 --2- 172.21.15.42:0/4008115394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf9c05fe30 0x7fcf9c106cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcfa1034700 1 -- 172.21.15.42:0/4008115394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf9c05fe30 msgr2=0x7fcf9c106cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcfa1034700 1 --2- 172.21.15.42:0/4008115394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf9c05fe30 0x7fcf9c106cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcfa1034700 1 -- 172.21.15.42:0/4008115394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf90005040 con 0x7fcf9c0f9600 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcfa1034700 1 --2- 172.21.15.42:0/4008115394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf9c0f9600 0x7fcf9c1071c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fcf90009f50 tx=0x7fcf90009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcf8e7fc700 1 -- 172.21.15.42:0/4008115394 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf90018070 con 0x7fcf9c0f9600 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcf8e7fc700 1 -- 172.21.15.42:0/4008115394 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7fcf90007ba0 con 0x7fcf9c0f9600 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcf8e7fc700 1 -- 172.21.15.42:0/4008115394 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf9001c620 con 0x7fcf9c0f9600 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.051+0000 7fcfa3a99700 1 -- 172.21.15.42:0/4008115394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9c10c2b0 con 0x7fcf9c0f9600 2024-02-15T18:36:12.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.052+0000 7fcfa3a99700 1 -- 172.21.15.42:0/4008115394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf9c10c790 con 0x7fcf9c0f9600 2024-02-15T18:36:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.052+0000 7fcfa3a99700 1 -- 172.21.15.42:0/4008115394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9c027a00 con 0x7fcf9c0f9600 2024-02-15T18:36:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.053+0000 7fcf8e7fc700 1 -- 172.21.15.42:0/4008115394 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7fcf9001c890 con 0x7fcf9c0f9600 2024-02-15T18:36:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.053+0000 7fcf8e7fc700 1 --2- 172.21.15.42:0/4008115394 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf88060f20 0x7fcf880633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.053+0000 7fcf8e7fc700 1 -- 172.21.15.42:0/4008115394 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fcf9007e0c0 con 0x7fcf9c0f9600 2024-02-15T18:36:12.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.053+0000 7fcfa1835700 1 --2- 172.21.15.42:0/4008115394 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf88060f20 0x7fcf880633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:12.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.054+0000 7fcfa1835700 1 --2- 172.21.15.42:0/4008115394 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf88060f20 0x7fcf880633e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fcf9c1081c0 tx=0x7fcf98009c80).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:12.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.057+0000 7fcf8e7fc700 1 -- 172.21.15.42:0/4008115394 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcf9004e3d0 con 0x7fcf9c0f9600 2024-02-15T18:36:12.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.274+0000 7fcfa3a99700 1 -- 172.21.15.42:0/4008115394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fcf9c0fed70 con 0x7fcf9c0f9600 2024-02-15T18:36:12.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.275+0000 7fcf8e7fc700 1 -- 172.21.15.42:0/4008115394 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v28) v1 ==== 107+0+4587 (secure 0 0 0) 0x7fcf900519f0 con 0x7fcf9c0f9600 2024-02-15T18:36:12.266 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:12.266 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34360,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1562993914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1562993914},{"type":"v1","addr":"172.21.15.42:6829","nonce":1562993914}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44313,"name":"cephfs.smithi191.erlpxp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6827/1725194059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":1725194059},{"type":"v1","addr":"172.21.15.191:6827","nonce":1725194059}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":44343,"name":"cephfs.smithi191.icwcnb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6825/540280968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":540280968},{"type":"v1","addr":"172.21.15.191:6825","nonce":540280968}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:44.003070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34356":{"gid":34356,"name":"cephfs.smithi042.vtwvlo","rank":0,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.42:6827/2962563293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2962563293},{"type":"v1","addr":"172.21.15.42:6827","nonce":2962563293}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:12.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.277+0000 7fcfa3a99700 1 -- 172.21.15.42:0/4008115394 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf88060f20 msgr2=0x7fcf880633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:12.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.277+0000 7fcfa3a99700 1 --2- 172.21.15.42:0/4008115394 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf88060f20 0x7fcf880633e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fcf9c1081c0 tx=0x7fcf98009c80).stop 2024-02-15T18:36:12.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.277+0000 7fcfa3a99700 1 -- 172.21.15.42:0/4008115394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf9c0f9600 msgr2=0x7fcf9c1071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:12.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.277+0000 7fcfa3a99700 1 --2- 172.21.15.42:0/4008115394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf9c0f9600 0x7fcf9c1071c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fcf90009f50 tx=0x7fcf90009b30).stop 2024-02-15T18:36:12.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.277+0000 7fcfa3a99700 1 -- 172.21.15.42:0/4008115394 shutdown_connections 2024-02-15T18:36:12.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.277+0000 7fcfa3a99700 1 --2- 172.21.15.42:0/4008115394 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7fcf88060f20 0x7fcf880633e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.277+0000 7fcfa3a99700 1 --2- 172.21.15.42:0/4008115394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf9c05fe30 0x7fcf9c106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.277+0000 7fcfa3a99700 1 --2- 172.21.15.42:0/4008115394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf9c0f9600 0x7fcf9c1071c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.277+0000 7fcfa3a99700 1 -- 172.21.15.42:0/4008115394 >> 172.21.15.42:0/4008115394 conn(0x7fcf9c0f0b70 msgr2=0x7fcf9c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:12.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.277+0000 7fcfa3a99700 1 -- 172.21.15.42:0/4008115394 shutdown_connections 2024-02-15T18:36:12.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.277+0000 7fcfa3a99700 1 -- 172.21.15.42:0/4008115394 wait complete. 2024-02-15T18:36:12.269 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 24 2024-02-15T18:36:12.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:11.196479+0000 mon.smithi042 (mon.0) 793 : audit [DBG] from='client.? 172.21.15.42:0/1087969726' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-15T18:36:12.279 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:11.554258+0000 mon.smithi042 (mon.0) 794 : audit [DBG] from='client.? 172.21.15.42:0/4144351121' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-15T18:36:12.279 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:12 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:11.915755+0000 mon.smithi042 (mon.0) 795 : audit [DBG] from='client.? 172.21.15.42:0/2074196669' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-15T18:36:12.280 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-02-15T18:36:12.280 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-02-15T18:36:12.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.410+0000 7f2d342ff700 1 -- 172.21.15.42:0/3158332858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d2c0f7780 msgr2=0x7f2d2c0f7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:12.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.410+0000 7f2d342ff700 1 --2- 172.21.15.42:0/3158332858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d2c0f7780 0x7f2d2c0f7be0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f2d28004660 tx=0x7f2d28009b30).stop 2024-02-15T18:36:12.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.410+0000 7f2d342ff700 1 -- 172.21.15.42:0/3158332858 shutdown_connections 2024-02-15T18:36:12.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.410+0000 7f2d342ff700 1 --2- 172.21.15.42:0/3158332858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d2c0f7780 0x7f2d2c0f7be0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.410+0000 7f2d342ff700 1 --2- 172.21.15.42:0/3158332858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c0f64e0 0x7f2d2c0f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.410+0000 7f2d342ff700 1 -- 172.21.15.42:0/3158332858 >> 172.21.15.42:0/3158332858 conn(0x7f2d2c0f1c40 msgr2=0x7f2d2c0f4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:12.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.410+0000 7f2d342ff700 1 -- 172.21.15.42:0/3158332858 shutdown_connections 2024-02-15T18:36:12.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.410+0000 7f2d342ff700 1 -- 172.21.15.42:0/3158332858 wait complete. 2024-02-15T18:36:12.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.410+0000 7f2d342ff700 1 Processor -- start 2024-02-15T18:36:12.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.410+0000 7f2d342ff700 1 -- start start 2024-02-15T18:36:12.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d342ff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c0f64e0 0x7f2d2c06b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:12.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d342ff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d2c0f7780 0x7f2d2c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:12.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d342ff700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c06bd70 con 0x7f2d2c0f7780 2024-02-15T18:36:12.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d342ff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c06a310 con 0x7f2d2c0f64e0 2024-02-15T18:36:12.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d3189a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d2c0f7780 0x7f2d2c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:12.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d3189a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d2c0f7780 0x7f2d2c069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43300/0 (socket says 172.21.15.42:43300) 2024-02-15T18:36:12.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d3189a700 1 -- 172.21.15.42:0/1831586219 learned_addr learned my addr 172.21.15.42:0/1831586219 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:12.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d3209b700 1 --2- 172.21.15.42:0/1831586219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c0f64e0 0x7f2d2c06b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:12.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d3189a700 1 -- 172.21.15.42:0/1831586219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c0f64e0 msgr2=0x7f2d2c06b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:12.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d3189a700 1 --2- 172.21.15.42:0/1831586219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c0f64e0 0x7f2d2c06b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d3189a700 1 -- 172.21.15.42:0/1831586219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d28005040 con 0x7f2d2c0f7780 2024-02-15T18:36:12.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d3189a700 1 --2- 172.21.15.42:0/1831586219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d2c0f7780 0x7f2d2c069e20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f2d28013fb0 tx=0x7f2d28009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:12.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d22ffd700 1 -- 172.21.15.42:0/1831586219 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d28018070 con 0x7f2d2c0f7780 2024-02-15T18:36:12.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d22ffd700 1 -- 172.21.15.42:0/1831586219 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f2d280097d0 con 0x7f2d2c0f7780 2024-02-15T18:36:12.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d342ff700 1 -- 172.21.15.42:0/1831586219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c06a530 con 0x7f2d2c0f7780 2024-02-15T18:36:12.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d22ffd700 1 -- 172.21.15.42:0/1831586219 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d2801c7c0 con 0x7f2d2c0f7780 2024-02-15T18:36:12.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.411+0000 7f2d342ff700 1 -- 172.21.15.42:0/1831586219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c06aa90 con 0x7f2d2c0f7780 2024-02-15T18:36:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.412+0000 7f2d342ff700 1 -- 172.21.15.42:0/1831586219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2c046330 con 0x7f2d2c0f7780 2024-02-15T18:36:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.413+0000 7f2d22ffd700 1 -- 172.21.15.42:0/1831586219 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f2d2801c980 con 0x7f2d2c0f7780 2024-02-15T18:36:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.413+0000 7f2d22ffd700 1 --2- 172.21.15.42:0/1831586219 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2d18060f20 0x7f2d180633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:12.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.413+0000 7f2d22ffd700 1 -- 172.21.15.42:0/1831586219 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f2d2807e220 con 0x7f2d2c0f7780 2024-02-15T18:36:12.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.413+0000 7f2d3209b700 1 --2- 172.21.15.42:0/1831586219 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2d18060f20 0x7f2d180633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:12.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.414+0000 7f2d3209b700 1 --2- 172.21.15.42:0/1831586219 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2d18060f20 0x7f2d180633e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f2d2c0f75e0 tx=0x7f2d1c009040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:12.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.417+0000 7f2d22ffd700 1 -- 172.21.15.42:0/1831586219 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f2d28051f60 con 0x7f2d2c0f7780 2024-02-15T18:36:12.440 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:11.196479+0000 mon.smithi042 (mon.0) 793 : audit [DBG] from='client.? 172.21.15.42:0/1087969726' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-15T18:36:12.440 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:11.554258+0000 mon.smithi042 (mon.0) 794 : audit [DBG] from='client.? 172.21.15.42:0/4144351121' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-15T18:36:12.440 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:12 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:11.915755+0000 mon.smithi042 (mon.0) 795 : audit [DBG] from='client.? 172.21.15.42:0/2074196669' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-15T18:36:12.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.634+0000 7f2d342ff700 1 -- 172.21.15.42:0/1831586219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f2d2c027a00 con 0x7f2d2c0f7780 2024-02-15T18:36:12.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.635+0000 7f2d22ffd700 1 -- 172.21.15.42:0/1831586219 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v28) v1 ==== 107+0+3805 (secure 0 0 0) 0x7f2d28051b50 con 0x7f2d2c0f7780 2024-02-15T18:36:12.626 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:12.626 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34360,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1562993914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1562993914},{"type":"v1","addr":"172.21.15.42:6829","nonce":1562993914}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44343,"name":"cephfs.smithi191.icwcnb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6825/540280968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":540280968},{"type":"v1","addr":"172.21.15.191:6825","nonce":540280968}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:44.003070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34356":{"gid":34356,"name":"cephfs.smithi042.vtwvlo","rank":0,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.42:6827/2962563293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2962563293},{"type":"v1","addr":"172.21.15.42:6827","nonce":2962563293}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:12.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.637+0000 7f2d342ff700 1 -- 172.21.15.42:0/1831586219 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2d18060f20 msgr2=0x7f2d180633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:12.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.637+0000 7f2d342ff700 1 --2- 172.21.15.42:0/1831586219 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2d18060f20 0x7f2d180633e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f2d2c0f75e0 tx=0x7f2d1c009040).stop 2024-02-15T18:36:12.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.637+0000 7f2d342ff700 1 -- 172.21.15.42:0/1831586219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d2c0f7780 msgr2=0x7f2d2c069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:12.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.637+0000 7f2d342ff700 1 --2- 172.21.15.42:0/1831586219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d2c0f7780 0x7f2d2c069e20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f2d28013fb0 tx=0x7f2d28009b30).stop 2024-02-15T18:36:12.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.637+0000 7f2d342ff700 1 -- 172.21.15.42:0/1831586219 shutdown_connections 2024-02-15T18:36:12.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.637+0000 7f2d342ff700 1 --2- 172.21.15.42:0/1831586219 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f2d18060f20 0x7f2d180633e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.637+0000 7f2d342ff700 1 --2- 172.21.15.42:0/1831586219 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d2c0f64e0 0x7f2d2c06b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.637+0000 7f2d342ff700 1 --2- 172.21.15.42:0/1831586219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d2c0f7780 0x7f2d2c069e20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.637+0000 7f2d342ff700 1 -- 172.21.15.42:0/1831586219 >> 172.21.15.42:0/1831586219 conn(0x7f2d2c0f1c40 msgr2=0x7f2d2c0fb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:12.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.637+0000 7f2d342ff700 1 -- 172.21.15.42:0/1831586219 shutdown_connections 2024-02-15T18:36:12.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.637+0000 7f2d342ff700 1 -- 172.21.15.42:0/1831586219 wait complete. 2024-02-15T18:36:12.630 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 25 2024-02-15T18:36:12.640 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-02-15T18:36:12.640 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-02-15T18:36:12.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.771+0000 7f6dec21e700 1 -- 172.21.15.42:0/1581319246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de405fe30 msgr2=0x7f6de40f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:12.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.771+0000 7f6dec21e700 1 --2- 172.21.15.42:0/1581319246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de405fe30 0x7f6de40f9110 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f6dd8004660 tx=0x7f6dd8009b30).stop 2024-02-15T18:36:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.771+0000 7f6dec21e700 1 -- 172.21.15.42:0/1581319246 shutdown_connections 2024-02-15T18:36:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.771+0000 7f6dec21e700 1 --2- 172.21.15.42:0/1581319246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6de40f9600 0x7f6de40fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.771+0000 7f6dec21e700 1 --2- 172.21.15.42:0/1581319246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de405fe30 0x7f6de40f9110 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.771+0000 7f6dec21e700 1 -- 172.21.15.42:0/1581319246 >> 172.21.15.42:0/1581319246 conn(0x7f6de40f0b70 msgr2=0x7f6de40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.771+0000 7f6dec21e700 1 -- 172.21.15.42:0/1581319246 shutdown_connections 2024-02-15T18:36:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.771+0000 7f6dec21e700 1 -- 172.21.15.42:0/1581319246 wait complete. 2024-02-15T18:36:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.771+0000 7f6dec21e700 1 Processor -- start 2024-02-15T18:36:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.771+0000 7f6dec21e700 1 -- start start 2024-02-15T18:36:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6dec21e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de405fe30 0x7f6de4106c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6dec21e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6de40f9600 0x7f6de4107180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6dec21e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4107780 con 0x7f6de405fe30 2024-02-15T18:36:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6dec21e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de41078f0 con 0x7f6de40f9600 2024-02-15T18:36:12.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6de9fba700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de405fe30 0x7f6de4106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6de9fba700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de405fe30 0x7f6de4106c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43316/0 (socket says 172.21.15.42:43316) 2024-02-15T18:36:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6de9fba700 1 -- 172.21.15.42:0/167073702 learned_addr learned my addr 172.21.15.42:0/167073702 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6de97b9700 1 --2- 172.21.15.42:0/167073702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6de40f9600 0x7f6de4107180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6de9fba700 1 -- 172.21.15.42:0/167073702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6de40f9600 msgr2=0x7f6de4107180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6de9fba700 1 --2- 172.21.15.42:0/167073702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6de40f9600 0x7f6de4107180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6de9fba700 1 -- 172.21.15.42:0/167073702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd8005040 con 0x7f6de405fe30 2024-02-15T18:36:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6de97b9700 1 --2- 172.21.15.42:0/167073702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6de40f9600 0x7f6de4107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-15T18:36:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6de9fba700 1 --2- 172.21.15.42:0/167073702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de405fe30 0x7f6de4106c90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f6dd8002040 tx=0x7f6dd800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6dd67fc700 1 -- 172.21.15.42:0/167073702 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dd8019070 con 0x7f6de405fe30 2024-02-15T18:36:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6dd67fc700 1 -- 172.21.15.42:0/167073702 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f6dd8014420 con 0x7f6de405fe30 2024-02-15T18:36:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6dec21e700 1 -- 172.21.15.42:0/167073702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de410c300 con 0x7f6de405fe30 2024-02-15T18:36:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6dd67fc700 1 -- 172.21.15.42:0/167073702 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dd801d5c0 con 0x7f6de405fe30 2024-02-15T18:36:12.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.772+0000 7f6dec21e700 1 -- 172.21.15.42:0/167073702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de410c830 con 0x7f6de405fe30 2024-02-15T18:36:12.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.773+0000 7f6dec21e700 1 -- 172.21.15.42:0/167073702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de4046330 con 0x7f6de405fe30 2024-02-15T18:36:12.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.774+0000 7f6dd67fc700 1 -- 172.21.15.42:0/167073702 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f6dd8025070 con 0x7f6de405fe30 2024-02-15T18:36:12.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.774+0000 7f6dd67fc700 1 --2- 172.21.15.42:0/167073702 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6dd0060f20 0x7f6dd00633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:12.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.774+0000 7f6dd67fc700 1 -- 172.21.15.42:0/167073702 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f6dd807d1e0 con 0x7f6de405fe30 2024-02-15T18:36:12.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.774+0000 7f6de97b9700 1 --2- 172.21.15.42:0/167073702 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6dd0060f20 0x7f6dd00633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:12.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.774+0000 7f6de97b9700 1 --2- 172.21.15.42:0/167073702 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6dd0060f20 0x7f6dd00633e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f6de41081b0 tx=0x7f6de000b040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:12.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.778+0000 7f6dd67fc700 1 -- 172.21.15.42:0/167073702 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6dd804d4f0 con 0x7f6de405fe30 2024-02-15T18:36:12.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.991+0000 7f6dec21e700 1 -- 172.21.15.42:0/167073702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f6de4027a00 con 0x7f6de405fe30 2024-02-15T18:36:12.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.992+0000 7f6dd67fc700 1 -- 172.21.15.42:0/167073702 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v28) v1 ==== 107+0+4584 (secure 0 0 0) 0x7f6dd8023020 con 0x7f6de405fe30 2024-02-15T18:36:12.983 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:12.983 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34360,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1562993914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1562993914},{"type":"v1","addr":"172.21.15.42:6829","nonce":1562993914}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44343,"name":"cephfs.smithi191.icwcnb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6825/540280968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":540280968},{"type":"v1","addr":"172.21.15.191:6825","nonce":540280968}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44347,"name":"cephfs.smithi191.erlpxp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6827/289198038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":289198038},{"type":"v1","addr":"172.21.15.191:6827","nonce":289198038}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:44.003070+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34356":{"gid":34356,"name":"cephfs.smithi042.vtwvlo","rank":0,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.42:6827/2962563293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2962563293},{"type":"v1","addr":"172.21.15.42:6827","nonce":2962563293}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:12.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.993+0000 7f6dec21e700 1 -- 172.21.15.42:0/167073702 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6dd0060f20 msgr2=0x7f6dd00633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:12.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.993+0000 7f6dec21e700 1 --2- 172.21.15.42:0/167073702 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6dd0060f20 0x7f6dd00633e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f6de41081b0 tx=0x7f6de000b040).stop 2024-02-15T18:36:12.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.993+0000 7f6dec21e700 1 -- 172.21.15.42:0/167073702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de405fe30 msgr2=0x7f6de4106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:12.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.994+0000 7f6dec21e700 1 --2- 172.21.15.42:0/167073702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de405fe30 0x7f6de4106c90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f6dd8002040 tx=0x7f6dd800b040).stop 2024-02-15T18:36:12.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.994+0000 7f6dec21e700 1 -- 172.21.15.42:0/167073702 shutdown_connections 2024-02-15T18:36:12.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.994+0000 7f6dec21e700 1 --2- 172.21.15.42:0/167073702 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f6dd0060f20 0x7f6dd00633e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.994+0000 7f6dec21e700 1 --2- 172.21.15.42:0/167073702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6de405fe30 0x7f6de4106c90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.994+0000 7f6dec21e700 1 --2- 172.21.15.42:0/167073702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6de40f9600 0x7f6de4107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:12.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.994+0000 7f6dec21e700 1 -- 172.21.15.42:0/167073702 >> 172.21.15.42:0/167073702 conn(0x7f6de40f0b70 msgr2=0x7f6de40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:12.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.994+0000 7f6dec21e700 1 -- 172.21.15.42:0/167073702 shutdown_connections 2024-02-15T18:36:12.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:12.994+0000 7f6dec21e700 1 -- 172.21.15.42:0/167073702 wait complete. 2024-02-15T18:36:12.986 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-02-15T18:36:12.997 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-02-15T18:36:12.997 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-02-15T18:36:13.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.128+0000 7f97fb919700 1 -- 172.21.15.42:0/3481802629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97f40f9660 msgr2=0x7f97f40fba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:13.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.128+0000 7f97fb919700 1 --2- 172.21.15.42:0/3481802629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97f40f9660 0x7f97f40fba50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f97e4004660 tx=0x7f97e4009b30).stop 2024-02-15T18:36:13.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.128+0000 7f97fb919700 1 -- 172.21.15.42:0/3481802629 shutdown_connections 2024-02-15T18:36:13.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.128+0000 7f97fb919700 1 --2- 172.21.15.42:0/3481802629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97f40f9660 0x7f97f40fba50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.128+0000 7f97fb919700 1 --2- 172.21.15.42:0/3481802629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97f405fdd0 0x7f97f40f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-15T18:36:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.128+0000 7f97fb919700 1 -- 172.21.15.42:0/3481802629 >> 172.21.15.42:0/3481802629 conn(0x7f97f40f0b70 msgr2=0x7f97f40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.128+0000 7f97fb919700 1 -- 172.21.15.42:0/3481802629 shutdown_connections 2024-02-15T18:36:13.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.128+0000 7f97fb919700 1 -- 172.21.15.42:0/3481802629 wait complete. 2024-02-15T18:36:13.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.129+0000 7f97fb919700 1 Processor -- start 2024-02-15T18:36:13.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.129+0000 7f97fb919700 1 -- start start 2024-02-15T18:36:13.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.129+0000 7f97fb919700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97f405fdd0 0x7f97f4106d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:13.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.129+0000 7f97fb919700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97f40f9660 0x7f97f4107230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:13.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.129+0000 7f97fb919700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f4107870 con 0x7f97f40f9660 2024-02-15T18:36:13.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.129+0000 7f97fb919700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f41079b0 con 0x7f97f405fdd0 2024-02-15T18:36:13.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.129+0000 7f97f8eb4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97f40f9660 0x7f97f4107230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:13.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.129+0000 7f97f8eb4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97f40f9660 0x7f97f4107230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43328/0 (socket says 172.21.15.42:43328) 2024-02-15T18:36:13.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.129+0000 7f97f8eb4700 1 -- 172.21.15.42:0/4280794341 learned_addr learned my addr 172.21.15.42:0/4280794341 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-02-15T18:36:13.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.130+0000 7f97f96b5700 1 --2- 172.21.15.42:0/4280794341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97f405fdd0 0x7f97f4106d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:13.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.130+0000 7f97f8eb4700 1 -- 172.21.15.42:0/4280794341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97f405fdd0 msgr2=0x7f97f4106d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:13.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.130+0000 7f97f8eb4700 1 --2- 172.21.15.42:0/4280794341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97f405fdd0 0x7f97f4106d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:13.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.130+0000 7f97f8eb4700 1 -- 172.21.15.42:0/4280794341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e4005040 con 0x7f97f40f9660 2024-02-15T18:36:13.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.130+0000 7f97f96b5700 1 --2- 172.21.15.42:0/4280794341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97f405fdd0 0x7f97f4106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-15T18:36:13.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.130+0000 7f97f8eb4700 1 --2- 172.21.15.42:0/4280794341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97f40f9660 0x7f97f4107230 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f97e4009f50 tx=0x7f97e4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:13.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.130+0000 7f97e9ffb700 1 -- 172.21.15.42:0/4280794341 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97e4018070 con 0x7f97f40f9660 2024-02-15T18:36:13.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.130+0000 7f97e9ffb700 1 -- 172.21.15.42:0/4280794341 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1143+0+0 (secure 0 0 0) 0x7f97e4007d70 con 0x7f97f40f9660 2024-02-15T18:36:13.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.130+0000 7f97fb919700 1 -- 172.21.15.42:0/4280794341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f410c3f0 con 0x7f97f40f9660 2024-02-15T18:36:13.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.130+0000 7f97e9ffb700 1 -- 172.21.15.42:0/4280794341 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97e401c690 con 0x7f97f40f9660 2024-02-15T18:36:13.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.130+0000 7f97fb919700 1 -- 172.21.15.42:0/4280794341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f410c8d0 con 0x7f97f40f9660 2024-02-15T18:36:13.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.131+0000 7f97fb919700 1 -- 172.21.15.42:0/4280794341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f4046330 con 0x7f97f40f9660 2024-02-15T18:36:13.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.132+0000 7f97e9ffb700 1 -- 172.21.15.42:0/4280794341 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 77144+0+0 (secure 0 0 0) 0x7f97e401daa0 con 0x7f97f40f9660 2024-02-15T18:36:13.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.132+0000 7f97e9ffb700 1 --2- 172.21.15.42:0/4280794341 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f97e0060f20 0x7f97e00633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-15T18:36:13.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.132+0000 7f97e9ffb700 1 -- 172.21.15.42:0/4280794341 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f97e407e100 con 0x7f97f40f9660 2024-02-15T18:36:13.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.132+0000 7f97f96b5700 1 --2- 172.21.15.42:0/4280794341 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f97e0060f20 0x7f97e00633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-15T18:36:13.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.132+0000 7f97f96b5700 1 --2- 172.21.15.42:0/4280794341 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f97e0060f20 0x7f97e00633e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f97f405fba0 tx=0x7f97f0008040).ready entity=mgr.24435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-15T18:36:13.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.136+0000 7f97e9ffb700 1 -- 172.21.15.42:0/4280794341 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f97e4051e60 con 0x7f97f40f9660 2024-02-15T18:36:13.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:12.275697+0000 mon.smithi042 (mon.0) 796 : audit [DBG] from='client.? 172.21.15.42:0/4008115394' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-15T18:36:13.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:12.635853+0000 mon.smithi042 (mon.0) 797 : audit [DBG] from='client.? 172.21.15.42:0/1831586219' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-15T18:36:13.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: cluster 2024-02-15T18:36:12.818167+0000 mgr.smithi042.kfoihb (mgr.24435) 485 : cluster [DBG] pgmap v371: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:36:13.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:13 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:12.992875+0000 mon.smithi042 (mon.0) 798 : audit [DBG] from='client.? 172.21.15.42:0/167073702' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-15T18:36:13.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.349+0000 7f97fb919700 1 -- 172.21.15.42:0/4280794341 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f97f4027a00 con 0x7f97f40f9660 2024-02-15T18:36:13.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.349+0000 7f97e9ffb700 1 -- 172.21.15.42:0/4280794341 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v28) v1 ==== 107+0+4584 (secure 0 0 0) 0x7f97e4051a50 con 0x7f97f40f9660 2024-02-15T18:36:13.340 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-02-15T18:36:13.340 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34360,"name":"cephfs.smithi042.idnxkc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/1562993914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1562993914},{"type":"v1","addr":"172.21.15.42:6829","nonce":1562993914}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":44343,"name":"cephfs.smithi191.icwcnb","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6825/540280968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":540280968},{"type":"v1","addr":"172.21.15.191:6825","nonce":540280968}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44347,"name":"cephfs.smithi191.erlpxp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6827/289198038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":289198038},{"type":"v1","addr":"172.21.15.191:6827","nonce":289198038}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-15T18:23:40.469506+0000","modified":"2024-02-15T18:32:55.237594+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":34356},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34356":{"gid":34356,"name":"cephfs.smithi042.vtwvlo","rank":0,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.42:6827/2962563293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2962563293},{"type":"v1","addr":"172.21.15.42:6827","nonce":2962563293}]},"join_fscid":1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-02-15T18:36:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.351+0000 7f97fb919700 1 -- 172.21.15.42:0/4280794341 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f97e0060f20 msgr2=0x7f97e00633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.351+0000 7f97fb919700 1 --2- 172.21.15.42:0/4280794341 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f97e0060f20 0x7f97e00633e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f97f405fba0 tx=0x7f97f0008040).stop 2024-02-15T18:36:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.351+0000 7f97fb919700 1 -- 172.21.15.42:0/4280794341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97f40f9660 msgr2=0x7f97f4107230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-15T18:36:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.351+0000 7f97fb919700 1 --2- 172.21.15.42:0/4280794341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97f40f9660 0x7f97f4107230 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f97e4009f50 tx=0x7f97e4009b30).stop 2024-02-15T18:36:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.352+0000 7f97fb919700 1 -- 172.21.15.42:0/4280794341 shutdown_connections 2024-02-15T18:36:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.352+0000 7f97fb919700 1 --2- 172.21.15.42:0/4280794341 >> [v2:172.21.15.42:6800/3393955655,v1:172.21.15.42:6801/3393955655] conn(0x7f97e0060f20 0x7f97e00633e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.352+0000 7f97fb919700 1 --2- 172.21.15.42:0/4280794341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f97f405fdd0 0x7f97f4106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.352+0000 7f97fb919700 1 --2- 172.21.15.42:0/4280794341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97f40f9660 0x7f97f4107230 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-15T18:36:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.352+0000 7f97fb919700 1 -- 172.21.15.42:0/4280794341 >> 172.21.15.42:0/4280794341 conn(0x7f97f40f0b70 msgr2=0x7f97f40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-15T18:36:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.352+0000 7f97fb919700 1 -- 172.21.15.42:0/4280794341 shutdown_connections 2024-02-15T18:36:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-02-15T18:36:13.352+0000 7f97fb919700 1 -- 172.21.15.42:0/4280794341 wait complete. 2024-02-15T18:36:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-02-15T18:36:13.355 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-15T18:36:13.367 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-15T18:36:13.367 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-15T18:36:13.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-15T18:36:13.367 DEBUG:teuthology.orchestra.run.smithi042:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-15T18:36:13.405 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-02-15T18:36:13.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:36:13.405 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-15T18:36:13.440 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:12.275697+0000 mon.smithi042 (mon.0) 796 : audit [DBG] from='client.? 172.21.15.42:0/4008115394' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-15T18:36:13.440 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:12.635853+0000 mon.smithi042 (mon.0) 797 : audit [DBG] from='client.? 172.21.15.42:0/1831586219' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-15T18:36:13.440 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: cluster 2024-02-15T18:36:12.818167+0000 mgr.smithi042.kfoihb (mgr.24435) 485 : cluster [DBG] pgmap v371: 65 pgs: 65 active+clean; 193 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-02-15T18:36:13.441 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:13 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:12.992875+0000 mon.smithi042 (mon.0) 798 : audit [DBG] from='client.? 172.21.15.42:0/167073702' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-15T18:36:13.486 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-15T18:36:13.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-15T18:36:13.486 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-02-15T18:36:13.540 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2024-02-15T18:36:13.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:36:13.540 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-15T18:36:13.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:36:13.579 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-02-15T18:36:13.599 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-15T18:36:13.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:36:13.599 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-15T18:36:13.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:36:13.703 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-02-15T18:36:13.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:36:13.791 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2024-02-15T18:36:13.813 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-15T18:36:13.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:36:13.814 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-15T18:36:13.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-15T18:36:13.916 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2024-02-15T18:36:13.984 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-15T18:36:13.995 INFO:tasks.cephadm:Teardown begin 2024-02-15T18:36:13.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-15T18:36:14.026 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-15T18:36:14.057 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-15T18:36:14.057 DEBUG:teuthology.orchestra.run.smithi042:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-15T18:36:14.085 DEBUG:teuthology.orchestra.run.smithi191:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-15T18:36:14.115 INFO:tasks.cephadm:Stopping all daemons... 2024-02-15T18:36:14.116 INFO:tasks.cephadm.mon.smithi042:Stopping mon.smithi042... 2024-02-15T18:36:14.116 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi042 2024-02-15T18:36:14.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:14 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: audit 2024-02-15T18:36:13.350526+0000 mon.smithi042 (mon.0) 799 : audit [DBG] from='client.? 172.21.15.42:0/4280794341' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-15T18:36:14.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: audit 2024-02-15T18:36:13.350526+0000 mon.smithi042 (mon.0) 799 : audit [DBG] from='client.? 172.21.15.42:0/4280794341' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-15T18:36:14.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:14 smithi042 systemd[1]: Stopping Ceph mon.smithi042 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:36:14.303 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:14 smithi042 bash[99363]: Error: no container with name or ID "ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi042" found: no such container 2024-02-15T18:36:14.803 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:36:14.454+0000 7fd93858c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi042 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-15T18:36:14.803 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:14 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042[77646]: debug 2024-02-15T18:36:14.454+0000 7fd93858c700 -1 mon.smithi042@0(leader) e2 *** Got Signal Terminated *** 2024-02-15T18:36:15.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Feb 15 18:36:15 smithi042 bash[99404]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi042 2024-02-15T18:36:15.404 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi042.service' 2024-02-15T18:36:15.451 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:36:15.451 INFO:tasks.cephadm.mon.smithi042:Stopped mon.smithi042 2024-02-15T18:36:15.452 INFO:tasks.cephadm.mon.smithi191:Stopping mon.smithi191... 2024-02-15T18:36:15.452 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi191 2024-02-15T18:36:15.822 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:15 smithi191 systemd[1]: Stopping Ceph mon.smithi191 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:36:15.822 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:15 smithi191 bash[70913]: Error: no container with name or ID "ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon.smithi191" found: no such container 2024-02-15T18:36:16.190 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:36:15.831+0000 7f636a387700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi191 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-15T18:36:16.191 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:15 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191[59657]: debug 2024-02-15T18:36:15.831+0000 7f636a387700 -1 mon.smithi191@1(peon) e2 *** Got Signal Terminated *** 2024-02-15T18:36:16.690 INFO:journalctl@ceph.mon.smithi191.smithi191.stdout:Feb 15 18:36:16 smithi191 bash[70951]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-mon-smithi191 2024-02-15T18:36:16.935 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@mon.smithi191.service' 2024-02-15T18:36:16.990 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:36:16.990 INFO:tasks.cephadm.mon.smithi191:Stopped mon.smithi191 2024-02-15T18:36:16.990 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-15T18:36:16.990 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.0 2024-02-15T18:36:17.291 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:36:17 smithi042 systemd[1]: Stopping Ceph osd.0 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:36:17.291 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:36:17 smithi042 bash[99535]: Error: no container with name or ID "ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.0" found: no such container 2024-02-15T18:36:17.553 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:36:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-0[80976]: debug 2024-02-15T18:36:17.299+0000 7f7bf7427700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:36:17.553 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:36:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-0[80976]: debug 2024-02-15T18:36:17.299+0000 7f7bf7427700 -1 osd.0 114 *** Got signal Terminated *** 2024-02-15T18:36:17.553 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:36:17 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-0[80976]: debug 2024-02-15T18:36:17.299+0000 7f7bf7427700 -1 osd.0 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:36:23.303 INFO:journalctl@ceph.osd.0.smithi042.stdout:Feb 15 18:36:22 smithi042 bash[99576]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-0 2024-02-15T18:36:25.006 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.0.service' 2024-02-15T18:36:25.055 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:36:25.055 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-15T18:36:25.055 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-15T18:36:25.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.1 2024-02-15T18:36:25.303 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:36:25 smithi042 systemd[1]: Stopping Ceph osd.1 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:36:25.303 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:36:25 smithi042 bash[99814]: Error: no container with name or ID "ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.1" found: no such container 2024-02-15T18:36:25.804 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:36:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-1[83769]: debug 2024-02-15T18:36:25.370+0000 7f3339d3b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:36:25.804 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:36:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-1[83769]: debug 2024-02-15T18:36:25.370+0000 7f3339d3b700 -1 osd.1 114 *** Got signal Terminated *** 2024-02-15T18:36:25.804 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:36:25 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-1[83769]: debug 2024-02-15T18:36:25.370+0000 7f3339d3b700 -1 osd.1 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:36:31.053 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:36:30 smithi042 bash[99854]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-1 2024-02-15T18:36:32.814 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:36:32 smithi042 systemd[1]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.1.service: Succeeded. 2024-02-15T18:36:32.814 INFO:journalctl@ceph.osd.1.smithi042.stdout:Feb 15 18:36:32 smithi042 systemd[1]: Stopped Ceph osd.1 for 51458a44-cc2e-11ee-95ba-87774f69a715. 2024-02-15T18:36:32.815 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.1.service' 2024-02-15T18:36:32.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:36:32.862 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-15T18:36:32.862 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-15T18:36:32.862 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.2 2024-02-15T18:36:33.162 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:36:32 smithi042 systemd[1]: Stopping Ceph osd.2 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:36:33.162 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:36:32 smithi042 bash[100093]: Error: no container with name or ID "ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.2" found: no such container 2024-02-15T18:36:33.553 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:36:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-2[86626]: debug 2024-02-15T18:36:33.170+0000 7fe5656ad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:36:33.553 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:36:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-2[86626]: debug 2024-02-15T18:36:33.170+0000 7fe5656ad700 -1 osd.2 114 *** Got signal Terminated *** 2024-02-15T18:36:33.554 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:36:33 smithi042 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-2[86626]: debug 2024-02-15T18:36:33.170+0000 7fe5656ad700 -1 osd.2 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:36:38.804 INFO:journalctl@ceph.osd.2.smithi042.stdout:Feb 15 18:36:38 smithi042 bash[100134]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-2 2024-02-15T18:36:40.437 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.2.service' 2024-02-15T18:36:40.482 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:36:40.482 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-15T18:36:40.482 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-15T18:36:40.482 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.3 2024-02-15T18:36:40.789 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:36:40 smithi191 systemd[1]: Stopping Ceph osd.3 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:36:40.789 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:36:40 smithi191 bash[71074]: Error: no container with name or ID "ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.3" found: no such container 2024-02-15T18:36:41.191 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:36:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:36:40.799+0000 7f492e66e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:36:41.191 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:36:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:36:40.799+0000 7f492e66e700 -1 osd.3 114 *** Got signal Terminated *** 2024-02-15T18:36:41.191 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:36:40 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:36:40.799+0000 7f492e66e700 -1 osd.3 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:36:43.940 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:36:43 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:36:43.551+0000 7f492b5eb700 -1 osd.3 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:20.307729+0000 front 2024-02-15T18:36:20.307784+0000 (oldest deadline 2024-02-15T18:36:42.607561+0000) 2024-02-15T18:36:44.940 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:36:44 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:36:44.536+0000 7f492b5eb700 -1 osd.3 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:20.307729+0000 front 2024-02-15T18:36:20.307784+0000 (oldest deadline 2024-02-15T18:36:42.607561+0000) 2024-02-15T18:36:45.940 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:36:45 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3[62233]: debug 2024-02-15T18:36:45.559+0000 7f492b5eb700 -1 osd.3 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:20.307729+0000 front 2024-02-15T18:36:20.307784+0000 (oldest deadline 2024-02-15T18:36:42.607561+0000) 2024-02-15T18:36:46.440 INFO:journalctl@ceph.osd.3.smithi191.stdout:Feb 15 18:36:46 smithi191 bash[71114]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-3 2024-02-15T18:36:47.441 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:36:47.209+0000 7f18cf91c700 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.692949+0000 front 2024-02-15T18:36:21.693006+0000 (oldest deadline 2024-02-15T18:36:46.992940+0000) 2024-02-15T18:36:47.441 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:47 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:47.015+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:36:48.082 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.3.service' 2024-02-15T18:36:48.125 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:36:48.125 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-15T18:36:48.125 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-15T18:36:48.125 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.4 2024-02-15T18:36:48.440 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:48 smithi191 systemd[1]: Stopping Ceph osd.4 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:36:48.440 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:48 smithi191 bash[71351]: Error: no container with name or ID "ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.4" found: no such container 2024-02-15T18:36:48.440 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:36:48.252+0000 7f18cf91c700 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.692949+0000 front 2024-02-15T18:36:21.693006+0000 (oldest deadline 2024-02-15T18:36:46.992940+0000) 2024-02-15T18:36:48.441 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:48.042+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:36:48.940 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:36:48.474+0000 7f18d69a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:36:48.940 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:36:48.474+0000 7f18d69a7700 -1 osd.4 114 *** Got signal Terminated *** 2024-02-15T18:36:48.940 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:48 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:36:48.474+0000 7f18d69a7700 -1 osd.4 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:36:49.255 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:49.006+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:36:49.690 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:36:49.266+0000 7f18cf91c700 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.692949+0000 front 2024-02-15T18:36:21.693006+0000 (oldest deadline 2024-02-15T18:36:46.992940+0000) 2024-02-15T18:36:50.303 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:49 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:49.966+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:36:50.690 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:36:50.312+0000 7f18cf91c700 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.692949+0000 front 2024-02-15T18:36:21.693006+0000 (oldest deadline 2024-02-15T18:36:46.992940+0000) 2024-02-15T18:36:51.190 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:50 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:50.930+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:36:51.690 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:36:51.278+0000 7f18cf91c700 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.692949+0000 front 2024-02-15T18:36:21.693006+0000 (oldest deadline 2024-02-15T18:36:46.992940+0000) 2024-02-15T18:36:51.690 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:36:51.278+0000 7f18cf91c700 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-02-15T18:36:28.093741+0000 front 2024-02-15T18:36:28.093794+0000 (oldest deadline 2024-02-15T18:36:50.993709+0000) 2024-02-15T18:36:52.190 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:51.919+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:36:52.190 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:51 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:51.919+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-02-15T18:36:27.446551+0000 front 2024-02-15T18:36:27.446588+0000 (oldest deadline 2024-02-15T18:36:51.546437+0000) 2024-02-15T18:36:52.690 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:36:52.256+0000 7f18cf91c700 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.692949+0000 front 2024-02-15T18:36:21.693006+0000 (oldest deadline 2024-02-15T18:36:46.992940+0000) 2024-02-15T18:36:52.690 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:36:52.256+0000 7f18cf91c700 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-02-15T18:36:28.093741+0000 front 2024-02-15T18:36:28.093794+0000 (oldest deadline 2024-02-15T18:36:50.993709+0000) 2024-02-15T18:36:53.190 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:52.916+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:36:53.190 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:52 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:52.916+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-02-15T18:36:27.446551+0000 front 2024-02-15T18:36:27.446588+0000 (oldest deadline 2024-02-15T18:36:51.546437+0000) 2024-02-15T18:36:53.690 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:36:53.251+0000 7f18cf91c700 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.692949+0000 front 2024-02-15T18:36:21.693006+0000 (oldest deadline 2024-02-15T18:36:46.992940+0000) 2024-02-15T18:36:53.690 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4[64520]: debug 2024-02-15T18:36:53.251+0000 7f18cf91c700 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-02-15T18:36:28.093741+0000 front 2024-02-15T18:36:28.093794+0000 (oldest deadline 2024-02-15T18:36:50.993709+0000) 2024-02-15T18:36:54.190 INFO:journalctl@ceph.osd.4.smithi191.stdout:Feb 15 18:36:54 smithi191 bash[71389]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-4 2024-02-15T18:36:54.191 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:53.916+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:36:54.191 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:53 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:53.916+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-02-15T18:36:27.446551+0000 front 2024-02-15T18:36:27.446588+0000 (oldest deadline 2024-02-15T18:36:51.546437+0000) 2024-02-15T18:36:55.190 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:54.889+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:36:55.190 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:54 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:54.889+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-02-15T18:36:27.446551+0000 front 2024-02-15T18:36:27.446588+0000 (oldest deadline 2024-02-15T18:36:51.546437+0000) 2024-02-15T18:36:55.940 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:55.923+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:36:55.940 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:55 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:55.923+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-02-15T18:36:27.446551+0000 front 2024-02-15T18:36:27.446588+0000 (oldest deadline 2024-02-15T18:36:51.546437+0000) 2024-02-15T18:36:56.099 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.4.service' 2024-02-15T18:36:56.144 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:36:56.145 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-15T18:36:56.145 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-15T18:36:56.145 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.5 2024-02-15T18:36:56.419 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:56 smithi191 systemd[1]: Stopping Ceph osd.5 for 51458a44-cc2e-11ee-95ba-87774f69a715... 2024-02-15T18:36:56.419 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:56 smithi191 bash[71635]: Error: no container with name or ID "ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd.5" found: no such container 2024-02-15T18:36:56.690 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:56.427+0000 7fb323ccf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-15T18:36:56.690 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:56.427+0000 7fb323ccf700 -1 osd.5 114 *** Got signal Terminated *** 2024-02-15T18:36:56.690 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:56.427+0000 7fb323ccf700 -1 osd.5 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-15T18:36:57.440 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:56.950+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:36:57.440 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:56 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:56.950+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-02-15T18:36:27.446551+0000 front 2024-02-15T18:36:27.446588+0000 (oldest deadline 2024-02-15T18:36:51.546437+0000) 2024-02-15T18:36:58.190 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:57.926+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:36:58.190 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:57 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:57.926+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-02-15T18:36:27.446551+0000 front 2024-02-15T18:36:27.446588+0000 (oldest deadline 2024-02-15T18:36:51.546437+0000) 2024-02-15T18:36:59.190 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:58.931+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:36:59.190 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:58 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:58.931+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-02-15T18:36:27.446551+0000 front 2024-02-15T18:36:27.446588+0000 (oldest deadline 2024-02-15T18:36:51.546437+0000) 2024-02-15T18:37:00.190 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:59.908+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:37:00.190 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:59.908+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-02-15T18:36:27.446551+0000 front 2024-02-15T18:36:27.446588+0000 (oldest deadline 2024-02-15T18:36:51.546437+0000) 2024-02-15T18:37:00.191 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:36:59 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:36:59.908+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6822 osd.2 since back 2024-02-15T18:36:36.747518+0000 front 2024-02-15T18:36:36.747502+0000 (oldest deadline 2024-02-15T18:36:59.047418+0000) 2024-02-15T18:37:01.440 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:37:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:37:00.950+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-02-15T18:36:21.045711+0000 front 2024-02-15T18:36:21.045686+0000 (oldest deadline 2024-02-15T18:36:46.945578+0000) 2024-02-15T18:37:01.440 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:37:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:37:00.950+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-02-15T18:36:27.446551+0000 front 2024-02-15T18:36:27.446588+0000 (oldest deadline 2024-02-15T18:36:51.546437+0000) 2024-02-15T18:37:01.440 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:37:00 smithi191 ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5[66731]: debug 2024-02-15T18:37:00.950+0000 7fb32144d700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.42:6822 osd.2 since back 2024-02-15T18:36:36.747518+0000 front 2024-02-15T18:36:36.747502+0000 (oldest deadline 2024-02-15T18:36:59.047418+0000) 2024-02-15T18:37:02.191 INFO:journalctl@ceph.osd.5.smithi191.stdout:Feb 15 18:37:01 smithi191 bash[71676]: ceph-51458a44-cc2e-11ee-95ba-87774f69a715-osd-5 2024-02-15T18:37:03.629 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-51458a44-cc2e-11ee-95ba-87774f69a715@osd.5.service' 2024-02-15T18:37:03.671 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-15T18:37:03.672 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-15T18:37:03.672 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-15T18:37:23.933 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-15T18:37:32.944 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-15T18:37:32.982 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-15T18:37:33.012 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-15T18:37:33.014 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561652/remote/ubuntu@smithi042.front.sepia.ceph.com/crash 2024-02-15T18:37:33.015 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/crash -- . 2024-02-15T18:37:33.053 INFO:teuthology.orchestra.run.smithi042.stderr:tar: /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-15T18:37:33.054 INFO:teuthology.orchestra.run.smithi042.stderr:tar: Error is not recoverable: exiting now 2024-02-15T18:37:33.056 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561652/remote/ubuntu@smithi191.front.sepia.ceph.com/crash 2024-02-15T18:37:33.057 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/crash -- . 2024-02-15T18:37:33.083 INFO:teuthology.orchestra.run.smithi191.stderr:tar: /var/lib/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-15T18:37:33.083 INFO:teuthology.orchestra.run.smithi191.stderr:tar: Error is not recoverable: exiting now 2024-02-15T18:37:33.084 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-15T18:37:33.084 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v 'Degraded data redundancy' | egrep -v '\(PG_' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-15T18:37:33.134 INFO:teuthology.orchestra.run.smithi042.stdout:2024-02-15T18:30:00.000241+0000 mon.smithi042 (mon.0) 244 : cluster [WRN] [WRN] OSD_DOWN: 1 osds down 2024-02-15T18:37:33.134 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-15T18:37:33.134 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[SEC\]' /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v 'Degraded data redundancy' | egrep -v '\(PG_' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-15T18:37:33.210 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]' /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v 'Degraded data redundancy' | egrep -v '\(PG_' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-15T18:37:33.286 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[WRN\]' /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v 'Degraded data redundancy' | egrep -v '\(PG_' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-15T18:37:33.361 INFO:teuthology.orchestra.run.smithi042.stdout:2024-02-15T18:30:00.000241+0000 mon.smithi042 (mon.0) 244 : cluster [WRN] [WRN] OSD_DOWN: 1 osds down 2024-02-15T18:37:33.361 INFO:tasks.cephadm:Compressing logs... 2024-02-15T18:37:33.361 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-15T18:37:33.403 DEBUG:teuthology.orchestra.run.smithi191:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-15T18:37:33.430 INFO:teuthology.orchestra.run.smithi191.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-15T18:37:33.430 INFO:teuthology.orchestra.run.smithi191.stderr:: No such file or directory 2024-02-15T18:37:33.430 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-15T18:37:33.430 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mgr.smithi191.gftdzo.log 2024-02-15T18:37:33.433 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-15T18:37:33.433 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-15T18:37:33.433 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi191.erlpxp.log 2024-02-15T18:37:33.433 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-volume.log: /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mgr.smithi191.gftdzo.log: /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.audit.log: /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.3.log 2024-02-15T18:37:33.433 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi191.erlpxp.log: 86.7% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-15T18:37:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-15T18:37:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:: No such file or directory 2024-02-15T18:37:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-15T18:37:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose /var/log/ceph/cephadm.log: -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.2.log 2024-02-15T18:37:33.433 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-15T18:37:33.434 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-15T18:37:33.434 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mon.smithi042.log 2024-02-15T18:37:33.434 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.cephadm.log: /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mon.smithi042.log: 86.6% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-15T18:37:33.435 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.3.log: 91.3% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-15T18:37:33.435 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mon.smithi191.log 2024-02-15T18:37:33.435 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.log 2024-02-15T18:37:33.435 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mon.smithi191.log: /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.4.log 2024-02-15T18:37:33.436 INFO:teuthology.orchestra.run.smithi042.stderr: 91.2% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-15T18:37:33.436 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.0.log 2024-02-15T18:37:33.436 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mgr.smithi042.kfoihb.log 2024-02-15T18:37:33.436 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi042.vtwvlo.log 2024-02-15T18:37:33.436 INFO:teuthology.orchestra.run.smithi042.stderr: 90.6% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-15T18:37:33.436 INFO:teuthology.orchestra.run.smithi042.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-15T18:37:33.436 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mgr.smithi042.kfoihb.log: gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi042.idnxkc.log 2024-02-15T18:37:33.436 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi042.vtwvlo.log: gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.log 2024-02-15T18:37:33.437 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.4.log: 89.5% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-15T18:37:33.437 INFO:teuthology.orchestra.run.smithi191.stderr: 90.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-15T18:37:33.437 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi191.icwcnb.log 2024-02-15T18:37:33.437 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.5.log 2024-02-15T18:37:33.437 INFO:teuthology.orchestra.run.smithi191.stderr: 88.2% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-15T18:37:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi042.idnxkc.log: /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.log: 88.2% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-15T18:37:33.439 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.1.log 2024-02-15T18:37:33.466 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi191.icwcnb.log: /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.5.log: 89.4% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mgr.smithi191.gftdzo.log.gz 2024-02-15T18:37:33.648 INFO:teuthology.orchestra.run.smithi191.stderr: 92.5% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mon.smithi191.log.gz 2024-02-15T18:37:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.1.log: 90.7% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mgr.smithi042.kfoihb.log.gz 2024-02-15T18:37:34.321 INFO:teuthology.orchestra.run.smithi042.stderr: 90.5% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mon.smithi042.log.gz 2024-02-15T18:37:38.562 INFO:teuthology.orchestra.run.smithi042.stderr: 93.1% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.0.log.gz 2024-02-15T18:37:38.570 INFO:teuthology.orchestra.run.smithi042.stderr: 93.5% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.2.log.gz 2024-02-15T18:37:38.724 INFO:teuthology.orchestra.run.smithi042.stderr: 94.9% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi042.idnxkc.log.gz 2024-02-15T18:37:39.614 INFO:teuthology.orchestra.run.smithi191.stderr: 93.3% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.4.log.gz 2024-02-15T18:37:40.431 INFO:teuthology.orchestra.run.smithi042.stderr: 94.9% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi042.vtwvlo.log.gz 2024-02-15T18:37:40.591 INFO:teuthology.orchestra.run.smithi191.stderr: 93.0% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.3.log.gz 2024-02-15T18:37:40.733 INFO:teuthology.orchestra.run.smithi042.stderr: 93.4% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.1.log.gz 2024-02-15T18:37:40.735 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-02-15T18:37:40.735 INFO:teuthology.orchestra.run.smithi042.stderr:real 0m7.315s 2024-02-15T18:37:40.735 INFO:teuthology.orchestra.run.smithi042.stderr:user 0m30.064s 2024-02-15T18:37:40.735 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m0.993s 2024-02-15T18:37:40.864 INFO:teuthology.orchestra.run.smithi191.stderr: 95.0% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi191.erlpxp.log.gz 2024-02-15T18:37:41.142 INFO:teuthology.orchestra.run.smithi191.stderr: 93.6% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-osd.5.log.gz 2024-02-15T18:38:51.117 INFO:teuthology.orchestra.run.smithi191.stderr: 93.0% -- replaced with /var/log/ceph/51458a44-cc2e-11ee-95ba-87774f69a715/ceph-mds.cephfs.smithi191.icwcnb.log.gz 2024-02-15T18:38:51.118 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-02-15T18:38:51.118 INFO:teuthology.orchestra.run.smithi191.stderr:real 1m17.699s 2024-02-15T18:38:51.118 INFO:teuthology.orchestra.run.smithi191.stderr:user 1m42.796s 2024-02-15T18:38:51.118 INFO:teuthology.orchestra.run.smithi191.stderr:sys 0m3.452s 2024-02-15T18:38:51.119 INFO:tasks.cephadm:Archiving logs... 2024-02-15T18:38:51.120 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561652/remote/ubuntu@smithi042.front.sepia.ceph.com/log 2024-02-15T18:38:51.121 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-15T18:38:54.701 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/log/ceph to /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561652/remote/ubuntu@smithi191.front.sepia.ceph.com/log 2024-02-15T18:38:54.702 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-15T18:39:10.391 INFO:tasks.cephadm:Removing cluster... 2024-02-15T18:39:10.391 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 --force 2024-02-15T18:39:10.744 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 51458a44-cc2e-11ee-95ba-87774f69a715 --force 2024-02-15T18:39:11.190 INFO:tasks.cephadm:Removing cephadm ... 2024-02-15T18:39:11.191 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-15T18:39:11.211 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-15T18:39:11.229 INFO:tasks.cephadm:Teardown complete 2024-02-15T18:39:11.229 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-15T18:39:11.241 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-15T18:39:11.241 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-15T18:39:11.254 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-15T18:39:11.347 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-15T18:39:11.359 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-15T18:39:11.359 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-15T18:39:11.362 DEBUG:teuthology.orchestra.run.smithi191:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-15T18:39:11.378 INFO:teuthology.orchestra.run.smithi042.stderr:bash: ntpq: command not found 2024-02-15T18:39:11.378 INFO:teuthology.orchestra.run.smithi191.stderr:bash: ntpq: command not found 2024-02-15T18:39:11.382 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-15T18:39:11.382 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-02-15T18:39:11.382 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 9 +1214us[+1214us] +/- 110ms 2024-02-15T18:39:11.382 INFO:teuthology.orchestra.run.smithi042.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 +170us[ +187us] +/- 61ms 2024-02-15T18:39:11.382 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -883us[ -865us] +/- 68ms 2024-02-15T18:39:11.382 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:39:11.383 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-15T18:39:11.383 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-02-15T18:39:11.383 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 9 +1204us[+1210us] +/- 110ms 2024-02-15T18:39:11.384 INFO:teuthology.orchestra.run.smithi191.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 +166us[ +172us] +/- 61ms 2024-02-15T18:39:11.384 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -904us[ -898us] +/- 68ms 2024-02-15T18:39:11.384 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-15T18:39:11.384 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-15T18:39:11.418 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-15T18:39:11.418 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-15T18:39:11.459 DEBUG:teuthology.orchestra.run.smithi042:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-15T18:39:11.526 DEBUG:teuthology.orchestra.run.smithi191:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-15T18:39:11.587 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-15T18:39:11.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:39:11.618 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-15T18:39:11.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:39:11.647 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-15T18:39:11.658 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-15T18:39:11.704 INFO:teuthology.task.internal:Duration was 2628.117619 seconds 2024-02-15T18:39:11.705 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-15T18:39:11.771 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-15T18:39:11.772 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-15T18:39:11.774 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-15T18:39:11.820 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-15T18:39:11.821 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-15T18:39:12.199 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-15T18:39:12.199 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-02-15T18:39:12.199 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-15T18:39:12.225 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi191.front.sepia.ceph.com 2024-02-15T18:39:12.225 DEBUG:teuthology.orchestra.run.smithi191:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-15T18:39:12.248 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-15T18:39:12.248 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-15T18:39:12.268 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-15T18:39:12.343 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-15T18:39:12.344 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-15T18:39:12.346 DEBUG:teuthology.orchestra.run.smithi191:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-15T18:39:12.471 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-15T18:39:12.483 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-15T18:39:12.483 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-15T18:39:12.510 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-15T18:39:12.537 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-15T18:39:12.548 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-15T18:39:12.552 DEBUG:teuthology.orchestra.run.smithi191:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-15T18:39:12.577 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-02-15T18:39:12.603 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = core 2024-02-15T18:39:12.616 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-15T18:39:12.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:39:12.649 DEBUG:teuthology.orchestra.run.smithi191:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-15T18:39:12.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-15T18:39:12.673 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-15T18:39:12.685 INFO:teuthology.task.internal:Transferring archived files... 2024-02-15T18:39:12.686 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561652/remote/smithi042 2024-02-15T18:39:12.687 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-15T18:39:12.734 DEBUG:teuthology.misc:Transferring archived files from smithi191:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561652/remote/smithi191 2024-02-15T18:39:12.735 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-15T18:39:12.774 INFO:teuthology.task.internal:Removing archive directory... 2024-02-15T18:39:12.775 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-15T18:39:12.777 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-15T18:39:12.826 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-15T18:39:12.839 INFO:teuthology.task.internal:Not uploading archives. 2024-02-15T18:39:12.840 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-15T18:39:12.910 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-15T18:39:12.910 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-15T18:39:12.913 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-15T18:39:12.928 INFO:teuthology.orchestra.run.smithi042.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 15 18:39 /home/ubuntu/cephtest 2024-02-15T18:39:12.930 INFO:teuthology.orchestra.run.smithi191.stdout: 393224 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 15 18:39 /home/ubuntu/cephtest 2024-02-15T18:39:12.931 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-15T18:39:13.002 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-15T18:39:13.179 INFO:teuthology.nuke:Checking targets against current locks 2024-02-15T18:39:13.206 DEBUG:teuthology.nuke:shortname: smithi042 2024-02-15T18:39:13.206 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-15T18:39:13.226 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-15 17:46:24.426094', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKlhCUm2k1q1gsq3CSgb49Hu3cnWkSpNr0fOsXKpIwQANleMzfbTUWiP4y8V0TcnpowcO0hDB2U/GPAJRCoIUsc='} 2024-02-15T18:39:13.229 DEBUG:teuthology.nuke:shortname: smithi191 2024-02-15T18:39:13.229 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-15T18:39:13.256 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-02-15_17:32:20-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7561652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-15 17:46:24.427538', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBLU5bsDTV0HTr8VBBaD3qxl333FTANBHQUsnaFRnkoACjZk/+YCDpW6l79Isbj8JbERr5d6JyU6yaIOBfCD4UI='} 2024-02-15T18:39:13.282 INFO:teuthology.orchestra.console.smithi042:Power off 2024-02-15T18:39:13.283 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-15T18:39:13.308 DEBUG:teuthology.orchestra.console.smithi042:power off output: Chassis Power Control: Down/Off 2024-02-15T18:39:13.309 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-15T18:39:13.323 INFO:teuthology.orchestra.console.smithi191:Power off 2024-02-15T18:39:13.323 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-15T18:39:13.333 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is on 2024-02-15T18:39:13.346 DEBUG:teuthology.orchestra.console.smithi191:power off output: Chassis Power Control: Down/Off 2024-02-15T18:39:13.346 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-15T18:39:13.371 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-02-15T18:39:17.335 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-15T18:39:17.372 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-15T18:39:17.446 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is on 2024-02-15T18:39:17.482 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-02-15T18:39:21.450 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-15T18:39:21.483 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-15T18:39:21.560 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is off 2024-02-15T18:39:21.592 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-02-15T18:39:21.662 INFO:teuthology.orchestra.console.smithi042:Power off completed 2024-02-15T18:39:25.596 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-15T18:39:25.706 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2024-02-15T18:39:25.806 INFO:teuthology.orchestra.console.smithi191:Power off completed 2024-02-15T18:39:25.911 INFO:teuthology.run:Summary data: description: rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} duration: 2628.117619037628 failure_reason: '"2024-02-15T18:30:00.000241+0000 mon.smithi042 (mon.0) 244 : cluster [WRN] [WRN] OSD_DOWN: 1 osds down" in cluster log' flavor: default owner: scheduled_lflores@teuthology success: false 2024-02-15T18:39:25.911 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-15T18:39:25.986 INFO:teuthology.run:FAIL